For people interested in developing Hadoop analytic applications there is a plethora of options. The options range from writing low-level, hand-tuned Java map-reduce code, to using a higher level language to manipulate the data such as Pig and Hive. There are pros and cons for each option. For the first, the code becomes complex for anything other than the canonical word-count example, and for the latter, to do anything meaningful, you almost always end up augmenting the higher level language with user-defined functions written in a different language to regain power and flexibility, causing maintenance nightmares. A happy medium in between is to use one of the data-flow libraries for Hadoop, of which Cascading is one.
Since Cascading has been around for some time, the online documentation is relatively mature, and includes a gentle introduction to the library, with example source code, and a well written user's guide. However this does not obviate the need for a book that describes the library and walks the reader gently through its usage and subtleties. "Enterprise Data Workflows with Cascading" is such a book.
The book starts with a simple example of copying a file on Hadoop, and introduces the concepts of taps for data sources, and data sinks, as well as data pipes that connect them. It then graduates to the canonical word count example, using it as a vehicle to explain flows, and the operations that can be performed on them through the use of functions and aggregation functions.
Next comes more complex tasks that require joins. The book starts with HashJoins, and then progresses to LeftJoins and distributed joins. The book then uses a meaty example of a text analytics pipeline to calculate term frequencies/inverse document frequency for a text corpus (TF-IDF), and uses that as a vehicle to walk through splits, merges, and more complex joins.
By then, the reader has become familiar and comfortable with Cascading, and the author walks him through the benefits of developing applications in a data-flow language instead of the other options available for Hadoop developers. Some of these benefits are the ability to test the code before deployment, and the author walks through an example of a TDD pipeline. Others include using a consistent pattern language to describe the workflows, and having a single deployable JAR that can be used in dev/test/production environments.
Toward the end the author lists other language bindings for Cascading, such as Scalding (Scala), and Cascalog (Clojure). The later chapters contain good references for further reading on TDD/Scala/Clojure. The book closes with an open-data use-case.
Throughout the book, the author provides ample links to the source code, and code gists on github, as well as alternate implementations in different languages.
I liked the style of the book: it is a gentle introduction to Cascading, interspersed with some good advice on doing TDD for enterprise applications, the use of a pattern language for describing data-flows, and an introduction to other language bindings for Cascading.
Since Cascading has been around for some time, the online documentation is relatively mature, and includes a gentle introduction to the library, with example source code, and a well written user's guide. However this does not obviate the need for a book that describes the library and walks the reader gently through its usage and subtleties. "Enterprise Data Workflows with Cascading" is such a book.
The book starts with a simple example of copying a file on Hadoop, and introduces the concepts of taps for data sources, and data sinks, as well as data pipes that connect them. It then graduates to the canonical word count example, using it as a vehicle to explain flows, and the operations that can be performed on them through the use of functions and aggregation functions.
Next comes more complex tasks that require joins. The book starts with HashJoins, and then progresses to LeftJoins and distributed joins. The book then uses a meaty example of a text analytics pipeline to calculate term frequencies/inverse document frequency for a text corpus (TF-IDF), and uses that as a vehicle to walk through splits, merges, and more complex joins.
By then, the reader has become familiar and comfortable with Cascading, and the author walks him through the benefits of developing applications in a data-flow language instead of the other options available for Hadoop developers. Some of these benefits are the ability to test the code before deployment, and the author walks through an example of a TDD pipeline. Others include using a consistent pattern language to describe the workflows, and having a single deployable JAR that can be used in dev/test/production environments.
Toward the end the author lists other language bindings for Cascading, such as Scalding (Scala), and Cascalog (Clojure). The later chapters contain good references for further reading on TDD/Scala/Clojure. The book closes with an open-data use-case.
Throughout the book, the author provides ample links to the source code, and code gists on github, as well as alternate implementations in different languages.
I liked the style of the book: it is a gentle introduction to Cascading, interspersed with some good advice on doing TDD for enterprise applications, the use of a pattern language for describing data-flows, and an introduction to other language bindings for Cascading.
No comments :
Post a Comment