Install Spark/Shark on CDH 4

CDH 4 is the currently stable version of Cloudera Distribution of Hadoop. http://cloudera.com/ Apache Spark is a fast and general engine for large-scale data processing. http://spark.incubator.apache.org/ Shark is a Hive compatible query engine Based on Spark. http://shark.cs.berkeley.edu/ Cloudera provides a parcel for Apache Spark, official parcel at http://archive.cloudera.com/spark/ and you can get it from my … Continue reading “Install Spark/Shark on CDH 4”

Shark 0.9.1 Snapshot Working with CDH 4.6.0 Now

The Shark 0.9.x(0.9.0, 0.9.1) are still pre-release: https://github.com/amplab/shark/releases Previously supplied 0.9.0 Prebuild with Hadoop2, CDH4.5.0: shark-0.9.0-hadoop2-bin is not really working with CDH 4.5.0, so I tried to compile Build Shark 0.9 for CDH 4, Install Spark/Shark on CDH 4 Some days later, 0.9.1 is out at the https://github.com/amplab/shark/tree/branch-0.9, the patched hive is uploaded to maven … Continue reading “Shark 0.9.1 Snapshot Working with CDH 4.6.0 Now”

Build Shark 0.9 for CDH 4

Cloudera provides the parcel of latest Apache Spark(0.9) for Cloudera Manager, which is incompatible with old versions of Shark (0.8.1, 0.8.0 or earlier). The official release/pre-release of Shark 0.9.0 for CDH 4 is still not available for downloading, build from source might be a choice. Shark’s wiki: Build Shark From Source Code This page is … Continue reading “Build Shark 0.9 for CDH 4”