Build Shark 0.9 for CDH 4

Cloudera provides the parcel of latest Apache Spark(0.9) for Cloudera Manager, which is incompatible with old versions of Shark (0.8.1, 0.8.0 or earlier). The official release/pre-release of Shark 0.9.0 for CDH 4 is still not available for downloading, build from source might be a choice.

Shark’s wiki: Build Shark From Source Code
This page is a little bit old but still useful.

1 Install CDH 4 + Spark

Install from parcels, follow Install Spark/Shark on CDH 4

2 Install git

Continue reading “Build Shark 0.9 for CDH 4” »

Build Shark 0.9 for CDH 4 by @sskaje: https://sskaje.me/2014/02/build-shark-for-cdh-4/

Install Spark/Shark on CDH 4

CDH 4 is the currently stable version of Cloudera Distribution of Hadoop. http://cloudera.com/
Apache Spark is a fast and general engine for large-scale data processing. http://spark.incubator.apache.org/
Shark is a Hive compatible query engine Based on Spark. http://shark.cs.berkeley.edu/

Cloudera provides a parcel for Apache Spark, official parcel at http://archive.cloudera.com/spark/ and you can get it from my mirror (only if you’re on CentOS/RHEL 6 x86_64) Cloudera Mirror.

Environment

CentOS 6.4 x86_64, host names hadoop1-hadoop5.
Cloudera Manager 4.8.1
CDH 4.5.0
Continue reading “Install Spark/Shark on CDH 4” »

Install Spark/Shark on CDH 4 by @sskaje: https://sskaje.me/2014/02/install-spark-shark-cdh-4/

Incoming search terms:

Fix Hadoop Conf Alternatives for CDH5

I’m using CDH5, upgraded failed from CDH4 and then reinstalled directly.
/etc/hadoop/conf is linked to /etc/hadoop/conf/conf.cloudera.mapreduce1/.
Deploy Client Configuration does not make it right.

The way fix it is manually set a new path and remove the old one, like:

But the next time you try Deploy Client Configuration would corrupt it again.

Continue reading “Fix Hadoop Conf Alternatives for CDH5” »

Fix Hadoop Conf Alternatives for CDH5 by @sskaje: https://sskaje.me/2014/02/fix-hadoop-conf-alternatives-cdh5/

Hive MetaStore Schema Upgrade Failed When Upgrading CDH5

Manually upgrade:

Continue reading “Hive MetaStore Schema Upgrade Failed When Upgrading CDH5” »

Hive MetaStore Schema Upgrade Failed When Upgrading CDH5 by @sskaje: https://sskaje.me/2014/02/hive-metastore-schema-upgrade-failed-upgrading-cdh5/

Incoming search terms:

Generate Certificate with GnuTLS and Sign with OpenSSL

In iOS IPSec VPN Server on Ubuntu, I create a local CA with openssl.
I’m setting up an OpenConnect VPN, which uses GnuTLS’s certtool generating ca and sign certificates.

I want to use share the same Root CA for both OpenSSL and GnuTLS, so I’m generating request from GnuTLS and signing with OpenSSL.
Apple has it’s own certtool different from GnuTLS, the MacPorts one is named as gnutls-certtool

Prepare

Create private key

Continue reading “Generate Certificate with GnuTLS and Sign with OpenSSL” »

Generate Certificate with GnuTLS and Sign with OpenSSL by @sskaje: https://sskaje.me/2014/02/generate-certificate-gnutls-sign-openssl/

Incoming search terms: