Skip to end of metadata
Go to start of metadata

Apache Hadoop is a framework that allows distributed processing of large datasets.

The Hadoop SmartMachine includes the following components:

  • hadoop
  • hbase
  • hcatalog
  • hive
  • flume
  • templeton
  • zookeeper
  • oozie
  • pig
  • sqoop
  • mahout

To learn more about Hadoop, see the Hadoop Documentation.

In this topic:

Provisioning a Hadoop SmartMachine

Since Hadoop runs on Java, you should provision your Hadoop SmartMachine with a comfortable amount of memory. For a stand-alone machine, 4GB should be enough. If your machine is part of a cluster, you should use 8GB or more.

Logging into your Hadoop SmartMachine

The Hadoop SmartMachine is configured with the two standard accounts: root and admin. You can use SmartLogin to log into your account using the keys in your my.joyentcloud.com account. Both accounts also have generated passwords that you can see in the Credentials section of the machine's detail page.

Log into your Hadoop SmartMachine the same way you log into a standard SmartMachine:

or

When you log in to your Hadoop SmartMachine for the first time, it is a good idea to bring the pkgsrc repository up to date and to upgrade the installed packages.

Location of Hadoop Files

Item Location
executables (hadoop, hbase, pig, etc) /opt/local/bin
shell scripts (start-all.sh, hadoop-create-user.sh, etc) /opt/local/sbin
configuration files /opt/local/etc/hadoop
/opt/local/etc/hbase
/opt/local/etc/hcatalog
/opt/local/etc/hive
/opt/local/etc/pig
/opt/local/etc/templeton
examples /opt/local/share/hadoop
/opt/local/share/hbase
/opt/local/share/hcatalog
/opt/local/share/hive
/opt/local/base/pig
/opt/local/share/templeton

Environment Variables

Some of the Hadoop tools rely on the JAVA_HOME environment variable to be set. This environment variable is set automatically when you run java and in /opt/local/etc/hadoop/hadoop-env.sh.

If you need to set it yourself, you can do so like this:

If you are using version 13.1.0 of this image, JAVA_HOME is set to /opt/local/java/sun6.

The HADOOP_HOME variable is set by /opt/local/etc/hadoop/hadoop-env.sh relative to the startup scripts.

Hadoop Documentation

Release Notes

Version 13.x

13.3.1

Name hadoop
Description A SmartOS 64-bit image with Hadoop Hortonworks components pre-installed. Includes hadoop, hbase, hcatalog, hive, pig, templeton, flume, pig, oozie, sqoop, and mahout.
UUID 6d03452a-9a3c-11e3-8773-23a29a9b75d8
Based on base64 13.3.1
pkgsrc 2013Q3

What's New in This Image

  • Base image is base64-13.3.1 (2013Q3)
  • Change java from sun-jdk to openjdk
  • OpenJDK version is 1.7.25
  • Npm Manta installed by default
  • Hadoop version is 1.1.2
  • Hbase version is 0.94.7
  • Hcatalog version is 0.5.0
  • Hive version is 0.11.0
  • Pig version is 0.10.1
  • Flume version is 1.4.0
  • Templeton version is 0.1.0
  • Zookeeper version is 3.4.5
  • Oozie version is 3.3.1
  • Sqoop version is 1.4.2
  • Mahout version is 0.7

Software Included in This Image

Software installed as pkgsrc packages:

Software installed as npm packages:

Software installed as Ruby gems:

For a detailed list of every package installed with this image, click here.

13.1.0

Name hadoop
Description A SmartOS 64-bit image with Apache Hadoop components pre-installed. Includes hadoop, hbase, hcatalog, hive, pig, templeton, flume, pig, oozie, sqoop, and mahout.
UUID 274bc2bc-d919-11e2-b797-83245409fbeb
Based on base64 13.1.0
pkgsrc 2013Q1

What's New in This Image

  • Based on base64 13.1.0
  • Uses Sun Java 1.6.x

Software Installed in This Image

For a detailed list of every package installed with this image, click here.

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.