Skip to main content

Posts

Showing posts from October, 2016

Install Apache hadoop Mahout in MacOS without Brew

Here are the steps to install Apache Mahout 1. Download latest package from http://mirror.nexcess.net/apache/mahout/ 2. Extract the package 3. Create a directory and put into HDFS <hadoop_directory> hdfs dfs -put /home/Hadoop/data/mydata.txt /mahout_data/ 4. Run clustering in mahout <mahout_directory>/bin/mahout seqdirectory -i hdfs://localhost:9000/mahout_data/ -o hdfs://localhost:9000/clustered_data/ 5. The output file will be in clustered_data directory

[Solved] Hive installation error: java.net.URISyntaxException: Relative path in absolute URI

Error: Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D at org.apache.hadoop.fs.Path.initialize(Path.java:205) Solution: Edit and update hive-site.xml <name>hive.exec.scratchdir</name> <value>/tmp/hive-${user.name}</value> <name>hive.exec.local.scratchdir</name> <value>/tmp/${user.name}</value> <name>hive.downloaded.resources.dir</name> <value>/tmp/${user.name}_resources</value>

Read a file and print all lines in Hadoop

#1. Compile this code and run in hadoop like #2. Put the "transaction.csv" into HDFS hadoop jar Cat.jar org.myorg.Cat #3. You will get the output in the console itself

How to verify Hadoop is running properly?

To verify hadoop's all node are running properly, type $jps The output should be like