One application I would like to test is to know how Olinuxino goes if you using it in an hadoop environment.
At first sight it might not be a very good solution. Indeed it doesn’t have a very big storage habilitity and you can not think of using a smartmedia card as a normal filesystem.
The solution I propose to test is to use a laptop (or another machine) as a master (and hdfs filesystem). I would like to know if filesystem is a bottleneck or if it is the network.
Step 1:Install an hadoop server
In my case it is my laptop and I use a 30Go partion on my sd disk. The IP I will use for my server is 192.168.1.7.
This step will install a « normal » hadoop distribution on a PC and will be used as
To simplify the best thing is to add alias in /etc/hosts
Step 2: setting up the board
Download standard image fot olinuxino. It can be found here: https://www.olimex.com/wiki/images/2/29/Debian_FS_34_90_camera_A20-olimex.torrent taken from the official olinuxino github: (https://github.com/OLIMEX/OLINUXINO/tree/master/SOFTWARE/A20/A20-build).
First problem is that by default network is not enabled. Change the file /etc/network/interfaces and add
Get your inetaddress on the board by typing
and ensure that the line « PermitRootLogin yes » and « StrictModes no » line has theses values
To test if everything is correctly setup go to your server computer and type (password by defaut for root is olimex)
ssh 192.168.1.254 -l root
Try also to connect to your server (in my case adress is 192.168.1.75 and I create an account names local)
ssh 192.168.1.75 -l local
Step 1: Adding a User
Step 2: installing hadoop
sudo aptitude install openjdk-7-jre
tar zxvf hadoop-2.7.0.tar.gz
mv hadoop-2.7.0/hadoop hadoop
Now we have to modify the configuration file to acces to the masternode and the hdfs
edit the file core-site.xml
Now you can tst if evrything works:
hadoop fs -ls
Step 3: installing mahout
One easy way to test hadoop is to install mahout. This project includes several hadoop jobs to classify data. So we will using it for test purpose.
Download it from apache:
Now everything should be corectly set.
Step 4: Benchmarking
Go into hadoop/mahout
do: sudo apt-get install curl examples/bin/classify-wikipedia.sh
Now to bench do