Upload data to HDFS windows

Posted on January 21, 2012. Filed under: Uncategorized |

We are now ready to run the first Map/Reduce project but data is still missing. This section explains how to upload data to the Hadoop Distributed File System (HDFS).

Upload Files To HDFS

  1. Open a new CYGWIN command window.
  2. Execute the following commands in the new CYGWIN window as shown on the image above.
    cd hadoop-0.19.1
    bin/hadoop fs -mkdir In
    bin/hadoop fs -put *.txt In

    When the last of the above commands starts executing, you should see some activity in other Hadoop windows as shown in the image below.

    The result of these commands is a newly created directory — named In — in the HDFS which contains a set of text files that comes with the Hadoop distribution.

  3. Close the Cygwin Window.

Verify if the files were uploaded correctly

In this section we will check if the files were uploaded correctly.
  1. Open the Eclipse environment.
  2. Open DFS locations folder which is located in the Project Explorer tab ofMap/Reduce perspective.
  3. Open localhost folder in DFS locations folder.
  4. Keep opening HDFS folders until you navigate to the newly created Indirectory, as shown in the image below.
  5. Verifying that the data was uploaded correctly
  6. When you get to the In directory, double-click on the file LICENCE.txt to open it.
  7. If you see something similar to the image above then the data was uploaded correctly and you can proceed to your first Hadoop project.
You can now move on to the next step.

Make a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Liked it here?
Why not try sites on the blogroll...

%d bloggers like this: