Hadoop File System (HDFS)

In this tutorial we will see, some of the common unix shell commands applied on the HDFS. To execute the commands on the HDFS (Hadoop Distributed File System), make sure that the Hadoop is running.

All unix shell commands, will executed against the default home directory in HDFS. What is the default home directory in HDFS? A user’s home directory in HDFS is located at /user/username. For example, my home directory is /user/mramanujam.

Lets start with some commands, make sure that hadoop is running, if not please start hadoop.

$start-dfs.sh
.....
$start-yarn.sh

Once started, let's create the HDFS home directory, it the same unix shell command to create a directory.

$ hadoop fs -mkdir -p /user/mramanujam

Now list the files in your home directory

$ hadoop fs -ls This is will list the files in our HDFS directory.

Lets create a file and move it to the HDFS

$ vi newFile.txt

Now copy the file create from the local directory to the HDFS directory

$ hadoop fs -copyFromLocal newFile.txt newFile.txt

Now list the files in your home directory now

$ hadoop fs -ls It will show Found 1 item and the relevant details.

To get the list of all commands, please visit here

Happy Programming...!!!