第6章. Hadoop HDFS命令介紹


6.1 啟動Hadoop Multi-Node Cluster
start-all.sh
6.2 建立與查看HDFS目錄
Step1 建立HDFS目錄
hadoop fs -mkdir /user
hadoop fs -mkdir /user/hduser
hadoop fs -mkdir /user/hduser/test
Step2 查看使用者HDFS目錄
hadoop fs -ls
Step3 查看HDFS完整目錄
hadoop fs -ls /
hadoop fs -ls /user
hadoop fs -ls /user/hduser
Step4 查看所有HDFS子目錄
hadoop fs -ls -R /
Step5 一次建立所有HDFS子目錄
hadoop fs -mkdir -p /dir1/dir2/dir3
hadoop fs -ls -R /
6.3 從本機複製檔案到 HDFS
Step1 複製本機(local)檔案到 HDFS
hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test
hadoop fs -copyFromLocal/usr/local/hadoop/README.txt /user/hduser/test/test1.txt
hadoop fs -ls /user/hduser/test
Step2 列出HDFS目錄下的檔案
hadoop fs -cat /user/hduser/test/README.txt
hadoop fs -cat /user/hduser/test/README.txt|more
Step3 複製重複檔案至HDFS目錄
hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test
Step4 強迫複製重複檔案至HDFS目錄
hadoop fs -copyFromLocal -f /usr/local/hadoop/README.txt /user/hduser/test
Step5 複製多個本機檔案至HDFS目錄
hadoop fs -copyFromLocal /usr/local/hadoop/NOTICE.txt /usr/local/hadoop/LICENSE.txt  /user/hduser/test
Step6 copyFromLocal複製目錄至HDFS目錄
hadoop fs -copyFromLocal /usr/local/hadoop/etc /user/hduser/test
hadoop fs -ls /user/hduser/test
Step7 查看目錄下所有的檔案
hadoop fs -ls -R /user/hduser/test/etc
Step8 使用put複製檔案至HDFS目錄
hadoop fs -put /usr/local/hadoop/README.txt /user/hduser/test/test2.txt
Step9 使用put命令接受stdin(標準輸入)
echo abc | hadoop fs -put -  /user/hduser/test/echoin.txt
hadoop fs -cat /user/hduser/test/echoin.txt
Step10 使用put命令將本機目錄的列表,儲存到HDFS檔案
ls /usr/local/hadoop | hadoop fs -put - /user/hduser/test/hadooplist.txt
hadoop fs -cat /user/hduser/test/hadooplist.txt
6.4 將HDFS上的檔案複製到本機
Step1 將HDFS上的檔案複製到本機(local)
mkdir test
cd test
hadoop fs -copyToLocal /user/hduser/test/hadooplist.txt
ll
Step3 hadoop fs -get複製到本機(local)
hadoop fs -get  /user/hduser/test/README.txt localREADME.txt
6.5 複製與刪除HDFS檔案
Step1 複製HDFS檔案
hadoop fs -mkdir /user/hduser/test/temp
hadoop fs -cp /user/hduser/test/README.txt /user/hadoop/test/temp
hadoop fs -ls /user/hadoop/test/temp
Step2 刪除HDFS檔案
hadoop fs -ls /user/hduser/test
hadoop fs -rm /user/hduser/test/test2.txt
Step3 刪除HDFS目錄
hadoop fs -ls  /user/hduser/test
hadoop fs -rm -R /user/hduser/test/etc
6.6 Hadoop HDFS Web UI 介面瀏覽HDFS
http://master:50070




 


以上內容節錄自這本書。很適合入門初學者:
  Python+Spark 2.0+Hadoop機器學習與大數據分析實戰 http://pythonsparkhadoop.blogspot.tw/2016/10/pythonspark-20hadoop.html

《購買本書 限時特價專區》
博客來:http://www.books.com.tw/products/0010730134?loc=P_007_090  

天瓏:https://www.tenlong.com.tw/items/9864341537?item_id=1023658
露天拍賣:http://goods.ruten.com.tw/item/show?21640846068139
蝦皮拍賣:https://goo.gl/IEx13P 



Share on Google Plus

About kevin

This is a short description in the author block about the author. You edit it by entering text in the "Biographical Info" field in the user admin panel.
    Blogger Comment
    Facebook Comment

1 意見:

  1. Hi 很感謝你整理的指令,這是一本很好的書。請問第七章的指令在哪裡?

    回覆刪除