python--upload file into HDFS 加载文件到HDFS

模拟:https://creativedata.atlassian.net/wiki/spaces/SAP/pages/61177860/Python+-+Read+Write+files+from+HDFS

import pandas as pd
from hdfs import InsecureClient
import os

client_hdfs = InsecureClient('http://(your name node ip adress):50070/',user='hadoop')
# Creating a simple Pandas DataFrame
liste_hello = ['hello1','hello2']
liste_world = ['world1','world2']
df = pd.DataFrame(data = {'hello' : liste_hello, 'world': liste_world})
 
# Writing Dataframe to hdfs
with client_hdfs.write('/Learn/input/helloworld.csv', encoding = 'utf-8') as writer:
    df.to_csv(writer)

 

posted @ 2018-04-25 18:45  Jesse_Li  阅读(999)  评论(0编辑  收藏  举报