pandas 技巧
find index of top 3 largest values of each column:
df1.apply(lambda s: pd.Series(s.nlargest(3).index))
map dataframe column
df["ItemIdx"] = df["question"].map(lambda x: itemMap.get(x,np.NaN))
load a dictionay from a save pkl file
with open ("l.pkl","rb") as f: itemMap= pickle.lead(f)
find the startpoint of each session (after sorted):
offset = np.zeros(df["sessinId"].nunique()+1,dtype=np.int32) offset[1:] = df.groupby('sessinId').size().cumsum()
create a dictionary of two pandas Dataframe columns?
In [9]: pd.Series(df.Letter.values,index=df.Position).to_dict() Out[9]: {1: 'a', 2: 'b', 3: 'c', 4: 'd', 5: 'e'}
Remap values in pandas column with a dict
>>> df = pd.DataFrame({'col2': {0: 'a', 1: 2, 2: np.nan}, 'col1': {0: 'w', 1: 1, 2: 2}}) >>> di = {1: "A", 2: "B"} >>> df col1 col2 0 w a 1 1 2 2 2 NaN >>> df.replace({"col1": di}) col1 col2 0 w a 1 A 2 2 B NaN
去掉括号里数字字母
config .loc[:,'cc'] = config.insurance.apply(lambda x: re.sub("\([a-zA-Z0-9]\)","",x))
去掉括号里
config .loc[:,'cc'] = config.insurance.apply(lambda x: re.sub("\(.*?\)","",x))
index reset
dfff.reset_index(drop=True)
Translate every element in numpy array according to key
>>> a = np.array([[1,2,3], [3,2,4]]) >>> my_dict = {1:23, 2:34, 3:36, 4:45} >>> np.vectorize(my_dict.get)(a) array([[23, 34, 36], [36, 34, 45]]
pandas dataframe to nested json after groupby
#application: dataframe to mongoDB test_dict = {'id':[1,2,3,1,2,1], "name":[...], "math":[...], "English":[...]} df = pd.DataFrame(data=test_dict) e = df.groupby(["name","id"],as_index=False).apply(lambda x: x[["math","english"]].to_dict("r")) sss = e.reset_index().rename(columns={0:"questions"}) result_dict = sss.to_dict("records")