apache spark - Pyspark dataframe write to single json file with specific name -


i have dataframe want write single json file specific name. tried below

df2 = df1.select(df1.col1,df1.col2) df2.write.format('json').save('/path/file_name.json') # didnt work, writing in folder 'file_name.json' , files part-xxx df2.tojson().saveastextfile('/path/file_name.json')  # didnt work, writing in folder 'file_name.json' , files part-xxx 

appreciate if 1 can provide solution.

you need save on single file using below code:-

df2 = df1.select(df1.col1,df1.col2) df2.coalesce(1).write.format('json').save('/path/file_name.json') 

this make folder file_name.json. check folder can single file whole data part-000


Comments

Popular posts from this blog

'hasOwnProperty' in javascript -

python - ValueError: No axis named 1 for object type <class 'pandas.core.series.Series'> -

java - How to provide dependency injections in Eclipse RCP 3.x? -