Spark Scala - How to create new column for each group and then explode the column values in spark dataframe -


my dataframe below col names index, words

index | words 1 | {col1: a, col2: b} 1 | {col1: c, col2: d} 2 | {col1: a, col2: b} 2 | {col1: e, col2: f} 

i looking output below output df column names one,two

one | 2  {col1: a, col2: b} | {col1: a, col2: b}  {col1: c, col2: d} | {col1: e, col2: f} 

any 1 please me out on one. using spark 1.5.2 version

if sure every index value there 2 words can use below :

sqlcontext.sql("select index, collect_list(words)[0],collect_list(words)[1] yourtable group index").show 

Comments

Popular posts from this blog

'hasOwnProperty' in javascript -

python - ValueError: No axis named 1 for object type <class 'pandas.core.series.Series'> -

java - How to provide dependency injections in Eclipse RCP 3.x? -