apache spark - Read JSON with odd format -
i testing skills easy data. apparently data comes serious challenge, data comes in property called data
in array of arrays, not in array of jsons. troublesome because examples spark's json
support i've found require data comes in series of jsons.
{ ... "data" : [ [ 1, "5dc7f285-052b-4739-8dc3-62827014a4cd", 1, 1425450997, "714909", 1425450997, "714909", "{\n}", "2013", "gavin", "st lawrence", "m", "9" ] , [ 2, "17c0d0ed-0b12-477b-8a23-1ed2c49ab8af", 2, 1425450997, "714909", 1425450997, "714909", "{\n}", "2013", "levi", "st lawrence", "m", "9" ] , [ 3, "53e20da8-8384-4ec1-a9c4-071ec2ada701", 3, 1425450997, "714909", 1425450997, "714909", "{\n}", "2013", "logan", "new york", "m", "44" ] , [ 4, "8d47e747-e5c7-4413-b362-509a2fab3c9b", 4, 1425450997, "714909", 1425450997, "714909", "{\n}", "2013", "hudson", "new york", "m", "49" ] ... ] ... }
i'd know if it's possible read json
using sqlcontext.read.json
in order obtain formatted data.
Comments
Post a Comment