apache spark - Read JSON with odd format -


i testing skills easy data. apparently data comes serious challenge, data comes in property called data in array of arrays, not in array of jsons. troublesome because examples spark's json support i've found require data comes in series of jsons.

{ ...      "data" : [ [ 1, "5dc7f285-052b-4739-8dc3-62827014a4cd", 1, 1425450997, "714909", 1425450997, "714909", "{\n}", "2013", "gavin", "st lawrence", "m", "9" ]               , [ 2, "17c0d0ed-0b12-477b-8a23-1ed2c49ab8af", 2, 1425450997, "714909", 1425450997, "714909", "{\n}", "2013", "levi", "st lawrence", "m", "9" ]               , [ 3, "53e20da8-8384-4ec1-a9c4-071ec2ada701", 3, 1425450997, "714909", 1425450997, "714909", "{\n}", "2013", "logan", "new york", "m", "44" ]               , [ 4, "8d47e747-e5c7-4413-b362-509a2fab3c9b", 4, 1425450997, "714909", 1425450997, "714909", "{\n}", "2013", "hudson", "new york", "m", "49" ] ... ]   ... }

i'd know if it's possible read json using sqlcontext.read.json in order obtain formatted data.


Comments

Post a Comment

Popular posts from this blog

'hasOwnProperty' in javascript -

i use obj.hasownproperty judge whether object has property, when replaced obj[prop] !== undefined , not normal implementation, ask, why behind method can not use it? object.hasownproperty(prop); object[prop] !== undefined; obj[prop] !== undefined wrong 2 reasons: you can explicitly set property undefined , obj[prop] = undefined; . obj.hasownproperty(prop) return true in case. obj[prop] follow prototype chain, return property that's inherited. obj.hasownproperty(prop) returns true if property exists directly in object, returns false inherited properties.
Read more

How to put a lock and transaction on table using spring 4 or above using jdbcTemplate and annotations like @Transactional? -

i trying put lock on table while writing on table , if happened in between roll-back . trying convert below code lock table test_g1 read; lock table test_g write; -- begin; start transaction; insert test_g1 values(143); insert test_g values(145); select * test_g1; select * test_g; rollback; select * test_g; unlock tables; how convert above code @transactional spring jdbctemplate code? @transactional(rollbackfor={dataaccessexception.class}) public void test(){ jdbctemplate.execute("insert test1 (id, nam) values (4, 'a')"); throw new dataaccessexception("error") { }; } here trying throw error, insert statement should rollback it's not happening . thanks edit-1 attaching code , doing in jdbcdaoimpl.java , mentioned problem comment above test() . app.java package com.cgiri.javabrains.spring4; import org.springframework.beans.factory.annotation.autowired; import org.springframework.context.applic...
Read more

How to understand 2 main() functions after using uftrace to profile the C++ program? -

i trying uftrace profile following simple c++ program: #include <iostream> class { public: a() {std::cout << "a created" << std::endl;} ~a() {std::cout << "a destroyed" << std::endl;} }; int main() { a; return 0; } the profile result this: # uftrace a.out created destroyed # duration tid function 2.026 [ 4828] | __cxa_atexit(); [ 4828] | main() { [ 4828] | __static_initialization_and_destruction_0() { 89.397 [ 4828] | std::ios_base::init::init(); 0.768 [ 4828] | __cxa_atexit(); 93.029 [ 4828] | } /* __static_initialization_and_destruction_0 */ 94.425 [ 4828] | } /* main */ [ 4828] | main() { [ 4828] | a::a() { 11.104 [ 4828] | std::operator <<(); 10.825 [ 4828] | std::basic_ostream::operator <<(); 24.514 [ 4828] | } /* a::a */ [ 4828] | a::~a() { 0.978 [ 4828] | ...
Read more