python - how can I reliably access a single key-value pair from a JSON file that's too large to load into memory? -
i trying retrieve names of people file. file size 201gb
import json open("d:/dns.json", "r") fh: l in fh: d = json.loads(l) print(d["name"])
whenever try run program on windows, encounter memory error, says insufficient memory.
is there reliable way parse single key, value pair without loading whole file? have reading file in chunks in mind, don't know how start.
here sample: test.json
every line seperated newline. hope helps.
you may want give ijson
try : https://pypi.python.org/pypi/ijson
Comments
Post a Comment