JSON Lines: a format you do not have to read in full
A file where each line holds a separate JSON object rather than one big array. It looks like a misunderstanding until you meet a ten-gigabyte export.
What is different
Regular JSON is a single whole. To get the first element of an array you must formally make sure the document is valid up to the last bracket. The consequence: the file is parsed in full and memory is spent in proportion to its size.
JSON Lines works differently: every line is a self-contained document. You can read one line at a time, start in the middle, stop anywhere. A ten-gigabyte file is processed with a few megabytes of memory.
The extensions are .jsonl, .ndjson, sometimes just .json or .log. Judging the format by its extension is pointless: it is more reliable to check whether the first lines are self-contained objects.
Where you meet it
- Logs. A line can be appended to the end of the file without rereading it — for a journal that is the decisive property.
- Paginated API exports. Page responses are simply concatenated and the result stays valid.
- Exchange between data-processing systems. BigQuery, ClickHouse and similar tools read JSON Lines directly.
- Machine-learning datasets. It is the de facto standard.
What to open it with
A text editor shows the content, but without columns and filters — on a hundred thousand lines that is of little use. Excel will not understand the format at all. Python and jq fit if someone can write the code.
If the task is to look, find the records you need and export a subset, a viewer that understands the format is enough: lines become a table, keys become columns and nesting is expanded with dot notation.
An important detail: real files contain broken lines — a record was cut off, or service output slipped in. Parsing must not fall over on them. The right behavior is to skip such lines and say how many there were, which is what the app does. For JSON Lines there is no memory limit of the kind regular JSON has: lines are read one by one, and the ceiling is five million records.
Turning logs into something meaningful
A typical scenario with an application log:
- open the file — each record becomes a row, and the fields
ts,level,msgbecome columns; - filter by level:
level = 'error'; - group by code or message with a count — what repeats most often is visible at once;
- look at the distribution over time and export the selection for whoever will investigate.
All of this without a command line and without uploading the log — which may well contain sensitive data — to someone else’s server.
FAQ
Do I need to rename the file to .jsonl?
No, the format is detected from the content: if the first lines are self-contained objects, the file is read as JSON Lines.
What happens to broken lines?
They are skipped, and their count is shown as a warning: silently losing data would be wrong.
Can I convert regular JSON to JSON Lines?
Yes, export to JSON Lines is available. It is handy when you need to hand a big array to a system that reads line by line.
Free for personal use. Your file is not uploaded to a server. Windows version — 3.5 MB, no installation: details. Organizations: license.