What’s the difference between CSV, JSONL, Parquet, ORC, Arrow, and Avro?

A quick comparison of the six formats

Consider sales data. CSV can be convenient when a person needs to inspect a small amount in a spreadsheet, while JSONL is a natural fit for records appended one at a time as logs. If you repeatedly aggregate only date and revenue from a dataset with ten million rows and one hundred columns, however, Parquet or ORC becomes a better fit because analytical readers can focus on the columns they need.

CSV — The simplest text table

CSV is a text format that represents rows and columns using delimiters. Its major strength is accessibility: spreadsheets and many data tools can open it, and a person can inspect it in a text editor. It remains excellent for small tables, exports, and simple interchange.

CSV itself does not generally embed a shared schema saying that one field is an integer, another is a date, or another is an array. Delimiters, encodings, quoting conventions, and date representations can vary between implementations and workflows, and nested data is awkward. Its simplicity is both its advantage and its limitation.

JSONL — One JSON value at a time

JSONL (JSON Lines, often called NDJSON) stores one independent JSON value per line. Because each line is JSON, values can naturally contain numbers, booleans, nulls, arrays, and objects rather than only text-like cells. It can represent richer records than CSV while still being processed one line at a time.

That makes it useful for logs, events, machine-learning datasets, and other record-at-a-time workflows. A program can parse line by line instead of loading one enormous JSON array. JSONL itself, however, normally does not define one mandatory file-wide schema, so fields and types can vary unless the producing and consuming systems enforce additional rules.

Parquet and ORC — similar goals, different layouts and ecosystems

Parquet and ORC are both column-oriented file formats for large structured datasets. They are designed to read selected columns efficiently and use encoding, compression, and column statistics so analytical engines can avoid scanning more data than necessary. It is reasonable to start by thinking of them as formats with very similar goals.

The differences become clearer in their internal layout and historical ecosystems. Parquet organizes data into row groups, column chunks, and pages. ORC organizes the file into stripes that keep column data together with row-group indexes and a stripe footer. A useful mental model is “Parquet divides each row group into columns,” while “ORC packages column data and indexes inside each stripe.” ORC was created to speed up Hive and Hadoop workloads and provides file-, stripe-, and row-group-level statistics and indexes that readers can use to skip irrelevant ranges.

Parquet, meanwhile, was designed as a broadly usable columnar substrate across processing frameworks and has implementations in many languages and analytics tools. The practical choice is therefore usually not “Parquet is newer” or “ORC is faster,” but which format best matches the systems and tools already in your workflow.

Arrow — Optimized for working with and exchanging in-memory data

Arrow occupies a different position from Parquet and ORC. The core Apache Arrow format standardizes how tabular data is laid out in memory in a language-independent columnar representation. It is designed so modern CPUs can process data efficiently and so libraries and languages can exchange structured data with less serialization overhead.

Arrow IPC defines File and Stream representations for serializing schemas, record batches, dictionaries, and related buffers. You may therefore encounter `.arrow` or `.ipc` files, but minimizing long-term storage size is not the central goal in the same way it is for formats such as Parquet. A system might read Parquet from storage and then use Arrow as the in-memory representation while processing it.

Avro — Work with records defined by a schema

Avro serializes records in binary according to a schema. A commonly used file representation, the Avro Object Container File, stores the writer schema in file metadata and groups records into blocks that can also be compressed. A reader can inspect that writer schema to understand which fields and types were used to write the records.

JSONL is also convenient for record-at-a-time data, but it remains human-readable text and does not require a file-wide schema. Avro instead emphasizes machine interchange between records governed by schemas. Avro also has encodings other than Object Container Files, so not every Avro payload literally carries a complete schema inside the payload itself.

So which should you choose, Parquet or ORC? Arrow is a different layer

If you already have a data platform, start with the format that platform normally uses. Parquet is often an easy candidate when broad interchange across languages and analytics tools matters. If your existing environment is centered on Hive and ORC, staying with ORC is usually the more natural choice. Converting from one to the other does not automatically make files smaller or queries faster.

If compression ratio or query speed really matters, write the same real dataset with comparable settings and benchmark it using the engine and queries you actually run. Data ordering, column types, filter predicates, and writer configuration can all change the result.

Arrow is easier to understand as a different layer rather than as a storage format competing directly with those two. It provides an in-memory representation and IPC for processing and interchange. Apache Arrow itself distinguishes storage-oriented Parquet from Arrow’s computation-friendly in-memory representation.

So which one should you choose? Start from the job

A useful starting point is: CSV when people need spreadsheet-friendly tables, JSONL for record-at-a-time text workflows, Parquet or ORC for large analytical storage, Arrow for efficient in-memory interchange, and Avro when records are stored or exchanged according to an explicit schema.

Real systems do not need to choose only one. A pipeline might ingest JSONL, store analytical history as Parquet, use Arrow inside a processing engine, and publish schema-aware events as Avro. The number of formats reflects different stages and goals rather than six attempts to solve exactly the same problem.

Tips and limitations

• CSV and JSONL are usually plain text, so very large files may trade readability for processing efficiency. Workflows sometimes add an outer compression layer such as gzip.
• Parquet and ORC file sizes depend on the data, codec, encodings, and writer settings. The format name alone does not determine the compression ratio.
• Think of Arrow less as 'a faster Parquet file' and more as an in-memory columnar representation plus IPC for efficient processing and interchange.
• Avro supports schema resolution and evolution patterns, but practical compatibility still depends on writer schemas, reader schemas, and operational rules. Choosing Avro does not make every schema change automatically safe.
• For any format, being able to inspect a file is different from being able to edit and rewrite it. A read-only viewer can be useful when the goal is to examine the original without modifying it.