How to work with Parquet files

Open the file in a Parquet viewer that runs in your browser. Drag a .parquet file into viewparquet and DuckDB-WASM opens it locally — no Python, pandas, Spark, or install required. You can browse rows, read the schema, and run SQL after the initial load.

Command-line alternatives also exist (parquet-tools, the pq CLI, or the DuckDB CLI), but they require an install and a terminal. A browser viewer is the fastest path for a quick look at an unfamiliar file.

Use a browser-based Parquet viewer such as viewparquet. You drop the file into the page and it is parsed locally in your browser; its contents are not uploaded to viewparquet for viewing or SQL.

This is useful when you receive a .parquet file and just want to confirm its contents, column names, and row count before pulling it into a heavier tool.

Use a tool that can query Parquet through a view and paginate results instead of intentionally materializing every row up front. viewparquet uses that approach, but success still depends on browser and WebAssembly memory, the device, file layout and codecs, and the query being run; there is no guaranteed maximum size.

Because Parquet is columnar, you rarely need the whole file — query only the columns and rows you care about with SQL (for example `SELECT a, b FROM file LIMIT 1000`) to keep memory low on large datasets.

Run a `LIMIT` query such as `SELECT * FROM read_parquet('file.parquet') LIMIT 10`. In viewparquet the grid requests a limited page rather than deliberately materializing the whole table first. The work required still depends on row-group layout, codecs, the source, and the query plan.

Query it with DuckDB, which can read Parquet directly without a separate import step. In viewparquet the loaded file is exposed as a table, so you can write `SELECT … FROM <table> WHERE …` in the SQL editor and get results in the grid. The browser build supports joins, aggregates, window functions, and other DuckDB SQL features available in DuckDB-WASM.

With the DuckDB syntax you can also reference a file by path, e.g. `SELECT * FROM read_parquet('data.parquet')`, and combine multiple files with globs like `read_parquet('data/*.parquet')`.

Yes. Engines like DuckDB query Parquet in place without an ETL or table-creation step. Projection and predicate pushdown can skip unneeded column chunks or row groups when the query, file layout, and available statistics allow it; a full scan still reads the data it needs.

Run `SELECT COUNT(*) FROM read_parquet('file.parquet')`. Parquet stores row counts in row-group metadata, so DuckDB can often answer without decoding every value. Time and bytes read still depend on the engine, file, and source.

Use `DESCRIBE SELECT * FROM read_parquet('file.parquet')` in DuckDB, or open the file in viewparquet and check the schema panel. Parquet is self-describing — column names, types, and nullability are stored in the file footer, so the schema is available without scanning the data.

Command-line equivalents include `parquet-tools inspect file.parquet`, `pq schema file.parquet`, and PyArrow’s `pyarrow.parquet.read_schema('file.parquet')`.

Parquet supports nested types (structs, lists, and maps) natively, and DuckDB can query into them with dot and bracket notation — for example `SELECT col.field`, `col[1]`, or `UNNEST(list_col)`. In viewparquet nested values can be displayed in the grid and projected or flattened with SQL.

Use `UNNEST` to explode a list column into rows, and dotted paths to project a single field out of a struct so you can filter or aggregate on it.

Parquet stores logical types (timestamp, decimal, date) on top of physical types (INT64, BYTE_ARRAY). When a reader ignores the logical type it shows the raw physical value — for example a timestamp as a large integer. Use a reader that honors logical types, such as DuckDB or Arrow, to display the correct value.

Timestamp precision (milliseconds vs microseconds vs nanoseconds) and timezone metadata are also stored as logical-type annotations, which is why the same column can look different across Pandas, Spark, and Athena.

Parquet keeps file-level and row-group metadata in its footer: number of rows, number of row groups, per-column compression codec, encodings, and min/max statistics. viewparquet shows the loaded table, column names, and DuckDB types; use `parquet-tools inspect`, `pq inspect`, `pyarrow.parquet.read_metadata`, or DuckDB metadata functions for raw footer, row-group, codec, or GeoParquet metadata.

Row-group statistics (min, max, null count) are what let query engines skip data; inspecting them helps you understand why a query is fast or slow.

First identify where opening failed: retrieving the expected bytes, locating the footer, binding one file or a multi-file schema, decoding pages, or materializing a result. Wrong HTML/XML or Git LFS bytes, an unclosed or truncated writer, encryption or reader incompatibility, damaged pages, remote access, and browser memory need different repairs.

Ordinary Parquet files use `PAR1` boundaries; encrypted-footer mode uses `PARE`. Boundary markers and `COUNT(*)` are screening checks, not proof that every data page decodes.

Spark and Hive often write a directory such as `data.parquet/` containing `part-*.parquet` files and marker files. Test individual members before diagnosing the whole dataset as corrupt.

Change the bucket or object-host CORS policy; viewparquet cannot bypass a cross-origin block in the browser. Allow https://viewparquet.com, GET and HEAD, the Range and signing request headers shown by the preflight, and expose ETag, Content-Length, Content-Range, and Accept-Ranges when the host sends them.

Use DevTools → Network to separate failure classes: no readable response or a failed OPTIONS request points to CORS, 403 points to authorization or signing, 404 points to the bucket/key, and 416 or a 200 response to a Range request points to range handling.

The guide includes a minimal AWS-style policy and the corresponding R2, GCS HMAC, and MinIO connection settings. Restrict the origin to the site you use, then tighten allowed headers after observing the actual preflight.

Generate a fresh presigned URL and paste the complete URL without editing, decoding, or dropping query parameters. Presigned URLs expire and bind authorization to request details; the wrong region or method, an object move, clock or credential expiry, CORS, or a URL-rewriting proxy can also produce 403.

A presigned URL is a temporary bearer credential. viewparquet stores URLs containing query parameters in session storage rather than durable local storage, but you should still clear Recent sources before sharing the browser.

Do not also enter access keys for the same presigned URL. Use the Public or presigned tab; use the Private bucket tab only for an s3:// path signed with credentials in the current browser session.

Use a GeoParquet-aware validator such as GPQ for a specification check; opening the table in a general Parquet reader only proves that the reader could read that table. viewparquet can show attribute columns, DuckDB types, raw geometry values, and SQL results, but it does not currently validate the required geo metadata, encoding, geometry types, CRS, or bounds.

For a reproducible reader check, the upstream five-row example tested on August 19, 2026 returned six columns in the shipped DuckDB-WASM reader: pop_est, continent, name, iso_a3, gdp_md_est, and geometry. That is deliberately a reader test, not a validation certificate.

A row group is a horizontal slice of the table stored together, and it is the unit query engines read and skip. A common target is 128 MB–512 MB (or roughly 100k–1M rows) per row group, balancing read parallelism against the per-row-group metadata overhead of having too many tiny groups.

Too many small files or tiny row groups (the "small files problem") hurt performance because engines spend more time on metadata than data. Compacting them into larger files helps.

With DuckDB it is a single statement: `COPY (SELECT * FROM read_csv_auto('data.csv')) TO 'data.parquet' (FORMAT PARQUET)`. In viewparquet you can load a CSV, TSV, JSON, or JSON Lines file and export the result as Parquet directly from the browser.

CLI tools such as `pq convert data.csv -o data.parquet` and `parquet-tools import` do the same conversion from a terminal.

Run your query, then export the result set. In viewparquet the results grid can be exported to Parquet or CSV. With the DuckDB CLI, wrap the query in `COPY (…) TO 'out.parquet' (FORMAT PARQUET)` or `(FORMAT CSV, HEADER)`.

Use Parquet for analytics and storage of large or wide datasets, and CSV for small, human-readable, interchange data. Parquet is columnar, compressed, and self-describing (it keeps types and statistics), so it is far smaller and faster to query; CSV is plain text with no types and must be fully scanned.

A practical workflow: keep raw exports as CSV/JSON, convert to Parquet for repeated querying, and inspect either format the same way in a viewer before trusting it.

In viewparquet, click Open from S3 / URL, then use Public or presigned for HTTPS and public object paths, or Private bucket for an s3:// path signed with in-browser AWS, R2, GCS HMAC, or MinIO credentials. Compatible endpoints can serve HTTP byte ranges directly to DuckDB; actual transfer depends on the endpoint, query, statistics, and row-group layout.

To reproduce the public path, paste https://viewparquet.com/data/flights-200k.parquet with Auto-detect selected, then run `SELECT count(*) AS rows FROM data`; the repository copy contains 231,083 rows.

Private keys and STS tokens are not stored with Recent sources. Presigned URLs already contain temporary authorization and are kept only in session storage, so treat them as bearer credentials and clear recents on a shared browser.

For CLI workflows, DuckDB httpfs also supports read_parquet('s3://bucket/key.parquet') after configuring secrets in a local session.

Use a viewer that processes local files in your browser. In viewparquet, local opening, querying, and exporting run on-device with DuckDB-WASM and do not upload file contents to viewparquet; the Network panel can verify that. Remote sources connect to their configured host. Optional AI sends messages and applicable context to the provider you choose; sharing controls cover preloaded schema, profile fields, and samples, while chat tools and saved AI context may add structural metadata or SQL.

Open the shard in a viewer and check the things that break training: schema and column types, null counts, row count, and the shape of embedding or token columns. viewparquet lets you eyeball rows and run SQL (distincts, null checks, length of array columns) on each Parquet shard locally before kicking off a fine-tune or eval.