Convert OpenStreetMap PBF to GeoParquet with QuackOSM

A current OpenStreetMap workflow does not need to pretend that PBF is unqueryable. Native DuckDB can read raw nodes, ways, and relations from an .osm.pbf file with ST_ReadOSM. QuackOSM can reconstruct feature geometries and write analysis-ready Parquet/GeoParquet. Those are different jobs:

This walkthrough uses the mutable Monaco Geofabrik extract, writes an explicit output path, validates the result, and avoids treating longitude/latitude degrees as meters.

Capture the source identity#

The “latest” Geofabrik URL changes as OpenStreetMap changes:

curl -L --fail \
  -o monaco-latest.osm.pbf \
  https://download.geofabrik.de/europe/monaco-latest.osm.pbf
 
shasum -a 256 monaco-latest.osm.pbf

Record the retrieval time, SHA-256, byte size, response ETag/Last-Modified if present, the Geofabrik Monaco download page (opens in a new tab), and the OpenStreetMap attribution/license requirements. A tutorial result from a latest extract is reproducible only when that downloaded object is preserved or identified by hash.

Query raw PBF directly with DuckDB#

This is a native DuckDB example:

INSTALL spatial;
LOAD spatial;
 
SELECT
  kind,
  count(*) AS entities
FROM ST_ReadOSM('monaco-latest.osm.pbf')
GROUP BY kind
ORDER BY kind;

ST_ReadOSM parses compressed OSM PBF into raw entity columns including kind, id, tags, refs, lat, and lon. It does not reconstruct arbitrary way/relation geometry for you. The official ST_ReadOSM documentation (opens in a new tab) explicitly makes that distinction.

For tagged point nodes:

SELECT
  id,
  tags['amenity'] AS amenity,
  lat,
  lon
FROM ST_ReadOSM('monaco-latest.osm.pbf')
WHERE kind = 'node'
  AND tags['amenity'] IS NOT NULL
  AND lat IS NOT NULL
  AND lon IS NOT NULL
ORDER BY id
LIMIT 20;

Counts and IDs vary with the downloaded extract. Publish the input hash with any expected result.

Convert buildings with the current QuackOSM API#

The current documented function is convert_pbf_to_parquet, not the older convert_pbf_to_geoparquet name. Pin QuackOSM in your project lockfile and print the installed version:

from importlib.metadata import version
from pathlib import Path
 
from quackosm import convert_pbf_to_parquet
 
print("quackosm", version("quackosm"))
 
output = convert_pbf_to_parquet(
    pbf_path=Path("monaco-latest.osm.pbf"),
    tags_filter={"building": True},
    result_file_path=Path("monaco-buildings.parquet"),
    keep_all_tags=False,
)
 
print(output)

Expected behavior: the function returns the explicit monaco-buildings.parquet path or its Path representation, rather than forcing downstream code to guess a generated filename. The row count changes with the OSM snapshot.

The QuackOSM API (opens in a new tab) documents tag-filter values as True, one string, or a list of strings. A Python dictionary cannot contain two independent building keys — the later key overwrites the earlier one — and an invented nested not filter is not part of that simple documented shape. Use the library’s documented custom filter mechanism when the simple OR-style tag filter is insufficient.

QuackOSM sorts by geometry by default. Its basic usage guide (opens in a new tab) explains that behavior and shows the current Monaco URL. Record whether sorting was enabled because it can affect file size and spatial locality.

Validate before analysis#

Run a GeoParquet-aware validator:

gpq validate monaco-buildings.parquet

Then inspect schema and row count in native DuckDB:

DESCRIBE SELECT *
FROM read_parquet('monaco-buildings.parquet');
 
SELECT count(*) AS building_features
FROM read_parquet('monaco-buildings.parquet');

Do not assume the geometry column’s exposed type. If DESCRIBE says BLOB, create a view that decodes WKB:

CREATE OR REPLACE VIEW buildings AS
SELECT
  * EXCLUDE (geometry),
  ST_GeomFromWKB(geometry) AS geom
FROM read_parquet('monaco-buildings.parquet');

If it already says GEOMETRY, alias it without ST_GeomFromWKB:

CREATE OR REPLACE VIEW buildings AS
SELECT
  * EXCLUDE (geometry),
  geometry AS geom
FROM read_parquet('monaco-buildings.parquet');

Use one branch, not both. A file opening in a general Parquet reader is not a GeoParquet validation result. The GeoParquet version guide explains the WKB-versus-native logical-type transition.

Correct point distance in meters#

This raw-node query finds amenity nodes within 800 metres of an example Monaco coordinate:

WITH amenity_nodes AS (
  SELECT
    id,
    tags['amenity'] AS amenity,
    lat,
    lon
  FROM ST_ReadOSM('monaco-latest.osm.pbf')
  WHERE kind = 'node'
    AND tags['amenity'] IS NOT NULL
    AND lat IS NOT NULL
    AND lon IS NOT NULL
)
SELECT id, amenity, lat, lon
FROM amenity_nodes
WHERE ST_DWithin_Spheroid(
  ST_Point2D(lat, lon),
  ST_Point2D(43.7384, 7.4246),
  800
)
ORDER BY amenity, id;

DuckDB documents ST_DWithin_Spheroid inputs as POINT_2D in latitude, longitude order and the threshold in metres. That is intentionally different from GeoJSON/GeoParquet WKB’s usual x/y longitude,latitude order.

This is wrong if geom contains unprojected longitude/latitude:

-- 800 means 800 coordinate units, normally degrees here — not metres.
ST_DWithin(geom, another_geom, 800)

For polygon/line operations, transform from the verified source CRS into an appropriate projected CRS.

Calculate building area in a projected CRS#

Assuming validation confirms the source is OGC:CRS84 longitude/latitude, Monaco falls in UTM zone 32N:

SELECT
  feature_id,
  ST_Area(
    ST_Transform(geom, 'OGC:CRS84', 'EPSG:32632', true)
  ) AS area_sq_m
FROM buildings
WHERE ST_IsValid(geom)
ORDER BY area_sq_m DESC
LIMIT 20;

The final true requests x/y axis order. EPSG:32632 is a regional choice for this Monaco example; choose and document a suitable projected CRS for other areas. ST_Area returns the square of the input coordinate units, so the transform is what makes square metres meaningful.

If the QuackOSM version names the identifier column differently, use the name shown by DESCRIBE. Do not silently invent feature_id.

Aggregate with ST_Union_Agg#

ST_Union accepts two geometries. The aggregate over a column is ST_Union_Agg:

SELECT
  ST_Union_Agg(
    ST_Transform(geom, 'OGC:CRS84', 'EPSG:32632', true)
  ) AS merged_buildings
FROM buildings
WHERE ST_IsValid(geom);

A union can be expensive and may not be necessary for a count, area distribution, or map. Keep it only when the merged topology is the intended output.

Preserve OSM semantics#

OSM is not a simple “buildings table.” Important caveats include:

• tags are open-ended and can be missing or community-specific;
• multipolygon relations can carry geometry assembled from member ways;
• a way tagged building may represent an outline, part, or other mapped concept;
• edits, deletions, and tagging conventions change over time;
• a regional extract clips at its boundary;
• licensing and attribution travel with derived products.

Publish the PBF hash, filter, QuackOSM version, output hash, validator report, feature count, and rejected/invalid geometry count. Those facts make the derivative auditable.

Browser inspection boundary#

Open the result in the GeoParquet inspector to review attribute columns and run non-spatial DuckDB SQL in the browser. viewparquet does not currently load the spatial extension, render OSM geometry, or validate GeoParquet metadata. Use native DuckDB/GPQ for those steps, then follow the MapLibre/PMTiles delivery guide when the output needs a web map.

Primary references#

• QuackOSM current API (opens in a new tab)
• QuackOSM basic usage (opens in a new tab)
• DuckDB ST_ReadOSM and spatial functions (opens in a new tab)
• Geofabrik OpenStreetMap extracts (opens in a new tab)
• OpenStreetMap copyright and attribution (opens in a new tab)
• GeoParquet specification repository (opens in a new tab)