Bulk data

Snapshot

The snapshot is the complete OpenAlex database as downloadable files. It’s stored in Amazon S3 in the openalex bucket, under the data/ prefix — free to download, no AWS account needed (see Access & authentication). For step-by-step download commands, see the download recipe.

Two formats

The snapshot is published in two formats, each a complete copy of the same data:

Format Compression One record per Prefix
JSON Lines gzip (.gz) line /data/jsonl/
Apache Parquet snappy (.parquet) row /data/parquet/

Size

Check the current size before downloading — it grows over time. As of the June 2026 release, each format holds about 649 million records:

Compressed size
JSON Lines (/data/jsonl/) ~750 GB (works alone: ~670 GB)
Parquet (/data/parquet/) ~780 GB

Decompressed, the JSON Lines data runs to several terabytes. The two formats are separate complete copies, so downloading both roughly doubles the transfer — most users want a single format prefix. Live totals are always in each format’s manifest.json (content_length, per entity and overall).

Bucket structure

Under each format prefix there is one folder per entity type, plus a combined manifest.json:

s3://openalex/data/
├── jsonl/
│   ├── manifest.json          # all entities, this format
│   ├── works/
│   │   ├── manifest.json       # works only
│   │   └── updated_date=2026-06-24/
│   │       ├── part_0000.gz
│   │       └── part_0001.gz
│   ├── authors/
│   └── ...
└── parquet/
    ├── manifest.json
    ├── works/
    │   ├── manifest.json
    │   └── updated_date=2026-06-24/
    │       ├── part_0000.parquet
    │       └── part_0001.parquet
    └── ...

The entity folders under each format are:

Core entities Topic hierarchy Lookup / aggregation entities
works topics keywords, concepts
authors subfields continents, countries
institutions fields institution-types, source-types, work-types
sources domains languages, licenses, sdgs
publishers, funders, awards

You can browse the bucket at openalex.s3.amazonaws.com/browse.html.

Records are partitioned by updated_date: each partition holds the records that last changed on that date, in part files (part_0000.*, part_0001.*, …) of up to 400,000 records each. This is what makes incremental updates cheap — see Updates & releases for how partitions move between releases and how to keep a copy current.

Note: Pre-2026 snapshots used a flat data/{entity}/ layout (JSON Lines only) with no jsonl//parquet/ split. That older layout, along with the merged_ids/ directory, is preserved under the legacy-data/ prefix.

The manifest files

Every format has a combined manifest.json listing all data files across all entities (/data/{format}/manifest.json), and every entity has its own (/data/{format}/{entity}/manifest.json). The manifest is written last, after all data files are uploaded — if the manifest is present, the data for that format is complete.

A per-entity manifest looks like this:

{
  "date": "2026-06-25",
  "format": "jsonl",
  "entity": "works",
  "record_count": 510372821,
  "content_length": 665688383258,
  "files": [
    {
      "url": "s3://openalex/data/jsonl/works/updated_date=2026-06-24/part_0000.gz",
      "meta": { "content_length": 936733, "record_count": 499 }
    }
  ]
}

The combined manifest has the same shape but nests one entry per entity under an entities array, with a top-level meta carrying the totals. The manifest drives the incremental-update workflow.

Entity schemas

Each record is a complete entity object in the same shape the API returns — the field definitions in Work fields and the API entity pages apply directly:

How the snapshot differs from the API

People often compare counts or fields between their snapshot copy and the live API and worry when they differ. The known, expected differences:

Difference Why
Works count: snapshot ~510M vs API default ~322M The snapshot contains all works, including the expansion (XPAC) corpus. The API excludes XPAC works by default; add include_xpac=true to match. Filter locally on the is_xpac field to reproduce the API’s default view.
Freshness The snapshot is a point-in-time release (roughly monthly); the API updates continuously. Counts and records drift between releases — see Updates & releases.
content_urls is absent It’s generated at serve time, API-only. The snapshot has has_content; build download URLs via the content endpoint with work IDs.
Some works have abstract_inverted_index: null Not every work has an abstract (availability and publisher restrictions) — same as the API.
No n-grams The old n-grams dataset is retired and was never part of this snapshot layout.

Otherwise the snapshot work schema matches the API’s — including fwci, topics, citation_normalized_percentile, and is_xpac.

View as Markdown
View as Markdown