Pulled works
A work is pulled when its publisher, its host (a journal, preprint server or repository) or its authors retracted, withdrew or removed it. OpenAlex keeps every pulled work and marks it with is_pulled. “Pulled” reports what someone else did to the work. It is not our judgment of the research. See every pulled work on the website.
The one thing to know: pulled works are left out of works lists, searches and counts by default. Add
include_pulled=trueto bring them back. Looking a work up by its ID or DOI always answers, withis_pulledshowing.
Why hide them, and why keep them
People use OpenAlex in two ways: to get at the knowledge in the scholarly record, and to study the record itself. For the first, pulled works should mostly just go away. Pulled means someone in the know took the work out of the record, yet retracted papers keep turning up in AI answers, and chatbots catch fewer than half of them. Hiding them by default keeps us from passing along bad information.
For the second, pulled works are really useful: for finding papers other systems still cite and should take down, for studying retractions (most come from misconduct, not honest error), or for comparing publishers by their share of pulled works. So we hide them by default and keep every one.
What counts
Pulled:
- Retracted works (
is_retracted) - Works the publisher withdrew or removed, as its Crossref record says
- Preprints and repository records their host marks withdrawn or removed (arXiv withdrawals, “WITHDRAWN:”, “Removed due to policy violations”)
- Conference abstracts the meeting withdrew, including after a no-show
Not pulled:
- Corrections, errata and addenda
- Expressions of concern
- Partial retractions
- A retract-and-replace: the corrected version stands
- A Cochrane review withdrawn because it is out of date
The last two need a word. Retraction Watch records a retract-and-replace, and a Cochrane review withdrawn as out of date, as retractions, so some of these works read is_retracted: true but is_pulled: false. A Cochrane review withdrawn for a problem with the review itself (unreliable data, misconduct) is pulled.
Left out by default
Pulled works are the third rule of the default view, beside the expansion corpus and pieces of other works. Nothing is deleted:
# The default view: pulled works left out
curl "https://api.openalex.org/works?search=ivermectin"
# Pulled works brought back
curl "https://api.openalex.org/works?search=ivermectin&include_pulled=true"
# Only the pulled works
curl "https://api.openalex.org/works?filter=is_pulled:true&include_pulled=true"
A filter or group_by that names is_pulled or is_retracted asks about pulled works, so it brings them back too, exactly as if you had added include_pulled=true. include_pulled=false keeps them out even then.
So filter=is_retracted:false no longer means “the default view without retracted works”. It means every work that isn’t retracted, withdrawn ones included. To leave pulled works out, add nothing.
In OQL, the word is pulled, like retracted: get pulled works about ivermectin, get the percent pulled of works in [Heliyon](S2898612692) by year. A query that names pulled or retracted (in a condition, a split or a percent) counts the pulled works. The website and the AI agent connector follow the same default.
Counts on authors, journals and institutions
Pulled works still count toward an author’s, journal’s or institution’s works_count, cited_by_count, h-index and i10-index. A list of an author’s works leaves them out, so the list can come out shorter than works_count. This may change later.
Where the flag comes from
- Crossref: a retraction, withdrawal or removal the publisher deposits on the work’s record or in a notice that names it.
- Retraction Watch: works it lists as retracted and never reinstated, read every day.
- PubMed: retractions it records.
- arXiv: withdrawn preprints.
- Copernicus meeting sites (EGU and others): abstracts withdrawn from the meeting.
- The work’s own title: a stamp such as “Retracted:”, “WITHDRAWN:”, “REMOVED:” or IEEE’s “Notice of Retraction”, unless the work is the notice about another paper.
- Curations: our curators’ corrections to
is_retractedandis_pulledwin over every source.
The flag is rebuilt every night. A retraction Crossref records today shows in the API tomorrow (10 to 38 hours later); one known only to Retraction Watch waits for Retraction Watch’s own update. A pulled work’s updated_date is the night it changed.
Known gaps
Copernicus withdrawals come from a one-time read of the meeting sites, so an abstract withdrawn since then can still read false. The notice itself (with its date and reason) isn’t linked to the work yet. If a work is pulled and we say it isn’t, or the other way round, report it.
Last updated
View as Markdown