AI Research Assistants: Verify Every Citation Before You Cite

Buy these tools for recall, not for trust. Elicit scored 85.5 and SciSpace Deep Review 84.6 on ScholarQA-CS2, the literature-QA benchmark inside Ai2's AstaBench. Read the paper's methodology and it says those citation metrics never check whether the cited papers exist or whether the quoted lines appear in them. No vendor and no benchmark is doing that job. Below: what each tool hands you to verify with, and the four checks you run yourself.

Key takeaways

• AstaBench's authors write that the citation metrics 'do not verify that citations refer to real papers or that quoted snippets actually appear in the cited sources'
• For Elicit, SciSpace Deep Review and Ai2's own Asta Scholar QA, the paper says the higher performance is driven by the citation subscores — the exact metric that skips both checks
• Judge the tools by what they export. A CSV of DOIs is auditable in minutes; a PDF of prose is not. Elicit's help center states its library cannot be exported at this time
• The verification infrastructure is free and already exists: doi.org resolution, the Crossref REST API with no sign-up, and the Crossref/Retraction Watch open dataset
• Elicit's free plan searches more than 138 million papers; Pro is $49 per user/month billed as $588 annually (checked August 2026)

The scoreboard, and the 57 agents behind it#

AstaBench is the most detailed public scoreboard in this category. Ai2 published it as arXiv paper 2510.21652 for ICLR 2026, covering more than 2,400 problems across 11 benchmarks in four areas: literature understanding, code and execution, data analysis, and end-to-end discovery. The abstract reports an evaluation of 57 agents across 22 agent classes.

The headline is bleak and useful. The best open-source agent using closed models, Ai2's own Asta v0, reached 53.0 percent. The best open-source agent running open-weight models — Smolagents Coder with Llama-4-Scout — managed 11.1 percent, which the paper itself calls terrible. The authors' summary is that AI remains far from solving the challenge of science research assistance.

ScholarQA-CS2 is the literature-QA test everyone quotes from that paper. Here is what it recorded, with 95 percent confidence intervals.

ScholarQA-CS2 scores from AstaBench Table 6 (arXiv 2510.21652), selected systems

System	ScholarQA-CS2	Note
Asta Scholar QA with Tables (claude-sonnet-4)	87.9 ±1.2	Built by Ai2, which also built the benchmark
Asta v0 (mixture)	87.7 ±1.4	Best overall AstaBench agent at 53.0% across all four areas
Asta Scholar QA (gpt-5)	85.9 ±1.6	Same system, different underlying model
Elicit	85.5 ±1.6	Marked closed and UI-only; vendor self-submitted its answers
SciSpace Deep Review (claude-sonnet-4)	84.6 ±1.3	Commercial
FutureHouse Crow	81.1 ±1.7	Commercial
OpenAI Deep Research	79.4 ±1.4	General-purpose, not literature-specific
STORM	78.3 ±2.4	Open research system
Perplexity Sonar Deep Research	67.3 ±1.2	General-purpose
OpenSciLM	58.0 ±2.6	Open system; source of the test questions
You.com Research API	55.0 ±2.2	General-purpose

Two structural facts about that table matter more than the ordering. Elicit is marked closed and UI-only, and the authors record that Elicit chose to submit its own answers, which they then processed from an offline cache. And the top scorer was built by the same institute that wrote the benchmark.

Why an 85 percent citation score is not 85 percent real citations#

The ScholarQA-CS2 score averages four measures: citation recall, citation precision, answer relevance and answer coverage. All four are scored by a language model acting as judge — gemini-2.5-flash is the official evaluator, chosen over gemini-2.5-pro, with a Pearson correlation of 0.995 between the two judges.

Now read the methodology note. The authors write that these citation metrics "do not verify that citations refer to real papers or that quoted snippets actually appear in the cited sources."

That single sentence is the whole problem. The judge decides whether a claim looks supported by its citation. Whether the paper exists, and whether the quote is real, are assumed rather than tested.

The authors knew this was gameable. They record discarding snippets from systems that lack access to real literature, on the grounds that those systems would likely hallucinate and receive inflated scores. That is an explicit admission that the metric cannot tell fabrication from evidence on its own.

Then comes the line that should reorder your reading of every roundup quoting these numbers. The paper states that for Asta Scholar QA, Elicit and SciSpace Deep Review, the higher performance is driven by the citation subscores. Their lead comes from precisely the measure that skips the two checks you most need.

Three more caveats belong on the record. The test set is 100 questions, all from computer science. System-level agreement between human experts and the model judge was Kendall tau 0.467, rising to 0.800 only when Elicit outputs that experts systematically disliked were excluded. And a held-out bias check found statistically significant answer-coverage drops for Asta Scholar QA with Tables, Elicit and SciSpace Deep Review — an average 2.5 point drop, p ≤ 0.01 — when those systems were removed from the pool used to build the scoring rubrics.

None of that makes AstaBench bad. It makes it honest. The people who built the best measurement in this category are telling you, in the methods section, that the measurement does not cover the thing you care about.

The four-check verification protocol#

Every check below runs on free public infrastructure. Steps one to three are scriptable. Step four is not, which is why it is the one people skip and the one that catches the failures that matter.

1. Resolve the DOI

   Paste the DOI after https://doi.org/ and see whether it reaches a publisher record. The DOI Foundation's factsheet shows the pattern with a worked example: https://doi.org/10.1037/0003-066X.59.1.29 resolves to the publisher's landing page for that article, using the Handle System underneath. If the tool gave you no DOI, the citation is unresolved until you find the record another way — treat it as unverified, not as fine.

2. Pull the metadata from Crossref

   Query the Crossref REST API for the same DOI and compare title, authors, year and journal against what the tool told you. Crossref's documentation states that no sign-up is required to use the REST API, and that almost none of the metadata is subject to copyright. The single-record endpoint returns one metadata record for a Crossref DOI. Supply your email address to use the polite pool, as Crossref recommends. A mismatched year or a different author list is the cheapest early warning you will get.

3. Check retraction status

   Crossref acquired the Retraction Watch database from the Center for Scientific Integrity on 12 September 2023 and made it a public resource. At the time of that announcement, Crossref held roughly 14,000 retraction records and Retraction Watch around 43,000, combining to approximately 50,000 after overlap. For one-off checks, search retractiondatabase.org — it is free, and since 23 October 2024 it requires criteria in at least one field. It shows at most 50 entries per search, though a banner reports the total number of matches. For bulk checking, take the full dataset as a CSV from Crossref's Labs API or its GitLab repository.

4. Find the sentence

   Open the paper and locate the text that supports the claim. If the tool supplied a quote, search for that exact phrase. If it supplied only a title and a summary, you are in the half-credit zone the benchmark tolerates and you should not: read the abstract, then the results section, then decide whether the paper says what the tool said it says. This is the check no vendor performs for you and no benchmark scores.

How many citations to check#

Check all of them when the work leaves your hands. Anything you submit, publish, file with a regulator or put in front of a client gets 100 percent verification, no exceptions and no sampling.

For internal decisions, sample. Check every citation that carries a number, every citation attached to a negative or null finding, and every citation you intend to quote — then take a random 20 percent of what remains. Negative findings and numbers are where a plausible-looking mismatch does the most damage.

For exploratory reading, check the citations your argument actually rests on and let the rest stay unverified — but mark them as unverified in your notes. The failure mode is not skipping a check. It is forgetting which ones you skipped. The AI output quality testing guide sets out a broader method for pressure-testing model output.

Elicit: the strongest exports, and a library you cannot take with you#

Elicit searches the Semantic Scholar database and draws on major model providers, per the AstaBench paper. It is the best-scoring non-Ai2 system on ScholarQA-CS2 and the most generous free tier in this roundup.

Export is where Elicit separates itself, and also where it disappoints. Research Reports export as PDF or Word on every plan. Find Papers and Paper Chat tables export on Elicit Plus and higher, systematic review tables export for Pro, Team/Scale and Enterprise, and the formats on offer are CSV and Excel for tables plus RIS and BIB for source lists.

Then the sentence that should govern your buying decision: Elicit's help center states that its library cannot be exported at this time. Your saved corpus — the accumulated result of your searching — stays inside Elicit.

On accuracy, Elicit is careful in public and thin on numbers. Its reliability page lists mitigations — process supervision, prompt engineering, ensembling multiple models, checking results with custom models — and says it runs internal evaluations of how common hallucinations are, without publishing the findings. Its evaluation page offers one performance statement, that in Elicit's testing the tool is "often more accurate than humans," with no percentage, task or method attached.

The vendor's own published study is more informative than its marketing. Elicit had 17 PhD-holding professional researchers rate whole reports from 0 to 10, then grade the top five claims in each for accuracy and citation support, across 29 Elicit reports and 120 competitor reports. Its own conclusion: several tools performed statistically equivalently to Elicit. The write-up lists its limitations too — it paid the evaluators, it reused the same researchers across rounds, and it notes their usage may not represent other users. Competitor sample sizes ranged from 11 reports for Google Gemini to 29 for Undermind.

Rating

4.1/5

Search recall

4.6

Export and auditability

3.8

Published evidence

3.5

Price transparency

3.2

Free tier

4.7

The best starting point in the category, and the free tier is genuinely usable for discovery. Buy Pro only when you need to screen at 5,000-paper scale — and accept that your library stays inside the product.

Gemini Notebook: closed-corpus grounding, and where its citations stop#

NotebookLM is now Gemini Notebook. Google describes it as the same standalone product under a new name, with code execution in a secure cloud environment and cross-app syncing with the Gemini app and Search.

It solves a different problem from Elicit. It does not search the literature; it answers from the sources you upload. Google's FAQ is explicit that if the answer is not in the source material, "it won't provide a response."

That closed corpus removes one failure mode entirely. The paper is real because you uploaded it. What remains is the harder question of whether the passage supports the claim — and here Google documents a specific limit: when source content is too short, Gemini Notebook references the whole document rather than citing individual text within it.

Free Standard gives 100 notebooks, 50 sources per notebook, 50 chats a day and 3 Audio Overviews a day. Plus raises that to 200 notebooks, 100 sources, 200 chats and 6 Audio Overviews. Pro gives 500 notebooks, 300 sources, 500 chats and 20 Audio Overviews. Its meeting-notes overlap is covered in the AI note-taking apps roundup.

Undermind, ResearchRabbit and Asta: three different verification surfaces#

Undermind runs a free plan at $0, Pro from $16/month billed annually (checked August 2026, described as saving 20 percent against monthly billing), and Team at $15 per person/month billed annually. Pro advertises 10x higher usage limits and the deepest analysis of full texts. Two named limitations. The free plan's ceiling is published only as "standard rate limits" on chats and searches, with no numbers, so you cannot size the free tier before signing up. And the homepage claims its v1 search engine delivered 10x better results than Google Scholar with no methodology published on the page — read it as marketing until Undermind shows the test. Undermind did appear as a comparator in Elicit's own study, with 29 reports evaluated.

ResearchRabbit works as a citation graph rather than a question-answering tool. You start from one paper and expand outward through authors, related works and emerging topics, watching connections form over time. It claims access to over 310 million academic papers. Its specific limitation is informational: the homepage publishes no pricing at all, and the only mention of Zotero sync appears inside a customer testimonial rather than a documented feature list — so plan your export path before you invest hours in a collection.

Ai2 Asta is the live product from the Allen Institute for AI, at asta.allen.ai, built in three parts: the Asta agents, AstaBench with its leaderboards, and developer resources. It is the only entry here whose maker publishes a full evaluation harness against itself. It is also the one topping a benchmark its own institute designed, and its Scholar QA variant was among the systems showing an answer-coverage drop in the held-out rubric check. The product page states neither pricing nor corpus size, so treat both as unknown until you check.

Your reference manager is the verification layer#

The tool that decides whether verification takes ten minutes or two hours is not the AI assistant. It is whatever holds the identifiers afterward.

Zotero is the sane default. Its homepage describes it as open source and "developed by an independent, nonprofit organization that has no financial interest in your private information," links its GitHub repository, and claims support for over 9,000 citation styles. Storage is where the money is: 300 MB free, 2 GB for $20/year, 6 GB for $60/year and unlimited for $120/year (checked August 2026).

Two honest limits. The free 300 MB fills quickly once you store PDFs rather than metadata. And Zotero's own export documentation confirms only Zotero RDF by name, recommending Zotero RDF with files and notes for the most complete transfer between users — so if you need a specific interchange format, confirm it in the app before committing a project to it.

The principle generalizes past any one product. A tool that hands you rows with identifiers is auditable; a tool that hands you a formatted document is a transcription job. That is why Elicit's unexportable library is a more serious constraint than its price, and why ResearchRabbit's undocumented export path deserves a test before your first long session.

The free infrastructure that does what the paid tools skip#

None of these services will write your review. All of them will tell you whether a citation is real, which is the part the paid tools leave unassigned.

Free scholarly infrastructure for citation verification (checked August 2026)

Service	What it verifies	Access terms as published
doi.org	That an identifier resolves to a publisher record	Resolution via the https://doi.org/ prefix, built on the Handle System
Crossref REST API	Title, authors, year, journal, funding, licensing, ORCID and ROR identifiers	No sign-up required; polite pool by supplying an email
Crossref / Retraction Watch dataset	Whether a paper has been retracted	Free and open; CSV via the Labs API and a GitLab repository
Semantic Scholar API	Papers, authors, citations and recommendations	214M papers, 2.49B citations, 79M authors; free with rate limits
OpenAlex	Works, authors and institutions as an open research graph	CC0 data; free API key required; $1/day of free usage credit

A note on rate limits, because they shape what you can automate. Semantic Scholar publishes 1,000 requests per second shared among all unauthenticated users, with throttling under load, and an introductory limit of 1 request per second for authenticated keys, with higher limits possible depending on use case. OpenAlex meters a free key by a daily credit rather than a request count, refreshes its free snapshot quarterly, and claims about twice the coverage of the other services, with significantly better coverage of non-English works and works from the Global South.

For a batch of 200 citations, that is enough to script steps one through three overnight and arrive at step four with a shortlist. If you are budgeting an automated pipeline around these calls, the LLM API cost estimation guide covers the arithmetic on the model side.

What could not be verified, and why it is not here#

Several tools that rank for this query publish pricing pages that refused automated requests, returning HTTP 403: Consensus, SciSpace and scite. Connected Papers answered, but served a page with no plan or price text in it. No prices, plan limits or corpus sizes appear here for any of the four — check current pricing on those pages yourself, on the day you buy.

Two figures widely repeated in other roundups are absent for the same reason. No primary vendor page states SciSpace's paper count, and none states scite's index size or its classifier accuracy. Search snippets are not sources.

Who this is not for#

If you need a systematic review that survives peer review, none of these tools is the method. They screen; you review. AstaBench's best result for generating literature-review tables was 42.9 on ArxivDIGESTables-Clean, with recall around 43 percent, which means more than half the expected content was missing from the best system tested.

If you work outside computer science, discount the benchmark accordingly. ScholarQA-CS2's 100 questions are all drawn from the computer science domain. Nothing in that table tells you how these tools behave on clinical trials, materials science or economics.

If your organization restricts sending unpublished manuscripts to third-party servers, resolve that before the trial, not after. Elicit is marked closed and UI-only in the benchmark, and Zotero is the only tool here whose homepage links public source code. Run the AI tool security checklist against any vendor before uploading a draft you have not published.

And if your work involves medical, legal or financial decisions, a citation check confirms that a paper exists and says something. It does not make you qualified to act on it.

How to choose#

Match the tool to how much the citation has to survive.

Discovery across the whole literature, with your own verification afterward: start on Elicit's free tier. Upgrade to Pro only when screening volume, not curiosity, forces it — and build your export habit from day one, because the library will not leave with you.

Reading a fixed set of documents you already have: Gemini Notebook. The closed corpus eliminates the existence question entirely, leaving only the support question, and the free tier's 50 sources per notebook covers most projects.

One hard question that needs exhaustive search: Undermind's free plan first, then $16/month billed annually if the depth earns it. Ignore the 10x headline and judge it on the papers it surfaces that you had not seen.

Mapping who cites whom: ResearchRabbit, with a test export before you commit a collection to it.

Every one of those paths: a reference manager holding DOIs, and the four checks. Resolve, compare, check retraction, find the sentence. The tools have made finding papers extraordinary and cheap. Deciding whether a paper says the thing is still your job, and every benchmark leader in this category quietly agrees.