01
The question I couldn't get answered
No paper stands by itself. Every one adopts a method from somebody, argues with somebody else, and takes a few things on faith without saying so. That's what I want to know when I read a paper, and the usual tools don't tell you. They all run the same pipeline: chunk the paper, embed it, retrieve, answer. They summarise.
So I built a reader that answers one question well: what is this paper standing on?
A keystone is the one stone an arch collapses without, and for almost every paper that stone belongs to somebody else.
02
The first version worked, and that was the problem
Keystone didn't start as a lineage tool. The first version audited a paper's arithmetic: totals that should add up, bolded “best” results that should be the best, numbers in the abstract that should match the tables. It shipped, and it worked.
Then I ran it across nine heavily reviewed papers and it correctly found nothing. Careful papers don't have arithmetic errors, so on exactly the papers people care about most, the best possible output was a blank page. The next day I moved the product to lineage and assumptions, and pushed the numeric checks down below them.
One lesson from those checks still shapes the code. Every false alarm came from a check firing on the wrong premise (assuming a table compares down its columns when it compares across rows, say), not from bad arithmetic. So a check now has to prove its premise before it's allowed to state a conclusion.
On a careful paper, the best possible output was a blank page.
03
Reading a paper the way a careful reader does
Keystone works from the arXiv LaTeX source, not a model's summary of it. The key idea is that a citation isn't a neutral pointer. “Following [12] we normalise each layer” and “Unlike [12], we normalise across the batch” say opposite things about the same work, and the difference is written right there in the sentence. Each citation is classified from its cue phrase, and the cue is shown, so you can see why it was read that way and disagree.
Assumptions get the same treatment. Papers announce them because the genre demands it (“we hypothesize”, “for simplicity”, “it is well known that”), and then nobody comes back to them. Keystone quotes each one next to whatever the paper offers for it in the same sentence: a citation, its own evidence, or nothing at all. ResNet's central hypothesis, that residual mappings are easier to optimise, is stated once in the introduction with nothing offered for it. Page 2.
None of this involves a language model. Every reading is a quote, and every quote is a click away from the exact words on the page.
04
Following the chain back
Where a cited paper is also in the library, the link works both ways, so a paper shows what it stands on and who stands on it. The library grows along its own citations: it fetches the papers its current papers take a stance on most, so each new paper arrives already connected.
The most useful view turned out to be asking how two papers relate. Every link carries the sentence that created it, so a path between two papers is a chain of things they actually said about each other. LLaMA traces back to VGG in three steps. Each step was visible on its own paper's page, but the chain wasn't visible anywhere.
I capped paths at four steps and chose the shortest over the “strongest”. Past four, almost everything in a library this size connects to everything else. And a long chain of weak links isn't a better answer than none. It's worse, because it looks like an answer.
05
Grading myself on someone else's test
For the first few days, every accuracy number I had came from 90 citations I'd labelled myself, drawn from the same papers I'd written the rules against. That isn't a measurement, it's a restatement. So I scored the rules against two public datasets that had contributed nothing to them.
Keystone is built to stay quiet rather than guess, and the numbers show both sides of that. When it takes a stance it's usually right; it just doesn't take one very often. Its macro-F1 is low, and on ACL-ARC it's below the 0.27 a cue-phrase system from 2000 scored. I published that number too. Leaving it out would have been the convenient choice.
The real value of the outside test was what it caught. Three genuine bugs came out of it:
| Benchmark | Citations | Takes a stance on | Right when it does | Macro-F1 |
|---|---|---|---|---|
| SciCite | 11,014 | 4.0% | 80.0% | 0.33 |
| ACL-ARC | 1,941 | 8.3% | 68.9% | 0.20 |
Neither dataset contributed a single cue phrase. Neural classifiers score around 0.84 macro-F1, because macro-F1 punishes silence as hard as error.
- Comparison cues never checked who was being compared. “Women are disproportionately more frequently affected compared to men [12]” compares two populations, not this paper against a citation, and 136 of 223 readings of that kind were wrong that way.
- Adoption cues insisted on “we” in the same clause, so “employ Collins head rules (Collins 1999)” was missed.
- A paper's own name counts as a subject. “Unlike recent models, BERT is designed to…” was losing BERT the two relationships it's best known for.
06
Pinning every claim to the page
A finding is only useful if you can click it and land on the exact words, so I measured the anchoring layer before building anything on top of it. 99.94% of anchors survive an independent round trip, and 0 of 359 deliberately fabricated quotes get anchored.
The most alarming result came out of the tests. Fuzzy matching briefly accepted 5.9% of fabricated quotes while matching every real one, because changing a single word barely moves the score. For a tool whose whole job is checking what a paper says, quoting something the paper doesn't say is the worst possible failure. Fuzzy matching is now opt-in, behind a guard against swapped words.
Later, as the library grew, anchoring quietly slipped to 89%. The cause was sentences allowed to run across breaks the page puts in but the LaTeX doesn't: run-in headings, the end of a list, an abbreviation at the end of a bullet. Each fix revealed the next one. Together they took anchoring back to 92%, the worst paper from 56% to 89%, and fabricated quotes from 4 to 0.
07
Things I got wrong
The commit history is honest about these, so the case study should be too.
- arXiv kept refusing downloads, and I blamed arXiv. Then I retried six refused papers 20 seconds apart and all six worked, so I decided it was my own impatience and slowed down. A full run later, it had barely helped: arXiv works on a budget over a time window, not a rate you can beat by being patient. I wrote that correction into the next commit rather than leave the first claim standing.
- Each paper's link preview worked on my machine and shipped to production reading “0 works it stands on” to anyone who pasted a link. Nothing in the build, the types or the tests would have caught it. Checking the deployed image did, and I've checked deployed output since.
- I tested a dropdown by clicking it in a script and confirming the right items came back. They did: the right sections, the right pages, and completely invisible, because the panel opened off the edge of the screen.
- Two databases were connected, and the code read the paid one first. That's the kind of mistake that shows up as a bill rather than a bug.
08
Rules it keeps
Keystone has a guide you can ask questions, and that's where a language model comes in, under one rule: it never writes a quotation. Retrieval picks candidate lines, the model replies with their numbers and a sentence on why each matters, and the server looks those numbers up in the stored readings. An invented number is dropped. An invented quote is impossible, because the text on screen never passes through the model.
Nothing is behind a login. Every paper, reading and assumption is open to a guest; signing in only adds highlights, notes and a synced reading list. And when I wanted to know how the site was used, I counted visits without cookies: a visitor is a one-way hash that changes daily, kept only as a count. It can say how many people came today, not who came back.
09
Letting agents read it too
A person reads Keystone in a browser. An AI assistant can't, so Keystone also runs as an MCP server, the protocol tools like Claude use to call other software. Any MCP client can point at keystone-research.vercel.app/api/mcp and get three tools: search what the library found, read one paper's citations and assumptions, and list every place the library argues with itself.
The point is the same as the site's: an assistant summarising a paper tends to paraphrase, and paraphrase invents. With Keystone it can quote the sentence instead, with the cue that decided it and a link to the exact line. A paper outside the library is parsed on demand in a few seconds.
I wrote it against the protocol directly rather than an SDK, as one plain web route: a few hundred lines with nothing to keep updated. It's stateless, because the hosting shuts each instance down after every request, so there's no session to lose. And it's public, read-only and unauthenticated, so there's nothing for an attacker to borrow. It's listed in the official MCP registry as io.github.Srikarmk/keystone.
10
Where it is now
73 commits over about five weeks. The library holds 197 papers with 437 paper-to-paper links you can walk. Any arXiv paper that ships its LaTeX can be read on demand in a few seconds. There's a timeline of how methods spread, a page of every place the library argues with itself, a feed of new papers that are one click from being read, and an MCP server so AI assistants can query the library and hand back links a person can check.
It's backed by 272 Python and 44 JavaScript tests. It is still often silent, and roughly one reading in five is wrong. That's why every reading shows the sentence and the cue it keyed on.