01
Where the votes are
The City publishes roll calls in two places. boston.gov has a search page with every roll call from 2008 to the latest meeting, updated as each meeting is posted: 183 pages of paragraphs, each with a docket number, the clerk's description and a list of names and votes. Analyze Boston, its open-data portal, has a public-domain dataset with one row per councillor per vote, but it stops in March 2024 and hasn't been updated since.
Before building anything I checked that the project was allowed. The dataset's licence is public domain. The City's terms let anyone republish its information as long as they don't imply the City endorses them, use its seal or copy its look, so the site says it is independent and has its own design. boston.gov's robots.txt doesn't block the roll-call pages, and the scraper fetches them three seconds apart with a user agent that names the project. Meeting video isn't copied, because YouTube's terms don't allow downloads.
So boston.gov is the source for every vote on the site, including all 642 since March 2024, and the dataset is a cross-check on the older ones: of the 1,179 roll calls both have, 1,158 agree vote for vote, and the dataset repeats 40 roll calls that boston.gov has once.
02
Reading the records
Councillors appear by surname, sometimes misspelled, and votes appear as short codes. The cleaning step turns those into one record per councillor and writes down every change on the vote's own page, 348 notes on 131 roll calls:
- Misspelled surnames are read as the councillor they refer to, from a list checked by hand.
- Vacant seats are dropped, and a council president election keeps each councillor's choice instead of a yes or no.
- A stray letter in a name field is resolved only when exactly one councillor who voted elsewhere that day is missing.
- A surname that disappears for more than a year and comes back is shown as two stretches, because the records can't tell a returning councillor from a new one with the same name.
03
Records that contradict themselves
Most descriptions end with the clerk's tally, such as “yeas 10, nays 2 (Flynn, Murphy)”. The check compares that sentence with the votes in the same record. Of the 1,530 records that state a tally, 84 don't match it. In November 2024 the clerk wrote that a resolution to put the Election Department under state receivership failed, 2 yes, 7 no and 4 present; the record lists all 13 councillors as voting yes. Seventeen records list some councillors more than once, two of them three times over.
The first version of the check flagged more than that. I read about 20 flagged records against the City's page, and four kinds of flag were my parser's mistakes: “not present” and “absent” in one tally overwrote each other, descriptions with two votes in them were checked against the first, a councillor listed with no vote was counted as a disagreement, and the check for councillors from an earlier Council was too eager. Each fix has a test. Separately, 24 records list councillors who had already left; those are flagged and left out of the councillors' own records.
When two published versions disagree, show both and say which is which.
04
The site
The site is built from the pipeline's JSON into 1,874 static pages: one per roll call, one per councillor, and a page each for search, contradictions and the data's sources. Search and filters run in the browser on a compact index, so there is no server to keep up. Each councillor's page has a strip for every year, one square per roll call, and a table of how often they vote with each colleague when the Council splits.
Titles come from the City's own one-line summaries of each docket. In March 2026 the City said it was testing summaries generated by AI, so the About page says that, and every vote's page also shows the clerk's full text.
05
Keeping it running
A GitHub Actions job runs every morning. It fetches the newest three boston.gov pages and the dataset, rebuilds, runs the tests and commits the data if anything changed. The commit redeploys the site on Vercel. If boston.gov hasn't answered for a week, the job fails, so the failure email goes out. A quiet week shouldn't do the same, so a successful fetch is recorded every few days even when there's nothing new.
Running it turned up three problems no test had caught. The first full crawl died on page 144 when a connection dropped, and the pipe I ran it through hid the exit code; fetches now retry, and a crawl can resume. The first scheduled run committed even though no vote had changed, because Python's set order shuffled one file and two timestamps were rewritten every run; the build is now byte-identical across runs, and only new votes make a commit. And Vercel refused the first deploy because the Next.js version had a published security vulnerability; I upgraded it to the patched release.