Case study · Product

The machine read
the paper.

In 2017 the word was machine learning and the job was quieter than it sounds: let a student photograph a paper answer sheet with a phone and get a real score back. Bubbles, shadows, lighting, a page that will not lie flat.

Five years before anybody was arguing about chatbots.

SAT / PSAT PRACTICE · 2017 PAPER NO. 2 PENCIL PHONE CAMERA FRAME SCORE INSTANTLY THE HARD PART BAD LIGHTING SHADOWS ACROSS THE SHEET A PAGE THAT WILL NOT LIE FLAT GROUND TRUTH: 3 ALL-CORRECT SHEETS
Pixels into a score

Phil remembers two clients.
The roster says one.

Phil’s account, unedited

A year of work and it bled into… another school platform. The testing platform. It worked well with them, their app… a big scholastic platform.

Phil · Innovation Theory, on the record

The Ideas By Nature client roster lists won deals by organisation. One row reads, in the organisation column, Wise/College Board, with two won deals against it. Not two rows. One. Internally it was filed as a single continuous relationship, which is the documentary form of “it bled into”.

The memory and the spreadsheet agree, and neither knew about the other.

Four dates carry
the whole engagement.

01
Fourteen weeks12 June to 22 September 2017
A named College Board workstream running across the summer, established by its own tracked hours.
02
Ground truth, three files, two minutes apart20 November 2017
SAT Practice Test 6, SAT Practice Test 8, and PSAT/NMSQT Practice Test 2, all filed as AllCorrect. Fully bubbled sheets with known answers. That is what you feed a scoring engine when you need to know whether it read the paper right, and it is the clearest evidence on this page of what the app was for.
03
Signed for release14 February 2018
An iOS distribution profile under bundle identifier com.ideasByNature.QOTD, team name Ideas By Nature, LLC, with TestFlight reporting enabled and debug access off. QOTD is Question of the Day. The bundle identifier is the tell: the App ID was registered under our company, not the client’s. We held the signing identity for the thing that shipped.
04
A twelfth point release15 February 2018
An Xcode archive of a release build, version 1.0.12, build 5-2, cut the day after the profile was minted. A twelfth point release is not a prototype. That is a product on a shipping cadence.

In November 2019 the College Board logo was added to our own website’s client strip, alongside ShapeShift, Komodo, SALT, CoinDesk, KeepKey and Neblio. A client we put on the front page.

The client’s own instructions
describe the product.

Printed on the practice answer sheets themselves:

On the front of the sheet

Download the College Board SAT Practice app to instantly score this test.

College Board’s printed instruction

On the reverse

If you’re using our mobile app keep in mind that bad lighting and even shadows cast over the answer sheet can affect your score.

The hard problem, stated by the client, on the paper

Computer vision doing optical mark recognition on a camera frame, in 2017.

That sentence is our inference from the fixtures and the printed instructions, not a claim from a surviving technical document. We are labelling it as inference because no document in our hands names the technique.

A year of work that left
no design record.

Every file in the drive whose title contains WISE is a financial or entity document for a company, and all of it was out of scope. There is no product brief, no wireframe, no screen, no spec and no roadmap for WISE the application anywhere we can reach. The year Phil remembers left nothing we can show you.

We can prove the College Board half. We cannot show you the WISE half.

Both sentences stay here. This page carries no pictures for the same reason: none survive in our hands, and we are not going to borrow a screenshot of somebody else’s app and imply it is ours.

Product · iOS · computer vision · 2017 to 2018 · engagement type: PRODUCT

Scope and available records

What the files hold, and what they do not.

This page is built from the project drive, and where the record is thin it says so. The fourteen-week College Board workstream is established by a billing file and its tracked hours; that file was not mined for anything else. The client roster it sits beside was last modified January 2020.

No screens from this engagement survive in our hands. The diagram on this page is our own schematic, not an artifact. The instructions quoted are printed on the College Board’s own practice answer sheets.

Speculation · not from the record

How AI would help
this now.

Everything above this line is in a file somebody can open. Nothing below it is. This is how we would run the same engagement today, with AI in the work — written as an opinion, because that is what it is.

This is the one case on the wall where the thing we built is the thing that got good. Reading a bubbled sheet off a phone camera in 2017 meant hand-built vision work and a stack of known-answer fixtures to test it against. A model does that now, out of the box, in worse lighting than we could handle, and it will read the handwriting in the margin too.

The fixtures would still be the job.

Three fully bubbled sheets with known answers is how you find out whether the machine actually read the paper, and that discipline gets more important as the reader gets more confident, not less. A 2017 vision pipeline failed loudly and obviously. A model fails fluently, hands you a plausible score, and never mentions the shadow across question forty.

WISE and College Board started the way
every entry here starts: with a question.

The question was whether a student could photograph a paper answer sheet with a phone and get a real score back. Bubbles, shadows, lighting, a page that will not lie flat — and it shipped, in 2017. An assessment finds out what holds before you fund the whole thing. Two days, no charge, answered in writing.

Let’s find out

Not ready? Take the Guide: the six rules with checklists you can run yourself. Free. No email required.