A phone app that read paper answer sheets and returned a score.
WISE and College Board were one continuous client relationship for us, starting with a school platform and moving into test preparation. We built machine learning image recognition that let students photograph a paper answer sheet and get a real score back.
- Product engineering
- iOS development
- Computer vision
- Test fixtures and QA
- Release management
The challenge
Students take SAT and PSAT practice tests on paper. College Board wanted them to score those tests instantly with a phone. The hard part was printed on the answer sheets themselves: bad lighting and shadows can affect the score. Add bubbles filled unevenly and pages that will not lie flat.
This was 2017. Reading marks off a camera frame meant building the vision work by hand, years before general AI models could do it.
What we did
A focused workstream
A named College Board workstream ran 14 weeks, from 12 June to 22 September 2017, established by its tracked hours. It grew out of earlier work with WISE, and our client roster records both as one relationship.
Ground truth first
On 20 November 2017 we filed three fully bubbled practice sheets with known answers: SAT Practice Tests 6 and 8 and PSAT/NMSQT Practice Test 2. Known answers are how you check that the engine read the paper correctly.
Ship and own the release
On 14 February 2018 we created the iOS distribution profile for the Question of the Day app under our own company identity, with TestFlight reporting on. The next day we archived release build 1.0.12. A twelfth point release is a product on a shipping schedule, not a prototype.
The result
The software shipped on a regular release cadence, with a twelfth point release in February 2018. In November 2019 we added College Board to the client list on our website.
Our records are uneven. The College Board work is documented. The WISE product side left no brief, wireframe or screen we can reach, so we show none.
What this means for your business
Before trusting any automated system with your data, build a small set of cases where you already know the right answer and check it against them. That discipline matters more as tools grow more confident, because a modern model can return a plausible wrong answer without any sign of trouble.
This project was delivered by the Innovation Theory team, the Denver firm behind InTheory, before InTheory existed as its AI practice.
Other case studies
Product · 2013–2020ShapeShift
An account-free crypto exchange we designed and helped build, which peaked at more than a million transactions a day.
Read the case study →
Product · 2016–2018SALT Lending
We held the fractional CTO seat at SALT Lending through launch, from a value proposition to a team of nearly a hundred.
Read the case study →
Brand · 2016–2019CoinDesk
Masthead, style guide, site, app and newsletter design, plus the Consensus event brand for its first New York shows.
Read the case study →Bring one expensive workflow. Leave with a clear next step.
In 30 minutes we look at one recurring workflow, what it costs you today and whether there is a credible case to improve it. If there is not, we will say so.
30 minutes with Phil or Martin. No technical brief, no pitch deck. Phil or Martin replies personally, usually within one business day.