ISSUE 02 // SHUBHAM GOEL // JUNE 2026
Can a small model tell you if you can park?
One week in the city. Two parking tickets, roughly $160, off a single pole carrying four stacked signs. So I taught a 3-billion-parameter model to read the pole I could not.
Two tickets, one week
I drove up from San Jose for a week in April and came home with two citations. Both times the cause was identical: I stood in front of a single pole holding four stacked signs and could not work out whether the curb in front of me was legal. A two-hour limit. Except with an Area permit. Except street-cleaning Tuesday mornings. Also tow-away during evening rush.
Every fact I needed was on the metal. The information was complete. What defeated me was the resolution step: take four overlapping rules, hold them against the current day and time, produce one answer before the meter maid does. Perception, plus rule-logic, plus a clock. My brain dropped a window. Twice.
That is a strange failure, because none of it is ambiguous. A parking pole is not a judgment call; it is a small, badly-formatted program. Which raised the question that became this project.
Read, then reason
The instinct is to show a model the photo and ask "can I park?" That fails, and it fails worst exactly where it matters, on the busy four-sign poles. Juggling four time windows, a permit exemption, and a severity ranking in one forward pass is too much for a 3B model. So curbcheck splits the job in two, and only the first half touches a neural network.
Read the pole
The VLM looks at the photo and transcribes every sign into structured JSON: kind, days, hours, limits, permit area, week-of-month. It only perceives.
Neural // Qwen2.5-VL-3BResolve the rules
A 120-line deterministic resolver (schema/rules.py) applies those rules to the current time. Strictest wins. No model in the loop.
Deterministic // pure PythonShow both halves
You get a verdict (ok / limited / no / tow-risk) and the reason, plus what each sign read. A misread is visible, not hidden in a black box.
UI // honest by designThe split is the whole trick. The model is allowed to be a fallible reader; it is never allowed to be a fallible logician. Because the logic is exact and inspectable, stacking signs does not break the answer, it just adds another line of JSON for the resolver to chew on. And because both halves are shown, you can always see whether a wrong verdict came from a misread or, impossibly, from the rules.
The pole that cost $160
PARKING
PARKING
CLEANING
permits: none
How it was built
The model
The base is Qwen2.5-VL-3B-Instruct. I fine-tuned it with QLoRA at rank 16, on the language layers only; the vision encoder stays frozen. The goal was to teach the model to reason about sign content, not to see differently. Training ran on Modal on a single A100, and the whole run cost roughly four dollars. Hold onto the frozen-encoder detail; it returns as the punchline.
The data
Two sources, labeled in completely different ways. About 77% is synthetic: sign-stacks rendered from public-domain Caltrans R-series specs (R26 no-parking, R30 time-limit, R32 street-cleaning), with rule distributions seeded by SFMTA's 144,333-row street-sign inventory so the mix looks like real San Francisco. Because each image is drawn from a rule object, the label exists before the pixels do; there is no annotation step and no label error.
The other 23% is real: SF street-space permit photos and 311 cases, pseudo-labeled by Claude Opus as a teacher (the same distillation move the Meta ads pipeline used) and human-verified. I added renderer realism on the synthetic side too, sky and building backgrounds, fading, occlusion, perspective, to attack the gap between clean renders and bleached Mission Street poles.
On clean signs, yes
| Metric | base | tuned |
|---|---|---|
| Read F1 | 0.34 | 0.985 |
| Reason, end-to-end (0.25 floor) | 0.16 | 0.82 |
| Reason, via pipeline | 0.83 | 0.99 |
The pipeline column is the product path: the model reads, the resolver decides. It posts the highest score because the resolver never makes a logic mistake.
Difficulty scales with the number of signs. The four-sign pole, the one that cost the tickets, is hardest at 0.56. The amber tick marks where an earlier model version sat (0.39). Route that same read through the resolver and four-sign accuracy jumps to 0.89.
The honest part
Synthetic scores are the easy victory. The real test is held-out San Francisco photos, faded, oblique, sticker-covered. Fine-tuning helps a lot there too: reading goes from near-blind to usable, and pipeline reasoning lands at a deployable 0.89.
I doubled the real data. Reading moved one point.
The whole v4 effort, double the real training photos, human-verified labels, two new sign types, and the renderer-realism augmentation, was aimed squarely at the sim-to-real reading gap. It moved real read F1 from 0.33 to 0.34.
And the reason is the detail to hold onto: the vision encoder is frozen. I fine-tuned the language layers; the eyes never changed. No amount of extra data, and no full SFT, can teach faded, oblique, real-world glyphs while the vision tower stays frozen. The next experiment writes itself: a vision-encoder LoRA, the one thing I have not unfrozen yet.
Meanwhile the architecture keeps paying off. Reading is stuck at 0.34, and pipeline reasoning still holds at 0.89, because the deterministic resolver never fluffs the logic. That split, the entire premise of the project, is what lets a half-blind reader still ship a useful verdict.
The poles it reads in the wild
Three held-out SF 311 photos. Real signs are sun-bleached, angled, and partly occluded; this is the distribution the frozen encoder struggles on.
Try it yourself
Curbcheck, live
Point it at any San Francisco parking pole. The Space runs the tuned model to read the signs, then the same deterministic resolver decides whether you can park, and shows both halves so you can catch a misread.