Building an OCR System for Restaurants: Harder Than Expected
Key lessons from developing Thaam-Sang: model tuning, edge cases, and real-world kitchen testing.
Introduction
Thaam-Sang began with a mission to help small local eateries manage rush-hour orders written by hand on slips of paper. In messy restaurant environments, illegible handwriting and misplaced order slips frequently cause chaos. Bringing computer vision and OCR to transform handwritten paper into digital kitchen tickets was an exciting challenge.
Kitchen Realities
Under real kitchen conditions, lighting is uneven, paper slips are stained with grease or creased, and staff write in hurried shorthand. Off-the-shelf cloud OCR models regularly misrecognized terms or completely skipped crucial custom modifiers.
“Good AI technology isn't what works cleanly in an academic lab; it's what stands up resiliently to real-world friction and unpredictable conditions.”
OCR Pipeline Architecture
I designed a robust multi-stage pipeline: automated image enhancement and contrast normalization first, followed by text bounding-box detection, and a recognition model tailored specifically for Thai culinary terminology.
Parsed lines are subsequently processed by an intelligent fuzzy matcher that pairs dishes with customization tags—like 'no spicy' or 'runny fried egg'—dispatching tickets straight to the kitchen queue.
Tech Stack & Models
The AI microservice is built using Python and FastAPI, serving fine-tuned vision models. The restaurant dashboard is implemented in Next.js with WebSockets for instantaneous real-time kitchen display updates.
Field Lessons
Handling edge cases matters far more than theoretical benchmark scores. Providing a 1-second human review verification interface created vastly better user trust than chasing a mythical 100% automated model accuracy.
What's Next
We are exploring edge quantization to run inference directly on local edge hardware, cutting cloud latency and ongoing operating expenses for family-run eateries.