IO List Generator

About the client & problem statement

Our client integrates building automation systems — installing and configuring the Honeywell, Priva, and ISMA controllers that run HVAC, climate control, and lighting in commercial buildings. Every project begins with a technical drawing: a multi-page Visio document showing every sensor, actuator, and control signal, marked with arrows and shapes that indicate its type and direction.

Before any controller can be selected or wired, an engineer has to work through that drawing point by point — identifying whether each connection is digital or analog, input or output, which device it belongs to, and what it's called — and transcribe all of it into a structured IO list in the correct order. For an experienced engineer, that's one to two hours of careful, repetitive reading per document, and a single missed or misread symbol can carry through to a wiring mistake on site.

It's exactly the kind of task that's valuable to get right, tedious to do by hand, and — until recently — hard to hand off to software, because it depends on reading a visual convention, not just text.

Solution

We didn't build this as a fully automatic black box, and deliberately so. AI extraction gets you very far, but no model hits 100% on a document that mixes vendor-specific symbol conventions, project-specific naming, and drawings that have quietly evolved their own local dialect over years of projects. So we designed a human-in-the-loop tool: the AI does the reading, an engineer does the final check, and the two together are dramatically faster than either alone.

At the time, we built this on Google's Gemini — specifically because that model was recognized as the strongest at genuinely multi-modal reasoning, reading a rendered drawing and its underlying text together in one pass rather than stitching separate tools together, and because it was cost-effective enough to run across a full multi-page drawing set without turning AI spend into its own line item on every project. Prompted with few-shot examples and read against each page's legend, it returns a clean, ordered, structured IO list, ready for an engineer to check rather than build from scratch.

We wrote our own preprocessing pipeline that extracts the exact position of every element on the page, so the model reads the drawing the way a person's eye actually would — left to right, top to bottom — instead of whatever order the PDF happens to store it in internally. That's the piece of engineering that turns a jumble of disconnected symbols into a sequence the model can actually reason about.

Making sure the model sees the drawing correctly, before it ever tries to reason about it, is half the engineering here — what you hand it, and how you hand it, matters as much as what you ask it.

Conclusions

The task that used to take an experienced engineer one to two hours per drawing now takes a few minutes of double-checking. The engineering judgment hasn't moved — controller selection and IO assignment are still expert calls — but the hours of manual transcription sitting in front of that judgment are gone. This isn't a lab result either: the tool is in daily production use on real projects.

The bigger takeaway isn't really about drawings at all. This is about as niche as a software problem gets — one company's internal process, built around one specific type of engineering diagram — and that's exactly why it's worth paying attention to. If a quick run at ChatGPT doesn't nail a task like this out of the box, and for most genuinely specific business processes it won't, that's not proof the task is stuck being manual forever. It usually just means the task needs an engineer around it, not a bigger prompt.