Tools, not gods.

I've worked with AI tools for over a decade. They can be very useful, and I use them heavily. I'm still skeptical of pitches that jump from a convincing demonstration to the promise of an autonomous business.

What interests me is the work a system can do reliably, what it takes to run it, and whether the result is worth the effort.

What I want to see in a demonstration #

Show me the task and the inputs. Include an awkward case. Let the person who normally does the work examine the result.

Then show me what happens when the system is wrong. Who notices? Who fixes it? What information do they have? A useful demonstration should make those questions easier to answer.

That is also where I would start when considering an agent. Its responsibilities, permissions, and stopping conditions matter more to the decision than the label.

Where we use these tools #

The work includes drafting structured material, extracting information from documents, searching a defined body of knowledge, and organizing material for expert review. Some tasks involve several steps and access to other systems.

One research firm we work with has a qualitative methodology that depends on interpreting interview transcripts. We built tooling that processes the transcripts, surfaces emotional and symbolic markers, and organizes early findings for analysts to review.

Preparation that took hours now takes minutes, leaving the analyst more time to interpret the material. The tooling helps prepare it; the analyst remains responsible for the interpretation.

Running costs belong in the design #

A working prototype leaves several questions open. How often will the system be used? How much review does each result require? Who maintains the integrations and checks quality when something changes?

Model usage is part of that cost. So are the people and systems around it. Compare the options against the task and measure the cost of producing a result the team can use.

For model usage, I look at tokens per useful answer. A shorter response is not automatically better, but paying for a long answer that someone has to rewrite is a cost worth seeing.

A smaller model or a conventional piece of software may be sufficient for some steps. There is little value in adding a model call to a task that an existing rule handles well.

Keep the argument close to the evidence #

I do not use predictions about artificial general intelligence to decide whether a client should adopt a tool. A project needs to make sense with the capabilities available for the work being proposed.

That leaves room to experiment. It also leaves room to say that a demonstration is promising but not ready to rely on.

The same standard applies to skepticism. Dismissing everything under the AI label is no more useful than accepting every claim made for it. Test the application and be willing to change your view.

What I would ask before starting #

What will the system do? What will people still need to do? How will you measure a useful result, and who will be responsible after launch?

Those questions do not settle every technical issue. They give the team a reason to examine the ones that matter to the project.

Tools, not gods. I want to understand what I am working with, use it where it helps, and keep checking the results.