The push is almost always top down. Leadership decides the organization needs to be doing AI, sets a target, and the teams below them start building. The problem is that the demand for most of what gets built was never really there. A goal was set and the goal was activity. Leadership is competing internally to show leadership above them how they are pushing the AI agenda, and the result is a race to ship more and ship faster where nobody stops to ask whether any of it is working.

Racing stripes on a Honda Accord

It is like putting racing stripes and a spoiler on a Honda Accord and calling it an F1 car. The optics are there but the engineering underneath has not changed. Quantity becomes the proof of progress, and quality never enters the conversation. And because each team is competing to show their own results, you end up with redundant tools solving the same problem in slightly different ways, cannibalizing each other’s adoption before either one has a chance to prove its value.

Looking for nails

It should not be a case of having a hammer and looking for something to hit. AI is not the answer to every problem, and treating it like one is how you end up with a catalog of tools that nobody asked for and nobody uses.

What that produces at the user level is cognitive overload. When there are too many prompts and too many tools, users do not know which one to trust or which one applies to their situation. Instead of reducing friction, you have added to it. The tool was supposed to make the job easier and instead it made the decision harder, because now the user has to figure out which tool to use before they can even start the work.

Show metrics versus impact metrics

What gets measured makes it worse. The metrics that get tracked are the ones that show activity, such as the number of models built, the number of prompts deployed, and how many teams have adopted AI in some form. These are show metrics because they demonstrate that something happened, not that anything changed. The impact metrics, whether users are actually using the tools, whether the tools are driving the outcome they were built for, and what the return on investment looks like, are rarely defined and almost never tracked.

What gets measured makes it worse. The metrics that get tracked are the ones that show activity, such as the number of models built, the number of prompts deployed, and how many teams have adopted AI in some form. These are show metrics because they demonstrate that something happened, not that anything changed. The impact metrics, whether users are actually using the tools, whether the tools are driving the outcome they were built for, and what the return on investment looks like, are rarely defined and almost never tracked.

The question that should come first

I build anyway. By the time it reaches the PM the decision has already been made above me, and I recognize that is the reality for most people in this position. But that does not mean the question should stop being asked earlier in the process, before the work starts and before the resources are committed.

Before any AI use case gets approved, someone should have to answer a few questions. What do you consider the value of the tool we are about to build, and how will it impact the users it is built for? Not in a vague sense, but specifically. What behavior are you trying to change, how will you know if it changed, and what does success look like beyond the number of people who have access to it.

AI is a capable tool. The problem is not the technology, it is the framing. When the goal is to show AI is happening rather than to solve a problem worth solving, the tools that get built reflect that. They exist to be counted, not to be used. And the data will eventually tell you what your users already know.

Author: Adam Dalal