Four signs your company may be using AI the wrong way

AI is everywhere, and your job probably has some sort of AI remit. So how do you measure success?

Think about the last campaign you ran that flopped. It probably didn’t fail for some mysterious reason. You can pinpoint what happened: the ICP shifted halfway through, the product got rescoped a week before launch, nobody could agree on which KPIs mattered, so strategy was scattered, etc.

AI adoption fails the same way. The problem is that nobody agreed on what success looked like before contracts were signed and licenses got handed out. AI adoption works best when you're clear about the outcome you want to achieve.

I've been working in AI since before ChatGPT went mainstream, and I've seen some companies get it really right, and some companies get it really wrong. 

So, let's jump into the pitfalls of poorly defined AI adoption, and how you can make it work without turning your company into a slop factory.

1) You're using more tokens, but you aren't seeing impact

Focus on problems to be solved instead of just pushing for usage without intention. If you just want teams to use ChatGPT instead of Google, that is a waste of everyone's time. Focus on problems to solve rather than generic token consumption.

Reducing content turnaround time from weeks to days is a great problem to solve. Eliminating out-of-date documentation before EOQ is another. Are you sick of spending time fighting procurement instead of working on campaigns? This is another fantastically solvable problem.

Notice what those have in common. Each one names a thing that changes, by how much, and by when. You can point at it in a quarter and say plainly whether it worked. Compare that to "increase weekly active users in our AI tool by 40%." That number tells you people opened an app. It tells you nothing about whether anything got better, and it's the metric most teams end up reporting on because it's the easiest one to move.

"We need AI" is not a problem; it's FOMO.

2) You gave everyone licenses, but no one is using them

If you're rolling out AI tooling across your organization, you should encourage people to experiment with it. People are sometimes seen as less creative or capable of using AI, and this bias prevents people from actually trying it.

Here's what that looks like day-to-day: Someone does easily automatable work manually because admitting they used AI feels like admitting they cut a corner. Nobody says this out loud, and you will never see it in your usage dashboard. You'll just see a license that never gets opened, and you'll assume the tool wasn't useful.

Leave space for experimentation and be open to the outputs. You should encourage AI adoption! Use Slack channels to share things like best practices, team fave prompts, etc. Hackathons are also another great way to bring people together to encourage experimentation and find new problems to solve, as outlined in the point above.

AI can be a fun and powerful tool for doing work, and giving space for experimentation empowers teams to do their best work with it.

3) Your company has become a slop factory

Scenarios two and three contradict each other. Tell everyone to experiment, then tell them to tear the results apart. They're really two extremes of the same adoption initiative: Either everyone overcommits, or everyone gets iced out.

This section is for every CTO who has considered replacing their marketing department with GPTs. 

Everyone hates AI slop. Do not accept AI content without reviewing it. Push back on em-dash-ridden, verbless, AI-generated strategies. Treat initial AI outputs very much as the first step.

If you're implementing a new AI-driven process, test all possible scenarios. If you're using AI for your own marketing projects, review and iterate on the outputs. In larger organizations, employees should feel empowered to call out leadership for presenting slop as strategy.

If you're building revenue projections, you should be able to explain them. If you're writing content, it should be coherent. If you are writing about your product or org, it should be reflective of your brand.

People often skip the most important step of working with AI: Iteration. A first output is a draft of a draft. It probably has the general shape of what you asked for, but none of the specifics that make it yours.

Getting to something usable usually means telling the model what's wrong with what it gave you, three or four times, in the same way you'd redline a junior employee's work. Teams that treat that loop as part of the job produce good work. Teams that treat the first pass as the deliverable produce a high-volume stream of AI slop.

Slop happens when people outsource all critical thinking to AI. Remember that AI is ultimately a very fancy math equation you can talk to. These algorithms can expedite cancer research, build safer airplanes, and find new ways to address climate change, but they aren't always good at talking about your brand or driving your corporate strategy.

You have to retain judgment over your own work outputs and maintain a high standard for company-wide initiatives.

4) You're struggling to see usage beyond basic research

AI needs context to be successful. If you have no connectors or centralized knowledge bases, you will forever limit how impactful AI can be. AI, like marketers, creates better work with maximum context.

This is why so many teams plateau at summarizing and light research. A model that can't see your roadmap, your positioning docs, your call transcripts, or last quarter's results can only give you advice that would apply to any company in your category. It doesn't know your ICP. It doesn't know which launch slipped, or why. So it hands back something reasonable and generic, and people conclude AI isn't that useful for real work.

The teams that get the most out of using AI are almost always the ones that share by default. Notes, decisions, docs, and post-mortems live somewhere the whole company can reach, which means the tooling can reach them too.

The teams that struggle tend to keep everything in DMs and private folders, then wonder why the output feels thin. AI is most powerful when context is shared, just like how teams do their best work with full context as to what's going on.

If your employees are using AI but your company doesn't have any pre-built connectors, your employees are probably copying and pasting context into their private accounts. It's worth looping in IT and legal if you haven't already, to keep your company's secrets safe but also give your employees room to explore.

So what does a well-defined outcome look like?

Four things have to be true at once:

1. There's a clear problem to solve, stated as a change you can measure
2. People are encouraged to experiment their way toward it without being judged for it
3. The outputs get critiqued and iterated on rather than shipped as-is
4. The tooling has enough context to do work that's specific to your company.

AI is a critical tool now. If you have a clear problem to solve with AI, a company that is encouraged to experiment and iterate on AI outputs, and your system has proper context, you'll be able to thoughtfully drive impact and see real results. You'll also be able to prove success, which is the part many teams can't do.

If you have an AI project that has failed, you've probably fallen into one of these four scenarios, so take a step back, re-evaluate, and hopefully you'll have one more problem to solve.