Introducing AI features into your product has never been faster. In this new era, any team can build AI features like text generation, image creation, and data analysis in a fraction of the time it used to take.
That increased speed is exposing new design and strategy traps. As Spencer and I shared at this year's Midwest House Summit, the hard part of product design was never building the thing. It's always been the strategy required to deliver a solution your users actually want, trust and adopt.
Building fast doesn't remove the strategy requirements. It just makes them easier to skip.
Across all industries, there is a widening gap between what teams ship and what their users adopt. More than 80% of AI projects fail to deliver their intended business value — roughly double the failure rate of non-AI projects — and 88% of AI proof-of-concepts never make it into production.
We've watched this play out across various client engagements. We’ve helped companies successfully avoid the widening strategy gap and ship AI features and AI-native products. From that experience, we’ve pulled together six traps that show up again and again, and what we'd do differently.
Trap 1: Treating research as a phase, not a practice
This trap shows up when AI features get scoped based on internal opinion or a board directive instead of real conversations with users.
When research gets pushed to "after the MVP is live” or “if there's time," that cadence gets expensive. Beyond the failure rates above, it means teams build with confidence but no evidence. Confidence doesn't transfer to adoption.
The fix starts with a hypothesis, not a feature.
Identify a near-term problem worth solving, then talk to your target users or buyers about their workflows, frustrations, and workarounds. Don't pitch. Ask. Treat that research as an ongoing practice, not a milestone you check off once.
We recently worked with Linnworks, where customer interviews revealed that users didn't actually want raw AI-generated insights. They wanted the system to recommend specific rules they could act on. That single research finding shaped an AI rule builder that delivered a 98% reduction in manual actions for users.
Trap 2: Picking the pattern before the pain
We see teams default to chat interfaces or a "magic wand" button because that's what AI is expected to look like right now. Familiarity with a pattern gets mistaken for user demand for it, and a pattern is chosen before the team knows what they are solving.
The cost is real. When a pattern is chosen because a team likes it rather than because users asked for it, people just don't feel pulled to use it. No metric can save a pattern nobody wanted in the first place.
A client came to us already committed to a chat-based AI overlay across their entire platform, described internally as "a natural language experience... like Claude or ChatGPT on top of the whole product."
Research told a different story. Across every persona tested, no one wanted the chat-based AI experience because users didn't want to formulate open-ended questions unprompted. Instead, users expressed enthusiasm for predictive insights and personalized conversation starters. That research led us to design a context-aware, action-prompting dashboard, with conversational AI available as a drill-down rather than the front door.
Start with the pain, not the pattern.
Find the specific moment AI could create value, then validate the interaction model with research before committing to it.
Trap 3: Not planning for when AI gets it wrong
Roadmaps tend to scope the happy path because that's where AI performs beautifully in the demo. What gets skipped is any plan for what happens when AI is wrong, uncertain, or simply can't complete the task. Those failure-mode questions get punted to V2.
The stakes are higher than they look. 70% of consumers say they'll take their business elsewhere after a single bad AI experience. Not to mention, the legal exposure is real: Air Canada was held liable after its chatbot fabricated a bereavement fare policy that a customer reasonably relied on.
Design failure modes as deliberately as you design success states.
Define what the feature does when it is wrong, uncertain, out of data, or unable to complete a request. Build in a way to relay confidence and route to a human when needed. AI should be able to say "I don't know yet" instead of guessing with confidence.
Trap 4: Using traditional UI for AI's new kind of output
Loading spinners, empty states that say "no results," and generic "something went wrong" error messages were all designed for interfaces that deliver the same answer every time. AI doesn't work that way, and using those old patterns for a new kind of output, without any visibility into sources, reasoning, or confidence, quietly erodes trust.
53% of consumers say they distrust AI search results specifically because they can't see the reasoning behind them. That distrust has a price tag attached: 76% of consumers say they'd switch brands over AI transparency, and more than half would pay a premium for it.
Design for transparency instead.
Give users context for a "miss" rather than a system error, replace generic loading and error states with specific ones that show the steps AI is taking toward an outcome, and let people dig into the reasoning and sources behind an answer when they want to.
Trap 5: Treating AI features as one-and-done
A feature ships, the team moves on, and there's no plan for iteration, removal, or ongoing quality. There’s no success metric beyond "live," no feedback loop, no owner watching performance as models change and user behavior shifts.
For every 100 features a team ships, only 6.4 ever drive meaningful usage, and even top-performing product teams can't get feature adoption above roughly 16%, per Pendo's 2024 Software Benchmarks Report.
Define what "working" looks like before you ship, and build usage and quality signals into the release itself rather than bolting them on later.
Name an owner responsible for the feature after launch, and set a date to evaluate it, with an actual plan to pull it or fix it if it's not working.
People often start with every intention of designing the full vision for an AI capability and validating it with research. But as things scale, priorities shift. The pressure builds to ship a stripped-down MVP of that validated concept, and customer feedback sessions quietly stop. "We have an AI feature" becomes the finish line instead of the starting line. Those decisions, made in the name of speed rather than strategy, often come back around as a roadmap rewrite and an uncomfortable conversation with the board.
Trap 6: Treating a behavior-changing feature like a minor update
A new AI feature ships quietly, folded into an existing workflow, on the assumption that users will notice simply because it's live and new. No single moment gets created to interrupt years of established user habit.
That assumption doesn't hold up. 99% of Americans use a product with AI every week, but only 36% realize it. If users don't know a capability exists, they can't decide to trust it, rely on it, or pay for it. The downstream cost shows up in support tickets and roadmap noise.
Amazon's Rufus assistant learned this the hard way. In early usability testing, almost no one noticed the feature because it was buried in the search bar with no visual cue it was there.
Treat a meaningful AI feature launch like a new product launch, not a minor upgrade.
Put the new way of doing things front and center with spotlights, banners, and walkthroughs, and create a deliberate moment that interrupts the old habit, instead of hoping users stumble onto the new one.
Ask before you ship: a quick checklist
Before your team ships the next AI feature, run it through these six questions:
- Have we talked to our users, or are we running on internal conviction?
- Did we choose this pattern because the problem required it, or because everyone else has one?
- Do we know what happens when AI can't answer or gets it wrong?
- Are we showing our work and earning trust through transparency?
- Will we know if this stops working in 60 days?
- Have we planned how we'll launch this to users and interrupt their habits, not just where we placed it?
If any of those don't have a clear answer, that's the gap to close before launch, not after.
Speed and strategy aren't in conflict
None of this is an argument for moving slower. Teams that can design and ship fast have a real advantage. But fast shipping doesn't excuse weak strategy. It demands sharper strategy, because the cost of building the wrong thing now compounds at the speed you're building it.
The teams avoiding these six traps aren't the ones with the most AI expertise. They're the ones who kept doing the fundamental product work: talking to users, matching the pattern to the problem, planning for failure, being transparent, measuring what matters, and launching deliberately. Let AI make that work faster instead of replacing it.
Have an AI feature on your roadmap you want a second set of eyes on?
Our product experience team helps translate product vision into products people actually use: sharpening UX, clarifying roadmaps, and designing systems built to scale. We'd love to learn what you're building and the problems you're solving.
