Nearly every leadership team has had some version of this conversation: "We should be using AI." Then the meeting ends and nobody's sure what that actually means for the software the company already runs on.
The gap between "we should use AI" and "AI is running in production and saving us money" is where most initiatives stall. Not because the technology isn't ready, but because the projects are chosen badly, the data isn't ready, or the effort is framed as a moonshot instead of an integration.
This guide is for owners, executives, and product leaders who need to make good decisions about AI without becoming machine learning engineers. It covers what AI is good and bad at, the four ways it gets added to existing software, what you need to have in place, the tradeoffs that matter, and a realistic 90-day plan.
What AI actually does well (and where it fails)
Before choosing a project, it helps to have an honest picture of where current AI delivers.
AI is excellent at:
- Reading and summarizing unstructured text. Contracts, support tickets, emails, reports, transcripts. Turning a 40-page document into a half-page summary, or extracting the ten fields you care about, is now reliable and cheap.
- Classification and routing. Deciding which category a ticket belongs in, which team should handle a request, or whether a transaction looks unusual.
- Search that understands meaning. Finding the right document when the user's words don't match the document's words.
- Drafting. First drafts of responses, reports, descriptions, and code that a human then reviews.
- Pattern-based prediction. Forecasting demand, flagging churn risk, predicting equipment failure, when you have enough historical data.
- Automating multi-step tasks that follow judgment-based rules. Processing an invoice that arrives in one of forty formats, for example.
AI is unreliable at:
- Being right every time. Language models produce confident, plausible errors. Any workflow where a single wrong answer is unacceptable needs a human check or a different design.
- Math and precise logic inside a language model. Use conventional code for calculations and let AI handle the language around them.
- Anything without data. A prediction model needs history. If you've never tracked the outcome you want to predict, there's nothing to learn from.
- Replacing judgment on high-stakes decisions. AI should inform hiring, lending, medical, and legal decisions, not make them, both for accuracy and regulatory reasons.
The best first AI projects live in the first list, have a clear metric, and have a human somewhere in the loop.
The four ways AI gets added to existing software
Almost every AI integration we build falls into one of four patterns. Knowing which one you need shapes cost, timeline, and risk.
Pattern 1: Language model features
Adding capabilities powered by a large language model (from providers like Anthropic, OpenAI, or Google, or open models you host) to your existing product. Examples: summarizing a customer's history when a rep opens their record, generating a first draft of a proposal from a template and CRM data, letting users describe what they want in plain language instead of navigating menus.
Effort: Lowest of the four. Often 4 to 10 weeks. The heavy lifting is in prompt design, connecting to your data, and building guardrails.
Best for: Anything involving reading, writing, or understanding text.
Pattern 2: Retrieval-augmented search and assistants
A language model connected to your own knowledge: documentation, policies, product data, past tickets, contracts. Users ask questions and get answers grounded in your actual content, with citations. This is how "chat with our knowledge base" and internal AI assistants are built, and it's what separates a useful assistant from one that makes things up.
Effort: 6 to 14 weeks. The key work is preparing your content, building the retrieval pipeline, and testing answer quality rigorously.
Best for: Support deflection, internal knowledge access, onboarding, sales enablement.
Pattern 3: Predictive models
Traditional machine learning trained on your historical data to predict an outcome: which leads will close, which customers will churn, how much inventory you'll need, which claims deserve review. This is older technology than language models and, for structured numeric problems, still usually the right tool.
Effort: 8 to 16 weeks, highly dependent on data quality. Much of the timeline is data preparation.
Best for: Forecasting, risk scoring, prioritization, anomaly detection.
Pattern 4: Automation and agents
AI that takes actions, not just answers: processing incoming documents and entering data into your systems, triaging and responding to routine requests, monitoring for conditions and kicking off workflows. Agents chain multiple steps together and can call your existing systems through their APIs.
Effort: 10 to 20 weeks. These require the most careful design because errors compound across steps, and they need robust logging, approval gates, and rollback capability.
Best for: High-volume, rules-plus-judgment processes currently done by hand: invoice processing, claims intake, order exceptions, compliance checks.
Many real projects combine patterns. A support assistant might use retrieval (pattern 2) to answer questions and an agent (pattern 4) to actually issue a refund when appropriate.
The data readiness checklist
The single most common reason AI projects fail is data. Before committing to a project, work through these questions:
Do we have the data at all? For predictive models, you need historical records of both the inputs and the outcome you want to predict, going back far enough to have meaningful volume. For retrieval-based assistants, you need the documents in a form software can read (not scanned images without OCR, not knowledge locked in people's heads).
Is it accessible? Data in a modern database or SaaS platform with an API is easy. Data in a legacy system with no export, or spread across personal spreadsheets, needs to be consolidated first.
Is it clean enough? "Clean" doesn't mean perfect. It means consistent enough that patterns are real. If the same customer appears under five different names, or half the records are missing a key field, that needs addressing.
Is it current? An assistant built on last year's policies gives last year's answers. You need a process for keeping the knowledge base updated.
Are we allowed to use it? Customer data, employee data, and regulated data may have contractual or legal restrictions on how it can be processed and where it can be sent. Check before building, not after.
If the answers are mostly "no," the first project isn't AI. It's data infrastructure. That's a less exciting sentence but a much better investment.
The tradeoffs that matter: cost, latency, accuracy, and privacy
Every AI integration involves balancing four things. Understanding them lets you have an informed conversation with whoever builds it.
Cost. AI features have two costs: building them and running them. Running costs come from model usage (charged per unit of text processed), which scales with how often the feature is used. A summarization feature used 100 times a day costs almost nothing. One used 100,000 times a day is a real line item. Good design (caching, using smaller models where they're sufficient, limiting what's sent to the model) can cut running costs by 5 to 10x.
Latency. Language models take time to respond, often one to several seconds. That's fine for generating a report. It's disruptive in the middle of a fast workflow. The design should match the user's expectations, and some features work better running in the background than on demand.
Accuracy. No AI feature is 100% accurate. The question is what accuracy is acceptable and what happens when it's wrong. A summary that's 95% accurate with a "view original" link is fine. An automated payment that's 95% accurate is not. Build in human review where errors are costly, and measure accuracy continuously, not just at launch.
Privacy and data residency. Using a cloud AI provider means sending data to a third party. Major providers offer enterprise agreements with no-training commitments and data-handling guarantees, which satisfy most needs. For highly sensitive data or strict residency requirements, self-hosted open models are an option, at higher cost and complexity. Decide this early; it affects architecture.
A 90-day pilot plan
The right way to start is small, measurable, and real. Here's a structure that works.
Weeks 1 to 3: Choose and scope
Pick one project using these criteria:
- It maps to something AI does well
- The data exists and is accessible
- There's a clear metric (hours saved, tickets deflected, accuracy improved, revenue influenced)
- Failure is survivable: a human can catch mistakes and the process doesn't stop if the feature is turned off
- Someone in the business genuinely wants it and will use it
Write a one-page brief: the problem, the users, the metric, and what "success" means numerically. Resist the urge to do three projects at once.
Weeks 3 to 5: Data and design
Consolidate and assess the data. Design the user experience, including how errors surface and how users give feedback. Decide the model and hosting approach based on privacy requirements. Build the evaluation set: 50 to 200 real examples with known correct answers, which you'll use to measure quality throughout.
Weeks 5 to 10: Build and test
Build the integration into your existing software. Test against the evaluation set repeatedly. Put it in front of a handful of real users early, with a clear "this is a pilot" framing, and watch how they actually use it. Tune based on what you see.
Weeks 10 to 13: Measure and decide
Run the pilot with a defined group for two to three weeks. Measure the metric from your brief. Gather qualitative feedback. Then make a decision: expand, adjust, or stop. All three are legitimate outcomes. A pilot that stops after 90 days with a clear reason is a success compared to a project that drifts for a year.
Everything you learn in the pilot (what data was harder than expected, what users actually wanted, what the running costs looked like) makes the second project faster and cheaper.
Frequently asked questions
Do we need to hire data scientists?
Should we build our own model?
How do we keep AI from making things up?
What does it cost to add an AI feature?
Will this replace our staff?
Where to start
If you take one thing from this guide: the best first AI project is a boring one. A narrow feature, real data, a clear metric, and a human still in the loop. Get that working, learn from it, and the ambitious projects become achievable.
We help companies scope, build, and integrate AI into the software they already run, from a first pilot through production systems. If you're trying to figure out where AI fits in your business, reach out for a free AI readiness conversation and we'll give you a candid read on what's worth building first.