# LLM integration patterns

> There are five main ways to put a large language model into a business system. Prompting gives the model instructions and text and takes back text. Retrieval fetches your own documents or records and puts them in front of the model so its answers are grounded. Structured extraction turns messy input into fields your systems can store. Tool use lets the model call functions such as a lookup or a draft. Agents chain tool calls toward a goal, with approvals where actions matter. Each is evaluated by comparing outputs to known-good answers on a fixed test set.

Source: https://risendigital.io/guides/llm-integration-patterns  
Publisher: Risen Digital  
Updated: 2026-09-21

## At a glance

- **Start with the simplest pattern:** Most business problems are solved by retrieval or extraction long before an agent is needed.
- **Grounding beats tuning:** Giving the model the right documents at the right moment fixes wrong answers; retraining it rarely does.
- **Structure is the bridge:** Extraction to a fixed schema is how model output becomes something your database, CRM or workflow can act on.
- **Evaluate with a test set:** A fixed set of real cases with known-good answers, re-run on every change to the prompt, the data or the model.

Almost every useful LLM feature is one of a handful of patterns, or a combination of them. Knowing which one you are building tells you what it needs, how it will fail and how to check its work. This guide lays out the patterns plainly, helps you pick one, and explains why fine-tuning is rarely where to start.

![Two colleagues mapping options on a board of sticky notes](https://risendigital.io/images/experience/sticky-board-1600.jpg)

## What are the main ways to put a language model into a business system?

Prompting, retrieval, structured extraction, tool use and agents. They build on each other: each adds a capability and a new way to go wrong.

Prompting is the base. You give the model instructions and some text, and it returns text: a summary, a rewrite, a classification, a draft. It knows nothing about your business except what is in the prompt. Retrieval fixes that by searching your documents or records for the pieces relevant to the question and placing them in the prompt, so the model reasons over your facts rather than its general knowledge and can cite what it used.

Structured extraction points the model at messy input, such as an email, a PDF or a call transcript, and asks for output in a fixed shape: named fields with types. That is what lets the result land in a database or trigger a workflow. Tool use gives the model a set of functions it may call, such as look up an order or create a ticket, and lets it decide when to call them. An agent runs tool use in a loop toward a goal, deciding step by step, with rules about what it may do alone and what needs a person.

| Pattern | Good for | Needs | Typical failure |
| --- | --- | --- | --- |
| Prompting | Summaries, rewrites, classification, first drafts | Clear instructions and examples | Confident answers with no basis in your data |
| Retrieval | Questions over policies, manuals, contracts, records | Documents you can search, with a way to chunk and rank them | The right document was never retrieved, so the answer is plausible and wrong |
| Structured extraction | Turning emails, forms and documents into records | A schema and a validation step | A field filled with a guess instead of left empty |
| Tool use | Assistants that look things up and prepare actions | Well-described functions with tight inputs | The wrong tool called, or the right one with the wrong arguments |
| Agent | Multi-step jobs with a clear definition of done | All of the above, plus limits and an approval path | Acting when it should have asked, or looping without progress |

## Which pattern fits the problem you have?

Answer a few questions about what the model should do with your information and what happens if it gets a step wrong. The result names the pattern and what it needs.

_Interactive on the web page: Find the right pattern for your case. The pathfinder asks what you want the model to do. Answering questions from your documents leads to retrieval, especially if answers must be current or cite a source. Turning messy input into records leads to structured extraction. Helping people look things up and prepare actions inside your systems leads to a tool-using assistant with a person confirming each action. Completing a multi-step job alone leads to an agent with approvals, but only when a wrong step can be caught or reversed; otherwise it routes back to an assistant. Wanting the model to learn your style or domain leads to the finding that fine-tuning is rarely the answer: wrong facts are fixed by retrieval, inconsistent output by extraction, and only tone and vocabulary sometimes justify tuning._

## Where do LLM integrations go wrong?

They fail quietly: a plausible answer with no basis, a field guessed instead of left blank, the wrong tool called, or an action taken that should have been a question. Each pattern has a characteristic failure and a matching guard.

- **Ungrounded answers** A prompt-only feature answers from general knowledge, fluently and wrongly. Guard: retrieval, an instruction to answer only from the provided material, and a visible citation.
- **Retrieval misses** The answer is wrong because the right document was never fetched. Guard: measure retrieval separately from generation, and show the sources so people can spot a miss.
- **Guessed fields** Extraction fills a field it could not find. Guard: allow empty values, require the source span for each field, and validate against the schema before storing.
- **Wrong tool, wrong arguments** The model calls a function it should not, or with a value it invented. Guard: tight input schemas, validation before execution, and read-only tools by default.
- **Overreach** An agent acts where it should have asked. Guard: written limits, approval for anything irreversible, and a hard stop after a set number of steps without progress.
- **Silent drift** The model provider updates the model, your documents change, or a prompt edit has a side effect. Guard: re-run the evaluation set on every change and watch the accepted-unchanged rate in production.

## How do you evaluate the outputs?

Build a fixed set of real cases with known-good answers, score the model against it on every change, and keep watching the live rate at which people accept its output unchanged.

1. **Collect real cases** Pull a spread of actual inputs from the process, including the awkward ones. Have the people who do the work today write the correct output for each.
2. **Decide what correct means per pattern** For extraction it is field-by-field match. For retrieval it is whether the right source was fetched and whether the answer is supported by it. For tool use it is the right call with the right arguments. For agents it is the outcome plus whether it asked when it should have.
3. **Score automatically where you can** Exact matches and schema checks are cheap. For free-text answers, a second model can grade against the reference answer, with a sample checked by a person.
4. **Re-run on every change** A new prompt, a new document set, a new model version. If the score drops, you find out before your customers do.
5. **Measure in production** Track how often a draft is accepted unchanged, how often a person corrects it and where. Corrections are new test cases.

> **Combine patterns deliberately** Most real features chain two or three: retrieval to ground an answer, extraction to structure it, a tool call to act on it. Evaluate each link on its own before you evaluate the chain, or you will not know which one broke.

## Frequently asked questions

**Q: What is retrieval-augmented generation in plain terms?**  
A: It is search plus a language model. When a question comes in, the system searches your own documents for the relevant passages, puts them in the prompt, and asks the model to answer from those passages only. The model contributes the reading and the writing; your documents contribute the facts.

**Q: Should we fine-tune a model on our own data?**  
A: Rarely, and never first. Fine-tuning changes how a model writes, not what it knows, and anything it learns goes stale. Retrieval keeps facts current, prompting with examples fixes most tone problems, and extraction fixes format. Consider tuning only when those are proven and style is still the gap.

**Q: How do you stop a language model from making things up?**  
A: Ground it with retrieval so it answers from your material, instruct it to say when the material does not cover the question, make it cite sources so misses are visible, validate any structured output before it is stored, and score it against a test set of known answers on every change.

**Q: What is the difference between tool use and an agent?**  
A: Tool use is the model calling a function when it decides one is needed, usually with a person acting on the result. An agent runs tool use in a loop toward a goal, deciding each next step itself. The difference is autonomy, which is why an agent needs written limits and an approval path that a tool-using assistant does not.

**Q: Which pattern should a business start with?**  
A: Retrieval or structured extraction, almost always. Both produce visible, checkable output, both are grounded in your own data, and both build the evaluation habits you need before giving a model any ability to act. Tool use comes next, and agents once the assistant has a track record.

## Related pages

- [How to integrate AI into existing software](https://risendigital.io/guides/how-to-integrate-ai-into-existing-software)
- [What is an AI agent?](https://risendigital.io/guides/what-is-an-ai-agent)
- [AI data privacy and security questions](https://risendigital.io/guides/ai-data-privacy-and-security-questions)
- [Reducing manual data entry errors](https://risendigital.io/guides/reducing-manual-data-entry-errors)