Introduction
The scariest thing in a concierge AI is when the AI tells a plausible-sounding lie. Being told confidently "this product is fully waterproof" when it actually isn't will cost you trust.
In this article I explain the RAG (retrieval-augmented generation) mechanism we used to give the concierge AI's recommendations real evidence, on an EC site selling motorcycle gear. It combines two kinds of information source — the product master and the knowledge base — and grounds every recommendation only in "what the documents say."
Why evidence is necessary
The danger of AI-only recommendations
If you hand an LLM just a product name and ask it to "recommend this," it generates a convincing-sounding explanation. But there is no guarantee that explanation matches the actual product spec.
The risk of hallucination
LLMs have a tendency to answer things they don't know as if they do. This is called hallucination. In a service setting, an explanation that differs from fact leads straight to returns and complaints.
That is exactly why every recommendation needed backing — "which document is this based on?"
Using two kinds of source for different jobs
The sources that back up the evidence are two, with different natures: a structured product master and a knowledge base stored as prose.
"A black jacket under 30,000 yen that's in stock" is the product master's forte. "Which one is least tiring on a long ride" is the knowledge base's turn. Only with both can you ground both the filtering and the explanation.
Building the knowledge base
Turning PDFs and web info into text
The raw material for the knowledge base is product manual PDFs and manufacturers' web product information. First we extract text from these and shape it into a searchable form.
Pull body text from manual PDFs and web info
Split long prose into moderate sizes to serve as search units
Convert each chunk into a vector (array of numbers)
Store in Neon Postgres (pgvector), linked to the product info
Running product materials through this pipeline turns scattered documents into "knowledge that can be searched by meaning."
Storing vectors in pgvector
The converted vectors are stored in pgvector, an extension of Neon Postgres. The advantage is completing everything on the existing Postgres without standing up a separate vector-only service.
Because we can keep them in the same database as the product master, the link "which product does this description chunk belong to" is expressed naturally. Operations become simpler, and it fit well with the Vercel environment too.
Tuning chunk granularity
If chunks are too small the context breaks; too large and search accuracy drops. Manuals are often organized by headings and sections, so we kept those boundaries in mind while splitting to a rough target of a few hundred characters.
With the right granularity, even a pinpoint question like "about the ventilation mechanism" pulls exactly the relevant passage. This is a spot to tune while watching the nature of the product materials.
Building evidence into recommendations
Search first, then generate
When crafting a recommendation, the AI doesn't just start writing. It first searches the knowledge base with the customer's requirements and builds the explanation using only the chunks it found. This is the core idea of RAG.
Plausible, but may diverge from the actual spec
Explains by quoting only what's written in the manual
Whether a recommendation can add the phrase "according to the manual" completely changes its trustworthiness.
Nailing down facts with the product master
Immovable facts like price, stock, and variants are pulled directly from the product master, not the knowledge base. It's safer to lock these down with accurate data synced from Shopify rather than leaving them to RAG.
We also link the size-recommendation logic — deriving the right size from body dimensions — to the product master side. Being able to declare "this is your size" in an EC context where you can't try things on is only possible thanks to an accurate master.
Making it say so honestly when nothing is found
Another benefit of RAG is that if search finds no hit, it can say "I don't know." By constraining it in the prompt to "not answer information absent from the search results," we prevent forced confabulation.
Designing it to hold its tongue
Not letting the AI fake knowledge is as important to protecting service trust as adding a feature. An AI that can say "I don't know" when it doesn't know ends up more relied upon.
Conclusion
To build grounded recommendations, we worked on three things.
- Two sources — nail down facts with the product master, search semantically with the knowledge base
- The RAG pipeline — turn documents into knowledge via text extraction, chunking, embedding, and pgvector storage
- No confabulation — ground in search results, and honestly say "I don't know" when there's nothing
How do we show these grounded recommendations to the customer so they're convinced? Visualization with comparison tables and charts is covered in "Visualizing Recommendations with Tables and Charts," and the dialog design that gathers requirements in "Designing the Interview Dialog and State Management."