u/Tron_tx6

Current challenge:
-We have a product recommendation/search system where precision matters more than recall.

Client expectation is:
- ~95% queries should resolve through deterministic/filter-based retrieval
- Only ~5% should go through RAG/semantic reasoning

Reason:
- Product catalog is limited
- Pure RAG/vector search gives decent recall but poor precision
- Earlier implementation used LLMs (Claude) to generate filters directly from prompts with confidence scoring > 90, but hallucinated filters caused poor SQL retrieval quality.

What I implemented:

1. Instead of relying on prompt-only filter extraction, I converted metadata into embeddings.

2. Stored metadata in PGVector using Cohere embeddings.

3. Each metadata entry is aligned with:

4. category, subcategory, normalized attributes/tags

5. Retrieval flow:

6. Vector similarity retrieval

7. Hybrid reranking for better precision + recall

8. Retrieved metadata candidates are then used to construct filters for SQL/product retrieval.

9. RAG is used only as fallback when filter confidence is low or query intent is ambiguous.

Observed improvements:
Better filter consistency
Reduced hallucinated attributes
Better precision compared to prompt-only extraction
More controllable retrieval pipeline

Questions:

1. Is this generally the right architecture direction for enterprise product recommendations/search?
2. Any better approaches for:
3. metadata normalization
4. filter confidence scoring
5. query-to-filter mapping
6. reducing semantic drift?
7. Would knowledge graphs/taxonomy mapping help more than embeddings here?
8. How do teams usually decide when to invoke RAG vs deterministic retrieval?

Would appreciate suggestions from people working on enterprise search, RAG systems, recommendation engines, or e-commerce or medical retrieval pipelines.