r/LangChain

I'm building an open-source multi-agent framework and spent last few weeks testing it against a real Enterprise solution design task — not a toy benchmark, an actual enterprise ticket requiring cross-referencing Jira comments, Java source code, Process Flow Config XMLs, and Confluence design docs to produce a correct technical document.

The setup:

• 4 specialist worker agents (Jira researcher, code analyst, config analyst, docs researcher) coordinated by an architect agent, with a synthesizer combining everything
• Each worker had focused MCP tools for their domain
• We tried 4 different multi-agent configurations over multiple days

What happened with multi-agent (4 attempts):

Attempt	Core Error
1	Invented an Attributes that doesn't exist
2	Misclassified the ticket as a different initiative entirely
3	Got the actual Ticket intention wrong.
4	Imported scope from a different ticket (which had similar name)

Each attempt used 30,000-70,000 tokens across different tools and agents. Each made a different fundamental error.

What happened with single agent:

• One agent with ALL tools (Jira + code + CDT + Confluence + output) in one context window
• Kimi K2.6 (cheap model, $0.73/1M input)
• Only 3,454 tokens total
• First doc to correctly identify the actual problem, name the right code sites, quote the right Jira comments, and recommend fixes.

It wasn't perfect but only needed minor fixes to make the solution workable.

Based on all the agent logs and traces which were captured at each agent level, here's my understanding of why multi-agent failed:

The task required connecting dots across multiple sources. A Jira comment mentions a class name -> read that class -> find it references a Config XMLS -> fetch that config -> discover a condition that gates the behavior the ticket wants to change. This chain of reasoning needs to happen in ONE context window.

Bue what wsa happening with multi-agent:

• Worker A finds the Jira comment but doesn't know about the code
• Worker B reads the code but doesn't know which Jira comment matters
• Worker C fetches Process Flow Config XMLss but doesn't know which code path to trace
• The architect gets summaries from each and tries to connect them — but summaries lose the specific details that matter

Information is really getting destroyed at every handoff. The architect is reasoning over shadows of the actual data and a lot of information was not even fetched because full information was never in the single context to work on.

This experience gave me the insight on when multi-agent just doesn't work:

• Anything requiring cross-source reasoning (solution design, root cause analysis, debugging)
• When the total data fits in one context window (most enterprise tasks)
• When coordination cost (token overhead, summarization loss) exceeds the parallelism benefit

When multi-agent DOES make sense:

• True parallelism on independent tasks (monitoring multiple services, processing document batches)
• Scale beyond one context window (millions of log lines need filtering before reasoning)
• Each agent has a genuinely independent domain (home automation: lighting agent, HVAC agent, security agent)

The final takeaway I could get from the full experiment would be:

The value isn't in agent count — it's in good tools and skills that give the model the right context. MCP servers, structured search, code reading tools — these are what made the single agent succeed. Adding more agents just added more ways to lose information.

Multi-agent is a tool, not a goal. Use it when parallelism genuinely helps. Default to single agent with good tools for anything requiring deep reasoning.

What have been your experience with multi-agents and where have they really worked and where they have failed.

Hey,

Software engineer here, relatively new to agentic workflows. Building a production AI concierge — user says "I'm going to Budapest tomorrow, plan my day" → agent searches our offer database, builds a plan, user books everything in one flow.

**Stack:** OpenAI GPT-5.5 + tool use, NestJS, SSE streaming, React Native. Tools: `search_offers`, `get_offer_details`, `calculate_price`, `prepare_booking_bundle`.

**The problem:** Consistency. Two main issues:

- Model hallucinates offers from training data instead of calling `search_offers`. It knows a lot about European tourism and just... uses that knowledge instead of querying our DB.
- Tool chains break mid-flow. After `search_offers` returns results, model sometimes responds in plain text instead of continuing to `get_offer_details` → `calculate_price`.

Tried explicit prompt rules, `__next` instructions embedded in tool results, reducing tool count. Helps but doesn't fully solve it.

**Questions:**
- What frameworks/tools are you using for production agentic flows?
- How do you enforce tool call sequences reliably?
- Any techniques for preventing hallucination in tool-use agents specifically?

Appreciate any advice from people who've shipped this stuff in production.

Is Langgraph/langchain still worth learning with the current progress of LLMs?
Do you build any projects with that still?

Been using Chainlit with LangChain for a while now on production legal AI apps — streaming agent responses, multi-step tool calls, the whole thing. Curious what others in this community have built with this combo and where the pain points are.

For me the rough edges have been:

• Auth in embedded/Copilot mode when the parent app already handles auth — the password_auth_callback flow gets messy fast
• Chat history persistence since LiteralAI shut down — self-hosting their open-sourced data layer works but it's extra ops nobody budgeted for
• WebSocket disconnects under moderate load — Chainlit drops connections and there's no built-in session recovery, you have to roll your own
• Debugging LangChain agent steps inside Chainlit's step visualizer when chains get deep — it can get noisy
• Mounting Chainlit inside an existing FastAPI app — the ASGI mount patterns are barely documented

What have you shipped? RAG pipelines, agents, internal tooling? And what forced you to reach for a workaround or abandon Chainlit entirely for something else?

PS: Claude helped me to write this as it knows my pain points while building with chainlit.

>ORKA — governance layer for AI agents

If you're running AI agents in production, you've probably run into at least one of these:

— An agent did something unexpected and you had no way to trace why
— You needed to prove to leadership or compliance what your agents are actually deciding
— A sensitive action happened that should have required human approval first

ORKA solves this. Full audit trail, policy engine, and human-in-the-loop approvals — works with OpenAI, Claude, LangChain, Firecrawl. Instruments on top of your existing stack, no rebuild.

Used by teams in production today. Free plan available.

orka.ia.br

Hey r/LangChain,

Just open-sourced a multi-agent research system built with LangGraph.

**What it does:**

You give it a complex clinical/research question. A network of AI agents

(Orchestrator → Researcher → Critic → Writer) researches the topic, critiques

data quality, loops back if insufficient, and only generates the final report

after human approval (HITL).

**Key architectural decisions:**

- LangGraph over CrewAI — explicit control over edges, state transitions, and interrupt points

- `operator.add` on `research_data` — append-only accumulation across critic revision cycles

- `interrupt_before=["writer"]` — human approves before report generation (true HITL)

- DeepSeek via OpenAI-compatible API — cost-efficient drop-in for GPT-4

**Stack:** LangGraph · LangChain · DeepSeek · Tavily · Pydantic · Python

The repo includes a real example output (clinical_report.md) generated with:

*"Latest evidence on semaglutide for obesity treatment in CKD patients"*

GitHub: https://github.com/Armandogith/langgraph-research-orchestrator

Happy to discuss the architecture — particularly around the critic loop design

and state checkpointing. What patterns are you all using for quality control

in multi-agent pipelines?

Hey everyone, my projects were tipping along with a small user base, but there has been a bit of an uptick in signups, which is great! But how might I manage costs?

I am slightly worried that costs might explode and erase my runway, any help appreciated!

Been thinking about something that keeps coming up in conversations with compliance and security teams at regulated firms, and I'm curious whether others are seeing the same thing.

I Had an interesting conversation with a compliance lead at a financial services firm last week and he was pretty confident their cloud AI vendor was handling their documents safely. They had DPA signed, opt-out enabled and the vendor was SOC 2 certified.

I asked if they knew what was being logged during inference and who at the vendor could access those logs and They didn't know.

It got me thinking about how narrow the training opt-out commitment actually is and how little people actually know about it. It says your data won't train future models but nothing about inference logging, shared GPU tenancy, log retention schedules or what happens if the vendor gets a government subpoena. Because those governed by separate policies.

Curious how others in regulated environments are actually handling this. Are your teams making a deliberate architectural decision here? Are you aware of the risks?

Hey r/LangChain,

I'm working on a lightweight tool to help developers monitor their AI agents in production.

The problem I'm trying to solve: when your agent fails or behaves weirdly, you currently have no easy way to see exactly what happened — which steps it took, where it went wrong, what it cost you in tokens.

My solution is basically a "black box recorder" for AI agents. You add one decorator to your code:

python

u/trace
def run_my_agent(user_input):
    # your existing code, untouched

And you get a dashboard showing:

• Every step your agent took
• Where it failed and why
• Cost per run
• Alerts when something breaks

Works with any model (OpenAI, Claude, Gemini, Llama) and any framework (LangChain, LangGraph, raw API calls).

Before I build this — would you actually use it? What's your biggest pain point when debugging agents in production?

Genuine feedback only, happy to be told this already exists or nobody wants it!

I'm trying to understand what teams building with AI SDKs struggle with the most once their app is in production.

So far I've heard a few things come up. Some people don't know which model to pick for each task and don't have a week to benchmark everything. Others mentioned costs creeping up but struggling to switch to cheaper models without breaking quality on edge cases.

I'd love to hear what's on your list. If you have 30 seconds, please drop your top 1 or 2 pains in the comments with a bit of context.

Was researching AI scraping tools for a project and noticed the existing awesome lists either cover traditional scraping (Scrapy, BeautifulSoup) or web agents broadly. Couldn't find one focused specifically on LLM-powered scraping, so I put one together.

Covers frameworks (Crawl4AI, Scrapling, ScrapeGraphAI, llm-scraper), hosted APIs (Firecrawl, Jina Reader, Diffbot), browser infrastructure for AI agents, MCP servers, and search APIs built for LLMs.

Open to more what am I missing?

Disclosure: I'm the developer of this project.

AI agents are taking actions without any oversight – calling APIs, writing data, triggering workflows. Most teams have zero visibility into what's actually happening.

I built ORKA to fix that.

ORKA sits between your agents and the outside world:

→ Every action goes through a policy check before executing

→ High-risk actions pause and wait for human approval

→ Everything is logged in a cryptographically chained audit trail

→ Real-time dashboard with risk scores per agent

It supports MCP, A2A, REST, and custom agent protocols.

Currently in private beta – free to use, no credit card required.

Search "ORKA governance AI agents" on GitHub or "orka.ia.br" to find it.

Would love feedback from anyone building with AI agents. What governance/visibility features are you missing today?