u/Conscious_Chapter_93

We just shipped a Rust-native learning overlay for Armorer Guard.

The idea: a scanner should be able to adapt from local feedback immediately, without silently mutating model weights or uploading prompts to a cloud service.

What changed:

• feedback-record / feedback-export / feedback-stats CLI modes
• stable scan IDs so teams can review findings without storing raw prompts
• local allow / block / review exemplars stored outside the repo
• no suppression for credentials, dangerous tool calls, or credential-disclosure policy reasons
• reviewed export path for later offline retraining

The claim we are trying to make precise is: live local learning, no silent cloud upload, no poisoning-by-default.

I am curious how people here would wire this into agent runtimes. Before the tool call? Around MCP/tool results? As a CI gate for agent evals?

A pattern keeps showing up in agent threads here: the first agent is not the hard part. The hard part starts when you have several agents running repeatedly, with tools, state, approvals, retries, and partial failures.

The questions become less glamorous:

• Which agent ran this task?
• Which tools or MCP servers were available?
• What did it change?
• Did it stop, fail, or wait for approval?
• Which verifier/test phase passed it?
• Can I replay or compare this run against the last good one?
• What do I do when context runs out mid-task?

I think a lot of agent reliability work is really agent operations work. Frameworks help build the agent, but teams still need an operating surface around runs, sessions, tools, approvals, and recovery.

Curious how others here are handling this today. Are you using LangSmith-style traces, custom dashboards, Temporal/workflows, git worktrees, spreadsheets, or just logs and vibes?

Armorer Guard is a source-available GitHub project for local AI-agent runtime safety. It scans prompts, model outputs, retrieved text, and tool-call arguments for prompt injection, credential disclosure, exfiltration attempts, and dangerous tool calls.

It is Rust-native, runs locally with no scanner network calls, returns structured JSON, includes credential redaction, and has a Python wrapper for Python agent stacks.

Current selected classifier metrics in the README: about 0.0247 ms average classifier latency, 0.9833 macro F1, and 1.0 micro recall. Model artifacts are on Hugging Face: https://huggingface.co/armorer-labs/armorer-guard-semantic-classifier

Would love feedback on the CLI/API contract and packaging.

I’ve been building a lot with AI agents lately, especially tool-using agents, MCP servers, browser agents, and local/self-hosted workflows.

One thing kept bothering me: agents are becoming more like applications, but we still manage many of them like random scripts.

Setup is fragmented. Config lives in different places. Logs are inconsistent. Tool access is often too broad. Secrets are easy to leak. And once an agent can use browsers, files, shells, GitHub, Slack, or APIs, the security model starts to matter a lot.

So I started building Armorer: an open-source control plane for AI agents.

The goal is to make it easier to:

• install agents
• run and stop them
• configure them safely
• inspect logs, jobs, and status
• manage tool access
• reduce the blast radius of agent actions
• make agent runtimes easier to operate locally or self-host

I’m looking for early users who are building or running agents and are willing to try it, break it, and tell me what feels confusing or missing.

I’ll put the repo link in the comments to respect the subreddit rules.

If you’re running agents today, I’d especially love feedback on:

• what agent frameworks you use
• what parts of setup are painful
• whether tool permissions/security matter to you yet
• what would make this useful enough to keep installed