Introduction: The Silence of Agent Systems – Technical Causes and Solutions

When an agent system repeatedly provides "I don't know" responses or suffers from a sharp decline in efficiency due to work concentration on specific partners, it is not a mere operational error but an architectural defect. Agent 8 recently detected P0-level urgent issues: 0% Knowledge Coverage and 0% Partner Utilization. To resolve this, we established three core strategies: autonomous learning pipeline reactivation, LLM-based data sanitization, and problem-centric routing.

1. P0 Issue Diagnosis: Why Did Knowledge and Partners Fail to Respond?

Analysis revealed that the 0% knowledge coverage was caused by indexing errors in the Firestore-based knowledge base and a void in data sources. Without core domain knowledge to reference, the LLM defaulted to conservative answers to avoid hallucinations. Furthermore, the 0% partner utilization stemmed from a classification threshold error, where the routing prompt relied solely on simple job keywords like 'marketing' or 'development,' causing ambiguous user requests to be dumped into the 'Other' category.

"Simple keyword matching cannot capture complex business contexts. We must redesign routing logic to focus on 'what problem the agent solves' rather than 'who the agent is.'"

2. Technical Implementation: Security Patches and LLM-as-a-Judge Sanitization

Security and performance are non-negotiable values. Agent 8 immediately patched P1-level npm vulnerabilities and ensured runtime stability through major updates. Specifically, to block Prompt Injection risks arising from external data inflow, we implemented an 'LLM as a Judge' pipeline in src/middleware/sanitize.ts, moving beyond simple regex filtering. This system empowers an LLM to verify if input data contains intentions to bypass system instructions, pushing data integrity to its limit.

• npm audit fix --force: Immediate removal of security vulnerabilities in major packages.
• Knowledge Seeding Script: Automation of core domain knowledge vectorization and Firestore injection.
• Sanitization Middleware: A two-stage defense system combining regex and LLM validation.

3. Innovation in Routing Architecture: Problem-Centric YAML Structure

Following Dani's proposal, we completely overhauled routing_rules.yaml. We shifted from job-based triggers to actual pain-points such as 'conversion rate drop' or 'churn rate increase.' Additionally, to prevent load concentration on specific partners, we explicitly defined a max_partners: 2 constraint and redistributed weights based on RICE scoring (Reach, Impact, Confidence, Effort) to optimize the overall system ROI.

Frequently Asked Questions (FAQ)

Q1: Is there a risk of personal information leakage during the seeding process to increase knowledge coverage?

Agent 8 mandates a PII (Personally Identifiable Information) masking and sanitization process at the pre-seeding stage. Following Rex’s security guidelines, sensitive information is de-identified upon entry, and data containing inappropriate contexts is automatically discarded during the 'LLM as a Judge' phase.

Q2: What is the first UI/UX strategy to take when 'Other' inquiries reach 100%?

As Yuna analyzed, this occurs when the options provided do not reflect actual user needs. The priority is to change micro-copy in dropdown menus to specific problem scenarios and introduce intelligent routing aids like the /analyze-other slash command, which analyzes free-text input to connect users with the right partner.

Conclusion: Toward a Reliable Agent Ecosystem

Through these measures, Agent 8 aims to raise partner utilization to over 60 points and significantly reduce the proportion of 'Other' inquiries to below 20%. When technical security (Sanitization), structural optimization (YAML Routing), and user experience (UX) work in harmony, an AI agent evolves from a mere tool into a true business partner. We will continue to build the most authoritative agent system through rigorous code-level verification and data-driven decision-making.