Percival Chat: An Eval Copilot for Agentic Systems

Percival Chat

Chat, your eval co-pilot, aims to simplify the agent evaluation process even more by providing you with in-context guidance on tracing, integrations, evaluation criteria, and prompting.

‍

Chat with Traces

Traces offer a treasure trove of information about opportunities for improvement. However, they are long, complex, and can be difficult to manage for large projects. Percival Chat allows you to analyze your traces conversationally, drill into details, and save context as you go.

Have some of your project names and trace IDs handy and get started!

Insights

Percival Chat elicits insights from your traces. It helps you identify recurring issues and patterns across multiple traces and time periods, detect security failures, cluster errors for easier resolution, and identify optimization opportunities.

Span Analysis

This allows you to explore trace components in detail, including understanding call chains and dependencies, categorizing failing operations, comparing span execution times, and monitoring tools and API usage patterns.

Detailed Inspection

You can go even further by examining specific fields, values, logs, LLM calls, database queries, and API requests to pinpoint exactly what may be causing the issue at hand.

‍

Features

Memory Files

With memory, Percival can remember information between chat sessions, allowing you to go deeper rather than starting from scratch each time. You can store trace context, system processes, log structure, important span attributes, and additional guidance on analysis or debugging to simplify future chat sessions.

Chat Sharing

After your conversation, you can share your debugging sessions with others on your team or publicly, allowing you to collaborate on even the most granular analysis.

The goal of Percival is to make working on evaluation and optimization even easier. 

‍

How To Get Started

You can chat with Percival in your main information panel. Log in and supercharge your analyses. Reach out to us with any questions.