AI observability/evaluation

Honeycombhoneycomb.ioAI observability/evaluation

Honeycomb is the observability platform built for AI-era software. Fast queries, unified telemetry, and LLM observability. Used by Slack, Intercom, and Dropbox.

Braintrust Data Inc.braintrust.devAI observability/evaluation

Ship quality agents at scale. Braintrust is the AI observability platform for tracing production, running evals, and catching regressions before they reach users.

Langchain (using Langsmith product)langchain.comAI observability/evaluation

LangChain enables every company to own their intelligence. Control, govern, and compound intelligence with an open agent engineering platform. Trusted by 7k+ organizations.

Cekuracekura.aiAI observability/evaluation

Cekura helps Conversational AI companies with automated QA. From pre-production simulation and evaluations to production call monitoring, to assisting customers in setting up the CI/CD pipeline, we help in QA across the agent lifecycle. We are working with 75+ customers across healthcare, BFSI, logistics, recruitment, and retail. Some of the largest CCAS players and enterprises use Cekura to ship AI voice and chat agents reliably

Langfuselangfuse.comAI observability/evaluation

Trace, evaluate, and improve AI agents with one open platform. Use production data to understand behavior, collaborate on fixes, and ship better quality at lower cost and latency.

Logfirepydantic.devAI observability/evaluation

Pydantic is the AI engineering stack for teams building with AI. Observe with Logfire, evaluate with Pydantic Evals, build type-safe agents with Pydantic AI, and track LLM costs with AI Gateway.

Invisible Tools Inc. (dba Raindrop.ai)raindrop.aiAI observability/evaluation

Trace every run. Auto-fix silent failures. Simulate before shipping.

Bluejaygetbluejay.aiAI observability/evaluation

Bluejay is the QA platform for AI agents. Rigorously test your voice agents—before and after deployment. Simulate edge cases, catch regressions, ensure security, and benchmark performance. Stop vibe testing. Quality is engineered.

Heliconehelicone.aiAI observability/evaluation

AI Gateway & LLM Observability

Oodleoodle.aiAI observability/evaluation

Observability built for agents. Give AI full production context across software and agents, with full-fidelity telemetry and sub-second queries at 80% lower cost.

Scale AIscale.comAI observability/evaluation

Scale delivers proven data, evaluations, and outcomes to AI labs, governments, and the Fortune 500.

Agnost AIagnost.aiAI observability/evaluation

Monitor AI agents in production. Detect silent failures, understand user frustration, and approve suggested fixes.

Anchorheadanchorhead.aiAI observability/evaluation

We have domain experts build real-world, custom evals for product teams — so you can assess capabilities, find edge cases, and prove value to customers.

Andon Labsandonlabs.comAI observability/evaluation

Andon Labs develops custom evaluations for AI models

Armaturearmature.techAI observability/evaluation

Be the tool coding agents choose. Get recommended and implemented, then see what users do and keep it working.

BentoLabs AIbentolabs.aiAI observability/evaluation

Monitor what runs. Improve what fails. Evaluate what changes. The closed-loop platform for production AI agents.

Besimple AIbesimple.aiAI observability/evaluation

Custom conversational audio datasets, expert annotation, and licensed voice actor data for training and evaluating speech and voice AI models.

Bilancbilanc.coAI observability/evaluation

Bilanc analyses every pull request with AI to measure engineering productivity, track AI tool ROI, and surface bottlenecks.

Chordiochordio.comAI observability/evaluation

Chordio's PX-bench measures how well frontier coding agents build product experience: design, UX, accessibility, performance, and functional correctness.

Confident AIconfident-ai.comAI observability/evaluation

Confident AI is the AI quality platform for enterprise teams to standardize AI evals and observability across the org — one consistent bar for how every team measures and monitors their AI.

Convergentconvergent.devAI observability/evaluation

Evals and optimization for production AI agents

DAGWorks Inc.dagworks.ioAI observability/evaluation

Build Reliable AI with Apache Hamilton & Apache Burr.

Dynamo AIdynamo.aiAI observability/evaluation

Dynamo AI offers end-to-end AI Performance, Security, and Compliance solutions for delivering Enterprise-grade Generative AI.

Dynatracedynatrace.comAI observability/evaluation

Innovate faster, operate more efficiently, and drive better business outcomes with observability, AI, automation, and application security in one platform.