Improve your AI applications with LLM evaluation.  Learn about online and offline evaluation methods, and key concepts like scores, datasets, and dataset runs to enhance model accuracy, performance, and user trust.

Evaluation Overview

Explore the ABV Scores Data Model for LLM evaluation.  Learn how to structure scores and use score configs to track numeric, categorical, and boolean metrics consistently.

Scores Data Model

Use LLM-as-a-judge for scalable, cost-effective LLM evaluation.  Our guide shows how to set up custom or managed evaluators in ABV to automate quality scoring.

LLM-as-a-Judge

Discover how to use Human Annotation in ABV for manual LLM evaluation.  Learn to collaboratively annotate traces, sessions, and observations, use Annotation Queues for large batches, and set a human baseline for scoring.

Human Annotation

Implement flexible LLM evaluation with Custom Scores.  Learn to ingest user feedback, run custom pipelines, and set guardrails using the ABV Python & JS/TS SDKs.

Custom Scores

Learn about the ABV data model for Datasets and Dataset Runs.  Understand how to use collections of inputs and expected outputs to evaluate and test your LLM applications effectively.

Dataset Runs Data Model

Create robust test cases for your LLM app with ABV Datasets.  Learn to build test data from production traces or synthetic examples using our UI and SDKs.

Datasets

Run Prompt Experiments (Native Dataset Runs) in the ABV UI to test and compare different prompt versions or language models side-by-side.  Learn how to use LLM-as-a-Judge evaluators for automatic scoring to prevent regressions.

Prompt Experiments

Automate LLM application testing with Remote Dataset Runs.  Get full flexibility to run experiments and apply custom scoring logic using the ABV Python & JS/TS SDKs.

Remote Dataset Runs

Link Prompts to Traces

Evaluations

Trace

ABV Developer Docs

Quickstart (Python SDK)

Quickstart (JS/TS SDK)

Observability & Tracing

Sessions

User Tracking

Environments

ABV Prompt Playground

Metadata

Trace IDs & Distributed Tracing

Log Levels

Comments on Objects

Masking of Sensitive LLM Data

Multi-Modality and Attachments

Event Queuing/Batching

Releases & Versioning

Sampling

Model Usage & Cost Tracking

Trace URLs

Query Data

API & Data Platform (cloned)

Metrics API

Metrics API (cloned with children)

Custom Dashboards

Basic Features

Prompt Management Overview

Get Started with Prompt Management

ABV Prompts Data Model

Prompt Version Control

Prompt Composability

Message Placeholders in Chat Prompts

A/B Testing of LLM Prompts

Caching of Prompts in Client SDKs

Prompt Config

Prompt Folders

Guaranteed Availability

Prompt Management

Overview of ABV SDKs

Python SDK - Overview

Python SDK - Setup

Python SDK - Instrumentation

Python SDK - Evaluations

Python SDK - Advanced Usage

Python SDK - Troubleshooting

Python SDK

TypeScript SDK - Overview

TypeScript SDK - Setup

TypeScript SDK - Instrumentation

TypeScript SDK - Advanced Configuration

TypeScript SDK - Troubleshooting and FAQ

Cookbook: ABV JS/TS SDK

JS/TS SDK

SDKs

Metrics overview

Metrics

API & Data Platform overview

Export Data from UI

Export via Blob Storage Integration

Export for Fine-Tuning

Public API

Query Data via SDKs

API & Data Platform

Role-Based Access Controls in ABV

Audit Logs

Data Deletion

Data Retention

LLM Connections

SCIM & Organization-Key Scoped API Routes

Usage Alerts

Administration

Platform

API Reference

References

Python SDK (v3)

JS SDK

Security & Compliance Overview

More