You can track the effect of changes to your LLM app on metrics in ABV. This allows you to:

Run experiments (A/B tests) in production and measure the impact on costs, latencies and quality.
- Example: “What is the impact of switching to a new model?”
Explain changes to metrics over time.
- Example: “Why did latency in this chain increase?”

Releases

A release tracks the overall version of your application. Commonly it is set to the semantic version or git commit hash of your application. The SDKs look for a release in the following order:

SDK initialization
Environment variable
Automatically on popular platforms

SDK initialization

Environment variable

The SDKs will look for a ABV_RELEASE environment variable. Use it to configure the release e.g. in your CI/CD pipeline.

ABV_RELEASE = "<release_tag>" # <- github sha or other identifier

Python SDK

Install packages

pip install abvdev, python-dotenv

Add credentials and ABV_RELEASE environment variable: Add your ABV credentials to your environment variables. Make sure that you have a .env file in your project root.

.env

ABV_API_KEY = "sk-abv-..."
ABV_HOST = "https://app.abv.dev" # US region
# ABV_HOST = "https://eu.app.abv.dev" # EU region
ABV_RELEASE = "3.2.25"

load environment variables and initialize client with get_client:

# Import the function to load the .env file
from dotenv import load_dotenv 
from abvdev import observe, get_client

# Load the environment variables from the .env file
load_dotenv() 

@observe
def my_function():
    return "Hello, world!" # Input/output and timings are automatically captured
 
my_function()
 
# Flush events in short-lived applications
abv = get_client()
abv.flush()

The Python SDK also allows you to set the release when initializing the client with client parameters:

from abvdev import ABV, observe

# Set the release when initializing the client
abv = ABV(
    release="v2.1.25",
    api_key="sk-abv-...", # your api key here
    host="https://app.abv.dev", # host="https://eu.app.abv.dev", for EU region
)

@observe
def my_function():
    return "Hello, world!" # Input/output and timings are automatically captured
 
my_function()
 
# Flush events in short-lived applications
abv.flush()

JS/TS SDK

Install packages

npm install @abvdev/tracing @abvdev/otel @opentelemetry/sdk-node

**Add credentials and **ABV_RELEASE environment variable Add your ABV credentials to your environment variables. Make sure that you have a .env file in your project root and a package like dotenv to load the variables.

.env

ABV_API_KEY = "sk-abv-..."
ABV_BASE_URL = "https://app.abv.dev" # US region
# ABV_BASE_URL = "https://eu.app.abv.dev" # EU region
ABV_RELEASE = "1.0.0"

Create a instrumentation.ts file that initializes the OpenTelemetry NodeSDK and registers the ABVSpanProcessor.

npm install dotenv

instrumentation.ts

import { NodeSDK } from "@opentelemetry/sdk-node";
import { ABVSpanProcessor } from "@abvdev/otel";

const sdk = new NodeSDK({
  spanProcessors: [new ABVSpanProcessor()],
});

sdk.start();

Modify instrumentation.ts file to use dotenv package to load the variables. Additional parameters are provided to get trace visible in the UI immediately.

instrumentation.ts

import dotenv from "dotenv";
dotenv.config();

import { NodeSDK } from "@opentelemetry/sdk-node";
import { ABVSpanProcessor } from "@abvdev/otel";

const sdk = new NodeSDK({
  spanProcessors: [
    new ABVSpanProcessor({
      apiKey: process.env.ABV_API_KEY,
      baseUrl: process.env.ABV_BASE_URL,
      exportMode: "immediate",
      flushAt: 1,
      flushInterval: 1,
      additionalHeaders: {
        "Content-Type": "application/json",
        "Accept": "application/json"
      }
    })
  ],
});

sdk.start();

Import the instrumentation.ts file at the top of your application.

index.ts

import "./instrumentation"; // Must be the first import

Instrument application The JS/TS SDK will look for a ABV_RELEASE environment variable. Use it also to configure the release e.g. in your CI/CD pipeline.

Automatically on popular platforms

If no other release is set, the ABV SDKs default to a set of known release environment variables. Supported platforms include: Vercel, Heroku, Netlify. See the full list of support environment variables for JS/TS and .

Versions

The version parameter can be added to traces and all observation types (span, generation, event). Thereby, you can track the effect of a new version on the metrics of an object with a specific name using ABV analytics.

With Python SDK

When using the @observe() decorator, version could be specified at both trace or observation levels:

from abvdev import ABV, observe

# ABV client initialization          
abv = ABV(
    api_key="sk-abv-...", # your api key here
    host="https://app.abv.dev", # host="https://eu.app.abv.dev", for EU region
)

@observe()
def process_data():
    # Set version at trace level
    abv.update_current_trace(version="1.0")

    # Set version at observation level
    abv.update_current_span(version="1.0")
    
process_data()

When creating spans directly:

from abvdev import ABV
from openai import OpenAI

# ABV client initialization          
abv = ABV(
    api_key="sk-abv-...", # your api key here
    host="https://app.abv.dev", # host="https://eu.app.abv.dev", for EU region
)

openai_client = OpenAI(api_key = "sk-proj-...")

# Set version when creating a span
with abv.start_as_current_span(
    name="process-data",
    version="1.0"
) as span:
    # Processing...
    span.update(output="Processing complete")

    # Create a generation with version
    with span.start_as_current_observation(
        as_type='generation',
        name="guess-countries",
        model="gpt-4o",
        version="1.1"
    ) as generation:
        # Generation code...
        response = openai_client.chat.completions.create(
            messages=[{"role": "user", "content": "Hi openai, how is that going?"}],
            model="gpt-4o",
        )
            
        generation.update(output=response.choices[0].message.content)
        
# Flush events in short-lived applications
abv.flush()

The version parameter can be included in both spans and generations, and can be set either during creation or updated later.

With JS/TS SDK

import "./instrumentation";
import { startActiveObservation } from "@abvdev/tracing";

async function main() {
  await startActiveObservation("my-first-trace-version1", async (span) => {
    span.update({
      input: "Hello, ABV!",
      output: "This is my first trace!",
      version: "1.0"
    });
  });
}

main();

abv.trace(), abv.span() and abv.event() also take an optional version parameter.

In UI

Version parameter in ABV interface

Getting Started

Basic Features

LLM Gateway

Guardrails

Evaluations

Prompt Management

Cookbook

SDKs

Platform

Support

Releases & Versioning

Releases

SDK initialization

Environment variable

Python SDK

JS/TS SDK

Automatically on popular platforms

Versions

With Python SDK

With JS/TS SDK

In UI

​Releases

​SDK initialization

​Environment variable

​Python SDK

​JS/TS SDK

​Automatically on popular platforms

​Versions

​With Python SDK

​With JS/TS SDK

​In UI

Releases

SDK initialization

Environment variable

Python SDK

JS/TS SDK

Automatically on popular platforms

Versions

With Python SDK

With JS/TS SDK

In UI