Documentation Index
Fetch the complete documentation index at: https://docs.abv.dev/llms.txt
Use this file to discover all available pages before exploring further.
You can track the effect of changes to your LLM app on metrics in ABV. This allows you to:
- Run experiments (A/B tests) in production and measure the impact on costs, latencies and quality.
- Example: “What is the impact of switching to a new model?”
- Explain changes to metrics over time.
- Example: “Why did latency in this chain increase?”
Releases
A release tracks the overall version of your application. Commonly it is set to the semantic version or git commit hash of your application.
The SDKs look for a release in the following order:
- SDK initialization
- Environment variable
- Automatically on popular platforms
SDK initialization
Environment variable
The SDKs will look for a ABV_RELEASE environment variable. Use it to configure the release e.g. in your CI/CD pipeline.
ABV_RELEASE = "<release_tag>" # <- github sha or other identifier
Python SDK
Install packages
pip install abvdev, python-dotenv
.env file in your project root.
ABV_API_KEY = "sk-abv-..."
ABV_HOST = "https://app.abv.dev" # US region
# ABV_HOST = "https://eu.app.abv.dev" # EU region
ABV_RELEASE = "3.2.25"
# Import the function to load the .env file
from dotenv import load_dotenv
from abvdev import observe, get_client
# Load the environment variables from the .env file
load_dotenv()
@observe
def my_function():
return "Hello, world!" # Input/output and timings are automatically captured
my_function()
# Flush events in short-lived applications
abv = get_client()
abv.flush()
from abvdev import ABV, observe
# Set the release when initializing the client
abv = ABV(
release="v2.1.25",
api_key="sk-abv-...", # your api key here
host="https://app.abv.dev", # host="https://eu.app.abv.dev", for EU region
)
@observe
def my_function():
return "Hello, world!" # Input/output and timings are automatically captured
my_function()
# Flush events in short-lived applications
abv.flush()
JS/TS SDK
Install packages
npm install @abvdev/tracing @abvdev/otel @opentelemetry/sdk-node
.env file in your project root and a package like dotenv to load the variables.
ABV_API_KEY = "sk-abv-..."
ABV_BASE_URL = "https://app.abv.dev" # US region
# ABV_BASE_URL = "https://eu.app.abv.dev" # EU region
ABV_RELEASE = "1.0.0"
instrumentation.ts file that initializes the OpenTelemetry NodeSDK and registers the ABVSpanProcessor.
import { NodeSDK } from "@opentelemetry/sdk-node";
import { ABVSpanProcessor } from "@abvdev/otel";
const sdk = new NodeSDK({
spanProcessors: [new ABVSpanProcessor()],
});
sdk.start();
instrumentation.ts file to use dotenv package to load the variables.
Additional parameters are provided to get trace visible in the UI immediately.
import dotenv from "dotenv";
dotenv.config();
import { NodeSDK } from "@opentelemetry/sdk-node";
import { ABVSpanProcessor } from "@abvdev/otel";
const sdk = new NodeSDK({
spanProcessors: [
new ABVSpanProcessor({
apiKey: process.env.ABV_API_KEY,
baseUrl: process.env.ABV_BASE_URL,
exportMode: "immediate",
flushAt: 1,
flushInterval: 1,
additionalHeaders: {
"Content-Type": "application/json",
"Accept": "application/json"
}
})
],
});
sdk.start();
instrumentation.ts file at the top of your application.
import "./instrumentation"; // Must be the first import
release is set, the ABV SDKs default to a set of known release environment variables.
Supported platforms include: Vercel, Heroku, Netlify. See the full list of support environment variables for JS/TS and .
Versions
The version parameter can be added to traces and all observation types (span, generation, event). Thereby, you can track the effect of a new version on the metrics of an object with a specific name using ABV analytics.
With Python SDK
When using the @observe() decorator, version could be specified at both trace or observation levels:
from abvdev import ABV, observe
# ABV client initialization
abv = ABV(
api_key="sk-abv-...", # your api key here
host="https://app.abv.dev", # host="https://eu.app.abv.dev", for EU region
)
@observe()
def process_data():
# Set version at trace level
abv.update_current_trace(version="1.0")
# Set version at observation level
abv.update_current_span(version="1.0")
process_data()
from abvdev import ABV
from openai import OpenAI
# ABV client initialization
abv = ABV(
api_key="sk-abv-...", # your api key here
host="https://app.abv.dev", # host="https://eu.app.abv.dev", for EU region
)
openai_client = OpenAI(api_key = "sk-proj-...")
# Set version when creating a span
with abv.start_as_current_span(
name="process-data",
version="1.0"
) as span:
# Processing...
span.update(output="Processing complete")
# Create a generation with version
with span.start_as_current_observation(
as_type='generation',
name="guess-countries",
model="gpt-4o",
version="1.1"
) as generation:
# Generation code...
response = openai_client.chat.completions.create(
messages=[{"role": "user", "content": "Hi openai, how is that going?"}],
model="gpt-4o",
)
generation.update(output=response.choices[0].message.content)
# Flush events in short-lived applications
abv.flush()
With JS/TS SDK
import "./instrumentation";
import { startActiveObservation } from "@abvdev/tracing";
async function main() {
await startActiveObservation("my-first-trace-version1", async (span) => {
span.update({
input: "Hello, ABV!",
output: "This is my first trace!",
version: "1.0"
});
});
}
main();
abv.trace(), abv.span() and abv.event() also take an optional version parameter.
In UI
Version parameter in ABV interface
