Skip to main content

How Sessions Work

A session groups multiple traces together to represent a complete user journey, conversation, or multi-step job. All traces with the same sessionId are linked, enabling end-to-end analysis. Session benefits:
  • End-to-end visibility: See complete user journeys from start to finish
  • Aggregated metrics: Track total cost, latency, and token usage across all traces in a session
  • Context preservation: Understand how earlier interactions influence later ones
Common session types:
All messages in a single conversation share the same session ID.Example conversation:
  • User: “What’s the weather in Paris?”
  • Bot: “It’s 72°F and sunny.”
  • User: “What about tomorrow?”
  • Bot: “Tomorrow will be 68°F with rain.”
All 4 traces share sessionId = "chat-abc-123"
All steps in a workflow share the same session ID.Example pipeline:
  1. Upload document → trace 1
  2. Extract text → trace 2
  3. Summarize content → trace 3
  4. Classify topic → trace 4
All 4 steps share sessionId = "doc-processing-xyz"
All activity during a single login session shares the same session ID.Example:
  • User logs in at 9:00 AM
  • Asks 10 questions over 2 hours
  • Logs out at 11:00 AM
All 10 queries share sessionId = "user-456-login-789"
Use session IDs that are meaningful to your application: user session tokens, conversation IDs, job IDs, order IDs, etc. ABV groups traces by exact string match.

Viewing and Analyzing Sessions

Navigate to the Sessions view in the ABV Dashboard to see all sessions with aggregated metrics.Available metrics:
  • Trace count per session
  • Total cost (sum of all LLM costs)
  • Total duration (first to last trace)
  • Error rate
  • User/tenant context
Click any session to see all traces in chronological order with a timeline visualization.
In the Traces view, filter by session ID to isolate specific sessions.Query examples:
  • sessionId = "chat-abc-789" - Specific session
  • sessionId LIKE "chat-%" - All chat sessions
  • sessionId = "doc-xyz" AND status = "error" - Failed traces in session
Fetch session data via API for custom analysis and reporting.Fetch session by ID:
import requests

response = requests.get(
    f"https://app.abv.dev/api/public/sessions/{session_id}",
    headers={"Authorization": f"Bearer {api_key}"}
)

session = response.json()
print(f"Traces: {len(session['traces'])}, Cost: ${session['totalCost']:.2f}")
List sessions for a user:
response = requests.get(
    "https://app.abv.dev/api/public/sessions",
    headers={"Authorization": f"Bearer {api_key}"},
    params={"userId": "user-123", "limit": 50}
)

Why Use Sessions?

Chatbots and assistants build context over multiple turns. When a conversation fails, you need the full history to understand why.Example: Customer support chatbot failureConversation flow:
  1. User: “I want to return my order”
  2. Bot: “Sure! What’s your order number?”
  3. User: “ORDER-123”
  4. Bot: “I found your order. What’s the reason for the return?”
  5. User: “The item was damaged”
  6. Bot: [ERROR] “I couldn’t process your return”
Without sessions:
  • You only see trace #6 (the failure)
  • No context about what the user asked before
  • Can’t reproduce the issue without the full conversation
  • Guessing why the bot failed
With sessions:
  • Click the session to see all 6 traces in order
  • See that the bot stored order_id = "ORDER-123" in context
  • Notice the bot attempted to call process_return(order_id, reason) but the API timed out
  • Root cause: The return API is slow for order “ORDER-123” (large order with 10 items)
Benefits:
  • Full conversation history for debugging
  • See how context accumulated over turns
  • Identify which turn introduced the error
  • Reproduce issues by replaying the exact conversation
Multi-step workflows (RAG pipelines, agent workflows, document processing) span multiple traces. Sessions aggregate metrics across all steps.Example: Document processing pipelineWorkflow:
  1. Upload document (0.5s, $0.00)
  2. Extract text with OCR (3.2s, $0.10)
  3. Chunk into paragraphs (0.3s, $0.00)
  4. Embed chunks with text-embedding-ada-002 (1.8s, $0.02)
  5. Summarize with GPT-4 (4.5s, $0.15)
  6. Classify topic with GPT-3.5 (1.2s, $0.03)
  7. Format output (0.2s, $0.00)
Session metrics:
  • Total duration: 11.7 seconds (sum of all steps)
  • Total cost: $0.30 (sum of all LLM calls)
  • Bottleneck: Step 5 (GPT-4 summarization) takes 38% of total time
Optimization decision:
  • Switch from GPT-4 to GPT-3.5-turbo for summarization
  • New duration: 9.2 seconds (21% faster)
  • New cost: $0.18 (40% cheaper)
  • Quality: No degradation (validated with evaluations)
Benefits:
  • Measure end-to-end latency instead of just individual steps
  • Calculate total cost per workflow
  • Identify bottlenecks across multi-step processes
  • Justify optimizations with concrete data
Users interact with your application over multiple requests. Sessions group their journey for behavioral analysis.Example: Onboarding flowUser journey (session “onboarding-user-456”):
  1. User signs up
  2. User completes profile
  3. User asks: “How do I upload documents?”
  4. Bot explains document upload
  5. User uploads first document
  6. User asks: “How do I share this?”
  7. Bot explains sharing
  8. User shares document with team
Session analysis:
  • Total traces: 8
  • Total duration: 12 minutes (from signup to first share)
  • Questions asked: 2 (steps 3 and 6)
  • Feature usage: Document upload (step 5), sharing (step 8)
Insights:
  • Users need help with document upload and sharing (common questions)
  • Average time to first share: 12 minutes (can we reduce this?)
  • 80% of users who complete onboarding ask at least one question
Improvements:
  • Add tooltips for document upload and sharing
  • Measure impact: Time to first share drops to 8 minutes
  • Onboarding completion rate increases from 60% to 75%
Benefits:
  • Understand user behavior end-to-end
  • Identify friction points in user journeys
  • Measure impact of UX changes
  • Personalize experiences based on session history
AI agents execute complex multi-step workflows: planning, tool calls, reflection, iteration. Sessions capture the entire agent execution.Example: Research agentAgent workflow (session “research-task-789”):
  1. Agent: Plan research strategy (LLM call)
  2. Agent: Search web for “climate change impact” (tool call)
  3. Agent: Read 3 articles (tool calls)
  4. Agent: Summarize findings (LLM call)
  5. Agent: Search for “climate policy solutions” (tool call)
  6. Agent: Read 2 more articles (tool calls)
  7. Agent: Generate final report (LLM call)
Session metrics:
  • Total traces: 7 (3 LLM calls, 6 tool calls)
  • Total cost: $0.45
  • Total duration: 23 seconds
  • Tools used: Web search (2×), article reader (5×)
Analysis:
  • The agent made 2 rounds of research (steps 2-4, then 5-7)
  • Reading articles took 15 seconds (65% of total time)
  • LLM calls were cheap (0.15)buttoolcallswereexpensive(0.15) but tool calls were expensive (0.30)
Optimization:
  • Cache article summaries to avoid re-reading
  • Parallelize article reads (5 sequential → 2 parallel)
  • New duration: 12 seconds (48% faster)
  • New cost: $0.35 (22% cheaper)
Benefits:
  • See full agent reasoning and tool usage
  • Measure cost and latency of agent workflows
  • Identify inefficient tool usage patterns
  • Optimize agent behavior with data
For multi-step interactions, analyze total cost per session rather than per individual trace.Customer support chatbot example:
  • Short sessions (1-2 turns): Lower cost, fast resolution
  • Medium sessions (3-5 turns): Moderate cost, typical cases
  • Long sessions (6+ turns): Higher cost, complex issues
Insight: Long sessions (15% of volume) may drive 40% of costs. Optimize by using faster/cheaper models for simple queries and premium models only for complex cases.Benefits:
  • Accurate cost accounting per user journey
  • Identify expensive session patterns
  • Justify pricing tiers based on session costs
  • Optimize model selection by session complexity
Support teams need full context when taking over from a chatbot or another agent.Example: Bot-to-human handoffSession flow:
  1. User: “I need a refund” (bot handles)
  2. Bot: “Can you provide your order number?” (bot handles)
  3. User: “ORDER-123” (bot handles)
  4. Bot: “This order requires manager approval. Escalating to human support.” (bot escalates)
  5. Human agent joins → sees full session history instantly
  6. Agent: “I see you ordered 10 items and one was damaged. I’ll process your refund now.”
Without sessions:
  • Human agent asks user to repeat everything
  • User frustrated: “I already told the bot this!”
  • Time wasted re-gathering context
With sessions:
  • Human agent clicks session replay
  • Sees full conversation history in 5 seconds
  • Jumps directly to resolution without asking user to repeat
Benefits:
  • Seamless bot-to-human handoffs
  • Reduced user frustration
  • Faster time to resolution
  • Better support experience

Implementation Guide

Use the @observe() decorator and update the session ID with abv.update_current_trace().Setup:
pip install abvdev
Basic usage:
from abvdev import observe, ABV

abv = ABV(api_key="sk-abv-...", host="https://app.abv.dev")

@observe()
def process_request(session_id, user_message):
    # Add session ID to the trace
    abv.update_current_trace(session_id=session_id)

    # Process the message
    response = llm.generate(user_message)
    return response

# Simulate a conversation
session_id = "chat-abc-123"
process_request(session_id, "What is the capital of France?")
process_request(session_id, "What about Germany?")
# Both traces belong to session "chat-abc-123"
Generate session IDs:
import uuid

# Option 1: Random UUID
session_id = str(uuid.uuid4())  # "f47ac10b-58cc-4372-a567-0e02b2c3d479"

# Option 2: User-based session ID
user_id = "user-123"
login_timestamp = "20250115-143000"
session_id = f"{user_id}-{login_timestamp}"  # "user-123-20250115-143000"

# Option 3: Conversation ID from your database
session_id = conversation.id  # "conv-abc-789"
Set session IDs when creating spans manually.Example:
from abvdev import ABV

abv = ABV(api_key="sk-abv-...", host="https://app.abv.dev")

# Multi-step workflow with session ID
session_id = "doc-processing-xyz"

# Step 1: Extract text
with abv.start_as_current_span(name="extract-text") as span:
    span.update_trace(session_id=session_id)
    text = extract_text_from_pdf(document)

# Step 2: Summarize text
with abv.start_as_current_span(name="summarize-text") as span:
    span.update_trace(session_id=session_id)
    summary = llm.summarize(text)

# Step 3: Classify topic
with abv.start_as_current_span(name="classify-topic") as span:
    span.update_trace(session_id=session_id)
    topic = llm.classify(summary)

# All 3 traces belong to session "doc-processing-xyz"
Update session ID without direct span reference:
with abv.start_as_current_span(name="another-operation"):
    # Add to the current trace
    abv.update_current_trace(session_id="your-session-id")
Use updateActiveTrace() to set session IDs.Setup:
npm install @abvdev/tracing @abvdev/otel @opentelemetry/sdk-node dotenv
Configuration (instrumentation.ts):
import dotenv from 'dotenv';
dotenv.config();

import { NodeSDK } from '@opentelemetry/sdk-node';
import { ABVSpanProcessor } from '@abvdev/otel';

const sdk = new NodeSDK({
  spanProcessors: [
    new ABVSpanProcessor({
      apiKey: process.env.ABV_API_KEY,
      baseUrl: process.env.ABV_BASE_URL,
      exportMode: 'immediate',
      flushAt: 1,
      flushInterval: 1
    })
  ]
});

sdk.start();
Add session IDs:
import './instrumentation';
import { startActiveObservation, updateActiveTrace } from '@abvdev/tracing';

async function processChatMessage(sessionId: string, message: string) {
  await startActiveObservation('process-chat-message', async (span) => {
    // Set session ID on trace
    updateActiveTrace({
      sessionId: sessionId
    });

    // Process message
    const response = await llm.generate(message);
    return response;
  });
}

// Conversation
const sessionId = 'chat-abc-123';
await processChatMessage(sessionId, 'What is the capital of France?');
await processChatMessage(sessionId, 'What about Germany?');
Wrap existing functions with automatic tracing and session IDs.Example:
import './instrumentation';
import { observe, updateActiveTrace } from '@abvdev/tracing';

// Original function
async function fetchData(sessionId: string, source: string) {
  // Set session ID
  updateActiveTrace({
    sessionId: sessionId
  });

  // Fetch data
  const response = await fetch(source);
  return await response.json();
}

// Wrap with observe
const tracedFetchData = observe(fetchData, {
  name: 'fetch-data-operation'
});

// Use traced version
async function main() {
  const sessionId = 'workflow-xyz-789';
  const result = await tracedFetchData(sessionId, 'https://api.example.com/data');
}

main();
Create spans manually and set session IDs on traces.Example:
import './instrumentation';
import { startObservation } from '@abvdev/tracing';

const sessionId = 'chat-abc-123';

const span = startObservation('manual-observation', {
  input: { query: 'What is the capital of France?' }
});

// Set session ID on the trace
span.updateTrace({
  sessionId: sessionId
});

span.update({ output: 'Paris' }).end();

Related Features