Skip to main content

How Sessions Work

A session groups multiple traces together to represent a complete user journey, conversation, or multi-step job. All traces with the same sessionId are linked, enabling end-to-end analysis. Session benefits:
  • End-to-end visibility: See complete user journeys from start to finish
  • Aggregated metrics: Track total cost, latency, and token usage across all traces in a session
  • Context preservation: Understand how earlier interactions influence later ones
Common session types:
All messages in a single conversation share the same session ID.Example conversation:
  • User: “What’s the weather in Paris?”
  • Bot: “It’s 72°F and sunny.”
  • User: “What about tomorrow?”
  • Bot: “Tomorrow will be 68°F with rain.”
All 4 traces share sessionId = "chat-abc-123"
All steps in a workflow share the same session ID.Example pipeline:
  1. Upload document → trace 1
  2. Extract text → trace 2
  3. Summarize content → trace 3
  4. Classify topic → trace 4
All 4 steps share sessionId = "doc-processing-xyz"
All activity during a single login session shares the same session ID.Example:
  • User logs in at 9:00 AM
  • Asks 10 questions over 2 hours
  • Logs out at 11:00 AM
All 10 queries share sessionId = "user-456-login-789"
Use session IDs that are meaningful to your application: user session tokens, conversation IDs, job IDs, order IDs, etc. ABV groups traces by exact string match.

Viewing and Analyzing Sessions

Navigate to the Sessions view in the ABV Dashboard to see all sessions with aggregated metrics.Available metrics:
  • Trace count per session
  • Total cost (sum of all LLM costs)
  • Total duration (first to last trace)
  • Error rate
  • User/tenant context
Click any session to see all traces in chronological order with a timeline visualization.
In the Traces view, filter by session ID to isolate specific sessions.Query examples:
  • sessionId = "chat-abc-789" - Specific session
  • sessionId LIKE "chat-%" - All chat sessions
  • sessionId = "doc-xyz" AND status = "error" - Failed traces in session
Fetch session data via API for custom analysis and reporting.Fetch session by ID:
List sessions for a user:

Why Use Sessions?

Chatbots and assistants build context over multiple turns. When a conversation fails, you need the full history to understand why.Example: Customer support chatbot failureConversation flow:
  1. User: “I want to return my order”
  2. Bot: “Sure! What’s your order number?”
  3. User: “ORDER-123”
  4. Bot: “I found your order. What’s the reason for the return?”
  5. User: “The item was damaged”
  6. Bot: [ERROR] “I couldn’t process your return”
Without sessions:
  • You only see trace #6 (the failure)
  • No context about what the user asked before
  • Can’t reproduce the issue without the full conversation
  • Guessing why the bot failed
With sessions:
  • Click the session to see all 6 traces in order
  • See that the bot stored order_id = "ORDER-123" in context
  • Notice the bot attempted to call process_return(order_id, reason) but the API timed out
  • Root cause: The return API is slow for order “ORDER-123” (large order with 10 items)
Benefits:
  • Full conversation history for debugging
  • See how context accumulated over turns
  • Identify which turn introduced the error
  • Reproduce issues by replaying the exact conversation
Multi-step workflows (RAG pipelines, agent workflows, document processing) span multiple traces. Sessions aggregate metrics across all steps.Example: Document processing pipelineWorkflow:
  1. Upload document (0.5s, $0.00)
  2. Extract text with OCR (3.2s, $0.10)
  3. Chunk into paragraphs (0.3s, $0.00)
  4. Embed chunks with text-embedding-ada-002 (1.8s, $0.02)
  5. Summarize with GPT-4 (4.5s, $0.15)
  6. Classify topic with GPT-3.5 (1.2s, $0.03)
  7. Format output (0.2s, $0.00)
Session metrics:
  • Total duration: 11.7 seconds (sum of all steps)
  • Total cost: $0.30 (sum of all LLM calls)
  • Bottleneck: Step 5 (GPT-4 summarization) takes 38% of total time
Optimization decision:
  • Switch from GPT-4 to GPT-3.5-turbo for summarization
  • New duration: 9.2 seconds (21% faster)
  • New cost: $0.18 (40% cheaper)
  • Quality: No degradation (validated with evaluations)
Benefits:
  • Measure end-to-end latency instead of just individual steps
  • Calculate total cost per workflow
  • Identify bottlenecks across multi-step processes
  • Justify optimizations with concrete data
Users interact with your application over multiple requests. Sessions group their journey for behavioral analysis.Example: Onboarding flowUser journey (session “onboarding-user-456”):
  1. User signs up
  2. User completes profile
  3. User asks: “How do I upload documents?”
  4. Bot explains document upload
  5. User uploads first document
  6. User asks: “How do I share this?”
  7. Bot explains sharing
  8. User shares document with team
Session analysis:
  • Total traces: 8
  • Total duration: 12 minutes (from signup to first share)
  • Questions asked: 2 (steps 3 and 6)
  • Feature usage: Document upload (step 5), sharing (step 8)
Insights:
  • Users need help with document upload and sharing (common questions)
  • Average time to first share: 12 minutes (can we reduce this?)
  • 80% of users who complete onboarding ask at least one question
Improvements:
  • Add tooltips for document upload and sharing
  • Measure impact: Time to first share drops to 8 minutes
  • Onboarding completion rate increases from 60% to 75%
Benefits:
  • Understand user behavior end-to-end
  • Identify friction points in user journeys
  • Measure impact of UX changes
  • Personalize experiences based on session history
AI agents execute complex multi-step workflows: planning, tool calls, reflection, iteration. Sessions capture the entire agent execution.Example: Research agentAgent workflow (session “research-task-789”):
  1. Agent: Plan research strategy (LLM call)
  2. Agent: Search web for “climate change impact” (tool call)
  3. Agent: Read 3 articles (tool calls)
  4. Agent: Summarize findings (LLM call)
  5. Agent: Search for “climate policy solutions” (tool call)
  6. Agent: Read 2 more articles (tool calls)
  7. Agent: Generate final report (LLM call)
Session metrics:
  • Total traces: 7 (3 LLM calls, 6 tool calls)
  • Total cost: $0.45
  • Total duration: 23 seconds
  • Tools used: Web search (2×), article reader (5×)
Analysis:
  • The agent made 2 rounds of research (steps 2-4, then 5-7)
  • Reading articles took 15 seconds (65% of total time)
  • LLM calls were cheap (0.15)buttoolcallswereexpensive(0.15) but tool calls were expensive (0.30)
Optimization:
  • Cache article summaries to avoid re-reading
  • Parallelize article reads (5 sequential → 2 parallel)
  • New duration: 12 seconds (48% faster)
  • New cost: $0.35 (22% cheaper)
Benefits:
  • See full agent reasoning and tool usage
  • Measure cost and latency of agent workflows
  • Identify inefficient tool usage patterns
  • Optimize agent behavior with data
For multi-step interactions, analyze total cost per session rather than per individual trace.Customer support chatbot example:
  • Short sessions (1-2 turns): Lower cost, fast resolution
  • Medium sessions (3-5 turns): Moderate cost, typical cases
  • Long sessions (6+ turns): Higher cost, complex issues
Insight: Long sessions (15% of volume) may drive 40% of costs. Optimize by using faster/cheaper models for simple queries and premium models only for complex cases.Benefits:
  • Accurate cost accounting per user journey
  • Identify expensive session patterns
  • Justify pricing tiers based on session costs
  • Optimize model selection by session complexity
Support teams need full context when taking over from a chatbot or another agent.Example: Bot-to-human handoffSession flow:
  1. User: “I need a refund” (bot handles)
  2. Bot: “Can you provide your order number?” (bot handles)
  3. User: “ORDER-123” (bot handles)
  4. Bot: “This order requires manager approval. Escalating to human support.” (bot escalates)
  5. Human agent joins → sees full session history instantly
  6. Agent: “I see you ordered 10 items and one was damaged. I’ll process your refund now.”
Without sessions:
  • Human agent asks user to repeat everything
  • User frustrated: “I already told the bot this!”
  • Time wasted re-gathering context
With sessions:
  • Human agent clicks session replay
  • Sees full conversation history in 5 seconds
  • Jumps directly to resolution without asking user to repeat
Benefits:
  • Seamless bot-to-human handoffs
  • Reduced user frustration
  • Faster time to resolution
  • Better support experience

Implementation Guide

Use the @observe() decorator and update the session ID with abv.update_current_trace().Setup:
Basic usage:
Generate session IDs:
Set session IDs when creating spans manually.Example:
Update session ID without direct span reference:
Use updateActiveTrace() to set session IDs.Setup:
Configuration (instrumentation.ts):
Add session IDs:
Wrap existing functions with automatic tracing and session IDs.Example:
Create spans manually and set session IDs on traces.Example:

Related Features

User Tracking

Link traces to user accounts to correlate sessions with specific users for support and analysis

Metadata

Attach structured context to sessions for precise filtering and business analysis

Trace IDs

Use custom trace IDs for distributed tracing across microservices within sessions

Cost Tracking

Calculate total costs per session to optimize pricing and model selection