Skip to main content
Enterprise security and compliance requirements demand complete visibility into who accessed what data, when changes occurred, and what those changes were. Without comprehensive audit trails, investigating security incidents becomes guesswork, compliance audits require manual record reconstruction, and insider threats go undetected.

​
How Audit Logs Work

ABV’s audit logging system automatically captures comprehensive activity records:

Automatic capture of all activities

ABV logs all significant system activities without requiring configuration. Every API call, UI action, and automated process that modifies resources generates an audit log entry.Captured information includes: user or API key that performed the action, exact action taken (create, update, delete), timestamp with millisecond precision, organization and project context, and complete before/after state for modifications.

Immutable storage

Audit logs are immutable—once created, they cannot be modified or deleted. This immutability ensures audit trails remain trustworthy for security investigations and compliance audits.Even users with Owner permissions cannot alter or delete audit logs. Logs persist according to your data retention policy, with export options for long-term archival.

Access control and viewing

Audit log access requires the auditLogs:read permission, typically granted to Owner and Admin roles. This restricted access ensures sensitive activity records are only visible to authorized personnel.View audit logs through the ABV UI with filtering by time period, project, action type, and user. Pagination handles large audit trails efficiently.

Export for external analysis

Export audit logs directly from the UI for external analysis, archival, or integration with SIEM systems (Splunk, DataDog, Elastic).Exported logs include all captured fields in structured format (JSON or CSV), enabling automated processing, compliance reporting, and long-term retention outside ABV.

​
What Gets Logged

​
Auditable Resources and Actions

ABV tracks specific actions across all system resources. The following table shows exactly what gets logged:
ResourceLogged Actions
Tracebookmark, delete, publish, updateTags
Sessionbookmark, publish
Scorecreate, delete, update
Score Configcreate, update
Datasetcreate, delete, update
Dataset Itemcreate, delete, update
Dataset Rundelete
Commentcreate, delete
Why this matters: Track data modifications for compliance (GDPR deletion requests), investigate quality regressions (which scores changed when?), and audit data access patterns.
ResourceLogged Actions
Promptcreate, delete, promote, setLabel, update, updateTags
Prompt Protected Labelcreate
Why this matters: Understand prompt evolution (who changed the production prompt?), investigate quality regressions (what changed between version 5 and 6?), and enforce deployment controls (who promoted to production without approval?).
ResourceLogged Actions
Evaluation Templatecreate
Job (Evaluation Jobs)create, delete, update
Annotation Queuecreate, delete, update
Annotation Queue Itemcomplete, create, delete
Why this matters: Track evaluation workflow changes, audit human annotation activities, and investigate evaluation result discrepancies.
ResourceLogged Actions
Organizationcreate, delete, update
Projectcreate, delete, transfer, update
Organization Membershipcreate, delete, update
Project Membershipcreate, delete, update
Membership Invitationcreate, delete
API Keycreate, delete, update
LLM API Keycreate, delete
Why this matters: Critical for security monitoring (who created API keys?), compliance audits (demonstrate access control), and incident investigation (when did the compromised user gain Admin access?).
ResourceLogged Actions
Modelcreate, delete, update
Batch Exportcreate
Batch Actioncreate, delete
Blob Storage Integrationupdate
PostHog Integrationdelete, update
Why this matters: Track configuration changes affecting system behavior, audit integration modifications, and investigate unexpected behavior caused by configuration changes.
ResourceLogged Actions
Stripe Checkout Sessioncreate
Why this matters: Audit subscription changes, track billing modifications, and demonstrate financial transaction records for accounting.

​
Log Entry Structure

​
What Each Log Entry Contains

Every log entry identifies the actor who performed the action:User actions (USER type):
  • User ID and email address
  • User’s organizational and project roles at the time of action
  • Authentication method used
API key actions (API_KEY type):
  • API key ID and name
  • Project or organization the key is scoped to
  • Key creation date and creator
Why this matters: Distinguish between human and automated actions, track individual accountability, and identify compromised credentials.
Each log entry specifies exactly what happened:
  • Resource type: Trace, Prompt, Project, API Key, etc.
  • Action: create, update, delete, or resource-specific actions (bookmark, publish, transfer)
  • Resource ID: Unique identifier of the affected resource
  • Resource name: Human-readable name when available
For update operations, logs capture:
  • Before state: Complete resource state prior to modification (JSON)
  • After state: Complete resource state after modification (JSON)
This complete state capture enables precise understanding of what changed—essential for debugging regressions and compliance audits.
Each entry includes contextual information:
  • Timestamp: Millisecond-precision UTC timestamp
  • Organization ID and name
  • Project ID and name (for project-scoped actions)
  • IP address (when available)
  • User agent (for UI actions)
Use case: Filter logs by project to see only production changes, search by time range to investigate incidents, or track geographic access patterns via IP addresses.

​
Using Audit Logs

​
Viewing Logs in the UI

Access audit logs

Navigate to project settings and select the Audit Logs section. Requires auditLogs:read permission (Owner and Admin roles).

Filter and search

Use filters to narrow results:
  • Time range: Last 24 hours, last 7 days, last 30 days, or custom range
  • Project: View logs for specific projects or all projects in the organization
  • Action type: Filter by create, update, delete, or specific actions
  • User: Show actions by specific users or API keys
Pagination handles large audit trails efficiently—logs load incrementally as you scroll.

Examine entry details

Click any log entry to see full details: complete before/after states for updates, all contextual information, and related resource links.For update actions, view side-by-side diff showing exactly what changed.

​
Exporting Audit Logs

Export logs for external analysis, archival, or SIEM integration:

Configure filters

Set filters to select the logs you want to export (time range, project, action type).

Click export button

Click the export button in the top-right corner of the audit logs table.

Choose format and download

Select export format:
  • JSON: Full fidelity with all fields and nested structures (recommended for programmatic analysis)
  • CSV: Flattened format for spreadsheet analysis
Download begins immediately. Large exports may take time to generate.

Integrate with external systems

Import exported logs into SIEM systems (Splunk, DataDog, Elastic), data warehouses (Snowflake, BigQuery), or compliance management platforms.

​
Common Use Cases

Investigate suspicious API activity by filtering logs around the incident timeframe, searching for compromised credentials, and reviewing all actions performed by that identity.Audit log value: Complete activity timeline, before/after states, IP addresses, blast radius analysis
Export logs for the audit period, filter to deletion actions, verify proper authorization, and generate compliance reports showing who deleted what and when.Audit log value: Immutable records, access control enforcement proof, complete attribution, exportable evidence
Filter logs by resource type and time range to find unexpected configuration changes. Review before/after states, identify who made the change, and roll back if needed.Audit log value: Complete change history with diffs, timestamp correlation, attribution
Monitor logs for unusual patterns: off-hours access, bulk exports, privilege escalation, unauthorized project access. Export to SIEM systems for automated anomaly detection.Patterns to watch: Off-hours activity, bulk data exports, unexpected role changes, unauthorized access
Filter logs for production modifications and cross-reference with change approval tickets to verify compliance with change management policies.Integration: Export logs to change management systems (Jira, ServiceNow) for automated verification
Filter logs by user ID to review all actions over the last 30-90 days. Check for suspicious patterns before offboarding.Audit log value: Complete activity history, exportable reports for HR and security teams

​
Security and Retention

Audit logs cannot be modified or deleted—even by users with Owner permissions. This immutability ensures logs remain trustworthy for security investigations and compliance audits.Technical implementation: Logs are written to append-only storage with cryptographic verification preventing tampering.
Only users with auditLogs:read permission can view audit logs. This permission is typically granted to Owner and Admin roles.Best practice: Limit audit log access to security personnel and senior administrators. Regular users and Members should not have access to audit logs.Learn more about RBAC →
Audit logs follow your project’s data retention policy. Configure retention periods based on compliance requirements (SOC 2 typically requires 1-2 years, HIPAA requires 6 years).Recommendation: Export and archive audit logs to external storage for long-term retention beyond ABV’s retention period. This is especially important for compliance requirements exceeding your configured retention.Learn more about data retention →
Export audit logs regularly for:
  • Long-term archival: Compliance requirements often exceed online retention periods
  • SIEM integration: Feed logs into security monitoring systems
  • Compliance reporting: Generate audit reports for regulators
  • Disaster recovery: Maintain backup copies of critical audit records
Automation: Schedule regular exports via API for automated archival workflows.

​
Integration with External Systems

​
SIEM Integration

Integrate audit logs with Security Information and Event Management (SIEM) systems for automated monitoring:

Schedule regular exports

Use ABV’s API to automate daily or hourly audit log exports in JSON format.

Ingest into SIEM

Configure your SIEM (Splunk, DataDog, Elastic) to ingest exported audit logs. Parse JSON structure to extract fields for indexing.

Create detection rules

Define rules for suspicious patterns:
  • Multiple failed authentication attempts
  • Unusual API key usage patterns
  • Off-hours access to production projects
  • Bulk data exports
  • Privilege escalation (role changes)

Configure alerts

Set up alerts to notify security teams when detection rules trigger. Integrate with incident response workflows (PagerDuty, Slack).

​
Compliance Management Platforms

Export audit logs for compliance management platforms (Vanta, Drata, SecureFrame):
  1. Scheduled exports: Weekly exports covering all audit activities
  2. Evidence collection: Logs demonstrating access control enforcement, data modification tracking, and security monitoring
  3. Automated compliance: Platforms analyze logs to verify compliance controls are working correctly

​
Related Topics

Role-Based Access Controls

Control who can view audit logs and perform auditable actions

Data Retention

Configure how long audit logs and other data are retained in ABV

Public API

Automate audit log exports via API for SIEM integration

Data Deletion

Understand what actions trigger deletion audit logs for compliance