Skip to main content
LLM observability platforms accumulate significant data over time: development traces that pollute production analytics, personally identifiable information (PII) requiring GDPR compliance, erroneously created test data, or entire projects that need removal. Without flexible deletion capabilities, you’re stuck with data you don’t want, compliance violations you can’t fix, and storage costs for unnecessary information.

​
How Data Deletion Works

ABV provides deletion at multiple levels, from individual traces to complete organizations:

Select deletion scope

Choose the appropriate deletion scope for your needs:
  • Single trace: Delete one specific trace (e.g., trace containing PII)
  • Batch of traces: Select multiple traces and delete together
  • Query-based deletion: Delete all traces matching filters (e.g., all traces for user_id=“john”)
  • Project deletion: Remove entire project with all data
  • Organization deletion: Remove organization and all associated data
  • User account deletion: Remove user account from ABV

Execute deletion

Deletion methods vary by scope:
  • UI: Visual selection and deletion for traces, projects, and organizations
  • API: Programmatic deletion for automation and bulk operations
  • Automatic: Scheduled deletion via data retention policies
All deletions create audit log entries showing who deleted what and when.

Deletion processing

Trace deletions: Most trace deletions are asynchronous due to data warehouse optimization. Traces typically disappear within 15 minutes but can take longer during high-volume periods. Rate limiting prevents excessive load.Project/organization deletions: Immediate revocation of API keys, followed by complete data removal within minutes.

Verification and confirmation

No confirmation for successful deletion: ABV doesn’t send completion notifications. Verify deletion by querying the data again—deleted items won’t appear.Deletion is irreversible: Once deleted, data cannot be recovered. Ensure backups exist before deleting critical data.

​
Deleting Traces

Trace deletion removes the trace and all related entities: observations (LLM calls, retrieval steps), scores (evaluations, user feedback), and media assets (images, audio).

​
Single Trace Deletion

​
Via UI

Navigate to traces view

Open the traces table in your project.

Locate the trace

Find the trace you want to delete using filters, search, or by scrolling.

Open action menu

Scroll right to the Action column, click the three dots button for the trace you want to delete.

Confirm deletion

Click Delete and confirm in the dialog. The trace and all related data (observations, scores) will be scheduled for deletion.

​
Via API

Delete a specific trace by ID: 
DELETE https://app.abv.dev/api/public/traces/{traceId}
Authentication: Requires API key with traces:delete permission (Owner or Admin roles). Response: 204 No Content on success. API Reference →

​
Batch Trace Deletion

Delete multiple selected traces in a single operation.

​
Via UI

Select traces

In the traces table, check the boxes next to traces you want to delete. Select individual traces or use “select all on page.”

Open batch actions

Click the Actions dropdown that appears when traces are selected.

Confirm deletion

Select Delete from the dropdown and confirm. All selected traces and related data will be deleted.

​
Via API

Delete multiple traces in a single API call: 
DELETE https://app.abv.dev/api/public/traces
Content-Type: application/json

{
  "traceIds": ["trace-123", "trace-456", "trace-789"]
}
Rate limiting: Batch deletions are rate-limited to prevent excessive load. Large batches may take longer to process. API Reference →

​
Query-Based Deletion

Delete all traces matching specific filters—useful for GDPR compliance (delete all traces for a user) or cleanup (delete all test data).

​
Via UI

Configure filters

In the traces table, configure filters to match the traces you want to delete. For example:
  • User ID equals “john@example.com”
  • Tag contains “test-data”
  • Date range for development period

Select all matching traces

Click the checkbox at the top of the table to select all traces on the current page. A banner appears offering to “select all items matching the filter.”Click the banner to select all traces matching your filter, not just the current page.

Delete all selected

With all matching traces selected, click Actions > Delete and confirm.
Warning: Verify your filters match only the intended traces before confirming deletion. Query-based deletion can affect large numbers of traces.

​
Via API

Delete traces matching query filters programmatically: 
DELETE https://app.abv.dev/api/public/traces
Content-Type: application/json

{
  "filter": {
    "userId": "john@example.com"
  }
}
Use cases:
  • GDPR compliance: Delete all traces for a specific user
  • Test cleanup: Delete all traces with tag “test-data”
  • Time-based cleanup: Delete traces older than a certain date (combine with data retention for automation)

​
Deletion Timing and Limitations

Trace deletions are asynchronous and rate-limited due to data warehouse architecture.
Timeline:
  • Most traces deleted within 15 minutes of deletion request
  • During high deletion volume, processing may take longer
  • No confirmation sent when deletion completes
Rate limiting:
  • ABV limits concurrent deletion operations to protect system performance
  • Large deletion requests queue and process incrementally
  • If you need to delete millions of traces, consider using data retention policies instead
Verification:
  • Query the data again to verify deletion (deleted traces won’t appear in results)
  • Check audit logs to confirm deletion request was logged
Data warehouse optimization: Removing records from analytical storage (data warehouse) is resource-intensive, requiring table reorganization and index updates. Asynchronous processing and rate limiting ensure system stability during large deletions.

​
Deleting Projects

Project deletion removes all data associated with the project: traces, scores, prompts, datasets, evaluations, configurations, and API keys.

Navigate to project settings

Open the project you want to delete and navigate to Project Settings > General.

Scroll to Danger Zone

Scroll to the bottom of the General settings page to find the Danger Zone section.

Delete project

Click Delete Project and confirm the action.

Immediate effects

  • All API keys are immediately revoked (applications stop functioning)
  • Project scheduled for complete deletion within minutes
  • All data irreversibly removed: traces, observations, scores, prompts, datasets, configurations
Project deletion is irreversible. All data is permanently removed and cannot be recovered. Ensure backups exist before deleting projects.
What gets deleted:
  • All traces and observations
  • All scores and evaluation results
  • All prompts and prompt versions
  • All datasets and dataset items
  • All configurations (LLM connections, integrations)
  • All API keys (immediately revoked)
  • All project-specific settings
What persists:
  • Audit logs (immutable, for compliance)
  • Organization-level resources (if the project was part of an organization)
Use cases:
  • Project reached end-of-life (POC concluded, feature sunset)
  • Customer churned (remove their dedicated project in multi-tenant deployment)
  • Security incident (delete compromised project to prevent further access)

​
Deleting Organizations

Organization deletion removes the organization and all associated data. Projects must be deleted first—organizations can only be deleted when empty.

Delete all projects

Before deleting an organization, delete all projects within it. Organizations with projects cannot be deleted.

Navigate to organization settings

Open Organization Settings > General.

Scroll to Danger Zone

Find the Danger Zone section at the bottom of the page.

Delete organization

Click Delete Organization and confirm. The organization and all associated user information will be removed.
What gets deleted:
  • Organization record and settings
  • Organization memberships (user associations with this organization)
  • Organization-scoped API keys
  • Billing information associated with the organization
What persists:
  • User accounts (users remain in ABV, just removed from this organization)
  • Other organizations the users belong to
  • Audit logs of organization activities (for compliance)
Use case: Company closes account, merges with another organization, or consolidated multiple organizations.

​
Deleting User Accounts

User account deletion removes the user from ABV entirely, including all organization memberships and personal data.
User account deletion requires contacting ABV support—it cannot be performed through the UI or API.
To delete a user account: Email support@abv.dev with:
  • Email address of the account to delete
  • Confirmation that you own the account or have authorization to request deletion
  • Organization context (if deleting as part of offboarding)
What gets deleted:
  • User account and authentication credentials
  • Organization and project memberships
  • Personal information (name, email, profile)
  • User-created API keys
What persists:
  • Audit logs showing historical activities (for compliance, but user personally identifiable information is anonymized)
  • Resources created by the user (prompts, traces, datasets) remain attributed to the organization/project, not the individual user
Timeline: Account deletion typically processes within 2-3 business days of support request.

​
Compliance Use Cases

​
GDPR Right to Be Forgotten

Identify user data

Determine which traces, datasets, and resources contain the user’s personal data. Use filters for userId or search for email addresses/names in trace data.

Delete traces via query

Use query-based deletion to remove all traces associated with the user:
  • Filter by userId matching the user’s identifier
  • Select all matching traces
  • Confirm deletion

Remove from datasets

Check datasets for test cases or examples containing the user’s data. Manually review and delete affected dataset items.

Delete user account

Contact support to delete the user’s account from ABV, removing their personal information from the platform.

Document for compliance

Export audit logs showing deletion actions as evidence of GDPR compliance. Retain these logs according to regulatory requirements (typically 3-7 years).
Timeline: Trace deletions complete within 15 minutes (typically), user account deletion within 2-3 business days.

​
HIPAA Data Deletion

Healthcare applications must delete protected health information (PHI) when legally required.

Identify PHI-containing traces

Search for traces containing patient identifiers: medical record numbers, patient names, dates of birth, or other PHI.Recommendation: Tag PHI-containing traces during ingestion for easier identification (e.g., tag: “contains-phi”).

Execute deletion

Use query-based deletion to remove all traces containing the patient’s PHI:
DELETE /api/public/traces
{
  "filter": {"tags": ["patient-123"]}
}

Verify complete removal

Query the data again to confirm no traces with the patient’s identifiers remain.

Document destruction

Export audit logs showing deletion actions and timing for HIPAA compliance documentation. Maintain these logs for minimum 6 years (HIPAA requirement).

​
Development Data Cleanup

Remove test and development data that pollutes production analytics.

Tag test data during creation

Preventive measure: Tag all development and test traces with “environment: development” or “test-data” tags during creation.

Filter and delete

Periodically delete test data:
  • Filter traces by tag “test-data”
  • Select all matching traces
  • Confirm deletion

Automate with retention policies

Set shorter retention periods for development projects, automatically deleting old test data.Learn more about data retention →

​
Common Scenarios

Scenario: User requests deletion under GDPR “right to be forgotten.”Steps:
  1. Identify the user’s identifier (email, userId)
  2. Use query-based deletion to remove all traces: filter by userId, select all matching, delete
  3. Check datasets for any test cases with user data, delete manually
  4. Contact support to delete the user’s account
  5. Export audit logs showing deletions for compliance documentation
Timeline: Traces deleted within 15 minutes, account deletion in 2-3 business days.
Scenario: Development team ran extensive tests in production project, creating noise in analytics.Steps:
  1. Filter traces by tag “test-data” or other identifying marker
  2. Select all matching traces
  3. Delete
  4. Verify deletion by querying again (should return no results)
Prevention: Use separate development project, or tag all test data for easy cleanup later.
Scenario: Developer accidentally logged customer email addresses in trace metadata.Steps:
  1. Identify affected time range (when did the bug exist?)
  2. Filter traces by time range and any identifying markers
  3. Review sample traces to confirm they contain PII
  4. Execute batch or query-based deletion
  5. Fix the code to prevent future PII logging
  6. Document the incident and deletion for compliance/security review
Timeline: Immediate deletion request, complete within 15 minutes.
Scenario: Proof-of-concept completed, project no longer needed.Steps:
  1. Export any valuable data (successful prompts, evaluation results, learnings)
  2. Notify team members that project will be deleted
  3. Navigate to Project Settings > General > Danger Zone
  4. Delete project (immediately revokes API keys, removes all data)
Note: If POC becomes production, transfer project to production organization instead of deleting.
Scenario: Customer cancels subscription, their dedicated project needs deletion.Steps:
  1. Verify customer confirmed cancellation and data deletion
  2. Export customer data if contractually required (for handoff or records)
  3. Delete customer’s project (removes all their traces, prompts, datasets)
  4. Update billing system to stop charges
  5. Document deletion in customer offboarding records
Compliance: Ensure deletion timeline matches contractual obligations (immediate vs. 30-day grace period).

​
Related Topics

Data Retention

Automate data cleanup with retention policies instead of manual deletion

Audit Logs

Track deletion actions for compliance and security monitoring

Export Data

Export data before deletion for backup or compliance requirements

RBAC

Control who can delete data with role-based access controls