LLM observability platforms accumulate significant data over time: development traces that pollute production analytics, personally identifiable information (PII) requiring GDPR compliance, erroneously created test data, or entire projects that need removal. Without flexible deletion capabilities, you’re stuck with data you don’t want, compliance violations you can’t fix, and storage costs for unnecessary information.

How Data Deletion Works

ABV provides deletion at multiple levels, from individual traces to complete organizations:

Select deletion scope

Choose the appropriate deletion scope for your needs:

Single trace: Delete one specific trace (e.g., trace containing PII)
Batch of traces: Select multiple traces and delete together
Query-based deletion: Delete all traces matching filters (e.g., all traces for user_id=“john”)
Project deletion: Remove entire project with all data
Organization deletion: Remove organization and all associated data
User account deletion: Remove user account from ABV

Execute deletion

Deletion methods vary by scope:

UI: Visual selection and deletion for traces, projects, and organizations
API: Programmatic deletion for automation and bulk operations
Automatic: Scheduled deletion via data retention policies

All deletions create audit log entries showing who deleted what and when.

Deletion processing

Trace deletions: Most trace deletions are asynchronous due to data warehouse optimization. Traces typically disappear within 15 minutes but can take longer during high-volume periods. Rate limiting prevents excessive load.Project/organization deletions: Immediate revocation of API keys, followed by complete data removal within minutes.

Verification and confirmation

No confirmation for successful deletion: ABV doesn’t send completion notifications. Verify deletion by querying the data again—deleted items won’t appear.Deletion is irreversible: Once deleted, data cannot be recovered. Ensure backups exist before deleting critical data.

Deleting Traces

Trace deletion removes the trace and all related entities: observations (LLM calls, retrieval steps), scores (evaluations, user feedback), and media assets (images, audio).

Single Trace Deletion

Via UI

Navigate to traces view

Open the traces table in your project.

Locate the trace

Find the trace you want to delete using filters, search, or by scrolling.

Open action menu

Scroll right to the Action column, click the three dots button for the trace you want to delete.

Confirm deletion

Click Delete and confirm in the dialog. The trace and all related data (observations, scores) will be scheduled for deletion.

Via API

Delete a specific trace by ID:

DELETE https://app.abv.dev/api/public/traces/{traceId}

Authentication: Requires API key with traces:delete permission (Owner or Admin roles). Response: 204 No Content on success. API Reference →

Batch Trace Deletion

Delete multiple selected traces in a single operation.

Via UI

Select traces

In the traces table, check the boxes next to traces you want to delete. Select individual traces or use “select all on page.”

Open batch actions

Click the Actions dropdown that appears when traces are selected.

Confirm deletion

Select Delete from the dropdown and confirm. All selected traces and related data will be deleted.

Via API

Delete multiple traces in a single API call:

DELETE https://app.abv.dev/api/public/traces
Content-Type: application/json

{
  "traceIds": ["trace-123", "trace-456", "trace-789"]
}

Rate limiting: Batch deletions are rate-limited to prevent excessive load. Large batches may take longer to process. API Reference →

Query-Based Deletion

Delete all traces matching specific filters—useful for GDPR compliance (delete all traces for a user) or cleanup (delete all test data).

Via UI

Configure filters

In the traces table, configure filters to match the traces you want to delete. For example:

User ID equals “john@example.com”
Tag contains “test-data”
Date range for development period

Select all matching traces

Click the checkbox at the top of the table to select all traces on the current page. A banner appears offering to “select all items matching the filter.”Click the banner to select all traces matching your filter, not just the current page.

Delete all selected

With all matching traces selected, click Actions > Delete and confirm.

Warning: Verify your filters match only the intended traces before confirming deletion. Query-based deletion can affect large numbers of traces.

Via API

Delete traces matching query filters programmatically:

DELETE https://app.abv.dev/api/public/traces
Content-Type: application/json

{
  "filter": {
    "userId": "john@example.com"
  }
}

Use cases:

GDPR compliance: Delete all traces for a specific user
Test cleanup: Delete all traces with tag “test-data”
Time-based cleanup: Delete traces older than a certain date (combine with data retention for automation)

Deletion Timing and Limitations

Trace deletions are asynchronous and rate-limited due to data warehouse architecture.

Timeline:

Most traces deleted within 15 minutes of deletion request
During high deletion volume, processing may take longer
No confirmation sent when deletion completes

Rate limiting:

ABV limits concurrent deletion operations to protect system performance
Large deletion requests queue and process incrementally
If you need to delete millions of traces, consider using data retention policies instead

Verification:

Query the data again to verify deletion (deleted traces won’t appear in results)
Check audit logs to confirm deletion request was logged

Data warehouse optimization: Removing records from analytical storage (data warehouse) is resource-intensive, requiring table reorganization and index updates. Asynchronous processing and rate limiting ensure system stability during large deletions.

Deleting Projects

Project deletion removes all data associated with the project: traces, scores, prompts, datasets, evaluations, configurations, and API keys.

Navigate to project settings

Open the project you want to delete and navigate to Project Settings > General.

Scroll to Danger Zone

Scroll to the bottom of the General settings page to find the Danger Zone section.

Delete project

Click Delete Project and confirm the action.

Immediate effects

All API keys are immediately revoked (applications stop functioning)
Project scheduled for complete deletion within minutes
All data irreversibly removed: traces, observations, scores, prompts, datasets, configurations

Project deletion is irreversible. All data is permanently removed and cannot be recovered. Ensure backups exist before deleting projects.

What gets deleted:

All traces and observations
All scores and evaluation results
All prompts and prompt versions
All datasets and dataset items
All configurations (LLM connections, integrations)
All API keys (immediately revoked)
All project-specific settings

What persists:

Audit logs (immutable, for compliance)
Organization-level resources (if the project was part of an organization)

Use cases:

Project reached end-of-life (POC concluded, feature sunset)
Customer churned (remove their dedicated project in multi-tenant deployment)
Security incident (delete compromised project to prevent further access)

Deleting Organizations

Organization deletion removes the organization and all associated data. Projects must be deleted first—organizations can only be deleted when empty.

Delete all projects

Before deleting an organization, delete all projects within it. Organizations with projects cannot be deleted.

Navigate to organization settings

Open Organization Settings > General.

Scroll to Danger Zone

Find the Danger Zone section at the bottom of the page.

Delete organization

Click Delete Organization and confirm. The organization and all associated user information will be removed.

What gets deleted:

Organization record and settings
Organization memberships (user associations with this organization)
Organization-scoped API keys
Billing information associated with the organization

What persists:

User accounts (users remain in ABV, just removed from this organization)
Other organizations the users belong to
Audit logs of organization activities (for compliance)

Use case: Company closes account, merges with another organization, or consolidated multiple organizations.

Deleting User Accounts

User account deletion removes the user from ABV entirely, including all organization memberships and personal data.

User account deletion requires contacting ABV support—it cannot be performed through the UI or API.

To delete a user account: Email support@abv.dev with:

Email address of the account to delete
Confirmation that you own the account or have authorization to request deletion
Organization context (if deleting as part of offboarding)

What gets deleted:

User account and authentication credentials
Organization and project memberships
Personal information (name, email, profile)
User-created API keys

What persists:

Audit logs showing historical activities (for compliance, but user personally identifiable information is anonymized)
Resources created by the user (prompts, traces, datasets) remain attributed to the organization/project, not the individual user

Timeline: Account deletion typically processes within 2-3 business days of support request.

Compliance Use Cases

Identify user data

Determine which traces, datasets, and resources contain the user’s personal data. Use filters for userId or search for email addresses/names in trace data.

Delete traces via query

Use query-based deletion to remove all traces associated with the user:

Filter by userId matching the user’s identifier
Select all matching traces
Confirm deletion

Remove from datasets

Check datasets for test cases or examples containing the user’s data. Manually review and delete affected dataset items.

Delete user account

Contact support to delete the user’s account from ABV, removing their personal information from the platform.

Document for compliance

Export audit logs showing deletion actions as evidence of GDPR compliance. Retain these logs according to regulatory requirements (typically 3-7 years).

Timeline: Trace deletions complete within 15 minutes (typically), user account deletion within 2-3 business days.

HIPAA Data Deletion

Healthcare applications must delete protected health information (PHI) when legally required.

Identify PHI-containing traces

Search for traces containing patient identifiers: medical record numbers, patient names, dates of birth, or other PHI.Recommendation: Tag PHI-containing traces during ingestion for easier identification (e.g., tag: “contains-phi”).

Execute deletion

Use query-based deletion to remove all traces containing the patient’s PHI:

DELETE /api/public/traces
{
  "filter": {"tags": ["patient-123"]}
}

Verify complete removal

Query the data again to confirm no traces with the patient’s identifiers remain.

Document destruction

Export audit logs showing deletion actions and timing for HIPAA compliance documentation. Maintain these logs for minimum 6 years (HIPAA requirement).

Development Data Cleanup

Remove test and development data that pollutes production analytics.

Tag test data during creation

Preventive measure: Tag all development and test traces with “environment: development” or “test-data” tags during creation.

Filter and delete

Periodically delete test data:

Filter traces by tag “test-data”
Select all matching traces
Confirm deletion

Automate with retention policies

Set shorter retention periods for development projects, automatically deleting old test data.Learn more about data retention →

Common Scenarios

Delete All Data for a User (GDPR)

Clean Up Test Data

Scenario: Development team ran extensive tests in production project, creating noise in analytics.Steps:

Filter traces by tag “test-data” or other identifying marker
Select all matching traces
Delete
Verify deletion by querying again (should return no results)

Prevention: Use separate development project, or tag all test data for easy cleanup later.

Remove Accidentally Exposed PII

Scenario: Developer accidentally logged customer email addresses in trace metadata.Steps:

Identify affected time range (when did the bug exist?)
Filter traces by time range and any identifying markers
Review sample traces to confirm they contain PII
Execute batch or query-based deletion
Fix the code to prevent future PII logging
Document the incident and deletion for compliance/security review

Timeline: Immediate deletion request, complete within 15 minutes.

Delete POC Project After Completion

Scenario: Proof-of-concept completed, project no longer needed.Steps:

Export any valuable data (successful prompts, evaluation results, learnings)
Notify team members that project will be deleted
Navigate to Project Settings > General > Danger Zone
Delete project (immediately revokes API keys, removes all data)

Note: If POC becomes production, transfer project to production organization instead of deleting.

Customer Churn in Multi-Tenant Deployment

Scenario: Customer cancels subscription, their dedicated project needs deletion.Steps:

Verify customer confirmed cancellation and data deletion
Export customer data if contractually required (for handoff or records)
Delete customer’s project (removes all their traces, prompts, datasets)
Update billing system to stop charges
Document deletion in customer offboarding records

Compliance: Ensure deletion timeline matches contractual obligations (immediate vs. 30-day grace period).

Data Retention

Automate data cleanup with retention policies instead of manual deletion

Audit Logs

Track deletion actions for compliance and security monitoring

Export Data

Export data before deletion for backup or compliance requirements

RBAC

Control who can delete data with role-based access controls

​How Data Deletion Works

​Deleting Traces

​Single Trace Deletion

​Via UI

​Via API

​Batch Trace Deletion

​Via UI

​Via API

​Query-Based Deletion

​Via UI

​Via API

​Deletion Timing and Limitations

​Deleting Projects

​Deleting Organizations

​Deleting User Accounts

​Compliance Use Cases

​GDPR Right to Be Forgotten

​HIPAA Data Deletion

​Development Data Cleanup

​Common Scenarios

​Related Topics

Data Retention

Audit Logs

Export Data

RBAC

How Data Deletion Works

Deleting Traces

Single Trace Deletion

Via UI

Via API

Batch Trace Deletion

Via UI

Via API

Query-Based Deletion

Via UI

Via API

Deletion Timing and Limitations

Deleting Projects

Deleting Organizations

Deleting User Accounts

Compliance Use Cases

GDPR Right to Be Forgotten

HIPAA Data Deletion

Development Data Cleanup

Common Scenarios

Related Topics