Skip to main content
Enterprise LLM observability platforms serve multiple teams with different access needs. Your security team needs read-only visibility across all projects, engineering needs full control over their projects but no access to finance’s cost analysis project, and external contractors need limited access to specific datasets without seeing sensitive production data.

​
How RBAC Works in ABV

ABV implements role-based access control through organizations, projects, and roles: Hierarchy overview:
  • Organizations contain users and projects
  • Users inherit organization-level roles by default
  • Project-level roles can override organization defaults for specific projects
  • API Keys are scoped to either a project or the entire organization

Organizations contain projects

Organizations are the top-level entities in ABV’s hierarchy. They contain projects, manage billing, and provide default role assignments for all members.All users in an organization receive a default role (Owner, Admin, Member, Viewer, or None) that applies across all projects unless overridden by project-level assignments.

Projects group observability data

Projects contain all your LLM observability data: traces, scores, prompts, datasets, and configurations. Projects enable fine-grained access control by isolating data and permissions.Common patterns: separate projects for production/staging environments, different teams or product lines, different customers in multi-tenant deployments, or sensitive workloads requiring restricted access.

Roles define permissions

Five roles provide increasing levels of access:
  • Owner: Complete control including billing and organization deletion
  • Admin: Project configuration and user management (cannot delete organization)
  • Member: Create and modify observability data (cannot change settings)
  • Viewer: Read-only access to data and dashboards
  • None: No default access (used when users need access to specific projects only)
Each role maps to specific permission scopes controlling what actions users can perform.

Project-level overrides provide flexibility

Users inherit their organization-level role by default, but you can assign project-specific roles to override this default.Example: A user with organization role “Viewer” can be assigned “Admin” on their team’s project, giving them full control where needed while maintaining restricted access to other projects.To grant access to only certain projects, set the organization role to “None” and assign project-specific roles.

API keys enable programmatic access

API keys authenticate programmatic access to ABV’s API. They’re scoped to a specific project and inherit that project’s data access, but are not tied to individual users.Create separate API keys for different systems (CI/CD, billing, integrations) to maintain clear audit trails and enable independent key rotation without affecting other services.

​
Roles and Permission Scopes

​
Role Overview

Complete organizational control. Unrestricted access to all resources including billing, organization deletion, and project transfers.For: Founders, executives, designated administratorsCapabilities: Manage billing, delete organization, transfer projects, manage members, plus all Admin/Member/Viewer permissions
Project configuration and user management without billing access.For: Team leads, engineering managers, technical administratorsCapabilities: Create projects, manage members, manage API keys, configure integrations, plus all Member/Viewer permissionsCannot: Manage billing, delete organization
Create and modify observability data without configuration access.For: Engineers, data scientists, product managersCapabilities: Create/modify traces, scores, datasets, prompts, evaluations; run experiments; export data; create dashboardsCannot: Change project settings, manage users, create API keys, configure integrations
Read-only access for monitoring and auditing.For: Security teams, auditors, executives, external stakeholdersCapabilities: View traces, scores, dashboards, metrics, configurations, prompts, evaluation templates, audit logsCannot: Modify data, export data, create resources, change settings
No default access. Use for project-specific assignments only.For: Contractors, external collaborators, employees working on specific projectsUsage: Set organization role to “None”, then assign project-specific roles (Viewer/Member/Admin) as needed

​
Detailed Permission Scopes

Permissions available at the organization level:
PermissionOwnerAdminMemberViewerNone
Manage billing (create/update billing)✓
Delete organization✓
Transfer projects between organizations✓✓
Create projects✓✓
Manage organization API keys (CRUD)✓✓
Update organization settings✓✓
Manage organization members (create/update/delete)✓✓
Read organization members✓✓✓
Permissions available at the project level:
PermissionOwnerAdminMemberViewerNone
Delete project✓
Update project settings✓✓
Read project settings✓✓✓✓
Manage project members (CUD)✓✓
Read project members✓✓✓
Manage API keys (CUD)✓✓
Read API keys✓✓✓
Read audit logs✓✓
Manage LLM connections (CRUD)✓✓
Read LLM connections✓✓✓✓
Manage integrations (CRUD)✓✓
Manage automations (CUD)✓✓
Read automations✓✓✓✓
Delete traces✓✓
Tag/bookmark/publish objects✓✓✓
Manage datasets (CUD)✓✓✓
Manage prompts (CUD)✓✓✓
Read prompts✓✓✓✓
Manage prompt experiments (CUD)✓✓✓
Read prompt experiments✓✓✓✓
Manage evaluations (CUD)✓✓✓
Read evaluations✓✓✓✓
Manage scores (CUD)✓✓✓
Read score configs✓✓✓✓
Manage annotation queues (CUD)✓✓✓
Read annotation queues✓✓✓✓
Manage dashboards (CUD)✓✓✓
Read dashboards✓✓✓✓
Create batch exports✓✓✓
Read batch exports✓✓✓
Manage comments (CUD)✓✓✓
Read comments✓✓✓✓
CUD = Create, Update, Delete

​
Managing Users

​
Adding Users to an Organization

Navigate to organization settings

Access organization settings from the dropdown in the top navigation bar.

Invite users by email

Enter the user’s email address and assign them an organization-level role (Owner, Admin, Member, Viewer, or None).Users receive an email notification and can access the organization after logging in. Users without existing ABV accounts appear as “pending invites” until they sign up.

User accepts invitation

New users sign up through the invitation email, existing users see the organization immediately after accepting the invite.

​
Changing User Roles

Any user with the organizationMembers:CUD or projectMembers:CUD permission can modify user roles. Organization-level role changes:
  • Affect the user’s default permissions across all projects
  • Can only assign roles equal to or lower than your own role
  • Takes effect immediately across the organization
Project-level role changes:
  • Override the organization-level role for specific projects
  • Useful for granting elevated access where needed or restricting access to sensitive projects

​
Managing Projects

​
Creating Projects

Verify permissions

Ensure you have the projects:create permission (available to Owners and Admins).

Create new project

Navigate to the projects section and create a new project within your ABV organization.Choose a descriptive name reflecting the project’s purpose: environment (staging/production), team (engineering/data science), customer (for multi-tenant deployments), or use case (chatbot/analysis).

Configure initial settings

Set up project-specific configurations: data retention policies, LLM connections, integrations, and initial team member access.

​
Transferring Projects Between Organizations

Only users with the projects:transfer_organization permission can transfer projects. What gets transferred:
  • All project data (traces, scores, datasets, prompts)
  • All project configurations and settings
  • API keys (remain valid without interruption)
  • All features continue working during and after transfer
What changes:
  • Access control shifts to the new organization’s role assignments
  • Users from the original organization lose access unless they’re members of the new organization
  • Billing moves to the new organization
Data integrity: No data is lost during transfer. The project remains fully operational throughout the process.

​
Project-Level Roles

By default, users inherit their organization-level role across all projects. Project-level roles provide fine-grained control when you need different permissions for different projects.

​
When to Use Project-Level Roles

Different teams need different access levels within the same organization.Example: Engineering team members get Admin access on their projects but Viewer access on the finance team’s cost analysis project. Finance team gets Admin on their project but None on engineering projects.
Restrict production access while keeping staging open.Example: Set organization role to Member, then assign Admin roles only on staging projects. Production projects use the default Member role, allowing engineers to view production data but requiring senior engineers (with project-level Admin) to modify production configurations.
Grant contractors or partners access to specific projects without organization-wide visibility.Example: Set contractor organization role to None, then assign Member or Viewer roles only on the projects they’re working on. They see only what they need, nothing more.
Isolate customer data in dedicated projects with restricted access.Example: Each customer gets a dedicated project. Customer success managers get Viewer access to all customer projects, but each customer’s integration uses a project-scoped API key that only accesses their specific project.

​
How Project-Level Roles Work

Inheritance: Users start with their organization-level role across all projects. Override: Assign a project-specific role to override the organization default for that project only. Independence: Project-level roles don’t affect other projects or the organization-level role. Complete isolation: Setting organization role to “None” with project-specific assignments ensures users only see the projects you’ve explicitly granted access to.

​
API Keys and Programmatic Access

API keys provide programmatic access to ABV’s API for integrations, automation, and CI/CD pipelines.

​
API Key Characteristics

Each API key is associated with a specific project and can only access that project’s data.For cross-project access, use organization-scoped API keys (available to Owners and Admins) that can manage projects, users, and organization settings.
API keys exist independently of user accounts. They remain valid when team members leave, are reassigned, or change roles.This independence provides stability for automated systems and clearer audit trails showing programmatic access separately from user actions.
Create separate API keys for different purposes:
  • Tracing SDK integration (write traces, read prompts)
  • Billing system (read-only metrics access)
  • CI/CD pipeline (write traces to staging project only)
  • External integrations (scoped to specific resources)
Rotate or revoke keys independently without affecting other systems.
Audit logs distinguish between user actions and API key actions, showing which API key performed each operation.This separation makes it easy to track automated system behavior separately from human actions during incident investigation or compliance auditing.

​
Managing API Keys

Creating API keys:
  • Requires apiKeys:CUD permission (Owner and Admin roles)
  • Specify the project the key should access
  • Optionally name the key to indicate its purpose (“CI/CD Pipeline”, “Billing Integration”, etc.)
Rotating API keys:
  • Create a new API key for the same project
  • Update your systems to use the new key
  • Delete the old key after confirming the new key works
Revoking access:
  • Delete an API key to immediately revoke its access
  • The key becomes invalid instantly across all ABV systems
Learn more about API authentication →

​
Related Topics

Audit Logs

Track all access and modifications for security monitoring and compliance

Data Retention

Configure automatic data deletion policies for compliance and storage management

SCIM & Organization API

Automate user provisioning and project management via SCIM and organization APIs

Public API Authentication

Learn how to use API keys for programmatic access to ABV