> ## Documentation Index
> Fetch the complete documentation index at: https://docs.abv.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Cost Tracking Reference

This reference guide covers advanced topics: configuring custom models, understanding tokenizers, handling reasoning models, and using the Public API to manage model definitions.

## Supported Tokenizers

ABV uses tokenizers to infer usage when token counts aren't provided. The following tokenizers are currently supported:

| Model Pattern | Tokenizer     | Package                                                                          | Notes                                              |
| ------------- | ------------- | -------------------------------------------------------------------------------- | -------------------------------------------------- |
| `gpt-4o*`     | `o200k_base`  | Built-in                                                                         | Used for GPT-4o models                             |
| `gpt*`        | `cl100k_base` | Built-in                                                                         | Used for GPT-4, GPT-3.5, and other GPT models      |
| `claude*`     | `claude`      | [@anthropic-ai/tokenizer](https://www.npmjs.com/package/@anthropic-ai/tokenizer) | Not fully accurate for Claude models per Anthropic |

<Warning>
  According to Anthropic, their tokenizer is not fully accurate for Claude models. For best results, ingest token counts directly from the Anthropic API response.
</Warning>

## Predefined Models

ABV maintains a list of predefined popular models with tokenizers and pricing for:

* **OpenAI** - GPT-4, GPT-3.5, GPT-4o, and other models
* **Anthropic** - Claude 4 family (Opus, Sonnet, Haiku)
* **Google** - Gemini models

<Tip>
  View the complete list of predefined models in the ABV UI (sign-in required). Navigate to **Settings → Model Definitions**.
</Tip>

## Custom Model Definitions

You can add your own model definitions for:

* Custom or proprietary models
* Fine-tuned models
* Models not yet supported by ABV
* Overriding default pricing

### Creating Models via the UI

1. Navigate to **Settings → Model Definitions** in the ABV UI
2. Click **Add Model**
3. Configure the model settings (see configuration details below)
4. Save the model definition

### Creating Models via the API

Use the Models API to programmatically manage model definitions:

```bash theme={null}
# List all models
GET /api/public/models

# Create a new model
POST /api/public/models

# Get a specific model
GET /api/public/models/{id}

# Delete a model
DELETE /api/public/models/{id}
```

[View the complete Public API documentation](/developer/platform/api-data-platform/public-api)

## Model Configuration

### Model Matching

Models are matched to generations using regular expressions:

| Generation Attribute | Model Attribute | Description                        |
| -------------------- | --------------- | ---------------------------------- |
| `model`              | `match_pattern` | Regex pattern to match model names |

**Example patterns:**

```regex theme={null}
# Exact match for GPT-4
(?i)^(gpt-4-0125-preview)$

# Match all GPT-4 variants
(?i)^gpt-4.*$

# Match custom model family
(?i)^my-custom-model-.*$
```

<Info>
  User-defined models take priority over ABV-maintained models. This allows you to override default pricing or add support for new models.
</Info>

### Pricing Configuration

Define pricing per usage type. Usage types must match exactly with the keys in the `usage_details` object:

```json theme={null}
{
  "pricing": {
    "input": 0.00001,           // Price per input token in USD
    "output": 0.00003,          // Price per output token in USD
    "cached_tokens": 0.000001   // Price per cached token in USD
  }
}
```

**Usage type matching:**

* Usage types are case-sensitive
* Must match exactly: `"input"` ≠ `"Input"`
* Supports arbitrary usage types for custom metrics

### Tokenization Configuration

For models using the `openai` tokenizer, specify the tokenization config:

```json theme={null}
{
  "tokenizer": "openai",
  "tokenization_config": {
    "tokenizerModel": "gpt-4-0125-preview",  // tiktoken model name
    "tokensPerMessage": 4,                   // Required for chat models
    "tokensPerName": -1                      // Required for chat models
  }
}
```

<Tip>
  Copy tokenization config from predefined OpenAI models in the UI. See the [OpenAI tokenization guide](https://platform.openai.com/docs/guides/text-generation) for more details on `tokensPerMessage` and `tokensPerName`.
</Tip>

## Complete Model Definition Example

Here's a complete example of a custom model definition:

<Tabs>
  <Tab title="JSON">
    ```json theme={null}
    {
      "name": "My Custom GPT-4 Model",
      "match_pattern": "(?i)^my-custom-gpt-4$",
      "tokenizer": "openai",
      "tokenization_config": {
        "tokenizerModel": "gpt-4-0125-preview",
        "tokensPerMessage": 4,
        "tokensPerName": -1
      },
      "pricing": {
        "input": 0.00001,
        "output": 0.00003
      }
    }
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl -X POST "https://app.abv.dev/api/public/models" \
      -H "Authorization: Bearer sk-abv-..." \
      -H "Content-Type: application/json" \
      -d '{
        "name": "My Custom GPT-4 Model",
        "match_pattern": "(?i)^my-custom-gpt-4$",
        "tokenizer": "openai",
        "tokenization_config": {
          "tokenizerModel": "gpt-4-0125-preview",
          "tokensPerMessage": 4,
          "tokensPerName": -1
        },
        "pricing": {
          "input": 0.00001,
          "output": 0.00003
        }
      }'
    ```
  </Tab>
</Tabs>

## Reasoning Models

Reasoning models (like OpenAI's o1 family) require special handling.

### Why Inference Doesn't Work

Reasoning models take multiple internal steps to arrive at a response:

1. Model generates reasoning tokens (internal thought process)
2. Model generates completion tokens (final response)
3. API bills you for: reasoning tokens + completion tokens

**The problem:** ABV cannot see the internal reasoning tokens, so it cannot accurately infer costs by tokenizing only the visible input and output.

### Solution: Always Ingest Usage

For reasoning models, you **must** ingest usage details from the API response:

```python theme={null}
# Python example for OpenAI o1
response = openai_client.chat.completions.create(
    model="o1-preview",
    messages=[{"role": "user", "content": "Solve this problem..."}]
)

generation.update(
    usage_details={
        "prompt_tokens": response.usage.prompt_tokens,
        "completion_tokens": response.usage.completion_tokens,
        "completion_tokens_details": {
            "reasoning_tokens": response.usage.completion_tokens_details.reasoning_tokens
        }
    }
)
```

<Warning>
  **Critical:** If you don't ingest usage for reasoning models, ABV cannot calculate accurate costs. The tokenized input + output will significantly underestimate actual usage.
</Warning>

[Learn more about OpenAI reasoning models](https://platform.openai.com/docs/guides/reasoning)

## Historical Data and Model Changes

### Retroactive Application

ABV does **not** retroactively apply model definition changes to historical generations. This includes:

* Price updates
* New tokenizers
* Changed model patterns

**Why:** Costs are calculated at ingestion time using the model and price information available at that moment. This ensures historical data remains accurate and auditable.

### Batch Reprocessing

If you need to apply new model definitions to existing generations:

1. Contact ABV support
2. Request a batch job to reprocess generations
3. Specify the time range and model changes

<Note>
  Batch reprocessing is typically used when:

  * Correcting a pricing error
  * Adding missing model definitions
  * Migrating to new tokenizers
</Note>

## OpenAI Usage Schema Compatibility

ABV supports the OpenAI usage schema for compatibility with OpenAI SDKs and tools.

### Automatic Mapping

When you use OpenAI field names, ABV automatically maps them:

| OpenAI Field                  | ABV Field  |
| ----------------------------- | ---------- |
| `prompt_tokens`               | `input`    |
| `completion_tokens`           | `output`   |
| `total_tokens`                | `total`    |
| `prompt_tokens_details.*`     | `input_*`  |
| `completion_tokens_details.*` | `output_*` |

**Example:**

```python theme={null}
# You send this (OpenAI schema):
usage_details={
    "prompt_tokens": 100,
    "completion_tokens": 50,
    "prompt_tokens_details": {
        "cached_tokens": 20
    }
}

# ABV stores this:
{
    "input": 100,
    "output": 50,
    "input_cached_tokens": 20,
    "total": 150  # automatically calculated
}
```

This makes it easy to integrate with OpenAI without changing your existing code.

## Requesting Official Model Support

Don't want to maintain custom model definitions? Request official support:

1. Visit the [ABV Model Support Request Form](https://smart-forms.saasjet.com/external?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbGllbnRLZXkiOiJlZmZmN2JhYy0xMzZlLTNkOTUtOGIxYi1hMWQ3NWNkZWM3NGYiLCJpYXQiOjE3NTY4NDc1MDc0MDl9.Q33DbNtKAfZvsY0qs4hOfTBAzLic-VPhTrIusjkOMtI)
2. Provide model details and documentation
3. ABV will add official support in a future release

Official support includes:

* Automatic tokenization
* Pricing updates
* Maintenance and updates

## Troubleshooting

### Usage and Cost Missing for Historical Data

**Problem:** After adding a new model definition, historical generations still show no usage or cost.

**Solution:** Model definitions are not applied retroactively. Options:

1. Request a batch reprocessing job from support
2. Accept that historical data uses old definitions
3. For future data: Ingest usage directly instead of relying on inference

### Incorrect Costs

**Problem:** Costs don't match your LLM provider's billing.

**Checklist:**

1. **Check pricing:** Verify the model definition pricing matches your provider's current rates
2. **Check usage types:** Ensure usage type keys match exactly (case-sensitive)
3. **Check model matching:** Verify the regex pattern correctly matches your model name
4. **Consider timing:** Prices are applied at ingestion time, not retroactively

**Best practice:** For billing-critical applications, always ingest both usage AND cost from the provider's API response.

### Model Not Matching

**Problem:** Generations aren't matching your custom model definition.

**Debug steps:**

1. Check the `model` field in your generation matches the `match_pattern` regex
2. Test the regex pattern in a regex tester
3. Verify user-defined model isn't being overridden by ABV model
4. Check for typos in the model name

**Example:**

```javascript theme={null}
// Generation model
"model": "gpt-4-custom-v1"

// Match pattern (correct)
"match_pattern": "(?i)^gpt-4-custom-v1$"

// Match pattern (incorrect - won't match)
"match_pattern": "^gpt-4-custom-v1$"  // missing (?i) flag
```

### Tokenization Errors

**Problem:** Tokenization failing or producing unexpected results.

**Solutions:**

1. **For Claude models:** Ingest token counts from API instead of inferring
2. **For OpenAI models:** Verify `tokensPerMessage` and `tokensPerName` are set correctly
3. **For custom tokenizers:** Ensure the tokenizer is supported (see supported tokenizers table)

## API Reference

### Models API Endpoints

**List all models:**

```bash theme={null}
GET /api/public/models
```

**Create a model:**

```bash theme={null}
POST /api/public/models
Content-Type: application/json

{
  "name": "string",
  "match_pattern": "string",
  "tokenizer": "string",
  "tokenization_config": {},
  "pricing": {}
}
```

**Get a specific model:**

```bash theme={null}
GET /api/public/models/{id}
```

**Delete a model:**

```bash theme={null}
DELETE /api/public/models/{id}
```

[View complete API documentation](/developer/platform/api-data-platform/public-api)

## Next Steps

<CardGroup cols={2}>
  <Card title="Implementation Guide" icon="code" href="/developer/basic-features/cost-tracking-implementation">
    Return to implementation examples
  </Card>

  <Card title="Daily Metrics API" icon="chart-line" href="/developer/platform/metrics/metrics-api">
    Export usage and cost data
  </Card>
</CardGroup>