> ## Documentation Index
> Fetch the complete documentation index at: https://docs.abv.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Dataset Runs Data Model

This page describes the data model of datasets and dataset runs. For detailed reference, see the [Datasets documentation](/developer/evaluations/datasets).

## Datasets

Datasets are a collection of inputs and, optionally, expected outputs that can be used during Dataset runs.

`Dataset`s are a collection of `DatasetItem`s.

<img src="https://mintlify.s3.us-west-1.amazonaws.com/abv-2be93c70/images/evaluations-dataset-runs-traces.png" alt="Dataset run with traces" />

### Dataset object

| Attribute     | Type   | Required | Description                         |
| ------------- | ------ | -------- | ----------------------------------- |
| `name`        | string | Yes      | Name of the dataset                 |
| `description` | string | No       | Description of the dataset          |
| `metadata`    | object | No       | Additional metadata for the dataset |

### DatasetItem object

| Attribute             | Type          | Required | Description                                                                                                                                                 |
| --------------------- | ------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `datasetName`         | string        | Yes      | Name of the dataset to add the item to                                                                                                                      |
| `input`               | object        | No       | Input data for the dataset item                                                                                                                             |
| `expectedOutput`      | object        | No       | Expected output data for the dataset item                                                                                                                   |
| `metadata`            | object        | No       | Additional metadata for the dataset item                                                                                                                    |
| `sourceTraceId`       | string        | No       | ID of the source trace to link this dataset item to                                                                                                         |
| `sourceObservationId` | string        | No       | ID of the source observation to link this dataset item to                                                                                                   |
| `id`                  | string        | No       | Unique identifier for the dataset item. Dataset items are upserted on their id. Id needs to be unique (project-level) and cannot be reused across datasets. |
| `status`              | DatasetStatus | No       | Status of the dataset item. Defaults to ACTIVE for newly created items. Possible values: `ACTIVE`, `ARCHIVED`                                               |

## DatasetRun

Dataset runs are used to run a dataset through your LLM application and optionally apply evaluation methods to the results.

<img src="https://mintlify.s3.us-west-1.amazonaws.com/abv-2be93c70/images/evaluations-dataset-runs-annotations.png" alt="Dataset run with annotations" />

### DatasetRun object

| Attribute     | Type   | Required | Description             |
| ------------- | ------ | -------- | ----------------------- |
| `datasetName` | string | Yes      | Name of the dataset     |
| `runName`     | string | Yes      | Name of the dataset run |

### DatasetRunItem object

| Attribute        | Type   | Required | Description                                                                                                                                                                |
| ---------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `runName`        | string | Yes      | Name of the dataset run to add the item to                                                                                                                                 |
| `runDescription` | string | No       | Description of the run. If run exists, description will be updated                                                                                                         |
| `metadata`       | object | No       | Metadata of the dataset run, updates run if run already exists                                                                                                             |
| `datasetItemId`  | string | Yes      | ID of the dataset item to link to this run                                                                                                                                 |
| `observationId`  | string | No       | ID of the observation to link to this run                                                                                                                                  |
| `traceId`        | string | No       | ID of the trace to link to this run. traceId should always be provided. For compatibility with older SDK versions it can also be inferred from the provided observationId. |

<Info>
  Most of the time, we recommend that DatasetRunItems reference TraceIDs directly. The reference to ObservationID exists for backwards compatibility with older SDK versions.
</Info>

## End to end data relations

DataSetRuns can combine a few ABV objects:

* `DatasetRuns` are created by looping through all or selected `DatasetItem`s of a `Dataset` with your LLM application.
* For each `DatasetItem` passed into the LLM application as an Input a `DatasetRunItem` & a `Trace` are created.
* Optionally `Score`s can be added to the `Trace`s to evaluate the output of the LLM application during the `DatasetRun`.

```mermaid theme={null}
graph LR
    Dataset[(Dataset)]
    DI1[DatasetItem 1]
    DI2[DatasetItem 2]
    DI3[DatasetItem 3]
    DR[DatasetRun]
    DRI1[DatasetRunItem 1]
    DRI2[DatasetRunItem 2]
    DRI3[DatasetRunItem 3]
    T1[Trace 1]
    T2[Trace 2]
    T3[Trace 3]
    S1[Score]
    S2[Score]
    S3[Score]

    Dataset -->|contains| DI1
    Dataset -->|contains| DI2
    Dataset -->|contains| DI3

    DR -->|references| Dataset
    DR -->|contains| DRI1
    DR -->|contains| DRI2
    DR -->|contains| DRI3

    DRI1 -->|references| DI1
    DRI1 -->|links to| T1
    DRI2 -->|references| DI2
    DRI2 -->|links to| T2
    DRI3 -->|references| DI3
    DRI3 -->|links to| T3

    T1 -.->|evaluated by| S1
    T2 -.->|evaluated by| S2
    T3 -.->|evaluated by| S3

    classDef datasetClass fill:#4fc3f7,stroke:#0288d1,color:#000
    classDef runClass fill:#ffb74d,stroke:#f57c00,color:#000
    classDef itemClass fill:#ba68c8,stroke:#8e24aa,color:#000
    classDef runItemClass fill:#ff8a65,stroke:#e64a19,color:#000
    classDef traceClass fill:#81c784,stroke:#388e3c,color:#000
    classDef scoreClass fill:#fff176,stroke:#f57f17,color:#000

    class Dataset datasetClass
    class DR runClass
    class DI1,DI2,DI3 itemClass
    class DRI1,DRI2,DRI3 runItemClass
    class T1,T2,T3 traceClass
    class S1,S2,S3 scoreClass
```