API ReferenceBenchmarks
Run Benchmark
Execute a benchmark and list available files
Run Benchmark
POST
/api/v1/benchmark/{schemaId}/run-benchmarkASYNCStarts a benchmark execution against a model. The benchmark re-extracts data from files whose rows are all accepted and compares the results against the ground truth. Addressing a paused or disabled deployment is refused with 409 DEPLOYMENT_PAUSED.
Parameters
| Name | Type | In | Required | Description |
|---|---|---|---|---|
schemaId | string | path | Yes | Addresses the model: its ID, the ID of a deployment that points at it, or its extraction schema ID. |
benchmarkRunName | string | body | Yes | Name for this benchmark run |
benchmarkRunDescription | string | body | Yes | Description of the benchmark run |
selectedFileIds | string[] | body | No | Specific file IDs to benchmark (defaults to all accepted files) |
configurationOverride | object | body | No | Override extraction configuration for this run |
Request
const res = await fetch(
"https://app.limai.io/api/v1/benchmark/es_abc123/run-benchmark",
{
method: "POST",
headers: {
Authorization: "Bearer YOUR_API_TOKEN",
"Content-Type": "application/json",
},
body: JSON.stringify({
benchmarkRunName: "Baseline v1",
benchmarkRunDescription: "Initial benchmark with default settings",
}),
}
);Response
Response200
{
"success": true,
"benchmarkExecutionId": "exec_abc123",
"deploymentId": "dep_001",
"modelId": "dep_001",
"viaDeploymentId": null,
"fileCount": 25
}List Available Files
GET
/api/v1/benchmark/{schemaId}/list-filesLists the reviewed files of a model, each with the number of accepted rows it has and whether it is in the example pool.
Parameters
| Name | Type | In | Required | Description |
|---|---|---|---|---|
schemaId | string | path | Yes | Addresses the model: its ID, the ID of a deployment that points at it, or its extraction schema ID. |
Request
const res = await fetch(
"https://app.limai.io/api/v1/benchmark/es_abc123/list-files",
{
headers: {
Authorization: "Bearer YOUR_API_TOKEN",
},
}
);
const files = await res.json();Response
Response200
{
"files": [
{
"id": "file_001",
"name": "invoice_001.pdf",
"createdAt": "2025-01-15T10:00:00.000Z",
"acceptedRowCount": 3,
"tableCount": 1,
"isTrainingData": false
}
],
"totalCount": 1,
"modelId": "dep_001",
"viaDeploymentId": null
}