Skip to main content
POST
/
evaluation
Evaluation
curl --request POST \
  --url https://inference-service-433968519479.us-central1.run.app/evaluation \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "hf_token": "<string>",
  "model_source": "<string>",
  "base_model_id": "<string>",
  "dataset_id": "<string>",
  "metrics": [],
  "max_samples": 123,
  "num_sample_results": 3,
  "use_vllm": false
}
'
{
  "metrics": {},
  "num_samples": 123,
  "dataset_id": "<string>",
  "samples": [
    {
      "prediction": "<string>",
      "reference": "<string>",
      "sample_index": 123,
      "input": [
        {}
      ]
    }
  ]
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
​
hf_token
string
required
​
model_source
string
required
​
model_type
enum<string>
required
Available options:
adapter,
merged,
base
​
base_model_id
string
required
​
dataset_id
string
required
​
task_type
enum<string> | null
Available options:
conversation,
qa,
summarization,
translation,
classification,
general
​
metrics
enum<string>[] | null
Available options:
rouge,
bertscore,
accuracy,
exact_match,
bleu,
meteor,
recall,
precision,
f1
​
max_samples
integer | null
​
num_sample_results
integer | null
default:3
​
use_vllm
boolean | null
default:false

Response

Successful Response

​
metrics
Metrics · object
required
​
num_samples
integer
required
​
dataset_id
string
required
​
samples
SampleResult · object[]
required