Sample from an interactive job

POST

https://api.fastino.ai

training

jobs

{job_id}

sample

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

POST /v1/training/jobs/{job_id}/sample Generates continuations of a tokenized prompt with one of the job’s sampler checkpoints. Sampling also works after the job is stopped. Read the generated tokens from the completed future’s result.sequences.

Request

string

required

Your Fastino API key.

string

required

Interactive job ID.

string

required

The name of a sampler checkpoint saved by this job, or its tinker://<job_id>/sampler_weights/<name> path.

integer[]

required

Prompt token IDs from the base model’s tokenizer. At least one token.

integer

default:"1"

Number of continuations.

object

Decoding settings.

Show properties

integer

Maximum generated tokens. Prompt plus max_tokens must fit the model’s sampling limit.

number

default:"1.0"

Sampling temperature.

number

default:"1.0"

Nucleus-sampling probability mass.

integer

default:"-1"

Top-k cutoff. -1 disables it.

string | string[] | integer[]

Stop strings or stop token IDs.

integer

Sampling seed.

boolean

default:"false"

Also return the log probability of each prompt token.

integer

default:"0"

Return the top-k alternatives at each prompt position.

integer

default:"0"

Return the top-k alternatives at each generated position.

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

Response

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Each sequence in the completed result has tokens, logprobs, stop_reason (stop or length), and topk_logprobs. Decode tokens with the same tokenizer. Cancel an unfinished sample with POST /v1/training/jobs/{job_id}/futures/{future_id}/cancel.

Status	Meaning
`400`	`checkpoint` is invalid, another job’s checkpoint, or a state checkpoint, or the prompt is too long.
`402`	The team is out of credits or has no valid payment setup.
`404`	The job or checkpoint doesn’t exist, or the job belongs to another team.
`422`	`checkpoint` or `prompt_tokens` is missing, or the body has an unknown field.

⌘I

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Inference

Training

Sample from an interactive job

Request

Response

​Request

​Response

Request

Response