从交互式作业采样

POST

https://api.fastino.ai

training

jobs

{job_id}

sample

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

POST /v1/training/jobs/{job_id}/sample Generates continuations of a tokenized prompt with one of the job’s sampler checkpoints. Sampling also works after the job is stopped. Read the generated tokens from the completed future’s result.sequences.

Request

string

必填

Your Fastino API key.

string

必填

Interactive job ID.

string

必填

The name of a sampler checkpoint saved by this job, or its tinker://<job_id>/sampler_weights/<name> path.

integer[]

必填

Prompt token IDs from the base model’s tokenizer. At least one token.

integer

默认值:"1"

Number of continuations.

object

Decoding settings.

显示 properties

integer

Maximum generated tokens. Prompt plus max_tokens must fit the model’s sampling limit.

number

默认值:"1.0"

Sampling temperature.

number

默认值:"1.0"

Nucleus-sampling probability mass.

integer

默认值:"-1"

Top-k cutoff. -1 disables it.

string | string[] | integer[]

Stop strings or stop token IDs.

integer

Sampling seed.

boolean

默认值:"false"

Also return the log probability of each prompt token.

integer

默认值:"0"

Return the top-k alternatives at each prompt position.

integer

默认值:"0"

Return the top-k alternatives at each generated position.

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

Response

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Each sequence in the completed result has tokens, logprobs, stop_reason (stop or length), and topk_logprobs. Decode tokens with the same tokenizer. Cancel an unfinished sample with POST /v1/training/jobs/{job_id}/futures/{future_id}/cancel.

Status	Meaning
`400`	`checkpoint` is invalid, another job’s checkpoint, or a state checkpoint, or the prompt is too long.
`402`	The team is out of credits or has no valid payment setup.
`404`	The job or checkpoint doesn’t exist, or the job belongs to another team.
`422`	`checkpoint` or `prompt_tokens` is missing, or the body has an unknown field.

⌘I

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

推理

训练

Request

Response

​Request

​Response

Request

Response