Aus einem interaktiven Job samplen

POST

https://api.fastino.ai

training

jobs

{job_id}

sample

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

POST /v1/training/jobs/{job_id}/sample Generates continuations of a tokenized prompt with one of the job’s sampler checkpoints. Sampling also works after the job is stopped. Read the generated tokens from the completed future’s result.sequences.

Request

string

erforderlich

Your Fastino API key.

string

erforderlich

Interactive job ID.

string

erforderlich

The name of a sampler checkpoint saved by this job, or its tinker://<job_id>/sampler_weights/<name> path.

integer[]

erforderlich

Prompt token IDs from the base model’s tokenizer. At least one token.

integer

Standard:"1"

Number of continuations.

object

Decoding settings.

Anzeigen properties

integer

Maximum generated tokens. Prompt plus max_tokens must fit the model’s sampling limit.

number

Standard:"1.0"

Sampling temperature.

number

Standard:"1.0"

Nucleus-sampling probability mass.

integer

Standard:"-1"

Top-k cutoff. -1 disables it.

string | string[] | integer[]

Stop strings or stop token IDs.

integer

Sampling seed.

boolean

Standard:"false"

Also return the log probability of each prompt token.

integer

Standard:"0"

Return the top-k alternatives at each prompt position.

integer

Standard:"0"

Return the top-k alternatives at each generated position.

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

Response

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Each sequence in the completed result has tokens, logprobs, stop_reason (stop or length), and topk_logprobs. Decode tokens with the same tokenizer. Cancel an unfinished sample with POST /v1/training/jobs/{job_id}/futures/{future_id}/cancel.

Status	Meaning
`400`	`checkpoint` is invalid, another job’s checkpoint, or a state checkpoint, or the prompt is too long.
`402`	The team is out of credits or has no valid payment setup.
`404`	The job or checkpoint doesn’t exist, or the job belongs to another team.
`422`	`checkpoint` or `prompt_tokens` is missing, or the body has an unknown field.

⌘I

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Inferenz

Training

Aus einem interaktiven Job samplen

Request

Response

​Request

​Response

Request

Response