Échantillonner depuis un job interactif

POST

https://api.fastino.ai

training

jobs

{job_id}

sample

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

POST /v1/training/jobs/{job_id}/sample Generates continuations of a tokenized prompt with one of the job’s sampler checkpoints. Sampling also works after the job is stopped. Read the generated tokens from the completed future’s result.sequences.

Request

string

requis

Your Fastino API key.

string

requis

Interactive job ID.

string

requis

The name of a sampler checkpoint saved by this job, or its tinker://<job_id>/sampler_weights/<name> path.

integer[]

requis

Prompt token IDs from the base model’s tokenizer. At least one token.

integer

défaut:"1"

Number of continuations.

object

Decoding settings.

Afficher properties

integer

Maximum generated tokens. Prompt plus max_tokens must fit the model’s sampling limit.

number

défaut:"1.0"

Sampling temperature.

number

défaut:"1.0"

Nucleus-sampling probability mass.

integer

défaut:"-1"

Top-k cutoff. -1 disables it.

string | string[] | integer[]

Stop strings or stop token IDs.

integer

Sampling seed.

boolean

défaut:"false"

Also return the log probability of each prompt token.

integer

défaut:"0"

Return the top-k alternatives at each prompt position.

integer

défaut:"0"

Return the top-k alternatives at each generated position.

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

Response

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Each sequence in the completed result has tokens, logprobs, stop_reason (stop or length), and topk_logprobs. Decode tokens with the same tokenizer. Cancel an unfinished sample with POST /v1/training/jobs/{job_id}/futures/{future_id}/cancel.

Status	Meaning
`400`	`checkpoint` is invalid, another job’s checkpoint, or a state checkpoint, or the prompt is too long.
`402`	The team is out of credits or has no valid payment setup.
`404`	The job or checkpoint doesn’t exist, or the job belongs to another team.
`422`	`checkpoint` or `prompt_tokens` is missing, or the body has an unknown field.

⌘I

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Inférence

Entraînement

Échantillonner depuis un job interactif

Request

Response

​Request

​Response

Request

Response