Muestrear desde un trabajo interactivo

POST

https://api.fastino.ai

training

jobs

{job_id}

sample

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

POST /v1/training/jobs/{job_id}/sample Generates continuations of a tokenized prompt with one of the job’s sampler checkpoints. Sampling also works after the job is stopped. Read the generated tokens from the completed future’s result.sequences.

Request

string

requerido

Your Fastino API key.

string

requerido

Interactive job ID.

string

requerido

The name of a sampler checkpoint saved by this job, or its tinker://<job_id>/sampler_weights/<name> path.

integer[]

requerido

Prompt token IDs from the base model’s tokenizer. At least one token.

integer

predeterminado:"1"

Number of continuations.

object

Decoding settings.

Mostrar properties

integer

Maximum generated tokens. Prompt plus max_tokens must fit the model’s sampling limit.

number

predeterminado:"1.0"

Sampling temperature.

number

predeterminado:"1.0"

Nucleus-sampling probability mass.

integer

predeterminado:"-1"

Top-k cutoff. -1 disables it.

string | string[] | integer[]

Stop strings or stop token IDs.

integer

Sampling seed.

boolean

predeterminado:"false"

Also return the log probability of each prompt token.

integer

predeterminado:"0"

Return the top-k alternatives at each prompt position.

integer

predeterminado:"0"

Return the top-k alternatives at each generated position.

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

Response

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Each sequence in the completed result has tokens, logprobs, stop_reason (stop or length), and topk_logprobs. Decode tokens with the same tokenizer. Cancel an unfinished sample with POST /v1/training/jobs/{job_id}/futures/{future_id}/cancel.

Status	Meaning
`400`	`checkpoint` is invalid, another job’s checkpoint, or a state checkpoint, or the prompt is too long.
`402`	The team is out of credits or has no valid payment setup.
`404`	The job or checkpoint doesn’t exist, or the job belongs to another team.
`422`	`checkpoint` or `prompt_tokens` is missing, or the body has an unknown field.

⌘I

curl -X POST "https://api.fastino.ai/v1/training/jobs/$TRAINING_JOB_ID/sample" \
  -H "X-API-Key: $FASTINO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "checkpoint": "step-000100",
    "prompt_tokens": [151644, 872, 198, 3838],
    "num_samples": 2,
    "sampling_params": {"max_tokens": 64, "temperature": 0.7}
  }'

{
  "future_id": "FUTURE_ID",
  "job_id": "SESSION_ID:train:0",
  "type": "sample"
}

Inferencia

Entrenamiento

Muestrear desde un trabajo interactivo

Request

Response

​Request

​Response

Request

Response