Up to 7.2× the speed of standard llama.cpp Runs 100% on your PC
Free for personal use
Download free
Local API Guide/Endpoints/Text completions

Text completions

POST/v1/completions
Counts as a Local API promptBase URL http://localhost:1234/v1
Local API Guide contents

Overview

The legacy completions API: the model continues prompt exactly as written, with no chat template, system prompt or identity added. Use it for autocomplete-style tasks and your own prompt formats. For questions and instructions, chat completions give much better answers.

Request body

ParameterDescription
promptstring | [string]requiredThe text to continue (a list with a single string also works).
modelstringOptional, as for chat completions.
max_tokensintegerThe longest continuation, in tokens. Default: until the model stops or the context is full.
stopstring | string[]Up to 16 strings that end the continuation.
response_formatobjectStructured output, as in OpenAI's API: {"type": "json_object"} for any JSON object, or {"type": "json_schema", "json_schema": {"name": "…", "schema": {…}}} for JSON matching your schema. The answer is always valid JSON (unless max_tokens cuts it short). {"type": "text"} is the default. See Structured output.
streambooleanServer-sent events of text_completion objects with choices[0].text pieces, then data: [DONE].
stream_options.include_usagebooleanUsage in an extra last chunk.
temperature, top_p, top_k, min_p, seed, …numberSampling, as in chat completions (greedy without them).
Authorization headerAuthorization: Bearer <key>, only when Require API key is on in the app.

web_search and knowledge are not available here (400 unsupported_extension).

Response

FieldDescription
idstringcmpl-…
objectstringtext_completion
choices[0].textstringThe continuation.
choices[0].finish_reasonstringstop, length or cancelled.
usage, timingsobjectAs for chat completions.

Example

curl http://localhost:1234/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3.6-35b-a3b", "prompt": "def fibonacci(n):\n", "max_tokens": 120, "stop": ["\n\n\n"]}'
Response
{
  "id": "cmpl-0b6e3f2a91c84d7e9a5b1c2d",
  "object": "text_completion",
  "created": 1792390000,
  "model": "qwen3.6-35b-a3b",
  "choices": [{ "index": 0, "text": "    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a + b\n    return a", "finish_reason": "stop" }],
  "usage": { "prompt_tokens": 6, "completion_tokens": 31, "total_tokens": 37, "prompt_tokens_details": { "cached_tokens": 0 } }
}

Errors

StatusCodeWhen
401invalid_api_keyRequire API key is on and the key is missing or wrong.
400invalid_jsonThe body is not a JSON object.
400model_not_loadedThe request names the other bundled model. 5090Ai runs one model at a time: switch in the app, or leave model out.
400(none)A parameter is invalid, for example max_tokens below 1 or a prompt longer than the context.
402trial_exhaustedToday's Local API allowance is used up (type insufficient_quota). See Daily limits and plans.
503model_not_loadedNo model is loaded, or it is still loading (retry in a few seconds).
502engine_unavailableThe engine did not respond (for example while the app restarts it).
400unsupported_extensionweb_search or knowledge was sent (chat completions only).