Chat Completions

Use the OpenAI Chat Completions format with the Ertas AI Gateway: streaming, tool calls, JSON mode and strict JSON Schema output, and token usage.

Ask AI about this page

On this page

The gateway accepts the OpenAI Chat Completions format at POST /v1/chat/completions. Any OpenAI SDK, or a framework that can point at an OpenAI-compatible base URL, works with the base URL and key changed.

Streaming

Set stream: true to receive the reply as server-sent events, the same way the OpenAI API sends them.

stream = client.chat.completions.create(
    model="ertas-ai/medertas-27b",
    messages=[{"role": "user", "content": "Summarise the CHA2DS2-VASc score in two lines."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Tool calls

Pass tools and the model can answer with tool_calls. Run the tool, append its result as a tool message, and call again.

tools = [{
    "type": "function",
    "function": {
        "name": "calculate_bmi",
        "description": "Body mass index from weight and height",
        "parameters": {
            "type": "object",
            "properties": {
                "weight_kg": {"type": "number"},
                "height_m": {"type": "number"},
            },
            "required": ["weight_kg", "height_m"],
        },
    },
}]

resp = client.chat.completions.create(
    model="ertas-ai/medertas-27b",
    messages=[{"role": "user", "content": "BMI for 82 kg and 1.78 m?"}],
    tools=tools,
)
print(resp.choices[0].message.tool_calls)

Structured output

Both JSON modes work through response_format:

JSON mode, {"type": "json_object"}, returns valid JSON. Say in the prompt which fields you want.
Strict JSON Schema, {"type": "json_schema", "json_schema": {..., "strict": true}}, returns JSON that follows your schema, including nullable fields.

resp = client.chat.completions.create(
    model="ertas-ai/medertas-31b",
    messages=[{"role": "user", "content": "Extract the vitals: BP 156/94, pulse 82."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "vitals",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {"bp": {"type": "string"}, "hr": {"type": "integer"}},
                "required": ["bp", "hr"],
                "additionalProperties": False,
            },
        },
    },
)
print(resp.choices[0].message.content)

Validate the JSON in your app before you rely on it, as you would with any model.

Multi-turn conversations

Chat Completions is stateless: send the earlier turns in messages on every API call.

Token usage and cost

Each reply carries a usage object with prompt_tokens, completion_tokens and cached tokens. Setting max_tokens caps the reply and lowers the credits set aside while the API call runs. See Credits and usage.