Fandefasana mivantana

POST https://api.shannon-ai.com/v1/chat/completions

Ny SDK OpenAI dia manome anao ny chunk iray isaky ny iray. Vakio ny chunk tsirairay araka izay entiny: sombin'ny reasoning, sombin'ny valiny, tool call, na ny faran'ny valiny.

from openai import OpenAI, APIError

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.shannon-ai.com/v1",
)

stream = client.chat.completions.create(
    model="shannon-3",
    messages=[{"role": "user", "content": "Explain server-sent events in three sentences."}],
    stream=True,
)

answer, reasoning, tool_calls = [], [], []
finish_reason = usage = None

try:
    for chunk in stream:
        if chunk.usage:
            usage = chunk.usage
        if not chunk.choices:
            continue
        choice = chunk.choices[0]
        delta = choice.delta

        thought = getattr(delta, "reasoning_content", None)
        if thought:
            reasoning.append(thought)
        if delta.content:
            answer.append(delta.content)
            print(delta.content, end="", flush=True)
        for call in delta.tool_calls or []:
            tool_calls.append((call.id, call.function.name, call.function.arguments))
        if choice.finish_reason:
            finish_reason = choice.finish_reason
except APIError as error:
    # A failure after the stream started arrives as an error chunk.
    print(f"\nstream failed: {error}")

print(f"\n{finish_reason} {usage}")

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.shannon-ai.com/v1",
});

const stream = await client.chat.completions.create({
  model: "shannon-3",
  messages: [{ role: "user", content: "Explain server-sent events in three sentences." }],
  stream: true,
});

let answer = "";
let reasoning = "";
const toolCalls = [];
let finishReason = null;
let usage = null;

try {
  for await (const chunk of stream) {
    if (chunk.usage) usage = chunk.usage;
    const choice = chunk.choices?.[0];
    if (!choice) continue;
    const delta = choice.delta;

    if (delta.reasoning_content) reasoning += delta.reasoning_content;
    if (delta.content) {
      answer += delta.content;
      process.stdout.write(delta.content);
    }
    for (const call of delta.tool_calls ?? []) {
      toolCalls.push({ id: call.id, name: call.function.name, arguments: call.function.arguments });
    }
    if (choice.finish_reason) finishReason = choice.finish_reason;
  }
} catch (error) {
  // A failure after the stream started arrives as an error chunk.
  console.error("\nstream failed:", error.message);
}

console.log("\n", finishReason, usage);

curl -N https://api.shannon-ai.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "shannon-3",
    "messages": [{"role": "user", "content": "Explain server-sent events in three sentences."}],
    "stream": true
  }'

Ny fomba fanaterana stream

• Ny valiny dia manana status 200 sy ny header content-type: text/event-stream ary cache-control: no-cache.
• Ny event tsirairay dia andalana iray manomboka amin'ny data: ary arahin'ny andalana foana. Ny soratra aorian'ny data: dia chunk JSON iray. Tsy misy andalana event: na id:.
• Ny andalana manomboka amin'ny roa teboka, toy ny : ping, dia fanamarihana izay mitazona misokatra ny fifandraisana. Lavo izany.
• Mifarana amin'ny data: [DONE] ny stream. Ny stream mijanona tsy misy azy dia tsy feno.

Chunk araka ny filaharana

Ny chunk tsirairay dia object chat.completion.chunk:

{
  "id": "chatcmpl-5f0c1e7a9b3d4c62a8e1f07d2b46c9a3",
  "object": "chat.completion.chunk",
  "created": 1791625200,
  "model": "shannon-3",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": "Server-sent"
      },
      "finish_reason": null
    }
  ]
}

Ny chunk rehetra amin'ny stream iray dia mitondra id, created ary model mitovy. Ny choices dia misy entry iray foana. Ny delta-ny dia mitazona izay vaovao, ary null ny finish_reason mandra-pahatongan'ny chunk farany.

Ny reasoning no tonga voalohany, avy eo ny valiny, avy eo ny tool call rehetra. Vakio ny chunk araka ny key ao amin'ny delta-ny, fa tsy araka ny toerany ao anaty stream: ny delta dia mety mitazona role miaraka amin'ny sombintsoratra voalohany, ary foana ny farany.

Stream manontolo araka ny fandehanany. Mba hahafohy ny andalana, ny id, object, created ary model dia tsy asiana ao amin'ny chunk tsirairay eto.

data: {"choices":[{"index":0,"delta":{"reasoning_content":"Three sentences:"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"reasoning_content":" what it is, how it is framed, why it is used."},"finish_reason":null}]}

: ping

data: {"choices":[{"index":0,"delta":{"content":"Server-sent"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"content":" events let a server push text to a client over one HTTP reply."},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":1190,"completion_tokens":84,"total_tokens":1274}}

data: [DONE]

Delta reasoning

Ny model manao reasoning dia mandefa ny reasoning-ny ao amin'ny delta.reasoning_content alohan'ny valiny. Tazony misaraka ny soratra roa: aseho ho ampahany mitokana azo atopy ny reasoning, na lavo izany.

Ny model open-weight hosted dia tsy mandefa reasoning rehefa off ny reasoning_effort, ary tsy mandefa koa rehefa manana response_format ny fangatahana.

Ny shannon-2-lite sy shannon-2-pro koa dia manao reasoning alohan'ny hamaliany. Amin'ity endpoint ity dia mandefa andalana fanamarihana : thinking izy ireo raha manao izany, ary avy eo ny valiny.

Ny reasoning_content dia tsy ao amin'ny typed model an'ny SDK OpenAI. Amin'ny Python vakio amin'ny getattr(delta, "reasoning_content", None); amin'ny JavaScript mandeha araka izy ny delta.reasoning_content.

Delta tool call

Ny tool call dia tonga manontolo: chunk iray isaky ny antso, izay mitazona ny antso miaraka amin'ny string arguments feno ao amin'ny delta-ny. Ny index dia manisa ny antso amin'ny valiny manomboka amin'ny 0. Ny kaody mampiaraka ny arguments ampahany isan-tsombiny dia mandeha tsy miova, satria mampiray sombiny iray ihany.

{
  "tool_calls": [
    {
      "index": 0,
      "id": "call_3d9a7c1e5b2f4a60c8e1d7f09b24a6c5",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\":\"Paris\"}"
      }
    }
  ]
}

Aorian'ny antso, ny chunk farany dia manana finish_reason tool_calls.

Rehefa manana tools ny fangatahana, ny shannon-1.6-lite, shannon-1.6-pro ary shannon-coder-1 dia mandefa ny soratry ny valiny ho delta iray amin'ny faran'ny valiny.

Ny chunk farany

Ny chunk farany alohan'ny data: [DONE] dia manana delta foana, ny finish_reason ary ny usage an'ny fangatahana, ao anaty chunk iray. Ny stream rehetra dia manana azy; tsy mila stream_options ianao.

{
  "id": "chatcmpl-5f0c1e7a9b3d4c62a8e1f07d2b46c9a3",
  "object": "chat.completion.chunk",
  "created": 1791625200,
  "model": "shannon-3",
  "choices": [
    {
      "index": 0,
      "delta": {},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 1190,
    "completion_tokens": 84,
    "total_tokens": 1274
  }
}

Amin'ny model open-weight hosted, ny usage dia manana prompt_tokens_details.cached_tokens sy completion_tokens_details.reasoning_tokens koa.

Aorian'ny fangatahana misy web_search: true izay nahita zavatra ny fikarohany, ny chunk farany dia mitondra sources koa, ary ny chunk iray alohany dia mitondra ny marika fanononana ho delta.annotations. Fikarohana web

Ny saha ao amin'ny usage dia voasoratra miaraka amin'ny endpoint. Chat Completions

Andalana keep-alive

Raha mbola miasa ny model nefa tsy mbola manana zavatra halefa, ny stream dia mitondra andalana fanamarihana. Tsy misy data izany. Ny parser SSE dia mandingana azy ho azy; ny kaody mamaky andalana manta dia tsy maintsy mandingana ny andalana rehetra manomboka amin'ny roa teboka.

Hadisoana ao anaty stream

Rehefa nanomboka ny stream dia 200 ny status-ny, ka ny tsy fahombiazana dia tonga ao anaty stream: chunk misy mpikambana error any amin'ny toerana ahitana ny choices.

data: {"error":{"message":"The model backend failed to answer. Please retry.","type":"api_error","code":null,"param":null}}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":1190,"completion_tokens":12,"total_tokens":1202}}

data: [DONE]

Aorian'ny chunk hadisoana, mbola manaraka ny chunk farany, misy finish_reason sy usage, ary data: [DONE]. Hamarino ny chunk rehetra raha misy error. Raha tsy atao izany dia toa feno ny valiny tsy nahomby tapany.

Ny model open-weight hosted dia mandefa ny mpikambana error misy type sy message ihany.

Ny SDK Python sy JavaScript an'ny OpenAI dia mampiakatra chunk hadisoana ho hadisoana API raha mbola mandalo azy ianao, ka asio ao anaty fikarakarana hadisoana mahazatra ny loop.

Ny code status, ny body hadisoana ary izay hatao retry dia manana pejy manokana. Fitantanana hadisoana

Fanakatonana ny fifandraisana

Ny fanakatonana ny fifandraisana dia manampitsahatra ny fandefasana, fa tsy ny fangatahana. Ny model dia manoratra ny valiny hatramin'ny farany ary vidiana ny fangatahana toy ny hoe novakinao daholo. Raha te-handoa kely kokoa, mangataha kely kokoa: apetraho max_tokens ambany kokoa amin'ny model mampihatra azy.

Ny stream dia mety mifarana aloha ihany koa teny an-dalana mankany aminao, ohatra rehefa tapaka ny tamba-jotra na rehefa havaozina ny server. Mijanona tsy misy chunk farany sy tsy misy data: [DONE] izy. Heverina ho tsy feno io valiny io ary alefaso indray ny fangatahana.

Famakiana stream tsy misy SDK

Ny mpamaky marina dia manao zavatra dimy: mandefa any amin'ny fikarakarana hadisoana ny hadisoana tonga alohan'ny stream, mizara ho andalana ny body, mandingana ny andalana fanamarihana, mijanona amin'ny data: [DONE], ary manamarina ny chunk tsirairay raha misy error.

import json
import requests

response = requests.post(
    "https://api.shannon-ai.com/v1/chat/completions",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "shannon-3",
        "messages": [{"role": "user", "content": "Explain server-sent events in three sentences."}],
        "stream": True,
    },
    stream=True,
)

# 1. Errors before the stream are plain JSON with their own status.
if response.status_code != 200:
    raise RuntimeError(f"{response.status_code}: {response.text}")

answer, done = [], False
for raw in response.iter_lines():          # 2. one line at a time
    line = raw.decode("utf-8")
    if not line.startswith("data: "):      # 3. empty lines and ": ping" comments
        continue
    data = line[len("data: "):]
    if data == "[DONE]":                   # 4. the end of the stream
        done = True
        break
    chunk = json.loads(data)
    if "error" in chunk:                   # 5. a failure inside the stream
        raise RuntimeError(chunk["error"]["message"])
    delta = chunk["choices"][0]["delta"]
    if delta.get("content"):
        answer.append(delta["content"])
    if chunk.get("usage"):
        print(chunk["choices"][0]["finish_reason"], chunk["usage"])

if not done:
    raise RuntimeError("the stream ended before [DONE]")
print("".join(answer))

const response = await fetch("https://api.shannon-ai.com/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "shannon-3",
    messages: [{ role: "user", content: "Explain server-sent events in three sentences." }],
    stream: true,
  }),
});

// 1. Errors before the stream are plain JSON with their own status.
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);

const decoder = new TextDecoder();
let buffer = "";
let answer = "";
let done = false;

read: for await (const bytes of response.body) {
  buffer += decoder.decode(bytes, { stream: true });
  const lines = buffer.split("\n");       // 2. one line at a time
  buffer = lines.pop();                    //    keep the unfinished line
  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;   // 3. empty lines and ": ping" comments
    const data = line.slice(6);
    if (data === "[DONE]") {               // 4. the end of the stream
      done = true;
      break read;
    }
    const chunk = JSON.parse(data);
    if (chunk.error) throw new Error(chunk.error.message);   // 5. a failure inside the stream
    const choice = chunk.choices[0];
    if (choice.delta.content) answer += choice.delta.content;
    if (chunk.usage) console.log(choice.finish_reason, chunk.usage);
  }
}

if (!done) throw new Error("the stream ended before [DONE]");
console.log(answer);

Ny ohatra Python dia mampiasa ny package requests. Ny ohatra JavaScript dia mandeha amin'ny Node.js 18 na vaovao kokoa.

Stream amin'ny endrika hafa

Ny endpoint Responses sy Messages koa dia mandefa stream. Ny frame-ny dia mitondra andalana event: misy ny anaran'ny event sy andalana data: misy ny JSON-ny. Ny pejy tsirairay dia mitanisa ny event araka ny filaharana.