Rere

POST https://api.shannon-ai.com/v1/chat/completions

Ka hoatu e ngā SDK OpenAI ngā chunk ki a koe ia tētahi. Pānuitia ia chunk mā tāna e kawe ana: he wāhanga reasoning, he wāhanga whakautu, he karanga tool, te mutunga rānei o te whakautu.

from openai import OpenAI, APIError

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.shannon-ai.com/v1",
)

stream = client.chat.completions.create(
    model="shannon-3",
    messages=[{"role": "user", "content": "Explain server-sent events in three sentences."}],
    stream=True,
)

answer, reasoning, tool_calls = [], [], []
finish_reason = usage = None

try:
    for chunk in stream:
        if chunk.usage:
            usage = chunk.usage
        if not chunk.choices:
            continue
        choice = chunk.choices[0]
        delta = choice.delta

        thought = getattr(delta, "reasoning_content", None)
        if thought:
            reasoning.append(thought)
        if delta.content:
            answer.append(delta.content)
            print(delta.content, end="", flush=True)
        for call in delta.tool_calls or []:
            tool_calls.append((call.id, call.function.name, call.function.arguments))
        if choice.finish_reason:
            finish_reason = choice.finish_reason
except APIError as error:
    # A failure after the stream started arrives as an error chunk.
    print(f"\nstream failed: {error}")

print(f"\n{finish_reason} {usage}")

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.shannon-ai.com/v1",
});

const stream = await client.chat.completions.create({
  model: "shannon-3",
  messages: [{ role: "user", content: "Explain server-sent events in three sentences." }],
  stream: true,
});

let answer = "";
let reasoning = "";
const toolCalls = [];
let finishReason = null;
let usage = null;

try {
  for await (const chunk of stream) {
    if (chunk.usage) usage = chunk.usage;
    const choice = chunk.choices?.[0];
    if (!choice) continue;
    const delta = choice.delta;

    if (delta.reasoning_content) reasoning += delta.reasoning_content;
    if (delta.content) {
      answer += delta.content;
      process.stdout.write(delta.content);
    }
    for (const call of delta.tool_calls ?? []) {
      toolCalls.push({ id: call.id, name: call.function.name, arguments: call.function.arguments });
    }
    if (choice.finish_reason) finishReason = choice.finish_reason;
  }
} catch (error) {
  // A failure after the stream started arrives as an error chunk.
  console.error("\nstream failed:", error.message);
}

console.log("\n", finishReason, usage);

curl -N https://api.shannon-ai.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "shannon-3",
    "messages": [{"role": "user", "content": "Explain server-sent events in three sentences."}],
    "stream": true
  }'

Te āhua o te frame o tētahi stream

• He tūnga 200 tō te whakautu me ngā header content-type: text/event-stream me cache-control: no-cache.
• He rārangi kotahi ia kaupapa ka tīmata ki te data: ā, ka whai ko tētahi rārangi kau. Ko te kuputuhi i muri i te data: he chunk JSON kotahi. Kāore he rārangi event: he id: rānei.
• He kōrero tētahi rārangi ka tīmata ki te tohu rua kikini, pēnei i te : ping, e pupuri ana i te hononga tuwhera. Whakarerea.
• Ka mutu te stream me te data: [DONE]. He kāore i oti tētahi stream ka tū me te kore ia.

Ngā chunk i te raupapa

He ahanoa chat.completion.chunk ia chunk:

{
  "id": "chatcmpl-5f0c1e7a9b3d4c62a8e1f07d2b46c9a3",
  "object": "chat.completion.chunk",
  "created": 1791625200,
  "model": "shannon-3",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": "Server-sent"
      },
      "finish_reason": null
    }
  ]
}

Ka kawe ngā chunk katoa o tētahi stream i te id, created me te model ōrite. Kotahi te urunga i te choices i ngā wā katoa. Kei tōna delta te mea hou, ā, ko null te finish_reason tae noa ki te chunk whakamutunga.

Ka tae mai te reasoning i te tuatahi, kātahi te whakautu, kātahi ētahi karanga tool. Pānuitia tētahi chunk mā ngā kī i tōna delta, kaua mā tōna wāhi i te stream: ka taea e tētahi delta te kawe i te role i te taha o te wāhanga kuputuhi tuatahi, ā, kau te mea whakamutunga.

He stream katoa i tana haere. Kia poto ai ngā rārangi, kua waihohia i konei te id, object, created me te model i ia chunk.

data: {"choices":[{"index":0,"delta":{"reasoning_content":"Three sentences:"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"reasoning_content":" what it is, how it is framed, why it is used."},"finish_reason":null}]}

: ping

data: {"choices":[{"index":0,"delta":{"content":"Server-sent"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"content":" events let a server push text to a client over one HTTP reply."},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":1190,"completion_tokens":84,"total_tokens":1274}}

data: [DONE]

Ngā delta reasoning

Ka tuku tētahi model e whakaaro ana i tana reasoning i te delta.reasoning_content i mua i te whakautu. Wehea ngā kuputuhi e rua: whakaatuhia te reasoning hei wāhanga motuhake ka taea te kopi, waihohia rānei.

Kāore ngā model open-weight hosted e tuku i tētahi reasoning ina he off te reasoning_effort, ā, kāore hoki ina he response_format tā te tono.

Ka whakaaro hoki te shannon-2-lite me te shannon-2-pro i mua i tā rāua whakautu. I runga i tēnei endpoint ka tuku rāua i ngā rārangi kōrero : thinking i a rāua e mahi ana, kātahi te whakautu.

Kāore te reasoning_content i roto i ngā model momo a ngā SDK OpenAI. I Python pānuitia mā te getattr(delta, "reasoning_content", None); i JavaScript ka mahi te delta.reasoning_content pēnei i a ia.

Ngā delta karanga tool

Ka tae mai tētahi karanga tool katoa: kotahi chunk mō ia karanga, kei tōna delta te karanga me te aho arguments katoa. Ka tau te index i ngā karanga o te whakautu mai i te 0. Ka mahi tonu te waehere e hono ana i te arguments wāhanga ki te wāhanga, nā te mea ka hono ia i te wāhanga kotahi.

{
  "tool_calls": [
    {
      "index": 0,
      "id": "call_3d9a7c1e5b2f4a60c8e1d7f09b24a6c5",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\":\"Paris\"}"
      }
    }
  ]
}

I muri i ngā karanga, kei te chunk whakamutunga te finish_reason tool_calls.

Ina he tools tā te tono, ka tuku te shannon-1.6-lite, shannon-1.6-pro me te shannon-coder-1 i te kuputuhi whakautu hei delta kotahi i te mutunga o te whakautu.

Te chunk whakamutunga

Kei te chunk whakamutunga i mua i te data: [DONE] tētahi delta kau, te finish_reason me te usage o te tono, i te chunk kotahi. Kei ia stream; kāore koe e hiahia ki te stream_options.

{
  "id": "chatcmpl-5f0c1e7a9b3d4c62a8e1f07d2b46c9a3",
  "object": "chat.completion.chunk",
  "created": 1791625200,
  "model": "shannon-3",
  "choices": [
    {
      "index": 0,
      "delta": {},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 1190,
    "completion_tokens": 84,
    "total_tokens": 1274
  }
}

I runga i ngā model open-weight hosted kei te usage hoki te prompt_tokens_details.cached_tokens me te completion_tokens_details.reasoning_tokens.

I muri i tētahi tono me te web_search: true i kitea he mea e tana rapu, kei te chunk whakamutunga hoki te sources, ā, kei te chunk kotahi i mua i a ia ngā tohu whakahua hei delta.annotations. Rapu paetukutuku

Kua whakaahuatia ngā āpure o te usage me te endpoint. Chat Completions

Ngā rārangi keep-alive

I a te model e mahi ana ā, kāore anō kia whai mea hei tuku, ka kawe te stream i ngā rārangi kōrero. Kāore he data i roto. Ka whakarere tētahi kaitātari SSE i a rātou mōna ake; me whakarere e te waehere e pānui ana i ngā rārangi mata ia rārangi ka tīmata ki te tohu rua kikini.

Ngā hapa i roto i tētahi stream

Ina tīmata tētahi stream ko 200 tōna tūnga, nō reira ka tae mai tētahi rahua i roto i te stream: he chunk me tētahi mema error i te wāhi e noho ai te choices.

data: {"error":{"message":"The model backend failed to answer. Please retry.","type":"api_error","code":null,"param":null}}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":1190,"completion_tokens":12,"total_tokens":1202}}

data: [DONE]

I muri i te chunk hapa ka whai tonu te chunk whakamutunga, me te finish_reason me te usage, me te data: [DONE]. Tirohia ia chunk mō te error. Ki te kore tērā tirohanga, ka rite tētahi whakautu i rahua i waenganui ki te mea kua oti.

Ka tuku ngā model open-weight hosted i te mema error me te type me te message anake.

Ka whakaara ngā SDK OpenAI Python me JavaScript i tētahi chunk hapa hei hapa API i a koe e tukutuku ana, nō reira takaia te loop ki tō whakahaere hapa noa.

Kei tōna ake whārangi ngā waehere tūnga, te tinana hapa me te mea hei whakamātau anō. Whakahaere hapa

Te kati i te hononga

Ka whakamutu te kati i te hononga i te tuku, kaua i te tono. Ka tuhi te model i te whakautu ki tōna mutunga, ā, ka utua te tono me te mea kua pānuitia katoatia e koe. Kia iti ake te utu, tonoa he iti ake: tautuhia he max_tokens iti ake i runga i tētahi model e whakamahi ana i a ia.

Ka taea hoki te mutu wawe o tētahi stream i tana haere ki a koe, hei tauira ina pakaru te whatunga, i a tētahi tūmau e whakahōutia ana rānei. Ka mutu ia me te kore chunk whakamutunga me te kore data: [DONE]. Whakaarohia taua whakautu he kāore i oti, ā, tukuna anō te tono.

Te pānui i tētahi stream me te kore SDK

E rima ngā mea ka mahia e te kaipānui tika: ka tuku i ngā hapa ka tae mai i mua i te stream ki tana whakahaere hapa, ka wehe i te tinana ki ngā rārangi, ka whakarere i ngā rārangi kōrero, ka tū i te data: [DONE], ā, ka tirotiro i ia chunk mō te error.

import json
import requests

response = requests.post(
    "https://api.shannon-ai.com/v1/chat/completions",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "shannon-3",
        "messages": [{"role": "user", "content": "Explain server-sent events in three sentences."}],
        "stream": True,
    },
    stream=True,
)

# 1. Errors before the stream are plain JSON with their own status.
if response.status_code != 200:
    raise RuntimeError(f"{response.status_code}: {response.text}")

answer, done = [], False
for raw in response.iter_lines():          # 2. one line at a time
    line = raw.decode("utf-8")
    if not line.startswith("data: "):      # 3. empty lines and ": ping" comments
        continue
    data = line[len("data: "):]
    if data == "[DONE]":                   # 4. the end of the stream
        done = True
        break
    chunk = json.loads(data)
    if "error" in chunk:                   # 5. a failure inside the stream
        raise RuntimeError(chunk["error"]["message"])
    delta = chunk["choices"][0]["delta"]
    if delta.get("content"):
        answer.append(delta["content"])
    if chunk.get("usage"):
        print(chunk["choices"][0]["finish_reason"], chunk["usage"])

if not done:
    raise RuntimeError("the stream ended before [DONE]")
print("".join(answer))

const response = await fetch("https://api.shannon-ai.com/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "shannon-3",
    messages: [{ role: "user", content: "Explain server-sent events in three sentences." }],
    stream: true,
  }),
});

// 1. Errors before the stream are plain JSON with their own status.
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);

const decoder = new TextDecoder();
let buffer = "";
let answer = "";
let done = false;

read: for await (const bytes of response.body) {
  buffer += decoder.decode(bytes, { stream: true });
  const lines = buffer.split("\n");       // 2. one line at a time
  buffer = lines.pop();                    //    keep the unfinished line
  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;   // 3. empty lines and ": ping" comments
    const data = line.slice(6);
    if (data === "[DONE]") {               // 4. the end of the stream
      done = true;
      break read;
    }
    const chunk = JSON.parse(data);
    if (chunk.error) throw new Error(chunk.error.message);   // 5. a failure inside the stream
    const choice = chunk.choices[0];
    if (choice.delta.content) answer += choice.delta.content;
    if (chunk.usage) console.log(choice.finish_reason, chunk.usage);
  }
}

if (!done) throw new Error("the stream ended before [DONE]");
console.log(answer);

Ka whakamahi te tauira Python i te kete requests. Ka rere te tauira JavaScript i runga i a Node.js 18 neke atu rānei.

Ngā stream o ētahi atu hōputu

Ka stream hoki ngā endpoint Responses me Messages. Ka kawe ō rātou frame i tētahi rārangi event: me te ingoa o te kaupapa me tētahi rārangi data: me tōna JSON. Ka rārangi ia whārangi i āna kaupapa i te raupapa.