Sruthadh

POST https://api.shannon-ai.com/v1/chat/completions

Bheir na OpenAI SDKan na chunks dhut fear air fhear. Leugh gach chunk a-rèir na tha e a’ giùlan: pios reasoning, pios dhen fhreagairt, gairm inneal, no deireadh an fhreagairt.

from openai import OpenAI, APIError

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.shannon-ai.com/v1",
)

stream = client.chat.completions.create(
    model="shannon-3",
    messages=[{"role": "user", "content": "Explain server-sent events in three sentences."}],
    stream=True,
)

answer, reasoning, tool_calls = [], [], []
finish_reason = usage = None

try:
    for chunk in stream:
        if chunk.usage:
            usage = chunk.usage
        if not chunk.choices:
            continue
        choice = chunk.choices[0]
        delta = choice.delta

        thought = getattr(delta, "reasoning_content", None)
        if thought:
            reasoning.append(thought)
        if delta.content:
            answer.append(delta.content)
            print(delta.content, end="", flush=True)
        for call in delta.tool_calls or []:
            tool_calls.append((call.id, call.function.name, call.function.arguments))
        if choice.finish_reason:
            finish_reason = choice.finish_reason
except APIError as error:
    # A failure after the stream started arrives as an error chunk.
    print(f"\nstream failed: {error}")

print(f"\n{finish_reason} {usage}")

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.shannon-ai.com/v1",
});

const stream = await client.chat.completions.create({
  model: "shannon-3",
  messages: [{ role: "user", content: "Explain server-sent events in three sentences." }],
  stream: true,
});

let answer = "";
let reasoning = "";
const toolCalls = [];
let finishReason = null;
let usage = null;

try {
  for await (const chunk of stream) {
    if (chunk.usage) usage = chunk.usage;
    const choice = chunk.choices?.[0];
    if (!choice) continue;
    const delta = choice.delta;

    if (delta.reasoning_content) reasoning += delta.reasoning_content;
    if (delta.content) {
      answer += delta.content;
      process.stdout.write(delta.content);
    }
    for (const call of delta.tool_calls ?? []) {
      toolCalls.push({ id: call.id, name: call.function.name, arguments: call.function.arguments });
    }
    if (choice.finish_reason) finishReason = choice.finish_reason;
  }
} catch (error) {
  // A failure after the stream started arrives as an error chunk.
  console.error("\nstream failed:", error.message);
}

console.log("\n", finishReason, usage);

curl -N https://api.shannon-ai.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "shannon-3",
    "messages": [{"role": "user", "content": "Explain server-sent events in three sentences."}],
    "stream": true
  }'

Mar a tha stream air a fhrèamadh

• Tha staid 200 aig an fhreagairt agus na headers content-type: text/event-stream agus cache-control: no-cache.
• Is e aon loidhne a th’ anns gach tachartas a thòisicheas le data: agus tha loidhne fhalamh às a dhèidh. Is e aon chunk JSON an teacsa às dèidh data: . Chan eil loidhnichean event: no id: ann.
• Is e beachd a th’ ann an loidhne a thòisicheas le colon, leithid : ping, a chumas an ceangal fosgailte. Leum thairis air.
• Crìochnaichidh an stream le data: [DONE]. Tha stream a stadas às aonais neo-choileanta.

Chunks nan òrdugh

Is e nì chat.completion.chunk a th’ anns gach chunk:

{
  "id": "chatcmpl-5f0c1e7a9b3d4c62a8e1f07d2b46c9a3",
  "object": "chat.completion.chunk",
  "created": 1791625200,
  "model": "shannon-3",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": "Server-sent"
      },
      "finish_reason": null
    }
  ]
}

Tha an aon id, created agus model aig gach chunk de dh’aon stream. Tha aon inntrigeadh an-còmhnaidh ann an choices. Tha na tha ùr ann an delta, agus tha finish_reason null gus an chunk mu dheireadh.

Thig reasoning an toiseach, an uair sin am freagairt, an uair sin gairmean inneal sam bith. Leugh chunk a-rèir nan iuchraichean ann an delta, chan ann a-rèir an àite san stream: faodaidh delta role a ghabhail còmhla leis a’ chiad phios teacsa, agus tha am fear mu dheireadh falamh.

Stream gu lèir mar a shiubhlas e. Gus na loidhnichean a chumail goirid, tha id, object, created agus model air am fàgail a-mach às gach chunk an seo.

data: {"choices":[{"index":0,"delta":{"reasoning_content":"Three sentences:"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"reasoning_content":" what it is, how it is framed, why it is used."},"finish_reason":null}]}

: ping

data: {"choices":[{"index":0,"delta":{"content":"Server-sent"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"content":" events let a server push text to a client over one HTTP reply."},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":1190,"completion_tokens":84,"total_tokens":1274}}

data: [DONE]

Deltas reasoning

Cuiridh modail a nì reasoning a reasoning ann an delta.reasoning_content ron fhreagairt. Cùm an dà theacsa air leth: seall an reasoning mar phàirt fa leth ris an gabh pasgadh, no fàg a-mach e.

Chan eil na modailean open-weight hosted a’ cur reasoning nuair a tha reasoning_effort off, agus chan eil gin nuair a tha response_format san iarrtas.

Nì shannon-2-lite agus shannon-2-pro reasoning mus freagair iad cuideachd. Air an endpoint seo cuiridh iad loidhnichean beachd : thinking fhad ’s a nì iad sin, agus an uair sin am freagairt.

Chan eil reasoning_content anns na modailean le seòrsa aig na OpenAI SDKan. Ann am Python leugh e le getattr(delta, "reasoning_content", None); ann an JavaScript obraichidh delta.reasoning_content mar a tha e.

Deltas gairm inneal

Thig gairm inneal slàn: aon chunk gach gairm, le delta anns a bheil a’ ghairm leis an t-sreang arguments choileanta. Àiridh index gairmean an fhreagairt o 0. Obraichidh còd a chuireas arguments ri chèile pios air phios gun atharrachadh, oir cuiridh e aon phios ri chèile.

{
  "tool_calls": [
    {
      "index": 0,
      "id": "call_3d9a7c1e5b2f4a60c8e1d7f09b24a6c5",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\":\"Paris\"}"
      }
    }
  ]
}

Às dèidh nan gairmean, tha finish_reason tool_calls aig an chunk mu dheireadh.

Nuair a tha tools san iarrtas, cuiridh shannon-1.6-lite, shannon-1.6-pro agus shannon-coder-1 teacsa an fhreagairt mar aon delta aig deireadh an fhreagairt.

An chunk mu dheireadh

Tha delta falamh, an finish_reason agus usage an iarrtais aig a’ chunk mu dheireadh ro data: [DONE], uile san aon chunk. Tha e aig gach stream; chan fheum thu stream_options.

{
  "id": "chatcmpl-5f0c1e7a9b3d4c62a8e1f07d2b46c9a3",
  "object": "chat.completion.chunk",
  "created": 1791625200,
  "model": "shannon-3",
  "choices": [
    {
      "index": 0,
      "delta": {},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 1190,
    "completion_tokens": 84,
    "total_tokens": 1274
  }
}

Air na modailean open-weight hosted tha prompt_tokens_details.cached_tokens agus completion_tokens_details.reasoning_tokens ann an usage cuideachd.

Às dèidh iarrtas le web_search: true a lorg rannsachadh rudeigin, tha sources air a’ chunk mu dheireadh cuideachd, agus tha aon chunk ro-làimh a’ giùlan nan comharran-iomraidh mar delta.annotations. Rannsachadh Lìn air a thoirt a-steach

Tha raointean usage air am mìneachadh leis an endpoint. Chat Completions

Loidhnichean keep-alive

Fhad ’s a tha am modail ag obair agus gun dad aige ri chur fhathast, bidh loidhnichean beachd san stream. Chan eil dàta annta. Leumaidh parser SSE thairis orra leis fhèin; feumaidh còd a leughas loidhnichean amh leum thairis air gach loidhne a thòisicheas le colon.

Mearachdan am broinn stream

Aon uair ’s gu bheil stream air tòiseachadh is e 200 a staid, agus mar sin thig fàillinn am broinn an stream: chunk le ball error far am biodh choices.

data: {"error":{"message":"The model backend failed to answer. Please retry.","type":"api_error","code":null,"param":null}}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":1190,"completion_tokens":12,"total_tokens":1202}}

data: [DONE]

Às dèidh an chunk mearachd thig an chunk mu dheireadh, le finish_reason agus usage, agus data: [DONE] fhathast. Dèan sgrùdadh air gach chunk airson error. Às aonais an sgrùdaidh sin tha coltas coileanta air freagairt a dh’fhàillig letheach slighe.

Cuiridh na modailean open-weight hosted am ball error le type agus message a-mhàin.

Togaidh na OpenAI SDKan Python agus JavaScript chunk mearachd mar mhearachd API fhad ’s a tha thu ag ath-aithris, agus mar sin cuir an lùb ann an làimhseachadh mearachd àbhaisteach.

Tha duilleag fhèin aig còdan staid, bodhaig na mearachd agus na tha ri ath-fheuchainn. Làimhseachadh mearachd

A’ dùnadh a’ cheangail

Le bhith a’ dùnadh a’ cheangail stadaidh tu an lìbhrigeadh, chan e an t-iarrtas. Sgrìobhaidh am modail am freagairt chun a dheireadh agus thèid cosgais a chuir air an iarrtas mar gum biodh tu air a leughadh uile. Gus nas lugha a phàigheadh, iarr nas lugha: suidhich max_tokens nas ìsle air modail a chuireas an gnìomh e.

Faodaidh stream crìochnachadh tràth air an t-slighe thugad cuideachd, mar eisimpleir aig briseadh lìonraidh no fhad ’s a tha frithealaiche ga ùrachadh. Stadaidh e an uair sin gun an chunk mu dheireadh agus gun data: [DONE]. Gabh ris an fhreagairt sin mar neo-choileanta agus cuir an t-iarrtas a-rithist.

A’ leughadh stream gun SDK

Bidh leughadair ceart a’ dèanamh còig rudan: cuiridh e mearachdan a ruigeas ron stream gu làimhseachadh mearachd, sgoilidh e am bodhaig na loidhnichean, leumaidh e thairis air loidhnichean beachd, stadaidh e aig data: [DONE], agus sgrùdaidh e gach chunk airson error.

import json
import requests

response = requests.post(
    "https://api.shannon-ai.com/v1/chat/completions",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "shannon-3",
        "messages": [{"role": "user", "content": "Explain server-sent events in three sentences."}],
        "stream": True,
    },
    stream=True,
)

# 1. Errors before the stream are plain JSON with their own status.
if response.status_code != 200:
    raise RuntimeError(f"{response.status_code}: {response.text}")

answer, done = [], False
for raw in response.iter_lines():          # 2. one line at a time
    line = raw.decode("utf-8")
    if not line.startswith("data: "):      # 3. empty lines and ": ping" comments
        continue
    data = line[len("data: "):]
    if data == "[DONE]":                   # 4. the end of the stream
        done = True
        break
    chunk = json.loads(data)
    if "error" in chunk:                   # 5. a failure inside the stream
        raise RuntimeError(chunk["error"]["message"])
    delta = chunk["choices"][0]["delta"]
    if delta.get("content"):
        answer.append(delta["content"])
    if chunk.get("usage"):
        print(chunk["choices"][0]["finish_reason"], chunk["usage"])

if not done:
    raise RuntimeError("the stream ended before [DONE]")
print("".join(answer))

const response = await fetch("https://api.shannon-ai.com/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "shannon-3",
    messages: [{ role: "user", content: "Explain server-sent events in three sentences." }],
    stream: true,
  }),
});

// 1. Errors before the stream are plain JSON with their own status.
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);

const decoder = new TextDecoder();
let buffer = "";
let answer = "";
let done = false;

read: for await (const bytes of response.body) {
  buffer += decoder.decode(bytes, { stream: true });
  const lines = buffer.split("\n");       // 2. one line at a time
  buffer = lines.pop();                    //    keep the unfinished line
  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;   // 3. empty lines and ": ping" comments
    const data = line.slice(6);
    if (data === "[DONE]") {               // 4. the end of the stream
      done = true;
      break read;
    }
    const chunk = JSON.parse(data);
    if (chunk.error) throw new Error(chunk.error.message);   // 5. a failure inside the stream
    const choice = chunk.choices[0];
    if (choice.delta.content) answer += choice.delta.content;
    if (chunk.usage) console.log(choice.finish_reason, chunk.usage);
  }
}

if (!done) throw new Error("the stream ended before [DONE]");
console.log(answer);

Cleachdaidh sampall Python am pacaid requests. Ruithidh sampall JavaScript air Node.js 18 no nas ùire.

Streams de na cruthan eile

Bidh na h-endpoints Responses agus Messages a’ sruthadh cuideachd. Tha loidhne event: le ainm an tachartais agus loidhne data: le a JSON anns na frèamaichean aca. Tha gach duilleag a’ liostadh nan tachartasan aice san òrdugh.