Làimhseachadh mearachd

Togaidh na SDKan nì mearachd a bhios a’ giùlan còd na staid agus raointean na bodhaig. Ainmichidh an t-iarrtas seo modail nach eil ann:

import openai
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.shannon-ai.com/v1",
)

try:
    client.chat.completions.create(
        model="no-such-model",
        messages=[{"role": "user", "content": "Hello"}],
    )
except openai.APIStatusError as error:
    print(error.status_code)                           # 400
    print(error.type)                                  # invalid_request_error
    print(error.body["message"])                       # unknown model: no-such-model
    print(error.response.headers.get("x-request-id"))  # the id of this request

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.shannon-ai.com/v1",
});

try {
  await client.chat.completions.create({
    model: "no-such-model",
    messages: [{ role: "user", content: "Hello" }],
  });
} catch (error) {
  if (!(error instanceof OpenAI.APIError)) throw error;
  console.log(error.status);         // 400
  console.log(error.type);           // invalid_request_error
  console.log(error.error.message);  // unknown model: no-such-model
}

curl -i https://api.shannon-ai.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "no-such-model",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Tillidh e staid 400 agus a’ bhodhaig seo:

{
  "error": {
    "type": "invalid_request_error",
    "message": "unknown model: no-such-model"
  }
}

Bodhaig na mearachd

Tha dà bhodhaig ann. Freagraidh /v1/messages ann an cruth Anthropic; freagraidh a h-uile endpoint eile ann an cruth OpenAI. Anns an dà chuid, tha am fiosrachadh ann an error.type agus error.message, agus mar sin fritheilidh aon leughadair gach endpoint.

{
  "error": {
    "type": "authentication_error",
    "message": "Invalid API key",
    "code": "invalid_api_key",
    "param": null
  }
}

{
  "type": "error",
  "error": {
    "type": "authentication_error",
    "message": "Invalid API key"
  }
}

Seòrsaichean mearachd

Còdan staid

Ruithidh na sgrùdaidhean san òrdugh seo: iuchair API, bodhaig an iarrtais, id modail, dìon thar-thuiltidh, balance. Mar sin tillidh iarrtas le iuchair cheàrr agus bodhaig cheàrr 401. Chan eil iarrtas a thèid a dhiùltadh le staid 4xx air a chosg.

Balance agus dìon thar-thuiltidh

Thèid gach iarrtas a fhrithealadh co-ionnan: chan eil ìrean ìre-chuingeachaidh no cuota API fa leth ann. Faodaidh dà rud fhathast 429 a fhreagairt, an dà chuid leis an t-seòrsa rate_limit_error. Seallaidh an teachdaireachd an diofar.

Balance

Is e do bhalance cuibhreann plana an-diugh, a dh’ath-shuidhicheas aig 00:00 UTC, agus às a dhèidh creideas a cheannaich thu. Tarraingidh còmhradh agus an API às an aon bhalance.

Cuiridh gach iarrtas crìoch an output aige an dàrna taobh an toiseach: max_tokens, no max_output_tokens air /v1/responses, 4,096 mura cuir thu gin. Nuair a bhios am freagairt coileanta, thèid na chaidh a chur an dàrna taobh a chur an àite na chleachd an t-iarrtas. Mura h-eil an t-suim a’ freagairt dhen bhalance a tha air fhàgail, tillidh an t-iarrtas Quota exceeded mus tòisich am modail.

Cuiridh iarrtasan a ruitheas aig an aon àm an t-suim aca fhèin an dàrna taobh. Mar sin faodaidh mòran iarrtasan co-shìnte le max_tokens àrd am balance a chleachdadh airson greis, fiù ‘s ma tha gach freagairt goirid. Cuiridh max_tokens nas lugha nas lugha an dàrna taobh.

Dìon thar-thuiltidh

Leigidh dìon thar-thuiltidh 120 iarrtas gach mionaid gach cunntais. Cunntaidh e an cunntas, chan e an iuchair, thar /v1/chat/completions, /v1/responses agus /v1/messages còmhla. Tòisichidh a’ mhionaid leis a’ chiad iarrtas agad.

Tillidh iarrtas àireamh 121 sa mhionaid sin Too many requests. Retry in <n>s. agus chan eil e a’ cosg dad. Thèid iarrtasan a chaidh a dhiùltadh a chunntadh cuideachd, agus mar sin fanaidh client a chuireas a-rithist gun feitheamh air a dhiùltadh gus an tig a’ mhionaid gu crìch. Fuirich na <n> diogan a tha san teachdaireachd.

Chan eil iarrtasan a thèid a chur ann an co-shìnte air an diùltadh air sgàth sin. Fuirichidh iad sa chiudha agus thèid am freagairt an tur.

Chan eil GET /v1/models agus na h-endpoints cunntais token air an cunntadh.

Tuilleadh mu chrìochan agus balance, agus càite am faic thu do bhalance agus do chleachdadh fhèin: Crìochan agus balance Iuchraichean & cleachdadh

Dè ri fheuchainn a-rithist

Cuir iarrtas a-rithist dìreach nuair a dh’fhaodadh an t-aon iarrtas soirbheachadh. Bidh SDKan OpenAI agus Anthropic mar-thà a’ feuchainn freagairtean 429 agus 5xx a-rithist dà thuras leotha fhèin; suidhich max_retries (maxRetries ann an JavaScript) gus sin atharrachadh.

Lùb feuchainn a-rithist a leanas an clàr seo:

import re
import time

import openai
from openai import OpenAI

# max_retries=0: this loop does the retrying, the SDK does not add its own.
client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.shannon-ai.com/v1", max_retries=0)


def create_with_retry(attempts=5, **request):
    for attempt in range(attempts):
        try:
            return client.chat.completions.create(**request)
        except openai.RateLimitError as error:
            text = str(error)
            flood = re.search(r"Retry in (\d+)s", text)
            if flood:
                time.sleep(int(flood.group(1)))   # flood protection: wait out the minute
            elif "temporarily busy" in text:
                time.sleep(2 ** attempt)
            else:
                raise                             # balance: a wait of seconds does not help
        except (openai.InternalServerError, openai.APIConnectionError):
            time.sleep(2 ** attempt)              # 1, 2, 4, 8 seconds
    raise RuntimeError("the request failed after all attempts")


response = create_with_retry(
    model="shannon-3",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

import OpenAI from "openai";

// maxRetries: 0 — this loop does the retrying, the SDK does not add its own.
const client = new OpenAI({ apiKey: "YOUR_API_KEY", baseURL: "https://api.shannon-ai.com/v1", maxRetries: 0 });

const sleep = (seconds) => new Promise((resolve) => setTimeout(resolve, seconds * 1000));

async function createWithRetry(request, attempts = 5) {
  for (let attempt = 0; attempt < attempts; attempt++) {
    try {
      return await client.chat.completions.create(request);
    } catch (error) {
      const text = String(error.message);
      const flood = text.match(/Retry in (\d+)s/);
      if (error.status === 429 && flood) {
        await sleep(Number(flood[1]));            // flood protection: wait out the minute
      } else if (error.status === 429 && text.includes("temporarily busy")) {
        await sleep(2 ** attempt);
      } else if (error.status >= 500 || error instanceof OpenAI.APIConnectionError) {
        await sleep(2 ** attempt);                // 1, 2, 4, 8 seconds
      } else {
        throw error;                              // wrong request, wrong key or balance
      }
    }
  }
  throw new Error("the request failed after all attempts");
}

const response = await createWithRetry({
  model: "shannon-3",
  messages: [{ role: "user", content: "Hello" }],
});
console.log(response.choices[0].message.content);

Mearachdan am broinn stream

Tha staid 200 aig stream bhon chiad byte. Thig fàilligeadh às dèidh sin mar thachartas am broinn an stream: chunk le ball error air /v1/chat/completions, tachartas error air /v1/messages, tachartas response.failed no error air /v1/responses. Thig tachartasan dùnaidh an stream fhathast às a dhèidh. Làimhsich a leithid de fhreagairt mar fhàilligeadh agus cuir an t-iarrtas a-rithist, mar airson staid 5xx.

Thèid còmhradh a tha nas fhaide na uinneag cho-theacsa a’ mhodail aithris san dòigh seo, leis an t-seòrsa invalid_request_error agus an còd context_length_exceeded. Giorraich an còmhradh mus cuir thu a-rithist e.

Tha na frèaman cruinn air am mìneachadh còmhla ri streaming. Sruthadh

Ids iarrtais

Tha header x-request-id aig gach freagairt, mearachdan nam measg. Is e an luach a chuir thu san header x-request-id agad fhèin, no 12 caractaran sia-deugach mura do chuir thu gin. Clàraich e, agus thoir iomradh air nuair a dh’iarras tu taic le iarrtas.