Mtundu wa Inkling API Wosasidwa

Ambiri mwa mitundu khumi ndi iwiri pa gateway yathu ndi mawu okha. Inayi si zimenezo, ndipo mwa mitundu inayiyi Inkling ili ndi malo enieni: mtundu wa masomphenya umene mumaugwiritsa ntchito pamene chithunzicho ndicho gawo lovuta la vutolo koma ndalama sizikwana flagship. Kaadi iyi ikufotokoza komwe maziko amachokera, chomwe chowonjezera W4A16-AUTOROUND-REAP chinachichitira, chomwe chidzachita ndi chomwe sichidzachita pa mbali ya API, ndipo — moona — nthawi imene mtundu wotsika mtengo mu mndandanda womwewo ndicho chisankho chabwino kwambiri.

01Kodi Inkling ndi chiyani, ndipo ndani anaimanga?

Inkling ndi mtundu woyamba wa zolemera zotseguka wochokera ku Thinking Machines Lab, wofalitsidwa pa July 15, 2026. Mosiyana ndi zotulutsidwa zambiri zoikidwa ngati "zotseguka," inatumizidwa ndi zolemera zonse zotsitsika pa Hugging Face ndi chithandizo cha tsiku loyamba mu transformers, SGLang, vLLM, ndi llama.cpp — ndicho chifukwa chake inatha kuyenda pa zomangamanga ngati zathu konse. Manambala aakulu, ochokera ku zolemba za kutulutsidwa za Thinking Machines zokha:

Kameneko ndi kakonzedwe ka Mixture-of-Experts kosakanikirana: zosanjikiza za decoder 66 pa kukula kwa 6,144 kobisika, ziwiri zoyambirira zolimba, zotsalazo 64 zonyamula akatswiri 256 ogawidwa ntchito kuphatikiza akatswiri 2 ogawana, ndi akatswiri 6 ogawidwa ntchito ogwira ntchito pa token. Pafupifupi 4% ya zigawo imayaka pa token iliyonse yoperekedwa — ndicho chimapangitsa mtundu wa pafupi ndi tiriliyoni imodzi ya zigawo kukhala wotheka kutumikiridwa konse.

Zisankho ziwiri za kapangidwe ndi zosazolowereka mokwanira kuti zitchulidwe. Chisamalira ndi chosinthasintha — zosanjikiza 55 za sliding-window attention pa zenera la ma token 512 motsutsana ndi zosanjikiza 11 za global, mu mawonekedwe obwerezabwereza a 5:1. Ndipo Inkling sigwiritsa ntchito RoPE: imagwiritsa ntchito chikoka chophunziridwa, chodalira zolowetsa cha malo achibale, pamodzi ndi ma convolution aafupi a causal pa ma projection a key/value ndi nthambi za residual. Zolemba za kapangidwe za Sebastian Raschka zikuwonetsa zonse ziwiri ngati kupatuka kwenikweni kwa njira ya wamba ya 2026. Maphunziro anagwiritsa ntchito Muon pa ma matrix parameter aakulu ndi Adam pa ena onse, kenako kupitirira 30 miliyoni a ma RL rollout osalunjika mu maphunziro apambuyo pake.

Mfundo kwa wogwiritsa ntchito API si zochititsa chidwi zokha. Ndikuti awa ndi maziko a mlingo wa frontier, a zamitundu yambiri mwachibadwa, ophunzitsidwa pa matiriliyoni 45 a ma token okhudza mawu, zithunzi, mawu oyimbidwa ndi makanema — osati mtundu wa mawu okhala ndi adapter ya masomphenya yowonjezeredwa pambuyo pake. Zithunzi zimasinthidwa kukhala zidutswa za pixel 40×40 kudzera mu MLP ya patchifier ya zosanjikiza zinayi, ndipo zimayendetsedwa pamodzi ndi mawu mu decoder yomweyo. Kumvetsetsa kwa masomphenya kunali mu cholinga cha pretraining.

02Kodi W4A16-AUTOROUND-REAP amatanthauza chiyani?

Njira zitatu zosiyana zaikidwa mu chowonjezerachi, ndipo zimachita ntchito zosiyana. Zoyenera kumveka, chifukwa chowonjezerachi ndicho kusiyana kwakukulu kwambiri pakati pa mtundu umene mumaitana pa API yathu ndi checkpoint imene Thinking Machines inafalitsa.

REAP — kudula akatswiri amene palibe amene amawapatsa ntchito

REAP ndi Router-weighted Expert Activation Pruning, yochokera ku Cerebras Research (arXiv 2510.13999, yovomerezedwa ku ICLR 2026). Mu MoE yokhala ndi akatswiri 256 pa chosanjikiza chilichonse router siimagawira ntchito mofanana: akatswiri ena amagunda nthawi zonse, ena amayatsa pang'ono ndipo amapereka pang'ono pamene akuchita zimenezo. REAP imayerekezera katswiri aliyense pa zonse ziwiri mfundo za router gate ndi ma norm a kugwira ntchito kwa katswiri — katswiri wosankhidwa nthawi zambiri koma amene amapereka zosintha zocheperapo ndi wofunika pang'ono kuposa mmene chiwerengero cha kusankhidwa kwake chikuwonetsera — ndipo imachotsa ofunika pang'ono kwambiri mu kudutsa kamodzi, quantization isanachitike.

Chifukwa timagwiritsa ntchito kudula m'malo mokongola kwambiri kwa kuphatikiza akatswiri ndi chakuti ziwirizi zimasiyana potengera mtundu wa ntchito. Kuphatikiza kumaoneka bwino pa ma benchmark odziwitsa koma kumagwa pa amene amapanga. Olemba a REAP akunena kuti kudula kumasunga mtundu bwino kwambiri pa ntchito zenizeni zimene anthu amagulira API — kupanga code, kuyitanitsa ntchito, kuganiza kwa masitepe ambiri — pa magawo a kupanikiza oposa 50%, pa mitundu kuchokera ku 20B mpaka 1T zigawo. Kuphatikiza kumawononga kudziwa kwapadera kwa router; kudula kumasiya akatswiri opulumuka ndi kugawa pakati pawo mosasinthidwa.

AutoRound — kuphunzira kumene kuzungulira

AutoRound ndi njira ya Intel ya quantization pambuyo pa maphunziro (arXiv 2309.05516, yofalitsidwa ngati SignRound). Quantization yosaganizira imazungulira kulemera kulikonse ku mtengo woyandikira kwambiri wa 4-bit — chachikulu chabwino ndipo choipa pa mlingo waukulu, chifukwa zolakwika za kuzungulira siziri zodziyimira payokha ndipo zimasonkhana kudutsa unyolo wa zosanjikiza 66. AutoRound imasamalira chisankho cha kuzungulira ngati chinthu chokonza: signed gradient descent imakonza pamodzi njira ya kuzungulira ndi malire a kudula kwa zolemera pa masitepe angapo mazana, kutenga kulondola kwa quantization-aware training popanda kuphunzitsanso. Intel ikunena za phindu lalikulu kwambiri pa makonzedwe a bits otsika kwambiri ndi phindu logwirizana pa INT4.

W4A16 — kumene ma biti amapita

W4A16 ndi mawonekedwe a manambala: zolemera za 4-bit, zochitika za 16-bit. Zolemera — zimene zimalamulira kukumbukira mu mtundu wa 975B — zimachepa mpaka pafupifupi gawo limodzi mwa anayi la malo a BF16, pomwe masamu amakhalabe pa mabiti 16. Kugawidwa kumeneku ndikofunika kwa mtundu wa masomphenya kwenikweni: njira za activation-quantized zimawononga ndendende ntchito ya manambala yotsatanetsatane imene kumvetsetsa zithunzi kumadalira — kuwerenga chizindikiro chaching'ono cha axis, kutanthauzira mzere wowonda mu graph, kusiyanitsa mithunzi iwiri yofanana. Kusunga zochitika pa mabiti 16 kumalipiritsa bandwidth ndikubweza kulondola kumene ntchito ya masomphenya imazindikira.

Zophatikizidwa, zitatuzi zimapereka mtundu umene umakwana ndi kupitiriza kufulumira popanda ngozi ya mtundu ya mlingo wolimba wa 3BIT-REAP umene timagwiritsa ntchito pa maziko akulu kwambiri. Icho ndicho kusinthana kumene mtengo umawonetsa.

03Masomphenya: API ya kusanthula zithunzi yopanda kukana

Inkling ndi imodzi mwa mitundu inayi ya masomphenya mu mndandanda wathu, pamodzi ndi Kimi-K3, Kimi-K2.6, ndi MiniMax-M3. Zotsatira za masomphenya zofalitsidwa za mtundu wa maziko, kuchokera ku kutulutsidwa kwa Thinking Machines, ndi 73.3% pa MMMU Pro (Standard 10) ndi 78.1% pa mafunso a kuganiza a CharXiv — kukwera mpaka 82.0% pamene mtunduwo ukuloledwa kuyendetsa Python motsutsana ndi graph. Zimenezo ndi ziwerengero za labotale yokha za checkpoint yosapanikizidwa; ziwoneni ngati malire aakulu a banjali, osati muyeso wa build yathu.

CharXiv ndicho choyenera kukhala nacho nthawi, chifukwa chikugwirizana ndi chifukwa chomwe anthu amagulira API ya masomphenya m'malo mwa ntchito ya OCR. Imayesa kuganiza pa magraph a sayansi — kuwerenga mtengo kuchokera pa axis, kuyerekezera mizere iwiri, kuzindikira mmene chizolowezi chikusinthira. Osati "mawu ati ali mu chithunzichi" koma "chithunzicho chikunena chiyani, ndipo kodi chinenerocho ndi choona." Icho ndicho chimalungamitsa kutumiza chithunzi ku mtundu wa kuganiza wa 975B m'malo mwa wolemba zolongosola waung'ono.

Kutumiza chithunzi ndi ntchito wamba yogwirizana ndi OpenAI — gawo la zinthu la image_url pa /v1/chat/completions, chosanjikiza cha mtundu wa Anthropic cha image pa /v1/messages, kapena chofanana pa /v1/responses. Zilankhulo zonse zitatu zimafikira zolemera zomwezo ndipo zonse zitatu zimachita streaming; sankhani chilichonse chimene SDK yanu yomwe ilipo kale imalankhula. Chitsanzo chogwiridwa chili mu gawo 09.

Gawo losasidwa ndicho lofunika kwambiri pa masomphenya kuposa pa mawu. Wosanjikiza wa kukana pa endpoint ya masomphenya ndi wolimba kwambiri mosazolowereka, chifukwa nthawi zambiri umaika chithunzi m'gulu mtunduwo usanaganize za pemphocho. Ntchito zeniyeni zimagwidwa nthawi zonse: chithunzi cha chilonda mu ntchito ya zolemba za zachipatala, screenshot ya umboni mu lipoti la chochitika, frame ya kanema kamene kakusanthulidwa pa nkhani ya kudzudzula, chithunzi cha zachipatala, chithunzi cha chikalata chomwe chili ndi dzina. Inkling pa gateway yathu ilibe chowunika choterocho patsogolo pake. Imayang'ana chomwe munatumiza ndikuyankha funso limene munafunsa. Ndicho chinthu chogulitsidwa.

04Zotuluka zokonzedwa: response_format inde, json_schema ayi

Ichi ndicho malo okha amene Inkling ili ndi kuthekera kochepa kuposa mndandanda wina, ndipo ndikwabwino kukonzekera izi kuposa kuzizindikira mukugwira ntchito.

Kuthekera	inkling	Kimi-K2.6	MiMo-V2.5	Laguna-S-2.1
Masomphenya (zolowetsa za zithunzi)	Inde	Inde	Inde	Ayi
response_format	Inde	Inde	Inde	Ayi
json_schema	Ayi	Inde	Inde	Ayi
Kuyitanitsa ntchito	Inde	Inde	Inde	Inde
Ndalama za kuganiza	Inde	Inde	Inde	Inde

response_format: {"type": "json_object"} imagwira ntchito — mumapeza JSON yolondola pa mawonekedwe. Chomwe simupeza ndi kupanga kokakamizidwa ndi grammar motsutsana ndi schema imene mumapereka, kumene sampler imaletsedwa mwathupi kutulutsa token imene ikuphwanya kapangidwe kanu. Mayina a magawo, kukhala membala wa enum, ndi kupezeka kwa makiyi ofunika ndi chisankho cha mtunduwo, osati chitsimikizo.

Njira zitatu zokhalira ndi izo, m'dongosolo la kudalirika kokwera:

1. Funsani schema ndikutsimikizira. Ikani schema mu funso la dongosolo ngati JSON yeniyeni, ikani response_format kukhala json_object, ndipo tsimikizirani yankho mbali yanu. Yesaninso pamene kutsimikizira kwalephera ndi uthenga wa cholakwika cha validator wowonjezeredwa. Yotsika mtengo, imagwira ntchito, imafunika ndalama za kuyesanso.
2. Gwiritsani ntchito malongosoledwe a chida ngati schema. Kuyitanitsa ntchito kumathandizidwa, ndipo malongosoledwe a parameter ya ntchito ndi JSON Schema. Longosolani chida chimodzi chotchulidwa dzina la kutulutsa deta kumene mukufuna, kakamizani, ndikuwerenga ma parameter. Ichi nthawi zambiri ndicho chosankha cholimba kwambiri pa Inkling ndipo sichikulipiritsani kalikonse kowonjezera.
3. Gawani ntchitoyo pakati pa mitundu iwiri. Lolani Inkling ichite masomphenya ndi kuganiza mu mawu, kenako perekani mawuwo ku MiMo-V2.5-W8A16 pa $0.50 / $2.00 ndi json_schema yeniyeni pa sitepe yokonza. Mayitanidwe awiri, koma achiwiri ndi otsika mtengo, olondola mu mawonekedwe, ndipo mtundu umene uyenera kukhala wanzeru ndi umene unaona chithunzicho.

Njira 3 ndiyo tikanalangiza pa njira iliyonse yotulutsa deta ya kuchuluka kwakukulu. Lipirani mtengo wowonjezera pa sitepe yokha imene ikuyifunikira.

05Kodi mtengo wowonjezera umagula chiyani?

Pa $1.42 kulowa / $6.07 kutuluka pa miliyoni ya ma token, Inkling ili mwadala pakati pa mndandanda wathu. Asanu ndi limodzi mwa mitundu khumi ndi iwiri ali pa mtengo wotsika kwambiri wa $0.50 / $2.00. Kimi-K3 — flagship, yomwenso ndi ya masomphenya — ndi $3.83 / $19.12. Inkling ili pafupifupi kuwirikiza katatu kwa mtengo wotsika kwambiri pa kutuluka ndipo pafupifupi gawo limodzi mwa atatu la flagship.

Mtengo wa kutuluka, USD pa ma token 1M, mitengo ya mndandanda wa Shannon AI Gateway, Seputembala 2026. Kutakata kwa bala ndi kofanana ndi mtengo; kutalika ndikokwera mtengo.

Nayi kuwerenga koona kwa kumene mtengo wowonjezerawo umapindulitsa — ndi kumene sukupindulitsa.

Ntchito	Gwiritsani ntchito Inkling?	Chifukwa
Kuganiza kwa graph / chithunzi	Inde	Ziwerengero za CharXiv ndi MMMU Pro zofalitsidwa ndizo maziko awa ali amphamvu kwambiri.
Agent zoyendetsedwa ndi screenshot	Inde	Masomphenya kuphatikiza kuyitanitsa ntchito kuphatikiza mbiri ya 262K mu mtundu umodzi, palibe kusinthana.
Zikalata zazitali zokhala ndi zithunzi	Inde	Mawu ndi zithunzi zimagawana decoder imodzi ndi zenera limodzi la context.
Kupanga mawu okha	Ayi	Mitundu isanu ndi umodzi pa $0.50 / $2.00 imachita izi. Mukulipira njira ya masomphenya imene simuyitanapo.
Kutulutsa deta ya schema ya bulki	Ayi	Palibe json_schema. MiMo-V2.5 ndi yotsika mtengo ndipo yokakamizidwa.
Malire enieni a frontier	Ayi	Kimi-K3 ndicho flagship. Lipirani 3.1× ngati malirewo ndicho chotchinga.
Masomphenya otsika mtengo kwambiri kwenikweni	Ayi	MiniMax-M3 ili ndi masomphenya pa $0.50 / $2.00. Yambani kumeneko, kwezani ngati mtundu ukukakamiza.

Werengani izi ngati lamulo la kutumiza, osati mndandanda wa malo. Inkling ndi yoyenera pa gulu lopapatiza la ntchito kuposa tsamba la kutsatsa lingatchule, ndipo mkati mwa gululo n'zovuta kupambana pa mtengo. Kunja kwake, tikanakonda kukugulitsani mtundu wotsika mtengo ndikukusungani.

06Zenera la context la 262K, ndi zomwe zimakwana kwenikweni

Mitundu yonse khumi ndi iwiri pa gateway imatumikiridwa ndi context ya kulowa ya ma token 262,144 ndi kutuluka kwambiri kwa ma token 16,384 pa pempho lililonse. Yofanana mwadala: kasamalidwe ka context sikuyenera kukhala nthambi ya mtundu uliwonse mu code yanu, ndipo kusinthira inkling ndi MiniMax-M3 mu fayilo ya config sikuyenera kusintha mwakachetechete zomwe zimakwana.

Onani mpata pakati pa maziko ofalitsidwa: Thinking Machines imalemba Inkling pa ma token 1,048,576, ndipo timatumikira gawo la 262K la izo. Ngati muli ndi ntchito yeniyeni ya pempho limodzi la miliyoni ya ma token, sitiri endpoint yoyenera pa izo, ndipo ndi bwino mudziwe musanamange.

Pa kukula pa 262K: PDF ya ukadaulo ya masamba 400 pamodzi ndi zithunzi zake, chosungiramo code chapakati, maola angapo a zolembedwa, kapena gawo lalitali la agentic lonyamula zotsatira za zida makumi angapo ndi ma screenshot ochepa — chilichonse chimakwana mu pempho limodzi. Pa mtundu wa masomphenya, kumbukirani kuti zithunzi zimadyanso context; screenshot yolimba si yaulere, ndipo gawo limene limatumiza ma frame mumaseshoni limadzaza zenera mofulumira kuposa gawo la mawu okha. Kapangidweko kamayenda bwino pa izi: mawonekedwe a 5:1 sliding-window-to-global amatanthauza kuti zosanjikiza zambiri zimayang'ana mkati mwa dera la ma token 512, ndi zosanjikiza za global zimabweretsanso kapangidwe kakutali nthawi ndi nthawi.

07Ndalama za kuganiza: kulipira kuganiza pokha pamene mukukufunika

Inkling imasonyeza ndalama za kuganiza zowongoleredwa — imodzi mwa zowongolera zothandiza kwambiri mu mndandanda. Mtundu wakuluwo umalandira milingo kuchokera none ndi minimal kudutsa low, medium, high, xhigh, mpaka max, yosonyezedwa pa zilankhulo zonse zitatu kudzera mu parameter yamba ya reasoning_effort.

{
  "model": "inkling-W4A16-AUTOROUND-REAP",
  "reasoning_effort": "low",
  "messages": [ ... ]
}

Iyi ndi mphamvu ya mtengo yokhala ndi ano, chifukwa ma token a kuganiza amalipiritsidwa ngati ma token otuluka pa $6.07 pa miliyoni. Thinking Machines ikunena kuti pa Terminal Bench 2.1 Inkling imafika pa zotsatira zofanana ndi Nemotron 3 Ultra pa pafupifupi gawo limodzi mwa atatu la ma token — kuchita bwino kwa token ndi cholinga chonenedwa cha kapangidwe ka dongosolo la ndalama za kuganiza, osati ngozi.

Mu ntchito zenizeni: yendetsani low kapena minimal pa kulemba zolongosola, kuika m'magulu, ndi kutulutsa deta kolunjika, kumene yankho lili mu chithunzicho ndipo palibe unyolo wa kuganiza umene ukufunikira. Sungani high ndi kupitirira pa kuganiza kwa graph, kuyerekezera kwa masomphenya kwa masitepe ambiri, ndi masitepe a agentic kumene sitepe yolakwika yapakati ndi yokwera mtengo. Magulu ambiri sawika parameteryo konse, ndipo amasiya kusiyana kwa mtengo wa kutuluka kwakukulu pa tebulo.

08Chomwe "chosasidwa" chimatanthauza pano — ndi chomwe sichitanthauza

Zenizeni: timayendetsa mtunduwu ndi palibe wosanjikiza wa kukana ndi palibe kusefa kwa zotulutsa. Palibe chowunika chimene chimawerenga funso lanu ndikusankha ngati mtunduwo ungathe kuyankha, ndipo palibe chimene chimawerenga yankho lisanatumizidwe mu streaming. Pamene mtunduwo sadziwa kapena ulibe chidziwitso, umanena zimenezo — ndicho mtunduwo, osati ndondomeko. Chomwe simudzalandira ndi kukana kwachizolowezi pa mutu umene chinthu cha ogula wamba chimaika kunja kwa malire.

Pa masomphenya ichi ndicho cholinga chonse cha phindu. Zowunika za zithunzi patsogolo pa ma API a masomphenya ndi zosalondola, ndipo mlingo wa zolakwika zabodza umagwera kwambiri pa ntchito ya akatswiri imene ikufunikira kuthekera kumeneku kwambiri: kafukufuku wa chitetezo, malo a zachipatala, uthenga ndi umboni, kudzudzula kwa makanema ndi luso, zida za kuwunika zomwe ziyenera kuyang'ana chinthu chomwe zikuwunika. Endpoint imene imakana chifukwa cha gulu singathe kutumikira imodzi mwa izi modalirika.

Zodziwitsa ziwiri zoona. Choyamba, Thinking Machines imafalitsa kuwunika kwa chitetezo pa kutumizidwa kwawo koyerekeza — 78.0% pa FORTRESS adversarial, 95.9% pa FORTRESS benign, 98.6% pa StrongREJECT. Ziwerengerozo zikufotokoza za chomangamanga chawo ndi zoteteza zawo zili pomwepo. Sizikufotokoza za zathu, ndipo sitidzazitchula ngati kuti ndi zathu. Tinachotsa wosanjikiza umene manambala amenewo amayeza gawo lake; icho ndicho chinthu chogulitsidwa, ndi udindo wake.

Chachiwiri, "palibe wosanjikiza wa kukana" si "palibe malamulo." Shannon Lab LLC imayendetsedwa pansi pa Ndondomeko ya Kagwiritsidwe Kaudindo ndi malamulo ogwira ntchito a US. Kuchotsa chowunika kumasuntha chiweruzo kukhala chanu, sikuchifafaniza. Ngati mukumanga pa izi, werengani ndondomekoyo musanakuze.

09Kuyamba

Base URL yomweyo, kiyi yomweyo, zilankhulo zitatu. Ngati code yanu ikuyitana kale OpenAI, Anthropic, kapena Responses API, kusintha mawu a mtundu nthawi zambiri ndicho kusamuka konse.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.shannon-ai.com/v1",
    api_key=SHANNON_API_KEY,
)

stream = client.chat.completions.create(
    model="inkling-W4A16-AUTOROUND-REAP",
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "What does this figure actually show?"},
        {"type": "image_url", "image_url": {"url": image_data_url}},
    ]}],
    reasoning_effort="high",
    max_tokens=4000,
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

Mndandanda wonse wa ma parameter, mawonekedwe a /v1/messages ndi /v1/responses, mawonekedwe a zochitika za streaming, ndi malo oyeserera amoyo zili mu zolemba za API. Makaadi a mtundu a mitundu ina khumi ndi imodzi ali mu Kafukufuku wa Shannon.

Ziwerengero za maziko ndi manambala ofalitsidwa ndi Thinking Machines Lab za checkpoint ya Inkling yosapanikizidwa ndipo zikufotokoza malire a banjali, osati build yathu ya quantization. Mitengo ndi zizindikiro za kuthekera ndi mitengo ya mndandanda wa Shannon AI Gateway pofika Seputembala 2026.

10Mafunso ofunsidwa kawirikawiri

---

Magwero: Thinking Machines Lab — Kudziwitsa Inkling · Hugging Face — Kulandira Inkling · REAP the Experts (arXiv 2510.13999) · Optimize Weight Rounding via Signed Gradient Descent (arXiv 2309.05516) · Sebastian Raschka — zolemba za kapangidwe ndi benchmark za Inkling. Ma benchmark a maziko afalitsidwa ndi labotale za checkpoint yosapanikizidwa.