China Telcom’s AI Analysis Institute claims it skilled a 100-billion-parameter mannequin utilizing solely domestically produced computing energy – a feat that implies Center Kingdom entities aren’t colossally perturbed by sanctions that stifle exports of Western tech to the nation.
The mannequin is named TeleChat2-115B and, in accordance with a GitHub replace posted on September 20, was “skilled completely with home computing energy and open sourced.”
“The open supply TeleChat2-115B mannequin is skilled utilizing 10 trillion tokens of high-quality Chinese language and English corpus,” the challenge’s GitHub web page states.
The web page additionally comprises a touch about how China Telecom could have skilled the mannequin, in a point out of compatibility with the “Ascend Atlas 800T A2 coaching server” – a Huawei product listed as supporting the Kunpeng 920 7265 or Kunpeng 920 5250 processors, respectively working 64 cores at 3.0GHz and 48 cores at 2.6GHz.
Huawei builds these processors utilizing the Arm 8.2 structure and payments them as produced with a 7nm course of.
At 100 billion parameters, TeleChat2 trails the likes of latest Llama fashions that apparently prime 400 billion parameters, or Open AI’s o1 which has been guesstimated to have been skilled with 200 billion parameters. Whereas parameter depend alone would not decide a mannequin’s energy or utility, the low-ish parameter depend suggests coaching TeleChat2 would likley have required much less computing energy than was wanted for different tasks.
Which can be why we will not discover a point out of a GPU – though the Ascend coaching server has a really modest one to drive a show at 1920 × 1080 at 60Hz with 16 million colours.
It due to this fact appears just like the infrastructure used to coach this mannequin was not at parity with the type of rigs accessible exterior China, suggesting that tech export sanctions aren’t stopping the Center Kingdom from pursuing its AI ambitions.
Or that it may ship in different methods, reminiscent of China Telecom’s monumental scale. The provider has income of over $70 billion, drawn from its provision of over half a billion wired and wi-fi subscriptions. It is also one of many largest customers and promoters of OpenStack. Even with out entry to the most recent and best AI {hardware}, China Telecom can muster loads of energy. ®
Source link
