Li Auto is exploring the development of an in-house cloud inference chip that could use the same dataflow architecture as the company’s assisted driving processor, according to a report by Chinese media outlet LatePost. The project is reportedly still at an early stage.
The potential cloud chip would be designed to handle some artificial intelligence inference workloads currently performed on GPUs, including assisted driving model processing, testing and simulation, as well as requests from large language models.
Li Auto Launches Lower-Priced Rear-Wheel-Drive Li i8 to Boost EV Sales
Li Auto Considers Extending Vehicle Chip Architecture
AI data centres commonly rely on GPUs for model training and inference, although dedicated inference processors are also used for some workloads.
According to LatePost, industry sources said Li Auto is considering transferring elements of its vehicle-side dataflow architecture to cloud computing applications.
One potential approach would involve adapting the compute architecture used in Li Auto’s vehicle-side neural processing unit and combining multiple AI compute dies. High-bandwidth memory and high-speed interconnects could then be added to create a larger inference processor.
Such an approach could allow Li Auto to reuse portions of its hardware design and software tools across vehicle and cloud applications, potentially reducing some research and development costs.
Li Auto Establishes Shanghai Chip Company Amid Semiconductor Expansion
Cloud and Vehicle Computing Have Different Requirements
A cloud inference processor would nevertheless face different requirements from an automotive AI chip.
Vehicle-side computing typically operates with relatively defined models, sensor inputs and execution patterns. Key requirements include low power consumption, low latency and consistent performance.
Cloud infrastructure must support a broader range of models and accommodate more frequent model updates, varying input lengths and changing numbers of simultaneous requests.
Cloud systems also require technologies such as high-bandwidth memory, multi-chip interconnects, dynamic batching and cluster-level scheduling.
Li Auto Launches New Li L6 With Larger Battery and Updated Smart Driving System
An industry source told LatePost that achieving a target performance level with an individual chip is not necessarily the main challenge. Instead, the overall cost of running inference at scale is a critical factor.
Whether a vehicle-derived dataflow architecture can provide a cost advantage in the cloud would therefore depend on factors including model adaptation and system-level operating efficiency.
Dataflow Architecture Faces GPU Competition
Companies including SambaNova, Groq and Tenstorrent are also developing processors based on dataflow architectures.
The teams behind SambaNova and Groq include engineers associated with Stanford and Google’s TPU programme, respectively. Tenstorrent is led by Jim Keller, who previously led Tesla’s self-driving chip development.
Compared with GPUs, dataflow architectures still face questions around model flexibility, software ecosystems and large-scale deployment.
These factors could determine whether architectures originally designed for specific AI workloads can compete effectively with more general-purpose GPU platforms in cloud environments.
Chip Team Undergoes Personnel Changes
As the cloud chip project progresses, Li Auto’s chip organisation has also experienced personnel changes.
LatePost reported that Jin Yihua, head of chip software research and development, and Dai Jie, who led one of the company’s chip front-end design groups, have left Li Auto. The publication said it confirmed the departures through multiple sources.
Li Auto Plans Organizational Restructuring to Speed Up EV Product Development
Both previously reported to Luo Min, head of Li Auto’s computing power unit, who reports to group Chief Technology Officer Xie Yan.
It remains unclear whether the departures will affect development of the reported cloud inference chip.
Li Auto Expands In-House AI Hardware
Li Auto officially introduced its Mach M100 assisted driving chip on May 12. The processor is manufactured using a 5-nanometre automotive-grade process and provides 1,280 TOPS of computing power per chip.
The Mach M100 is now in mass production for the latest Li L9, L8 and L6 models. Vehicles using two chips can achieve combined computing power of 2,560 TOPS.
Li Auto founder, chairman and CEO Li Xiang has previously said the company’s investment in in-house chip development is intended to support practical AI applications in physical environments rather than solely demonstrate chip design capabilities.
The company also registered Xinchuang Zhihe (Shanghai) Technology Co Ltd on July 13. Its business scope includes integrated circuit chip design.
The reported cloud inference project would represent a potential expansion of Li Auto’s in-house AI hardware strategy beyond vehicle computing, although the company has not publicly confirmed the project or disclosed a development timeline.