Ion Stoica: Memory is AI's Main Bottleneck, Demanding Infrastructure Innovation Across All Layers
Baca dalam 60 detik
- Ion Stoica mengungkapkan bahwa memori masih menjadi bottleneck krusial dalam pengembangan AI karena penyimpanan satu bit data tetap membutuhkan satu transistor.
- Efisiensi di setiap lapisan tumpukan teknologi dinilai wajib untuk mengimbangi lonjakan biaya pelatihan dan inferensi yang terus meningkat eksponensial.
- Pendekatan human-in-the-loop dan formal methods dipercaya dapat menjembatani kesenjangan antara intent pengguna dan kebutuhan sistem di era agentic AI.

Ion Stoica, a UC Berkeley Professor and co-founder of Databricks and Anyscale, stated that memory has become the biggest impediment to accelerating AI adoption in the agentic era. During a discussion titled The Next Endeavor, hosted by Imagination in Action in collaboration with Stanford HAI on September 14–15 at Google Bay View, Mountain View, California, Stoica emphasized that memory limitations remain a fundamental, unresolved issue.
Stoica explained that, to this day, storing one bit of information still requires one transistor. "You can make transistors smaller, but this one-to-one mapping doesn't change," he said. According to him, computing capacity will grow faster than memory capacity, and bandwidth is also predicted to increase more rapidly because it is easier to upgrade. Consequently, memory is increasingly becoming a bottleneck, necessitating layer decomposition and system modularity for maximum efficiency.
In a session moderated by Daniela Rus, Director of MIT CSAIL, Stoica also highlighted the prohibitive and still exponentially growing costs of training and inference. "Efficiency becomes incredibly important, and to get it, you almost have to innovate at every layer of the stack," he asserted. He added that many key innovations, including algorithmic evolution, are actually driven from the system side.
Stoica also discussed the challenges of alignment and the risks of recursive self-improvement. According to him, the primary goal must be to build reliable AI applications and systems that are aligned with user intent. "This is very difficult because there is a fundamental gap between user intent and system requirements, as well as between the real world and the environmental models we have," he explained. This gap, he continued, becomes increasingly difficult to close in an open and constantly changing world.
Responding to Rus's question about workload distribution between cloud and local devices, Stoica believes that local AI on edge devices can help address privacy concerns. He also linked this development to the big data era, where Hadoop and traditional queries were gradually replaced by much faster methods. Stoica himself was involved in the development of Spark, an SQL-related processing technology that now appears to be becoming obsolete in the AI era.
"As long as you're building this for humans, humans will be part of the loop; because you need to fulfill human intent... humans can be a bottleneck. We have to make sure that we use humans in the most effective way, to provide feedback, and to continuously improve the system."
— Ion Stoica
Rus then raised the issue of physical AI, envisioning 100,000 robots operating to perform tasks. According to her, such an armada of robots with physical intelligence would require robust infrastructure. Stoica countered by highlighting the Hugging Face incident and the threats of reward hacking and hallucinations. He suggested that AI operations in many cases should be isolated to prevent unintended risks.
Stoica concluded by emphasizing the importance of strengthening formal methods. "You will try to close the gap and capture, in formal specifications, more of human intent and more of the real world," he concluded. This call for clarity is deemed crucial as the adoption of agentic AI accelerates.
Indonesia's Context
For Indonesia, Stoica's spotlight on memory bottlenecks and AI infrastructure efficiency reflects the challenges faced. Dependence on foreign data centers and limited local computing resources make national AI development vulnerable to cost surges. However, the opportunity for AI on edge devices—which can reduce latency and maintain privacy—could be a solution for remote areas with limited connectivity. The government and industry players need to encourage research at the system layer, not just applications, so as not to always be technology consumers.
Looking ahead, the question is: can Indonesia build a robust AI ecosystem by prioritizing innovation at the infrastructure level, or will it continue to rely on breakthroughs from abroad?



