Advancing to the next frontier of AI

The recent Nvidia GTC 2026 conference in San Jose served as a pivotal platform for discussions surrounding the evolution of artificial intelligence (AI) and its implications for software tools. Key insights were shared by Jeff Dean, Google’s chief scientist, and Bill Dally, Nvidia’s chief scientist, who emphasized the urgent need for a fundamental redesign of current software tools to keep pace with AI agents that operate at machine speed. This shift is essential as traditional software development practices may not suffice in a landscape where AI can process information significantly faster than human developers.

Dean pointed out that while human programmers seldom worry about the startup time of a C compiler, the existing tooling can introduce substantial performance delays. The rapid pace at which AI agents function necessitates a transformation in coding tools, which Dean believes is already underway. He highlighted the necessity for business applications to evolve in tandem, enabling AI agents to manipulate spreadsheets and documents for information extraction in a more programmatic manner. This evolution is crucial as the industry transitions from prompt-and-wait AI systems to more advanced agentic systems capable of self-correction, negotiation, and even designing their successors.

The Road to Autonomous AI

For Google, the journey towards the next frontier of AI involves empowering models to act as autonomous research and development (R&D) laboratories. Dally inquired about the industry's proximity to achieving an AI model that can autonomously experiment, curate data, and train its next iteration. While Dean acknowledged that the technology is not fully developed yet, he pointed to the emergence of neural architecture search as a significant step forward. This innovative approach allows users to define research parameters in natural language, enabling AI to conduct experiments autonomously. Dean described it as a “super-powerful multiplier for research and productivity,” indicating its potential to revolutionize how AI contributes to scientific inquiry and technological advancement.

Achieving this level of autonomy will require models to overcome existing training limitations. Instead of relying on pre-training across the entirety of the internet's data, models could engage in predictive actions or learn from specific environments before returning to the learning phase. This method promises to enhance learning efficiency dramatically, a crucial factor as inference is projected to dominate AI workloads. Dally emphasized Nvidia's focus on minimizing communication latency to facilitate uninterrupted AI processing, which is vital for the agents to function effectively.

Innovations in Hardware and Energy Efficiency

To address the challenges of latency, Dally revealed Nvidia's exploration of simplified router architectures that prioritize latency over bandwidth. By reducing data transmission speeds from 400 to 200 gigabits per second, Nvidia aims to achieve router latency below 50 nanoseconds. This approach could enable the execution of large models at an impressive rate of 10,000 to 20,000 tokens per second.

Energy consumption remains a significant concern as AI systems require substantial power. Dally proposed a straightforward yet unconventional solution: minimizing data movement. He explained that while performing low-precision calculations consumes minimal energy, retrieving data from external memory incurs vastly higher energy costs. To mitigate this, Nvidia is investigating advanced 3D stacking technologies that integrate memory and processing units, thereby reducing energy expenditure associated with data transfer.

In addition to hardware innovations, achieving greater algorithmic efficiency is essential for addressing the AI power crisis. Dally highlighted the importance of sparsity—the practice of omitting calculations for parameters that do not significantly influence a model's output—as a key opportunity for reducing energy consumption. However, he cautioned that striving for higher sparsity levels could disrupt the predictable computation patterns that GPUs rely on, necessitating more sophisticated control and data routing strategies.

The Future of AI in Chip Design

The vision of an agentic future is already manifesting within the engineering teams at Nvidia and Google, where AI is being utilized to design the next generation of silicon. Dean cited Google’s achievements in employing AI for chip design, particularly through its AlphaChip research, while Dally elaborated on Nvidia’s integration of AI throughout its design pipeline.

One of Nvidia’s notable tools is NVCell, a reinforcement learning program that streamlines the porting of standard cell libraries during transitions to new semiconductor processes. This innovation has significantly reduced the time and manpower required for such tasks. Additionally, Nvidia has developed ChipNeMo, a custom large language model designed to enhance engineering productivity by mentoring junior engineers and automating the routing of bug reports.

Dally expressed a desire for AI to eventually automate the most labor-intensive aspects of chip development. He envisioned a future where engineers could simply request a new GPU design and return to find it completed, although he acknowledged that this reality is still a long way off. Even when such advancements are realized, Dally anticipates that AI chip designers will function under a master agent coordinating specialized sub-agents, mirroring the collaborative dynamics of human engineering teams today.