Google reportedly developing ‘Frozen v2’ AI chip optimized for Gemini models

Google LLC is reportedly advancing its efforts in artificial intelligence by developing a new chip, dubbed Frozen v2, which is specifically optimized for its Gemini series of AI models. This move signifies a strategic shift towards creating custom hardware tailored to the unique demands of AI workloads, potentially enhancing efficiency and performance in AI applications.

The current landscape of AI hardware often relies on off-the-shelf chips that may include unnecessary components. For instance, an AI startup purchasing a graphics card might find that it includes both inference and rendering cores, leaving some of these modules idle during operation. This inefficiency can hinder the performance of AI projects, as resources are not utilized to their full potential.

By designing a custom chip like Frozen v2, Google aims to eliminate these inefficiencies. A custom chip allows a company to exclude circuits that are not essential for its specific workloads, thereby reducing manufacturing costs. Furthermore, it can replace these unused circuits with cores that are optimized for particular tasks, enhancing overall performance.

Google has a history of providing custom AI chips known as Tensor Processing Units (TPUs) through its public cloud services. The latest additions to this lineup, the TPU 8t and TPUi, are tailored for training and inference tasks, respectively. The anticipated Frozen v2 chip is expected to take this customization further, as it will be specifically designed to align with the architecture of the Gemini models.

The expected efficiency improvements from Frozen v2 are anticipated to come from several avenues. Notably, Google aims to reduce the number of calculations required to operate the Gemini models, as well as minimize data movement. Data movement becomes a significant challenge when a neural network's components cannot be accommodated within a graphics card's onboard memory. In such cases, elements of the neural network must be stored in off-chip storage, necessitating frequent transfers between this storage and the graphics card's processing units, which can slow down overall processing speed.

To mitigate this bottleneck, Google plans to equip the Frozen v2 chip with sufficient memory to allow the Gemini models to operate fully on-chip. This design choice would eliminate the need for constant data transfers to and from off-chip RAM, thereby enhancing processing speed and efficiency.

Moreover, the expectation that Frozen v2 will reduce the number of calculations needed for Gemini suggests it will incorporate operator fusion, a technique commonly used to accelerate AI models. Operator fusion combines multiple calculations into a single computation, which can be executed more quickly than if processed separately.

Google's TPUs are typically deployed in clusters that consist of multiple custom components. For example, the TPU 8i utilizes specialized devices known as optical circuit switches to manage data flow. It is likely that the Frozen v2 chip will be designed to work seamlessly with these existing components, avoiding the need for significant redesigns of the current cluster architecture.

The company is reportedly aiming to begin rolling out the Frozen v2 chip to its data centers by 2028, marking a significant step in its ongoing commitment to advancing AI technology through tailored hardware solutions.

Why it matters

The development of the Frozen v2 chip by Google underscores a critical trend in the AI industry: the shift towards custom hardware designed specifically for AI workloads. For creators and technologists, this represents a significant opportunity to enhance the efficiency and performance of AI applications. By leveraging chips like Frozen v2, developers can expect improved processing speeds and reduced operational costs, which could lead to faster iterations and more innovative AI solutions.

Additionally, the potential for operator fusion and reduced data movement could streamline the development process for AI models, allowing creators to focus more on innovation rather than optimization challenges. As companies like Google continue to invest in custom AI hardware, it may set new standards for performance and efficiency in the industry, prompting other tech companies to follow suit.

The implications of such advancements are vast, as they could democratize access to high-performance AI capabilities, enabling smaller startups and individual creators to compete on a more level playing field with larger enterprises. As the landscape evolves, staying informed about these technological advancements will be crucial for anyone involved in AI development and deployment.