Google’s Four TPU 8 Systems Explained: How Google Is Building Infrastructure for the Next Generation of AI

Artificial intelligence infrastructure is becoming more specialised. As organisations move beyond training large language models and begin deploying AI agents, real-time reasoning systems, and applications that process sensitive business data, the underlying hardware has to address a much wider range of requirements.

A single accelerator configuration is not necessarily the most efficient solution for every workload.

Google’s eighth-generation Tensor Processing Units (TPUs) reflect this shift. Rather than relying on one general-purpose accelerator design, Google has expanded its TPU portfolio to four specialised systems: TPU 8t, TPU 8i, TPU 8tx, and TPU 8i Max.

The first two systems, TPU 8t and TPU 8i, were introduced at Google Cloud Next in April 2026. On November 9, Google announced TPU 8tx and TPU 8i Max, extending the eighth-generation portfolio to address two additional requirements: confidential AI processing and workloads that demand substantially more memory and compute within a single package.

The distinction between these systems is important. Training a foundation model, serving thousands of inference requests, running an AI agent against confidential enterprise information, and processing millions of tokens of context are different computational problems. Each places different demands on processing power, memory, networking, and security.

Google’s latest TPU architecture is designed around these differences.

What Are Google TPUs?

Tensor Processing Units are Google’s custom-built application-specific integrated circuits (ASICs), designed to accelerate machine learning and AI workloads.

Unlike general-purpose CPUs, which are designed to handle a broad range of computing tasks, TPUs are optimised for the mathematical operations commonly used in machine learning. These include matrix multiplication and other tensor operations involved in training and running neural networks.

TPUs are particularly relevant for workloads involving large language models (LLMs), generative AI, recommendation systems, computer vision, and other computationally intensive applications.

However, the performance of an AI accelerator is not determined by its processing capability alone. Memory bandwidth, interconnect speed, data movement, software compatibility, and the efficiency of the surrounding infrastructure all influence how effectively an AI workload runs.

This is why Google’s eighth-generation TPU portfolio is built around specialised systems rather than treating every AI workload as the same problem.

The four systems form part of Google Cloud’s AI Hypercomputer, an integrated infrastructure architecture combining accelerators, networking, storage, and software for large-scale AI workloads.

Understanding Google’s Four Eighth-Generation TPU Systems

The four systems target different infrastructure requirements.

TPU system	Primary purpose	Main workload
TPU 8t	Large-scale training	Foundation model development
TPU 8i	Low-latency inference	Real-time reasoning and AI serving
TPU 8tx	Confidential inference	Sensitive data and proprietary AI models
TPU 8i Max	High-memory inference	Large-context and complex agentic workloads

Understanding the differences requires looking at the computational bottlenecks each system is designed to address.

1. TPU 8t: Designed for Large-Scale AI Model Training

Training a large AI model involves processing enormous datasets, performing repeated mathematical operations, calculating errors, and updating billions or even trillions of model parameters.

These operations need to run across large numbers of accelerators working together. As models grow, distributing computation efficiently becomes just as important as the processing power of an individual chip.

Google designed TPU 8t specifically for large-scale model training and embedding-heavy workloads.

How TPU 8t handles training workloads

One of the important architectural features of TPU 8t is its ability to scale across large accelerator clusters.

Google states that a single TPU 8t superpod can connect up to 9,600 chips using a three-dimensional torus network topology. This allows the system to distribute computation across a large number of accelerators while supporting communication between them.

This matters because distributed model training requires accelerators to exchange information continuously. If communication between chips becomes a bottleneck, processing units spend time waiting for data instead of performing calculations.

TPU 8t addresses this through several architectural improvements.

SparseCore acceleration: Large recommendation models and other embedding-heavy workloads frequently require irregular memory access patterns. Google’s SparseCore is designed to handle these operations separately from the main matrix computation units, reducing the impact of data-dependent operations on overall processing efficiency.

Native FP4 support: TPU 8t introduces native four-bit floating-point computation. Lower-precision numerical formats can reduce memory requirements and data movement while increasing computational throughput. The important consideration is maintaining acceptable model accuracy when using reduced precision.

Improved networking: Google’s Virgo Network provides the scale-out networking fabric required to connect large training clusters. Google reports up to four times the data centre network bandwidth compared with the previous generation for TPU 8t training.

Direct storage access: TPUDirect Storage allows TPU accelerators to access high-speed managed storage without routing all data through the host CPU and system memory. This helps reduce data-loading bottlenecks when training on extremely large datasets.

Google also reports up to 2.7 times better training performance per dollar compared with its previous-generation Ironwood TPU for large-scale training workloads.

Where TPU 8t fits in

TPU 8t is particularly relevant for organisations developing or training large foundation models, multimodal AI systems, and other computationally demanding machine learning applications.

For example, an organisation training a large language model on extensive text, image, and video datasets needs more than high-performance processors. It needs a distributed infrastructure capable of feeding data to thousands of accelerators and coordinating their operations without excessive communication overhead.

That is the problem TPU 8t is designed to address.

2. TPU 8i: Optimised for Real-Time AI Inference and Reasoning

Training develops a model. Inference is where that model is used to generate predictions, produce responses, or complete tasks.

When a user submits a question to an AI assistant, the model processes the input and generates an output through inference. The same process takes place when an AI agent analyses documents, generates code, retrieves information, or executes a sequence of tasks.

At production scale, inference introduces a different set of infrastructure challenges.

A model may need to serve thousands of concurrent requests while maintaining low response times. It also needs to manage the growing amount of information retained during a conversation or reasoning session.

This is where TPU 8i comes in.

Why inference requires a different architecture

One of the major bottlenecks in large-scale inference is memory movement.

During autoregressive generation, language models produce output tokens sequentially. They use a mechanism called the key-value (KV) cache to retain intermediate attention information from previously processed tokens.

Without this cache, the model would need to repeatedly recompute information from earlier tokens. However, as conversations and context windows become longer, the KV cache consumes increasingly significant amounts of memory.

Managing this memory efficiently becomes essential for maintaining inference performance.

Google designed TPU 8i around this requirement.

Its architecture includes:

• Larger on-chip SRAM: TPU 8i has three times more on-chip SRAM than the previous generation, allowing more KV cache data to remain close to the processing units.
• Collectives Acceleration Engine (CAE): This helps accelerate communication operations between accelerators.
• Boardfly network topology: Designed for low-latency communication, reducing the delays associated with distributed inference workloads.
• Optimised memory access: The system is designed to keep processors supplied with the information required for continuous token generation.

Google reports up to 80% better inference price-performance compared with its previous-generation Ironwood TPU.

Where TPU 8i fits in

TPU 8i is designed for real-time reasoning, high-concurrency AI serving, and post-training workloads.

Consider an enterprise AI assistant deployed across an organisation. Hundreds or thousands of employees may interact with it simultaneously, asking questions, retrieving information, and working with business documents.

The infrastructure must process requests quickly while managing multiple active sessions and their associated memory requirements.

TPU 8i is designed to address these inference demands.

It is also relevant to AI agents that need to reason through multiple steps, maintain context, and interact with external tools before returning a response.

The distinction is straightforward: TPU 8t focuses on efficiently training models at scale, while TPU 8i focuses on efficiently running models and handling real-time reasoning workloads.

3. TPU 8tx: Bringing Confidential Computing to AI Inference

As AI adoption expands into financial services, healthcare, government, and other regulated industries, data protection becomes a fundamental infrastructure requirement.

Organisations increasingly want to run AI models against information that cannot be exposed to unauthorised parties. This includes financial records, patient information, confidential business documents, customer data, and proprietary model weights.

Traditional encryption protects information while it is stored and while it travels across networks. However, data must generally be available in an accessible form while a workload processes it.

This creates an important security consideration: how can organisations protect sensitive information even while AI workloads are actively using it?

Google’s answer is TPU 8tx.

Announced on November 9, 2026, TPU 8tx is designed for confidential inference and sensitive AI workloads. It combines Google’s TPU architecture with hardware-enforced confidential computing capabilities and fifth-generation AMD EPYC host processors.

What makes TPU 8tx different?

The main distinction is its approach to protecting data during computation.

Confidential computing uses hardware-based security mechanisms to isolate workloads and protect information while it is being processed.

TPU 8tx extends this protection to AI accelerator workloads.

According to Google and AMD, the system uses AMD EPYC’s Secure Encrypted Virtualization (SEV) capabilities alongside Google’s confidential computing architecture. The system also verifies the accelerator’s identity and security configuration before allowing access to protected memory.

This helps protect sensitive information, including:

• User prompts and private input data.
• Proprietary AI model weights.
• Intermediate data exchanged between host processors and accelerators.
• Information used during inference and model customisation.

The involvement of AMD is also significant.

The fifth-generation AMD EPYC processors provide the host infrastructure responsible for workload orchestration, data preparation, and communication with TPU accelerators. AMD’s security capabilities contribute to extending confidential computing protections beyond the host CPU to the connected accelerators.

This is a hardware-level security approach rather than relying exclusively on application-level encryption or access controls.

Confidential computing and its importance for enterprise AI

Consider a financial institution deploying an AI assistant to analyse customer transaction records and internal financial documents.

The organisation may want to use generative AI to identify patterns, answer questions, and support internal decision-making. However, the underlying information is highly sensitive.

Traditional cloud security controls remain important, but organisations also need assurances about how data is protected while workloads are executing.

Confidential computing helps address this requirement by establishing hardware-enforced isolation around sensitive workloads.

The same principle applies to healthcare organisations processing patient records, government departments handling restricted information, and businesses using proprietary datasets to fine-tune their own AI models.

TPU 8tx is particularly relevant to these environments.

Running model customisation and inference on the same system

Another important capability is that TPU 8tx is not limited to inference.

Google also positions it for smaller-scale training and model customisation, including:

• Fine-tuning models using private enterprise data.
• Distilling larger models into smaller, more efficient versions.
• Training AI agents through reinforcement learning.
• Running confidential AI inference workloads.

Google says this configuration allows organisations to perform these activities without moving sensitive data between separate clusters.

It also uses the Boardfly network topology associated with TPU 8i. Google reports up to two times lower latency compared with TPU 8t, with a near-zero performance penalty.

The larger point is that security and performance are being considered together in the design of AI infrastructure. Organisations should not have to treat confidential processing as an entirely separate requirement from operational performance.

4. TPU 8i Max: Built for Large Context Windows and Memory-Intensive AI

As AI applications become more sophisticated, the amount of information a model needs to process in a single task is increasing.

An AI coding assistant, for example, may need to understand an entire software repository rather than a single source file. A video analysis system may need to process hours of footage. An enterprise agent may need to work across extensive documentation, multiple databases, and long-running conversations.

These workloads introduce a significant infrastructure challenge known as the memory wall.

Understanding the memory wall

The memory wall occurs when processors cannot access data quickly enough to keep up with their computational capabilities.

In AI inference, this can happen when models need to retrieve large amounts of context from memory while generating responses.

Increasing the number of processors alone does not necessarily solve the problem. If memory capacity or bandwidth cannot keep up, additional processing power may remain underutilised.

This becomes especially important for applications using large context windows and multiple AI agents.

Google designed TPU 8i Max to address this bottleneck.

What makes TPU 8i Max different?

Rather than introducing an entirely unrelated chip architecture, Google combines two low-latency TPU 8i chips into a single package.

According to Google’s announcement, TPU 8i Max provides:

• Twice the compute capacity of TPU 8i.
• Twice the on-chip SRAM.
• Twice the high-bandwidth memory (HBM) bandwidth.

By scaling compute and memory together, the system is designed to keep more information available to the processing units and reduce delays associated with data movement.

Google also states that the dual-chip configuration fits within the existing rack footprint, allowing greater compute density per square foot and per watt.

Where TPU 8i Max fits in

The benefits become clearer when looking at memory-intensive AI applications.

AI coding assistants: An AI coding agent working across a large software repository may need to retain information about multiple files, dependencies, functions, and previous interactions. More memory capacity and bandwidth can help support larger working contexts.

Long-form video analysis: Analysing extended video content requires processing and retaining information across many frames and sequences. Memory-intensive inference systems can help support these workloads.

Multi-agent systems: Multiple AI agents may need to maintain context, share information, and coordinate tasks. As the number of active agents increases, managing their working memory becomes more demanding.

Advanced reasoning: Complex reasoning workloads may involve longer contexts and multiple stages of computation. Infrastructure capable of handling larger amounts of context efficiently becomes increasingly important.

TPU 8i Max is designed for these workloads, where memory capacity and bandwidth are just as important as raw processing performance.

How the Four TPU Systems Fit Into the AI Lifecycle

The four systems are not simply four different versions of the same accelerator. They address different infrastructure requirements across the AI lifecycle.

AI requirement	TPU system	Infrastructure priority
Training large foundation models	TPU 8t	Distributed compute and training throughput
Serving AI models and real-time reasoning	TPU 8i	Low latency and inference efficiency
Processing sensitive enterprise information	TPU 8tx	Hardware-enforced confidential computing
Handling massive context windows	TPU 8i Max	Memory capacity and bandwidth

This specialisation reflects a broader change in how AI infrastructure is being designed.

Earlier generations of AI infrastructure were largely focused on scaling the computational capacity required to train increasingly large models.

That remains important. However, as AI moves into production, other requirements become equally significant.

Inference costs affect the economics of running AI applications. Memory constraints influence how much context a model can process. Networking determines how efficiently distributed workloads communicate. Security controls determine whether sensitive business information can be processed safely.

The four TPU systems address these requirements through different hardware and system configurations.

Why Google Is Moving Towards Specialised AI Infrastructure

Google’s latest TPU announcement is part of a larger industry shift towards purpose-built AI infrastructure.

The growth of generative AI has changed the computational requirements of enterprise applications. Organisations are no longer using AI only to generate text or classify information. They are building systems that can analyse large datasets, interact with software tools, execute workflows, and make decisions across multiple stages.

These developments introduce three important infrastructure considerations.

1. Inference is becoming a major infrastructure investment

Training a foundation model can require substantial computing resources, but production inference introduces a different cost structure.

Once an AI application is deployed, it may need to serve requests continuously. The total cost depends on request volume, model size, context length, response speed, and the number of concurrent users.

This makes inference performance and price-performance important considerations when selecting AI infrastructure.

Google’s separation of training-focused TPU 8t and inference-focused TPU 8i reflects this distinction.

2. Enterprise AI requires security beyond traditional infrastructure controls

Businesses operating in regulated industries cannot evaluate AI infrastructure based on performance alone.

They must also consider data isolation, access controls, encryption, workload integrity, and protection of proprietary information.

Confidential computing addresses a specific part of this requirement by protecting data during processing.

TPU 8tx brings these capabilities into Google’s custom AI accelerator infrastructure, extending confidential computing to sensitive AI inference and model customisation workloads.

3. Memory and networking are becoming critical performance factors

Larger models and increasingly complex AI agents require access to substantial amounts of information.

A system can have significant computational power and still perform poorly if memory bandwidth, storage access, or inter-chip communication becomes a bottleneck.

TPU 8i Max directly addresses memory-related constraints, while Google’s broader TPU architecture incorporates specialised networking and storage technologies to support distributed workloads.

The implication is that AI infrastructure performance must be evaluated as a complete system rather than by comparing processor specifications in isolation.

What This Means for Businesses Building AI Applications on Google Cloud

For organisations planning AI deployments, Google’s eighth-generation TPU portfolio introduces more options for matching infrastructure to workload requirements.

However, selecting an accelerator is only one part of the architecture decision.

A production AI environment also requires the right compute configuration, data pipelines, storage architecture, networking, security controls, orchestration, monitoring, and cost management strategy.

For example, a business building a customer-facing generative AI application may prioritise inference latency and cost per request. A financial services organisation processing confidential information may place greater emphasis on hardware-enforced security. A company developing an AI coding platform may need infrastructure capable of handling large context windows.

These requirements should influence infrastructure planning from the beginning.

Organisations should evaluate several factors before committing to a TPU-based deployment:

Workload characteristics: Determine whether the primary requirement is model training, real-time inference, confidential processing, or high-context reasoning.

Model compatibility: Evaluate supported frameworks, model architectures, numerical precision requirements, and software dependencies.

Data architecture: Assess how training datasets, inference inputs, KV caches, and application data will move through the infrastructure.

Security and compliance: Identify encryption requirements, workload isolation needs, regulatory obligations, and access-control policies.

Performance and cost: Benchmark actual workloads rather than relying exclusively on theoretical processing performance. Measure latency, throughput, utilisation, and total cost of ownership.

Scalability: Plan how the environment will handle increasing inference requests, additional users, longer context windows, and growing AI workloads.

The most suitable infrastructure is the one that meets the application’s actual requirements without introducing unnecessary complexity or operating costs.

TPU 8t vs 8i vs 8tx vs 8i Max: Which One Should Businesses Choose?

There is no single best TPU system for every organisation. The appropriate choice depends on the workload being deployed.

• Choose TPU 8t when the priority is large-scale foundation model training and distributed machine learning workloads.
• Choose TPU 8i when the requirement is high-concurrency inference, low-latency serving, and real-time AI reasoning.
• Choose TPU 8tx when AI workloads involve confidential information, proprietary model weights, or sensitive enterprise data that requires hardware-enforced protection during processing.
• Choose TPU 8i Max when applications need substantially greater compute and memory bandwidth to support large context windows and memory-intensive inference.

These systems also should not be viewed as mutually exclusive choices across an entire organisation. Different applications and stages of the AI lifecycle may require different configurations.

For example, a company may train a model using TPU 8t, deploy it for real-time inference using TPU 8i, and use TPU 8tx for workloads that require confidential processing.

The architecture should follow the workload rather than forcing every application onto the same accelerator configuration.

What We Still Don’t Know About Google’s New TPUs

Although Google has announced the capabilities and intended workloads of TPU 8tx and TPU 8i Max, some commercial and deployment details remain undisclosed.

As of November 10, 2026, Google has not published specific pricing or general availability dates for the two newly announced systems.

This matters for organisations evaluating a potential deployment.

Hardware capabilities alone are not enough to determine whether a system is commercially viable. Customers also need to understand regional availability, provisioning options, pricing models, supported configurations, and the software requirements for production workloads.

These details will influence how easily organisations can adopt the new systems and calculate their expected return on infrastructure investment.

For now, the announcement provides a clearer picture of Google’s hardware direction, but deployment decisions will require more information.

Conclusion

Google’s eighth-generation TPU portfolio represents a move towards more specialised AI infrastructure.

TPU 8t addresses the computational demands of large-scale model training. TPU 8i focuses on low-latency inference and real-time reasoning. TPU 8tx introduces hardware-enforced confidential computing for sensitive AI workloads, while TPU 8i Max targets applications that require substantially more memory and compute capacity.

The important development is not simply that Google has introduced two additional TPU systems. It is that AI infrastructure is being designed around the distinct requirements of models as they move from development into production.

As organisations deploy more sophisticated AI applications and autonomous agents, performance, memory, networking, security, and operating costs will increasingly need to be considered together.

Choosing the right accelerator will be an important part of that decision, but designing the surrounding cloud infrastructure will matter just as much.