TPU Debugging Overview

This section includes a collection of guides and reference docs for debugging TPU programs. Both JAX and TPU-SDK provide an extensive set of debugging tools and techniques, and choosing the right one is a technical decision itself.

Debugging can be divided into several levels, where each group offers diagnostics for a specific topic or area.

JAX

When starting debugging your JAX model, JAX offers a high-level and intuitive API and flags for logging and troubleshooting your model. If you are just starting, consider it as your first choice for any investigation, before jumping directly to a more complex tooling:

HLO debugging

For TPU, the JAX program is lowered to HLO, which can be extensively inspected for errors. HLO dumps provide an insight into generated modules:

Domain-specific debugging

For a range of domain-specific errors, check out the following documentation on some common cases: