Calculate the memory footprint of a tensor from its shape and data type.
A tensor's memory footprint is the product of all its dimensions (total element count) multiplied by the number of bytes each element occupies for the chosen data type: size_bytes = ∏(dimensions) × bytes_per_element. A (32, 512, 4096) FP16 activation tensor, for example, holds 32×512×4096 ≈ 67M elements at 2 bytes each, for roughly 134 MB. This calculation is the foundation for estimating activation memory, KV cache size, and intermediate buffer requirements when planning batch sizes and model architectures that must fit within a GPU's VRAM budget.
size_bytes = product(dimensions) × bytes_per_element
In PyTorch, call tensor.shape or tensor.size(); in TensorFlow, call tensor.shape — either gives you the dimension list to enter here (e.g. a shape of torch.Size([32, 512, 4096]) becomes "32, 512, 4096").
Size scales linearly with bytes per element, so simply switching a tensor from FP32 (4 bytes) to FP16 (2 bytes) halves its memory footprint with no change to its shape — this is a major reason mixed-precision and quantized inference reduce memory usage.
Sum the tensor size across every intermediate activation tensor retained for the backward pass (or for a KV cache during inference) — this per-tensor calculation is the building block for larger memory-budgeting calculators like the VRAM Usage Calculator.