GPU and inference performance fundamentals
Answer questions on GPU performance: Amdahl's law, HBM versus on-chip SRAM, matrix-multiply cost and tiling, CPU versus GPU execution, tensor and pipeline parallelism, and where transformer inference spends its time.
No answer yet. Write the first one; the best answer gets endorsed.