Implement a transformer block with SwiGLU
Write a transformer block from scratch: multi-head self-attention, the residual connections and normalisation, and a feed-forward layer that uses the SwiGLU gate (two parallel projections, one passed through SiLU, multiplied together, then projected back). Explain the tensor shapes at each step.
GoogleTechnical phone screenNo answer yet. Write the first one; the best answer gets endorsed.