Stable softmax and multi-head attention in NumPy
In NumPy, implement a numerically stable sigmoid and softmax, then scaled dot-product attention, then multi-head attention from scratch.
AmazonTechnical phone screenNo answer yet. Write the first one; the best answer gets endorsed.