Entropy from logits, stable and then streaming
Compute the entropy of softmax(logits) for a vector of logits without overflow or underflow. Then the logits come in one by one: keep a fixed amount of state so the exact entropy is available after every arrival.
No answer yet. Write the first one; the best answer gets endorsed.
