Count keywords across a fleet's logs
Many machines write logs, and you need a distributed system that gathers those logs and counts how often given keywords appear in them. The keyword set, log volume and freshness requirements were left to you to ask about. Cover collecting logs from each host, where the counting happens (on the host, in a stream processor, or in batch), aggregating partial counts, handling late and duplicate data, storing results so they can be queried by keyword and time, and how the design changes if the keyword set changes often.
No answer yet. Write the first one; the best answer gets endorsed.
