Deduplicate and order logs from batch and streaming sources

Log records come both in batches and as a live stream; some are duplicates and some arrive out of order. Produce a single output that has each record once, in timestamp order. Explain what you have to buffer in the streaming case and when it is safe to emit a record.