Diagnose inconsistent data in a distributed database

Users report reading different values for the same record from a replicated database. How do you find the cause? Go through replication lag, read and write consistency settings, clock skew, failed or split-brain failovers and application bugs, the evidence you would gather for each, and how to repair the data and prevent a repeat.