LLM provider pool with failover

Write a client that sends completion requests to a pool of external LLM providers. It should be able to switch between providers, and when one goes down, move traffic to another without the caller noticing. Decide how a provider is marked unhealthy and brought back, how retries and timeouts work, and how you would test the failover.