Design an LLM-based coding assistant

Design an AI coding assistant that helps developers inside their editor: completing code, answering questions about a codebase and making edits on request.

Discuss how it gathers context from the repository, which models serve which requests and at what latency, how suggestions are shown and accepted, how you evaluate quality offline and online, and how you keep cost under control as usage grows.