Truncate model output at the first stop sequence

Implement a function that returns the part of a string that comes before the earliest stop word.

def text_before_stop(text: str, stop_words: list[str]) -> str: ...

• text is any string. It may be empty.
• stop_words is a list of non-empty strings. Matching is exact and case-sensitive. A stop word may appear anywhere, including inside another word.
• Return text[:i], where i is the smallest index at which any stop word starts. If no stop word occurs, return text unchanged.

Example 1

text = "Final answer: 42<|eot|> ignore this"
stop_words = ["<|eot|>", "\n\n"]
-> "Final answer: 42"

Example 2. The earliest start wins, even when another stop word ends sooner.

text = "xyzw", stop_words = ["yz", "xyzw"]
-> ""

Part 2 (streaming). The text now arrives as chunks from a model that is still generating, and it may not fit in memory. Implement a class with feed(chunk) -> str, which returns the text that is now safe to show the user, and flush() -> str, which is called at end of stream. A stop word may be split across chunks. Never release a character that could still be part of a stop word, and hold back as little as possible. Once a stop word is confirmed, ignore all further input.

Constraints: up to 10^6 characters of text and up to 10^4 stop words, each at most 100 characters long.