An AI chat composer + A streaming-text typewriter
Spent a day on this weekend trying to build a ChatGPT-style streaming chat UI in a Compose Multiplatform as a side project and ran into a wall: the markdown rendering and streaming-token side of things on CMP is sparse.
The specific problem:
Rendering progressively-arriving markdown without flicker. If you have a Flow<String> of tokens from an LLM SDK and re-parse "Hello, **world" → "Hello, **wo" → "Hello, world!" as each token arrives, naive parsers will keep re-classifying the bold span and the text reflows constantly.
The fix is a prefix-stable parser: for any prefix of the input string, the same prefix of tokens must come out. That means treating an unclosed ** as plain text (not a half-rendered bold), and only flipping it to a Bold span when the closing ** arrives. Earlier tokens never re-classify.
I ended up writing this + a few other pieces (composer with slash commands and @ mentions,
send/stop state machine, syntax-highlighted code blocks that build up live) into two libraries:
- llm-typewriter is the renderer
- prompt-bar is the composer
Curious if anyone has tackled this differently — there's a streaming-markdown library in React (Vercel's streamdown) that takes a similar approach but I don't know of others in the JVM world.
Code:
val prompt = rememberPromptBarState()
val state = rememberStreamingTypewriterState()
// Send button auto-becomes Stop while the typewriter streams.
LaunchedEffect(state.isStreaming) {
if (state.isStreaming) prompt.markStreaming() else prompt.markReady()
}
PromptBar(prompt, onSend = { vm.send() }, onStop = { state.stop(); vm.cancel() })
StreamingTypewriter(
tokens = vm.responseFlow,
state = state,
renderer = rememberMarkdownTypewriterRenderer(),
)