Picking a Model for a Rude Chatbot

Running an AI product means picking a model, and picking a model is a set of tradeoffs between cost, latency, quality, and the specific behaviors you want. For a rude chatbot the priorities aren't obvious, so it's worth writing down how we ended up on the current stack.

The free tier: Llama 3.3 70B on Groq

The free tier gets five chats a day and one image upload. Those users cost real money if you route them through a frontier model, and most of what they ask isn't hard — a joke, a definition, a quick script. Llama 3.3 70B is genuinely strong at that class of question, and Groq's hosted version is close to free and returns tokens absurdly fast.

Fast tokens matter more than you'd think for a persona bot. When the AI drags an insult out over several seconds it stops feeling snappy and starts feeling like it's buffering. Latency is part of the tone.

Premium: frontier GPT via Lovable AI Gateway

Premium users pay $11.99/month or $199.99 once. They expect the answer to actually be good on hard questions — debugging a codebase, thinking through a decision, summarizing a long document. That means a frontier reasoning model, which right now means GPT-5-mini or GPT-5.5 depending on whether they've toggled 'deep thinking'.

Cost per request is roughly 20–50x the free tier. That's fine because premium is metered by the seat, not the request.

Vision: same premium model, upgraded content array

Images are the special case. Groq's hosted Llama build doesn't take images, so any request with an attached image routes to the Lovable AI Gateway regardless of tier — otherwise the free tier would silently drop the attachment and hand back a hallucinated description. Users hate that more than they hate a paywall.

What we're not doing

We're not fine-tuning a custom rude model. The persona lives entirely in the system prompt. Fine-tuning would lock us into whatever base we picked six months ago; a prompt-based persona takes 10 seconds to port to a better model when one ships. And better models are shipping constantly.

What might change

The obvious upgrade path is a cheaper, faster reasoning model for premium — every quarter something new arrives at half the cost. When it does, we'll switch. Users won't have to do anything. The rudeness is portable.