Websauna Co / AI
I’ve been experimenting with on-device AI on phones for years, chasing a simple question: when does quantized inference actually make sense on a smartphone? It’s tempting to treat quantization as a silver bullet — smaller models, faster...
Read more... →
I want to walk you through a pattern I’ve used several times in the field: rolling out incremental on-device model updates using delta-only packages so users get improvements with minimal bandwidth, storage and downtime. I’ve shipped mobile and...
Read more... →
I’ve been wrestling with the same question a lot lately: if you’re working with regulated data, is it really cheaper (and safer) to use an off-the-shelf model like Mistral via an API, or to bring a model in-house and fine-tune it locally? The...
Read more... →
Keeping an LLM on-device up to date in a field app is deceptively hard. Over-the-air model updates, prompt tuning patches, and delta compression can change user experience, storage, and — critically — battery life. I've worked through many...
Read more... →
I’ve been benchmarking vector databases for months now, because keeping semantic search realtime at scale is one of those problems that sounds simple until you try to run it on real traffic. In this article I walk through the tests I ran comparing...
Read more... →
I remember the first time my team relied on an AI system in production: a recommendation model that quietly drifted for weeks until business metrics diverged and users complained. We had alerts—sort of—but they told us symptoms, not causes. That...
Read more... →
Embedding drift is one of those slow, sneaky problems you don’t notice until your semantic search suddenly returns garbage for queries that used to work. I’ve wrestled with it in production: models change, content evolves, third-party data...
Read more... →
I’ve been experimenting with on-device GPT-style models on phones and tablets for over a year now, trying to answer the question that keeps coming up in meetings, DMs, and comment threads: can on-device GPT replace cloud inference for offline...
Read more... →
I’ve been watching models quietly change behavior in production long enough to know the moment a product team tells me “we didn’t see it coming.” Silent model drift — where performance degrades without obvious feature changes or clear data...
Read more... →
I recently spent a few weekends trying to answer a practical question I keep getting from field engineers and IT managers: can you run a reliable on-device LLM for field techs on a Raspberry Pi 5? The short answer is: yes—with important caveats....
Read more... →