LLM Inference Economics from First Principles
The main product LLM companies offer these days is access to their models via an API, and the key question that will determine the profitability they can enjoy is the inference cost structure.
The main product LLM companies offer these days is access to their models via an API, and the key question that will determine the profitability they can enjoy is the inference cost structure.
Learning shell scripting through code analyses of a real-world example, breaking the ‘z’ program line by line to understand what’s happening under the hood and hopefully become a better bash programmer -- Part 1.
The monks were freaking out long before Silicon Valley was
This week a fellow B12er was performing an ad-hoc data analysis. They had a spreadsheet with some data, and wanted to classify the rows in the spreadsheet by a few different criteria along which they would look for trends. For an engineer, this would have been…
<p>Tired of AI coders tripping on new libraries? MadKudu's CTO reveals how feeding targeted context (think Context7) to your LLM can dramatically boost accuracy and 10x your AI-assisted coding. Stop fighting outdated knowledge!</p>
This blog documented my setup for self-hosting a web server. Hopefully this is
Academic and journalist
Imagine, for a moment, a world of pure chaos.
Ever since I had my first Windows desktop, then a Windows laptop turned into a Linux laptop, then a Linux laptop running Hackintosh in a VM, then a Mac, I've...
Personal webpage of Alexandre Dulaunoy - from information security to open source and art
As you may recall from previous posts and elsewhere I have been busy writing a new solver for APT. Today I want to share some of the latest changes in how to approach solving. The idea for the solver was that manually installed packages are always protected fr…
Glitch is shutting down, and it's a bummer. Here's what I think about it.
In my most recent blogpost I gave an introduction to the Mojo programming language. This blogpost will provide a workflow how to archive peak performance on...
In small to mid-sized engineering teams, unchecked diversity in backend technology stacks can lead to increased cognitive load, fragmented knowledge, and reduced team mobility. While each language or tool may offer individual advantages, the lack of alignment …
If you're anything like me, you’ve probably typed in authenticator codes a hundred times a day, just...
Devlog video about "HomeGrown", the casual farming game I'm creating using my own engine. Support the channel on Patreon and get access to the game & code for Homegrown, the city-builder, and Equilinox: https://www.patreon.com/thinmatrix Play my previous gam…
Direct link to a PDF on Anthropic's CDN because they don't appear to have a landing page anywhere for this document. Anthropic's system cards are always worth a look, and …
One of the most harmful pieces of advice is to …
Earlier this week, it was discovered that the Chicago Sun-Times and the Philadelphia Inquirer had both published an externally-produced "special supplement" that contained facts, experts, and book titles entirely made up by an AI chatbot. There's been a lot wr…