Notes on private AI,
training, and deployment.

OCTOBER 19, 2026

How we cold start a 32B model in 1.1s.

GPU snapshots still take 13 to 17 seconds to spin up a 0.6B model. Lupine's checkpoint/restore engine cold starts Qwen3-32B on vLLM in 1.1 seconds. Here's where the time was going, and how we engineered around it.

Read more →

OCTOBER 8, 2026

Financial text classifiers: fine-tune, don't prompt.

Seven benchmarks, one 0.6B model, four approaches. Prompting scores 11.6 F1 on Banking77; a LoRA fine-tune of the same model scores 91.2, needs 91% fewer labels than full tuning, and answers in about 30 ms.

Read more →

SEPTEMBER 14, 2026

Training a language model for your internal wiki.

Pointing a training run at a Confluence export teaches the model every retired policy in it, with no last-edited date attached. What belongs in the weights, what belongs in the index, and the build order that keeps the two apart.

Read more →

SEPTEMBER 9, 2026

Building an agentic research assistant for a trading desk.

The concrete build: a six-stage architecture, why retrieval quality caps everything downstream, why an information barrier enforced in the prompt is not a barrier, and what a desk has to log once the output of an agent counts as a record.

Read more →

SEPTEMBER 8, 2026

AI agent failure modes, and how to catch them.

Eight ways an agent breaks in production, sorted by the stage of the loop each one enters at. The specific signal that catches each, why the quiet failures are the expensive ones, and the three layers of detection that most teams have only two of.

Read more →

SEPTEMBER 4, 2026

Evaluating agents: why public benchmarks don't transfer.

A leaderboard score belongs to a system, not a model: tasks, tools, harness, and grader all move the number, and your deployment replaces all four. Why the transfer fails, what a task-specific eval is made of, and the metrics that survive contact with production.

Read more →

SEPTEMBER 3, 2026

Synthetic data for fine-tuning: what to do when you have no labels.

Nobody has a labelled corpus of agent trajectories, including the teams shipping fine-tuned agents. Where synthetic training data comes from, why the verifier and not the generator sets the ceiling, how much you actually need, and what must never be generated.

Read more →

SEPTEMBER 2, 2026

Tool-use fine-tuning: teaching a model your internal APIs.

A frontier model has read every public API reference and none of yours. What tool-use fine-tuning changes, what one training record looks like, where the data comes from, and how to evaluate it beyond validation loss.

Read more →

SEPTEMBER 1, 2026

SFT or RL? Choosing how to train an agent.

Supervised fine-tuning teaches a model to imitate attempts that worked. Reinforcement learning fine-tuning teaches it to produce attempts that score well. Where SFT, RLHF, RLVR, and GRPO each win on multi-step tasks, and how to pick between them.

Read more →

AUGUST 31, 2026

What is agentic fine-tuning, actually?

Agentic fine-tuning trains a model on whole task trajectories, tool calls and observations included, and rewards it for finishing the job. What the data looks like, why it needs a live environment, and when it beats a better prompt.

Read more →

AUGUST 20, 2026

What is GPU orchestration, actually?

GPU orchestration is the software layer that pools many GPUs and schedules training and inference across them. What an orchestrator actually does, how it differs from provisioning and Kubernetes, and why utilization is the whole game on a fixed fleet.

Read more →

AUGUST 17, 2026

What is a model in a box, actually?

A model in a box is a self-contained AI system that trains, serves, and owns a model inside your own network. What's actually in the box, how it differs from an inference appliance and a cloud API, and when it beats renting a model.

Read more →

AUGUST 13, 2026

How to run a coding assistant air-gapped, and prove it to an auditor.

A six-step guide to deploying an AI coding assistant with no network path out: why air-gapping and SOC 2 are separate requirements, how each component runs offline, and the control-to-criterion map that turns isolation into audit evidence.

Read more →

AUGUST 11, 2026

Every way source code leaves, and how to close each path.

An on-premises blueprint for AI developer tools: the nine paths code takes out of a network, the control that closes each one, how far air-gapping actually goes, and the six artifacts that prove the boundary holds.

Read more →

AUGUST 10, 2026

How Python decides what to free, and when to go looking.

Reference counting, cycle detection, and the generational thresholds that actually trigger a collection. Plus how the free-threaded build counts allocations per thread, and what a negative count does to collection scheduling.

Read more →

AUGUST 10, 2026

Should your coding assistant run on your own hardware?

A self-hosted AI coding assistant versus a public copilot on the three axes that actually differ: what leaves your network, where the milliseconds go, and how far you can customize. With a decision table and a reference architecture.

Read more →

AUGUST 7, 2026

What it actually costs to run your own model.

The honest break-even math against a frontier API, with every assumption stated. One GPU pays for itself around 550M output tokens a month, and against the cheapest hosted models it may never pay off at all.

Read more →

AUGUST 7, 2026

How many GPUs to serve a model at sub-50ms?

The KV cache formula, why decode speed is capped by memory bandwidth, and what sub-50ms can honestly mean. With a sizing table from 8B to 70B.

Read more →

AUGUST 7, 2026

What your risk team will ask, and the answers.

The 24 questions a review board asks about an on-prem AI deployment, what each one is really asking, and the artifact that closes it. Written to be forwarded internally.

Read more →

AUGUST 7, 2026

Fine-tuning LLMs for finance: a practical guide.

How to fine-tune open-source LLMs for financial NLP: sentiment classification, entity extraction, and financial QA, using LoRA, QLoRA, and open-source tools like Unsloth and Axolotl.

Read more →

AUGUST 6, 2026

How to deploy private AI, step by step.

Deploying private AI means running training and inference entirely on infrastructure you control. Here's the five-step process, from your isolation model to verifying zero data egress.

Read more →

AUGUST 4, 2026

DeepSeek Explained

DeepSeek's R1 triggered Nvidia's biggest single-day market cap loss ever by proving frontier reasoning didn't need a frontier budget. Here's how it's trained and when to use it.

Read more →

AUGUST 4, 2026

Gemma Explained

Google's Gemma is built from the same research as Gemini but released as open weights, tuned for the best capability per parameter at small sizes. Here's how it's trained and when to use it.

Read more →

AUGUST 4, 2026

Llama Explained

Meta's Llama built the largest open-weight fine-tuning ecosystem, but its license was never truly open source and Llama 4 lost ground to Chinese rivals. Here's the full picture.

Read more →

AUGUST 4, 2026

Mistral Explained

Mistral AI's flagship went from a gated research license back to Apache 2.0 with Mistral Large 3, a 675B-parameter MoE. Here's who builds it, how it's trained, and when to use it.

Read more →

AUGUST 4, 2026

Kimi Explained

Moonshot AI's Kimi K2 trained a 1T-parameter open-weight model with zero loss spikes; K3 pushes that to 2.8T. Here's who builds it, how it's trained, and when to use it.

Read more →

AUGUST 4, 2026

Qwen Explained

Alibaba's Qwen family now runs from 0.6B up to Qwen3.5's 397B-parameter open-weight flagship. Here's who builds it, how it's trained, and which size fits your task.

Read more →

AUGUST 3, 2026

AI for Internal Tooling: 6 private AI use cases.

From code review to support copilots, here are six places private AI pays off in the tools teams build for themselves, without handing your codebase to a vendor.

Read more →

AUGUST 3, 2026

AI for Trading: 6 private AI use cases.

From signal generation to trade surveillance, here are six places private AI pays off on and around the desk, without leaking strategy or missing a latency budget.

Read more →

AUGUST 3, 2026

AI for Finance: 6 private AI use cases.

From transaction monitoring to trading signals, here are six places private, fine-tuned AI outperforms a public model API in finance.

Read more →

AUGUST 3, 2026

What is an inference engine, actually?

Training produces a model's weights. An inference engine is the separate piece of software that serves them, fast, at scale, under real request traffic.

Read more →

AUGUST 3, 2026

What is a small language model, actually?

A small language model trades general breadth for speed, cost, and control. Here's what actually makes a model "small," and when that trade is worth making.

Read more →

JULY 29, 2026

What is fine-tuning, actually?

Fine-tuning updates a pretrained model's weights to fit your task. Here's how it actually works, and how it differs from prompting and RAG.

Read more →

JULY 29, 2026

Real-time inference doesn't have minutes to spare.

Most of torch.compile()'s speedup comes from one thing: CUDA graphs. Here's how NinetyFive gets the same performance without the multi-minute warmup tax.

Read more →

JULY 29, 2026

Air-gapped LLM training: what it actually requires.

Most vendors' "private cloud" just means a VPC inside AWS or GCP. Air-gapped is a stricter, different claim, here's what it actually takes.

Read more →

JULY 29, 2026

Deployment-agnostic AI infra: why the tool shouldn't dictate where your GPUs live.

Coupling your training and inference tools to a specific cloud, data center, or region is a decision you'll regret. Here's what deployment-agnostic infrastructure looks like instead.

Read more →