Measuring LLM Prefix Caching: The Cache Hit Rate Metric

Prefix caching is one of the biggest LLM serving cost levers. But how do you know it’s actually working? A look at the cache hit rate metric and why the obvious ratio is wrong.

llmperf Is Archived: Alternatives for LLM Benchmarking

llmperf is in archive mode. A look at what’s left for benchmarking OpenAI-compatible LLM endpoints, and why I wrote llmperf-rs.

Switching to Secondary Is Faster

A practical workflow that routes boilerplate and spec generation to a smaller model, reserving the flagship for review and novel problems.

Learnings of the Poor

When hardware resources are scarce, understanding memory-efficient patterns becomes essential. This post explores practical optimization techniques using iterators and streaming pipelines in Python, demonstrating how to process large files without loading them entirely into memory.

Tiny Rust Compile Tricks

Small Rust compile-time tricks that I’ve found useful in my projects

Setting Up Docker CI for Rust with cargo-dist

A guide to building multi-arch Docker images for Rust projects using cargo-dist artifacts and GitHub Actions, without compiling inside Docker.

Learning Go: January 2026 Review

I spent January learning Go and building three projects: a Japanese kanji Telegram bot, a Claude API proxy with gitleaks integration, and a Kubernetes operator for vLLM load balancing. Documenting what I’ve learnt.

Making Compression a Habit with zstd

Practical guide to using zstd compression in Python and Linux for data processing, log management, and reducing storage and transfer costs.

Rust File I/O with Compression

Rust utility functions for transparent zstd compression/decompression when working with files

2025 -> 2026

My plan for 2026

Running Multiple LLM Models with llamacpp Router

Guide to setting up llamacpp router for managing multiple GGUF models including embeddings and LLMs

Benchmarking LLM Inference: TTFT, ITL & Throughput

A practical guide to LLM inference metrics — TTFT, ITL, and throughput — how to measure them, where they mislead, and the tool I wrote when llmperf no longer fit.

Using hf tokenizers in Rust

Example of how to use Tokenizers from Huggingface in Rust

Serverless Webhooks with Cerebrium

Processing large PDF files often leads to HTTP timeouts. You send a document, wait, and the connection dies before processing completes. Cerebrium’s serverless platform solves this with custom FastAPI webhooks and built-in security.

Ansible at Home

Using Ansible to automate and manage home environments, including server setups and configurations.

Re-Writing a Text Deduplication Tool in Rust

Exploratory project to re-write a text deduplication tool in Rust, comparing performance and accuracy with the original Python implementation.