Author of The AiEdge newsletter; writes on machine learning systems and LLMs.
A reranker takes a question and a shortlist of retrieved passages, reads them again, and changes their order before a large language model (LLM) receives its context. The first search finds candidates across a large...
A looped Transformer reuses a group of neural-network layers several times before predicting the next token. Each pass receives the hidden representations produced by the previous pass. Those representations are the...
An AI agent can spend most of a model call reading a history it has already processed. The KV cache preserves the keys and values that a causal Transformer computed for earlier tokens, at each attention layer. Later...
A hybrid LLM processes a sequence with more than one kind of memory. Some layers update a compact running state as tokens arrive. Other layers retain representations of individual positions and consult them through...
A persistent agent keeps a task moving after the model's current context, the worker process, or the execution environment has ended. It does that by saving enough information outside those components to reconstruct the...
A self-improving agent uses evidence from earlier tasks to change something retained for later tasks, then demonstrates better behavior on new runs. The change might be a prompt, a tool, a reusable procedure, or the...
DeepSeek V4.1 Flash generates text one token at a time. A token is a piece of text, often a word or part of one. To choose what comes next, the model transforms the text already available through successive processing...
An AI agent does more than write its final answer. It chooses a tool, reads the result, decides whether to continue, and sometimes has to admit that an action's outcome is unknown. TypeSafe released Jev in September...
Jev is a decision model whose interface asks bounded questions about a supplied state and returns probabilities over the allowed answers. A support system might ask which queue should receive a ticket, whether a tool...
An agent can say that a refund is complete while the customer's account shows no refund. For an agent that can use tools, a convincing reply is only one piece of evidence. A production evaluation has to check the...
An AI coding assistant can remember that you use Python for deployment scripts and still get the next deployment wrong after you switch to Go. The problem is not that it forgot everything. It may remember both...
An LLM request router sits in front of several replicas of the same model and chooses which replica should receive each prompt. Cache-aware routing makes that placement decision before execution by considering reusable...
Test-time compute is the work an already-trained model performs for one request while producing, checking, or revising its answer. Test-time scaling means giving the model more of that request-time work, or allocating...
An LLM can return JSON that parses perfectly, matches every required field, and is still dangerous to execute. That is not a contradiction. It is the exact boundary of constrained decoding.Normally, a language model can...
An LLM can accept a very long prompt without treating every earlier piece the same way. The model represents each piece as an internal token state. Inside each attention layer, an eligibility rule determines which...
Video looks like one input, but an AI cannot inspect it as one indivisible object. A two-hour recording is a sequence of changing images, sound, speech, and timestamps. Before a model can answer a question, some system...
An AI text watermark leaves detectable evidence in the words a model chooses. During generation, a secret scoring rule helps select the next piece of text. Later, a detector with the matching key can reconstruct that...
Speculative decoding can make a coding assistant respond faster by having a small model propose several future tokens for a larger model to check together. The larger model, called the target, still determines the...
This edition explores how a model can pass validation for the wrong reason, how an agent can earn a reward without completing its task, and what makes a tool retry safe after a crash.We'll also look inside speculative...
AI Agent frameworks are becoming just as important as model training itself! I am excited to introduce you to Li Yin. She is the CEO of SylphAI and the founder of AdalFlow, a PyTorch-like open-source library on GitHub...