Yash's Blog

Write

LLaMA 2: How Three Borrowed Techniques Fit a 70B Model on Two GPUs

Mar 15, 202615 min read

Attention Is All You Need: What the Paper's Heads Are Actually Doing at Each Layer

Mar 8, 202612 min read

Mixtral of Experts: Top-2 Routing Gives 47B Capacity at 13B Active Compute

At roughly 2.08e11 cumulative FLOPs in my own run, a dense baseline lands at 25.31 validation perplexity. At nearly the same compute budget, a sparse MoE lands at 20.98. The absolute delta is 4.324, a

Mar 22, 202612 min read

Beyond try/except: Architecting Robust Error Handling in Python Applications

Apr 23, 202511 min read

Demystifying Reinforcement Learning: A Beginner's Guide to the Math

Mar 4, 202512 min read

Exploration vs. Exploitation: A Deep Dive into Multi-armed Bandits

A k-armed Bandit Problem Imagine you're at a casino, faced with a row of slot machines (one-armed bandits), each with its own hidden probability of paying out. Your goal is to maximize your winnings over the night, but you don't know which machines h...

Feb 19, 202512 min read

Reinforcement Learning: First Principles

Initial Concepts and Core Ideas

Feb 5, 20258 min read

Command Palette

Latest articles