Patterns for Building Cybersecurity Evals
A sandboxed target, inputs that influence task difficulty, tools, and a grader.
Applied scientist writing on ML systems, recommendations, and building with LLMs in production.
A sandboxed target, inputs that influence task difficulty, tools, and a grader.
Build a threat model, discover vulnerabilities, verify, triage, and patch.
Context as infra, taste as config, verification for autonomy, scale via delegation, closing the loop.
An eventful year of progress in health and career, while making time for travel and reflection.
Label some data, align LLM-evaluators, and run the eval harness with each change.
Based on what I've learned from role models and mentors in Amazon
An LLM that can converse in English & item IDs, and make recommendations w/o retrieval or tools.