Using AI to solve hard security problems. Here’s a reasonably accurate profile of my other interests.
- Thanks be to God For the birth of my daughter, Magdalene Aurelia. Perhaps it’s a bit unfair to write on the arrival of my fifth child and not the other four, but I will not deny an occasion to practice gratitude: for my wife carrying and delivering this sweet girl for my mother caring for the other kids while we recover for my friends and the extended support they will provide in the near future for my coworkers making it easy for me to take time away And for the gift of life, which I do not take for granted.
- Who bankrolls the AI agent swarm? In the wake of the OpenAI-Hugging Face incident (OAI-HF), Anthropic CEO Dario Amodei recently expressed his concern that recursive self-improvement now allows AI to advance drastically faster, and consequently that in 6–12 months a swarm of more capable agents (i.e., than those involved in OAI-HF) could take over the entire internet with a persistent botnet. Could this plausibly happen? Let’s start with the basics. A botnet typically refers to a fleet of compromised machines around the world that have been infected with malware. These are often low-power IoT devices like your home router, and the malware is a small program (at least, small enough to reliably execute on whatever hardware it has compromised) that turns the device into a “bot” that listens for instructions from a command-and-control server (e.g., to flood some other server with unwanted traffic in a DDoS attack). Sometimes the compromised machine is more capable (an AWS EC2 instance) and sometimes the bot does other stuff (like using the machine’s processing power to mine cryptocurrency), but if we’re painting broadly, this is what you’d expect from a botnet: a widely distributed network of infected machines whose malware is powered by the (often modest) compute power resident on the machine itself.
- Nothing is just It’s just a symbol. It’s just a clump of cells. It’s just a social construct. The next time you reach for this diminutive qualifier, ask yourself: What obligation are you avoiding? When does “just” ever serve justice? The use of the word betrays its own intent; it implies the very thing it tries to ignore. The carefully (atrociously) separated parts still cast a shadow of the whole. Nothing is “just.” Art should evoke humility, not hubris. Learning to decompose something doesn’t entitle you to devalue the composition. Beauty demands a response—even responsibility.
- Pelican, or pelican't? A hint at Claude evals I’m a big fan and frequent user of Claude’s deep research feature, so it caught my eye when the task title from my research subagent was a bit longer than usual. I’d asked about WWI causes and got the research title “Causes of WWI research disabled warnings about…” with some unexpected trailing text at the end. Confused, I inspected the title with Chrome DevTools and found that the h2 header overflowed far past the truncated text shown in the UI. The full title discussed re-enabling system update warnings for Windows and macOS—clearly unrelated to my prompt!
- You can just say it There is a weird collection of arguments for appraising the value of humans and their creative artifacts. It usually goes something like this: In the age of AI, we should still prefer humans in certain roles because AI could never perform the tasks required for that role. Or, a human can at least do it better. Or, perhaps the output from a human and AI may look similar, but human output is preferable for subtle stylistic reasons that an AI cannot reproduce. Or, at least the AI cannot reproduce it consistently. Observe the scuff marks around the base of the goalpost from constant movement. Allow 28 days for the concrete to cure.
- O(N) the Money: Scaling Vulnerability Research with LLMs Let’s go back to the basics and consider a very simple goal: What if we could find and fix all the vulnerabilities in open source software? This is a pretty ambitious goal, admittedly, but…
- Slice: SAST + LLM Interprocedural Context Extractor Earlier this summer, Sean Heelan published a great blog post detailing his use of o3 to find a use-after-free vulnerability in the Linux kernel. The internet lit up in response, and for good…
- I used o3 to profile myself from my saved Pocket links Welp, Pocket shuts down tomorrow despite our pleas for it to stay. While migrating1 all of my saved articles, I noticed that I’ve got almost 900 saved articles spanning nearly 7 years. That’s a goldmine of stuff-I-like data! Some quick analysis using xsv2: 𝄢 unzip pocket.zip && xsv headers part_000000.csv 1 title 2 url 3 time_added 4 tags 5 status 𝄢 xsv sample 1 part_000000.csv | xsv flatten title The Uncertain Future of American Libraries url https://mek.fyi/posts/the-uncertain-future-of-american-libraries time_added 1678243022 tags status unread 𝄢 xsv count part_000000.csv 878 𝄢 xsv select time_added part_000000.csv | xsv stats | xsv select min,max | xsv flatten | while read header value; do echo -n "$header " date -I -d "@$value" done min 2018-07-12 max 2025-06-26 Experiment Recalling Simon Willison’s recent geoguessing challenge for o3, I considered, “What might o3 be able to tell me about myself, simply based on a list of URLs I’ve chosen to save?” I opened a temporary chat with o3 (doesn’t use ChatGPT’s memory feature) and prompted:
- Using LLMs to solve security problems TL;DR: Raink—a novel, general-purpose listwise document ranking algorithm using an LLM as the ranking model—can be used to solve non-trivial security problems. A very simple explanation of how the Raink algorithm works: Split big list of items into small groups (e.g., 10 items per group) Ask the LLM to rank/order each small group according prompt relevance Shuffle everything and repeat this process several times Keep track of how each item performs across different groups Focus more attention on the items that consistently rank highly The approach is fast, cheap, and handles far more items than would fit in a typical LLM’s context window. Key features of the algorithm and its implementation:
- Hard problems that reduce to document ranking There are two claims I’d like to make: LLMs can be used effectively1 for listwise document ranking. Some complex problems can (surprisingly) be solved by transforming them into document ranking problems. I’ve primarily explored both of these claims in the context of using patch diffing to locate N-day vulnerabilities—a sufficiently domain-specific problem that can be solved using general purpose language models as comparators in document ranking algorithms. I demonstrated at RVAsec ‘24 that listwise document ranking can be used to locate the specific function in a patch diff that actually fixes a vulnerability described by a security advisory, and later wrote on the Bishop Fox blog in greater defense of listwise ranking by publishing a command-line tool implementation (raink) to prove the idea.
- Why I left Air Force Cyber I wanted to write this before I’ve been out longer than I’d been in. I agonized over my decision to leave the Air Force—largely because I experienced it as an issue of professional identity. Why I joined After wrestling at a high school math and science academy, I wanted to keep up physical fitness and develop leadership skills in a technical environment. I also needed financial assistance to fund college. I had Army, Marine, and Naval officers represented among my grandparents—but rather than inspiring any overwhelming patriotism in me, I think military lineage simply enabled me to consider the possibility of going military at all. I considered attending a military service academy but chose a more balanced experience with Air Force ROTC at the University of Virginia.
- zip snip: Social engineering attack using a .zip domain Google recently launched the .zip TLD, and Twitter has been up in arms. It’s hard to imagine that .zip domains would be used for anything legitimate, and would instead give way to more creative phishing attacks. Accordingly, I quickly snagged source-code.zip while these domains were rapidly being bought up, and showed an example of how it might be used to trick a software developer into downloading a malicious ZIP file—yet there are still some that quite reasonably ask, “Should we really be so concerned about .zip TLDs?”
- How I keep up with new content Or, using a content aggregator like it’s 1999. Daniel Miessler writes a lot—and a lot of that lot is about RSS. Daniel inspired me to curate my media intake with RSS (and also more generally to start writing learning in public). Here are a few RSS-related tools and principles I’ve picked up over the past 6 months. It essentially boils down to discovering new content sources via Twitter, etc., and using a content aggregator (and supporting tools) to bring those many sources into a single view.
- Zero-knowledge appointment scheduler I recently (finally) started using online appointment scheduling software. There are many players in this space, but Calendly is the most popular and featureful. It’s awesome—but in order to use it, you have to give it direct access to whichever calendars you want it to check for conflicts. That means access to all sensitive details like online meeting URLs, attendee information, etc.—and it gives me the creeps. This isn’t unique to Calendly as most booking software works exactly this way. Additionally, Calendly’s free tier is limited in that:
- Free as in (tier) I tend to push a product’s free tier pretty hard. I’ve been using the same free Dropbox account for 11 years (originally 2 GB, but permanently upgraded to 22 GB through various referrals and promotions long ago). I have several free Zoho email accounts grandfathered into supporting custom domains, SMTP, and email forwarding (they still offer a free tier with custom domain support, but without those latter features). I use free Inoreader while compensating for its restrictions by layering on free equivalent RSS services like Kill the Newsletter!, siftrss, Grepfeed, and Pocket.