Posts

TL;DR: Raink—a novel, general-purpose listwise document ranking algorithm using an LLM as the ranking model—can be used to solve non-trivial security problems. A very simple explanation of how the Raink algorithm works: Split big list of items into small groups (e.g., 10 items per group) Ask the LLM to rank/order each small group according prompt relevance Shuffle everything and repeat this process several times Keep track of how each item performs across different groups Focus more attention on the items that consistently rank highly The approach is fast, cheap, and handles far more items than would fit in a typical LLM’s context window....

There are two claims I’d like to make: LLMs can be used effectively1 for listwise document ranking. Some complex problems can (surprisingly) be solved by transforming them into document ranking problems. I’ve primarily explored both of these claims in the context of using patch diffing to locate N-day vulnerabilities—a sufficiently domain-specific problem that can be solved using general purpose language models as comparators in document ranking algorithms. I demonstrated at RVAsec ‘24 that listwise document ranking can be used to locate the specific function in a patch diff that actually fixes a vulnerability described by a security advisory, and later wrote on the Bishop Fox blog in greater defense of listwise ranking by publishing a command-line tool implementation (raink) to prove the idea....

I wanted to write this before I’ve been out longer than I’d been in. I agonized over my decision to leave the Air Force—largely because I experienced it as an issue of professional identity. Why I joined After wrestling at a high school math and science academy, I wanted to keep up physical fitness and develop leadership skills in a technical environment. I also needed financial assistance to fund college....

Google recently launched the .zip TLD, and Twitter has been up in arms. It’s hard to imagine that .zip domains would be used for anything legitimate, and would instead give way to more creative phishing attacks. Accordingly, I quickly snagged source-code.zip while these domains were rapidly being bought up, and showed an example of how it might be used to trick a software developer into downloading a malicious ZIP file—yet there are still some that quite reasonably ask, “Should we really be so concerned about ....

Or, using a content aggregator like it’s 1999. Daniel Miessler writes a lot—and a lot of that lot is about RSS. Daniel inspired me to curate my media intake with RSS (and also more generally to start writing learning in public). Here are a few RSS-related tools and principles I’ve picked up over the past 6 months. It essentially boils down to discovering new content sources via Twitter, etc., and using a content aggregator (and supporting tools) to bring those many sources into a single view....

I recently (finally) started using online appointment scheduling software. There are many players in this space, but Calendly is the most popular and featureful. It’s awesome—but in order to use it, you have to give it direct access to whichever calendars you want it to check for conflicts. That means access to all sensitive details like online meeting URLs, attendee information, etc.—and it gives me the creeps. This isn’t unique to Calendly as most booking software works exactly this way....

I tend to push a product’s free tier pretty hard. I’ve been using the same free Dropbox account for 11 years (originally 2 GB, but permanently upgraded to 22 GB through various referrals and promotions long ago). I have several free Zoho email accounts grandfathered into supporting custom domains, SMTP, and email forwarding (they still offer a free tier with custom domain support, but without those latter features). I use free Inoreader while compensating for its restrictions by layering on free equivalent RSS services like Kill the Newsletter!...