Removing 3rd Party CVEs

Jeremiah: This will cause so much brand confusion. People will think we’re doing source code analysis. We’re a vulnerability management company.

‍

RSnake: No, I’ll make it clear that we don’t actually want to get into this business.

‍

‍Jeremiah: Okayyy…

‍

Ahem! A quick note to the reader. Root Evidence does not want to be in the business of static code analysis. We are a vulnerability management company. But we do have things to say about application security, given our decades of experience in this space. And with that, let’s get started!

‍

There are around 350,000 CVEs, many of which are in third-party code. These vulnerabilities almost universally do not matter at all, because the vast majority are never exploited. But they do waste an inordinate amount of time for VM teams who are required to fix them to be compliant, while the vulns that do require attention are ignored.

‍

For instance, let’s say you have a minor flaw in some library that isn’t externally facing. Should you fix it? If no one has ever exploited it before and it isn’t even customer-facing, why would you? And yet, there could be a number of compliance standards, policy documents, or customer requirements that may demand you do. So, do you waste time on this minor flaw that isn’t even exploitable, or has never been exploited in the wild? Or do you refocus your energies elsewhere? It is Root Evidence’s thesis that you should always focus your energies where they best improve your actual security, not wasting time on non-issues.

‍

A couple of weeks ago, Jeremiah Grossman and I had a very interesting conversation with Drew Hintz at OpenAI, where he explained that there is a theoretical new strategy for companies to just re-write all of the 3rd party code libraries so that they were no longer vulnerable. Jeremiah and I stopped in our tracks. It seemed so obvious to us what the ramifications were when he said it. Drew said there were a couple of companies doing this and that it was really high-dollar consulting. To Jer and I it seemed like something worth thinking about.

‍

We began to look into it, and while very interesting, we do not believe there is a moat here, and that the likes of OpenAI Codex and Anthropic Claude can easily build this technology into their products and wipe out anyone who attempted to monetize it.  But it was worth the research nonetheless. So, to help get your head wrapped around it, let’s discuss the ship of Theseus.

‍

For those of you not familiar with Greek philosophy, let’s say you take King Theseus’ ship, and you replace one board. Is it still the same ship? What if you replaced two? What about 50? What about every board? Is it still the ship of Theseus, or is it a different ship? It is a deep philosophical question.

Instead of a ship, what if it were code? If I take a 3rd party library, and I replace one line of code, is it the same library? What about two lines of code? What about 50? What if I replace every function but the one I am leveraging? What if I even modify each of those lines? Is it the same library? This has some HUGE implications for security, it turns out.

‍

Let’s talk about the good and bad aspects of this. Good first:

‍

1. It makes you compliant because the vulnerable library is no longer there.
2. It removes future exploits since the programmer won't update to the next version with that vulnerability in it anymore, since there is nothing to update to, it is now being first-party code.
3. If attackers take over the code-base somehow, the programmer won’t ever download and run it in the future since they no longer use it. The supply chain attack therefore disappears.
4. It removes functions within a library that isn't used. That code may have vulnerabilities in it that could theoretically be used elsewhere at a later date, but is no longer a risk since it doesn’t exist.
5. Removes libraries that are unused by validating that there are simply no functions that are called from said libraries. Sometimes these are just left-over artifacts from previous developer tests that don’t belong in the code any longer.
6. It removes real vulns by simply re-writing source code without the same vulnerable code path.
7. Obscures vulns in some cases where the vulnerability may still exist, but is not instantiated the same way and, therefore automated attacks against that CVE will no longer work.
8. In the worst case scenario, the vulnerable code is identical and, the code is still vulnerable, but the compliance team won’t be able to tell because it’s more or less been renamed, making you compliant. (I consider this the most BS upside, but it’s worth mentioning because these vulns are effectively never used, but compliance mandates their fix. So, it can allow companies to focus on the vulnerabilities that matter.)
9. It may remove some copyright/copyleft issues since you are no longer using the original code that forces you to inherit the copyright/copyleft, allowing the business to use the functionality without the code.

‍

Now the bad:

‍

1. If the code isn’t substantially modified and the vulnerability persists in the exact same way as the original code, and because the programmer is removing the original 3rd party code, auditors can’t use npm audit (or equivalent) to identify the flaws. So, the vulnerable code will persist even if there is an update for it. It effectively requires the use of SAST, as a result, if you actually want to remove the vulnerability.
2. It could introduce new bugs/flaws/performance issues.
3. It could cause issues with copyrights that do not allow for derivative works. There is a complicated set of case law for derivative works that I won’t even begin to have a stance on, and you really should consult a lawyer.

‍

Okay, so there are more positives than negatives, even if some of the positives are questionable. Either way, it is interesting enough for me to do some research on, so I spent a few days putting together a new skill.

‍

Introducing… Lib-Theseus.

‍

Lib-Theseus is a skill designed to be used by Claude Code or OpenAI Codex, that identifies 3rd party libraries in your code, audits their use, and replaces them. It has a number of steps, and it’s worth explaining how it all works, step by step, because this has huge implications for those who use it.

‍

Step 1 is identifying the 3rd party libraries in the codebase.

‍

Step 2 is studying the code to see how the 3rd party code is used (or not) and propose how to re-write the changes requested by the LLM.

‍

Step 3 is developing test cases, abuse cases, and performance test cases.

‍

Step 4 is to write out the code changes identified/recommended.

‍

Step 5 is a testing phase, and if they pass, move on and if not, they loop back and continue making edits until the test cases are all passed.

‍

Step 6 is writing up the PRD to explain what happened and inform future LLMs how to use the new API endpoints that were written.

‍

Step 7 is the cleanup phase, where old code that was downloaded is removed so it is no longer on disc and therefore would not be found in SBOM-esq checks.

‍

So? How well does it work, you may be asking.  After running it against a rather large internal project that I developed, and therefore know quite well, here is the state of the sample codebase I set up, to see how it would handle it without any additional context on the test code.

With zero additional prompting, Lib-Theseus chose to re-write 14 modules out of 18. The other modules it chose not to re-write were Electron, Electron Packager, Electron Universal and node-pty, with the explanation that it considered these modules platform risks and not 3rd party risks. That is a reduction of approximately 78% of 3rd party code libraries with no additional prompting in this test. The only additional issue I saw was some noticeable UI bugs that needed to be fixed, but all were easily solved within 6 additional prompts. An overall huge improvement in 3rd party security with minimal bugs that were easily addressed. I feel confident that with some additional prompting/prodding, it could have reached closer to 100% code re-written. I don’t have an opinion about introduced bugs, because I think that’s incredibly complicated, but if you use Lib-Theseus, I’d expect them.

‍

Software security has been focused on code security for so long; it is time to reconsider what architectural security can accomplish, and Lib-Theseus allows companies to reconsider their entire architecture, ditching the old hacky way that we used to leverage 3rd party code with a brand-spanking new shiny… hacky solution. This is really only possible at scale due to the rise of AI-enabled development software, and therefore, potentially a huge win for the defenders that really wasn’t possible before without enormous dev team resources.

‍

This is something every dev team on earth can just install and use, and it could easily get baked into OpenAI or Anthropic. Therefore, we decided to make Lib-Theseus open source, to let the community figure out what to do with it, fork it, make it better, etc.

‍

But with all that spare time you’ve created by removing all that 3rd party code, we’d recommend focusing on the vulnerabilities that DO matter.

‍

You can download Lib-Theseus here (use at your own risk). Let us know what you think!