I joined Brian Gracely on episode 710 of the Cloudcast to discuss SBOMs, supply chain security and cloud-native security.
Here’s a quick breakdown of the topics we covered, starting at 4:45.
Starting EdgeBit
Welcome to the show Rob. Tell us a little bit about your background and what led you to start Edgebit.
I’m founder and CEO of Edgebit and we’re building a software supply chain security platform. But my background is in kind of cloud infrastructure and containers.
During my and my cofounders’ career experiences, we’ve seen that dependency sprawl is just out of control. Russell, ran security engineering at Okta previous to starting the company. And they’ve got tons of teams, tons of dependencies, lots of moving pieces, and just like doing compliance and running a security program and doing vulnerability remediation is just really hard at scale. The scale is just getting worse and worse every single day.
Why are we discussing Software Bill of Materials (SBOMs) now?
SBOM (Software Bill of Materials) has become a big topic, especially around the cloud-native community. Software and security have been around a long time, why the uptick in discussions around SBOM now?
I think the best example to start with is yes, dependencies have existed forever and you’ve got ways of upgrading them. But look what happened with log4j.
To start investigating, ask your teams, “are we using this?” and “where are we using this?” Sometimes with one team you get an answer in 2 minutes, another team two weeks, or “we don’t know”. That’s a huge problem and still has not been fixed to this day.
I think I saw a stat where the federal government spent 33,000 hours remediating log4j and downstream security problems. Just one event — absolutely mind boggling!
What’s different about now is before everyone was doing “best effort”. Some people sort of follow some NIST guidelines. If you’re under a compliance regime, it may be strong or not strong in certain areas. The government kind of finally said, all right, we need to level this up.
We’ve got certified engineers that build skyscrapers and submarines and stuff like that. Certification doesn’t exist on the software side. The federal guidelines are going to step in and do that.
The Biden Administration put out a cybersecurity directive, which kicked off a lot of the focus in the supply chain space. And I think that’s really powerful for pushing industry to do the right thing.
Day-in-the-life of an Engineering Team with Security Tools
Let’s walk through the day-in-the-life of a typical team these days. Where are there holes in their current toolset and how are things potentially improving?
On the cloud native side, you’re probably using containers, you’ve got strong automation via some pipelines. Those teams are probably doing more DevSecOps type stuff where a smaller team is responsible for deploying and securing their own software that they run. But then on the other side, a lot of these orgs have more applications than engineers they employ — like most of the Fortune 500. And so that means that you’re not 100% on one application. You’re switching hats and you’re context switching constantly.
That means that the security tools you’re using might be piecemeal compared to some of those highly automated platforms. You may do some vulnerability scanning, but it’s “every once in a while we think about it” versus something that’s very highly automated or doesn’t exist at all.
There’s no standardization for those tools. There are some gartner categories and what we’ve observed is that a lot of those tools are very siloed. Something that runs in a production CI pipeline doesn’t actually know if that software ever got deployed — the version that was scanned or looked through. Is that the same thing is something live in production? Where did this thing come from? What is even in this thing? Nobody knows.
This is where the supply chain problem starts. And there are some emerging standards for how to do this. Mostly a Software Bill of Materials (SBOM).
An SBOM is a piece of metadata that says what’s in a piece of software. And this is actually what’s driving the federal guidelines. If you sell software to the government, you now need to provide one of these SBOMs to prove that you understand what’s in your software.
EdgeBit’s Role in Supply Chain Security
Tell us about Edgebit. Where does Edgebit play a role in the Software Supply Chain?
Brian: How does the concept of SBOM in a more dynamic world? How do we know what’s been updated, like you said, how do we keep track of inventory, what’s been changed, what versions? How does this work in a more dynamic world?
The short answer is — it’s tricky — because if you think about it, you might pull in some open source libraries and maybe they either do or don’t have their own SBOM related to them. So you either generate that and add it onto the pile that you already have, or you need to mix that in with some of your first party code that you still want to track in the same manner. And so now you’ve kind of aggregated all this metadata together and need to make sense of it.
One is doing that at scale as you’ve got a frontend and a backend that make up an application. But your customer probably wants to understand that as one piece, right? It’s two teams inside your organization, but they don’t care about that. So there’s that aggregation and trakcing piece of it.
Now that you understand what’s in your software, what are the vulnerabilities attached to these? What do we know about this? You can think of one step of enrichment is taking that inventory and saying “oh, this piece of code has five known vulnerabilities”.
Maybe you care about only if they’re high and critical, etc. You have an SLA that you want to fix those within a certain timeframe. Then the last piece is — how exploitable are those vulnerabilities? A lot of the times, a lot of this code is dormant.
So if it doesn’t run as part of your product, but it is packaged in there, that’s a good thing for your customers to know because they don’t have to worry about that. That whole set of problems, multiplied by every app and every team and every front end, every component, every microservice becomes a huge surface area for just looking at this metadata, aggregating it and understanding it.
Taking Targeted Actions
Edgebit take a broad view of a company’s software landscape, but a narrow view of action. Less of a boil the ocean approach. How do you find this approach is appreciated by developers vs. security teams?
The two stakeholders in this world are a central security function and then developers/engineers that are actually writing code.
The central security function — this matches my cofounder Russell’s experience at Okta perfectly — are not experts in each application. If you have 100 applications, they can’t possibly understand the security content of those. So they’re just throwing tickets over the wall or they say “go investigate this.”
The engineer looking at the ticket asks: “does this matter to us?” and “do we run this code?”
Solving that problem x100 or x1000 engineers is really tough and time consuming. That roughly falls into the category called “vulnerability management”. Are we at risk from this issue and now that we understand it, how do we fix it? And did we actually fix it?
We want to help security teams and developers have a frictionless experience with that problem. For security teams, it means having a place to detect and actually figure out what’s real, what’s the prioritization of these threats, and then go get engineers to fix them.
Context for your Engineers
And then for developers, they hate change. They just want to stick inside of their pipeline and in their IDE. We want to be as close to that process as possible.
What we do is we take context from production about how this code is actually running — in the Linux kernel — and then give that context all the way back to the very beginning of that pipeline. Shifting left as far as you can go, to the engineer when they’re upgrading a dependency. Show them that you’re going from version one to version two that either fixes or introduces these problems.
And tell them: this code actually is running in production right now. The change that you’re making here is more weighty. It matters more than some other dormant piece of code.
And so that’s either, yes, checkmark, you remediated this thing, let’s get it live, or careful, you’re about to introduce this critical vulnerability. Let’s not put that live and then fix it.
In the past, before it was called software supply chain, there’s always been lots of tools that would go look at your container registry or your repositories and run some vulnerability scanning. They give you these huge reports with green, yellow, red stuff in it. But people would get those big reports and see hundreds and hundreds of things on here.
It’s a fire hose and so you have a limited capacity of things that you’re going to fix and that’s just a function of the time and engineers that you have. And so let’s say you can only fix ten items a week. You obviously want that to be the most impactful ten items.
And so that’s what we try to get to your engineering team and for your security team to understand that. And I’ve talked to folks that some of these large enterprises are tracking 100,000 to a million known vulnerabilities that they know of right now. You’re never going to burn that backlog down. You just need to be ruthlessly prioritizing what you are going to fix.
And the interesting thing about this is, due to the supply chain regulations, now you need to communicate that information to your customers and to other parties. It’s not a good look when you’ve got 100,000 known vulnerabilities, but if you can prove that a vast majority of those are dormant code or we’ve got this remediation plan in place, it’s a lot less scary to start sharing that information.
Where are SBOMS and Supply Chain Going Over the Next Year?
What are some of the ways you expect to see the SBOM and Software Supply Chain over the next year?
There’s always some standard work that’s going to be happening since the government regulations right now are pretty fuzzy, kind of on purpose. I think they’re going to crisp up as they walk through each successive federal agency. The executive branch is handing stuff down to the Office of Management and Budget, and then NIST gets involved, and then it’s going to keep coming all the way down. What I’m excited about is it’s just a matter of time before this makes it into SOC2 and ISO 27001 compliance — more of the commercial security standards.
Those changes are going to impact basically the entire software industry. Every SaaS is SOC2 compliance. So keep your eyes peeled for that coming down. The entire space has been kind of optional right now, as some of the NIST guidelines already cover this and some folks just optionally implement some of what NIST says.
But overall, if you think of where we are as an industry — think about the early days of the Internet and SaaS providers — before everybody had a status page. We’re kind of like that in the security realm. We aren’t getting enough infromation about issues.
Like when log4j comes out or Heartbleed. Which services have fixed and remediated Heartbleed? Am I safe to start using that SaaS service again?
If I’m worried about my data security right now, there are no status pages for security. It just doesn’t exist. There’s no metadata, there’s no way for you to make a correct assumption. What’s going to start changing is these SBOMS — as you communicate them and aggregate them and send them to your customers.
People have a better way to make a decision about something related to the security of something that they don’t own anymore. And as you consume more and more SaaS, this becomes more and more critical.
Enjoy the episode and I hope to be back on the Cloudcast for a third time soon!
