By now, hopefully everyone in the software development ecosystem is aware of the grave security risks that lie in unmanaged open source vulnerabilities. Most of us are familiar with at least a few security vulnerabilities that made headlines, but not many realize that those constitute only a handful of the 5000 vulnerabilities that are disclosed each year.
Considering that number will continue to grow, and that publicly disclosed vulnerabilities offer hackers an easy shot at hitting the jackpot, having a vulnerability remediation process in place is a must for any organization that cares about their customer’s safety, not to mention their reputation.
Before we dive into the essential steps of the vulnerability remediation process, it’s important to understand what it consists of and why having a vulnerability remediation process in place should be mandatory for every organization.
In order to always remain one step ahead of malicious attacks, DevSecOps professionals need to have a process in place to track and manage known vulnerabilities. Once a tedious, time consuming manual process, today teams can continuously track their organization’s software inventory with automated tools, and match them against the various security advisories, databases, or issue trackers in the software development space, so that they can ensure that their services and products are not relying on risky code. Then, if tracking results show that they are, they need to locate the vulnerable component and mitigate the risk in the most efficient way possible.
These steps might sound simple, but without a vulnerability remediation process that all stakeholders have signed off on, an organization might find itself a day late and a dollar short in their race against the hackers.
The very first step in the vulnerability remediation process is knowing what you’re working with. That means continuously tracking your software inventory to know which software components you are using, and what might need immediate attention.
SAST & DAST & Pen Testing, Oh My!
When it comes to proprietary code, SAST (Static application security testing) and DAST (Dynamic application security testing) tools that analyze source code or applications in their dynamic running state enable early detection of any vulnerabilities that could risk an organization’s security.
In addition, pen testing your organization’s applications periodically to assess your products’ security is also a good way to locate any security vulnerabilities that might be hidden in your code, before releasing it.
While these tools combined provide a good solution for keeping your proprietary code vulnerability-free, they don’t cover open source components.
With open source security vulnerabilities, Software Composition Analysis (or SCA) tools are the way to go. This family of automated tracking tools enable software developers and security professionals to automatically detect all open source components in an organization’s systems, identifying them as soon as they are added to the code base, and alerting admins when risky open source components are added, or when new vulnerabilities are published in one of the various security advisories or databases. This is crucial, since most popular open source components are supported by a large community that continuously collaborates on checking and updating the code, and security vulnerabilities are sometimes found in versions that have been used and trusted for years. Without a automated tool continuously tracking your open source inventory and matching it against updated security advisories, these vulnerabilities are easy to miss and a mess when discovered too late.
In addition to continuously tracking your teams’ open source usage, SCA tools allow you to adopt the recommended DevSecOps practice of shifting security left, by locating issues as early as possible in the software development life cycle, significantly lowering the cost of fixing them, not to mention avoiding the chaos of detecting security vulnerabilities close to release dates.
Good news: you know what’s in your code, and what requires your attention.
Bad news: So many vulnerabilities, so little time. There are only so many hours in a developer’s day, and your team is most probably racing through a sprint. There is no way your team can attend to all the issues that your superior tracking processes have uncovered.
How can you make it to the finish line with a solid and secure release? When in doubt, prioritize.
When working with proprietary code, organizations need to have prioritization policies in place. They need to evaluate the risk of the vulnerabilities they found by looking at the organization’s system configuration, the likelihood of an occurrence, its impact, and the security controls already put in place.
Ideally, an organization's critical systems and components, the ones that will be most damaged by an attack, will be separated beforehand.
Once all these are assessed, and it is clear to all DevSecOps leaders whether an important system is being threatened, and the impact of an exploit might be, remediation efforts can be prioritized and the workload can be distributed among team members to ensure their products and systems are risk free, without having to halt the development lifecycle.
This can be more of a challenge. As we already know, the information about the vulnerability and its remediation might be found in a number of repositories, advisories and bug trackers, so a tool that aggregates all open source vulnerability information from across the open source bazaar continuously will help your organization get the most accurate information.
However, having a vulnerable open source component in your software doesn’t necessarily mean that your code is affected, since you might be not be using the vulnerable file or function in your system architecture.
In order to prioritize remediation wisely, you need to make sure you have a complete and accurate overview of the open source vulnerabilities in your code base, how they work, and what they impact. This is where the third generation of SCA tools, with their Effective Usage Analysis, come in. Effective Usage Analysis helps teams gain insight into which specific functionalities in open source components actually impact your code, providing a trace analysis that pinpoints the exact location of the effective vulnerability, and which other components it impacts.
These actionable insights can cut down the volume of security alerts by 70%, allowing development teams to prioritize the remediation of open source vulnerabilities that carry real risk, without having to spend too much time searching for the insecure function, or replacing important libraries that pose no real threat.
Once you’ve established which security vulnerabilities require the most immediate attention, and mapped out a timeline and workplan for the fix, it’s time to roll up your sleeves and get fixing.
Vulnerability remediation in proprietary code requires that you consider the root cause of the security vulnerability when you attend to the fix, and includes both manual and automated processes. Remediation to proprietary code might include: patching, disabling the vulnerable process, removing a vulnerable component, updating system configuration, or updating the platform or service that your teams are using.
All these can serve to provide a good permanent solution to a security vulnerability. In all cases, it’s best to test the update or fix outside of your production environment, to ensure the fix doesn’t cause any regression in your system or products. After the fix or patch are deployed, it’s important to continue monitoring it to ensure its security, and that it doesn’t affect other processes or configurations in the system.
In addition, newly discovered security vulnerabilities might raise a need to add more security to your systems perimeters. However, it’s important to keep in mind that having good perimeter protection doesn’t eliminate the need to monitor and manage security vulnerabilities, patching, updating and reconfiguring when necessary.
The remediation process for open source security vulnerabilities at the fixing stage brings additional challenges that organizations need to address. The collaborative and decentralized nature of the open source community means that fixes to vulnerabilities in open source components can be published across a number of advisories. Some projects are quick to offer detailed documentation and support, while others require a little more digging through forums and message boards.
This makes manually searching for a fix for an open source vulnerability virtually impossible, or at the very least extremely time consuming. Automated SCA tools will alert admins of available fixes as soon as they are published. Sometimes there will be more than one fix offered online, and an advanced SCA tool will recommend the best one.
After prioritizing the security vulnerabilities that actually require a fix, there are a few methods you can use to remediate, depending on the effective usage and trace analysis results that your automated SCA tool provided.
Patching is usually the go-to solution for an open source vulnerability, since it attends to the heart of the issue without requiring updating any additional libraries or components. However, you might find some patches incompatible with other functionalities in a component, or sometimes a patch simply isn’t available.
The next best thing would be updating the specific source file. This solution is also minimally invasive, promising little risk to the rest of your libraries. Unfortunately, updating your source file won’t cut it if a vulnerability is threatening more than just that file. And what happens if an update isn’t available?
In cases where updating one source file isn’t a viable solution, it’s best to update the entire open source component. A newer version of a component may even provide an upgrade to the quality and security of your product. But in some cases, it might be incompatible with other components in your system. That’s why it's best to test that solution in a sandboxed environment to ensure your good intentions don’t cause more harm than good.
If all of the above aren’t available and you have trace analysis in place and you can locate the functionalities that are putting your code at risk, it might be relatively easy to find and block the use of the vulnerable methods within the components, or change the system configuration to avoid the security risks without impacting your environment.
Security vulnerabilities, both proprietary and open source can put the most innovative products and services at risk of attack. As the application security ecosystem continues to evolve along with the DevSecOps approach, allowing us to integrate automated tools throughout our development lifecycle, we have many tools available to us to address the ongoing risk of software security vulnerabilities and uphold security.
It’s important that we as development organizations have the policies and processes in place to leverage the innovative software composition analysis and testing tools out there to secure our systems, our offerings, and our customers privacy.