On Friday, F5 reported an unauthenticated remote code execution vulnerability (along with a second slightly less serious XSS bug that could also result in RCE) in the Traffic Management User Interface of their flagship Big-IP Load Balancer products (though it seems in-support customers got a heads-up on Wednesday) and they were so bad that US Cyber Command had put out a tweet saying “don’t postpone [patching] over the weekend “.
By Sunday there were work Proof of Concept exploits and “the sky is falling” type news articles across the InfoSec press.
As with most headline grabbing vulnerabilities of the type, people immediately worked on a Shodan search to tell us how many vulnerable systems were connected to the Internet.
and for the people who manage those 8460 appliances, the Cyber Command advice was sound, because they were in danger of having a REALLY bad weekend. But I’d wager most of those people are used to having bad days, because the problem here isn’t how quickly can you apply a patch (when as best we can tell, even paying customer only got 3 days advance warning of f5 publishing this) it’s that we have a mind set where the answer to every vulnerability related problem like this is PATCH PATCH PATCH! But that only addresses the symptom, not the disease. If you’re having to get your team to give up their 4th of July Holiday weekend to do emergency out-of-band patching for this type of vulnerability, you’ve already got a MUCH bigger problem on your hands
The vulnerability was in a Management Interface. In an idea world, Management Interface would only be accessible to just the people who need to access it (regardless of whether they can log in once they access it), in reality (until network micro-segmentation become commonplace) it would be at least be only accessible by trusted IT Ops staff who generally manage systems (whether they manage that specific one of not) on some kind of management VLAN). It’s nearly impossible to come up with a use case where your entire organisation should be able to access that UI, much less then entire Internet!
So, why would anyone do this? Well, the problem is either a lack of quality Security Architecture skills/oversight, or a lack of controls ensuring that good Security Architecture is being implemented. Or as Kevin “Gossi The Dog” summed it on during a twitter discussion of the problem (perhaps unfairly). DEVOPS!
The “move fast and break things” attitude of DevOps is great for rapid development, and is fine if the cost of getting the architecture wrong is only technical debt. But if the cost is having a message from US Cyber command urging you to cancel your engineers 4th July holiday weekends and then praying they can patch before you’re exploited, then you really should be focusing on both ensuring there are controls in place to ensure dangerous architectural decisions aren’t made unknowingly and on giving your DevOps engineers (or whoever you have putting management UIs on the Internet) sufficient time, resource and training to understand why when it comes to a tactical solution and a little tech debt, it could easily manifest itself as a national holiday spent away from their family doing emergency patching, or even worse, dealing with an active compromise.