This blog post has changed tack a few times in the last two days. It started to document my attempts to replicate a relevant and actively exploited vulnerability in a piece of software I know well and ended up a rant against vendors clamouring for attention without really caring about the impact inaccurate intel can have.
I first came across Plex back when my job involved many nights away in hotel rooms and laptops with DVD/Blu-ray players were becoming less common so I wanted a way to watch my extensive Blu-ray and DVD collection whilst on the road with relying on physical media. Since them it’s become THE media player of choice in my house (especially as I’m part of that rare breed that still likes to own physical media) and I’ve been a happy user and general advocate of the software for many years.
Over the years I’ve watched Plex grow from a hobby project based on XBMC (now Kodi) to a stand alone commercial product with great support and a great reputation. From a security standpoint, whilst Plex can’t boast a perfect security history, their approach to security has also been good and it always felt like solid security architecture with generally good practices.
This is why when on 4th Feb NetScout put out a blog post (here, but they’ve since updated the content and that’s relevant, I have a copy of the text from the original here) saying the Plex PMS server (which is what your normally run on your home network) sets up a public SSDP server which is being actively used in reflective DDoS attacks, it caught my eye.
Now because I know NetScout are specialists is analysing the huge amount of data flowing through their devices (and they backed it up with real world data I had no reason to question) I wondered, could this be true, could Plex have screwed up? The starting point where these two paragraphs
It also appears to make use of SSDP probes to locate UPnP gateways on broadband internet access routers that have SSDP enabled. When a UPnP gateway is discovered via this methodology, Plex attempts to utilize NAT-PMP to instantiate dynamic NAT forwarding rules on the broadband internet access router.
https://www.netscout.com/blog/asert/plex-media-ssdp-pmssdp-reflectionamplification-ddos-attack (Before it was later modified)
When successful, this has the effect of exposing a Plex UPnP-enabled service registration responder to the general internet where it can be abused to generate reflection/amplification DDoS attacks
Now, what was in the first paragraph I knew was true, partly thanks to debugging a fauxmo issue, I came to learn much more about SSDP than I ever cared to and I had observed Plex (quite sensibly and legitimately) using SSDP and NAT-PMP to open a public-facing TCP port on a router and mapping it to a Plex port. But the second bit, to be useful in reflection/amplification DDoS attacks it means it was also opening a UDP port. The blog post goes on to confirm they have seen traffic from UDP/32414 which IS a port Plex uses on the internal network!
Now, one thing about being a Blue Teamer and Security Researcher, it means I actually have extensive logging, even on my home network, so after confirming I currently wasn’t mapping any exposed UDP ports to Plex, I went and checked both my edge firewall logs and my natty-little firewall-on-a-stick Firewalla device for recent UPnP mappings and found nothing.
Still curious, I hit my lab network, where thanks to the recent release of Plex Arcade I happened to already have the Windows version of PMS running and broke our wireshark. As expected I could see the SSDP conversation, but only for their default TCP port, no UDP ports.
At this point I reached out over twitter to both Plex and NetScout for more technical info and Plex very soon put out this statement.
Now, this was interesting, they mentioned two scenarios that if I was being generous I’d put down to “poor configuration / architecture due to a lack of infrastructure and security understanding” (or less generous, “self-owns due to negligence or stupidity”). The first being running your plex server directly onto the internet, not behind any kind of firewall or NAT based router. The second being intentionally manually allowing traffic designed for local networks to be accepted from the internet.
As my threat model doesn’t include those two cases, I was curious about the claim from NetScout that Plex itself could map a UDP port to the outside would, so I continued to try and attract NetScouts attention on twitter, but unexpectedly a Plex engineer replied up confirming my findings and they had been confused by the report (especially saying it was fixed in v1.21). She also confirmed my suspicion that nobody (including the original Chinese researchers at Baidu who first discovered it and NetScout) had ever reached out to them and attempted any kind of responsible disclosure.
And that was where this post was going to end, a question about whether we need responsible disclosure for in-the-wild attacks, a massive hat-tip to Plex for not only being super responsive, but for one of their engineers to actually jump on twitter and give some much appreciated background and a quick nod to the fact it was odd the somebody the size and standing of NetScout started this mystery around the public UDP port without ever really answering it……..
……. Then NetScout updated their blog post. On the surface it looks like a great and responsible update, they link to Plex’s own response, credit Baidu Labs with the original disclosure and include more data on their own findings. But what it doesn’t address is the claim about NAT-PMP opening a UDP port to the world.
However, intentionally or not, the insertion of two new paragraphs and a minor change in one paragraph changes the emphasis entirely. Compare the quote above with how it now reads. Now it (correctly) says Plex uses NAT-PMP to initiate port forwarding and three paragraphs later when it says “These actions can have the effect of exposing a Plex UPnP-enabled service registration responder to the general Internet” it’s now referring to the “self-own” actions outline by Plex, not NAT-PMP! Quite a change from how it read originally!
…. This is where I started to get upset
Plex’s target market is tech-savvy, but not necessarily security focused. The initial news tore through it, spreading FUD and opening the door for all kinds of well-meaning but terrible (or at least, utterly irrelevant) security advice. When in reality, the number of people impacted is exceptionally small and given most consumer internet connectivity ships with a router that includes a firewall that denies inbound traffic by default, you’re mostly limited to people running a big enough operation to afford a data centre or cloud infrastructure but not the knowledge to perform basic security, or people who have manually misconfigured their home setup (perhaps following some of the above “well meaning but terrible” security advice). In both cases, a very small but non-zero percentage of people.
It also doesn’t look great on Plex, their security is being questioned when currently (as far as we can tell) their product is doing nothing wrong or unusual. If you stuck almost any IOT device (from an Amazon Echo, to a DVR to a Sonos) directly onto the internet I’d wager it would open an UDP port for SSDP and be an willing participant in a DDoS amplification. (I haven’t actually checked if these actual devices only allow SSDP on RFC1918 addresses, but shodan currently report 400,000 SSDP enabled devices on the internet, so this is obviously not just a plex thing).
In care you’re wondering, I have a theory as to how a reputable company like NetScout can get something like the mapping of SSDP so wrong. This is all supposition, I don’t have evidence, it’s just a theory
I suspect they saw the Baidu research and looked at their own devices for it in the wild and saw it in spades. They then stood up their own PMS Server, observed the SSDP traffic on a port they’d seen in the wild, saw the PMP-NAT negation (of the TCP port) and assumed the same was happening for UDP. Alternatively it could have been taken from the Baidu article (I can’t find a translation) and either assumed to be true or is possibly a mistranslation.
Then later, when they updated their article in light of the Plex statement, the reworded in (intentionally or not) to disassociate NAT-PMP (though why still mention it?).
If this turns out to be true, I’ve lost a lot of faith in NetScout as a vendor and the quality of intel they provide. It wouldn’t have taken much to remove the NAT-PMP stuff from the advisory and in the addendum at the top to mention they no longer believe this to be relevant to the attack. Instead they let a tech community run around scared their stuff is insecure, possibly because it saves a little face and sounds a lot better and an advisory that says “2014 attack using SSDP for reflective DDoS currently in use via manually misconfigured Plex Servers” which is hardly headline grabbing. I hope that isn’t the case and it’s an honest mistake.