In a simple penetration test the tester is presented with a target, say a web application, and attacks it from his workstation. However, systems can be far more complicated than standard web applications and can span multiple servers with different accessibility. Komodo routinely tests such systems. This blog details some of the challenges we faced during one such test and the path we took to overcome them.
Background
The system we tested was meant to be embedded in a corporate network. It is composed of an endpoint agent and a cloud-hosted, “event aggregator” server which gathers events from endpoints, analyzes them and presents them. In addition, special events are sent to the manufacturer for analysis through a RabbitMQ message queue. A very basic diagram of the system:
During the pentest, we tested the endpoint and event aggregator in a black box fashion. While this proved fruitful and many vulnerabilities were found, some components of the system could not be tested. Namely the cloud RabbitMQ server as it can only be accessed from the event aggregator servers. And so, in the second part of the test, we simulated a scenario where an attacker gained access to an event aggregator server of an organization and were asked to test the RabbitMQ implementation.
Initial probing
We were given an SSH connection with a root-capable user to the aggregator server. The first step was to figure out what processes/services on the aggregator server communicates with the RabbitMQ server. Using the netstat utility (netstat -pa), we quickly found an outgoing connection to port 5672 by a Java application – a port used by RabbitMQ. A port scan of the server didn’t reveal any other services, so we were left to probe the implementation itself.
First, we needed credentials. Digging into the Java VM using the ‘JVM’ command, we found the JAR file making the connection. Searching the configuration file for credentials proved unsuccessful as we found they were dynamically retrieved using an encrypted connection. We decided to pull them from memory.
We started by disassembling the JAR file and locating the object holding the credentials for the AMQP connection. We then continued to dump the process’s memory, which was straight forward. Since it runs under the Java VM, we used the ‘jmap’ utility. By loading the memory dump into a Java memory analyzer and querying for the desired object we found the plaintext credentials:
We used the newly obtained credentials and probed the RabbitMQ instance for misconfigured permissions, for example read access in shared events queues or write access for configuration queues, but came up empty handed. The management interface for the RabbitMQ instance was closed as well. Since the RabbitMQ instance appeared properly configured, we started to probe the application reading from it. This is where things got complicated.
Let’s deep dive
We started by sending a standard event. We generated an event on the endpoint and watched as it appeared at the backend dashboard. We now wanted to modify the event’s values midway. However, the message sent to the queue was packed in a way that made reproducing it difficult. We needed to debug the production JAR on the remote server and copy the routines and data involved.
The JAR was started by an ‘init.d’ script. We altered the initialization script to include a Java debug server on port 5005 by including the parameter:
“-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005”.
We then tunneled the port through the SSH session, which allowed us to connect to the debug server on the remote host. We placed a breakpoint right before the event was sent and through a slow and sometimes agonizing process, managed to copy various relevant parameters and routines to our own basic Java program:
We now had complete control over the event’s parameters. And indeed, shortly afterwards, we discovered the dashboard application was not verifying some of the event’s parameters against the RabbitMQ user used to deliver it – for instance we could inject events under the name of other customers.
Our journey through the cloud concluded with the discovery of this vulnerability – the backend systems were mostly out of the test’s scope, but we sure did enjoy the ride!