We Tested the Defenses — and Found an Attack. How a pentester can shift gears into an Incident Response scenario. And why, for the client, this matters more than even the most thorough report
Forensic examiners in spite of ourselves
Ethical hacking, also known as pentesting, is always a walk along a razor's edge. You act like an attacker, but with a noble purpose: at the client's request, you find their vulnerabilities and point them out before cybercriminals can take advantage of the weaknesses in the client's security perimeter. As a rule, the work follows a well-worn path and often yields predictable results. Here's an operating system that hasn't been updated in ages. There — weak passwords. Over there, undocumented APIs. In the public cloud — forgotten test staging environments. Everything is perfectly clear.
But one day, a routine pentest suddenly went off script. In the course of our authorized "intrusion," we discovered that someone had already been inside the client's security perimeter. And that someone hadn't just dropped in out of curiosity. They had established persistence, left behind an espionage toolkit, set up access, and were pulling the strings to use the client's infrastructure for their own ends.
We decided to take on more than a pentest team usually does. What had begun as a security assessment for a report in the format of "here's a vulnerability, and here's a recommendation" turned into a genuine cyber investigation, spiced with a set of rapid-response measures to address the very fact of the hackers' presence. And so, entirely spontaneously, we found ourselves trying on the role of digital forensic examiners — a role we hadn't even contemplated when the project began.
Part one. A planned breach with unplanned findings
A few years ago, at the height of the active exploitation of the ProxyLogon set of vulnerabilities in Microsoft Exchange, our team was conducting a comprehensive pentest for a large company. The objective: obtain domain administrator rights, gain access to the backup system, and find a way to copy confidential information. The chain was standard: CVE-2021-26855 (SSRF) → CVE-2021-27065 (file upload) → and there we were on the Exchange mail server with SYSTEM privileges.
Success? Without a doubt. Next, we began classic persistence in the infrastructure: looking for paths to other systems, studying the internal network, trying to plant our tools for reverse connections (reverse shell). And that's exactly where the first warning sign was waiting for us. In the key directories — the ones where autostart entries are usually written or where it's convenient to drop executables (for example, C:\ProgramData and the IIS temporary directories) — we found unfamiliar executables and libraries. Their names matched neither our tools nor the server's standard software.
Part two. "Colleague, I can see you"
A quick static analysis of the files, a check of their digital signatures (or rather, the absence thereof), and a review of the server's network connections left no doubt: we had stumbled upon traces of someone else's activity. Several remote-access tools were already living on the Exchange server: from the familiar C++ backdoors to more cunning script-based implants.
It was a veritable "zoo," which usually means one of two things: either the server had already survived several independent attacks, or the attackers were deliberately keeping a set of different tools on hand — in case one of them was discovered and someone tried to block it.
At that moment, the pentest plan instantly became obsolete. We were now facing a twofold task.
First, do no harm. Any action could tip off the attacker that they had been noticed. The very fact of discovery would, in all likelihood, push the hackers to try to wipe the digital traces that mattered for the investigation. And with such a varied toolkit at their disposal, they could have done that in several different ways.
Second, regain control. The client wanted to know not only about the vulnerabilities we had found, but also about the signs of a full-scale compromising intrusion, so as to move from a "stress test" to investigation, containment, and cleanup of the infrastructure.
Part three. Changing roles: from white-hat hackers to investigators
We immediately agreed with the client on changes to the legal aspects of the pentest and brought the paperwork in line with the new circumstances. The client authorized and welcomed the expansion of the team's mandate: the pentest engagement was officially extended to include incident response.
After that, the priority shifted from exploiting vulnerabilities to reconstructing the picture of what was happening. We pulled up and began going through the Exchange logs, the Windows logs (Security, System, PowerShell), and the IIS logs. The focus was on hunting for anomalies. These could be unusual times of activity, suspicious accounts, series of successful and failed login attempts, and traces of commands and scripts being run.
The next step was artifacts and "persistence points." We looked for exactly where the remote-access tools had been placed, how they survived a reboot (services, the task scheduler, autostart via the registry), and where the connections were going — that is, which external addresses were configured as command-and-control (C2) servers.
In parallel, we were assembling a dynamic map of the attack almost in real time, overlaying the attackers' actions onto a timeline: how and when they got in through the same vulnerability (ProxyLogon), how quickly they deployed their tools, and when they tried to move across the network and reach the data.
We worked with extreme caution. We proceeded on the assumption that the attackers could notice changes in the system at any moment — and would then start covering their tracks or speed up their operations. Such a turn of events would have made both the investigation and the recovery of control much harder.
The caution paid off: we completed our part of the job without arousing any suspicion among our "colleagues."
Part four. Cleaning up the aftermath
Once we had documented the key artifacts and gathered the evidence, the team, together with the client, moved on to the containment and cleanup stage: we needed to remove the traces of the intruders' presence and close the very door through which the uninvited guests had, in fact, gotten in.
First of all, we removed the foreign executables and scripts we had found — carefully, preserving copies and checksums for a possible further investigation. Then we cleaned out the persistence mechanisms: the registry entries, tasks, and services that ensured the malicious components would start automatically.
At the same time, we closed off the root cause of the incident — we rolled out the Exchange updates, eliminating the vulnerability through which the attackers had gained access.
A separate block of recommendations concerned post-incident cyber hygiene. We strongly advised the client to change the passwords of privileged accounts, to consider implementing two-factor authentication, and to audit the network for secondary access points. And, to be on the safe side, to step up monitoring. Simply to make sure that no hidden "backdoors" remained and that the attacker would not attempt to get back in.
Conclusion: lessons from an unplanned operation
In retrospect, I can say that we handled it correctly. But "in the moment," every decision had to be made with a multitude of real and probable circumstances in mind. The work went on around the clock. The team had plenty of nerve-racking moments, and that's normal: in situations like these, the cost of a mistake is too high. But if we ever face a similar combined operation again, we already know how to act.
The lessons we drew together with the client
1. A pentest is no guarantee that threats are absent. It's a snapshot of vulnerabilities at the moment of testing. Penetration testing becomes a step toward practical (that is, effective) cybersecurity only when its results are taken as a guide for implementation within the overall information security process. In other words, a pentest is an occasion to talk about improving infosec — but by no means a certificate of the absence of risk.
2. Flexibility is a key quality. The readiness to quickly restructure the work around new, more acute circumstances is critically important. The more diverse the staff's experience, the better the pentest team's chances of "pulling off" a complex case like this one.
3. Discovering the traces of someone else's attack is an undeniable success, and by no means a failure. We accomplished the main task: we found a hidden problem that was already causing real damage. The client received a concrete plan for getting out of the crisis. In their case, that was a far weightier gain than a mere statement of the status of their corporate infosec.
4. EDR and monitoring really do make the difference. The outcome of the project showed the client, better than any argument could, the need to deploy an Endpoint Detection and Response-class system. Had one been in place, the RATs we found would probably have been detected much earlier.
Proof that a vulnerability is already being exploited by some third party, for purposes unknown to the pentest client, is a pentester's most valuable find. But it also confronts the provider of ethical hacking services with a difficult choice: stick to the immediate task, or step beyond it and become the first to give the client real help right now.
The "help immediately" option entails maximum responsibility. But it is precisely this option that most fully answers what the client wants from the "cybersecurity people": save us right now, before things get really bad.