Examining electronic storage media when information security tools are unavailable

In brief

What to do when the corporate network is already down and the usual security tool consoles and telemetry are gone. The analysis is built on a single end-to-end case: a leaked online store database containing the personal email address of a contractor's employee → access to his email and cloud → open-source reconnaissance → entry into the customer through a jump host under the guise of contract work → persistence, exfiltration of databases, the discovery of an unencrypted backup of a personal phone on the workstation of an IT guy → destruction of backups and of core switch configurations → encryption and extortion over a messenger. Then the practical part: when to isolate a node and when you must not; what to collect as volatile data and in what order; how to connect to switches with a console cable and why the first command should be the shell history. The section on persistent data had begun — and that is where the recording breaks off.

Key points

• The opening thesis: even a full arsenal of protection (SIEM, EDR, DLP, TI, WAF, NGFW, antiviruses, including certified ones), regular pentests, exercises and regulator audits "in no way guarantee" that in a serious attack you will keep access to the security tools themselves and their telemetry.
• The case, step 1: a database of an online store with personal data, email addresses and passwords ended up in the public domain. Among those affected was an employee of the developer of a well-known IDM system who used his personal email address as his login.
• Step 2: the discovered password gave access to that employee's personal email and cloud service; open sources established his additional addresses, his corporate email, his position and his organization.
• Step 3: the contents of the email and the cloud yielded information about the customer organizations, about privileged AD accounts, about the hosts used and the remote connection methods.
• Step 4: the attackers continued the correspondence with one of the customers and under the pretext of additional work obtained remote access to a jump host. For the first week they "behaved as if they really were a contractor", then started gathering data about the domain infrastructure, the network architecture, accounts and rights, critical servers and databases.
• Step 5: compromise of AD accounts, use of privileged and service accounts, a persistent presence in the infrastructure; exfiltration of databases with data on finances, employees, customers and logistics.
• Step 6, particularly telling: the workstation of an IT department employee held an unencrypted backup of his personal phone. It yielded photos of confidential documents, correspondence with credentials that had been passed to a contractor, network diagrams and accounts kept in the notes.
• Step 7, destructive: the servers' backups were destroyed, the config files of core-level switches and of other network equipment were deleted, the configurations of the core switches were changed.
• The consequences: the network lost connectivity, the domain environment degraded, normal access to the security tools was lost and centralized restrictions on telemetry appeared. The key phrase: at this stage "a rapid reconstruction of the attack with standard tools was no longer possible".
• In parallel the database passwords were changed, after which the databases were encrypted.
• Extortion: contact with the IT employee over a messenger, a ransom demand, a threat to publish the captured backup of his personal phone; to confirm that the threat was serious, a fragment of the data was in fact published in a public channel.
• Conclusions from the case: the compromise of a single personal account can lead to an incident of this scale; remote access for contractors without control is a high risk; storing confidential information on personal devices and in their backups hurts both the company and the employee himself.
• The main conclusion of the talk: when there is no access to the usual security tools, "the only key instrument for restoring the full picture of the incident is exclusively digital forensics".
• An important correction to the perception: only the viewer of the presentation sees the whole chain. At the moment of detection the information security staff have only a network that does not work, failures of nodes and databases, a message from the attacker and a published fragment of data.
• The response process is broken down into 10 interrelated stages — from recording the initial report and preliminarily assessing the scale through to reconstructing the attack, restoring the infrastructure and closing the incident. A detailed document, "General Response Procedure", is posted behind a QR code that hangs on all the following slides.
• The logic for each node: there is no universal rule — neither "immediately isolate every host" nor "collect everything from everyone". The rules must be "flexible and not categorical".
• Why not isolate everything: a system that survived the incident may break precisely because of the isolation (especially a database that something was being written into at that moment), and there is a risk of "thoughtlessly losing the necessary artifacts".
• Why not collect everything: with 2,000, 10,000 or 100,000 hosts "we'll be collecting information until our company shuts down because of this incident".
• The decision algorithm: assess the node's connection to the incident → is the damage ongoing on the host or through it, are there suspicious network connections → does network access to the systems that are still alive remain (even in a dead network a host in the same VLAN as them may keep connectivity) → account for technical limitations.
• On technical limitations separately: an unstable machine may go down for good because of careless collection, and for network equipment from one vendor running the same OS the set of commands and their impact on the device differ from model to model.
• Case 1: the node is connected to the incident, but the damage is not ongoing, there are no suspicious connections and there is no network path to the resources — there is no need to isolate it, "the host, essentially, is already isolated". Collect the volatile data and do not touch it at all until persistent data is obtained.
• Case 2: active encryption, modification, deletion or copying of data is going on. Isolation is still not an automatic decision: on a critical node a dropped connection may leave an operation unfinished, and "the database itself could go down", and restoring it with the backups destroyed is already problematic. If it is clear that nothing catastrophic will happen in two minutes — collect the volatile data first, because with isolation its loss is guaranteed.
• This is not done by hand: on Linux it is a matter of "about 67 or more commands", which will not fit into two minutes. An automated script is needed — behind the QR code there are both full lists of commands for Windows, macOS and Linux and ready-made scripts that export the data and generate a report in a couple of minutes.
• Case 3: the node is not connected to the incident — do not touch it at all. Case 4: the node is powered off — do not power it on and do not boot the installed OS, otherwise the persistent data will change.
• Virtual machines: if there is the technical ability and the confidence that the hypervisor will not make unacceptable changes to the guest system's data, a snapshot preserving the RAM is created — and before that button is pressed, do nothing on the machine.
• How to reach the hypervisor in a dead network: if the servers are not outsourced, take a non-domain laptop and connect to the SAN switch or directly to the equipment's management port.
• The order of volatile collection. What is of interest is not the entire contents of the media but what may be altered or lost because of the OS running, the attacker's actions or the isolation itself. First the basics about the node: the hostname, the current account, the OS or kernel version, the system date, time and time zone, the time from an independent source with any discrepancies recorded, the uptime since boot.
• Then user activity: active users and sessions, cached authentication data for the current sessions, running processes and services with their launch parameters, open files.
• Then local logs with a high risk of being overwritten — yes, they sit in persistent storage, but while the collection is going on some of the entries may get overwritten. Do not export all of it: logs sometimes "weigh in at terabytes", the past 24 hours is enough.
• The specific logs: for Windows — System, Security, WinRM Operational, the terminal services logs, OpenSSH Operational and, "most importantly", both PowerShell logs; for Linux — syslog, audit, secure, kern, messages, daemon, the xrdp logs and auth; for macOS — syslog and Unified.
• Next — loaded drivers and kernel modules, active inter-process communication mechanisms and the node's network state: interfaces, established connections, listening ports, the routing table, neighboring hosts via ARP and NDP.
• A hard storage rule: what is collected is "under no circumstances" saved to the persistent drive of the node under examination — only to external media or to an isolated storage independent of the corporate network.
• Network equipment. A loss of connectivity does not in itself mean a compromise: it may be a technical failure (which happens often), mistakes by IT staff during configuration — or indeed an attacker, as in the case under discussion.
• Connect from a standalone non-domain laptop with a console cable. Before establishing the session you must start recording the terminal session and record the time from an independent source.
• The first command after logging in is to view the shell history for all users: later commands will push out the old entries. Then — the hostname, the account, the date, time, time zone and the deviation from an independent source, the uptime, the OS version, local logs, administrative sessions, network routes and whatever remains in the operational tables.
• A categorical ban: no write commands. Only commands verified in advance, whose result and load on the device are both clear. Any connection and even a viewing command shows up in the shell history, the local logs and the AAA systems and can briefly raise the load. Collection is done by script here too, one or two minutes.
• Persistent data: the task is to obtain a copy whose origin and integrity can be verified later. For that, the data source, the extraction method, the tool and its version, the place where the result is saved and the time of the operation are recorded, and checksums are calculated. The recording breaks off on this sentence.

Tools, artifacts, technologies

• Security tools mentioned as background: SIEM, EDR, DLP, TI, WAF, NGFW, antivirus software.
• The attack: a leaked database with email logins, access to the personal email and cloud, OSINT enrichment, a jump host, privileged and service AD accounts, an unencrypted backup of a personal phone on a workstation, destruction of backups and of core switch configurations, encryption of the databases, extortion over a messenger and publication of a fragment of the data.
• Host volatile data: basic information about the node and time, active sessions, the authentication cache, processes and services, open files, local logs (System, Security, WinRM Operational, the terminal services logs, OpenSSH Operational, Windows PowerShell and PowerShell Operational; syslog, audit, secure, kern, messages, daemon, xrdp, auth; syslog and Unified on macOS), drivers and kernel modules, inter-process communication, the network state (interfaces, connections, ports, routes, ARP/NDP).
• Network equipment: a console cable and a non-domain laptop, terminal session recording, the shell history, the operational tables, AAA systems; the remark about commands differing between models from the same vendor.
• Virtualization: a snapshot with memory, access to the hypervisor through a SAN switch or a management port.
• The author's materials behind the QR code: the document "General Response Procedure", lists of commands by operating system and automated collection scripts (PowerShell ones among them).

Legal and organizational context

The procedural frame appears in the last section: the copy of persistent data must be verifiable — with the source, the extraction method, the tool and its version, the storage location, the time of the operation and the checksums all recorded. The organizational part of the case is about responsibility for contractors: remote access granted to a "contractor" without control became the entry point, and the unencrypted backup of a personal phone on a workstation turned into leverage against the employee personally, including a threat to his reputation. It is separately stressed that none of the collected data may be written to the drive of the node under examination.

Questions from the audience

There are no questions in the recording: the stream breaks off mid-sentence in the section on persistent data, so the end of the talk and any discussion did not make it into the recording.

The speaker's position

A methodical talk by a practitioner, built not around tools but around decisions: the main message is not to act by the template of "isolate everything" or "collect everything", but to assess each time the node's connection to the incident, the ongoing damage and the technical risks. The limitations are stated consistently: isolation destroys volatile data, careless collection brings down an unstable machine, a viewing command on a switch still changes the state of the device. The tone is emphatically cautious, with qualifiers like "at least I wouldn't do it" and "this question is rather philosophical". The practical part is tied to the materials behind the QR code — neither the scripts nor the lists of commands are shown in full in the recording itself.

Quotes

• "…no matter how modern and strong your information security system is in practice, it in no way guarantees that… you will necessarily keep access to your usual security tools and their telemetry."
• "…the only key instrument for restoring the full picture of the incident is exclusively digital forensics."
• "I'm afraid we'll be collecting information until our company shuts down because of this incident."
• "…if we do network isolation, we'll lose a huge number of such artifacts. That's 100%."
• "…when collecting volatile data we never, under any circumstances, use write commands."