Human beings are lazy and frugal. As soon as we can stop using a person to do something simple, we do. People are much better suited to doing expensive, complex things. And so, more than 200 years after the beginning of the industrial revolution, we still carry on automating the workplace. 

The latest incarnation is the public cloud, which runs at a massive scale, far beyond that of our own data centres. That very scale is both a benefit and a risk: it gives access to vast amounts of compute and memory — but where there are resources, there are criminals who want to get something for nothing, hijacking your cloud infrastructure for their own purposes and leaving you with the bill at the end of the month. 

SEE: Social engineering: A cheat sheet for business professionals (free PDF) (TechRepublic)  

It’s a big problem, and one that’s going to get bigger, as our virtual infrastructures grow and add scale automatically. We’ve moved from a world where servers were much-loved pets, carefully cared for and given individual names, to one where we treat them as sheds full of chickens, where all we care about is what gets delivered. That hands-off approach is attractive to attackers, who can drop rootkits into images and steal resources running cryptocurrency miners or sniffing through data for valuable snippets. With thousands of servers, who’s going to be looking for the signs of a malware attack on one or two, or a dozen, or a hundred? 

Attackers have invested in smarter malware that can get around traditional security tooling, hiding underneath the operating system in memory, masking tell-tale signatures, and even deleting itself as soon as it detects security systems in action. There’s a lot of value in the hyperscale cloud’s massive scale, and that value is what attackers want to steal. 

Scanning the cloud: all of it

A Microsoft research project, Project Freta, aims to change that, providing tools to identify malware running on virtual machines in the cloud. It takes an economic approach to managing malware, which is only valuable to bad actors as long as it’s undetected: once identified on one system, malware code is no longer reusable, as its signature can be added to active scanning tools. But if we’re to have any success, we need to be able to scan many thousands of devices, at a push of a button. 

The very industrial scale of the cloud means that traditional scanning techniques are too slow, looking for one or two compromised images in an ever-growing fleet. It’s a reminder of that old Cold War adage: your attackers only have to be lucky once, you have to be lucky every time. 

Microsoft Research’s security specialists have been thinking about this problem, and Project Freta encapsulates much of this thinking in a cloud-centric proof-of-concept. Designed to look for in-memory malware, it provides a portal where you can scan memory snapshots from Linux and Windows virtual machines. Initially focusing on virtual machine instances, it’s intended to show the techniques and tools that can be used to scan for malware at massive scale. 

Under the hood of Project Freta 

A key part of the Project Freta thinking revolves around the concept of ‘survivorship bias’. We’re used to thinking that devices that show no sign of malware are clean, not that they may well be the hosts for undetected malware. Attackers want to get around our sensing, as we let our defences down when we trust that our tools are doing the necessary work for us. But there’s a fundamental problem in how we look for malware: much of what we use is designed to work in a pre-virtualisation world, and recent research has shown that it’s possible for malware to detect whether it’s being monitored by hypervisor security tools that are working outside the virtual machine. 

That led to the Project Freta team rethinking security from scratch, treating it as a green field. The team came up with four principles for developing sensing tools to target modern malware. First: malware can’t detect a sensor before it’s installed. Second: no malware can hide out of reach of sensors. Third: no malware can change itself before it is sampled. Fourth: no malware can change a sensor to avoid detection and acquisition. The aim is to have a resilient security environment that can rapidly test many thousands of physical and virtual machines, making it impossible for stealthy malware to work. 

Capturing memory snapshots 

Project Freta builds on these principles by accepting that the perfect is the enemy of the good, and that trade-offs are necessary to achieve these goals. First and foremost was the realisation that the only way to deliver on the project’s goals was to capture all the memory used, without running any code in the captured memory space. That capture would then be analysed offline, using cloud resources for speed and the ability to test many captures in parallel, with the whole system build using memory-safe programming languages and techniques. 

SEE: Checklist: Securing Windows 10 systems (TechRepublic Premium)

The cloud is necessary here, as it avoids having to wait hours or days for analysis to complete, reducing overall risk to your systems. There’s another reason why using the cloud is essential, as modern memory protection techniques randomise memory usage and copying to decode memory quickly could alert malware that it is being attacked, so analysis requires significant compute resources to unscramble and decode memory using brute-force techniques. Microsoft has had some success here, working initially with Linux and quickly delivering support for over 4,000 different kernel versions. 

Using the experimental portal 

Microsoft has now shipped a prototype portal that works with hypervisor memory snapshots, running on Azure. It has been tested with Hyper-V, but also works with VMware and with AVML and LiME memory snapshots. However, only Hyper-V is trusted at this stage, as it can, as the Project Freta team put it, “provide a reasonable approximation of the element of surprise” that’s needed. 

Once uploaded to the portal, a snapshot’s contents are analysed, allowing you to examine just what’s happening in a virtual machine at a specific point in time. You can see what processes are in memory, along with current system calls and open Unix sockets and files. It’s an interesting tool that gives a feel for the type of data Project Freta can get from an image, with an indicator of possible hidden malware for further analysis. Don’t expect it to be particularly user-friendly, as this is the first public pass at this type of security tooling, and the team has a lot more work to do. 

It’s easy to image a more user-focused future version of Project Freta that’s continuously sampling all the VMs running in Azure, providing you with information about compromised images while still providing Microsoft with the information needed to harden its base images. At that scale, Microsoft will need to use AI techniques to analyse and fingerprint malware in thousands, or even millions of images. It’s an intriguing vision of a future where the economics of cloud security have shifted, making it cheap to harden virtual machines, and expensive to attack them. 

Microsoft Weekly Newsletter

Be your company’s Microsoft insider by reading these Windows and Office tips, tricks, and cheat sheets. Delivered Mondays and Wednesdays

Sign up today

Also see