Quite a few of my customers have a Security Information and Event Management (SIEM) on premise. They spent thousands of dollars fine tuning it to detect and correlate security events, it’s integrated with their ticketing system, but mostly… it has been sanctioned as the corporate security tool.
So the next question is: how do I integrate that with Azure resources?
By monitoring, I mean that we can have insights about what’s going on within our solutions, for instance:
- Have any failures occurred or are currently occurring?
- What is the current and past load?
- Are resources well utilized or do we have under / over provisioning?
… and so on. This doesn’t address a lot of security requirements, such as treat detection, though.
For a customer who do not have a legacy SIEM in place, I would recommend to take a look at Security Center. It is a solid base for security monitoring and takes minutes to setup.
Security Center does threat detection and correlation between security events.
Microsoft Azure Log Integration
Back to my legacy SIEM scenario.
In July 2016, Microsoft released Microsoft Azure Log Integration. As of November 2016, it is still in Preview.
You can download the tool at https://aka.ms/azlogdownload. It is meant to be installed on a VM along standard SIEM connector agent (e.g. Splunk Universal Forwarder, ArcSight Windows Event Collector agent or Qradar wincollect). The VM is meant to act as a log gateway: it collects logs from Storage Accounts and send them to the SIEM via standard SIEM connector.
A complete guide to set this up is available here.
Of course, this setup assume we are exporting all the logs to Azure Storage.
Now that we’ve looked at the general solution, let’s consider where the different components could reside since as with all things depending on a pesky thing called physics, latency matter.
Following the diagram from previous section from left to right, the Azure resources & Azure storage are, of course, in Azure. The VM running the Log Integration could be either in Azure or on Premise so can our SIEM. So let’s consider the different possibilities.
Log Integration & SIEM on premise
In many ways, that might be the easiest way to install the log integration but also most likely the less efficient.
It is easy because you simply need a VM on premise with access to the SIEM and the internet, in order to access Azure Storage.
It is likely inefficient because of the latency between on premise site and Azure. The Log Integration tool will probe different parts of Azure Storage at regular intervals with and the latency will build up. Network bandwidth, which do not come for free on premise, will also add up.
For many Azure resources with verbose logs, the network bandwidth requirement might be so demanding that connecting over the internet is too slow or unreliable, in which case we would consider Azure Express Route.
Log Integration in Azure & SIEM on premise
This setup is slightly less straightforward because of hybrid connectivity. Indeed, the Log Integration VM, being in Azure, needs some hybrid connectivity in order to communicate with the on premise SIEM.
This can easily be achieved with a site-to-site connection through an Azure VPN Gateway.
This setup is likely going to be more efficient network wise since all Azure Storage probing will occur in Azure with no bandwidth on premise. Only the gathered logs shipped on premise to the SIEM will incur bandwidth.
This can still be a lot if we have a few resources in Azure and gather verbose logs. Again Express Route could be considered.
Log Integration & SIEM in Azure
In this scenario, we migrate the SIEM in Azure. Bam! No more latency.
Unless we also moved all on premise workload in Azure we now have the reverse problem we had, i.e. we need to push on premise logs to Azure.
A variant on that scenario actually brings the best of both worlds.
Azure SIEM to generate security events
In this scenario, we install a SIEM in Azure while also keeping the one we have on premise.
The one in Azure acts as an aggregator of logs and generates security events. Only those security events are shipped to the SIEM on premise. This reduces the network pipe requirement considerably but still achieve the same goal, i.e. for the “main SIEM” to have enough information to be able to correlate security events from different sources in order to detect attack patterns.
A blocker for this approach would obviously be the license costs to install a SIEM in Azure. I wouldn’t recommend this for a small deployment.
If workload migration is happening from on premise to Azure, as a corporate strategy for instance, we could consider inversing the SIEM at some point, i.e. having the main SIEM in Azure and an aggregator one on premise. This could make sense early on since storage is cheaper in the cloud than on premise and SIEM tends to consume lots of those. Also, a SIEM being a critical system, it tends to be easier / cheaper to have an high SLA in Azure than on premise.
Azure Log Integration with on premise SIEM is often a corporate requirement. In this article we went through the solution and some variants regarding networking consideration.