Ensure Critical Log Collection in Microsoft Sentinel

Ensure Critical Log Collection in Microsoft Sentinel

This solution was inspired by one of my customers. They wanted to know if they could use a Microsoft Sentinel Watchlist in combination with Usage logs to be alerted if they were no longer getting log data on one of their collections.

For this example, I created a Watchlist that contained one column, LogName. This watchlist would be used to keep a collection of critical logs that you want to make sure you are collecting. I am checking to ensure that there was a collection in the last 24 hours.

Create the Watchlist

I will not get into the details of Watchlist creation in this blog. Please check out Use Watchlists in Microsoft Sentinel for more information. I created a simple .csv file with three log names: AzureActivity, AzureMetrics, and MissingCriticalLog. I then created the list from that .csv

Create the Query

Now came the fun part, creating and debugging my KQL query. What you see is the result. No use making you suffer as I did.

let checklog = Usage | where TimeGenerated >= ago(24h) | project DataType;
let watchlist = (_GetWatchlist('CriticalLogs') | project LogName);
watchlist
| where LogName !in (checklog)

Yep, it was a simple as that. OK, I will break it down for you:

let checklog = Usage | where TimeGenerated >= ago(24h) | project DataType;
Gather all the usage data from the last 24 hours. "DataType" is the log name from the "Usage" log.

let watchlist = (_GetWatchlist('CriticalLogs') | project LogName);
Gather the list of critical logs from the Watchlist I created.

watchlist
| where LogName !in (checklog)
These lines take the LogName from the Watchlist and see if it appears in the Usage log (checklog). If it does not, it displays that LogName. The ones that are displayed, received no data in the last 24 hours.

Build and Schedule the Alert

From the query, Create Azure Sentinel alert

I changed the Severity to Low

I scheduled it to run once a day

Here is my resulting alert.