MSTICPy versions >= 1.5.0
This Notebook provides details and examples of how to connect to and query data from the Microsoft Defender Advanced Hunting API.
Note: This notebook reflects a partially-updated component and still uses the "MDATP" abbreviation to refer to the Microsoft 365 Defender and Microsoft Defender for Endpoint data services.
%pip install --upgrade msticpy
Authentication for the Microsoft Defender Advanced Hunting API is handled via an Azure AD application. Before you can authenticate you will need to register an application and provide it with the required permissions. MSTICpy supports Application Context authentication to the API. Detailed instructions on registering an application can be found here:
Once created you will require the following details:
These details can be found in the Azure Portal under Azure Active Directory > App Registrations.
Once collected the easiest way to manage these details is via msticpyconfig.yaml - simply add them to the file in the following format:
DataProviders:
MicrosoftDefender:
Args:
ClientId: "CLIENT ID"
ClientSecret:
KeyVault:
TenantId: "TENANT ID"
You can then initialize a data provider for Microsoft Defender and connect the provider.
Note: you can also provide these values to the connect function. See Microsoft Defender data provider
Note: If you want to access the Microsoft Defender for Endpoint APIs rather than the M365 Defender API (the latter is a subset of the former), please use "MDE" as the parameter to QueryProvider.
from msticpy.data.data_providers import QueryProvider
md_prov = QueryProvider("M365D")
md_prov.connect()
Connected.
Once connected the Microsoft Defender data connector functions in a similar manner to other data connectors. You can list queries:
md_prov.list_queries()
['MDATP.file_path', 'MDATP.host_alerts', 'MDATP.host_connections', 'MDATP.ip_alerts', 'MDATP.ip_connections', 'MDATP.list_alerts', 'MDATP.list_connections', 'MDATP.list_filehash', 'MDATP.list_files', 'MDATP.list_host_processes', 'MDATP.process_cmd_line', 'MDATP.process_creations', 'MDATP.process_paths', 'MDATP.protocol_connections', 'MDATP.sha1_alerts', 'MDATP.url_alerts', 'MDATP.url_connections', 'MDATP.user_files', 'MDATP.user_logons', 'MDATP.user_network', 'MDATP.user_processes', 'MDATPHunting.accessibility_persistence', 'MDATPHunting.av_sites', 'MDATPHunting.b64_pe', 'MDATPHunting.brute_force', 'MDATPHunting.cve_2018_1000006l', 'MDATPHunting.cve_2018_1111', 'MDATPHunting.cve_2018_4878', 'MDATPHunting.doc_with_link', 'MDATPHunting.dropbox_link', 'MDATPHunting.email_link', 'MDATPHunting.email_smartscreen', 'MDATPHunting.malware_recycle', 'MDATPHunting.network_scans', 'MDATPHunting.powershell_downloads', 'MDATPHunting.service_account_powershell', 'MDATPHunting.smartscreen_ignored', 'MDATPHunting.smb_discovery', 'MDATPHunting.tor', 'MDATPHunting.uncommon_powershell', 'MDATPHunting.user_enumeration']
Get details about avaliable queries:
md_prov.MDATP.list_alerts('?')
Query: list_alerts Data source: MDE Retrieves list of alerts Parameters ---------- add_query_items: str (optional) Additional query clauses end: datetime Query end time start: datetime Query start time table: str (optional) Table name (default value is: DeviceAlertEvents) Query: {table} | where Timestamp >= datetime({start}) | where Timestamp <= datetime({end}) {add_query_items}
Execute queries with default parameters:
md_prov.MDATP.list_alerts(start=-10)
AlertId | Timestamp | DeviceId | DeviceName | Severity | Category | Title | FileName | SHA1 | RemoteUrl | RemoteIP | AttackTechniques | ReportId | Table | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | ed637828714417577685_889446803 | 2022-03-14 16:03:11.918913300+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | High | UnwantedSoftware | Notepad Execution Detection | notepad.exe | 5b80bbb07b1a84384e61fb3f9366cad97904ebea | 25654 | Not supported | |||
1 | ed637828714417577685_889446803 | 2022-03-14 16:03:11.918913300+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | High | UnwantedSoftware | Notepad Execution Detection | notepad.exe | 5b80bbb07b1a84384e61fb3f9366cad97904ebea | 25654 | DeviceProcessEvents | |||
2 | da637831027930681075_399960450 | 2022-03-17 08:32:13.537246500+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | Low | Malware | 'ATPMiniDump' hacktool was prevented | ATPMiniDump.exe | 2659afd376e84ea8dab21d73412069677212abd6 | 50216 | DeviceEvents | |||
3 | da637831027930681075_399960450 | 2022-03-17 08:32:41.619019500+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | Low | Malware | 'ATPMiniDump' hacktool was prevented | ATPMiniDump.exe | 2659afd376e84ea8dab21d73412069677212abd6 | 50232 | DeviceEvents | |||
4 | da637831027930681075_399960450 | 2022-03-17 08:32:51.787218300+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | Low | Malware | 'ATPMiniDump' hacktool was detected | ATPMiniDump.exe | 2659afd376e84ea8dab21d73412069677212abd6 | 50322 | DeviceEvents | |||
5 | da637831027930681075_399960450 | 2022-03-17 08:32:51.795963100+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | Low | Malware | 'ATPMiniDump' hacktool was detected | ATPMiniDump.exe | 2659afd376e84ea8dab21d73412069677212abd6 | 50351 | DeviceEvents | |||
6 | da637831029556671978_1988175326 | 2022-03-17 08:33:14.289264600+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | Medium | Malware | An active 'ATPMiniDump' hacktool process was d... | ATPMiniDump.exe | 2659afd376e84ea8dab21d73412069677212abd6 | 50540 | DeviceEvents | |||
7 | da637831027930681075_399960450 | 2022-03-17 08:33:14.289301300+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | Low | Malware | 'ATPMiniDump' hacktool was detected | ATPMiniDump.exe | 2659afd376e84ea8dab21d73412069677212abd6 | 50541 | DeviceEvents | |||
8 | da637831027930681075_399960450 | 2022-03-17 08:33:14.505995400+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | Low | Malware | 'ATPMiniDump' hacktool was detected | ATPMiniDump.exe | 2659afd376e84ea8dab21d73412069677212abd6 | 50521 | DeviceEvents | |||
9 | da637831931160791947_311302490 | 2022-03-18 09:36:44.691315900+00:00 | e09b459fcd3aa36f23084211c34bfca046848b65 | atevet06cl005.defenderatevet06.onmicrosoft.com | Medium | Execution | Suspicious PowerShell command line | powershell.exe | f43d9bb316e30ae1a3494ac5b0624f6bea1bf054 | ["PowerShell (T1059.001)"] | 127261 | DeviceImageLoadEvents | ||
10 | da637831931152888965_11461089 | 2022-03-18 09:36:44.691315900+00:00 | e09b459fcd3aa36f23084211c34bfca046848b65 | atevet06cl005.defenderatevet06.onmicrosoft.com | Medium | Execution | Suspicious PowerShell command line | powershell.exe | f43d9bb316e30ae1a3494ac5b0624f6bea1bf054 | ["PowerShell (T1059.001)"] | 127261 | DeviceImageLoadEvents |
Execute queries with custom parameters:
md_prov.MDATP.list_alerts(start="-30", add_query_items="| summarize count() by Severity")
Severity | count_ | |
---|---|---|
0 | High | 53 |
1 | Medium | 11 |
2 | Low | 19 |
3 | Informational | 6 |
Print a fully constructed query for debug purposes:
md_prov.MDATP.list_alerts("print", start="-30", add_query_items="| summarize count() by Severity")
' DeviceAlertEvents | where Timestamp >= datetime(2022-02-19T18:31:48.373684Z) | where Timestamp <= datetime(2022-03-22T18:31:44.860951Z) | summarize count() by Severity'
Execute a custom query:
query = "DeviceAlertEvents | sample 10"
md_prov.exec_query(query)
AlertId | Timestamp | DeviceId | DeviceName | Severity | Category | Title | FileName | SHA1 | RemoteUrl | RemoteIP | AttackTechniques | ReportId | Table | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | da637807989907659907_-236366407 | 2022-02-21 09:47:47.938238100+00:00 | efeb32222798ea6c2eb747514dd34e646cf751aa | win-6l54m49tpgn.woanware.co.uk | Low | Discovery | Suspicious System Service Discovery | powershell.exe | 5b1a36a8b206fe9daece7e2cd702cdd4e10bb895 | ["System Service Discovery (T1007)","Windows M... | 45923 | DeviceEvents | ||
1 | da637824441902905094_1268174690 | 2022-03-09 17:34:15.638562100+00:00 | 8a2d7be2ee62093058401b4d50c481fe9ec7aedb | attackervm.attack.vm | High | Persistence | A suspicious DLL was loaded by the ADFS service | version.dll | ebdcd4c1743adc9714a94a09eeb770c8b67747cc | ["T1606"] | 718 | DeviceImageLoadEvents | ||
2 | da637807989904403259_2043617699 | 2022-02-21 09:50:28.356919+00:00 | efeb32222798ea6c2eb747514dd34e646cf751aa | win-6l54m49tpgn.woanware.co.uk | Low | Discovery | Suspicious User Account Discovery | explorer.exe | dbf66924d79d246da08df40db1f20595385b6347 | ["Remote System Discovery (T1018)","PowerShell... | 46011 | DeviceRegistryEvents | ||
3 | ed637810396041195799_-785916563 | 2022-02-23 12:42:05.423489800+00:00 | efeb32222798ea6c2eb747514dd34e646cf751aa | win-6l54m49tpgn.woanware.co.uk | High | UnwantedSoftware | Notepad Execution Detection | notepad.exe | 58bf532f93949cc62821273f778b87da85787afa | 7173 | DeviceProcessEvents | |||
4 | ed637810396041195799_-785916563 | 2022-02-21 12:50:38.523957+00:00 | efeb32222798ea6c2eb747514dd34e646cf751aa | win-6l54m49tpgn.woanware.co.uk | High | UnwantedSoftware | Notepad Execution Detection | notepad.exe | 58bf532f93949cc62821273f778b87da85787afa | 48272 | DeviceProcessEvents | |||
5 | da637831931152888965_11461089 | 2022-03-18 09:36:44.691315900+00:00 | e09b459fcd3aa36f23084211c34bfca046848b65 | atevet06cl005.defenderatevet06.onmicrosoft.com | Medium | Execution | Suspicious PowerShell command line | powershell.exe | f43d9bb316e30ae1a3494ac5b0624f6bea1bf054 | ["PowerShell (T1059.001)"] | 127261 | DeviceImageLoadEvents | ||
6 | da637823935780152927_1691659474 | 2022-03-09 03:31:17.271601600+00:00 | e09b459fcd3aa36f23084211c34bfca046848b65 | atevet06cl005.defenderatevet06.onmicrosoft.com | Medium | Persistence | A WMI event filter was bound to a suspicious e... | ["Windows Management Instrumentation Event Sub... | 771 | DeviceEvents | ||||
7 | da637819900757032477_-2012458047 | 2022-03-04 11:20:44.426056800+00:00 | efeb32222798ea6c2eb747514dd34e646cf751aa | win-6l54m49tpgn.woanware.co.uk | Low | DefenseEvasion | Windows Sysinternals tool renamed | powershell.exe | 5b1a36a8b206fe9daece7e2cd702cdd4e10bb895 | ["Masquerading (T1036)"] | 43063 | DeviceProcessEvents | ||
8 | da637823935780204986_825228657 | 2022-03-09 03:31:17.272119+00:00 | e09b459fcd3aa36f23084211c34bfca046848b65 | atevet06cl005.defenderatevet06.onmicrosoft.com | Medium | Persistence | A WMI event filter was bound to a suspicious e... | ["Windows Management Instrumentation Event Sub... | 774 | DeviceEvents | ||||
9 | ed637828714417577685_889446803 | 2022-03-14 16:03:11.918913300+00:00 | a085f9e3a00689554d351bdcdd4d503a1983eecd | desktop-6qf1uf2 | High | UnwantedSoftware | Notepad Execution Detection | notepad.exe | 5b80bbb07b1a84384e61fb3f9366cad97904ebea | 25654 | Not supported |