In this notebook we will explore how to obtain attributes and relationship for different entities using VirusTotal API v3. Finally we can render all the relationships we have obtained using VTGraph.
from msticpy.context.vtlookupv3 import VTLookupV3, VTEntityType
import networkx as nx
import matplotlib.pyplot as plt
import os
import pandas as pd
pd.set_option('max_colwidth', 200)
try:
import nest_asyncio
except ImportError as err:
print("nest_asyncio is required for running VTLookup3 in notebooks.")
resp = input("Install now? (y/n)")
if resp.strip().lower().startswith("y"):
%pip install nest_asyncio
import nest_asyncio
else:
raise err
nest_asyncio.apply()
from msticpy.common.provider_settings import get_provider_settings
# Try to obtain key from env varaible
vt_key = os.environ.get("VT_API_KEY")
if not vt_key:
# if not try provider settings to get from msticpyconfig.yaml
vt_key = get_provider_settings("TIProviders")["VirusTotal"].args["AuthKey"]
# Instantiate vt_lookup object
vt_lookup = VTLookupV3(vt_key)
# The ID (SHA256 hash) of the file to lookup
FILE = 'ed01ebfbc9eb5bbea545af4d01bf5f1071661840480439c6e5babe8e080e41aa'
example_attribute_df = vt_lookup.lookup_ioc(observable=FILE, vt_type='file')
example_attribute_df
last_submission_date | size | times_submitted | meaningful_name | type_description | first_submission_date | detections | scans | first_submission | last_submission | type | |
---|---|---|---|---|---|---|---|---|---|---|---|
id | |||||||||||
ed01ebfbc9eb5bbea545af4d01bf5f1071661840480439c6e5babe8e080e41aa | 1605582797 | 3514368 | 1325 | diskpart.exe | Win32 EXE | 1494574270 | 67 | 76 | 2017-05-12 07:31:10+00:00 | 2020-11-17 03:13:17+00:00 | file |
We can use get_object to retrieve all details or just look it up directly at https://www.virustotal.com/gui/home/search
vt_lookup.get_object(FILE, "file")
attributes | |
---|---|
authentihash | 4b2c4c7f06f5ffaeea6efc537f0aa66b0a30c7ccd7979c86c7f4f996002b99fd |
autostart_locations | [{'entry': ' ', 'location': ' '}, {'entry': 'HKLM\SYSTEM\CurrentControlSet\Control\NetworkProvider\Order\ProviderOrder', 'location': 'HKLM\SYSTEM\CurrentControlSet\Control\NetworkProvider\Order'},... |
capabilities_tags | [win_registry, str_win32_winsock2_library, win_files_operation] |
creation_date | 1290243905 |
crowdsourced_yara_results | [{'author': 'ReversingLabs', 'description': 'Yara rule that detects WannaCry ransomware.', 'rule_name': 'Win32_Ransomware_WannaCry', 'ruleset_id': '005e5fc7e3', 'ruleset_name': 'Win32.Ransomware.W... |
downloadable | True |
exiftool | {'CharacterSet': 'Unicode', 'CodeSize': '28672', 'CompanyName': 'Microsoft Corporation', 'EntryPoint': '0x77ba', 'FileDescription': 'DiskPart', 'FileFlagsMask': '0x003f', 'FileOS': 'Windows NT 32-... |
first_seen_itw_date | 1578568742 |
first_submission_date | 1494574270 |
last_analysis_date | 1605638619 |
last_analysis_results | {'ALYac': {'category': 'malicious', 'engine_name': 'ALYac', 'engine_update': '20201117', 'engine_version': '1.1.1.5', 'method': 'blacklist', 'result': 'Trojan.Ransom.WannaCryptor'}, 'APEX': {'cate... |
last_analysis_stats | {'confirmed-timeout': 0, 'failure': 0, 'harmless': 0, 'malicious': 67, 'suspicious': 0, 'timeout': 1, 'type-unsupported': 4, 'undetected': 4} |
last_modification_date | 1605645885 |
last_submission_date | 1605582797 |
magic | PE32 executable for MS Windows (GUI) Intel 80386 32-bit |
md5 | 84c82835a5d21bbcf75a61706d8ab549 |
meaningful_name | diskpart.exe |
names | [diskpart.exe, C:\Users\Work PC\Downloads\Test\Ransomware\Ransomware.WannaCry\ed01ebfbc9eb5bbea545af4d01bf5f1071661840480439c6e5babe8e080e41aa.exe, ed01ebfbc9eb5bbea545af4d01bf5f1071661840480439c6... |
packers | {'PEiD': 'Microsoft Visual C++'} |
pe_info | {'compiler_product_versions': ['id: 12, version: 7291 count=2', 'id: 11, version: 8047 count=1', 'id: 14, version: 7299 count=4', 'id: 10, version: 8047 count=11', 'id: 4, version: 8047 count=4', ... |
reputation | -2633 |
sha1 | 5ff465afaabcbf0150d1a3ab2c2e74f3a4426467 |
sha256 | ed01ebfbc9eb5bbea545af4d01bf5f1071661840480439c6e5babe8e080e41aa |
sigma_analysis_stats | {'critical': 2, 'high': 0, 'low': 1, 'medium': 2} |
sigma_analysis_summary | {'Sigma Integrated Rule Set (GitHub)': {'critical': 2, 'high': 0, 'low': 1, 'medium': 2}} |
signature_info | {'copyright': '© Microsoft Corporation. All rights reserved.', 'description': 'DiskPart', 'file version': '6.1.7601.17514 (win7sp1_rtm.101119-1850)', 'internal name': 'diskpart.exe', 'original nam... |
size | 3514368 |
ssdeep | 98304:QqPoBhz1aRxcSUDk36SAEdhvxWa9P593R8yAVp2g3x:QqPe1Cxcxk3ZAEUadzR8yc4gB |
tags | [peexe, self-delete, overlay, runtime-modules, direct-cpu-clock-access, via-tor, executes-dropped-file] |
times_submitted | 1325 |
tlsh | T173F533F4E221B7ACF2550EF64855C59B6A9724B2EBEF1E26DA8001A70D44F7F8FC0491 |
total_votes | {'harmless': 28, 'malicious': 292} |
trid | [{'file_type': 'Win32 Executable MS Visual C++ (generic)', 'probability': 38.5}, {'file_type': 'Microsoft Visual C++ compiled executable (generic)', 'probability': 20.4}, {'file_type': 'Win16 NE e... |
type_description | Win32 EXE |
type_extension | exe |
type_tag | peexe |
unique_sources | 980 |
vhash | 036046656d1570a8z3631lz1fz |
zemana_behaviour | [dll-injection] |
example_relationship_df = vt_lookup.lookup_ioc_relationships(
observable=FILE,
vt_type='file',
relationship='execution_parents')
example_relationship_df
target_type | source_type | relationship_type | ||
---|---|---|---|---|
source | target | |||
ed01ebfbc9eb5bbea545af4d01bf5f1071661840480439c6e5babe8e080e41aa | 018ac8f95d5e14b92011cdbfc8c48056ca4891161ed6bdd268770a5b56bb327f | file | file | execution_parents |
02a7977d1faf7bfc93a4b678a049c9495ea663e7065aa5a6caf0f69c5ff25dbd | file | file | execution_parents | |
06b020a3fd3296bc4c7bf53307fe7b40638e7f445bdd43fac1d04547a429fdaf | file | file | execution_parents | |
06c676bf8f5c6af99172c1cf63a84348628ae3f39df9e523c42447e2045e00ff | file | file | execution_parents | |
070f603e0443b1fae57425210fb3b27c2f77d8983cfefefb0ee185de572df33d | file | file | execution_parents | |
... | ... | ... | ... | |
f1aa23299987eed2173e83d26b6078232051f885586ebba35699143b83bc68ad | file | file | execution_parents | |
f2916486e380d0c0bbd31694b05509b91f0f622478595eba89b30031f9f64c3c | file | file | execution_parents | |
fbf74ee5d011dfb0d6c3357446ea3999ef62b088c553d665847aece28a1d3e2b | file | file | execution_parents | |
ff6af3f113f61f823e422b7eb9e379495b81bdbb66a4e4e159b4caee8a79bada | file | file | execution_parents | |
0d592a8d7e13210140f106a897a211b839608c2e9e86f20419e30d4087b7ac03 | file | file | execution_parents |
106 rows × 3 columns
The function lookup_iocs
is able to obtain attributes for all the rows in a DataFrame. If no observable_column
and observable_type
parameters are specified, the function will obtain the attributes of all the entities that are in the column target
, and will obtain their types from the target_type
column.
This function is especially useful when a user has obtained a set of relationships, and would like to obtain their attributes.
Note: it can take some time to fetch results, depending on the number of nodes and relationships.
example_multiple_attribute_df = vt_lookup.lookup_iocs(example_relationship_df)
example_multiple_attribute_df
last_submission_date | size | times_submitted | meaningful_name | type_description | first_submission_date | detections | scans | first_submission | last_submission | type | |
---|---|---|---|---|---|---|---|---|---|---|---|
id | |||||||||||
018ac8f95d5e14b92011cdbfc8c48056ca4891161ed6bdd268770a5b56bb327f | 1526215996 | 3723264 | 6 | 8479206ff1a47362199ddabebb7358d2.virus | Win32 EXE | 1495139411 | 67 | 74 | 2017-05-18 20:30:11+00:00 | 2018-05-13 12:53:16+00:00 | file |
02a7977d1faf7bfc93a4b678a049c9495ea663e7065aa5a6caf0f69c5ff25dbd | 1571387079 | 9164800 | 4 | =?UTF-8?B?572R5piT5bel5YW3566x56uv5ZCv5YqoLmV4ZQ==?= | Win32 EXE | 1570020111 | 52 | 75 | 2019-10-02 12:41:51+00:00 | 2019-10-18 08:24:39+00:00 | file |
06b020a3fd3296bc4c7bf53307fe7b40638e7f445bdd43fac1d04547a429fdaf | 1588342161 | 3991221 | 1 | Tender.pdf.exe | Win32 EXE | 1588342161 | 55 | 75 | 2020-05-01 14:09:21+00:00 | 2020-05-01 14:09:21+00:00 | file |
06c676bf8f5c6af99172c1cf63a84348628ae3f39df9e523c42447e2045e00ff | 1595479073 | 4535704 | 1 | car.exe | Win32 EXE | 1595479073 | 51 | 76 | 2020-07-23 04:37:53+00:00 | 2020-07-23 04:37:53+00:00 | file |
070f603e0443b1fae57425210fb3b27c2f77d8983cfefefb0ee185de572df33d | 1601363298 | 3723264 | 9 | lhdfrgui.exe | Win32 EXE | 1504687270 | 68 | 74 | 2017-09-06 08:41:10+00:00 | 2020-09-29 07:08:18+00:00 | file |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
f1aa23299987eed2173e83d26b6078232051f885586ebba35699143b83bc68ad | 1563994865 | 3723392 | 1 | lhdfrgui.exe | Win32 EXE | 1563994865 | 64 | 72 | 2019-07-24 19:01:05+00:00 | 2019-07-24 19:01:05+00:00 | file |
f2916486e380d0c0bbd31694b05509b91f0f622478595eba89b30031f9f64c3c | 1518624409 | 3676610 | 5 | acdsee.ultimate.10.x.unipatch_WannaCry.exe | Win32 EXE | 1498115823 | 54 | 69 | 2017-06-22 07:17:03+00:00 | 2018-02-14 16:06:49+00:00 | file |
fbf74ee5d011dfb0d6c3357446ea3999ef62b088c553d665847aece28a1d3e2b | 1573073940 | 3811580 | 1 | Presentation.exe | Win32 EXE | 1573073940 | 28 | 72 | 2019-11-06 20:59:00+00:00 | 2019-11-06 20:59:00+00:00 | file |
ff6af3f113f61f823e422b7eb9e379495b81bdbb66a4e4e159b4caee8a79bada | 1576634480 | 3597101 | 1 | ShieldPassword.exe | Win32 EXE | 1576634480 | 22 | 70 | 2019-12-18 02:01:20+00:00 | 2019-12-18 02:01:20+00:00 | file |
0d592a8d7e13210140f106a897a211b839608c2e9e86f20419e30d4087b7ac03 | 1583318742 | 3723264 | 1 | lhdfrgui.exe | Win32 EXE | 1583318742 | 66 | 75 | 2020-03-04 10:45:42+00:00 | 2020-03-04 10:45:42+00:00 | file |
106 rows × 11 columns
Also, if we would like to obtain the relationships for a set of entities, we have the function lookup_iocs_relationships
. Here also, if no observable_column
and observable_type
parameters are specified, the function will obtain the relationships of all the entities that are in the column target
, and will obtain their types from the target_type
column.
Note: it can take some time to fetch results
example_multiple_relationship_df = vt_lookup.lookup_iocs_relationships(example_relationship_df, 'contacted_domains')
example_multiple_relationship_df
target_type | source_type | relationship_type | ||
---|---|---|---|---|
source | target | |||
018ac8f95d5e14b92011cdbfc8c48056ca4891161ed6bdd268770a5b56bb327f | www.iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com | domain | file | contacted_domains |
02a7977d1faf7bfc93a4b678a049c9495ea663e7065aa5a6caf0f69c5ff25dbd | fkksjobnn43.org | domain | file | contacted_domains |
070f603e0443b1fae57425210fb3b27c2f77d8983cfefefb0ee185de572df33d | www.iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com | domain | file | contacted_domains |
76jdd2ir2embyv47.onion | domain | file | contacted_domains | |
xxlvbrloxvriy2c5.onion | domain | file | contacted_domains | |
... | ... | ... | ... | ... |
0d592a8d7e13210140f106a897a211b839608c2e9e86f20419e30d4087b7ac03 | 76jdd2ir2embyv47.onion | domain | file | contacted_domains |
xxlvbrloxvriy2c5.onion | domain | file | contacted_domains | |
gx7ekbenv2riucmf.onion | domain | file | contacted_domains | |
57g7spgrzlojinas.onion | domain | file | contacted_domains | |
cwwnhwhlz52maqm7.onion | domain | file | contacted_domains |
202 rows × 3 columns
We can display a simple plot of the relataionships locally but it doesn't tell us much about what the nodes are and they types of relationships between them.
from bokeh.io import output_notebook, show
from bokeh.plotting import figure, from_networkx
from bokeh.models import HoverTool
graph = nx.from_pandas_edgelist(
example_multiple_relationship_df.reset_index(),
source="source",
target="target",
edge_attr="relationship_type",
)
plot = figure(
title="Simple graph plot", x_range=(-1.1, 1.1), y_range=(-1.1, 1.1), tools="hover"
)
g_plot = from_networkx(graph, nx.spring_layout, scale=2, center=(0, 0))
plot.renderers.append(g_plot)
output_notebook()
show(plot)
Once we have some DataFrames with the relationships, we are able to generate and visualize a VT Graph in our notebook. The function create_vt_graph
accepts as input a list of Relationship DataFrames.
Note: it can take some time to generate the graph, depending on the number of nodes and relationships.
Unlike our local graph, this displays rich information about the nodes and relationship and allows us to expand our investigation with further searches or ad hoc nodes.
Note: - the inline graph displays node attributes but doesn't allow you edit or to add to the graph with further searches.
Click on the link in the frame to go to the VirusTotal site to view.
graph_id = vt_lookup.create_vt_graph(
relationship_dfs=[example_relationship_df, example_multiple_relationship_df],
name="My first Jupyter Notebook Graph",
private=False,
)
graph_id
'g20091e04457e441ab3d061480caf5e3c626208e1da5a41e08522f78b4e31b574'
vt_lookup.render_vt_graph(
graph_id = graph_id,
width = 900,
height = 600
)