---

Workshop 1.2: Acquiring Data

• Contributors:

  • Roberto Rodriguez (@Cyb3rWard0g)

  • Jose Rodriguez (@Cyb3rPandah)

  • Ian Hellen (@ianhellen, gh:@ianhelle)
    
    

• Agenda:

  • Reading data from SIEMs and Databases

  • OTR Security Datasets - (aka Mordor)
    
    

• Notebook: https://aka.ms/Jupyterthon-ws-1-2

• License: Creative Commons Attribution-ShareAlike 4.0 International

• Q&A - OTR Discord #Jupyterthon #WORKSHOP DAY 1 - ACQUIRING DATA

---

Reading from SIEMs and Databases

Elasticsearch

https://elasticsearch-py.readthedocs.io/

python -m pip install elasticsearch

• Importing libraries:

# Elasticsearch connector
from elasticsearch import Elasticsearch
from elasticsearch_dsl import Search
# Data manipulation
import pandas as pd

• Initializing an Elasticsearch client:

  Initialize an Elasticsearch client using a specific Elasticsearch URL. Next, you can pass the client to the Search object that we will use to represent the search request in a little bit.

es = Elasticsearch(['http://<elasticsearch-ip>:9200'])
searchContext = Search(using=es, index='logs-*', doc_type='doc')

• Setting the query search context:

  In addition, we will need to use the query class to pass an Elasticsearch query_string . For example, what if I want to query event_id 1 events?.

s = searchContext.query('query_string', query='event_id:1')

• Running query & Exploring response:

  Finally, you can run the query and get the results back as a DataFrame.

response = s.execute()

if response.success():
    df = pd.DataFrame((d.to_dict() for d in s.scan()))

df

Connect to Elasticsearch (Elasticsearch DSL Libraries)

Reference: https://medium.com/threat-hunters-forge/jupyter-notebooks-from-sigma-rules-️-to-query-elasticsearch-31a74cc59b99

Splunk

Huntlib - https://github.com/target/huntlib

Also

SplunkSDK - https://github.com/splunk/splunk-sdk-python

• Importing libraries:

from huntlib.splunk import SplunkDF

• Running query & Exploring response

df = s.search_df(
  spl="search index=win_events EventCode=4688",
  start_time="-2d@d",
  end_time="@d"
)

Sqlite

• Importing libraries:

!pip install ipython-sql

• Loading library

%%capture
%load_ext sql

• Connecting to database

%sql sqlite:///../data/browser2.db

• Executing queries

%%sql
SELECT
    name
FROM
    sqlite_master
WHERE
    type='table';

 * sqlite:///../data/browser2.db
Done.

name
android_metadata
bookmarks
sqlite_sequence
history
images
searches
settings
thumbnails
_sync_state
_sync_state_metadata

%%sql
SELECT * FROM history;

 * sqlite:///../data/browser2.db
Done.

_id	title	url	created	date	visits	user_entered
1	Google	http://www.google.es/?gfe_rd=cr&dcr=0&ei=_OtlWu_eK5St8wespoTYBg	0	1516628986890	2	0
2	mobile congress - Buscar con Google	http://www.google.es/search?dcr=0&source=hp&ei=_OtlWqfYOIH-UPnHrqAH&sjs=16383&q=mobile+congress&oq=mobile+congress&gs_l=mobile-gws-hp.3..0l5.9699.17759..20649.......143.1677.2j13............mobile-gws-wiz-hp.....0..0i131.9Koqktw5naA%3D	0	1516629009457	1	0
3	Home | Mobile World Congress	https://www.mobileworldcongress.com/	0	1516629026677	3	0
4	Google	https://www.google.es/webhp?source=android-home&gws_rd=cr&dcr=0&ei=LexlWsqTBMXvUOOsg6gF	0	1516629037678	2	0
5	apk mirror - Buscar con Google	https://www.google.es/search?source=android-home&dcr=0&source=hp&ei=LexlWvvkIIzXUbDmn8gB&sjs=16383&q=apk+mirror&oq=apk+mirror&gs_l=mobile-gws-hp.3..0l5.4318.7753..7951.......136.1152.1j9............mobile-gws-wiz-hp.....0..0i131j0i10.GOGxmTJuhIg%3D	0	1516629047435	1	0
6	APKMirror - Free APK Downloads - Download Free Android APKs #APKPLZ	https://www.apkmirror.com/	0	1516629052435	2	0
7	Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more	https://www.amazon.com/	0	1516629109159	2	0
8	http://192.168.74.128/i6ADxOqMEyyI	http://192.168.74.128/i6ADxOqMEyyI	0	1516629327266	1	0
9	http://192.168.74.128/i6ADxOqMEyyI/EeMVfx/	http://192.168.74.128/i6ADxOqMEyyI/EeMVfx/	0	1516629327667	1	0
10	apks - Google Search	http://www.google.es/search?hl=en&source=android-browser-type&q=apks&gws_rd=cr&dcr=0&ei=dO1lWoSxGMjSUYrwkpgG	0	1516629364685	2	0

• Save query results in a Pandas DataFrame

df = _.DataFrame()

df

	_id	title	url	created	date	visits	user_entered
0	1	Google	http://www.google.es/?gfe_rd=cr&dcr=0&ei=_OtlW...	0	1516628986890	2	0
1	2	mobile congress - Buscar con Google	http://www.google.es/search?dcr=0&source=hp&ei...	0	1516629009457	1	0
2	3	Home | Mobile World Congress	https://www.mobileworldcongress.com/	0	1516629026677	3	0
3	4	Google	https://www.google.es/webhp?source=android-hom...	0	1516629037678	2	0
4	5	apk mirror - Buscar con Google	https://www.google.es/search?source=android-ho...	0	1516629047435	1	0
5	6	APKMirror - Free APK Downloads - Download Free...	https://www.apkmirror.com/	0	1516629052435	2	0
6	7	Amazon.com: Online Shopping for Electronics, A...	https://www.amazon.com/	0	1516629109159	2	0
7	8	http://192.168.74.128/i6ADxOqMEyyI	http://192.168.74.128/i6ADxOqMEyyI	0	1516629327266	1	0
8	9	http://192.168.74.128/i6ADxOqMEyyI/EeMVfx/	http://192.168.74.128/i6ADxOqMEyyI/EeMVfx/	0	1516629327667	1	0
9	10	apks - Google Search	http://www.google.es/search?hl=en&source=andro...	0	1516629364685	2	0

Log analytics workspace

Requirements:

• Azure AD application

  • Secret text (credentials)

  • API permissions granted:

    • Service: Log Analytics API

    • Permission: Data.Read

    • Type: AppRole

• Add application with Log Analytics Reader role to Log Analytic Workspace Access Control list.

Get OAUTH Access Token

• Application ID

• Scope: https://api.loganalytics.io/.default

• TenantId

• Application secret

https://securitydatasets.com/create/azureloganalyticsapi.html

import requests
import time

appId = "AppId"
scope = "https://api.loganalytics.io/.default"
tenantId = "TenantID"
secret = 'ApplicationSecret'

endpoint = f'https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/token'
http_headers = {'Accept': 'application/json','Content-Type': 'application/json'}
data = {"scope" : scope, 'grant_type' : 'client_credentials', 'client_id' : appId, 'client_secret' : secret}
results = requests.get(endpoint, json=data, headers=http_headers, stream=False).json()

access_token = results["access_token"]

Run Query

workspaceId = 'Workspace ID'
apiUri = f'https://api.loganalytics.io/v1/workspaces/{workspaceId}/query'

query = 'query'

HttpHeaders = {"Authorization" : f'Bearer {access_token}', 'Accept': 'application/json','Content-Type': 'application/json'}
data = {'query' : query}

query_results = requests.get(apiUri, json=data, headers=http_headers, stream=False).json()

M365 advanced hunting APIs

Requirements:

• Azure AD application

  • Secret text (credentials)

  • API permissions granted:

    • Service: Microsoft Threat Protection

    • Permission:

      • AdvancedHunting.Read.All

      • Incident.Read.All

    • Type: AppRole

Get OAUTH Access Token

• Application ID

• Scope: https://api.security.microsoft.com/.default

• TenantId

• Application secret

https://securitydatasets.com/create/m365defenderhuntapi.html

MSTICPy Data Providers

There’s a later session devoted to MSTICPy

%pip install msticpy

MSTICPy isn’t a data source - just wraps a bunch of data sources in common API.

Currently supports:

• MS Sentinel (aka Azure Sentinel, Log Analytics), MS Defender, MS Graph, Azure Resource Graph

• Splunk

• Sumologic

• Local data (CSV or Pickle) - would be easy to add supported pandas formats (any requests?)

• Experimental support for Kusto/Azure Data explorer

Typical usage:

• import QueryProvider class

• instantiate a QueryProvider object with the required provider name.

• run query_provider.connect() method - params vary (e.g. connection string)

• pre-defined, parameterized queries appear as methods of the query_provider class

• ad hoc queries via .exec_query() method

• output returned as a pandas dataframe

If you use the MSTICPy init (init_notebook), the QueryProvider is imported for you

import msticpy
msticpy.init_notebook(globals())

# To install
# %pip install msticpy

# Alternative import - init_notebook imports QueryProvider and a bunch of other stuff
# import msticpy
# msticpy.init_notebook(globals())

from msticpy.data import QueryProvider
sentinel_prov = QueryProvider("AzureSentinel")

local_prov = QueryProvider("LocalData", query_paths=["../data"], data_paths=["../data"])

Accessing queries as functions

(usually need to connect before running one)

sentinel_prov.

sentinel_prov.browse()

---

Gets latest VMComputer record for Host

Parameters

add_query_items: str (optional)

Additional query clauses

end: datetime (optional)

Query end time

host_name: str

The Computer name of the VM

start: datetime (optional)

Query start time

(default value is: -5)

table: str (optional)

Table name

(default value is: VMComputer)

Query

{table} 
| where TimeGenerated >= datetime({start}) 
| where TimeGenerated <= datetime({end}) 
| where Computer has "{host_name}" 
| take 1

Example

{QueryProvider}[.QueryPath].QueryName(params...)

qry_prov.Azure.get_vmcomputer_for_host(start=start, end=end, hostname=host)

local_prov.Network.list_network_flows().head()

	TenantId	TimeGenerated	FlowStartTime	FlowEndTime	FlowIntervalEndTime	FlowType	ResourceGroup	VMName	VMIPAddress	PublicIPs	...	DestPort	FlowDirection	AllowedOutFlows	AllowedInFlows	DeniedInFlows	DeniedOutFlows	RemoteRegion	VMRegion	AllExtIPs	TotalAllowedFlows
0	52b1ab41-869e-4138-9e40-2a4457f09bf0	2019-02-14 13:23:59.512	2019-02-14 12:21:58	2019-02-14 12:21:58	2019-02-14 13:00:00	AzurePublic	asihuntomsworkspacerg	msticalertswin1	10.0.3.5	[13.67.143.117]	...	443.0	O	1.0	0.0	0.0	0.0	centralus	eastus	13.67.143.117	1.0
1	52b1ab41-869e-4138-9e40-2a4457f09bf0	2019-02-14 13:23:59.512	2019-02-14 12:29:02	2019-02-14 12:29:02	2019-02-14 13:00:00	AzurePublic	asihuntomsworkspacerg	msticalertswin1	10.0.3.5	[40.77.232.95]	...	443.0	O	1.0	0.0	0.0	0.0	westcentralus	eastus	40.77.232.95	1.0
2	52b1ab41-869e-4138-9e40-2a4457f09bf0	2019-02-14 03:26:06.765	2019-02-14 02:08:46	2019-02-14 02:48:45	2019-02-14 03:00:00	AzurePublic	asihuntomsworkspacerg	msticalertswin1	10.0.3.5	[13.65.107.32, 40.124.45.19]	...	443.0	O	4.0	0.0	0.0	0.0	southcentralus	eastus	13.65.107.32	4.0
3	52b1ab41-869e-4138-9e40-2a4457f09bf0	2019-02-14 03:26:06.765	2019-02-14 02:08:46	2019-02-14 02:48:45	2019-02-14 03:00:00	AzurePublic	asihuntomsworkspacerg	msticalertswin1	10.0.3.5	[13.65.107.32, 40.124.45.19]	...	443.0	O	4.0	0.0	0.0	0.0	southcentralus	eastus	40.124.45.19	4.0
4	52b1ab41-869e-4138-9e40-2a4457f09bf0	2019-02-14 03:26:06.828	2019-02-14 02:30:56	2019-02-14 02:30:56	2019-02-14 03:00:00	AzurePublic	asihuntomsworkspacerg	msticalertswin1	10.0.3.5	[20.38.98.100]	...	443.0	O	1.0	0.0	0.0	0.0	eastus	eastus	20.38.98.100	1.0

5 rows × 24 columns

sentinel_prov.connect(
    "loganalytics://code().tenant('72f988bf-86f1-41af-91ab-2d7cd011db47').workspace('8ecf8077-cf51-4820-aadd-14040956f35d')"
)

Connecting... 

RRS3HK9GP Copy code to clipboard and authenticate

popup schema 8ecf8077-cf51-4820-aadd-14040956f35d@loganalytics

connected

---

OTR Security Datasets (aka Mordor)

Reading OTR-SecurityDatasets from code

The Security Datasets project is an open-source initiatve that contributes pre-recorded datasets that describe malicious activity, from different platforms, to the infosec community to expedite data analysis and threat research.

We will consider the following steps to access Security Datasets:

- Importing required Python Libraries
- Making a HTTP request to GitHub repository
- Generating ZipFile (Compressed) object from Bytes data
- Extracting JSON file from ZipFile object
- Reading JSON file (Lines = True)

• Let’s start by importing the required Python libraries in order to access Security Datasets’ content:

# Generate HTTP request
import requests
# Zip file object manipulation
from zipfile import ZipFile
# Byte data manipulations
from io import BytesIO
# Read JSON file
from pandas.io import json

• We will make a HTTP request to the Security Datasets GitHub repo using the get method, and we are storing the reponse content in the variable zipFileRequest.

  It is important to note that we are using the raw data link related to the dataset. This type of links usually starts with https://raw.githubusercontent.com/ + Project reference.

url = 'https://raw.githubusercontent.com/OTRF/Security-Datasets/master/datasets/atomic/windows/discovery/host/empire_shell_net_localgroup_administrators.zip'
zipFileRequest = requests.get(url)
type(zipFileRequest)

requests.models.Response

• The type of data of the content of HTTP repsonse is Bytes.

type(zipFileRequest.content)

bytes

• We will create a BytesIO object to access the response content and store it in a ZipFile object. All the data manipulation is performed in memory.

zipFile = ZipFile(BytesIO(zipFileRequest.content))
type(zipFile)

zipfile.ZipFile

• Any ZipFile object can contain more than one file. We can access the list of files’ names using the namelist method. Since the Security Datasets contains one file, we will reference the first element of the list when extracting the JSON file.

zipFile.namelist()

['empire_shell_net_localgroup_administrators_2020-09-21191843.json']

• We will extract the JSON file from the compressed folder using the extract method. After running the code below, we will download and store the file in the directory specified in the path parameter.

  It is important to note that this method returns the normalized path to the JSON file. We are storing the directory path in the datasetJSONPAth variable and use it when trying to read the file.

datasetJSONPath = zipFile.extract(zipFile.namelist()[0], path = '../data')

print(datasetJSONPath)

../data/empire_shell_net_localgroup_administrators_2020-09-21191843.json

• Now that the file was downloaded and know its direcotry path, we can read the JSON file using the read_json method.

  It is important to note that, when recording Security Dataset, each line of the JSON file represents an event. Therefore, it is important to set the parameter lines to True.

dataset = json.read_json(path_or_buf = datasetJSONPath, lines=True)

• The read_json method returns a DataFrame object. We will share more details of what a dataframe is in the next section of this workshop.

type(dataset)

pandas.core.frame.DataFrame

• Finally, we should be able to start exploring our dataset using different functions or method such as head.

dataset.head(n=1)

	Keywords	SeverityValue	TargetObject	EventTypeOrignal	EventID	ProviderGuid	ExecutionProcessID	host	Channel	UserID	...	SourceIsIpv6	DestinationPortName	DestinationHostname	Service	Details	ShareName	EnabledPrivilegeList	DisabledPrivilegeList	ShareLocalPath	RelativeTargetName
0	-9223372036854775808	2	HKU\S-1-5-21-4228717743-1032521047-1810997296-...	INFO	12	{5770385F-C22A-43E0-BF4C-06F5698FFBD9}	3172	wec.internal.cloudapp.net	Microsoft-Windows-Sysmon/Operational	S-1-5-18	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

1 rows × 155 columns

Using MSTICPy to access Security Datasets

#%pip install msticpy

import pandas as pd
from msticpy.data import QueryProvider
from msticpy.vis import mp_pandas_plot


qry_prov_sd = QueryProvider("Mordor")

qry_prov_sd.connect()

Retrieving Mitre data...
Retrieving Mordor data...

Downloading Mordor metadata: 100%|██████████| 95/95 [00:22<00:00,  4.27 files/s]

qry_prov_sd.list_queries()[:10]

['atomic.aws.collection.ec2_proxy_s3_exfiltration',
 'atomic.linux.defense_evasion.host.sh_binary_padding_dd',
 'atomic.linux.discovery.host.sh_arp_cache',
 'atomic.windows.collection.host.msf_record_mic',
 'atomic.windows.credential_access.host.cmd_lsass_memory_dumpert_syscalls',
 'atomic.windows.credential_access.host.cmd_psexec_lsa_secrets_dump',
 'atomic.windows.credential_access.host.cmd_sam_copy_esentutl',
 'atomic.windows.credential_access.host.covenant_dcsync_dcerpc_drsuapi_DsGetNCChanges',
 'atomic.windows.credential_access.host.empire_dcsync_dcerpc_drsuapi_DsGetNCChanges',
 'atomic.windows.credential_access.host.empire_mimikatz_backupkeys_dcerpc_smb_lsarpc']

qry_prov_sd.search_queries("empire + localgroup")

['atomic.windows.discovery.host.empire_shell_net_localgroup_administrators (Empire Net Local Administrators Group)']

emp_df = qry_prov_sd.atomic.windows.discovery.host.empire_shell_net_localgroup_administrators()
emp_df.head()

https://raw.githubusercontent.com/OTRF/Security-Datasets/master/datasets/atomic/windows/discovery/host/empire_shell_net_localgroup_administrators.zip
Extracting empire_shell_net_localgroup_administrators_2020-09-21191843.json

	Keywords	SeverityValue	TargetObject	EventTypeOrignal	EventID	ProviderGuid	ExecutionProcessID	host	Channel	UserID	...	SourceIsIpv6	DestinationPortName	DestinationHostname	Service	Details	ShareName	EnabledPrivilegeList	DisabledPrivilegeList	ShareLocalPath	RelativeTargetName
0	-9223372036854775808	2	HKU\S-1-5-21-4228717743-1032521047-1810997296-...	INFO	12	{5770385F-C22A-43E0-BF4C-06F5698FFBD9}	3172	wec.internal.cloudapp.net	Microsoft-Windows-Sysmon/Operational	S-1-5-18	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
1	0	2	NaN	NaN	4103	{A0C1853B-5C40-4B15-8766-3CF1C58F985A}	7456	wec.internal.cloudapp.net	Microsoft-Windows-PowerShell/Operational	S-1-5-21-4228717743-1032521047-1810997296-1104	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
2	0	2	NaN	NaN	4103	{A0C1853B-5C40-4B15-8766-3CF1C58F985A}	7456	wec.internal.cloudapp.net	Microsoft-Windows-PowerShell/Operational	S-1-5-21-4228717743-1032521047-1810997296-1104	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
3	-9214364837600034816	2	NaN	NaN	5158	{54849625-5478-4994-A5BA-3E3B0328C30D}	4	wec.internal.cloudapp.net	Security	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
4	-9214364837600034816	2	NaN	NaN	5156	{54849625-5478-4994-A5BA-3E3B0328C30D}	4	wec.internal.cloudapp.net	Security	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

5 rows × 155 columns

Make sure that timestamps actually are timestamps, not strings

emp_df["EventTime"] = pd.to_datetime(emp_df["EventTime"])

emp_df.mp_plot.timeline(time_column="EventTime", group_by="EventID")

Loading BokehJS ...

Column(

id = '1530', …)

align = 'start',

aspect_ratio = None,

background = None,

children = [Figure(id='1068', ...), Figure(id='1100', ...)],

css_classes = [],

disabled = False,

height = None,

height_policy = 'auto',

js_event_callbacks = {},

js_property_callbacks = {},

margin = (0, 0, 0, 0),

max_height = None,

max_width = None,

min_height = None,

min_width = None,

name = None,

rows = 'auto',

sizing_mode = None,

spacing = 0,

subscribed_events = [],

syncable = True,

tags = [],

visible = True,

width = None,

width_policy = 'auto')

Security Datasets Browser

• Browser properties

• Filter by MITRE Tactic/Technique

• Search across metadata, file names

• Download selected datasets

from msticpy.data.browsers.mordor_browser import MordorBrowser
m_browser = MordorBrowser()

Retrieving Mitre data...
Retrieving Mordor data...

	Message	EventID	SourceName	TimeCreated	Hostname	Task	Level	Keywords	Channel	ProviderGuid	...	ParentProcessGuid	LogonGuid	LogonId	Device	StartFunction	TargetProcessGuid	StartModule	SourceProcessGuid	StartAddress	NewThreadId
0	The audit log was cleared.\r\nSubject:\r\n\tSe...	1102	Microsoft-Windows-Eventlog	2020-10-21T09:40:38.926Z	WORKSTATION5	104	4	0x4020000000000000	Security	{fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
1	The Windows Filtering Platform has permitted a...	5158	Microsoft-Windows-Security-Auditing	2020-10-21T09:40:40.709Z	WORKSTATION5	12810	0	0x8020000000000000	Security	{54849625-5478-4994-a5ba-3e3b0328c30d}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
2	The Windows Filtering Platform has permitted a...	5156	Microsoft-Windows-Security-Auditing	2020-10-21T09:40:40.709Z	WORKSTATION5	12810	0	0x8020000000000000	Security	{54849625-5478-4994-a5ba-3e3b0328c30d}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
3	An attempt was made to duplicate a handle to a...	4690	Microsoft-Windows-Security-Auditing	2020-10-21T09:40:43.571Z	WORKSTATION5	12807	0	0x8020000000000000	Security	{54849625-5478-4994-a5ba-3e3b0328c30d}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
4	The handle to an object was closed.\r\n\r\nSub...	4658	Microsoft-Windows-Security-Auditing	2020-10-21T09:40:43.571Z	WORKSTATION5	12801	0	0x8020000000000000	Security	{54849625-5478-4994-a5ba-3e3b0328c30d}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
504	Process accessed:\r\nRuleName: -\r\nUtcTime: 2...	10	Microsoft-Windows-Sysmon	2020-10-21T09:41:04.166Z	WORKSTATION5	10	4	0x8000000000000000	Microsoft-Windows-Sysmon/Operational	{5770385f-c22a-43e0-bf4c-06f5698ffbd9}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
505	Process accessed:\r\nRuleName: -\r\nUtcTime: 2...	10	Microsoft-Windows-Sysmon	2020-10-21T09:41:04.166Z	WORKSTATION5	10	4	0x8000000000000000	Microsoft-Windows-Sysmon/Operational	{5770385f-c22a-43e0-bf4c-06f5698ffbd9}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
506	Process accessed:\r\nRuleName: -\r\nUtcTime: 2...	10	Microsoft-Windows-Sysmon	2020-10-21T09:41:04.166Z	WORKSTATION5	10	4	0x8000000000000000	Microsoft-Windows-Sysmon/Operational	{5770385f-c22a-43e0-bf4c-06f5698ffbd9}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
507	Process accessed:\r\nRuleName: -\r\nUtcTime: 2...	10	Microsoft-Windows-Sysmon	2020-10-21T09:41:04.166Z	WORKSTATION5	10	4	0x8000000000000000	Microsoft-Windows-Sysmon/Operational	{5770385f-c22a-43e0-bf4c-06f5698ffbd9}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
508	The System log file was cleared.	104	Microsoft-Windows-Eventlog	2020-10-21T09:40:38.973Z	WORKSTATION5	104	4	0x8000000000000000	System	{fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

509 rows × 114 columns

Downloaded data available in browser.current_dataset

m_browser.current_dataset.head(3)

	Message	EventID	SourceName	TimeCreated	Hostname	Task	Level	Keywords	Channel	ProviderGuid	...	ParentProcessGuid	LogonGuid	LogonId	Device	StartFunction	TargetProcessGuid	StartModule	SourceProcessGuid	StartAddress	NewThreadId
0	The audit log was cleared.\r\nSubject:\r\n\tSe...	1102	Microsoft-Windows-Eventlog	2020-10-21T09:40:38.926Z	WORKSTATION5	104	4	0x4020000000000000	Security	{fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
1	The Windows Filtering Platform has permitted a...	5158	Microsoft-Windows-Security-Auditing	2020-10-21T09:40:40.709Z	WORKSTATION5	12810	0	0x8020000000000000	Security	{54849625-5478-4994-a5ba-3e3b0328c30d}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
2	The Windows Filtering Platform has permitted a...	5156	Microsoft-Windows-Security-Auditing	2020-10-21T09:40:40.709Z	WORKSTATION5	12810	0	0x8020000000000000	Security	{54849625-5478-4994-a5ba-3e3b0328c30d}	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

3 rows × 114 columns

Cached datasets available in browser.datasets

m_browser.datasets

{'https://raw.githubusercontent.com/OTRF/Security-Datasets/master/datasets/atomic/windows/defense_evasion/host/psh_mavinject_dll_notepad.zip':                                                Message  EventID  \
 0    The audit log was cleared.\r\nSubject:\r\n\tSe...     1102   
 1    The Windows Filtering Platform has permitted a...     5158   
 2    The Windows Filtering Platform has permitted a...     5156   
 3    An attempt was made to duplicate a handle to a...     4690   
 4    The handle to an object was closed.\r\n\r\nSub...     4658   
 ..                                                 ...      ...   
 504  Process accessed:\r\nRuleName: -\r\nUtcTime: 2...       10   
 505  Process accessed:\r\nRuleName: -\r\nUtcTime: 2...       10   
 506  Process accessed:\r\nRuleName: -\r\nUtcTime: 2...       10   
 507  Process accessed:\r\nRuleName: -\r\nUtcTime: 2...       10   
 508                   The System log file was cleared.      104   
 
                               SourceName               TimeCreated  \
 0             Microsoft-Windows-Eventlog  2020-10-21T09:40:38.926Z   
 1    Microsoft-Windows-Security-Auditing  2020-10-21T09:40:40.709Z   
 2    Microsoft-Windows-Security-Auditing  2020-10-21T09:40:40.709Z   
 3    Microsoft-Windows-Security-Auditing  2020-10-21T09:40:43.571Z   
 4    Microsoft-Windows-Security-Auditing  2020-10-21T09:40:43.571Z   
 ..                                   ...                       ...   
 504             Microsoft-Windows-Sysmon  2020-10-21T09:41:04.166Z   
 505             Microsoft-Windows-Sysmon  2020-10-21T09:41:04.166Z   
 506             Microsoft-Windows-Sysmon  2020-10-21T09:41:04.166Z   
 507             Microsoft-Windows-Sysmon  2020-10-21T09:41:04.166Z   
 508           Microsoft-Windows-Eventlog  2020-10-21T09:40:38.973Z   
 
          Hostname   Task  Level            Keywords  \
 0    WORKSTATION5    104      4  0x4020000000000000   
 1    WORKSTATION5  12810      0  0x8020000000000000   
 2    WORKSTATION5  12810      0  0x8020000000000000   
 3    WORKSTATION5  12807      0  0x8020000000000000   
 4    WORKSTATION5  12801      0  0x8020000000000000   
 ..            ...    ...    ...                 ...   
 504  WORKSTATION5     10      4  0x8000000000000000   
 505  WORKSTATION5     10      4  0x8000000000000000   
 506  WORKSTATION5     10      4  0x8000000000000000   
 507  WORKSTATION5     10      4  0x8000000000000000   
 508  WORKSTATION5    104      4  0x8000000000000000   
 
                                   Channel  \
 0                                Security   
 1                                Security   
 2                                Security   
 3                                Security   
 4                                Security   
 ..                                    ...   
 504  Microsoft-Windows-Sysmon/Operational   
 505  Microsoft-Windows-Sysmon/Operational   
 506  Microsoft-Windows-Sysmon/Operational   
 507  Microsoft-Windows-Sysmon/Operational   
 508                                System   
 
                                ProviderGuid  ... ParentProcessGuid  LogonGuid  \
 0    {fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}  ...               NaN        NaN   
 1    {54849625-5478-4994-a5ba-3e3b0328c30d}  ...               NaN        NaN   
 2    {54849625-5478-4994-a5ba-3e3b0328c30d}  ...               NaN        NaN   
 3    {54849625-5478-4994-a5ba-3e3b0328c30d}  ...               NaN        NaN   
 4    {54849625-5478-4994-a5ba-3e3b0328c30d}  ...               NaN        NaN   
 ..                                      ...  ...               ...        ...   
 504  {5770385f-c22a-43e0-bf4c-06f5698ffbd9}  ...               NaN        NaN   
 505  {5770385f-c22a-43e0-bf4c-06f5698ffbd9}  ...               NaN        NaN   
 506  {5770385f-c22a-43e0-bf4c-06f5698ffbd9}  ...               NaN        NaN   
 507  {5770385f-c22a-43e0-bf4c-06f5698ffbd9}  ...               NaN        NaN   
 508  {fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}  ...               NaN        NaN   
 
     LogonId Device StartFunction TargetProcessGuid  StartModule  \
 0       NaN    NaN           NaN               NaN          NaN   
 1       NaN    NaN           NaN               NaN          NaN   
 2       NaN    NaN           NaN               NaN          NaN   
 3       NaN    NaN           NaN               NaN          NaN   
 4       NaN    NaN           NaN               NaN          NaN   
 ..      ...    ...           ...               ...          ...   
 504     NaN    NaN           NaN               NaN          NaN   
 505     NaN    NaN           NaN               NaN          NaN   
 506     NaN    NaN           NaN               NaN          NaN   
 507     NaN    NaN           NaN               NaN          NaN   
 508     NaN    NaN           NaN               NaN          NaN   
 
      SourceProcessGuid StartAddress NewThreadId  
 0                  NaN          NaN         NaN  
 1                  NaN          NaN         NaN  
 2                  NaN          NaN         NaN  
 3                  NaN          NaN         NaN  
 4                  NaN          NaN         NaN  
 ..                 ...          ...         ...  
 504                NaN          NaN         NaN  
 505                NaN          NaN         NaN  
 506                NaN          NaN         NaN  
 507                NaN          NaN         NaN  
 508                NaN          NaN         NaN  
 
 [509 rows x 114 columns]}

---

End of Session

Break: 5 Minutes