vbrandl/masterthesis
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Content

Browse Source
This commit is contained in:
Valentin Brandl 2022-04-07 19:37:02 +02:00
parent dc10f471cd
commit e7d8c3131d
5 changed files with 67 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
abstract.tex
View File

@ -1,6 +1,10 @@
\begin{abstract}
% TODO
TODO: abstract
Botnets pose a huge risk on general internet infrastructure and services.
Decentralized \Acs*{p2p} topologies make it harder to detect monitor and take those botnets offline.
This work explores ways to make monitoring of fully distributed botnets more efficient, resilient and harder to detect, by using a collaborative, coordinated approach.
\todo{do me}
\end{abstract}
% vim: set filetype=tex ts=2 sw=2 tw=0 et :
\keywords{\Acs*{p2p}, botnet, monitoring, collaboration}
% vim: set filetype=tex ts=2 sw=2 tw=0 et spell :

13
bibliography.bib
View File

@ -115,6 +115,17 @@
archivedate = {2021-10-22},
}
@online{bib:netlab_mozi,
title = {The Mostly Dead Mozi and Its' Lingering Bots},
date = {2021-08-30},
author = {Turing, Alex and Wang, Hui and Ye, Genshen},
organization = {360 Netlab},
url = {https://blog.netlab.360.com/the-mostly-dead-mozi-and-its-lingering-bots/},
urldate = {2022-04-07},
archiveurl = {https://web.archive.org/web/20220130162722/https://blog.netlab.360.com/the-mostly-dead-mozi-and-its-lingering-bots/},
archivedate = {2022-01-30},
}
@article{bib:fan_p2p_2014,
title = {A P2P Botnet Detection Method Used On-line Monitoring and Off-line Detection},
volume = {8},
@ -137,7 +148,7 @@
isbn = {978-1-4503-6747-9},
url = {https://dl.acm.org/doi/10.1145/3319535.3363281},
doi = {10.1145/3319535.3363281},
shorttitle = {Poster},
shorttitle = {Challenges of Accurately Measuring Churn in P2P Botnets},
eventtitle = {{CCS} '19: 2019 {ACM} {SIGSAC} Conference on Computer and Communications Security},
pages = {2661--2663},
booktitle = {Proceedings of the 2019 {ACM} {SIGSAC} Conference on Computer and Communications Security},

7
commands.tex
View File

@ -1,4 +1,11 @@
\newcommand{\eg}{\textit{e}.\textit{g}.}
% Keywords command
\providecommand{\keywords}[1]
{
\small
\textbf{\textit{Keywords---}} #1
}
% absolute value
\DeclarePairedDelimiter\abs{\lvert}{\rvert}%

90
content.tex
View File

@ -4,27 +4,27 @@
The internet has become an irreplaceable part of our day-to-day lives.
We are always connected via numerous \enquote{smart} and \ac{iot} devices.
We use the internet to communicate, shop, handle financial transactions, and much more.
Many personal and professional workflows are so dependent on the internet, that they won't work when being offline, and with the pandemic, we are living through, this dependency grew even bigger.
Many personal and professional workflows are so dependent on the internet, that they won't work when being offline, and with the pandemic, we are living through, this dependency grew even stronger.
%{{{ motivation
\subsection{Motivation}
The number of connected \ac{iot} devices is around 10 billion in 2021 and is estimated to be constantly growing over the next years up to 25 billion in 2030~\cite{bib:statista_iot_2020}.
Many of these devices run on outdated software, don't receive any updates, and don't follow general security best practices.
In 2021 there were around 10 billion internet connected \ac{iot} devices and this number is estimated to more than double over the next years up to 25 billion in 2030~\cite{bib:statista_iot_2020}.
Many of these devices run on outdated software, don't receive regular updates, and don't follow general security best practices.
While in 2016 only \SI{77}{\percent} of German households had a broadband connection with a bandwidth of \SI{50}{\mega\bit\per\second} or more, in 2020 it was already \SI{95}{\percent} with more than \SI{50}{\mega\bit\per\second} and \SI{59}{\percent} with at least \SI{1000}{\mega\bit\per\second}~\cite{bib:statista_broadband_2021}.
Their nature as small devices---often without any direct user interaction---that are always online and behind internet connections that are getting faster and faster makes them a desirable target for botnets.
Their nature as small, always online devices---often without any direct user interaction---behind internet connections that are getting faster and faster makes them a desirable target for botnet operators.
In recent years, \ac{iot} botnets have been responsible for some of the biggest \ac{ddos} attacks ever recorded---creating up to \SI{1}{\tera\bit\per\second} of traffic~\cite{bib:ars_ddos_2016}.
A botnet is a network of infected computers with some means of communication to control the infected systems.
Classic botnets use one or more central coordinating hosts called \ac{c2} servers.
These \ac{c2} servers could use any protocol from \ac{irc} over \ac{http} to Twitter~\cite{bib:pantic_covert_2015} as communication channel with the infected hosts.
Abusive use of infected systems includes several things\todo{things = bad}---\ac{ddos} attacks, banking fraud, as proxies to hide the attacker's identity, send spam emails\dots{}
Centralized botnets use one or more coordinating hosts called \ac{c2} servers.
These \ac{c2} servers can use any protocol from \ac{irc} over \ac{http} to Twitter~\cite{bib:pantic_covert_2015} as communication channel with the infected hosts.
The abuse of infected systems includes several activities---\ac{ddos} attacks, banking fraud, as proxies to hide the attacker's identity, send spam emails\dots{}
Analyzing and shutting down a centralized botnet is comparatively easy since the central means of communication (the \ac{c2} IP address or domain name, Twitter handle or \ac{irc} channel) are publicly known.
Analyzing and shutting down a centralized botnet is comparatively easy since the central means of communication (the \ac{c2} IP address or domain name, Twitter handle or \ac{irc} channel), can be extracted from the malicious binaries and are therefore publicly known.
A coordinated operation with help from law enforcement, hosting providers, domain registrars, and platform providers could shut down or take over the operation by changing how requests are routed or simply shutting down the controlling servers/accounts.
To complicate take-down attempts, botnet operators came up with a number of ideas: \acp{dga} use pseudorandomly generated domain names to render simple domain blacklist-based approaches ineffective~\cite{bib:antonakakis_dga_2012} or fast-flux \ac{dns}, where a large pool of IP addresses is used assigned randomly to the \ac{c2} domains to prevent IP based blacklisting~\cite{bib:nazario_as_2008}.
To complicate take-down attempts, botnet operators came up with a number of ideas: \acp{dga} use pseudorandomly generated domain names to render simple domain blacklist-based approaches ineffective~\cite{bib:antonakakis_dga_2012} or fast-flux \ac{dns}, where a large pool of IP addresses is assigned randomly to the \ac{c2} domains to prevent IP based blacklisting~\cite{bib:nazario_as_2008}.
%{{{ fig:c2vsp2p
\begin{figure}[h]
@ -45,18 +45,20 @@ To complicate take-down attempts, botnet operators came up with a number of idea
A number of botnet operations were shut down like this~\cite{bib:nadji_beheading_2013} and as the defenders upped their game, so did attackers---the concept of \ac{p2p} botnets emerged.
The idea is to build a decentralized network without \acp{spof} in the form of \ac{c2} servers as shown in \autoref{fig:p2p}.
In a \ac{p2p} botnet, each node in the network knows a number of its neighbors and connects to those, each of these neighbors has a list of neighbors on his own, and so on.
Any of the nodes in \autoref{fig:p2p} could be the bot master but they don't even have to be online all the time since the peers will stay connected autonomously.
In a \ac{p2p} botnet, each node in the network knows a number of its neighbors and connects to those, each of these neighbors has a list of neighbors on its own, and so on.
The bot master only needs to join the network to send new commands or receive stolen data.
Any of the nodes in \autoref{fig:p2p} could be the bot master but they don't even have to be online all the time since the peers will stay connected autonomously.
In fact there have been arrests of operators of \ac{p2p} botnets but due to the autonomy offered by the decentralized approach, the botnet keeps communicating~\cite{bib:netlab_mozi}.
Especially worm-like botnets, where each peer tries to find and infect other systems, the network can keep lingering for many years.
This lack of a \ac{spof} makes \ac{p2p} botnets more resilient to take-down attempts since the communication is not stopped and bot masters can easily rejoin the network and send commands.
The constantly growing damage produced by botnets has many researchers and law enforcement agencies trying to shut down these operations~\cite{bib:nadji_beheading_2013, bib:nadji_still_2017, bib:dittrich_takeover_2012, bib:fbiTakedown2014}.
The monetary value of these botnets directly correlates with the amount of effort bot masters are willing to put into implementing defense mechanisms against take-down attempts.
Some of these countermeasures include deterrence, which limits the number of allowed bots per IP address or subnet to 1; blacklisting, where known crawlers and sensors are blocked from communicating with other bots in the network (mostly IP based); disinformation, when fake bots are placed in the neighborhood lists, which invalidates the data collected by crawlers; and active retaliation like \ac{ddos} attacks against sensors or crawlers~\cite{bib:andriesse_reliable_2015}.
\todo{take-down? take down?}
Some of these countermeasures include deterrence, which limits the number of allowed bots per IP address or subnet to 1; blacklisting, where known crawlers and sensors are blocked from communicating with other bots in the network (mostly IP based); disinformation, when fake bots are placed in the peer lists, which invalidates the data collected by crawlers; and active retaliation like \ac{ddos} attacks against sensors or crawlers~\cite{bib:andriesse_reliable_2015}.
Successful take-downs of a \ac{p2p} botnet requires intricate knowledge over the network topology, protocol characteristics and participating peers.
This work aims to make the monitoring and information gathering phase more efficient and resilient to detection.
%}}} motivation
@ -69,15 +71,20 @@ A \ac{p2p} botnet can be modelled as a digraph
G &= (V, E)
\end{align*}
With the set of vertices \(V\) describing the bots in the network and the set of edges \(E\) describing the communication flow between bots.
With the set of vertices \(V\) describing the peers in the network and the set of edges \(E\) describing the communication flow between bots.
\(\forall v \in V\), the predecessors \(\text{pred}(v)\) and successors \(\text{succ}(v)\) are defined as:
\(G\) is not required to be a connected graph but might consist of multiple disjoint components~\cite{bib:rossow_sok_2013}. Components consisting of peers, that are infected by the same bot, are considered part of the same graph.
\(\forall v \in V\), the \textbf{predecessors} \(\text{pred}(v)\) and \textbf{successors} \(\text{succ}(v)\) are defined as:
\begin{align*}
\text{succ}(v) &= \{ u \in V \mid (u, v) \in E \} \\
\text{pred}(v) &= \{ u \in V \mid (v, u) \in E \}
\end{align*}
The set of edges \(\text{pred}(v)\) is also called the \textbf{peer list} of \(v\).
Those are the nodes, a peer will connect to, to request new commands and other peers.
For a vertex \(v \in V\), the in and out degree \(\deg^{+}\) and \(\deg^{-}\) describe how many bots know \(v\) or are known by \(v\) respectively.
\begin{align*}
@ -85,6 +92,8 @@ For a vertex \(v \in V\), the in and out degree \(\deg^{+}\) and \(\deg^{-}\) de
\deg^{-}(v) &= \abs{\text{succ}(v)}
\end{align*}
\todo{more details}
%}}} formal model
%{{{ detection techniques
@ -123,7 +132,7 @@ There are three subtypes of active detection:
\begin{enumerate}
\item Crawlers: recursively ask known bots for their neighbourhood lists
\item Crawlers: recursively ask known bots for their peer lists
\item Sensors: implement a subset of the botnet protocol and become part of the network without performing malicious actions
@ -155,7 +164,7 @@ There are three subtypes of active detection:
%{{{ methodology
\section{Methodology}
The implementation of the concepts of this work will be done as part of \ac{bms}\footnotemark, a monitoring platform for \ac{p2p} botnets described by \citeauthor{bib:bock_poster_2019} in~\cite{bib:bock_poster_2019}.
The implementation of the concepts of this work will be done as part of \ac{bms}\footnotemark, a monitoring platform for \ac{p2p} botnets described by \citeauthor{bib:bock_poster_2019} in \citetitle{bib:bock_poster_2019}.
\footnotetext{\url{https://github.com/Telecooperation/BMS}}
\Ac{bms} uses a hybrid active approach of crawlers and sensors (reimplementations of the \ac{p2p} protocol of a botnet, that won't perform malicious actions) to collect live data from active botnets.
@ -168,7 +177,7 @@ The goal of this work is to complicate detection mechanisms like this for bot ma
The coordinated work distribution also helps in efficiently monitoring large botnets where one sensor is not enough to track all peers.
The changes should allow the current sensors to use the new abstraction with as few changes as possible to the existing code.
The final results should be as general as possible and not depend on any botnet's specific behaviour, but it assumes, that every \ac{p2p} botnet has some kind of \enquote{getNeighbourList} method in the protocol, that allows other peers to request a list of active nodes to connect to.
The final results should be as general as possible and not depend on any botnet's specific behaviour, but it assumes, that every \ac{p2p} botnet has some kind of \enquote{getPeerList} method in the protocol, that allows other peers to request a list of active nodes to connect to.
In the current implementation, each crawler will itself visit and monitor each new node it finds.
The idea for this work is to report newfound nodes back to the \ac{bms} backend first, where the graph of the known network is created, and a fitting worker is selected to archive the goal of the according coordination strategy.
@ -179,12 +188,6 @@ If it is not possible, to select a specific sensor so that the monitoring activi
The improved crawler system should allow new crawlers to register themselves and their capabilities (\eg{} bandwidth, geolocation ), so the amount of work can be scaled accordingly between hosts.
Further work might even consider autoscaling the monitoring activity using some kind of cloud computing provider.
To validate the result, the old sensor implementation will be compared to the new system using different graph metrics.
\todo{maybe?}
If time allows, \ac{bsf}\footnotemark{} will be used to simulate a botnet place sensors in the simulated network and measure the improvement achieved by the coordinated monitoring effort.
\footnotetext{\url{https://github.com/tklab-tud/BSF}}
%}}} methodology
%{{{ primitives
@ -202,7 +205,7 @@ The coordination protocol must allow the following operations:
\subsubsection{Report Edge}
\mintinline{go}{reportEdge(edges)}: Report found edges. Edges are found by querying the neighbourhood list of known peers. This is how new peers are detected.
\mintinline{go}{reportEdge(edges)}: Report found edges. Edges are found by querying the peer list of known peers. This is how new peers are detected.
\subsubsection{Request Tasks}
@ -336,16 +339,6 @@ for _, peer := range peers {
\end{minted}
\todo{reference for wrr}
\begin{table}[H]
\center
\begin{tabular}{lll}
\(C_n\) & \(B_c\) & \(W_c\) \\
0 & 100 & \(\frac{10}{16}\) \\
1 & 10 & \(\frac{1}{16}\) \\
2 & 50 & \(\frac{5}{16}\) \\
\end{tabular}
\end{table}
\todo{remove me}
\subsubsection{IP-based Partitioning}\label{sec:ip_part}
@ -485,12 +478,12 @@ One of those, \enquote{SensorBuster} uses \acp{wcc} since crawlers don't have an
Building a complete graph \(G_C = K_{\abs{C}}\) between the crawlers by making them return the other crawlers on peer list requests would still produce a disconnected component and while being bigger and maybe not as obvious at first glance, it is still easily detectable since there is no path from \(G_C\) back to the main network (see~\autoref{fig:sensorbuster2} and~\autoref{fig:metrics_table}).
\todo{rank? deg+ - deg-?}
With \(v \in V\), \(\text{succ}(v)\) being the set of successors of \(v\) and \(\text{pred}(v)\) being the set of predecessors of \(v\), PageRank is recursively defined as~\cite{bib:page_pagerank_1998}:
\[
\text{PR}(v) = \text{dampingFactor} \times \sum\limits_{p \in \text{pred}(v)} \frac{\text{PR}(p)}{\abs{\text{succ}(p)}} + \frac{1 - \text{dampingFactor}}{\abs{V}}
\]
\begin{align*}
\text{PR}_0(v) &= \text{initialRank} \\
\text{PR}_{n+1}(v) &= \text{dampingFactor} \times \sum\limits_{p \in \text{pred}(v)} \frac{\text{PR}_n(p)}{\abs{\text{succ}(p)}} + \frac{1 - \text{dampingFactor}}{\abs{V}}
\end{align*}
For the first iteration, the PageRank of all nodes is set to the same initial value. When iterating often enough, any value can be chosen~\cite{bib:page_pagerank_1998}.
@ -498,7 +491,7 @@ The dampingFactor describes the probability of a person visiting links on the we
For simplicity---and since it is not required to model human behaviour for automated crawling and ranking---a dampingFactor of \(1.0\) will be used, which simplifies the formula to
\[
\text{PR}(v) = \sum\limits_{p \in \text{pred}(v)} \frac{\text{rank}(p)}{\abs{\text{succ}(p)}}
\text{PR}_{n+1}(v) = \sum\limits_{p \in \text{pred}(v)} \frac{\text{PR}_n(p)}{\abs{\text{succ}(p)}}
\]
Based on this, SensorRank is defined as
@ -526,11 +519,10 @@ Applying PageRank once with an initial rank of \(0.25\) once on the example grap
\caption{Values for metrics from~\autoref{fig:sensorbuster} (a/b)}\label{fig:metrics_table}
\end{table}
\todo{big graphs, how many Kn to get significant?}
While this works for small networks, the crawlers must account for a significant amount of peers in the network for this change to be noticeable.
The generated \(K_n\) needs to be at least as big as the smallest regular component in the botnet, which is not feasible.
While this works for small networks, the crawlers must account for a significant amount of peers in the network for this change to be noticeable.\todo{for bigger (generated) graphs?}
In our experiments on a snapshot of the Sality~\cite{bib:falliere_sality_2011} botnet exported from \ac{bms} over the span of \daterange{2021-04-21}{2021-04-28}\todo{export timespan}, even 1 iteration were enough to get distinct enough values to detect sensors and crawlers.
In our experiments on a snapshot of the Sality~\cite{bib:falliere_sality_2011} botnet obtained from \ac{bms} over the span of \daterange{2021-04-21}{2021-04-28} even 1 iteration were enough to get distinct enough values to detect sensors and crawlers.
\begin{table}[H]
\centering
@ -632,9 +624,9 @@ Looking at the data in smaller buckets of one hour each, the average number of s
\todo{use better data?}
%}}}fig:avg_out_edges
Since crawlers never respond to neighbourhood list requests, they will always be detectable by the described approach but sensors might benefit from the following technique.
Since crawlers never respond to peer list requests, they will always be detectable by the described approach but sensors might benefit from the following technique.
By responding to neighbourhood list requests with plausible data, one can move make those metrics less suspicious, because it produces valid outgoing edges from the sensors.
By responding to peer list requests with plausible data, one can move make those metrics less suspicious, because it produces valid outgoing edges from the sensors.
The hard part is deciding which peers can be returned without actually supporting the network.
The following candidates to place into the NL will be investigated:
@ -684,9 +676,9 @@ Also this does not help against the \ac{wcc} metric since this would create a bi
\subsubsection{Use Churned Peers After IP Rotation}
Churn describes the dynamics of peer participation of \ac{p2p} systems, \eg{} join and leave events~\cite{bib:stutzbach_churn_2006}.\todo{übergang}
Detecting if a peer just left the system, in combination with knowledge about \acp{as}, peers that just left and came from an \ac{as} with dynamic IP allocation (\eg{} many consumer broadband providers in the US and Europe), can be placed into the crawler's neighbourhood list.\todo{what is an AS}
Detecting if a peer just left the system, in combination with knowledge about \acp{as}, peers that just left and came from an \ac{as} with dynamic IP allocation (\eg{} many consumer broadband providers in the US and Europe), can be placed into the crawler's peer list.\todo{what is an AS}
If the timing of the churn event correlates with IP rotation in the \ac{as}, it can be assumed, that the peer left due to being assigned a new IP address---not due to connectivity issues or going offline---and will not return using the same IP address.
These peers, when placed in the neighbourhood list of the crawlers, will introduce paths back into the main network and defeat the \ac{wcc} metric.
These peers, when placed in the peer list of the crawlers, will introduce paths back into the main network and defeat the \ac{wcc} metric.
It also helps with the PageRank and SensorRank metrics since the crawlers start to look like regular peers without actually supporting the network by relaying messages or propagating active peers.
%}}} churned peers
@ -776,7 +768,7 @@ The new implementation consists of three main interfaces:
\item \mintinline{go}{ReportPeer}, to report newly found peers
\item \mintinline{go}{Protocol}, the actual botnet protocol implementation used to ping a peer and request its neighbourhood list
\item \mintinline{go}{Protocol}, the actual botnet protocol implementation used to ping a peer and request its peer list
\end{itemize}
Currently there are two sources \mintinline{go}{FindPeer} can use: read peers from a file on disk or request them from the \ac{grpc} BMS coordinator.

BIN
report.pdf
View File

Binary file not shown.