vbrandl/masterthesis
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Content

Browse Source
This commit is contained in:
Valentin Brandl 2022-04-18 13:50:47 +02:00
parent db2a4c42d2
commit 65a3ffecc8
5 changed files with 168 additions and 121 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
abstract.tex
View File

@ -1,7 +1,10 @@
\begin{abstract}
Botnets pose a huge risk on general internet infrastructure and services.
Distributed \Acs*{p2p} topologies make it harder to detect, monitor and take those botnets offline.
This work explores ways to make monitoring of fully distributed botnets more efficient, resilient and harder to detect, by using a collaborative, coordinated approach.
Botnets pose a huge risk to general internet infrastructure and services.
Distributed \Acs*{p2p} topologies make it harder to detect and take those botnets offline.
To take a \ac{p2p} botnet down, it has to be monitored to estimate the size and learn about the network topology.
With the growing damage and monetary value produced by such botnets, ideas emerged on how to detect and prevent monitoring activity in the network.
This work explores ways to make monitoring of fully distributed botnets more efficient, resilient, and harder to detect, by using a collaborative, coordinated approach.
Further, we show how the coordinated approach helps in circumventing anti-monitoring techniques deployed by botnets.
\todo{do me}
\end{abstract}

27
bibliography.bib
View File

@ -82,6 +82,16 @@
archivedate = {2021-10-25}
}
@online{bib:msZloader,
title = {Notorious cybercrime gang’s botnet disrupted},
organization = {Microsoft},
author = {Hogan-Burney, Amy},
url = {https://blogs.microsoft.com/on-the-issues/2022/04/13/zloader-botnet-disrupted-malware-ukraine/},
urldate = {2022-04-15},
archiveurl = {https://web.archive.org/web/20220413210653/https://blogs.microsoft.com/on-the-issues/2022/04/13/zloader-botnet-disrupted-malware-ukraine/},
archivedate = {2022-04-13},
}
@online{bib:fbiTakedown2014,
title = {Taking Down Botnets},
organization = {Federal Bureau of Investigation},
@ -448,4 +458,21 @@
note = {Series Title: Lecture Notes in Computer Science}
}
@article{greengard_war_2012,
title = {The war against botnets},
volume = {55},
issn = {0001-0782, 1557-7317},
url = {https://dl.acm.org/doi/10.1145/2076450.2076456},
doi = {10.1145/2076450.2076456},
abstract = {Increasingly sophisticated botnets have emerged during the last several years. However, security researchers, businesses, and governments are attacking botnets from a number of different angles---and sometimes winning.},
pages = {16--18},
number = {2},
journaltitle = {Communications of the {ACM}},
shortjournal = {Commun. {ACM}},
author = {Greengard, Samuel},
urldate = {2022-04-18},
date = {2012-02},
langid = {english}
}
/* vim: set filetype=bib ts=2 sw=2 tw=0 et :*/

253
content.tex
View File

@ -4,23 +4,23 @@
The Internet has become an irreplaceable part of our day-to-day lives.
We are always connected via numerous \enquote{smart} and \ac{iot} devices.
We use the Internet to communicate, shop, handle financial transactions, and much more.
Many personal and professional workflows are so dependent on the Internet, that they won't work when being offline, and with the pandemic we are living through, this dependency grew even stronger.
Many personal and professional workflows are so dependent on the Internet, that they won't work when being offline, and with the pandemic, we are living through, this dependency grew even stronger.
In 2021, there were around 10 billion Internet connected \ac{iot} devices and this number is estimated to more than double over the next years up to 25 billion in 2030~\cite{bib:statista_iot_2020}.
In 2021, there were around 10 billion Internet-connected \ac{iot} devices and this number is estimated to more than double over the next years up to 25 billion in 2030~\cite{bib:statista_iot_2020}.
Many of these devices run on outdated software, don't receive regular updates, and don't follow general security best practices.
While in 2016 only \SI{77}{\percent} of German households had a broadband connection with a bandwidth of \SI{50}{\mega\bit\per\second} or more, in 2020 it was already \SI{95}{\percent} with more than \SI{50}{\mega\bit\per\second} and \SI{59}{\percent} with at least \SI{1000}{\mega\bit\per\second}~\cite{bib:statista_broadband_2021}.
Their nature as small, always online devices---often without any direct user interaction---behind Internet connections that are getting faster and faster makes them a desirable target for \emph{botnet} operators.
A \emph{botnet} is a network of malware infected computers, called \emph{bots}, controlled by a \emph{botmaster}.
A \emph{botnet} is a network of malware-infected computers, called \emph{bots}, controlled by a \emph{botmaster}.
Botnets are controlled via a \emph{\ac{c2} channel}.
The communication patterns of a \ac{c2} channel can be \emph{centralized}, \emph{decentralized} or \emph{distributed}.
The communication patterns of a \ac{c2} channel can be \emph{centralized}, \emph{decentralized}, or \emph{distributed}.
Centralized or decentralized botnets use one or more coordinating hosts to contact and receive new commands.
Distributed botnets create a \emph{\ac{p2p}} network as their communication layer.
The \ac{c2} channel for centralized and decentralized botnets can use anything from \ac{irc} over HTTP to Twitter~\cite{bib:pantic_covert_2015}.
In recent years, \ac{iot} botnets have been responsible for some of the biggest \ac{ddos} attacks ever recorded---creating up to \SI{1}{\tera\bit\per\second} of traffic~\cite{bib:ars_ddos_2016}.
Other malicious use of bots includes several activities---\ac{ddos} attacks, banking fraud, proxies to hide the attacker's identity, sending of spam emails, just to name a few.
Other malicious use of bots includes several activities---\ac{ddos} attacks, banking fraud, proxies to hide the attacker's identity, and sending of spam emails, just to name a few.
The constantly growing damage produced by botnets has many researchers and law enforcement agencies trying to shut down these operations~\cite{bib:nadji_beheading_2013, bib:nadji_still_2017, bib:dittrich_takeover_2012, bib:fbiTakedown2014}.
A coordinated operation with help from law enforcement, hosting providers, domain registrars, and platform providers could shut down or take over the operation by changing how requests are routed or simply shutting down the controlling servers/accounts.
@ -28,7 +28,7 @@ A coordinated operation with help from law enforcement, hosting providers, domai
The monetary value of these botnets directly correlates with the amount of effort botmasters are willing to put into implementing defense mechanisms against take-down attempts.
Botnet operators came up with a number of ideas: \acp{dga} use pseudorandomly generated domain names to render simple domain blacklist-based approaches ineffective~\cite{bib:antonakakis_dga_2012} or fast-flux DNS entries, where a large pool of IP addresses is randomly assigned to the \ac{c2} domains to prevent IP based blacklisting and hide the actual \ac{c2} servers~\cite{bib:nazario_as_2008}.
Analyzing and shutting down a centralized or decentralized botnet is comparatively easy since the central means of communication (the \ac{c2} IP addresses or domain names, Twitter handles or \ac{irc} channels), can be extracted from the malicious binaries or determined by analyzing network traffic and can therefore be considered publicly known.
A number of botnet operations were taken down by shutting down the \ac{c2} channel~\cite{bib:nadji_beheading_2013} and as the defenders upped their game, so did attackers---the concept of \ac{p2p} botnets emerged.
A number of botnet operations were taken down by shutting down the \ac{c2} channel~\cite{bib:nadji_beheading_2013, bib:msZloader} and as the defenders upped their game, so did attackers---the concept of \ac{p2p} botnets emerged.
The idea is to build a distributed network without \acp{spof} in the form of \ac{c2} servers as shown in \Fref{fig:p2p}.
%{{{ fig:c2vsp2p
@ -50,12 +50,13 @@ The idea is to build a distributed network without \acp{spof} in the form of \ac
This lack of a \ac{spof} makes \ac{p2p} botnets more resilient to take-down attempts since there is no easy way to stop the communication and botmasters can easily rejoin the network and send new commands.
Taking down a \ac{p2p} botnet requires intricate knowledge over botnet's characteristics, \eg{} size, risk, distribution over IP subnets or geolocations, network topology, participating peers and protocol characteristics.
This can be obtained by monitoring peers activity of known participants in the botnet.
Taking down a \ac{p2p} botnet requires intricate knowledge of the botnet's characteristics, \eg{} size, risk, distribution over IP subnets or geolocations, network topology, participating peers, and protocol characteristics.
Just like for centralized and decentralized botnets, to take down a \ac{p2p} botnet, the \ac{c2} channel needs to be identified and disrupted.
By \emph{monitoring} peer activity of known participants in the botnet, this knowledge can be obtained and used to find attack vectors in the botnet protocol.
\todo{few words about monitoring}
In this work, we will show how a collaborative system of crawlers and sensors can make the monitoring and information gathering phase more efficient, resilient to detection and how collaborative monitoring can help circumventing anti-monitoring techniques.
In this work, we will show how a collaborative system of crawlers and sensors can make the monitoring and information gathering phase of a \ac{p2p} botnet more efficient, resilient to detection and how collaborative monitoring can help circumvent anti-monitoring techniques.
%}}} introduction
@ -70,14 +71,14 @@ In this work, we will show how a collaborative system of crawlers and sensors ca
%%}}} motivation
\clearpage{}
\section{Background and Related Work}
\section{Background}
In a \ac{p2p} botnet, each node in the network knows a number of its neighbors and connects to those. Each of these neighbors has a list of neighbors on its own, and so on.
The botmaster only needs to join the network to send new commands or receive stolen data but there is no need for a coordinating host, that is always connected to the network.
Any of the nodes in \Fref{fig:p2p} could be the botmaster but they don't even have to be online all the time since the peers will stay connected autonomously.
In fact, there have been arrests of \ac{p2p} botnet operators but due to the autonomy offered by the distributed approach, the botnet keeps intact and continues operating~\cite{bib:netlab_mozi}.
Especially worm-like botnets, where each peer tries to find and infect other systems, can keep lingering for many years.
Especially worm-like botnets, where each peer tries to actively find and infect other systems, can keep lingering for many years.
Bots in a \ac{p2p} botnet can be split into two distinct groups according to their reachability: peers that are not publicly reachable (\eg{} because they are behind a \ac{nat} router or firewall) and those, that are publicly reachable, also known as \emph{superpeers}.
@ -92,8 +93,8 @@ This is achieved by periodically querying their neighbor's neighbors in a proces
\Ac{mm} can be distinguished into two categories: \emph{structured} and \emph{unstructured}~\cite{bib:baileyNextGen}.
Structured \ac{p2p} botnets have strict rules for a bot's neighbors based on its unique ID and often use a \ac{dht}, which allows persisting data in a distributed network.
The \ac{dht} could contain a ordered ring structure of IDs and neighborhood in the structure also means neighborhood in the network, as is the case in the Kademila botnet~\cite{bib:kademlia2002}.
In botnets that employ unstructured \ac{mm} on the other hand, bots ask any peer they know for new peers to connect to, in a process called \emph{peer discovery}.
To enable peers to connect to unstructured botnets, the malware binaries include hardcoded lists of superpeers for the newly infected systems to connect to.
In \ac{p2p} botnets that employ unstructured \ac{mm} on the other hand, bots ask any peer they know for new peers to connect to, in a process called \emph{peer discovery}.
To enable peers to join a unstructured \ac{p2p} botnets, the malware binaries include hardcoded lists of superpeers for the newly infected systems to connect to.
The concept of \emph{churn} describes when a bot becomes unavailable.
There are two types of churn:
@ -137,8 +138,6 @@ For a node \(v \in V\), the in and out degree \(\deg^{+}\) and \(\deg^{-}\) desc
\deg^{-}(v) &= \abs{\text{succ}(v)}
\end{align*}
\todo{more details}
%}}} formal model
%{{{ detection techniques
@ -149,12 +148,12 @@ There are two distinct methods to map and get an overview of the network topolog
%{{{ passive detection
\subsubsection{Passive Monitoring}
For passive detection, traffic flows in large amounts of collected network traffic often obtained from \acp{isp} or network telescopes~\cite{bib:carnaNetworkTelescope2014} are analysed.
This has some advantages: \eg{} it is not possible for botmasters to detect or prevent data collection of that kind, though it is not trivial to distinguish valid \ac{p2p} application traffic (\eg{} BitTorrent, Skype, cryptocurrencies, \ldots) from \ac{p2p} bots.
\citeauthor{bib:zhang_building_2014} propose a system of statistical analysis to solve some of these problems in~\cite{bib:zhang_building_2014}.
For passive detection, traffic flows in large amounts of collected network traffic often obtained from \acp{isp} or network telescopes~\cite{bib:carnaNetworkTelescope2014} are analyzed.
This has some advantages: \eg{} it is not possible for botmasters to detect or prevent data-collection of that kind, though it is not trivial to distinguish valid \ac{p2p} application traffic (\eg{} BitTorrent, Skype, cryptocurrencies, \ldots) from \ac{p2p} bots.
\citeauthor{bib:zhang_building_2014} propose a system of statistical analysis to solve some of these problems in \citetitle{bib:zhang_building_2014}~\cite{bib:zhang_building_2014}.
Also getting access to the required datasets might not be possible for everyone.
As most botnet detection mechanisms, also the passive ones work by building communication graphs and finding tightly coupled subgraphs that might be indicative of a botnet~\cite{bib:botgrep2010}. An advantage of passive detection is, that it is independent of protocol details, specific binaries, or the structure of the network (\ac{p2p} vs.\ centralized/decentralized)~\cite{bib:botminer2008}.
Like most botnet detection mechanisms, also the passive ones work by building communication graphs and finding tightly coupled subgraphs that might be indicative of a botnet~\cite{bib:botgrep2010}. An advantage of passive detection is, that it is independent of protocol details, specific binaries, or the structure of the network (\ac{p2p} vs.\ centralized/decentralized)~\cite{bib:botminer2008}.
% \begin{itemize}
@ -173,15 +172,15 @@ Passive monitoring is only mentioned for completeness and not further discussed
\subsubsection{Active Monitoring}
For active detection, a subset of the botnet protocol and behavior is reimplemented to participate in the network.
To do so, samples of the malware are reverse engineered to unterstand and recreate the protocol.
This partial implementation includes the communication part of the botnet but ignores the malicious functionality as to not support and take part in illicit activity.
To do so, samples of the malware are reverse engineered to understand and recreate the protocol.
This partial implementation includes the communication part of the botnet but ignores the malicious functionality to not support and take part in illicit activity.
% The difference in behaviour from the reference implementation and conspicuous graph properties (\eg{} high \(\deg^{+}\) vs.\ low \(\deg^{-}\)) of these sensors allows botmasters to detect and block the sensor nodes.
There are two subtypes of active detection: \emph{sensors} wait to be contacted by other peers, while \emph{crawlers} actively query known bots and recursively ask for their neighbors~\cite{bib:karuppayah_sensorbuster_2017}.
Obviously crawlers can only detect superpeers and therefore only see a small subset of the network, while sensors are also contacted by peers in private networks and behind firewalls.
Crawlers can only detect superpeers and therefore only see a small subset of the network, while sensors are also contacted by peers in private networks and behind firewalls.
To accurately monitor a \ac{p2p} botnet, a hybrid approach of crawlers and sensors is required.
A crawler starts with a predefined list of known bots, connects to those and uses the peer exchange mechanism to request other bots.
A crawler starts with a predefined list of known bots, connects to those, and uses the peer exchange mechanism to request other bots.
Each found bot is crawled again, slowly building the graph of superpeers on the way.
Every entry \textit{E} in the peer exchange response received from bot \textit{A} represents an edge from \textit{A} to \textit{E} in the graph.
\citetitle{bib:andriesse_reliable_2015} describes disinformation attacks, in which bots will include invalid entries in their peer list replies~\cite{bib:andriesse_reliable_2015}.
@ -206,7 +205,35 @@ These include deterrence, which limits the number of bots per IP address or subn
%}}} detection techniques
\subsection{Related Work}
In \citetitle{bib:karuppayah_boobytrap_2016}~\cite{bib:karuppayah_boobytrap_2016}, the authors evaluate criteria to detect monitoring attempts in a \ac{p2p} botnet:
\begin{description}
\item[Defiance] Peers that don't abide by the \ac{mm} protocol rules are most likely crawlers or sensors, \eg{} peers that query other peers that shouldn't be in their neighborhood according to geolocation or IP subnet rules.
\item[Abuse] Higher \ac{mm} frequency as an indicator for a sensor or crawler
\item[Avoidance] Peers that avoid aiding the botnet, \eg{} by returning empty replies on \ac{mm} requests are potential monitoring nodes
\end{description}
\citetitle{bib:karuppayah_sensorbuster_2017} explores graph ranking algorithms to detect monitoring activity in a \ac{p2p} botnet.
They depend on suspicious graph properties to enumerate candidate peers~\cite{bib:karuppayah_sensorbuster_2017}.
\begin{description}
\item[PageRank] The algorithm used by Google to rank their search results uses the ratio of \(\deg^{+}\) and \(\deg^{-}\) to detect sensors since they have many incoming but few outgoing edges~\cite{bib:page_pagerank_1998}
\item[SensorRank] A deviation of PageRank that normalizes the result, to better account for churn and valid peers with few high-ranked predecessors but only a few successors
\item[SensorBuster] evaluates \ac{wcc} in a graph since sensors have only incoming but no outgoing edges, thereby creating a disconnected graph component
\end{description}
\Ac{bms}\footnotemark is a monitoring platform for \ac{p2p} botnets described by \citeauthor{bib:bock_poster_2019} in \citetitle{bib:bock_poster_2019}.
\footnotetext{\url{https://github.com/Telecooperation/BMS}}
\Ac{bms} is intended for a hybrid active approach of crawlers and sensors (reimplementations of the \ac{p2p} protocol of a botnet, that won't perform malicious actions) to collect live data from active botnets.
In an earlier project, we implemented different graph ranking algorithms---among others \emph{PageRank}~\cite{bib:page_pagerank_1998} and \emph{SensorRank}---to detect sensor candidates in a botnet, as described in \citetitle{bib:karuppayah_sensorbuster_2017}.
In an earlier project, we implemented the ranking algorithms described in \citetitle{bib:karuppayah_sensorbuster_2017} for \ac{bms}.
%%{{{ detection criteria
%\subsection{Detection Criteria}
@ -227,19 +254,10 @@ These include deterrence, which limits the number of bots per IP address or subn
\clearpage{}
\section{Methodology}
The implementation of the concepts of this work will be done as part of \ac{bms}\footnotemark, a monitoring platform for \ac{p2p} botnets described by \citeauthor{bib:bock_poster_2019} in \citetitle{bib:bock_poster_2019}.
\footnotetext{\url{https://github.com/Telecooperation/BMS}}
\Ac{bms} is intended for a hybrid active approach of crawlers and sensors (reimplementations of the \ac{p2p} protocol of a botnet, that won't perform malicious actions) to collect live data from active botnets.
In an earlier project, we implemented different graph ranking algorithms---among others \emph{PageRank}~\cite{bib:page_pagerank_1998} and \emph{SensorRank}---to detect sensor candidates in a botnet, as described in \citetitle{bib:karuppayah_sensorbuster_2017}.
Both ranking algorithms exploit the differences in a sensor's or crawler's \(\deg^+\) and \(\deg^-\) to weight the nodes.
Sensors will have few to none outgoing edges, since they don't participate actively in the botnet, while crawlers have only outgoing edges.
Another way to enumerate candidates for sensors in a \ac{p2p} botnet is to find \acp{wcc} in the graph.
\citeauthor{bib:karuppayah_sensorbuster_2017} call this method \emph{SensorBuster}.
The goal of this work is to complicate detection mechanisms like this for botmasters by coordinating the work of the system's crawlers and sensors, thereby reducing the node's rank for specific graph metrics.
The implementation of the concepts of this work will be done as part of \ac{bms}.
The goal of this work is to complicate detection and anti-monitoring mechanisms for botmasters by coordinating the work of the system's crawlers and sensors.
The coordinated work distribution also helps in efficiently monitoring large botnets where one crawler is not enough to track all peers.
The changes should allow the current \ac{bms} crawlers and sensors to use the new abstraction with as few changes as possible to the existing code.
The changes should allow the current \ac{bms} crawlers and sensors to use the new implementation with as few changes as possible to the existing code.
We will explore how cooperative monitoring of a \ac{p2p} botnet can help with the following problems:
@ -251,13 +269,13 @@ We will explore how cooperative monitoring of a \ac{p2p} botnet can help with th
\item Make crawling more efficient
\end{itemize}
The final results should be as general as possible and not depend on any botnet's specific behaviour (except for the mentioned anti-monitoring techniques which might be unique to some botnets), but we assume, that every \ac{p2p} botnet has some way of querying a bot's neighbors for the concept of crawlers and sensors to be applicable.
The final results should be as general as possible and not depend on any botnet's specific behavior (except for the mentioned anti-monitoring techniques which might be unique to some botnets), but we assume that every \ac{p2p} botnet has some way of querying a bot's neighbors for the concept of crawlers and sensors to be applicable.
In the current implementation, each crawler will itself visit and monitor each new node it finds.
The general idea for the implementation of the concepts in this thesis is to report newfound nodes back to the \ac{bms} backend first, where the graph of the known network is created, and a fitting worker is selected to achieve the goal of the according coordination strategy.
That worker will be responsible to monitor the new node.
If it is not possible, to select a sensor so that the monitoring activity stays inconspicuous, the coordinator can do a complete shuffle of all nodes between the sensors to restore the wanted graph properties or warn if more sensors are required to fulfill the goal defined by the strategy.
If it is not possible to select a sensor so that the monitoring activity stays inconspicuous, the coordinator can do a complete shuffle of all nodes between the sensors to restore the wanted graph properties or warn if more sensors are required to fulfill the goal defined by the strategy.
The improved crawler system should allow new crawlers to register themselves and their capabilities (\eg{} bandwidth, geolocation), so the amount of work can be scaled accordingly between hosts.
@ -270,8 +288,8 @@ The coordination protocol must allow the following operations:
\begin{description}
\item[Register Worker] Register a new worker with capabilities (which botnet, available bandwidth and processing power, if it is a crawler or sensor, \dots{}).
This is called periodically and used to determine which worker is still active, when assigning new tasks.
\item[Register Worker] Register a new worker with capabilities (which botnet, the available bandwidth and processing power, if it is a crawler or sensor, \dots{}).
This is called periodically and used to determine which worker is still active when assigning new tasks.
\item[Report Peer] Report found peers.
Both successful and failed attempts are reported, to detect churned peers, and blacklisted crawlers as soon as possible.
@ -320,7 +338,7 @@ Without loss of generality, if not stated otherwise, we assume that \(C\) is kno
There will be no joining or leaving crawlers.
This assumption greatly simplifies the implementation due to the lack of changing state that has to be tracked while still exploring the described strategies.
A production-ready implementation of the described techniques can drop this assumption but might have to recalculate the work distribution once a crawler joins or leaves.
The protocol primitives described in \Fref{sec:protPrim} already allow for this to be implemented by first creating tasks with the \mintinline{go}{StopCrawling} flag set to true for all active tasks, run the strategy again and create the according tasks to start crawling again.
The protocol primitives described in \Fref{sec:protPrim} already allow for this to be implemented by first creating tasks with the \mintinline{go}{StopCrawling} flag set to true for all active tasks, running the strategy again, and creating the according tasks to start crawling again.
%{{{ load balancing
\subsection{Load Balancing}\label{sec:loadBalancing}
@ -328,7 +346,7 @@ The protocol primitives described in \Fref{sec:protPrim} already allow for this
Depending on a botnet's size, a single crawler is not enough to monitor all superpeers.
While it is possible to run multiple, uncoordinated crawlers, two or more of them can find and monitor the same peer, making the approach inefficient with regard to the computing resources at hand.
The load balancing strategy solves this problem by systematically splitting the crawl tasks into chunks and distributes them among the available crawlers.
The load balancing strategy solves this problem by systematically splitting the crawl tasks into chunks and distributing them among the available crawlers.
The following load balancing strategies will be investigated:
\begin{description}
@ -345,7 +363,7 @@ Load balancing allows scaling out, which can be more cost-effective.
This strategy distributes work evenly among crawlers by either naively assigning tasks to the crawlers rotationally or weighted according to their capabilities\todo{1 -- 2 sentences about naive rr?}.
To keep the distribution as even as possible, we keep track of the last crawler a task was assigned to and start with the next in line in the subsequent round of assignments.
For the sake of simplicity, only the bandwidth will be considered as capability but it can be extended by any shared property between the crawlers, \eg{} available memory or processing power.
For the sake of simplicity only the bandwidth will be considered as a capability but it can be extended by any shared property between the crawlers, \eg{} available memory or processing power.
For a given crawler \(c_i \in C\) let \(cap(c_i)\) be the capability of the crawler.
The total available capability is \(B = \sum\limits_{c \in C} cap(c)\).
With \(G\) being the greatest common divisor of all the crawler's capabilities, the weight \(W(c_i) = \frac{cap(c_i)}{G}\).
@ -388,25 +406,25 @@ func WeightedCrawlerList(crawlers ...Crawler) []string {
return crawlerIds
}
\end{minted}
\caption{Pseudocode for weighted round robin}\label{lst:wrr}
\caption{Pseudocode for weighted round-robin}\label{lst:wrr}
\end{listing}
This creates a list of crawlers where a crawler can occur more than once, depending on its capabilities.
To ensure better distribution, first every crawler is assigned one task, then, according to the capabilities, every crawler with a weight of 2 or more is assigned a task, repeating this process until all tasks are assigned.
To ensure better distribution, first, every crawler is assigned one task, then, according to the capabilities, every crawler with a weight of 2 or more is assigned a task, repeating this process until all tasks are assigned.
The set of crawlers \(\{a, b, c\}\) with the capabilities \(cap(a) = 3\), \(cap(b) = 2\), \(cap(c) = 1\) would produce \(<a, b, c, a, b, a>\), allocating two and three times the work to crawlers \(b\) and \(a\) respectively.
\subsubsection{IP-based Partitioning}\label{sec:ipPart}
The output of cryptographic hash functions is uniformly distributed.
Given the hash function \(H\), calculating the hash of an IP address and distributing the work with regard to \(H(\text{IP}) \mod \abs{C}\) creates about evenly sized buckets for each worker to handle.
Given the hash function \(H\), calculating the hash of an IP address and distributing the work with regard to \(H(\text{IP}) \mod \abs{C}\) creates almost evenly sized buckets for each worker to handle.
This gives us the mapping \(m(i) = H(i) \mod \abs{C}\) to sort peers into buckets.
Any hash function can be used but since it must be calculated often, a fast function should be used.
While the \ac{md5} hash function must be considered broken for cryptographic use~\cite{bib:stevensCollision}, it is faster to calculate than hash functions with longer output.\todo{md5 crypto broken, distribution not?}
For the use case at hand, only the uniform distribution property is required so \ac{md5} can be used without scarifying any kind of security.
This strategy can also be weighted using the crawlers capabilities by modifying the list of available workers so that a worker can appear multiple times according to its weight.
This strategy can also be weighted using the crawlers' capabilities by modifying the list of available workers so that a worker can appear multiple times according to its weight.
The weighting algorithm from \Fref{lst:wrr} is used to create the weighted multiset of crawlers \(C_W\) and the mapping changes to \(m(i) = H(i) \mod \abs{C_W}\).
% \begin{figure}[H]
@ -420,8 +438,8 @@ The weighting algorithm from \Fref{lst:wrr} is used to create the weighted multi
The Go standard library includes helpers for arbitrarily sized integers\footnote{\url{https://pkg.go.dev/math/big\#Int}}.
This helps us in implementing the mapping \(m\) from above.
By exploiting the even distribution offered by hashing, the work of each crawler is also about evenly distributed over all IP subnets, \acp{as} and geolocations.
This ensures neighboring peers (\eg{} in the same \ac{as}, geolocation or IP subnet) get visited by different crawlers.
By exploiting the even distribution offered by hashing, the work of each crawler is also about evenly distributed over all IP subnets, \acp{as}, and geolocations.
This ensures neighboring peers (\eg{} in the same \ac{as}, geolocation, or IP subnet) get visited by different crawlers.
It also allows us to get rid of the state in our strategy since we don't have to keep track of the last crawler we assigned a task to, making it easier to implement and reason about.
%}}} load balancing
@ -429,8 +447,8 @@ It also allows us to get rid of the state in our strategy since we don't have to
%{{{ frequency reduction
\subsection{Reduction of Request Frequency}
The GameOver Zeus botnet limited the amount of requests a peer was allowed to perform, and blacklisted peers, that exceeded the limit, as an anti-monitoring mechanism~\cite{bib:andriesse_goz_2013}.
In a uncoordinated crawler approach, the crawl frequency has to be limited to prevent hitting the request limit.
The GameOver Zeus botnet limited the number of requests a peer was allowed to perform and blacklisted peers, that exceeded the limit, as an anti-monitoring mechanism~\cite{bib:andriesse_goz_2013}.
In an uncoordinated crawler approach, the crawl frequency has to be limited to prevent hitting the request limit.
%{{{ fig:old_crawler_timeline
\begin{figure}[h]
@ -447,7 +465,7 @@ In a uncoordinated crawler approach, the crawl frequency has to be limited to pr
Using collaborative crawlers, an arbitrarily fast frequency can be achieved without being blacklisted.
With \(l \in \mathbb{N}\) being the maximum allowed frequency as defined by the botnet's protocol, \(f \in \mathbb{N}\) being the crawl frequency that should be achieved.
The amount of crawlers \(n\) required to achieve the frequency \(f\) without being blacklisted and the offset \(o\) between crawlers are defined as
The number of crawlers \(n\) required to achieve the frequency \(f\) without being blacklisted and the offset \(o\) between crawlers are defined as
\begin{align*}
n &= \left\lceil \frac{f}{l} \right\rceil \\
@ -551,7 +569,7 @@ With \(v \in V\), \(\text{succ}(v)\) being the set of successors of \(v\) and \(
For the first iteration, the PageRank of all nodes is set to the same initial value. \citeauthor{bib:page_pagerank_1998} argue that when iterating often enough, any value can be chosen~\cite{bib:page_pagerank_1998}.
The dampingFactor describes the probability of a person visiting links on the web to continue doing so, when using PageRank to rank websites in search results.
For simplicity---and since it is not required to model human behaviour for automated crawling and ranking---a dampingFactor of \(1.0\) will be used, which simplifies the formula to
For simplicity---and since it is not required to model human behavior for automated crawling and ranking---a dampingFactor of \(1.0\) will be used, which simplifies the formula to
\[
\text{PR}_{n+1}(v) = \sum\limits_{p \in \text{pred}(v)} \frac{\text{PR}_n(p)}{\abs{\text{succ}(p)}}
@ -568,11 +586,11 @@ Since crawlers never respond to peer list requests, they will always be detectab
By responding to peer list requests with plausible data and thereby producing valid outgoing edges from the sensors, we will try to make those metrics less suspicious.
The challenge here is deciding which peers can be returned without actually supporting the network.
The following candidates to place into the neighbor list will be investigated:
The following candidates to place on the neighbor list will be investigated:
\begin{itemize}
\item Return the other known sensors, effectively building an complete graph \(K_{\abs{C}}\) containing all sensors
\item Return the other known sensors, effectively building a complete graph \(K_{\abs{C}}\) containing all sensors
\item Detect churned peers from \ac{as} with dynamic IP allocation
@ -587,12 +605,12 @@ The following candidates to place into the neighbor list will be investigated:
Returning all the other sensors when responding to peer list requests, thereby effectively creating a complete graph \(K_\abs{C}\) among the workers, creates valid outgoing edges.
The resulting graph will still form a \ac{wcc} with now edges back into the main network.
Also this would leak the information about all known sensors to the botmasters.
Also, this would leak the information about all known sensors to the botmasters.
%{{{ churned peers
\subsubsection{Churned Peers After IP Rotation}
Churn describes the dynamics of peer participation of \ac{p2p} systems, \eg{} join and leave events~\cite{bib:stutzbach_churn_2006}.\todo{übergang}
Churn describes the dynamics of peer participation in \ac{p2p} systems, \eg{} join and leave events~\cite{bib:stutzbach_churn_2006}.\todo{übergang}
Detecting if a peer just left the system, in combination with knowledge about \acp{as}, peers that just left and came from an \ac{as} with dynamic IP allocation (\eg{} many consumer broadband providers in the US and Europe), can be placed into the crawler's peer list.\todo{what is an AS}
If the timing of the churn event correlates with IP rotation in the \ac{as}, it can be assumed, that the peer left due to being assigned a new IP address---not due to connectivity issues or going offline---and will not return using the same IP address.
These peers, when placed in the peer list of the crawlers, will introduce paths back into the main network and defeat the \ac{wcc} metric.
@ -603,8 +621,8 @@ It also helps with the PageRank and SensorRank metrics since the crawlers start
%{{{ cg nat
\subsubsection{Peers Behind Carrier-Grade \acs*{nat}}
Some peers show behaviour, where their IP address changes almost after every request.
Those peers can be used as fake neighbors and create valid looking outgoing edges for the sensor.
Some peers show behavior, where their IP address changes almost after every request.
Those peers can be used as fake neighbors and create valid-looking outgoing edges for the sensor.
%}}} cg nat
@ -616,12 +634,12 @@ Those peers can be used as fake neighbors and create valid looking outgoing edge
\clearpage{}
\section{Evaluation}
To evaluate the strategies from above, we took a snapshot of the Sality~\cite{bib:falliere_sality_2011} botnet obtained from \ac{bms} over the span of \daterange{2021-04-21}{2021-04-28}.
To evaluate the strategies from above, we took a snapshot of the Sality~\cite{bib:falliere_sality_2011} botnet obtained from \ac{bms} throughout of \daterange{2021-04-21}{2021-04-28}.
%{{{ eval load balancing
\subsection{Load Balancing}
To evaluate the real-world applicability of IP based partitioning, we will partition the dataset containing \num{1595} distinct IP addresses among \num{2}, \num{4}, \num{6} and \num{10} crawlers and verify if the work is about evenly distributed between crawlers.
To evaluate the real-world applicability of IP based partitioning, we will partition the dataset containing \num{1595} distinct IP addresses among \num{2}, \num{4}, \num{6}, and \num{10} crawlers and verify if the work is about evenly distributed between crawlers.
We will compare the variance \(\sigma^2\) and standard derivation \(\sigma\) to evaluate the applicability of this method.
@ -630,30 +648,27 @@ We will compare the variance \(\sigma^2\) and standard derivation \(\sigma\) to
\centering
\includegraphics[width=.7\linewidth]{ip_part_c02.png}
\caption{IP based partitioning for 2 crawlers}\label{fig:ipPartC02}
\end{figure}
%}}}fig:ipPartC02
\begin{align*}
n &= 2 \\
\mu &= 1595 / n = 797.5 \\
\mu &= \frac{1595}{n} = 797.5 \\
s &= \sum\limits_{i=1}^{n} {(x_i - \mu)}^2 \\
&= {(808 - 797.5)}^2 + {(787 - 797.5)}^2 \\
&= 220.5 \\
\sigma^2 &= \frac{s}{n} = 110.2 \\
\sigma &= \sqrt{\sigma^2} = 10.5
\end{align*}
\end{figure}
%}}}fig:ipPartC02
%{{{ fig:ipPartC04
\begin{figure}[H]
\centering
\includegraphics[width=.7\linewidth]{ip_part_c04.png}
\caption{IP based partitioning for 4 crawlers}\label{fig:ipPartC04}
\end{figure}
%}}}fig:ipPartC04
\begin{align*}
n &= 4 \\
\mu &= 1595 / n = 398.8 \\
\mu &= \frac{1595}{n} = 398.8 \\
s &= \sum\limits_{i=1}^{n} {(x_i - \mu)}^2 \\
&= {(403 - 398.8)}^2 + {(369 - 398.8)}^2 + {(405 - 398.8)}^2 \\
&+ {(418 - 398.8)}^2 \\
@ -661,18 +676,18 @@ We will compare the variance \(\sigma^2\) and standard derivation \(\sigma\) to
\sigma^2 &= \frac{s}{n} = 328.2 \\
\sigma &= \sqrt{\sigma^2} = 18.1
\end{align*}
\end{figure}
%}}}fig:ipPartC04
%{{{ fig:ipPartC06
\begin{figure}[H]
\centering
\includegraphics[width=.7\linewidth]{ip_part_c06.png}
\caption{IP based partitioning for 6 crawlers}\label{fig:ipPartC06}
\end{figure}
%}}}fig:ipPartC06
\begin{align*}
n &= 6 \\
\mu &= 1595 / n = 265.8 \\
\mu &= \frac{1595}{n} = 265.8 \\
s &= \sum\limits_{i=1}^{n} {(x_i - \mu)}^2 \\
&= {(258 - 265.8)}^2 + {(273 - 265.8)}^2 + {(257 - 265.8)}^2 \\
&+ {(264 - 265.8)}^2 + {(293 - 265.8)}^2 + {(250 - 265.8)}^2 \\
@ -681,17 +696,17 @@ We will compare the variance \(\sigma^2\) and standard derivation \(\sigma\) to
\sigma &= \sqrt{\sigma^2} = 14.0
\end{align*}
\end{figure}
%}}}fig:ipPartC06
%{{{ fig:ipPartC10
\begin{figure}[H]
\centering
\includegraphics[width=.7\linewidth]{ip_part_c10.png}
\caption{IP based partitioning for 10 crawlers}\label{fig:ipPartC10}
\end{figure}
%}}}fig:ipPartC10
\begin{align*}
n &= 10 \\
\mu &= 1595 / n = 159.5 \\
\mu &= \frac{1595}{n} = 159.5 \\
s &= \sum\limits_{i=1}^{n} {(x_i - \mu)}^2 \\
&= {(140 - 159.5)}^2 + {(175 - 159.5)}^2 + {(186 - 159.5)}^2 \\
&+ {(166 - 159.5)}^2 + {(159 - 159.5)}^2 + {(152 - 159.5)}^2 \\
@ -701,6 +716,8 @@ We will compare the variance \(\sigma^2\) and standard derivation \(\sigma\) to
\sigma^2 &= \frac{s}{n} = 196.4 \\
\sigma &= \sqrt{\sigma^2} = 14.0
\end{align*}
\end{figure}
%}}}fig:ipPartC10
\begin{table}[H]
\centering
@ -716,25 +733,25 @@ We will compare the variance \(\sigma^2\) and standard derivation \(\sigma\) to
\Fref{tab:varSmall} shows that the derivation from the expected even distribution is within \SI{10}{\percent}.
Since the used sample is not very big, according to the law of big numbers we would expect the derivation to get smaller, the bigger the sample gets.
Therefore, we simulated the partitioning on a bigger sample of \num{1000000} random IP addresses.
Therefore, we simulate the partitioning on a bigger sample of \num{1000000} random IP addresses.
%{{{ fig:randIpPartC02
\begin{figure}[H]
\centering
\includegraphics[width=.8\linewidth]{rand_ip_part_c02.png}
\caption{IP based partitioning for 2 crawlers on generated dataset}\label{fig:randIpPartC02}
\begin{align*}
n &= 2 \\
\mu &= \frac{1000000}{n} = 500000 \\
s &= \sum\limits_{i=1}^{n} {(x_i - \mu)}^2 \\
&= {(499322 - 500000)}^2 + {(500678 - 500000)}^2 \\
&= 919368 \\
\sigma^2 &= \frac{s}{n} = 459684 \\
\sigma &= \sqrt{\sigma^2} = 678
\end{align*}
\end{figure}
%}}}fig:randIpPartC02
\begin{align*}
n &= 2 \\
\mu &= \frac{1000000}{n} = 500000.0 \\
s &= \sum\limits_{i=1}^{n} {(x_i - \mu)}^2 \\
&= {(499322 - 500000.0)}^2 + {(500678 - 500000.0)}^2 \\
&= 919368.0 \\
\sigma^2 &= \frac{s}{n} = 459684.0 \\
\sigma &= \sqrt{\sigma^2} = 678.0
\end{align*}
%{{{ fig:randIpPartC04
@ -742,28 +759,25 @@ Therefore, we simulated the partitioning on a bigger sample of \num{1000000} ran
\centering
\includegraphics[width=.8\linewidth]{rand_ip_part_c04.png}
\caption{IP based partitioning for 4 crawlers on generated dataset}\label{fig:randIpPartC04}
\end{figure}
%}}}fig:randIpPartC04
\begin{align*}
n &= 4 \\
\mu &= \frac{1000000}{n} = 250000.0 \\
\mu &= \frac{1000000}{n} = 250000 \\
s &= \sum\limits_{i=1}^{n} {(x_i - \mu)}^2 \\
&= {(249504 - 250000.0)}^2 + {(250451 - 250000.0)}^2 + {(249818 - 250000.0)}^2 \\
&+ {(250227 - 250000.0)}^2 \\
&= 534070.0 \\
&= {(249504 - 250000)}^2 + {(250451 - 250000)}^2 + {(249818 - 250000)}^2 \\
&+ {(250227 - 250000)}^2 \\
&= 534070 \\
\sigma^2 &= \frac{s}{n} = 133517.5 \\
\sigma &= \sqrt{\sigma^2} = 365.4
\end{align*}
\end{figure}
%}}}fig:randIpPartC04
%{{{ fig:randIpPartC06
\begin{figure}[H]
\centering
\includegraphics[width=.8\linewidth]{rand_ip_part_c06.png}
\caption{IP based partitioning for 6 crawlers on generated dataset}\label{fig:randIpPartC06}
\end{figure}
%}}}fig:randIpPartC06
\begin{align*}
n &= 6 \\
\mu &= \frac{1000000}{n} = 166666.7 \\
@ -774,27 +788,30 @@ Therefore, we simulated the partitioning on a bigger sample of \num{1000000} ran
\sigma^2 &= \frac{s}{n} = 62068.9 \\
\sigma &= \sqrt{\sigma^2} = 249.1
\end{align*}
\end{figure}
%}}}fig:randIpPartC06
%{{{ fig:randIpPartC10
\begin{figure}[H]
\centering
\includegraphics[width=1\linewidth]{rand_ip_part_c10.png}
\caption{IP based partitioning for 10 crawlers on generated dataset}\label{fig:randIpPartC10}
\end{figure}
%}}}fig:randIpPartC10
\begin{align*}
n &= 10 \\
\mu &= \frac{1000000}{n} = 100000.0 \\
\mu &= \frac{1000000}{n} = 100000 \\
s &= \sum\limits_{i=1}^{n} {(x_i - \mu)}^2 \\
&= {(100424 - 100000.0)}^2 + {(99650 - 100000.0)}^2 + {(99307 - 100000.0)}^2 \\
&+ {(100305 - 100000.0)}^2 + {(99403 - 100000.0)}^2 + {(100562 - 100000.0)}^2 \\
&+ {(100277 - 100000.0)}^2 + {(99875 - 100000.0)}^2 + {(99911 - 100000.0)}^2 \\
&+ {(100286 - 100000.0)}^2 \\
&= 1729874.0 \\
&= {(100424 - 100000)}^2 + {(99650 - 100000)}^2 + {(99307 - 100000)}^2 \\
&+ {(100305 - 100000)}^2 + {(99403 - 100000)}^2 + {(100562 - 100000)}^2 \\
&+ {(100277 - 100000)}^2 + {(99875 - 100000)}^2 + {(99911 - 100000)}^2 \\
&+ {(100286 - 100000)}^2 \\
&= 1729874 \\
\sigma^2 &= \frac{s}{n} = 172987.4 \\
\sigma &= \sqrt{\sigma^2} = 415.9
\end{align*}
\end{figure}
%}}}fig:randIpPartC10
\begin{table}[H]
\centering
@ -840,7 +857,7 @@ Also this does not help against the \ac{wcc} metric since this would create a bi
\caption{Differences in graph metrics}\label{fig:sensorbuster}
\end{figure}
Applying PageRank once with an initial rank of \(0.25\) once on the example graphs in \Fref{fig:sensorbuster} results in:
Applying PageRank with an initial rank of \(0.25\) once on the example graphs in \Fref{fig:sensorbuster} results in:
\begin{table}[H]
\centering
@ -1012,7 +1029,7 @@ Experiments were performed, in which a percentage of random outgoing edges were
% \caption{SensorRank distribution with initial rank \(\forall v \in V : \text{PR}(v) = 0.75\)}\label{fig:dist_sr_75}
\end{figure}
These results showed, that simply adding new edges is not enough and we need to limit the incoming edges to improve the Page- and SensorRank metrics.
These results show, that simply adding new edges is not enough and we need to limit the incoming edges to improve the Page- and SensorRank metrics.
For the \ac{wcc} metric, it is obvious that even a single edge back into the main network is enough to connect the sensor back to the main graph and therefore beat this metric.
%}}} eval creating edges
@ -1024,12 +1041,12 @@ For the \ac{wcc} metric, it is obvious that even a single edge back into the mai
\section{Implementation}
Crawlers in \ac{bms} report to the backend using \acp{grpc}\footnote{\url{https://www.grpc.io}}.
Both crawlers and the backend \ac{grpc} server are implemented using the Go\footnote{\url{https://go.dev/}} programming language, so to make use of existing know-how and to allow others to use the implementation in the future, the coordinator backend and crawler abstraction were also implemented in Go.
Both crawlers and the backend \ac{grpc} server are implemented using the Go\footnote{\url{https://go.dev/}} programming language, so to make use of existing know-how and to allow others to use the implementation in the future, the coordinator backend, and crawler abstraction were also implemented in Go.
\Ac{bms} already has an existing abstraction for crawlers.
This implementation is highly optimized but also tightly coupled and grown over time.
The abstraction became leaky and extending it proved to be complicated.
A new crawler abstraction was created with testability, extensibility and most features of the existing implementation in mind, which can be ported back to be used by the existing crawlers.
A new crawler abstraction was created with testability, extensibility, and most features of the existing implementation in mind, which can be ported back to be used by the existing crawlers.
%{{{ fig:crawler_arch
\begin{figure}[h]
@ -1049,12 +1066,12 @@ The new implementation consists of three main interfaces:
\item \mintinline{go}{Protocol}, the actual botnet protocol implementation used to ping a peer and request its peer list
\end{itemize}
Currently there are two sources \mintinline{go}{FindPeer} can use: read peers from a file on disk or request them from the \ac{grpc} BMS coordinator.
Currently, there are two sources \mintinline{go}{FindPeer} can use: read peers from a file on disk or request them from the \ac{grpc} BMS coordinator.
The \mintinline{go}{ExactlyOnceFinder} delegate can wrap another \mintinline{go}{FindPeer} instance and ensures the source is only requested once.
This is used to implement the bootstrapping mechanism of the old crawler, where once, when the crawler is started, the list of bootstrap nodes is loaded from a textfile.
This is used to implement the bootstrapping mechanism of the old crawler, where once when the crawler is started, the list of bootstrap nodes is loaded from a text file.
\mintinline{go}{CombinedFinder} can combine any amount of \mintinline{go}{FindPeer} instances and will return the sum of requesting all the sources.
The \mintinline{go}{PeerTask} instances returned by \mintinline{go}{FindPeer} contain the IP address and port of the peer, if the crawler should start or stop the operation, when to start and stop crawling and in which interval the peer should be crawled.
The \mintinline{go}{PeerTask} instances returned by \mintinline{go}{FindPeer} contain the IP address and port of the peer, if the crawler should start or stop the operation, when to start and stop crawling, and in which interval the peer should be crawled.
For each task, a \mintinline{go}{CrawlPeer} and \mintinline{go}{PingPeer} worker is started or stopped as specified in the received \mintinline{go}{PeerTask}.
These tasks use the \mintinline{go}{ReportPeer} interface to report any new peer that is found.
@ -1065,7 +1082,7 @@ Current report possibilities are \mintinline{go}{LoggingReport} to simply log ne
\mintinline{go}{PingPeer} and \mintinline{go}{CrawlPeer} use the implementation of the botnet \mintinline{go}{Protocol} to perform the actual crawling in predefined intervals, which can be overwritten on a per \mintinline{go}{PeerTask} basis.
The server-side part of the system consists of a \ac{grpc} server to handle the client requests, a scheduler to assign new peers, and a \mintinline{go}{Strategy} interface for modularity over how work is assigned to crawlers.
The server-side part of the system consists of a \ac{grpc} server to handle the client requests, a scheduler to assign new peers, and a \mintinline{go}{Strategy} interface for modularity over how tasks are assigned to crawlers.
%{{{ fig:bachend_arch
\begin{figure}[h]
@ -1079,7 +1096,7 @@ The server-side part of the system consists of a \ac{grpc} server to handle the
%{{{ conclusion
\clearpage{}
\section{Conclusion, Lessons Learned}\todo{decide}
\section{Conclusion}
Collaborative monitoring of \ac{p2p} botnets allows circumventing some anti-monitoring efforts.
It also enables more effective monitoring systems for larger botnets, since each peer can be visited by only one crawler.
@ -1099,11 +1116,11 @@ Another way to expand on this work is automatically scaling the available crawle
Doing so would allow a constant crawl interval for even highly volatile botnets.
Placing churned peers or peers with suspicious network activity (those behind carrier-grade \acp{nat}) might just offer another characteristic to flag sensors in a botnet.
This should be investigated and maybe there are ways to mitigate this problem.
The feasibility of this approach should be investigated and maybe there are ways to mitigate this problem.
Autoscaling features offered by many cloud-computing providers should be evaluated to automatically add or remove crawlers based on the monitoring load, a botnet's size and number of active peers.
This should also allow create workers with new IP addresses in different geolocations fast and easy.
The current implementation assumes a immutable set of crawlers.
Autoscaling features offered by many cloud-computing providers can be evaluated to automatically add or remove crawlers based on the monitoring load, a botnet's size, and the number of active peers.
This should also allow the creation of workers with new IP addresses in different geolocations in a fast, easy and automated way.
The current implementation assumes an immutable set of crawlers.
For autoscaling to work, efficient reassignment of peers has to be implemented to account for added or removed workers.
%}}} further work

BIN
references/2012-war_agains_botnets.pdf Normal file
View File

Binary file not shown.

BIN
report.pdf
View File

Binary file not shown.