Content

assets/backend_architecture.drawio

@@ -0,0 +1 @@
+<mxfile host="app.diagrams.net" modified="2022-03-31T12:27:11.044Z" agent="5.0 (X11)" etag="7-oxMMRev6E7jJLz77DP" version="17.3.0" type="device"><diagram id="lSPjqpDcrzCjySd24YSd" name="Page-1">7VpZc6M4EP41VM08uIrTx2NsT3ayu6lKOTubrXmTTdtmBhAj5Gt+/UogcQhIiMsHqeTFhkZqqbu//hoJadYk2P9BULS+xy74mqm7e82aaqY57A/ZLxccUoE9tFPBinhuKjJywaP3G4RQF9KN50Jcakgx9qkXlYULHIawoCUZIgTvys2W2C+PGqEVVASPC+RXpU+eS9fCLEfP5V/BW63lyIYungRINhaCeI1cvCuIrC+aNSEY0/Qq2E/A576Tfkn73TY8zSZGIKRtOmz/+nH3X/Tn5Pds9+/f373enf7d6Dmpli3yN8Jgzez7TN844lOmB+GH/q8Nn+c4QGTlhZp1w57q0Z79MmFiLZf3KI7SZ3bhGYU97SHfW4l+CzZhILlOdrUS/8nIXkGAAqZw7Ffv7riOJVpAJi53KWlkTvHUUZgsUmXXs3kuBY+UIAqrQ2Ga80pj0saYNeHWyISSAxvNczjKeh+WtGB+VTdvexPHzBMPACT+NMc0BHrncmSG9DP7+xSgSHPGMSVeuNKcKbtmv7y1Zk7Yc2BpTD63Nd4sWWFugVCPpfNNGotpEq2xiMw0nf0Ys1ZLP0nOpceSxhovcUgFGRmmuL9FgedzGvsK/ha4Vu5lGvi8UTY2HxD2jTlqZJnPGBNwAJSwWOuigzkSCSnY0hgK8tjl3GOOhGxd4B3ZDgm6W2Wqc0ZgF4IUXkEQ/UaCyGA5w5vQneE5Q0ktaJWIFDzmEhz9w3AE3Ar9NC6UnpD1purAzKlFB47O5cBBjQMVj7DKEPHLALsbLhv/AEoPstwMpSCrM3ZbTzVHtNF9RhVvNd6SMgI+ot62XCvrXCjGe8As5/PBeuagFKxBWQFeLmMGDTUC2aSPD8rwZVTfo/0tgV8bCBcNZHxZXFtOx4A96hqwhx/A7snX5ueQ/ZTYC+49jwruArb7ZhnbxuDa4DaMrqHb0J914TuBt9mCuDsL66tTtmF1DtXmB6p7ht2etDv9st0B3q7b0Lguwu0PhPeMumWkEhYI3Ru+Ycfu5j5e/OSQRfEaXIFf9vw2WZwn4GV3xfV5AeeEZ0jSKwc5uJVdPgXibCp4QxbwjBHCBipzqKmdUx/wYk70TxNPWykwpqPkVWqT6JWHtaLIGb6gKDW6ouhk+GixSu48PiSNdwgg5kAhaPtIgFiqIpXBzw2QuhX7mwOI3j2AqBX8WAaxVAbRLwyQFtsRnQfIsHv4GCn46B+LD0VR9oHvQviQlfJN40MufzsEEEtvWAK8FiC2oih7570UQOp2fNIVluttX/xe94TJz+QD3+u+2pVUX3btpm4oWzIjr7Z4kynZncXbCy58H4s3s7prNK2iVQbGRRTFFBMem93ao/AYoYTUdgRFJ/9gahtlENtOFcT9mpioZex0GK7u5XyL4blSgyMIq+WkVHguUVuy0zXdqS3q8tY6traoy9uKonPXluZjNoUC8ERYtsQti8MbBlDKJ+8DQSxG6FBoFvEG8Vkw1uKkxmr2MHkEsm16UbnuC4jTv/oLSOcOa5gfpzWYE6qbPzNAbvwWSLHtpvEVSdFW8+lYUqwoOiMpstv86GzaPD9/bH35Hw==</diagram></mxfile>

assets/sensorbuster1.drawio

@@ -0,0 +1 @@
+<mxfile host="app.diagrams.net" modified="2022-03-31T13:37:03.283Z" agent="5.0 (X11)" etag="9UKWEBxLBe_wRfpIkXR4" version="17.3.0" type="device"><diagram id="d8KPRqQcHJQ_maWGryHp" name="Page-1">1ZpNc5swEIZ/jY/x6AshHZu0aS+dyTQzba8qqDYzGDlCjp38+opB2EayCUmxsS8ZaxErePfRaqUwwXeLzVctlvPvKpX5BIF0M8GfJwgxyuzfyvBSGwgjtWGms7Q2wZ3hMXuVzgicdZWlsmx1NErlJlu2jYkqCpmYlk1ordbtbn9V3h51KWYyMDwmIg+tv7LUzN1rRWBn/yaz2bwZGQJ3ZSGazs5QzkWq1nsm/GWC77RSpv612NzJvNKu0aW+7/7I1e2DaVmYPjeskvviM/x9/+f5KTZPP8FmZh5voHPzLPKVe+PEmUrz0ohgHVm9beN2Pc+MfFyKpLqythG3trlZ5LYF7U9RLusg/M020o5767xLbeTm6HPDrRqWIqkW0ugX26W5oRHQEYSIa6/34uFM871QNDbhCJhtPe9Esj+cTu/RLA40u9NinUtdBsqV62yRi6KSbq509qoKIyqxwDDakLY0W/j2pEHggDY4PpU4NOQJBKqcmKeOqB1VknmQjc0YCHVE16gjHVtIHgj5Q85WudDW+CDfO2e1MsJkqrDNG8gGmsW+aBcwjUPViguZxrw7IUaXNY1RoGOoYpF+qgoV20pyUZZZ0hZPq1WRVqrtLRoyDaqWXrrZgdVKJ7Kjo1vcjNAz2eXwSBz2lI4OKN3YtMztPHpuv8Qh+d0IDyqzr7cN8w2k7UBDP4T1e7rb9quiwBP0PGHPU61E4MnGTLzsdVtWHcrjjwxjcPCJd3jVHnewbVX9D/7w1fHHe/J3ZBk6E38+fvzD+HmOyIno8yvFs9BHro4+dhXZL/JXuY/S58EXnQY+7g2DzsFedHXs9c18bFT2/AoLfpA9jwl6GvTiEdBjAXnF2Y9UBqid4djnLOExS3EhW+D37UFG3wKHZzKXngr7LsOjFoGQwynHkDDCAIwog62wczSNAOEUxtgiQKHnvm+aZFMQx4ACwFDMIAyyb2xHiSPOIcExYAz/ZxZtgTZALmw24Gdlr8m2b1FGe1I2arFHQLtyD/JEX5JI5DnqudEdDIXwXw3Xh8KoCQdxbxPnH//1rr3wG45OjQK6YBSagvay0wJibIqrbXyEEbVeI9qOKINTQgnlke1qx0Txx0ix69aUMsoJiziEEebtFBIzu/xQwEgUx5hQdF6KxjjcGpyikTNKN0V8IIqwv4bx85IyxkFUb1KazxouOt9g0EkKAkOR0plv+Kj5ZowjpcEpGvX0CMNuiuBAFJEuiiqUT0GRbe6+5qm77z6Jwl/+AQ==</diagram></mxfile>

assets/sensorbuster2.drawio

@@ -0,0 +1 @@
+<mxfile host="app.diagrams.net" modified="2022-03-31T13:34:40.111Z" agent="5.0 (X11)" etag="a6XsfpmlWGMFJNmlumPt" version="17.3.0" type="device"><diagram id="d8KPRqQcHJQ_maWGryHp" name="Page-1">5ZtNk5s4EIZ/jY9D6fvjmEwyyWWrUjtVm1wJKDZVGBzAY09+/YoCbCPZRONghHcvLtOIFrz9uCW18AI/rvefinCz+iuPVbpAIN4v8IcFQoIJ/VkbXhsDEaQxLIskbkzwaHhOfqnWCFrrNolV2WtY5XlaJZu+McqzTEVVzxYWRb7rN/uRp/1eN+FSWYbnKExt69ckrlbtY1FwtH9WyXLV9QxBe2Yddo1bQ7kK43x3YsIfF/ixyPOq+bbeP6q01q7Tpbnu6cLZw40VKqtcLthGT9kH+O3p+8tPXv38B+yX1fMDbN28hOm2feKoNZXVayeCdqT11gfvd6ukUs+bMKrP7HTEtW1VrVN9BPXXsNw0QfiR7JXu933rXRWV2l+8b3hQQ1Ok8rWqilfdpLugE7AlCJH2eHcSj9a0OglFZwtbApYHz0eR9JdWp7doxi3NHotwl6qitJQrd8k6DbNaulVeJL/yrAprscA42pC+NAf4TqRB4Iw2mN9KHGbzBCxVbszTQNQuKikMyHwzBmwd0T3qyDwLie0EZ8uYxe/qoUIfRWlYlknUV6/It1lcy3bys1WxNW44Cac7zrdFpBxiX4XFUg15ZOcjcSI1PSN1ZytUGlbJS/8pzunf9vAlT/TzHQKNmZF7zBA2z9ledToumY4MP9Tw08hg+dERC19Pmm3qBuXl+71wu0e2GodH0g6K/gF86O7gY47sQbD4L8BHoOEI34Y+E/Ip6IPSou9vtdymYaGNX9RbpytFXumQ5Zk+fKjvf5QJjDlezGAGY6uWzWQGIwelJLQ/8vqewcwn90nH3Mcdc9+FOEyT+h4g6wf66tynF3+GpxslP8h9ZD98d/xJR/4uzMAn4s/ET16Nn+GI3Ig+c5E8CX3k7ugTd5H9qDnKXUufAd+NVh3S6AZNwR69O/ZcM5/wyp45w4JXsmcwwW6DHveAnrDIyyavJo8wd4a+S8x2hTk78xuevY7eq3/QLkfPPRW6DsNeJ4FQwkBiSAQRAFImYC/sEgUUEMkgxxoBBg33rmlSBIBzwAAQiAsIrezLdS+cSgkJ5kAI/IdZtAfaCLmwW4BPyl6Xbccq83md7BHQn7lbecK5ykcNR44L3dFQ8LEJMTYKXhMOksYiziz/Oc+9zEqs6ejWKKAZo+C88+Q1LSAhAlwv4ylGTHulrB9RAQPCCJNUN9V9In4dKXrcCphgkggqIaRY9lMIF3r4YUAQyjkmDE1LkY/i1ugUec4owxTJkSjC5hgmpyXFRyHKmZTuja5Z5xsMBklBYCxSBvON9JpvfJSURqfIa/UIw2GK4EgUkSGKapQ9UuRjNT76qAUv7Gj/T9ZEb95+MfrBZII6JLILaDMizTVf+X3Hxty1td64cCXNcjTRNvM0pNkl7zskzeubhJCzgHIqqSSyrr13CnZhRDIgVDfQYyNGgmJyLYbD3XAQUEIQwYAwDgljk6RDNMW2DLLfaZoPpM5vHHodd6EYpgePBOlvuhHTQMpHhVQfHv/m0jQ//lcIf/wX</diagram></mxfile>

content.tex

@@ -266,6 +266,45 @@ The weight \(W(c_i) = \frac{B}{B(c_i)}\)\todo{proper def for weight} defines whi
 The set of target peers \(P = <p_0, p_1, \ldots, p_{n-1}>\), is partitioned into \(|C|\) subsets according to \(W(c_i)\) and each subset is assigned to its crawler \(c_i\).
 The mapping \mintinline{go}{gcd(C)} is the greatest common divisor of all peers in \mintinline{go}{C}, \(\text{maxWeight}(C) = \max \{ \forall c \in C : W(c) \}\).
 
+The following algorithm distributes the work according to the crawler's capabilities:
+
+\begin{minted}{go}
+func WeightCrawlers(crawlers ...Crawler) map[string]uint {
+  weights := []int{}
+  totalWeight := 0
+  for _, crawler := range crawlers {
+    totalWeight += crawler.Bandwith
+    weights = append(weights, crawler.Bandwith)
+  }
+  gcd := Fold(Gcd, weights...)
+  weightMap := map[string]uint{}
+  for _, crawler := range crawlers {
+    weightMap[crawler.ID] = uint(crawler.Bandwith / gcd)
+  }
+  return weightMap
+}
+
+func WeightedCrawlerList(crawlers ...Crawler) []string {
+  weightMap := WeightCrawlers(crawlers...)
+  didSomething := true
+  crawlerIds := []string{}
+  for didSomething {
+    didSomething = false
+    for k, v := range weightMap {
+      if v != 0 {
+        didSomething = true
+        crawlerIds = append(crawlerIds, k)
+        weightMap[k] -= 1
+      }
+    }
+  }
+  return crawlerIds
+}
+\end{minted}{go}
+
+This creates a list of crawlers where a crawler can occur more than once, depending on its capabilities.
+The set of crawlers \(\{a, b, c\}\) with capabilities \(cap(a) = 3, cap(b) = 2, cap(c) = 1\) would produce \(<a, b, c, a, b, a>\), allocating two and three times the work to crawlers \(b\) and \(a\) respectively.
+
 The following weighted round-robin algorithm distributes the work according to the crawlers' capabilities:
 
 \begin{minted}{go}
@@ -318,6 +357,8 @@ Any hash function can be used but since it must be calculated often, a fast func
 While the \ac{md5} hash function must be considered broken for cryptographic use, it is faster to calculate than hash functions with longer output.
 For the use case at hand, only the uniform distribution property is required so \ac{md5} can be used without scarifying any kind of security.
 
+This strategy can also be weighted using the crawlers capabilities by modifying the list of available workers so that a worker can appear multiple times according to its weight.
+
 \begin{figure}[H]
   \centering
   \includegraphics[width=1\linewidth]{./md5_ip_dist.png}
@@ -436,7 +477,7 @@ While the effective frequency of the whole system is halved compared to~\autoref
 %}}} frequency reduction
 
 %{{{ against graph metrics
-\subsection{Preventing Suspicious Graph Metrics}
+\subsection{Creating Outgoing Edges for Crawlers and Sensors}
 
 \citetitle*{bib:karuppayah_sensorbuster_2017} describes different graph metrics to find sensors in \ac{p2p} botnets.
 These metrics depend on the uneven ratio between incoming and outgoing edges for crawlers.
@@ -445,7 +486,7 @@ One of those, \enquote{SensorBuster} uses \acp{wcc} since crawlers don't have an
 Building a complete graph \(G_C = K_{\abs{C}}\) between the crawlers by making them return the other crawlers on peer list requests would still produce a disconnected component and while being bigger and maybe not as obvious at first glance, it is still easily detectable since there is no path from \(G_C\) back to the main network (see~\autoref{fig:sensorbuster2} and~\autoref{fig:metrics_table}).
 
 \todo{rank? deg+ - deg-?}
-With \(v \in V\), \(\text{succ}(v)\) being the set of successors of \(v\) and \(\text{pred}(v)\) being the set of predecessors of \(v\), PageRank recursively is defined as~\cite{bib:page_pagerank_1998}:
+With \(v \in V\), \(\text{succ}(v)\) being the set of successors of \(v\) and \(\text{pred}(v)\) being the set of predecessors of \(v\), PageRank is recursively defined as~\cite{bib:page_pagerank_1998}:
 
 \[
   \text{PR}(v) = \text{dampingFactor} \times \sum\limits_{p \in \text{pred}(v)} \frac{\text{PR}(p)}{\abs{\text{succ}(p)}} + \frac{1 - \text{dampingFactor}}{\abs{V}}
@@ -468,7 +509,7 @@ Based on this, SensorRank is defined as
 
 \todo{percentage of botnet must be crawlers to make a significant change}
 
-Applying SensorRank PageRank once with an initial rank of \(0.25\) once on the example graphs above results in:
+Applying PageRank once with an initial rank of \(0.25\) once on the example graphs above results in:
 
 \todo{pagerank, sensorrank calculations, proper example graphs, proper table formatting}
 \begin{table}[H]
@@ -625,17 +666,16 @@ Also this does not help against the \ac{wcc} metric since this would create a bi
 \centering
 \begin{subfigure}[b]{.5\textwidth}
   \centering
-  \includegraphics[width=1\linewidth]{dot/sensorbuster1.pdf}
+  \includegraphics[width=1\linewidth]{sensorbuster1.drawio.pdf}
   \caption{\acp{wcc} for independent crawlers}\label{fig:sensorbuster1}
 \end{subfigure}%
 \begin{subfigure}[b]{.5\textwidth}
   \centering
-  \includegraphics[width=1\linewidth]{dot/sensorbuster2.pdf}
+  \includegraphics[width=1\linewidth]{sensorbuster2.drawio.pdf}
   \caption{\acp{wcc} for collaborated crawlers}\label{fig:sensorbuster2}
 \end{subfigure}%
   \caption{Differences in graph metrics}\label{fig:sensorbuster}
 \end{figure}
-\todo{these examples suck; chose better examples}
 
 
 %}}} other sensors
@@ -711,14 +751,25 @@ Current report possibilities are \mintinline{go}{LoggingReport} to simply log ne
 
 The server-side part of the system consists of a \ac{grpc} server to handle the client requests, a scheduler to assign new peers, and a \mintinline{go}{Strategy} interface for modularity over how work is assigned to crawlers.
 
+%{{{ fig:bachend_arch
+\begin{figure}[h]
+\centering
+\includegraphics[width=1\linewidth]{backend_architecture.drawio.pdf}
+\caption{Architecture of the \ac{grpc} backend}\label{fig:bachend_arch}
+\end{figure}
+%}}}fig:bachend_arch
+
 %}}} implementation
 
 %{{{ conclusion
 \section{Conclusion, Lessons Learned}\todo{decide}
 
+Collaborative monitoring of \ac{p2p} botnets allows circumventing some anti-monitoring efforts.
+It also enables more effective monitoring systems for larger botnets, since each peer can be visited by only one crawler.
+The current concept of independent crawlers in \ac{bms} can also use multiple workers but there is no way to ensure a peer is not watched by multiple crawlers thereby using unnecessary resources.
 
 
-%}}}
+%}}} conclusion
 
 %{{{ further work
 \section{Further Work}