[++doc] add benchmark design

doc/Design.tex

@@ -182,3 +182,29 @@ not the full headers are used, but the pseudo headers (compare figures
 To compensate the carry bit, our code uses 17 bit integers for
 correcting the carry.
 % FIXME: add note to python script / checksum diffing
+% ----------------------------------------------------------------------
+\section{\label{Design:Benchmarks}Benchmarks}
+The benchmarks were performed on two hosts, a load generator and a
+nat64 translator. Both hosts were equipped with a dual port
+Intel X520 10 Gbit/s network card. Both hosts were connected using DAC
+without any equipment in between. Figure \ref{fig:softwarenat64design}
+shows the setup.
+\begin{figure}[h]
+  \includegraphics[scale=0.5]{softwarenat64design}
+  \centering
+  \caption{NAT64 in software benchmark}
+  \label{fig:softwarenat64design}
+\end{figure}
+When testing the NetPFGA/P4 performance, the X520 cards in the NAT64
+translator were diconnected and instead the NetPFGA ports were
+connected, as show in figure \ref{fig:netpfgadesign}. The load
+generator is equipped with a quad core CPU (Intel(R) Core(TM) i7-6700
+CPU @ 3.40GHz), enabled with hyperthreading and 16 GB RAM. The NAT64
+translator is also equipped with a quard core CPU (Intel(R) Core(TM)
+i7-4770 CPU @ 3.40GHz) and 16 GB RAM.
+\begin{figure}[h]
+  \includegraphics[scale=0.5]{netpfgadesign}
+  \centering
+  \caption{NAT64 with NetFPGA benchmark}
+  \label{fig:netpfgadesign}
+\end{figure}

doc/Results.tex

@@ -1,15 +1,53 @@
 \chapter{\label{results}Results}
 %** Results.tex: What were the results achieved including an evaluation
 %
+This section describes the achieved results and compares the P4 based
+implementation with real world software solutions.
 
-\section{\label{results:general}General}
+We distinguish the software implementation of P4 (BMV2) and the
-Parser for all protocols (udp,tcp,icmp,icmp6)
+hardware implementation (NetFPGA) due to significant differences in
+deployment and development. We present benchmarks for the existing
+software solutions as well as for our hardware implementation. As the
+objective of this thesis was to demonstrate the high speed
+capabilities of NAT64 in hardware, no benchmarks were performed on the
+P4 software implementation.
+% ----------------------------------------------------------------------
+\section{\label{results:p4}NAT64 with P4}
+We successfully implemented P4 code to realise
+NAT64\cite{schottelius:thesisrepo}. It contains parsers
+for all related protocols (ipv6, ipv4, udp, tcp, icmp, icmp6, ndp,
+arp), supports EAMT as defined by RFC7757 \cite{rfc7757} and is
+feature equivalent to the two compared software solutions
+tayga\cite{lutchansky:_tayga_simpl_nat64_linux} and
+jool\cite{mexico:_jool_open_sourc_siit_nat64_linux}.
+Due to limitations in the P4 environment of the
+NetFPGA\cite{conclusion:netfpga} environment, the BMV2 implementation
+is more feature rich. Table \ref{tab:benchmark} summarises the
+achieved bandwidths of the NAT64 solutions.
 
-BMV2: more feature rich, but software only solution
+\begin{table}[htbp]
-NetFPGA: capabale of line speed Nat64, focused port on nat64
+\begin{center}\begin{minipage}{\textwidth}
+\begin{tabular}{| c | p{130pt} | l |}
+\hline
+Solution & Column 2 \newline (additional line) & Column 3 \\
+\hline
+Tayga & C2,R2 & C2,R3 \\
+\hline
+Jool	& \multicolumn{2}{| c |}{C2\&C3,R3} \\
+\hline
+P4 / NetPFGA & C2,R4\footnote{Footnote to table~\ref{tab:benchmark}} & C3,R4\\
+\hline
+\end{tabular}
+\end{minipage}
+\caption{Table 1}
+\label{tab:benchmark}
+\end{center}
+\end{table}
 
-Both support EAMT as defined by RFC7757 \cite{rfc7757}.
+During the benchmarks the client
 
+
+% ----------------------------------------------------------------------
 \section{\label{Results:BMV2}BMV2}
 
 Responds to icmp, icmp6
@@ -28,11 +66,13 @@ RFC6145\cite{rfc6145}.
 
 Stateful : no automatic removal
 
-
+% ----------------------------------------------------------------------
 \section{\label{results:tayga}Tayga}
 
 3gbit
 
+% ----------------------------------------------------------------------
+
 \section{\label{results:jool}Jool}
 
 

doc/appendix.tex

@@ -2,22 +2,24 @@
 The following sections describe how to acquire the resources
 to reproduce the test results. All compilations were made on Ubuntu
 16.04 with kernels
-
 \begin{itemize}
-\item 4.15.0-54-generic (Supporting Desktop),
+\item 4.15.0-54-generic (Supporting Desktop)
 \item 4.4.0-143-generic (BMV2 test VM)
-\item 4.15.0-55-generic (Desktop with NetFPGA
+\item 4.15.0-55-generic (Desktop with NetFPGA card)
 \end{itemize}
-
+% ok
+% ----------------------------------------------------------------------
 \section{\label{chapterminus1:thesis:general}Master Thesis}
 The master thesis including all self developed source code is
 available by git via
-
 \begin{verbatim}
 git clone git@gitlab.ethz.ch:nicosc/master-thesis.git
+git clone https://gitlab.ethz.ch/nsg/student-projects/ma-2019-19_high_speed_nat64_with_p4
 \end{verbatim}
-
+It can be browsed online on
-It can be browsed online on \url{https://gitlab.ethz.ch/nicosc/master-thesis}.
+\url{https://gitlab.ethz.ch/nicosc/master-thesis} and on
+\url{https://gitlab.ethz.ch/nsg/student-projects/ma-2019-19_high_speed_nat64_with_p4}.
+% ok
 % ----------------------------------------------------------------------
 \section{\label{chapterminus1:thesis:xilinx}Xilinx Toolchain}
 A prerequisite for building the NetFGPA source code is the
@@ -26,10 +28,9 @@ installation of
 \item \verb=Xilinx_SDNet_2018.2_1005_9=
 \item \verb=Xilinx_Vivado_SDK_2018.2_0614_1954=
 \end{itemize}
-
 Both tools need to be installed to /opt/Xilinx/, as paths are
 hardcoded in various places.
-
+% ok
 % ----------------------------------------------------------------------
 \section{\label{chapterminus1:thesis}NetFGPA support scripts}
 To be able to compile P4 source code to the NetFPGA the collection of
@@ -45,7 +46,6 @@ After that the variable \verb=P4_PROJECT_NAME= in
 \texttt{~/projects/P4-NetFPGA/tools/settings.sh} needs to be modified to
 read \verb|export P4_PROJECT_NAME=minip4| instead of
 \verb|export P4_PROJECT_NAME=switch_calc|.
-
 Sample code for installation:
 
 \begin{verbatim}
@@ -53,38 +53,33 @@ Sample code for installation:
   git clone git@github.com:NetFPGA/P4-NetFPGA-live.git P4-NetFPGA
   sed -i 's/\(P4_PROJECT_NAME=\).*/\1minip4/' ~/projects/P4-NetFPGA/tools/settings.sh
 \end{verbatim}
-
 Version \textbf{v1.3.1-46-g97d3aaa} of the P4-NetPFGA repository was
 used for creating the bitfiles of this project.
-
-
 \begin{verbatim}
 nico@nsg-System:~/projects/P4-NetFPGA$ git describe --always
 v1.3.1-46-g97d3aaa
 \end{verbatim}
-
+% ok
 % ----------------------------------------------------------------------
 \chapter{\label{appendix0}BMV2 environment and tests}
 All BMV2 based compilations were made with the following compiler:
-
 \begin{verbatim}
 p4@ubuntu:~$ p4c --version
 p4c 0.5 (SHA: 5ae30ee)
 \end{verbatim}
 
 The installation is based on the vagrant files that were provided in
-the
+the ``Advanced Topics in
-``Advanced Topics in
 Communication Networks Fall 2018'' course of
 ETHZ (\url{https://adv-net.ethz.ch/2018/}) and contains p4tools as
 well as all utilities that came with the vagrant installation.
-
+% ok
-\section{\label{chapter0:bmv2-diff}Diff based checksumming}
+% ----------------------------------------------------------------------
+%\section{\label{chapter0:bmv2-diff}Diff based checksumming}
 For running the diff based checksum code, the following steps are
 necessary:
 
 Compiling the p4 code and starting the switch:
-
 \begin{verbatim}
 cd ~/master-thesis/p4app
 sudo p4run --config nat64-diff.json
@@ -100,9 +95,14 @@ sudo python ./controller.py --mode range_router
 
 % ----------------------------------------------------------------------
 \chapter{\label{appendixA}NetFPGA environment and tests}
+% ----------------------------------------------------------------------
 \section{\label{chapterA:netpfga-setup}NetFPGA Setup}
 Description of installation, commit of netpfga-live
+% ----------------------------------------------------------------------
 
+\section{\label{chapterA:netpfga:compile}NetFPGA Compile Flow}
+
+% ----------------------------------------------------------------------
 \section{\label{chapterA:section1}NetFPGA NAT64 Test cases}
 todo: add graphic of nsg <-> esprimo cabling
 \begin{verbatim}
@@ -131,8 +131,8 @@ For all test cases the following network settings on esprimo:
     inet6 fe80::faf2:1eff:fe09:62d1/64 scope link
        valid_lft forever preferred_lft forever
 \end{verbatim}
-
+% ----------------------------------------------------------------------
-\subsection{Test 1: IPv4 egress settings work}
+\subsection{Test 1: IPv4 egress}
 Scenario: simple egress port setting for the IPv4 addresses
 
 Step 1: getting correct values for table entries from python:
@@ -208,6 +208,7 @@ listening on enp2s0f0, link-type EN10MB (Ethernet), capture size 262144 bytes
 Result: sucess
 
 
+% ----------------------------------------------------------------------
 \subsection{Test 2: IPv6 egress}
 Similar to the IPv4 setting before, just for IPv6.
 
@@ -393,18 +394,12 @@ sudo ip -6 neighbor add 2001:db8:42::a00:2a lladdr f8:f2:1e:09:62:d0 dev enp2s0f
 
 Step 4: ping test should translate, but fail with wrong checksum:
 
-\begin{verbatim}
+% ----------------------------------------------------------------------
-\end{verbatim}
-
-
-\begin{verbatim}
-\end{verbatim}
-
-
-
 \chapter{\label{appendixB}NetFPGA Logs}
 Majority of the log files are stored inside the source code directory
-stored at ``netpfga/logs''. It follows a selection of log files
+stored at ``netpfga/logs''. It follows a selection of excerpts
+of log files that might be relevant for reproducing the work.
+% ----------------------------------------------------------------------
 
 \section{\label{chapterB:netpfga-flasherror}NetFPGA Flash Errors}
 Sometimes flashing bitfiles to the NetFPGA will fail. A random amount
@@ -458,6 +453,7 @@ nf3: ERROR while getting interface flags: No such device
 + bash config_writes.sh
 
 \end{verbatim}
+% ----------------------------------------------------------------------
 
 \section{\label{chapterB:netpfga-flashok}NetFPGA Flash Success}
 A successful flashing process also emits a couple of errors, however
@@ -511,6 +507,7 @@ nf3: ERROR while getting interface flags: No such device
 + bash config_writes.sh
 nico@nsg-System:~/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/simple_sume_switch/bitfiles$
 \end{verbatim}
+% ----------------------------------------------------------------------
 
 \section{\label{chapterB:netpfga-kernelmodule}NetFPGA Kernel module}
 After a successful flash, loading the kernel module will enable nf
@@ -582,6 +579,7 @@ nico@nsg-System:~$ ip l
 nico@nsg-System:~$
 
 \end{verbatim}
+% ----------------------------------------------------------------------
 
 \section{\label{chapterB:netpfga-nftraffic}NetFPGA misses packets on nf*}
 While the nf devices appear in the operating system, packets emitted
@@ -591,6 +589,8 @@ that is connected to the specific output port.
 
 %---------------------------------------------------------------------------------------------------------
 \chapter{\label{benchmark}Benchmark Logs}
+% ----------------------------------------------------------------------
+
 \section{\label{benchmark:iperf}iperf}
 Omitting startup time
 
@@ -2369,7 +2369,6 @@ Describe your task.
 
 %---------------------------------------------------------------------------------------------------------
 \printnomenclature
-
 \abbrev{ARP}{Address resolution protocol}
 \abbrev{ASIC}{Application-specific integrated circuit}
 \abbrev{FGPA}{Field-programmable gate array}

doc/graphviz/netpfgadesign.dot

@@ -4,7 +4,7 @@ graph G {
 
     x520_1 [ label="X520: IPv4" ];
     x520_2 [ label="X520: IPv6" ];
-    x520_nsg [ label="X520: Controller" ];
+    x520_nsg [ label="X520: P4 Controller" ];
 
     netpfga1 [ label="NetFPGA Port 1" ];
     netpfga2 [ label="NetFPGA Port 2" ];

doc/graphviz/softwarenat64design.dot

@@ -0,0 +1,27 @@
+graph G {
+    node [ shape="box"];
+    rankdir="LR";
+
+    x520_1 [ label="X520: IPv4" ];
+    x520_2 [ label="X520: IPv6" ];
+    x520_nsg_1 [ label="X520: IPv4" ];
+    x520_nsg_2 [ label="X520: IPv6" ];
+
+    subgraph cluster_esprimo {
+        label="Load generator";
+
+        x520_1;
+        x520_2;
+    }
+
+    subgraph cluster_nsg {
+        label="NAT64 Host";
+
+        x520_nsg_1;
+        x520_nsg_2;
+    }
+
+    x520_1--x520_nsg_1;
+    x520_2--x520_nsg_2;
+
+}

doc/refs/refs.bib

@@ -69,6 +69,11 @@
   title =     {TAYGA - Simple, no-fuss NAT64 for Linux},
   howpublished = {\url{http://www.litech.org/tayga/}}}
 
+@Misc{mexico:_jool_open_sourc_siit_nat64_linux,
+  author =    {NIC Mexico},
+  title =     {Jool an Open Source SIIT and NAT64 for Linux.},
+  howpublished = {\url{https://www.jool.mx/en/index.html}}}
+
 @Misc{vanbever:_progr_networ_data_planes,
   author =    {Laurent Vanbever},
   title =     {Programming Network Data Planes},
@@ -115,3 +120,8 @@
   title =     {Solicited-node multicast address},
   howpublished = {\url{https://en.wikipedia.org/wiki/Solicited-node_multicast_address}},
   note =      {Requested on 2019-08-13}}
+
+@Misc{schottelius:thesisrepo,
+  author =    {Nico Schottelius},
+  title =     {High speed NAT64 in P4 (git repository)},
+  howpublished = {\url{https://gitlab.ethz.ch/nsg/student-projects/ma-2019-19_high_speed_nat64_with_p4}}}