[++doc] add benchmark design

This commit is contained in:
Nico Schottelius 2019-08-14 17:23:12 +02:00
parent ce02b44164
commit 393a6ecd91
9 changed files with 142 additions and 40 deletions

View File

@ -182,3 +182,29 @@ not the full headers are used, but the pseudo headers (compare figures
To compensate the carry bit, our code uses 17 bit integers for
correcting the carry.
% FIXME: add note to python script / checksum diffing
% ----------------------------------------------------------------------
\section{\label{Design:Benchmarks}Benchmarks}
The benchmarks were performed on two hosts, a load generator and a
nat64 translator. Both hosts were equipped with a dual port
Intel X520 10 Gbit/s network card. Both hosts were connected using DAC
without any equipment in between. Figure \ref{fig:softwarenat64design}
shows the setup.
\begin{figure}[h]
\includegraphics[scale=0.5]{softwarenat64design}
\centering
\caption{NAT64 in software benchmark}
\label{fig:softwarenat64design}
\end{figure}
When testing the NetPFGA/P4 performance, the X520 cards in the NAT64
translator were diconnected and instead the NetPFGA ports were
connected, as show in figure \ref{fig:netpfgadesign}. The load
generator is equipped with a quad core CPU (Intel(R) Core(TM) i7-6700
CPU @ 3.40GHz), enabled with hyperthreading and 16 GB RAM. The NAT64
translator is also equipped with a quard core CPU (Intel(R) Core(TM)
i7-4770 CPU @ 3.40GHz) and 16 GB RAM.
\begin{figure}[h]
\includegraphics[scale=0.5]{netpfgadesign}
\centering
\caption{NAT64 with NetFPGA benchmark}
\label{fig:netpfgadesign}
\end{figure}

View File

@ -1,15 +1,53 @@
\chapter{\label{results}Results}
%** Results.tex: What were the results achieved including an evaluation
%
This section describes the achieved results and compares the P4 based
implementation with real world software solutions.
\section{\label{results:general}General}
Parser for all protocols (udp,tcp,icmp,icmp6)
We distinguish the software implementation of P4 (BMV2) and the
hardware implementation (NetFPGA) due to significant differences in
deployment and development. We present benchmarks for the existing
software solutions as well as for our hardware implementation. As the
objective of this thesis was to demonstrate the high speed
capabilities of NAT64 in hardware, no benchmarks were performed on the
P4 software implementation.
% ----------------------------------------------------------------------
\section{\label{results:p4}NAT64 with P4}
We successfully implemented P4 code to realise
NAT64\cite{schottelius:thesisrepo}. It contains parsers
for all related protocols (ipv6, ipv4, udp, tcp, icmp, icmp6, ndp,
arp), supports EAMT as defined by RFC7757 \cite{rfc7757} and is
feature equivalent to the two compared software solutions
tayga\cite{lutchansky:_tayga_simpl_nat64_linux} and
jool\cite{mexico:_jool_open_sourc_siit_nat64_linux}.
Due to limitations in the P4 environment of the
NetFPGA\cite{conclusion:netfpga} environment, the BMV2 implementation
is more feature rich. Table \ref{tab:benchmark} summarises the
achieved bandwidths of the NAT64 solutions.
BMV2: more feature rich, but software only solution
NetFPGA: capabale of line speed Nat64, focused port on nat64
\begin{table}[htbp]
\begin{center}\begin{minipage}{\textwidth}
\begin{tabular}{| c | p{130pt} | l |}
\hline
Solution & Column 2 \newline (additional line) & Column 3 \\
\hline
Tayga & C2,R2 & C2,R3 \\
\hline
Jool & \multicolumn{2}{| c |}{C2\&C3,R3} \\
\hline
P4 / NetPFGA & C2,R4\footnote{Footnote to table~\ref{tab:benchmark}} & C3,R4\\
\hline
\end{tabular}
\end{minipage}
\caption{Table 1}
\label{tab:benchmark}
\end{center}
\end{table}
Both support EAMT as defined by RFC7757 \cite{rfc7757}.
During the benchmarks the client
% ----------------------------------------------------------------------
\section{\label{Results:BMV2}BMV2}
Responds to icmp, icmp6
@ -28,11 +66,13 @@ RFC6145\cite{rfc6145}.
Stateful : no automatic removal
% ----------------------------------------------------------------------
\section{\label{results:tayga}Tayga}
3gbit
% ----------------------------------------------------------------------
\section{\label{results:jool}Jool}

Binary file not shown.

View File

@ -2,22 +2,24 @@
The following sections describe how to acquire the resources
to reproduce the test results. All compilations were made on Ubuntu
16.04 with kernels
\begin{itemize}
\item 4.15.0-54-generic (Supporting Desktop),
\item 4.15.0-54-generic (Supporting Desktop)
\item 4.4.0-143-generic (BMV2 test VM)
\item 4.15.0-55-generic (Desktop with NetFPGA
\item 4.15.0-55-generic (Desktop with NetFPGA card)
\end{itemize}
% ok
% ----------------------------------------------------------------------
\section{\label{chapterminus1:thesis:general}Master Thesis}
The master thesis including all self developed source code is
available by git via
\begin{verbatim}
git clone git@gitlab.ethz.ch:nicosc/master-thesis.git
git clone https://gitlab.ethz.ch/nsg/student-projects/ma-2019-19_high_speed_nat64_with_p4
\end{verbatim}
It can be browsed online on \url{https://gitlab.ethz.ch/nicosc/master-thesis}.
It can be browsed online on
\url{https://gitlab.ethz.ch/nicosc/master-thesis} and on
\url{https://gitlab.ethz.ch/nsg/student-projects/ma-2019-19_high_speed_nat64_with_p4}.
% ok
% ----------------------------------------------------------------------
\section{\label{chapterminus1:thesis:xilinx}Xilinx Toolchain}
A prerequisite for building the NetFGPA source code is the
@ -26,10 +28,9 @@ installation of
\item \verb=Xilinx_SDNet_2018.2_1005_9=
\item \verb=Xilinx_Vivado_SDK_2018.2_0614_1954=
\end{itemize}
Both tools need to be installed to /opt/Xilinx/, as paths are
hardcoded in various places.
% ok
% ----------------------------------------------------------------------
\section{\label{chapterminus1:thesis}NetFGPA support scripts}
To be able to compile P4 source code to the NetFPGA the collection of
@ -45,7 +46,6 @@ After that the variable \verb=P4_PROJECT_NAME= in
\texttt{~/projects/P4-NetFPGA/tools/settings.sh} needs to be modified to
read \verb|export P4_PROJECT_NAME=minip4| instead of
\verb|export P4_PROJECT_NAME=switch_calc|.
Sample code for installation:
\begin{verbatim}
@ -53,38 +53,33 @@ Sample code for installation:
git clone git@github.com:NetFPGA/P4-NetFPGA-live.git P4-NetFPGA
sed -i 's/\(P4_PROJECT_NAME=\).*/\1minip4/' ~/projects/P4-NetFPGA/tools/settings.sh
\end{verbatim}
Version \textbf{v1.3.1-46-g97d3aaa} of the P4-NetPFGA repository was
used for creating the bitfiles of this project.
\begin{verbatim}
nico@nsg-System:~/projects/P4-NetFPGA$ git describe --always
v1.3.1-46-g97d3aaa
\end{verbatim}
% ok
% ----------------------------------------------------------------------
\chapter{\label{appendix0}BMV2 environment and tests}
All BMV2 based compilations were made with the following compiler:
\begin{verbatim}
p4@ubuntu:~$ p4c --version
p4c 0.5 (SHA: 5ae30ee)
\end{verbatim}
The installation is based on the vagrant files that were provided in
the
``Advanced Topics in
the ``Advanced Topics in
Communication Networks Fall 2018'' course of
ETHZ (\url{https://adv-net.ethz.ch/2018/}) and contains p4tools as
well as all utilities that came with the vagrant installation.
\section{\label{chapter0:bmv2-diff}Diff based checksumming}
% ok
% ----------------------------------------------------------------------
%\section{\label{chapter0:bmv2-diff}Diff based checksumming}
For running the diff based checksum code, the following steps are
necessary:
Compiling the p4 code and starting the switch:
\begin{verbatim}
cd ~/master-thesis/p4app
sudo p4run --config nat64-diff.json
@ -100,9 +95,14 @@ sudo python ./controller.py --mode range_router
% ----------------------------------------------------------------------
\chapter{\label{appendixA}NetFPGA environment and tests}
% ----------------------------------------------------------------------
\section{\label{chapterA:netpfga-setup}NetFPGA Setup}
Description of installation, commit of netpfga-live
% ----------------------------------------------------------------------
\section{\label{chapterA:netpfga:compile}NetFPGA Compile Flow}
% ----------------------------------------------------------------------
\section{\label{chapterA:section1}NetFPGA NAT64 Test cases}
todo: add graphic of nsg <-> esprimo cabling
\begin{verbatim}
@ -131,8 +131,8 @@ For all test cases the following network settings on esprimo:
inet6 fe80::faf2:1eff:fe09:62d1/64 scope link
valid_lft forever preferred_lft forever
\end{verbatim}
\subsection{Test 1: IPv4 egress settings work}
% ----------------------------------------------------------------------
\subsection{Test 1: IPv4 egress}
Scenario: simple egress port setting for the IPv4 addresses
Step 1: getting correct values for table entries from python:
@ -208,6 +208,7 @@ listening on enp2s0f0, link-type EN10MB (Ethernet), capture size 262144 bytes
Result: sucess
% ----------------------------------------------------------------------
\subsection{Test 2: IPv6 egress}
Similar to the IPv4 setting before, just for IPv6.
@ -393,18 +394,12 @@ sudo ip -6 neighbor add 2001:db8:42::a00:2a lladdr f8:f2:1e:09:62:d0 dev enp2s0f
Step 4: ping test should translate, but fail with wrong checksum:
\begin{verbatim}
\end{verbatim}
\begin{verbatim}
\end{verbatim}
% ----------------------------------------------------------------------
\chapter{\label{appendixB}NetFPGA Logs}
Majority of the log files are stored inside the source code directory
stored at ``netpfga/logs''. It follows a selection of log files
stored at ``netpfga/logs''. It follows a selection of excerpts
of log files that might be relevant for reproducing the work.
% ----------------------------------------------------------------------
\section{\label{chapterB:netpfga-flasherror}NetFPGA Flash Errors}
Sometimes flashing bitfiles to the NetFPGA will fail. A random amount
@ -458,6 +453,7 @@ nf3: ERROR while getting interface flags: No such device
+ bash config_writes.sh
\end{verbatim}
% ----------------------------------------------------------------------
\section{\label{chapterB:netpfga-flashok}NetFPGA Flash Success}
A successful flashing process also emits a couple of errors, however
@ -511,6 +507,7 @@ nf3: ERROR while getting interface flags: No such device
+ bash config_writes.sh
nico@nsg-System:~/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/simple_sume_switch/bitfiles$
\end{verbatim}
% ----------------------------------------------------------------------
\section{\label{chapterB:netpfga-kernelmodule}NetFPGA Kernel module}
After a successful flash, loading the kernel module will enable nf
@ -582,6 +579,7 @@ nico@nsg-System:~$ ip l
nico@nsg-System:~$
\end{verbatim}
% ----------------------------------------------------------------------
\section{\label{chapterB:netpfga-nftraffic}NetFPGA misses packets on nf*}
While the nf devices appear in the operating system, packets emitted
@ -591,6 +589,8 @@ that is connected to the specific output port.
%---------------------------------------------------------------------------------------------------------
\chapter{\label{benchmark}Benchmark Logs}
% ----------------------------------------------------------------------
\section{\label{benchmark:iperf}iperf}
Omitting startup time
@ -2369,7 +2369,6 @@ Describe your task.
%---------------------------------------------------------------------------------------------------------
\printnomenclature
\abbrev{ARP}{Address resolution protocol}
\abbrev{ASIC}{Application-specific integrated circuit}
\abbrev{FGPA}{Field-programmable gate array}

View File

@ -4,7 +4,7 @@ graph G {
x520_1 [ label="X520: IPv4" ];
x520_2 [ label="X520: IPv6" ];
x520_nsg [ label="X520: Controller" ];
x520_nsg [ label="X520: P4 Controller" ];
netpfga1 [ label="NetFPGA Port 1" ];
netpfga2 [ label="NetFPGA Port 2" ];

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

View File

@ -0,0 +1,27 @@
graph G {
node [ shape="box"];
rankdir="LR";
x520_1 [ label="X520: IPv4" ];
x520_2 [ label="X520: IPv6" ];
x520_nsg_1 [ label="X520: IPv4" ];
x520_nsg_2 [ label="X520: IPv6" ];
subgraph cluster_esprimo {
label="Load generator";
x520_1;
x520_2;
}
subgraph cluster_nsg {
label="NAT64 Host";
x520_nsg_1;
x520_nsg_2;
}
x520_1--x520_nsg_1;
x520_2--x520_nsg_2;
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.2 KiB

View File

@ -69,6 +69,11 @@
title = {TAYGA - Simple, no-fuss NAT64 for Linux},
howpublished = {\url{http://www.litech.org/tayga/}}}
@Misc{mexico:_jool_open_sourc_siit_nat64_linux,
author = {NIC Mexico},
title = {Jool an Open Source SIIT and NAT64 for Linux.},
howpublished = {\url{https://www.jool.mx/en/index.html}}}
@Misc{vanbever:_progr_networ_data_planes,
author = {Laurent Vanbever},
title = {Programming Network Data Planes},
@ -115,3 +120,8 @@
title = {Solicited-node multicast address},
howpublished = {\url{https://en.wikipedia.org/wiki/Solicited-node_multicast_address}},
note = {Requested on 2019-08-13}}
@Misc{schottelius:thesisrepo,
author = {Nico Schottelius},
title = {High speed NAT64 in P4 (git repository)},
howpublished = {\url{https://gitlab.ethz.ch/nsg/student-projects/ma-2019-19_high_speed_nat64_with_p4}}}