Merge branch 'master' of gitlab.ethz.ch:nicosc/master-thesis

This commit is contained in:
Nico Schottelius 2019-08-04 12:21:44 +02:00
commit 100b0a9873
5 changed files with 563 additions and 142 deletions

View File

@ -1,4 +1,60 @@
\chapter{\label{appendixA}Test descriptions}
\chapter{\label{appendixminus1}Resources and code repositories}
The following sections describe how to acquire the resources
to reproduce the test results.
\section{\label{chapterminus1:thesis}Master Thesis}
The master thesis including all self developed source code is
available by git via
\begin{verbatim}
git clone git@gitlab.ethz.ch:nicosc/master-thesis.git
\end{verbatim}
It can be browsed online on \url{https://gitlab.ethz.ch/nicosc/master-thesis}.
% ----------------------------------------------------------------------
\section{\label{chapterminus1:thesis}Xilinx Toolchain}
A prerequisite for building the NetFGPA source code is the
installation of
\begin{itemize}
\item \verb=Xilinx_SDNet_2018.2_1005_9=
\item \verb=Xilinx_Vivado_SDK_2018.2_0614_1954=
\end{itemize}
Both tools need to be installed to /opt/Xilinx/, as paths are
hardcoded in various places.
% ----------------------------------------------------------------------
\section{\label{chapterminus1:thesis}NetFGPA}
To be able to compile P4 source code to the NetFPGA the collection of
scripts, Makefiles and sample code of P4-NetFGPA is required.
The repository \url{git@github.com:NetFPGA/P4-NetFPGA-live.git} needs
to be cloned to ``projects'' subdirectory as ``P4-NetPFGA''
of the user that wants to
compile the source code. Access to the repository is granted after
applying for access as described on
\url{https://github.com/NetFPGA/P4-NetFPGA-public/wiki}.
After that the variable \verb=P4_PROJECT_NAME= in
\url{~/projects/P4-NetFPGA/tools/settings.sh} needs to be modified to
read \verb|export P4_PROJECT_NAME=minip4| instead of
\verb|export P4_PROJECT_NAME=switch_calc|.
Sample code for installation:
\begin{verbatim}
mkdir -p ~/projects
git clone git@github.com:NetFPGA/P4-NetFPGA-live.git P4-NetFPGA
sed -i 's/\(P4_PROJECT_NAME=\).*/\1minip4/' ~/projects/P4-NetFPGA/tools/settings.sh
\end{verbatim}
% ----------------------------------------------------------------------
\chapter{\label{appendix0}BMV2 environment and tests}
\section{\label{chapter0:bmv2-diff}Diff based checksumming}
\begin{verbatim}
\end{verbatim}
\chapter{\label{appendixA}NetFPGA environment and tests}
\section{\label{chapterA:netpfga-setup}NetFPGA Setup}
Description of installation, commit of netpfga-live

View File

@ -8267,146 +8267,6 @@ table_cam_add_entry realmain_nat64_0 realmain.nat64_static 42540766411362381960
#+END_CENTER
*** TODO 2019-08-02: ipv4 checksum after nat64 is empty!
*** TODO 2019-08-03
** The NetPFGA saga
Problems encountered:
- The logfile for a compile run is 10k+ lines
- Many logged errors can actually be ignored (?) like:
ERROR: [VRFC 10-1491] unexpected EOF [/home/nico/master-thesis/netpfga/minip4/nf_sume_sdnet_ip/SimpleSumeSwitch/S_CONTROLLERs.HDL/S_CONTROLLER_SimpleSumeSwitch.vp:37]
ERROR: [VRFC 10-426] cannot find port tuple_out_sume_metadata_DATA on this module [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/wrapper/nf_sume_sdnet.v:219]
ERROR: [VRFC 10-426] cannot find port tuple_out_sume_metadata_VALID on this module [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/wrapper/nf_sume_sdnet.v:218]
ERROR: [VRFC 10-426] cannot find port tuple_in_sume_metadata_DATA on this module [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/wrapper/nf_sume_sdnet.v:185]
ERROR: [VRFC 10-426] cannot find port tuple_in_sume_metadata_VALID on this module [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/wrapper/nf_sume_sdnet.v:184]
ERROR: [VRFC 10-2063] Module <S_RESETTER_line> not found while processing module instance <S_RESET_clk_line> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/Simp
leSumeSwitch/SimpleSumeSwitch.v:332]
ERROR: [VRFC 10-2063] Module <S_RESETTER_lookup> not found while processing module instance <S_RESET_clk_lookup> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/
SimpleSumeSwitch/SimpleSumeSwitch.v:343]
ERROR: [VRFC 10-2063] Module <S_RESETTER_control> not found while processing module instance <S_RESET_clk_control> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_i
p/SimpleSumeSwitch/SimpleSumeSwitch.v:354]
ERROR: [VRFC 10-2063] Module <TopParser_t> not found while processing module instance <TopParser> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/SimpleSumeSwitc
h/SimpleSumeSwitch.v:436]
ERROR: [VRFC 10-2063] Module <TopPipe_lvl_t> not found while processing module instance <TopPipe_lvl> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/SimpleSumeS
witch/SimpleSumeSwitch.v:474]
ERROR: [VRFC 10-2063] Module <dummy_table_for_netpfga_t> not found while processing module instance <dummy_table_for_netpfga> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_s
ume_sdnet_ip/SimpleSumeSwitch/SimpleSumeSwitch.v:502]
ERROR: [VRFC 10-2063] Module <TopPipe_lvl_0_t> not found while processing module instance <TopPipe_lvl_0> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/SimpleS
umeSwitch/SimpleSumeSwitch.v:533]
ERROR: [VRFC 10-2063] Module <TopDeparser_t> not found while processing module instance <TopDeparser> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/SimpleSumeS
witch/SimpleSumeSwitch.v:561]
# launch_simulation -simset sim_1 -mode behavioral
INFO: [Vivado 12-5698] Checking validity of IPs in the design for the 'XSim' simulator...
CRITICAL WARNING: [BD 41-1356] Address block </M04_AXI/Reg> is not mapped into </S00_AXI>. Please use Address Editor to either map or exclude it.
CRITICAL WARNING: [BD 41-1356] Address block </M05_AXI/Reg> is not mapped into </S00_AXI>. Please use Address Editor to either map or exclude it.
WARNING: [VRFC 10-756] identifier state is used before its declaration [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/axis_sim_record_ip0/hdl/axis_sim_record.v:93]
WARNING: [VRFC 10-756] identifier ready_count is used before its declaration [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/axis_sim_record_ip0/hdl/axis_sim_record.v:94]
INFO: [#UNDEF] Sorry, too many errors..
ERROR: [XSIM 43-3322] Static elaboration of top level Verilog design unit(s) in library work failed.
INFO: [USF-XSim-69] 'elaborate' step finished in '1' seconds
INFO: [USF-XSim-99] Step results log file:'/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log'
ERROR: [USF-XSim-62] 'elaborate' step failed with error(s). Please check the Tcl console output or '/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log' file for more information.
nico@nsg-System:~/master-thesis$ find . -name elaborate.log
nico@nsg-System:~/master-thesis$ find ~ -name elaborate.log
nico@nsg-System:~/master-thesis$
- Scripts that "fail" (generate wrong data) do exit 0 ->
There is no easy / reliable error detection
- Writing tables resulted in ioctl errors
- Hardware test: unclear if first board was/is broken or not,
BUT: second board in different computer allows writing tables
- Many scripts depend on each other in later stages, without clear
dependencies
- There is basically no documentation for someone who "just wants to
compile from P4 to netpfga" or A LOT of documentation (if vivado,
vhld, sdnet documentation is counted)
- Very high complexity in toolchain, scripts that are generated
+ cd /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/simple_sume_switch/test/sim_switch_default
+ make
rm -f config_writes.py*
rm -f *.pyc
nico@nsg-System:~$ cat /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/testdata/config_writes.py
from NFTest import *
NUM_WRITES = 4
def config_tables():
nftest_regwrite(0x44020050, 0x22222208)
nftest_regwrite(0x44020054, 0x00000822)
nftest_regwrite(0x44020080, 0x00000201)
nftest_regwrite(0x44020040, 0x00000001)
nico@nsg-System:~$ cat /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/testdata/config_writes.sh
#!/bin/bash
${SUME_SDNET}/sw/sume/rwaxi -a 0x44020050 -w 0x22222208
${SUME_SDNET}/sw/sume/rwaxi -a 0x44020054 -w 0x00000822
${SUME_SDNET}/sw/sume/rwaxi -a 0x44020080 -w 0x00000201
${SUME_SDNET}/sw/sume/rwaxi -a 0x44020040 -w 0x00000001
nico@nsg-System:~$
- Misleading errors like
ERROR: [USF-XSim-62] 'elaborate' step failed with error(s). Please check the Tcl console output or '/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log' file for more information.
nico@nsg-System:~/master-thesis/netpfga$ ls /home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log
ls: cannot access '/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log': No such file or directory
- not using raise() and hiding source of errors (_hexify)
- sometimes flashing fails:
#+BEGIN_CENTER
nico@nsg-System:~/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/simple_sume_switch/bitfiles$ sudo bash -c ". $HOME/master-thesis/netpfga/bashinit && $(pwd -P)/program_switch.sh"
++ which vivado
+ xilinx_tool_path=/opt/Xilinx/Vivado/2018.2/bin/vivado
+ bitimage=minip4.bit
+ configWrites=config_writes.sh
+ '[' -z minip4.bit ']'
+ '[' -z config_writes.sh ']'
+ '[' /opt/Xilinx/Vivado/2018.2/bin/vivado == '' ']'
+ rmmod sume_riffa
+ xsct /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/tools/run_xsct.tcl -tclargs minip4.bit
rlwrap: warning: your $TERM is 'screen' but rlwrap couldn't find it in the terminfo database. Expect some problems.
RUN loading image file.
minip4.bit
100% 19MB 1.7MB/s 00:11
fpga configuration failed. DONE PIN is not HIGH
invoked from within
"::tcf::eval -progress ::xsdb::print_progress {::tcf::cache_enter tcfchan#0 {tcf_cache_eval {process_tcf_actions_cache_client ::tcfclient#0::arg}}}"
(procedure "::tcf::cache_eval_with_progress" line 2)
invoked from within
"::tcf::cache_eval_with_progress [dict get $arg chan] [list process_tcf_actions_cache_client $argvar] $progress"
(procedure "process_tcf_actions" line 1)
invoked from within
"process_tcf_actions $arg ::xsdb::print_progress"
(procedure "fpga" line 430)
invoked from within
"fpga -f $bitimage"
(file "/home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/tools/run_xsct.tcl" line 33)
+ bash /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/tools/pci_rescan_run.sh
Check programming FPGA or Reboot machine !
+ rmmod sume_riffa
rmmod: ERROR: Module sume_riffa is not currently loaded
+ modprobe sume_riffa
+ ifconfig nf0 up
nf0: ERROR while getting interface flags: No such device
+ ifconfig nf1 up
nf1: ERROR while getting interface flags: No such device
+ ifconfig nf2 up
nf2: ERROR while getting interface flags: No such device
+ ifconfig nf3 up
nf3: ERROR while getting interface flags: No such device
+ bash config_writes.sh
nico@nsg-System:~/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/simple_sume_switch/bitfiles$
#+END_CENTER
*** TODO 2019-08-04: test nat64: ipv4 checksums
**** DONE Setting table entries: ok
CLOSED: [2019-08-04 Sun 08:55]
@ -8556,7 +8416,8 @@ nico@nsg-System:~/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/project
>>
#+END_CENTER
**** TODO Testing translation: v4->v6
**** DONE setting up translation: v4->v6: ok
CLOSED: [2019-08-04 Sun 09:14]
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/bin$ ./socat-listen-tcp-v6
nico@ESPRIMO-P956:~/master-thesis$ ./bin/sniff-intf.sh enp2s0f0 netfpga-nat64
@ -8587,6 +8448,510 @@ nico@ESPRIMO-P956:~/master-thesis/bin$ ./set_ipv4_neighbor.sh
+ hwaddr=03
#+END_CENTER
**** DONE Testing v4->v6 tcp: ok (version 10.0)
CLOSED: [2019-08-04 Sun 09:15]
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/bin$ ./socat-connect-tcp-v4
+ echo from-v4-ok
+ socat - TCP:10.0.0.66:2345
TCPv6-ok
nico@ESPRIMO-P956:~/master-thesis/bin$ ./socat-listen-tcp-v6
from-v4-ok
#+END_CENTER
trace:
netfpga-nat64-2019-08-04-0907-enp2s0f0.pcap
netfpga-nat64-2019-08-04-0907-enp2s0f1.pcap
**** DONE Testing v4->v6 udp: ok (version 10.1)
trace:
create mode 100644 pcap/netfpga-nat64-udp-2019-08-04-0913-enp2s0f0.pcap
create mode 100644 pcap/netfpga-nat64-udp-2019-08-04-0913-enp2s0f1.pcap
**** TODO NOTE: outgoing ipv4 checksum incorrect -> doesn't make sense
As this one is generated by socat, no nat64 involed. Maybe tcp
offloading is active.
*** TODO 2019-08-04: Testing more traffic: fails. trace:
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/bin$ xxd /dev/urandom | socat - UDP:10.0.0.66:2345
nico@ESPRIMO-P956:~/master-thesis/bin$ while true; do xxd /dev/urandom | socat - UDP6-LISTEN:2345; sleep 0.2; done
#+END_CENTER
Too big for git:
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/pcap$ mv netfpga-nat64-tcp-xxd-2019-08-04-0923-enp2s0f* ~
nico@ESPRIMO-P956:~/master-thesis/pcap$ ls -alh ~/netfpga-nat64-tcp-xxd-2019-08-04-0923-enp2s0f*
-rw-r--r-- 1 root root 1.1G Aug 4 09:23 /home/nico/netfpga-nat64-tcp-xxd-2019-08-04-0923-enp2s0f0.pcap
-rw-r--r-- 1 root root 301M Aug 4 09:23 /home/nico/netfpga-nat64-tcp-xxd-2019-08-04-0923-enp2s0f1.pcap
nico@ESPRIMO-P956:~/master-thesis/pcap$
#+END_CENTER
Limiting to 200:
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis$ cd pcap; sudo tcpdump -ni enp2s0f0 -c 200 -w netfpga-nat64-tcp-xxd-2019-08-04-0926-enp2s0f0.pcap
nico@ESPRIMO-P956:~/master-thesis/pcap$ sudo tcpdump -ni enp2s0f0 -c 200 -w netfpga-nat64-tcp-xxd-2019-08-04-0926-enp2s0f1.pcap
#+END_CENTER
try again:
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/bin$ cat /etc/passwd | socat - UDP6-LISTEN:2345
nico@ESPRIMO-P956:~/master-thesis/bin$ cat /etc/issue | socat - UDP:10.0.0.66:2345
#+END_CENTER
Result:
Ubuntu 16.04.5 LTS \n \l
but no passwd!
*** TODO IPv6 options (fragments) set that were not set before!
Trace:
#+BEGIN_CENTER
create mode 100644 pcap/netfpga-nat64-tcp-xxd-2019-08-04-0930-enp2s0f0.pcap
create mode 100644 pcap/netfpga-nat64-tcp-xxd-2019-08-04-0930-enp2s0f1.pcap
#+END_CENTER
*** TODO retry manually: long packets break
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/pcap$ ~/master-thesis/bin/sniff-intf.sh enp2s0f0 netfpga-nat64-udp-manually
nico@ESPRIMO-P956:~/master-thesis/bin$ socat - UDP6-LISTEN:2345
nico@ESPRIMO-P956:~/master-thesis/bin$ socat - UDP:10.0.0.66:2345
#+END_CENTER
very
loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong
text
sent the very long text from ipv6:
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/bin$ socat - UDP6-LISTEN:2345
hello 1
from v6 hello
when does it break?
very
loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong
text
#+END_CENTER
only seeing this on ipv4:
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/bin$ socat - UDP:10.0.0.66:2345
hello 1
from v6 hello
when does it break?
very
text
#+END_CENTER
*** Analysing x520 features
#+BEGIN_CENTER
root@ESPRIMO-P956:~# ethtool -k enp2s0f0
Features for enp2s0f0:
Cannot get device udp-fragmentation-offload settings: Operation not supported
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: on [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: on [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: on
tx-ipxip6-segmentation: on
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off
hw-tc-offload: off
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: on
root@ESPRIMO-P956:~#
#+END_CENTER
Changing values:
#+BEGIN_CENTER
root@ESPRIMO-P956:~# ethtool -K enp2s0f0 rx off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
root@ESPRIMO-P956:~# ethtool -K enp2s0f0 tx off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
Actual changes:
tx-checksumming: off
tx-checksum-ip-generic: off
tx-checksum-sctp: off
tcp-segmentation-offload: off
tx-tcp-segmentation: off [requested on]
tx-tcp6-segmentation: off [requested on]
root@ESPRIMO-P956:~#
root@ESPRIMO-P956:~# ethtool -k enp2s0f0 |grep -i check
Cannot get device udp-fragmentation-offload settings: Operation not supported
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: off
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: on [fixed]
tx-checksum-sctp: off
root@ESPRIMO-P956:~# ethtool -K enp2s0f0 tx-checksum-fcoe-crc off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
Could not change any device features
root@ESPRIMO-P956:~#
root@ESPRIMO-P956:~# ethtool -K enp2s0f0 rx-udp_tunnel-port-offload off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
root@ESPRIMO-P956:~#
#+END_CENTER
same for 2nd nic:
#+BEGIN_CENTER
root@ESPRIMO-P956:~# ethtool -K enp2s0f1 rx-udp_tunnel-port-offload off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
root@ESPRIMO-P956:~# ethtool -K enp2s0f1 rx off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
root@ESPRIMO-P956:~# ethtool -K enp2s0f1 tx off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
Actual changes:
tx-checksumming: off
tx-checksum-ip-generic: off
tx-checksum-sctp: off
tcp-segmentation-offload: off
tx-tcp-segmentation: off [requested on]
tx-tcp6-segmentation: off [requested on]
root@ESPRIMO-P956:~#
#+END_CENTER
See also:
https://www.kernel.org/doc/ols/2009/ols2009-pages-169-184.pdf
#+BEGIN_CENTER
root@ESPRIMO-P956:~# ethtool -K enp2s0f1 gso off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
root@ESPRIMO-P956:~# ethtool -K enp2s0f0 gso off
Cannot get device udp-fragmentation-offload settings: Operation not supported
Cannot get device udp-fragmentation-offload settings: Operation not supported
root@ESPRIMO-P956:~#
#+END_CENTER
*** TODO 2019-08-04: post tuning test
v4 side:
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/bin$ socat - UDP:10.0.0.66:2345
short line from v4
looooooooooooooooooooooooooooooooonger line from v4
looooooooooooooooooooooooooooooooongerlooooooooooooooooooooooooooooooooongerlooooooooooooooooooooooooooooooooonger line from v4
from v4
loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong
#+END_CENTER
v6 side:
#+BEGIN_CENTER
nico@ESPRIMO-P956:~/master-thesis/bin$ socat - UDP6-LISTEN:2345
short line from v4
looooooooooooooooooooooooooooooooonger line from v4
looooooooooooooooooooooooooooooooongerlooooooooooooooooooooooooooooooooongerlooooooooooooooooooooooooooooooooonger line from v4
from v6 loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong
from v4
#+END_CENTER
*** DONE 2019-08-04: small repeating udp: works
CLOSED: [2019-08-04 Sun 09:58]
#+BEGIN_CENTER
^Cnico@ESPRIMO-P956:~/master-thesis/bin$ (while true; do echo v4: not long ok; sleep 1; done) | socat - UDP:10.0.0.66:2345
nico@ESPRIMO-P956:~/master-thesis/bin$ (while true; do echo v4: not long ok; sleep 1; done) | socat - UDP:10.0.0.66:2345
^Cnico@ESPRIMO-P956:~/master-thesis/bin$ (while true; do echo v6: not long ok too; sleep 1; done) | socat - UDP6-LISTEN:2345
nico@ESPRIMO-P956:~/master-thesis/bin$ (while true; do echo v4: not long ok; sleep 1; done) | socat - UDP:10.0.0.66:2345
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
v6: not long ok too
^C
nico@ESPRIMO-P956:~/master-thesis/bin$
^Cnico@ESPRIMO-P956:~/master-thesis/bin$ (while true; do echo v6: not long ok too; sleep 1; done) | socat - UDP6-LISTEN:2345
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
v4: not long ok
^C
nico@ESPRIMO-P956:~/master-thesis/bin$
#+END_CENTER
Trace:
create mode 100644 pcap/netfpga-nat64-udp-small-repeat-2019-08-04-0954-enp2s0f0.pcap
create mode 100644 pcap/netfpga-nat64-udp-small-repeat-2019-08-04-0954-enp2s0f1.pcap
*** TODO 2019-08-04: problem with longer: ethernet frame check sequence broken: 0x0000000
Points towards incorrect length:
ipv4 sending:
total length = 36, udp length = 16
50 bytes on wire
ipv6 receiving
80 byte on wire
working frames:
v4: 61 byte, v6 81 byte : +20
v4 total length: 47 diff 14
v6 payload length: 27 diff 54
v4: 94 byte, v6: 114 byte: +20
v4: 170 byte, v6: 190 byte: +20
frame 4 udp checksum v4 broken, coming FROM v6:
v6: 574 byte, v4: 554 byte
v6 payload legnth 520
v4 total length 540
v4 id is 0
udp data part 512 bytes in both cases
checksum --- v4 checksum correct -> is that one included in the UDP
checksum??? ==> order might matter!
frame 5: !!!! 30 bytes difference! => 10 zero bytes too much at the end!!
v4 50 bytes, v6 80 bytes
v4 total length: 36 diff 14
v6 payload length: 16 diff 54
10 bytes too much received in ipv6: exactly 10 0 bytes at the end
-> explains frame check sequence error
frame 6:
v4 546 bytes, v6 566 bytes: 20 byte difference, udp checksum broken
*** TODO 2019-08-04: compare netfpga behaviour with bmv2
***
** The NetPFGA saga
Problems encountered:
- The logfile for a compile run is 10k+ lines
- Many logged errors can actually be ignored (?) like:
ERROR: [VRFC 10-1491] unexpected EOF [/home/nico/master-thesis/netpfga/minip4/nf_sume_sdnet_ip/SimpleSumeSwitch/S_CONTROLLERs.HDL/S_CONTROLLER_SimpleSumeSwitch.vp:37]
ERROR: [VRFC 10-426] cannot find port tuple_out_sume_metadata_DATA on this module [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/wrapper/nf_sume_sdnet.v:219]
ERROR: [VRFC 10-426] cannot find port tuple_out_sume_metadata_VALID on this module [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/wrapper/nf_sume_sdnet.v:218]
ERROR: [VRFC 10-426] cannot find port tuple_in_sume_metadata_DATA on this module [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/wrapper/nf_sume_sdnet.v:185]
ERROR: [VRFC 10-426] cannot find port tuple_in_sume_metadata_VALID on this module [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/wrapper/nf_sume_sdnet.v:184]
ERROR: [VRFC 10-2063] Module <S_RESETTER_line> not found while processing module instance <S_RESET_clk_line> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/Simp
leSumeSwitch/SimpleSumeSwitch.v:332]
ERROR: [VRFC 10-2063] Module <S_RESETTER_lookup> not found while processing module instance <S_RESET_clk_lookup> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/
SimpleSumeSwitch/SimpleSumeSwitch.v:343]
ERROR: [VRFC 10-2063] Module <S_RESETTER_control> not found while processing module instance <S_RESET_clk_control> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_i
p/SimpleSumeSwitch/SimpleSumeSwitch.v:354]
ERROR: [VRFC 10-2063] Module <TopParser_t> not found while processing module instance <TopParser> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/SimpleSumeSwitc
h/SimpleSumeSwitch.v:436]
ERROR: [VRFC 10-2063] Module <TopPipe_lvl_t> not found while processing module instance <TopPipe_lvl> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/SimpleSumeS
witch/SimpleSumeSwitch.v:474]
ERROR: [VRFC 10-2063] Module <dummy_table_for_netpfga_t> not found while processing module instance <dummy_table_for_netpfga> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_s
ume_sdnet_ip/SimpleSumeSwitch/SimpleSumeSwitch.v:502]
ERROR: [VRFC 10-2063] Module <TopPipe_lvl_0_t> not found while processing module instance <TopPipe_lvl_0> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/SimpleS
umeSwitch/SimpleSumeSwitch.v:533]
ERROR: [VRFC 10-2063] Module <TopDeparser_t> not found while processing module instance <TopDeparser> [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/nf_sume_sdnet_ip/nf_sume_sdnet_ip/SimpleSumeS
witch/SimpleSumeSwitch.v:561]
# launch_simulation -simset sim_1 -mode behavioral
INFO: [Vivado 12-5698] Checking validity of IPs in the design for the 'XSim' simulator...
CRITICAL WARNING: [BD 41-1356] Address block </M04_AXI/Reg> is not mapped into </S00_AXI>. Please use Address Editor to either map or exclude it.
CRITICAL WARNING: [BD 41-1356] Address block </M05_AXI/Reg> is not mapped into </S00_AXI>. Please use Address Editor to either map or exclude it.
WARNING: [VRFC 10-756] identifier state is used before its declaration [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/axis_sim_record_ip0/hdl/axis_sim_record.v:93]
WARNING: [VRFC 10-756] identifier ready_count is used before its declaration [/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.srcs/sources_1/ip/axis_sim_record_ip0/hdl/axis_sim_record.v:94]
INFO: [#UNDEF] Sorry, too many errors..
ERROR: [XSIM 43-3322] Static elaboration of top level Verilog design unit(s) in library work failed.
INFO: [USF-XSim-69] 'elaborate' step finished in '1' seconds
INFO: [USF-XSim-99] Step results log file:'/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log'
ERROR: [USF-XSim-62] 'elaborate' step failed with error(s). Please check the Tcl console output or '/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log' file for more information.
nico@nsg-System:~/master-thesis$ find . -name elaborate.log
nico@nsg-System:~/master-thesis$ find ~ -name elaborate.log
nico@nsg-System:~/master-thesis$
- Scripts that "fail" (generate wrong data) do exit 0 ->
There is no easy / reliable error detection
- Writing tables resulted in ioctl errors
- Hardware test: unclear if first board was/is broken or not,
BUT: second board in different computer allows writing tables
- Many scripts depend on each other in later stages, without clear
dependencies
- There is basically no documentation for someone who "just wants to
compile from P4 to netpfga" or A LOT of documentation (if vivado,
vhld, sdnet documentation is counted)
- Very high complexity in toolchain, scripts that are generated
+ cd /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/simple_sume_switch/test/sim_switch_default
+ make
rm -f config_writes.py*
rm -f *.pyc
nico@nsg-System:~$ cat /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/testdata/config_writes.py
from NFTest import *
NUM_WRITES = 4
def config_tables():
nftest_regwrite(0x44020050, 0x22222208)
nftest_regwrite(0x44020054, 0x00000822)
nftest_regwrite(0x44020080, 0x00000201)
nftest_regwrite(0x44020040, 0x00000001)
nico@nsg-System:~$ cat /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/testdata/config_writes.sh
#!/bin/bash
${SUME_SDNET}/sw/sume/rwaxi -a 0x44020050 -w 0x22222208
${SUME_SDNET}/sw/sume/rwaxi -a 0x44020054 -w 0x00000822
${SUME_SDNET}/sw/sume/rwaxi -a 0x44020080 -w 0x00000201
${SUME_SDNET}/sw/sume/rwaxi -a 0x44020040 -w 0x00000001
nico@nsg-System:~$
- Misleading errors like
ERROR: [USF-XSim-62] 'elaborate' step failed with error(s). Please check the Tcl console output or '/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log' file for more information.
nico@nsg-System:~/master-thesis/netpfga$ ls /home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log
ls: cannot access '/home/nico/master-thesis/netpfga/minip4/simple_sume_switch/hw/project/simple_sume_switch.sim/sim_1/behav/xsim/elaborate.log': No such file or directory
- not using raise() and hiding source of errors (_hexify)
- sometimes flashing fails:
#+BEGIN_CENTER
nico@nsg-System:~/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/simple_sume_switch/bitfiles$ sudo bash -c ". $HOME/master-thesis/netpfga/bashinit && $(pwd -P)/program_switch.sh"
++ which vivado
+ xilinx_tool_path=/opt/Xilinx/Vivado/2018.2/bin/vivado
+ bitimage=minip4.bit
+ configWrites=config_writes.sh
+ '[' -z minip4.bit ']'
+ '[' -z config_writes.sh ']'
+ '[' /opt/Xilinx/Vivado/2018.2/bin/vivado == '' ']'
+ rmmod sume_riffa
+ xsct /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/tools/run_xsct.tcl -tclargs minip4.bit
rlwrap: warning: your $TERM is 'screen' but rlwrap couldn't find it in the terminfo database. Expect some problems.
RUN loading image file.
minip4.bit
100% 19MB 1.7MB/s 00:11
fpga configuration failed. DONE PIN is not HIGH
invoked from within
"::tcf::eval -progress ::xsdb::print_progress {::tcf::cache_enter tcfchan#0 {tcf_cache_eval {process_tcf_actions_cache_client ::tcfclient#0::arg}}}"
(procedure "::tcf::cache_eval_with_progress" line 2)
invoked from within
"::tcf::cache_eval_with_progress [dict get $arg chan] [list process_tcf_actions_cache_client $argvar] $progress"
(procedure "process_tcf_actions" line 1)
invoked from within
"process_tcf_actions $arg ::xsdb::print_progress"
(procedure "fpga" line 430)
invoked from within
"fpga -f $bitimage"
(file "/home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/tools/run_xsct.tcl" line 33)
+ bash /home/nico/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/tools/pci_rescan_run.sh
Check programming FPGA or Reboot machine !
+ rmmod sume_riffa
rmmod: ERROR: Module sume_riffa is not currently loaded
+ modprobe sume_riffa
+ ifconfig nf0 up
nf0: ERROR while getting interface flags: No such device
+ ifconfig nf1 up
nf1: ERROR while getting interface flags: No such device
+ ifconfig nf2 up
nf2: ERROR while getting interface flags: No such device
+ ifconfig nf3 up
nf3: ERROR while getting interface flags: No such device
+ bash config_writes.sh
nico@nsg-System:~/projects/P4-NetFPGA/contrib-projects/sume-sdnet-switch/projects/minip4/simple_sume_switch/bitfiles$
#+END_CENTER
** References / Follow up
*** RFC 791 IPv4 https://tools.ietf.org/html/rfc791
*** RFC 792 ICMP https://tools.ietf.org/html/rfc792

Binary file not shown.

Binary file not shown.