Eduroam failed three times due to an IDS being overloaded with all the icmp states.
Originally, an optical tap was proposed, but was replaced with a hardware tap using a p4-based intel switch.
Do not get the cheapest 40gbit nic. A lot of hardware is specced in terms of gigabits, but we mostly care about the amount of Mpps it can handle.
If lasert is involved, make sure that you are able to handle 100g across the pipe.
Make sure to reboot a few days before the event starts to ensure that everything is persistent.
Video is a mess, x264 has lossless profiles, but those are not supported in many browsers. The AV1 and vp9 implementations in gstreamer will also give you trouble and unknown browser support. Look into transcoding multiple versions depending on support.
Look into doing hardware encoding using on-chip or discrete graphics.
DPDK-gpudev looks very interesting if you want to go completely overkill and do all processing directly on a GPU.
Being able to restart the server while keeping the video stream alive is very nice to have.
MediaMTX is quite nice, but make sure to increase the buffer size
You will receive a lot of shit. Some common examples:
* Checksums of zero
* A lot of zero-alpha pixels
* Bogon source ranges
* All sorts of non-conforming ICMP traffic.
Make sure to test the blacklist beforehand
Make sure you have push access to all the repositories beforehand. Others won't like a new repo being created every time, but waiting days for a PR to come through is also a no-go.
Locks are the devil, pre-allocate where possible, but also think about cases where your preallocated space runs out.
DPDK has a lot of nice libraries for high performance packet processning and firewalling, but nothing can beat hardcoding.
A lot of people will take part. You should email the technical associations and their tech committees, but half of the participants will come from non-current-students. In 2024 we had about 100 different groups partaking, Easily reaching the upstream 50Mpps limit.
CFM, SU are hard to deal with. Don’t count on getting a screen from them. If you somehow do, make sure your moderation standards are relatively high.
Keep a way to test outside of prod. uio_pci_generic is nice, but some linux socket based solution would be nice to have.
Make sure others can read and understand your code and how to compile and run it. You will not be the only person looking at it anymore.
Silke has a golang-based version that can handle about 2Mpps per core. If you can find a way to improve the performance that should be a betteer choice than c++, but we were not able to.
If picking a socket/kernel-based approach, make sure to disable flow control and icmp replies.
Look into replacing gstreamer with ffmpeg. In theory gstreamer is better for streaming from an mbuf, buf we ran into quite a few badly documented cases.
Having statistics per source ip/block would be very nice. Make sure these are tracked separately per-core.
Do not pre-allocate too much. Linux’s virtual memory helps, but ideally you dont preallocate more than you have.
You want to be able to blacklist more or less than a /64
Make sure your program can deal with people sending from more than a single /64.
Auto restarts on the pings server are very nice to have for your sleeping rhytm.
LISA/ITO is your friend. They love the project, but keep them updated and give them time so they can prepare.
You will hear various names for the concept in this project: SNTPings, JinglePings, PixelFlut.