User Tools

Site Tools


harden_tcp:ip_services_against_syn_flood_attacks

The following cookbook will harden the IP stack for TCPIP Services for OpenVMS V5.3 and above against SYN FLOOD attacks. This can be useful if you are still exposing an OpenVMS web server to the outside world. These updates override the cautious HPE distributed defaults. The default was based on “small memory” system configurations. I'm running 1G or above in my installed base so I'm not worried about memory depletion from this specific change.

$!
$! Before
$!
$SEA SYS$SPECIFIC:[TCPIP$ETC]sysconfigtab.dat ":","="
$CREATE temp_table_mgr.stanza
$DECK
socket:
#
# To display current "socket" values:
#
# tcpip sysconfig -q socket
#
# HP TCP/IP Services for OpenVMS
# Tuning and Troubleshooting
# AA-RN1VB-TE
# Section 2.1.5.1 and 2.3.1
#
# Harden IP stack against SYN FLOOD attacks
#
somaxconn=65535
sominconn=65535
$EOD
$SET PROC/PRIV=ALL
$TCPIP SYSCONFIGDB -a -f temp_table_mgr.stanza SOCKET
$DEL/NOLOG temp_table_mgr.stanza;
$!
$! after
$!
$ECHO "++++++++++++++++++++++++++++++++++"
$ECHO ""
$SEA SYS$SPECIFIC:[TCPIP$ETC]sysconfigtab.dat ":","="

Recycle of TCPIP Services to bring change into effect required. Overall server node reboot often is the “simple” way to do that since other services tend to be layered on top of TCPIP Services.

I did a before/after test probing my test server running the OSU web server using a SYN FLOOD attack generated from Hyenae. The difference was like night and day.

For folks interested in this topic, see RFC 4987 for the common mitigations for a TCP SYN flood attack.

https://tools.ietf.org/html/rfc4987

And some introductory fodder on this particular mess:

https://www.cisco.com/c/en/us/about/press/internet-protocol-journal/back-issues/table-contents-34/syn-flooding-attacks.html

somaxconn=65535
sominconn=65535

The above increases the number of available sockets.

Which means the server will now have more sockets available to flood.

If that increase in the number of sockets has mitigated the effects of the SYN flood in this case, then that mitigation is likely due to some other factor. Maybe insufficient bandwidth in the host(s) or app(s) or network attempting the SYN flood, for instance.

I'd wonder if the Hyenae SYN flood packet generator or its host was running out of capacity, or running into a preconfigured limit.

I don't know if TCP/IP Services is coded to detect and drop SYN floods from a fixed source address. I'd tend to doubt it, though. That'll mitigate a simple SYN flood. Firewalls can provide this, if TCP/IP Services does not.

Either in addition to or in place of increasing the sockets — though pragmatically, there probably aren't all that many good reasons not to run with 64K sockets configured in this era — I'd suggest having a look at the tcp_keepinit setting, and specifically at reducing that value. That's one of the mitigations listed in the RFC, and it's something that means that flooding gets cleaned up more quickly. Downside is that the DDoS only gets bigger with a shorter timeout or with more sockets or both, and the server still gets flooded. This as discussed in the RFC.

If the services are targets for spoofed SYN floods or other DDoS shenanigans, then the configuration is probably headed for CloudFlare or other services, as I'd expect that increasing the number of sockets available on OpenVMS and shortening the tcp_keepinit will just consume more resources on the server and not markedly harden the server against a TCP SYN flood.

Receiving a hundred gigabytes of DDoS traffic — TCP SYN or otherwise — was possible five years ago, and more recently there've been various folks hit with terabyte-scale DDoS. Largest DDoS I'm aware of was 1.7 terabytes per second. I'm not aware of any OpenVMS server that has sufficient network controllers and capacity to even support that volume of traffic, all other factors and considerations aside.

I did a before/after test probing my test server running the OSU web server using a SYN FLOOD attack generated from Hyenae. The difference was like night and day.

Haven't tried Hyenae; will have a look at that one. That tool seems to have been forked to JHyenae, as well. hping is another common tool for that sort of load testing.

The following part of the quoted posting was reordered…

This can be useful if you are still exposing an OpenVMS web server to the outside world.

A directly-exposed OpenVMS server is often not a particularly auspicious configuration. I've long preferred OpeNVMS configured behind a firewall and preferably one with VPN and quite possibly one with appropriate filtering, and with additional capabilities for detecting the common sorts of IP shenanigans. If not additionally configured behind a DDoS prevention service, for that matter.

The above increases the number of available sockets.

It is the queue size/length to a specific socket, not? And the mitigation for SYN FLOOD is combined with lowering the timeout for packages in the queue before they are deleted. I guess the thought is to make it harder to reach the max number of queued SYN packages (connects).

What I do not fully understand is if that timeout functionality can detect “bad” packages from “good”. If not, I guess that good packages can also be deleted. Now, if not some firewall or similar detects a large volume of SYNs from a specific address and then shuts it down.

But OK, you get a more headroom before that specific service dies.

Which means the server will now have more sockets available to flood.

I think it will have larger queues for the available sockets. Not more sockets.

Some related info:

http://support.sas.com/kb/11/479.html

http://h41379.www4.hpe.com/doc/732final/6631/6631pro_005.html (2.1.5.1)

https://docs.oracle.com/cd/B12037_01/server.101/q20201/misc/perf-dec.html

http://www.frascati.enea.it/documentation/tru6450/ARH9GATE/CHNTXXXX.HTM

https://ftp.unpad.ac.id/orari/library/library-ref-eng/ref-eng-1/network/network-security/avoidtcpsynattack.txt

Test performed:

HP 600G2Mini Gigabit link W10 towards RX2600 gigabit link OVMS8.4 TCPIP 5.7 ECO5+

hyenae -I 1 -A 4 -a tcp -s %-%@80 -d (reacted MAC address)(redacted IP address)@80 -f S -t 128 -k 0 -w 0 -q 0 -Q 1

Ran HyenaeFE for 3+ minutes. 1767280 packets sent (95433120 bytes) in 203.467 seconds (That's 1.7 million flood packets)

Normal web performance

I wouldn't see that much traffic from an external site thru my ISP link so I'm satisfied it is a solid test. Johnny Billquist bqt@softjar.se: Aug 14 10:14PM +0200

On 2018-08-14 17:42, Stephen Hoffman wrote:

pragmatically, there probably aren't all that many good reasons not to
run with 64K sockets configured in this era — I'd suggest having a look
at the tcp_keepinit setting, and specifically at reducing that value.

Why the 64K limit? Why not bump it to a few million in that case?

» at the tcp_keepinit setting, and specifically at reducing that value.

> Why the 64K limit? Why not bump it to a few million in that case?

> Johnny

You cannot (or should not) set it higher according to the docs or unexpected “things” might happen. It is a built-in limit.

Or did you ask why that limit is where it is? Don't know… Johnny Billquist bqt@softjar.se: Aug 14 10:45PM +0200

On 2018-08-14 22:31, Jan-Erik Söderholm wrote:

> You cannot (or should not) set it higher according to the docs

or unexpected “things” might happen. It is a built-in limit.

> Or did you ask why that limit is where it is? Don't know…

Well, I guess the question was vague, but your explanation at least clarifies that there is some kind of limit at 64K in the system here. I could be interesting to learn why there is a limit there, but that would really just be to feed my curiosity.

harden_tcp/ip_services_against_syn_flood_attacks.txt · Last modified: 2021/10/25 18:48 by williams

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki