Copyright © 2009, 2010 Thomas M. Eastep
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover, and with no Back-Cover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License”.
2020/04/17
Table of Contents
Debian Lenny includes Shorewall version 4.0.15 while Squeeze includes Shorewall 4.4. Because there are significant differences between the two product versions, some users may experience upgrade issues. This article outlines those issues and offers advice for dealing with them.
Note
Although this article is targeted specifically at Lenny -> Squeeze upgrades, it should be useful to any Shorewall-shell user upgrading to Shorewall 4.4.x. Footnotes are used to flag areas where non-Debian users may experience different results.
The first key difference between Shorewall 4.0 and Shorewall 4.4 is in the packaging[1]. In Lenny, there are six Shorewall packages:
shorewall-common — Contains the basic components needed to create an IPv4 firewall.
shorewall-shell — The legacy Shorewall configuration compiler written in Bourne shell.
shorewall — A transitional package that depends on shorewall-common and shorewall-shell. Installing this package installs both shorewall-common and shorewall-shell.
shorewall-perl — A re-implementation of the Shorewall configuration compiler in Perl. This compiler has many advantages over the shell-based compiler:
The compiler is much faster
The compiler does a much better job of validating the configuration, thus avoiding run-time errors.
The compiler produces better and more consistent diagnostic messages.
The compiler produces a script that runs much faster and that does not reject/drop connections during start/restart.
shorewall-lite — A small package that can run scripts generated by shorewall-shell or shorewall-perl. Allows centralized firewall administration.
shorewall-doc — Documentation.
In Squeeze, there are six slightly different packages:
shorewall — Contains everything needed to create an IPv4 firewall. It combines the former shorewall-common and shorewall-perl packages.
shorewall6 — Depends on shorewall. Adds those components needed to create an IPv6 firewall.
shorewall-lite — Same as in Lenny; only runs IPv4 firewall scripts.
shorewall6-lite — Similar to shorewall-lite, except that it only runs IPv6 firewall scripts.
shorewall-init — Allows the firewall to be closed before interfaces are brought up and also allows the firewall to react to interfaces coming up and going down.
shorewall-doc — Documentation.
Warning
Do not purge the old packages (shorewall-common, shorewall-shell and shorewall-perl) until after the new shorewall package has been installed.
The key change in Squeeze that may produce upgrade issues is that Squeeze does not include the shell-based configuration compiler. As a consequence, unless you are already using Shorewall-perl on Lenny, an upgrade from Lenny to Squeeze will mean that you will be switching from the old shell-based compiler to the new Perl-based compiler[2]. While the two compilers are highly compatible, there are some differences. Those differences are detailed in the following sections.
As always, when upgrading from one major release of Shorewall to
another, the installer will prompt you about replacing your existing
shorewall.conf
with the updated one from the
package. Shorewall is designed with the assumption that users will never
replace shorewall.conf and retaining your existing file will always
produce upward-compatible behavior.
That having been said, there are a few settings that you may have in your shorewall.conf that will cause compilation warning or error messages after the upgrade.
- BLACKLISTNEWONLY
If you have BLACKLISTNEWONLY=No together with FASTACCEPT=Yes, you will receive this error:
ERROR: BLACKLISTNEWONLY=No may not be specified with FASTACCEPT=Yes
To eliminate the error, reverse the setting of one of the options.
Note
This combination never worked correctly in earlier versions -- to duplicate the earlier behavior, you will want to set BLACKLISTNEWONLY=Yes.
- BRIDGING
If you have set this option to Yes, you will receive the following error:
ERROR: BRIDGING=Yes is not supported by Shorewall 4.4.x
You should not be receiving this error if you are upgrading from Lenny since BRIDGING=Yes did not work in that release either[3]. If you have a bridge configuration where you want to control connections through the bridge, you will want to visit https://shorewall.org/bridge-Shorewall-perl.html[4].
- DELAYBLACKLISTLOAD
If you have set this option to Yes, you will receive the following warning:
WARNING: DELAYBLACKLIST=Yes is not supported by Shorewall 4.4.x
To eliminate the warning, set DELAYBLACKLISTLOAD=No or remove the setting altogether.
- DYNAMIC_ZONES
If you have set this option to Yes, you will receive the following warning:
WARNING: DYNAMIC_ZONES=Yes is not supported by Shorewall 4.4.x
To eliminate the warning, set DYNAMIC_ZONES=No or remove the setting altogether. See this article to learn how to set up Dynamic Zones under Shorewall 4.4.
- FW
If a setting for FW appears in your shorewall.conf file, you will receive this warning:
WARNING: Unknown configuration option (FW) ignored.
Remove the setting from the file and modify your
/etc/shorewall/zones
file as described below.- IPSECFILE
If you have specified IPSECFILE=ipsec or IPSECFILE= or if you do not have a setting for IPSECFILE, then you will receive the following error:
ERROR: IPSECFILE=ipsec is not supported by Shorewall 4.4.x
To eliminate the warning, you will need to:
Set IPSECFILE=zones
Modify your
/etc/shorewall/zones
file as described below.
- PKTTYPE
The PKTTYPE option is ignored by Shorewall-perl. Shorewall-perl will use Address type match if it is available; otherwise, it will behave as if PKTTYPE=No had been specified.
- RFC1918_LOG_LEVEL
If you have specified any setting for this option, you will receive the following warning:
WARNING: RFC1918_LOG_LEVEL=value ignored. The 'norfc1918' interface/host option is no longer supported.
To eliminate the warning, set RFC1918_LOG_LEVEL= or simply remove the setting altogether.
- RFC1918_STRICT
If you have set this option to Yes, you will receive the following warning:
WARNING: RFC1918_STRICT=Yes is not supported by Shorewall 4.4.x
To eliminate the warning, set RFC1918_STRICT=No or remove the setting altogether.
- SAVE_IPSETS
Shorewall 4.4.0-4.4.5 will issue a warning if you set SAVE_IPSETS=Yes in
shorewall.conf
:WARNING SAVE_IPSETS=Yes is not supported by Shorewall 4.4.x
To eliminate this message, you will need to set SAVE_IPSETS=No or remove the setting altogether.
See below for additional information regarding ipsets in Shorewall 4.4.
- SHOREWALL_COMPILER
If you have specified SHOREWALL_COMPILER=shell, you will receive the following warning message:
WARNING: SHOREWALL_COMPILER=shell ignored. Shorewall-shell support has been removed in this release
To eliminate the warning, set SHOREWALL_COMPILER=perl or simply remove the setting altogether.
- USE_ACTIONS
If you have set this option to No, you will receive the following warning:
WARNING: USE_ACTIONS=No is not supported by Shorewall 4.4.x
To eliminate the warning, set USE_ACTIONS=Yes or remove the setting altogether.
If the column headings in your /etc/shorewall/zones file look like this:
#ZONE DISPLAY COMMENTS net Net The big bad net loc Local The local LAN
then you are using the original zones file format that has been deprecated since Shorewall 3.0.
You will need to convert to the new file format which has the following headings:
#ZONE TYPE OPTIONS IN OUT # OPTIONS OPTIONS
You will need to add an entry for your firewall zone. The default
name for the firewall zone is 'fw' but may have been overriden in your
old configuration using the FW option in
shorewall.conf
.
#ZONE TYPE OPTIONS IN OUT # OPTIONS OPTIONS fw firewall
The remainder of your zones will have type 'ipv4' unless they are mentioned in your /etc/shorewall/ipsec file (see below).
#ZONE TYPE OPTIONS IN OUT # OPTIONS OPTIONS fw firewall net ipv4 # The big bad net loc ipv4 # The local LAN
This file is no longer used -- its specifications are now included
in /etc/shorewall/zones
.
Take this example:
#ZONE IPSEC OPTIONS IN OUT # ONLY OPTIONS OPTIONS ipsec1 Yes ipsec2 No
This would translate to the following entries in
/etc/shorewall/zones
:
#ZONE TYPE OPTIONS IN OUT # OPTIONS OPTIONS ipsec1 ipsec4 ipsec2 ipv4
Any OPTIONS, IN OPTIONS and OUT OPTIONS should simply be copied
from /etc/shorewall/ipsec
to
/etc/shorewall/zones
.
The BROADCAST column is essentially unused in Squeeze. If it contains anything except 'detect' or '-', then you will receive this warning[5]:
WARNING: Shorewall no longer uses broadcast addresses in rule generation when Address Type Match is available
To eliminate the warning, replace the contents of the BROADCAST column with '-' or 'detect'.
The 'norfc1918' option has been removed. If you specify the option, you will receive the following warning:
WARNING: Support for the norfc1918 interface option has been removed from Shorewall
To eliminate the warning, simply remove the 'norfc1918' option from the OPTIONS list. You may wish to consider NULL_ROUTE_RFC1918=Yes as a replacement (see shorewall.conf (5)).
The 'norfc1918' option has been removed. If you specify the option, you will receive the following warning:
WARNING: The 'norfc1918' option is no longer supported
To eliminate the warning, simply remove the 'norfc1918' option from the OPTIONS list. You may wish to consider NULL_ROUTE_RFC1918=Yes as a replacement (see shorewall.conf (5)).
Shorewall 4.4 detects dead policy file entries that result when an entry is masked by an earlier more general entry.
Example:
#SOURCE DEST POLICY LOG LEVEL all all REJECT info loc net ACCEPT
Shorewall-shell silently accepted the above even though the loc->net policy is useless. Shorewall-perl generates a fatal compilation error:
ERROR: Policy "loc net ACCEPT" duplicates earlier policy "all all REJECT"
There is a long tradition of specifying an interface name in the SOURCE column of this file.
Masquerading/SNAT occurs in the Netfilter POSTROUTING chain where an incoming interface may not be specified in iptables rules. Consequently, while processing the shorewall start and shorewall restart commands, the generated script must examine the firewall's main routing table to determine those networks that are routed out of the interface; the script then adds a MASQUERADE/SNAT rule for connections from each of those networks. This additional processing requires the named interface to be up and configured when Shorewall starts or restarts.
Users often complain that Shorewall fails to start at boot time because a VPN interface that is named as a masq SOURCE isn't up and configured during boot.
To emphasize this restriction, if an interface is named in the SOURCE column of one or more entries, a single warning is issued as follows:
WARNING: Using an interface as the masq SOURCE requires the interface to be up and configured when Shorewall starts/restarts
To suppress this warning, replace the interface name with the list of networks that are routed out of the interface.
Example.
Existing entry:
#INTERFACE SOURCE ADDRESS PROTO PORT(S) IPSEC MARK USER/ # GROUP eth0 eth1
Current routing configuration:
gateway:~# ip route ls dev eth1
172.20.1.0/24 proto kernel scope link src 172.20.1.254
224.0.0.0/4 scope link
gateway:~#
Replacement entry:
#INTERFACE SOURCE ADDRESS PROTO PORT(S) IPSEC MARK USER/
# GROUP
eth0 172.20.1.0/24
Note that no entry is included for 224.0.0.0/4 since that is the multicast IP range and there should never be any packets with a SOURCE IP address in that network.
If you include a destination zone in a 'nonat' rule, Shorewall issues the following warning:
WARNING: Destination zone (zonename) ignored.
Nonat rules include:
DNAT- REDIRECT- NONAT
To eliminate the warning, remove the DEST zone.
Example.
Before:
#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK CONNLIMIT TIME # PORT(S) PORT(S) DEST LIMIT GROUP NONAT loc net tcp 80
After:
#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK CONNLIMIT TIME # PORT(S) PORT(S) DEST LIMIT GROUP NONAT loc - tcp 80
Shorewall 4.4 versions prior to 4.4.19 do not support icmp type lists in the DEST PORT(S) column. Only a single ICMP type may be listed. If you have a shell variable with a list of ICMP types that you use in a rule, you can work around this limitation as follows. Replace this rule:
#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK CONNLIMIT TIME # PORT(S) PORT(S) DEST LIMIT GROUP ACCEPT z1 z2 icmp $ITYPES
with:
#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE USER/ MARK CONNLIMIT TIME # PORT(S) PORT(S) DEST LIMIT GROUP BEGIN SHELL for type in $ITYPES; do ACCEPT z1 z2 icmp $type done END SHELL
The 'critical' option is no longer needed and hence is no longer supported. If you have critical hosts defined, you will receive this warning:
WARNING: The 'critical' option is no longer supported (or needed)
To suppress the warning, simply remove the option.
Shorewall 4.4 also treats the routestopped
file differently from earlier releases. Previously, the
routestopped
file was parsed during
shorewall stop processing so that changes made to the
file while Shorewall was running would be applied at the next
stop. This is no longer the case -- the
routestopped
file is processed during compilation
just like the rest of the configuration files so that when
shorewall stop is issued, the firewall will pass
traffic based on the contents of the routestopped
file at the last start or
restart.
If you change the routestopped
file and now
want to stop the firewall, you can run this sequence of commands:
shorewall compile
shorewall stop
The /etc/shorewall/tos
file now has
zone-independent SOURCE and DEST columns as do all other files except
the rules and policy files.
The SOURCE column may be one of the following:
[all:]<address >[,...] |
[all:]<interface >[:<address >[,...]] |
$FW[:<address >[,...]] |
The DEST column may be one of the following:
[all:]<address >[,...] |
[all:]<interface >[:<address >[,...]] |
This is a permanent change. The old zone-based rules have never
worked right and this is a good time to replace them. We have tried to
make the new syntax cover the most common cases without requiring change
to existing files. In particular, it will handle the
tos
file released with Shorewall 1.4 and
earlier.
With the shell-based compiler, all extension scripts were copied into the compiled script and executed at run-time. In some cases, this approach doesn't work with Shorewall Perl because (almost) the entire rule set is built by the compiler. As a result, Shorewall-perl runs some extension scripts at compile-time rather than at run-time. Because the compiler is written in Perl, these extension scripts from earlier versions will no longer work.
The following table summarizes when the various extension scripts are run:
Compile-time (Must be written in Perl) | Run-time | Eliminated |
initdone | clear | continue |
maclog | init | |
Per-chain (including those associated with actions) | start | |
started | ||
stop | ||
stopped | ||
tcclear |
Compile-time extension scripts are executed using the Perl 'eval `cat <file>`' mechanism. Be sure that each script returns a 'true' value; otherwise, the Shorewall-perl compiler will assume that the script failed and will abort the compilation.
When a script is invoked, the $chainref scalar variable will usually hold a reference to a chain table entry.
$chainref->{name} contains the name of the chain |
$chainref->{table} holds the table name |
To add a rule to the chain:
add_rule $chainref,
the-rule |
Where
the rule is a scalar argument
holding the rule text. Do not include "-A
chain-name " |
Example:
add_rule $chainref, '-j ACCEPT'; |
To insert a rule into the chain:
insert_rule $chainref, rulenum ,
the-rule |
The log_rule_limit function works like it does in the shell compiler with three exceptions:
You pass the chain reference rather than the name of the chain.
The commands are 'add' and 'insert' rather than '-A' and '-I'.
There is only a single "pass as-is to iptables" argument (so you must quote that part
Example:
log_rule_limit 'info' , $chainref , $chainref->{name}, 'DROP' , '', #Limit '' , #Log tag 'add' '-p tcp ';
Here is an example of an actual initdone script used with Shorewall 3.4:
run_iptables -t mangle -I PREROUTING -p esp -j MARK --set-mark 0x50 run_iptables -t filter -I INPUT -p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT run_iptables -t filter -I OUTPUT -p udp --sport 1701 -j ACCEPT
Here is the corresponding script used with Shorewall 4.4:
use Shorewall::Chains; insert_rule $mangle_table->{PREROUTING}, 1, "-p esp -j MARK --set-mark 0x50"; insert_rule $filter_table->{INPUT}, 1, "-p udp --dport 1701 -m mark --mark 0x50 -j ACCEPT"; insert_rule $filter_table->{OUTPUT}, 1, "-p udp --sport 1701 -j ACCEPT"; 1;
The initdone script is unique because the $chainref variable is not set before the script is called. The above script illustrates how the $mangle_table, $filter_table, and $nat_table references can be used to add or insert rules in arbitrary chains.
Shorewall 4.4 insists that ipset names begin with a letter and be composed of alphanumeric characters, underscores (_) and dashes (-). When used in a Shorewall configuration file, the name must be preceded by a plus sign (+) as with the shell-based compiler.
Shorewall 4.4.6 re-introduced SAVE_IPSETS=Yes with slightly different semantics:
The contents of the ipsets are saved during processing of the stop command in addition to during processing of the save command.
The contents of the ipsets are restored during processing of the start command in addition to during processing of the restore command. When restore is being run when Shorewall is not in the stopped state (such as when it is run to recover from a failed start, restart or refresh) ipsets are not restored.
Specifying an ipset in shorewall-routestopped (5) is prohibited when SAVE_IPSETS=Yes.
If you find that output bandwidth is extremely limited, it is likely due to TCP Segmentation Offload (TSO) and/or Generic Segmentation Offload (GSO) being enabled in the network adapter. To verify that, install the ethtool package and use the -k command:
root@gateway:~# ethtool -k eth1 Offload parameters for eth1: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp-segmentation-offload: on udp-fragmentation-offload: off generic-segmentation-offload: on generic-receive-offload: off large-receive-offload: off ntuple-filters: off receive-hashing: off root@gateway:~#
If that is the case, you can correct the problem by adjusting the
minburst
setting in
/etc/shorewall/tcinterfaces. We suggest starting at 10-12kb and adjust
as necessary. Example:
#INTERFACE TYPE IN-BANDWIDTH OUT-BANDWIDTH
eth0 External 50mbit:200kb 5.0mbit:100kb:200ms:100mbit:10kb
Alternatively, you can turn off TSO and GSO using this command in
/etc/shorewall/init
:
ethtool -k ethN tso off gso off
The following articles provide additional information.
[1] Most distributions use a similar packaging structure. Note, however, that the 'shorewall' package in Simon Mater's RPMs for RedHat/Fedora/CentOS is like the Lenny shorewall-common package.
[2] Note that Perl is a required package on Debian. If you are running an embedded distribution which does not include Perl and it is not feasible to install Perl on your firewall, then you should consider installing Shorewall on another system in your network (may be a Windows™ system running Cygwin™ or an Apple™ MacIntosh™ running OS X) and installing Shorewall-lite on your firewall.
[3] If you are upgrading from a release using a kernel earlier than 2.6.20, then BRIDGING=Yes did work correctly with Shorewall-shell.
[4] Kernel 2.6.20 or later is required.
[5] Users whose kernel and/or iptables do not include Address Type Match Support can continue to list broadcast addresses in this column; no warning will be issued.