[mdlug] Dual NAT config questions

[David Lee Lambert]

I have two Internet connections via different providers, let's call them "A"
and "B".  For each one, I have a consumer-grade NAT device attached to the
modem.  Both devices map UDP, support static routing, some filtering, 
DNS proxy (also NAT basic UDP traffic like DNS or Teredo), and inbound 
port forwarding.  I've disabled the DHCP server on one device (the newer
one, actually).  The newer device also works as an IPv6 router with ability
to act as a 6to4 (RFC 3056) client.

A diagram:

		Service A ----  55.66.11.12 [ NAT A ] 192.168.123.1 ---+
                    /                                                  |
 ~~~~~~~~~~~~~~~~~ /                                                   |
 ( IPv4 Internet ) --- Service B -- 3.4.5.6 [ NAT B ] 192.168.123.254 -+
 ~~~~~~~~~~~~~~~~~                                                     |
      Ethernet (wire + wireless) 192.168.123.0/24, 169.254/16, IPv6... |
     +-------------+-------------+-------------+----------+------------+
     |             |             |             |          |
   Host A      Host B          Host C       Host D     Host E
		| (qemu virtual net(s)) 
           +----+-------+
           |            |
        VM B'        VM B''


I am able to point each host at one NAT device for IP and the other for 
DNS with no trouble.  In fact that's what I've set in DHCP for new 
hosts.  I can also do static routing for load-balancing or temporarily 
if one link goes down.

With just those tools, however, I don't think I can set up inbound port- 
forwarding from any provider to any host from both NATs.  Any inbound 
connection from an IP address that the host would not route back to via 
the same NAT will not succeed ... the return packets will go out the 
wrong NAT, and even if they do not get filtered earlier, they will reach 
the original host with the wrong source address.

Since several of the hosts run Linux, I suspect there's a way to write a 
script to set up routing/balnacing as desired on each host.  Now at a high
level, the rules would be:

  * Incoming TCP connections should be sticky to the NAT they came from
  * Outgoing TCP connections should be sticky to a NAT selected for
    the initial SYN packet, to include:
     - static assignment (provider A address blocks go out via NAT A, 
       and vice versa)
     - certain ports via certain NAT
     - certain process (owner, name, etc.) via certail NAT (think a qemu VM)
     - random load-balancing

I have not found a description of how to set exactly this up.  How-to guides
for similar scenarios (which may be badly out-of-date) suggest I might use
the following tools:

  "iptables -t mangle -A [INPUT|OUTPUT] ..."
  /etc/iproute2/rt_tables (config file)
  "ifconfig"  (add an additional address to each host's interface)   
  "ip route ... table [A|B]"
  "ip rule ..."

Has anyone on this list done something similar?  Or can anyone explain the 
difference between IPtables connection marks, IProute2 tables, and how to
connect them?

-- 
David Lee Lambert
Member ACM (david.lee.lambert at acm.org)
Ph# (616)676-7375  *  IM: davidleelambert (Yahoo!, Skype and Google Talk)
"Justicia, Tierra y Libertad"