marfillaster/router.cfg

 # WAN to LAN
 add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=established,related in-interface=ether1 new-connection-mark=ISP1_conn \
     passthrough=yes
 add action=mark-connection chain=prerouting connection-mark=no-mark connection-state=established,related in-interface=ether2 new-connection-mark=ISP2_conn \
     passthrough=yes

These rules are not only from WAN to LAN connections, they also catch connections directed to the router itself (if you remove the connection-state=stablished,related), thats the reason why this two rules also exist:

 add action=mark-routing chain=output connection-mark=ISP1_conn dst-address-list=!local new-routing-mark=to_ISP1 passthrough=yes
 add action=mark-routing chain=output connection-mark=ISP2_conn dst-address-list=!local new-routing-mark=to_ISP2 passthrough=yes

The output rules are using already marked connections that were directed to the router itself, and now they serve the purpose to route the connections to output from the same interface that they did enter. This have nothing to do with the ECMP routes, the ECMP routes are used only for new originated connections from the router itself to the internet, so you can ping 8.8.8.8 from the router and it will choose one of the two routes, then ping 8.8.4.4 and it may choose the another.

Sorry for my english.

how to deal with bgp routes? Their routing mark beeing overriden

@marfillaster thank you so much for this concise and coherent example, I didn't need PCC but two-way failover and routing specific clients via secondary ISP seems to work great based on this. no troubles with router-originated tunnels and OSPF either.

@marfillaster Thanks for this detailed script.

My question is – do you have any reading recommendations to actually understand how these things work, since I obviously don’t get it enough to adapt your solution to my setup described below?

The Microtik documentation is great for adapting understanding modern IP network management to their syntax if you already know all the lingo and principles involved in depth, and general reading on “how to IP” is extremely basic, I’d really like to read up to fill the gap between “I guess I understand a bit about consumer/SOHO networking” and “… and this is how you write the subtle art of networking in our language”.

My personal issue that has me confused is a bit more exotic since I have 2 ISPs, with asymmetric bandwidth (1x250, 1x500) and I’d like to distribute connections by available bandwidth proportions.

I tried to replicate the old RouterOS 6 syntax of just adding a default route with multiple gateways, with multiplying gateways by their proportion of the total bandwidth – so in my case gateway=ether1,pppoe-out1,pppoe-out1 with the recursive routes via public DNS addresses method in RouterOS 7.

I ended up with the extremely simple:

/ip route
add comment=Provider250 distance=1 dst-address=208.67.222.222 gateway=100.64.0.1 scope=10 target-scope=10
add comment=Provider500 distance=1 dst-address=1.1.1.1 gateway=pppoe-out1 scope=10 target-scope=10
add comment=Provider500 distance=1 dst-address=8.8.8.8 gateway=pppoe-out1 scope=10 target-scope=10
add check-gateway=ping dst-address=0.0.0.0/0 gateway=1.1.1.1 scope=10 target-scope=11
add check-gateway=ping dst-address=0.0.0.0/0 gateway=8.8.8.8 scope=10 target-scope=11
add check-gateway=ping dst-address=0.0.0.0/0 gateway=208.67.222.222 scope=10 target-scope=11
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

This seems to work flawlessly in that /ip route print shows all default routes being used for ECMP and /interface monitor-traffic pppoe-out1,ether1 shows both interfaces used, and pppoe-out1 more than ether1 (if not always in the exact proportions sought, but of course that’s to be expected if connections go different routes and then have different demands). Now seeing your much more sophisticated mangling-based setup, it makes me nervous that I’m missing something that opens me up to subtle errors.

I don’t see an obvious way however to adapt your script to actually use different providers for different proportions of traffic since this would seem to collide with the mangling, which can only work based on interfaces used and thus would conflate multiple routes for the same provider and possibly mess up the proportional routing?