Repository navigation
Conversation
f-hollow
changed the title
9bc53ec to
92747b6
Compare
92747b6 to
8f0cc95
Compare
|
@johnmaguire @bgolat PTAL |
jasikpark
left a comment
jasikpark
left a comment
There was a problem hiding this comment.
Thanks for this! The terminology cleanup, the move to -networks / -unsafe-networks, and the refreshed 1.11 output are all big improvements, and the 0.0.0.0/0 section is something people keep asking for. I think this also closes #107.
The main things to fix before merge are in the firewall/routing steps: the nftables policy drop, the iptables -A vs -I, and the ufw forward policy can each break or over-open forwarding on a host that already has other rules. There's also the scp destination, tun.dev on macOS, and the gateway side of the 0.0.0.0/0 setup. The rest are smaller nits.
One more that isn't on a diff line: network_diagram-unsafe_routes.png and mobile_nebula-unsafe_routes.png are no longer referenced, so they can be deleted in this PR.
— Caleb + Cache
| If connected, disconnect from the site. Now, connect again and try to access one of the hosts only accessible using | ||
| `unsafe_routes`. You should see a new line in your Mobile Nebula logs. | ||
|
|
||
| `level=warning msg="Adding UNSAFE Route" route=192.168.1.0/24 via=10.0.0.5` |
There was a problem hiding this comment.
Was this log line captured on a current Mobile Nebula build? Desktop 1.11 logs Added route now (as you show above), so this one may be out of date.
There was a problem hiding this comment.
To be honest, I just updated IP addresses in the existing log line.
I set up a new Nebula network and a full-tunnel config on an Android phone.
The connection log is given below.
Full-tunnel routing works fine, but I don't see any mention of added routes or unsafe routes... (the same on iOS)
@jasikpark Is it normal?
time=2026-10-20T01:41:34.867Z level=INFO msg="Firewall rule added" firewallRule="map[caName: caSha: cidr: direction:outgoing endPort:0 groups:[] host:any localCidr: proto:0 startPort:0]"
time=2026-10-20T01:41:34.868Z level=INFO msg="Firewall rule added" firewallRule="map[caName: caSha: cidr: direction:incoming endPort:0 groups:[] host:any localCidr: proto:0 startPort:0]"
time=2026-10-20T01:41:34.868Z level=INFO msg="Firewall started" firewallHashes=SHA:498215dec4e5687a2353f51c10838c113bd1af35ef72b8e8c9f536986ada5417,FNV:2782948616
time=2026-10-20T01:41:34.868Z level=INFO msg=listening addr=0.0.0.0:0
time=2026-10-20T01:41:34.868Z level=INFO msg="Main HostMap created" preferredRanges=[]
time=2026-10-20T01:41:34.868Z level=INFO msg="punchy disabled"
time=2026-10-20T01:41:34.868Z level=INFO msg="Loaded send_recv_error config" sendRecvError=always
time=2026-10-20T01:41:34.868Z level=INFO msg="Loaded accept_recv_error config" acceptRecvError=always
time=2026-10-20T01:41:34.869Z level=INFO msg="Nebula interface is active" interface=android networks=[10.0.10.14/24] build=1.11.1-0.20260922204501-7cfa47d569e8 udpAddr=[::]:38764 boringcrypto=false fips140Version=latest fips140Enabled=false fips140Enforced=false
time=2026-10-20T01:41:34.870Z level=INFO msg="Handshake message sent" vpnAddrs=[10.0.10.10] localIndex=134872465 remoteIndex=0 udpAddrs=[<lighthouse-public-ip>:4242] initiatorIndex=134872465 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-20T01:41:34.871Z level=INFO msg="Handshake message sent" vpnAddrs=[10.0.10.1] localIndex=421341123 remoteIndex=0 udpAddrs=[<gateway-public-ip>:4242] initiatorIndex=421341123 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-20T01:41:34.975Z level=INFO msg="Handshake message received" vpnAddrs=[10.0.10.10] from=<lighthouse-public-ip>:4242 certName=sgvps certVersion=2 fingerprint=5d8c3af750a49e2ee8e831ccdf8bfbc39b57902f4f32e6eb136349c2c922e6df issuer=2c1b5f8c0fc503970d14636df01f89eac17a8088ebadb87a131fe7ab5d926993 initiatorIndex=134872465 responderIndex=1171826692 handshake="map[stage:2 style:ix_psk0]" durationNs=105401823 sentCachedPackets=4
time=2026-10-20T01:41:35.085Z level=INFO msg="Handshake message received" vpnAddrs=[10.0.10.1] from=<gateway-public-ip>:4242 certName=jpvps certVersion=2 fingerprint=b6f31aaa78a3ce81326665f395e7a5f2698d4ae53b88874fa724f2dada2c01c8 issuer=2c1b5f8c0fc503970d14636df01f89eac17a8088ebadb87a131fe7ab5d926993 initiatorIndex=421341123 responderIndex=1425314157 handshake="map[stage:2 style:ix_psk0]" durationNs=214626354 sentCachedPackets=2| Mobile Nebula [Android](https://play.google.com/store/apps/details?id=net.defined.mobile_nebula) and | ||
| [iOS](https://apps.apple.com/us/app/mobile-nebula/id1509587936) apps route `0.0.0.0/0` correctly as long as you follow | ||
| the guidelines in [Use unsafe routes with Mobile Nebula](#use-unsafe-routes-with-mobile-nebula). |
There was a problem hiding this comment.
Have you verified this on both Android and iOS? It may well be true, since the mobile apps manage their own tunnel routes, but it'd be good to confirm before we state it.
There was a problem hiding this comment.
This is a very good call. I didn't question this, because I assumed if 0.0.0.0/0 routing is implemented correctly on Android, it would also be on iOS.
Tested it on an iPhone with Mobile Nebula v0.11: Full-tunnel routing didn't work :(
Should I mention that full-tunnel routing on iOS doesn't work for now?
Here are the logs BEFORE adding the unsafe route 0.0.0.0/0:
time=2026-10-19T21:18:08.054Z level=INFO msg="Firewall rule added" firewallRule="map[caName: caSha: cidr: direction:outgoing endPort:0 groups:[] host:any localCidr: proto:0 startPort:0]"
time=2026-10-19T21:18:08.054Z level=INFO msg="Firewall rule added" firewallRule="map[caName: caSha: cidr: direction:incoming endPort:0 groups:[] host:any localCidr: proto:0 startPort:0]"
time=2026-10-19T21:18:08.055Z level=INFO msg="Firewall started" firewallHashes=SHA:498215dec4e5687a2353f51c10838c113bd1af35ef72b8e8c9f536986ada5417,FNV:2782948616
time=2026-10-19T21:18:08.055Z level=INFO msg=listening addr=0.0.0.0:0
time=2026-10-19T21:18:08.055Z level=INFO msg="Main HostMap created" preferredRanges=[]
time=2026-10-19T21:18:08.056Z level=INFO msg="punchy disabled"
time=2026-10-19T21:18:08.056Z level=INFO msg="Loaded send_recv_error config" sendRecvError=always
time=2026-10-19T21:18:08.056Z level=INFO msg="Loaded accept_recv_error config" acceptRecvError=always
time=2026-10-19T21:18:08.056Z level=INFO msg="Nebula interface is active" interface=iOS networks=[10.0.10.25/24] build=1.11.1-0.20260922204501-7cfa47d569e8 udpAddr=[::]:52909 boringcrypto=false fips140Version=latest fips140Enabled=false fips140Enforced=false
time=2026-10-19T21:18:08.060Z level=INFO msg="Handshake message sent" vpnAddrs=[10.0.10.1] localIndex=3182111111 remoteIndex=0 udpAddrs=[<lighthouse-public-ip>:4242] initiatorIndex=3182111111 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:18:08.267Z level=INFO msg="Handshake message received" vpnAddrs=[10.0.10.1] from=<lighthouse-public-ip>:4242 certName=jpvps certVersion=2 fingerprint=b6f31aaa78a3ce81326665f395e7a5f2698d4ae53b88874fa724f2dada2c01c8 issuer=2c1b5f8c0fc503970d14636df01f89eac17a8088ebadb87a131fe7ab5d926993 initiatorIndex=3182111111 responderIndex=131141699 handshake="map[stage:2 style:ix_psk0]" durationNs=208554750 sentCachedPackets=1
time=2026-10-19T21:21:02.618Z level=INFO msg="Handshake message sent" vpnAddrs=[10.0.10.10] localIndex=173963653 remoteIndex=0 udpAddrs=[<gateway-public-ip>:4242] initiatorIndex=173963653 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:21:02.695Z level=INFO msg="Handshake message received" vpnAddrs=[10.0.10.10] from=<gateway-public-ip>:4242 certName=sgvps certVersion=2 fingerprint=5d8c3af750a49e2ee8e831ccdf8bfbc39b57902f4f32e6eb136349c2c922e6df issuer=2c1b5f8c0fc503970d14636df01f89eaHere are the logs AFTER adding the unsafe route 0.0.0.0/0:
(Note that the unsafe route 0.0.0.0/0 is not indicated in the logs, the same way as on Android)
time=2026-10-19T21:33:41.420Z level=INFO msg="Firewall rule added" firewallRule="map[caName: caSha: cidr: direction:outgoing endPort:0 groups:[] host:any localCidr: proto:0 startPort:0]"
time=2026-10-19T21:33:41.421Z level=INFO msg="Firewall rule added" firewallRule="map[caName: caSha: cidr: direction:incoming endPort:0 groups:[] host:any localCidr: proto:0 startPort:0]"
time=2026-10-19T21:33:41.421Z level=INFO msg="Firewall started" firewallHashes=SHA:498215dec4e5687a2353f51c10838c113bd1af35ef72b8e8c9f536986ada5417,FNV:2782948616
time=2026-10-19T21:33:41.421Z level=INFO msg=listening addr=0.0.0.0:0
time=2026-10-19T21:33:41.421Z level=INFO msg="Main HostMap created" preferredRanges=[]
time=2026-10-19T21:33:41.422Z level=INFO msg="punchy disabled"
time=2026-10-19T21:33:41.422Z level=INFO msg="Loaded send_recv_error config" sendRecvError=always
time=2026-10-19T21:33:41.422Z level=INFO msg="Loaded accept_recv_error config" acceptRecvError=always
time=2026-10-19T21:33:41.422Z level=INFO msg="Nebula interface is active" interface=iOS networks=[10.0.10.25/24] build=1.11.1-0.20260922204501-7cfa47d569e8 udpAddr=[::]:63173 boringcrypto=false fips140Version=latest fips140Enabled=false fips140Enforced=false
time=2026-10-19T21:33:41.426Z level=INFO msg="Handshake message sent" vpnAddrs=[10.0.10.1] localIndex=2135332346 remoteIndex=0 udpAddrs=[<lighthouse-public-ip>:4242] initiatorIndex=2135332346 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:33:41.427Z level=INFO msg="Handshake message sent" vpnAddrs=[10.0.10.10] localIndex=2072646949 remoteIndex=0 udpAddrs=[<gateway-public-ip>:4242] initiatorIndex=2072646949 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:33:48.123Z level=INFO msg="Handshake timed out" vpnAddrs=[10.0.10.1] localIndex=2135332346 remoteIndex=0 udpAddrs=[<lighthouse-public-ip>:4242] initiatorIndex=2135332346 durationNs=6697955208 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:33:48.124Z level=INFO msg="Handshake timed out" vpnAddrs=[10.0.10.10] localIndex=2072646949 remoteIndex=0 udpAddrs=[<gateway-public-ip>:4242] initiatorIndex=2072646949 durationNs=6697459583 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:33:51.546Z level=INFO msg="Handshake message sent" vpnAddrs=[10.0.10.10] localIndex=2276813806 remoteIndex=0 udpAddrs=[<gateway-public-ip>:4242] initiatorIndex=2276813806 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:33:51.547Z level=INFO msg="Handshake message sent" vpnAddrs=[10.0.10.1] localIndex=3326655433 remoteIndex=0 udpAddrs=[<lighthouse-public-ip>:4242] initiatorIndex=3326655433 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:33:58.223Z level=INFO msg="Handshake timed out" vpnAddrs=[10.0.10.10] localIndex=2276813806 remoteIndex=0 udpAddrs=[<gateway-public-ip>:4242] initiatorIndex=2276813806 durationNs=6678485042 handshake="map[stage:1 style:ix_psk0]"
time=2026-10-19T21:33:58.223Z level=INFO msg="Handshake timed out" vpnAddrs=[10.0.10.1] localIndex=3326655433 remoteIndex=0 udpAddrs=[<lighthouse-public-ip>:4242] initiatorIndex=3326655433 durationNs=6678948125 handshake="map[stage:1 style:ix_psk0]"| Then, you must ensure that underlay Nebula traffic bypasses the Nebula tunnel. For this, add explicit static routes in | ||
| your routing table for the public IPs of the lighthouse and any direct Nebula peers. |
There was a problem hiding this comment.
Static routes for the lighthouse work, but direct peers' public IPs come from the lighthouse at runtime and can change, so readers can't really pin those ahead of time. Maybe say that only the lighthouse(s) can realistically be pinned, and call out that this is the limitation on macOS/Windows?
There was a problem hiding this comment.
I see your point. I missed a lot of important details here.
Rewrote this part and ended up rewriting all other content related to full-tunnel routing.
Please check it again if you have a chance.
| so_mark: 0x6EB | ||
| ``` | ||
| After that, create a separate table for internal overlay network traffic by running: | ||
| ```sh | ||
| ip route replace default via 10.0.0.5 dev nebula1 table 4242 |
There was a problem hiding this comment.
One more on the Linux recipe, from your #1837 thread: John's Option B pairs so_mark with tun.use_system_route_table: true, so Nebula doesn't install default dev nebula1 in the main table. You got the same effect with install: false. As written, a reader who also adds the Step 4 unsafe_routes stanza for 0.0.0.0/0 gets that conflicting main-table default again. Could this show the full config? Either
listen:
so_mark: 0x6EB
tun:
use_system_route_table: truewith no unsafe_routes entry (the via on the table-4242 route is then required), or an unsafe_routes entry with install: false. Also, table 4242 holds the default route for all non-Nebula traffic, not internal overlay traffic. Maybe "a separate routing table for the full-tunnel default route"?
— Caleb + Cache
There was a problem hiding this comment.
The rewrite reads well, but the gap from this comment is still there. With only so_mark and the table-4242 route, the client has no unsafe_routes entry and no use_system_route_table, so Nebula drops the internet-bound packets it receives on nebula1. Could the snippet include one of the two configs above? Also, L557 still describes the table as holding "internal overlay network traffic". It actually holds the default route for everything except Nebula's own underlay traffic.
There was a problem hiding this comment.
I tested use_system_route_table again and left a comment in #387. I will act on this comment after we test the two configurations and decide which one to document.
jasikpark
left a comment
jasikpark
left a comment
There was a problem hiding this comment.
We ran the firewall steps from this PR (at 8f0cc95) live on fresh VMs with Nebula 1.11.2. The routing and NAT commands work as written for ufw, nftables, and iptables. Three of the persistence/setup notes don't hold up on a fresh host, and the ufw sed is confirmed unnecessary. Line comments below.
Test setup and results
Each run used a fresh Lima VM as subnet-gateway, with real systemd and real ufw/nft/iptables. nebula-roaming and the printer ran in network namespaces inside the VM, so roaming → printer traffic crossed the gateway's real FORWARD path. The printer had no default route, so replies only came back if the gateway's MASQUERADE worked. The probe was TCP to port 631 (see the note on ping below). The guide's eth0 was lan0 in the VMs.
| Scenario | Distro | Result |
|---|---|---|
| ufw as written | Ubuntu 24.04 | works; leaves FORWARD policy ACCEPT |
ufw without the DEFAULT_FORWARD_POLICY sed |
Ubuntu 24.04 | works with policy DROP; survives a reboot |
ufw with the sed, no ufw route allow |
Ubuntu 24.04 | works, so the scoped rule has no effect |
| ufw with neither | Ubuntu 24.04 | TCP blocked; ping still passes |
| nftables as written | Ubuntu 24.04, Debian 13 | works; loading the file twice is fine |
nftables include line after a reboot |
Ubuntu 24.04, Debian 13 | rules not loaded until nftables.service is enabled |
nftables policy drop vs. another table's accept |
Ubuntu 24.04 | drop blocks the other table's flow; accept doesn't |
ip_forward in /etc/sysctl.conf after a reboot |
Debian 13 | not applied |
ip_forward in /etc/sysctl.d/99-nebula.conf after a reboot |
Ubuntu 24.04, Debian 13 | applied |
| iptables as written | Ubuntu 24.04, FORWARD ACCEPT and DROP |
works |
iptables -A vs. -I FORWARD 1/2 with an existing catch-all REJECT |
Ubuntu 24.04 | -A blocked; -I works |
Every command suggested in this review's line comments was run in these VMs. We didn't test iptables persistence, because the guide doesn't give specific commands for it.
— Caleb + Cache
|
|
||
| This change will be lost upon reboot. | ||
|
|
||
| To make it persist across reboots, add `net.ipv4.ip_forward = 1` to `/etc/sysctl.conf`. |
There was a problem hiding this comment.
On Debian 13, /etc/sysctl.conf isn't shipped and isn't read at boot, so ip_forward came back as 0 after a reboot. A drop-in under /etc/sysctl.d/ survived a reboot on both Debian 13 and Ubuntu 24.04:
| To make it persist across reboots, add `net.ipv4.ip_forward = 1` to `/etc/sysctl.conf`. | |
| To make it persist across reboots, add it to a file under `/etc/sysctl.d/`: | |
| ```sh | |
| echo 'net.ipv4.ip_forward = 1' | sudo tee /etc/sysctl.d/99-nebula.conf | |
| ``` |
There was a problem hiding this comment.
This is what I am talking about in my general comment below.
I, personally, use a drop-in config on Debian for net.ipv4.ip_forward these days. I used to set it in /etc/sysctl.conf on Ubuntu years ago and it seemed to work fine. So I decided to simplify the instructions based on my prior experience without testing, because there are many more important things to test related to Nebula.
This is general knowledge that is already covered in so many places.
| Confirm the new rule is active: | ||
|
|
||
| ```sh | ||
| $ sudo ufw status | ||
| Status: active | ||
|
|
||
| To Action From | ||
| -- ------ ---- | ||
| 192.168.1.0/24 on eth0 ALLOW FWD 10.0.0.0/24 on nebula1 | ||
| ``` |
There was a problem hiding this comment.
This output assumes ufw is already enabled. On a fresh Ubuntu host it's inactive: ufw reload prints Firewall not enabled (skipping reload) and ufw status shows Status: inactive. Could we add a line saying this section assumes ufw is already active? Enabling it also needs SSH and Nebula's UDP port allowed first, or it can lock you out.
| #### nftables | ||
|
|
||
| Create a new nftables configuration (e.g. `/etc/nftables-routing.conf`) with the following configuration: | ||
| Create a new nftables configuration (e.g., `/etc/nftables/nebula.conf`). Configure the packet forwarding chain and then |
There was a problem hiding this comment.
/etc/nftables/ doesn't exist on fresh Ubuntu 24.04 or Debian 13, so creating the file fails until the directory exists. Could we add sudo mkdir -p /etc/nftables first? (The Debian 13 cloud image also doesn't include nft, so sudo apt install nftables may be needed there.)
| To make them persist across reboots, include this file in your main `/etc/nftables.conf` by adding the following line to | ||
| it: | ||
|
|
||
| ```sh | ||
| include "/etc/nftables/nebula.conf" | ||
| ``` |
There was a problem hiding this comment.
On both Ubuntu 24.04 and Debian 13, nftables.service is disabled by default, so the include line alone didn't load the rules after a reboot. It worked once the service was enabled:
| To make them persist across reboots, include this file in your main `/etc/nftables.conf` by adding the following line to | |
| it: | |
| ```sh | |
| include "/etc/nftables/nebula.conf" | |
| ``` | |
| To make them persist across reboots, include this file in your main `/etc/nftables.conf` by adding the following line to | |
| it, and enable the `nftables` service so the file is loaded at boot: | |
| ```sh | |
| include "/etc/nftables/nebula.conf" | |
| ``` | |
| ```sh | |
| sudo systemctl enable nftables | |
| ``` |
| As you can see in `msg="Added route"`, the unsafe route is being used. | ||
|
|
||
| Ping working? Great! Go ahead and add that printer, map file shares, and VNC to any host on that LAN. | ||
| Now you should be able to successfully ping any host on the subnet from your roaming host and access subnet devices: |
There was a problem hiding this comment.
On a ufw host, ping isn't a reliable test of the routing rules: ufw's default before.rules forwards ICMP echo even with no ufw route allow rule. Maybe suggest also testing a TCP port (e.g. the printer's 631) to confirm forwarding works?
|
@jasikpark Thank you for the detailed review and for testing the firewall configs! I have already implemented half of the comments and will push the changes once I implement and test a few others. Regarding the firewall setup instructions, there is one issue from the technical writing perspective. When core documentation includes instructions for setting up third-party tools or the environment, maintainers become responsible for tracking external changes and keeping those instructions current. A common documentation best practice is to focus the core docs on the product itself and provide high-level guidance or prerequisites for third-party tools and the environment, whatever might not be obvious from standard practices. If there is capacity to help less experienced users, a good alternative is to provide additional resources, for example a blog post, with more detailed explanations. In that case, changes in specific distros or third-party tools are much less likely to impact the accuracy of the core documentation. An outdated blog post will be perceived by users as something natural and can be updated periodically when there is capacity for it. What do you think of keeping the firewall section high-level and, optionally, writing a blog post with more detailed instructions? |
|
Thanks @f-hollow, that's a fair argument, and I agree with the general principle. Normally I'd say the same thing: put the detailed walkthrough in a blog post, let it age like a blog post, and fold the lasting parts into the docs. We talked it over internally, though, and we'd like to keep the iptables/nftables/ufw sections in this page. In Diátaxis terms, this page sits somewhere between a tutorial and a how-to guide. It's less a reference for how To handle the maintenance concern, could you add an admonition ( — Caleb + Cache |
|
@jasikpark I have resolved the straightforward comments (a4bdc8f), removed the outdated pictures (2ed9431) and replied to the comments that might need your follow-up, including the rewritten full-tunnel routing section (4fb426c). I will try to implement the rest of the comments by the end of the week. Regarding the firewall rules, I agree with your decision to keep the detailed steps and add an admonition. The detailed walkthrough makes this guide (or rather tutorial as you pointed out) much more user-friendly. I started using Tinc a few years ago with almost zero networking experience, and its documentation wasn't very beginner-friendly. Though I am sure it's very easy to maintain, and now that I have more experience, I like that it skips the basics. That's why I asked the question to see what you prefer. I will fill in the firewall details according to your comments and see if any other things can be further improved. |
jasikpark
left a comment
jasikpark
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround! The scp, local_cidr, ufw, nftables, and link fixes all look good, and the full-tunnel rewrite reads much more clearly.
Suggestion: split the full-tunnel section into a stacked follow-up PR. Steps 1–4 are a tutorial for reaching a subnet, and that part is close: once the firewall items from the last round land, it's ready to merge. Routing 0.0.0.0/0 is a different setup on the client side (routing loops, so_mark, the /1 routes), and it's where most of the open questions are. Moving it to its own PR, stacked on this one, would let this one merge without waiting on those. The full-tunnel comments below still apply wherever the section ends up. If you'd rather keep it here, that's fine too.
In this round:
- The iptables
-Ichange dropped the-iflags, so those two commands now fail (line comment). - The prerequisites now list macOS and Windows for the subnet gateway, but the steps are Linux-only (line comment).
- For the full-tunnel section: the gateway checklist needs a couple of additions (line comment), and the Linux recipe still has the routing gap from the earlier thread (replied there).
No rush on the remaining firewall items and the open questions on L516 and L534. I know those are still in progress.
— Caleb + Cache
| sudo iptables -I FORWARD 1 nebula1 -o eth0 -s 10.0.0.0/24 -d 192.168.1.0/24 -j ACCEPT | ||
| sudo iptables -I FORWARD 2 eth0 -o nebula1 -s 192.168.1.0/24 -d 10.0.0.0/24 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT |
There was a problem hiding this comment.
The -i got dropped when switching from -A. Without it, iptables rejects nebula1 with Bad argument `nebula1'.
| sudo iptables -I FORWARD 1 nebula1 -o eth0 -s 10.0.0.0/24 -d 192.168.1.0/24 -j ACCEPT | |
| sudo iptables -I FORWARD 2 eth0 -o nebula1 -s 192.168.1.0/24 -d 10.0.0.0/24 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT | |
| sudo iptables -I FORWARD 1 -i nebula1 -o eth0 -s 10.0.0.0/24 -d 192.168.1.0/24 -j ACCEPT | |
| sudo iptables -I FORWARD 2 -i eth0 -o nebula1 -s 192.168.1.0/24 -d 10.0.0.0/24 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT |
| - The **subnet gateway** running Linux, macOS, or Windows that will forward traffic from the overlay network to the | ||
| subnet using `unsafe_routes` |
There was a problem hiding this comment.
Sorry, my earlier comment was ambiguous. Other OSes can act as the gateway, but this guide only shows Linux commands in Step 3, so the prerequisite should match the note above:
| - The **subnet gateway** running Linux, macOS, or Windows that will forward traffic from the overlay network to the | |
| subnet using `unsafe_routes` | |
| - The **subnet gateway** running Linux that will forward traffic from the overlay network to the subnet using | |
| `unsafe_routes` |
| - Inbound Nebula firewall rules cover `local_cidr` for `0.0.0.0/0` | ||
| - System firewall rules for forwarding and NAT/masquerading do not contain destination restrictions (such as | ||
| `-d 192.168.1.0/24`) |
There was a problem hiding this comment.
Two additions here:
local_cidr: 0.0.0.0/0also matches the gateway's own addresses, so overlay hosts can reach any port on the gateway itself that the rule allows. Worth a sentence, or a separate narrower rule for the gateway's services.- The guide uses three tools, so the "no destination restrictions" example could name each one:
to 192.168.1.0/24inufw route allow,ip daddrin nft,-din iptables. The forwarding and masquerade rules should still be limited to the uplink interface (out on eth0,oifname eth0,-o eth0) once the destination is removed.
2 participants
This PR is my attempt to improve the unsafe routes guide mentioned in slackhq/nebula#1837 (comment) after my journey setting up a Nebula overlay network with unsafe routes.
In the near future, I plan to create a couple more PRs with some minor improvements in other documents related to this one.
Important
I took the liberty to update quite a few things.
Feel free to let me know if any proposed changes are not desirable or don't feel right for whatever reason.
Changes
Among the other changes in this PR, the following deserve a mention:
Linux router hostvs.linux hostorlocal networkvs.subnetvs.extended network-ip>-networks-subnets>-unsafe-networks0.0.0.0/0Proposals
Just because you use the same documentation for all Nebula versions, it might be good to mark documents with the version they were written for. It will help track how old the documents are and which of them need to be updated according to the breaking changes in newer Nebula versions.
For this, I propose adding metadata parameters in doc YAML frontmatter:
app_version-- Nebula version for which the guide was written and verified againstmin_app_version-- lowest supported version: newnebula-certoptions-networksand-unsafe-networksusedQuestions
192.168.100.0/24was the Nebula or LAN subnet, because192.168.86.0/24is very similar. Using10.0.0.0/24for Nebula and192.168.1.0/24for LAN would immediately solve the problem. But I see that you use192.168.100.0/24for the Nebula network inconfig.yml. Do you want to keep it that way or are you willing to reconsider example IP ranges in the docs?