krun: passt - don't map the gateway address#2099
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the libkrun VM configuration to use the NET_FLAG_DHCP_CLIENT flag and overrides the DNS server advertised by passt to use Podman's pasta DNS forwarder address. To ensure compatibility with older build environments and runtime versions of libkrun, it is recommended to provide a fallback definition for NET_FLAG_DHCP_CLIENT and handle fallback scenarios (such as catching -EINVAL) to retry the call without the flag.
|
Ephemeral COPR build failed. @containers/packit-build please check. |
112d741 to
d0a5fea
Compare
|
ok. looks like #2093 was happening in parallel here so I've dropped the first commit from the PR here. |
|
@slp PTAL |
|
This is a step in the right direction, thanks @dustymabe. Ideally, we should also use @sbrivio-rh any suggestions? |
I'm not quite sure what the problem would be here. If the DNS server is on the local network, and it's not a loopback address, it will anyway reach pasta and be forwarded (with translation, if necessary) to the resolver pasta derives from the host configuration. What is the actual issue with that?
I didn't quite understand what problem exactly we're fixing with this. Dropping Regardless of that: maybe it would be a good idea to stick to exactly the same as Podman does for simplicity. It took us a long time to get that right and it should fit as it is in the the krun situation as well if I understand correctly. |
The problem is, in this scenario, passt is using the host's DNS server instead of the |
Right, and you're suggesting that we should switch to the aardvark-dns resolver. But the current pull request doesn't do that, correct? I'm still trying to understand what problem specifically a particular |
I can try to answer that. I am certainly no expert here and sometimes "networking" makes my head hurt, so there's that disclaimer. The problem that I was having is that before the 2nd commit in this PR I couldn't reach the DNS server (which also happened to be my gateway for the local network, the router), but slotting in a public DNS server like 8.8.8.8 worked fine. I was chatting through this problem with opencode/anthropic/opus4.6 and this is what it came up with: My understanding of the 2nd commit (and believe me there's a lot of handwaving here from my side) is that the I can try to reproduce the full conversation I had with AI somewhere if that would be useful. Additionally, I will note that the 2nd commit in this PR might not be the correct solution (and I appreciate the conversation trying to fight the correct solution), but it has been working nicely for me for the past few days and I've finally gotten to the workflow I desire described in libkrun/libkrunfw#128 (comment) |
Thanks for the offer, but that part is what's confusing, because the very "assumptions" from that excerpt look somewhat plausible but they range from misleading ("For remote hosts like 8.8.8.8, passt creates a socket on the host and connects — this works because the host's kernel handles routing. But 192.168.86.1 is your local gateway/router [...]") to plain wrong ("This should work in principle — but there's a known interaction", "pasta provides internet access through NAT"). Anyway, I think I figured out what the problem is: by default (unless So, yes, something like what you did is needed. I'm not quite sure if that's sufficient to reach aardvark-dns, Podman networking people might help with that. Let me ask around. |
|
Thanks @sbrivio-rh for helping me understand! |
|
I've been digging a bit more on the issue. Let's say we have a VM with this config: When the VM attempts to communicate with the DNS server at Changing I've tried started passt with This is the network configuration in the netns where passt is running: @sbrivio-rh any idea why passt is translating |
That's not specific to DNS traffic, it's the default behaviour (unless you pass
that was picked as an address to refer to "the host" mostly for KubeVirt convenience, as they needed a valid known address without an explicit configuration, and usually the default gateway is not something you would try to reach directly as a connection destination from KubeVirt pods. This is just the default, and it can be changed in two ways:
I haven't completely thought this through but if you don't need / want an option for the guest to connect to the host, If you want to use another address to refer to the host, by the way, you can use |
|
Adding So I think we need to:
This would allows us to preserve the |
I wonder if keeping If you isolate that over DHCP / NDP / DHCPv6, passt validates the information and tells the guest only what it needs over the network and in a simpler (binary) format, with no separate channel involved. |
Both passt and the libkrun are running within the container context, so the This is the same thing we do with TSI. We don't replace the container's boundaries with ours, we add our own on top of the container's. |
Yes, I get it, that's why I was mentioning possible fingerprinting of Podman's version (not of NetworkManager or whatever is configuring things on the host). I still feel like handling this over network protocols is a marginally more "isolated" way of doing things.
Oh dear, another case of passt-in-pasta. 😄 I thought we only had that in our upstream tests.
I see, it's just that with passt you could add a bit more boundaries than that. I'm not sure if it's worth it, perhaps in this case it's really not substantial. |
|
Trying to track the conversation here (that is way over my head)... are you saying we want something like: diff --git a/src/libcrun/handlers/krun.c b/src/libcrun/handlers/krun.c
index 5e1f3e54..34d309b1 100644
--- a/src/libcrun/handlers/krun.c
+++ b/src/libcrun/handlers/krun.c
@@ -567,7 +567,7 @@ libkrun_start_passt (void *cookie, libcrun_container_t *container)
{
struct krun_config *kconf = (struct krun_config *) cookie;
pid_t pid;
- char *passt_argv[9];
+ char *passt_argv[10];
char fd_as_str[16];
int use_passt;
int argv_idx;
@@ -595,9 +595,13 @@ libkrun_start_passt (void *cookie, libcrun_container_t *container)
{
passt_argv[argv_idx++] = (char *) "-u";
passt_argv[argv_idx++] = (char *) "all";
- passt_argv[argv_idx++] = (char *) "--no-dhcp-dns";
}
+ /* Please help me write a good comment here!!!
+ */
+ passt_argv[argv_idx++] = (char *) "--no-dhcp-dns";
+ passt_argv[argv_idx++] = (char *) "--no-map-gw";
+
passt_argv[argv_idx++] = (char *) "--fd";
passt_argv[argv_idx++] = fd_as_str;
passt_argv[argv_idx] = NULL; |
@slp is suggesting something like that, yes. I'm suggesting that we could drop |
|
@slp What do you think of @sbrivio-rh suggestion? Should I try the patch in #2099 (comment) first? Can someone help me write an informed comment for that patch and I'll push it here to this PR. |
|
Ok I tried to experiment a bit. I've tried with just and I've tried with in both cases As part of this I rebased my stack (collection of RPMs) on top of the latest ones in Fedora so I just applied the patches to the most recent ones there. |
If the gateway for the network is also the DNS server we won't be able to query the DNS server without --no-map-gw. See discussion in containers#2099 Signed-off-by: Dusty Mabe <dusty@dustymabe.com>
Combining |
ok cool. I'll pull libkrun#698 into my list of backport patches and try it. |
|
@slp do you have any opinions on if we should pull the See the patch in #2099 (comment) Or we could do something like: diff --git a/src/libcrun/handlers/krun.c b/src/libcrun/handlers/krun.c
index 6e249a9b..6092988d 100644
--- a/src/libcrun/handlers/krun.c
+++ b/src/libcrun/handlers/krun.c
@@ -567,7 +567,7 @@ libkrun_start_passt (void *cookie, libcrun_container_t *container)
{
struct krun_config *kconf = (struct krun_config *) cookie;
pid_t pid;
- char *passt_argv[9];
+ char *passt_argv[10];
char fd_as_str[16];
int use_passt;
int argv_idx;
@@ -598,6 +598,12 @@ libkrun_start_passt (void *cookie, libcrun_container_t *container)
passt_argv[argv_idx++] = (char *) "--no-dhcp-dns";
}
+ /* Set --no-map-gw. If the gateway for the network is also the DNS server
+ * we won't be able to query the DNS server without --no-map-gw.
+ * See discussion in https://github.com/containers/crun/pull/2099
+ */
+ passt_argv[argv_idx++] = (char *) "--no-map-gw";
+
passt_argv[argv_idx++] = (char *) "--fd";
passt_argv[argv_idx++] = fd_as_str;
passt_argv[argv_idx] = NULL; |
That conditional is for AWS Nitro support and I don't have access to test it. We need to ask @tylerfanelli |
|
ok. i'll just limit the patch such that it only makes the smallest change possible then and test that. Then I'll push it up here if it works. |
d0a5fea to
10240f4
Compare
|
ok here is what I see now (when I combine the patch I just pushed here with libkrun/libkrun#698) From the host: Inside the krun VM (started with |
|
TMT tests failed. @containers/packit-build please check. |
I think the failures here are because of The CI failures are not related to your PR at all. Here's what's happening: CI Failure Analysis The two failed checks (testing-farm:fedora-rawhide-x86_64 and testing-farm:fedora-rawhide-aarch64) are both failing on the same test: /tests/provision/container/basic in the /plans/ The root cause is a Python 3.15 compatibility bug in the tmt testing framework itself: module 'dataclasses' has no attribute '_MISSING_TYPE' The traceback shows: File /usr/lib/python3.15/site-packages/tmt/container/init.py, line 435, in is_bare Python 3.15 (which is what Fedora Rawhide ships) removed or renamed dataclasses._MISSING_TYPE (a private/internal API). The tmt tool is accessing this private attribute, and it |
|
@slp @sbrivio-rh - mind another review? |
EDIT:
If the gateway for the network is also the DNS server
we won't be able to query the DNS server without --no-map-gw.
See discussion in #2099
Old:
The commits in this PR try to get some default networking into the krun VM started withpodman run --runtime=krun --annotation krun.use_passt=1. First we pass inNET_FLAG_DHCP_CLIENTtokrun_add_net_unixstreamand then pass in--dns 169.254.1.1to workaround a problem where if the DNS server is on the local network it won't hit NAT and won't be forwarded (thus unreachable from inside the krun VM).The hardcoding here of169.254.1.1isn't great. It's the default used by podman rootless (so that target is available inside the rootless container namespace, but not the krun VM). I'm interested to know the experts thoughts here and if there is a better alternative.