MPTCP is missing from system parameters inside Mininet hosts

https://github.com/mininet/mininet/issues/1161

 

https://github.com/multipath-tcp/mptcp/issues/506

 

Hello,

I am having problems with certain system parameters not being accessible from within a Mininet host even though they are accessible from the host system.

After a clean install with the following:

• Ubuntu 20.04.5 LTS
• The 5.4.230.mptcp kernel

I get the following output when running sysctl -a | grep mptcp outside Mininet:

net.mptcp.mptcp_checksum = 1
net.mptcp.mptcp_debug = 0
net.mptcp.mptcp_enabled = 1
net.mptcp.mptcp_path_manager = fullmesh
net.mptcp.mptcp_scheduler = default
net.mptcp.mptcp_syn_retries = 3
net.mptcp.mptcp_version = 0

 

which is correct. But when I do the same from within a Mininet host by starting Mininet with the default topology:

sudo mn

and then running the sysctl command on, for instance, the h1 node:

mininet> h1 sysctl -a | grep mptcp

I dont get any output. In other words, the system parameters can't be found by sysctl -a from within a Mininet host.

Expected/Desired Behavior:

mininet> h1 sysctl -a | grep mptcp

=>

net.mptcp.mptcp_checksum = 1
net.mptcp.mptcp_debug = 0
net.mptcp.mptcp_enabled = 1
net.mptcp.mptcp_path_manager = fullmesh
net.mptcp.mptcp_scheduler = default
net.mptcp.mptcp_syn_retries = 3
net.mptcp.mptcp_version = 0

 

Actual Behavior:

mininet> h1 sysctl -a | grep mptcp

=>

[No output]

 

Detailed Steps to Reproduce the Behavior

1. Install "Ubuntu 20.04.5 LTS" and the "5.4.230.mptcp" kernel.
2. Run sudo mn
3. Inside the Mininet console execute sysctl -a | grep mptcp
4. Compare output when running it outside the host

Additional Information

The following people seems to have had the same problem:

• TR : proc/sys/net/mptcp missing
• mptcp inside lxc container can't access /proc/sys/net/mptcp_enabled

Here are some outputs from both the host system and from within a Mininet host. It seems to be the same output regardless of if sudo or not is used in both cases.

Command	Result in Ubuntu (outside a Mininet host)	Result from within a Mininet host
ip netns identify $$	[No output]	[No output]
ls /proc/net/mptcp_net/	mptcp snmp	mptcp snmp
readlink /proc/self/ns/net	net:[4026532008]	net:[4026533846] (differs between Mininet hosts)
ls /proc/sys/net/mptcp	mptcp_checksum mptcp_debug mptcp_enabled mptcp_path_manager mptcp_scheduler mptcp_syn_retries mptcp_version	[No output]

I think it's relevant to note that creating hosts with inNamespace=False:

h1 = net.addHost('h1', inNamespace=False) # don't spawn in net namespaces
h2 = net.addHost('h2', inNamespace=False) # don't spawn in net namespaces
r1 = net.addHost('r1', inNamespace=False) # don't spawn in net namespaces

 

results in me being able to see the system parameters from inside a Mininet host by running sysctl -a | grep mptcp. While this solves the access problem it seems to result in the network parameters that you can set for each Mininet host to be ignored.

I think I somehow need to figure out how I can mount the namespace or perhaps the "/proc/sys/net/mptcp" folder. Running mount --bind /proc/sys/net/mptcp from within a Mininet host just results it:

mount: /proc/sys/net/mptcp: mount point does not exist.

 

https://github.com/multipath-tcp/mptcp/issues/506

 

mrconter1 commented on Mar 16, 2023

Hello,

I am having problems with certain system parameters not being accessible from within a Mininet host even though they are accessible from the host system.

After a clean install with the following:

• Ubuntu 20.04.5 LTS
• The 5.4.230.mptcp kernel

I get the following output when running sysctl -a | grep mptcp outside Mininet:

net.mptcp.mptcp_checksum = 1
net.mptcp.mptcp_debug = 0
net.mptcp.mptcp_enabled = 1
net.mptcp.mptcp_path_manager = fullmesh
net.mptcp.mptcp_scheduler = default
net.mptcp.mptcp_syn_retries = 3
net.mptcp.mptcp_version = 0

 

which is correct. But when I do the same from within a Mininet host by starting Mininet with the default topology:

sudo mn

and then running the sysctl command on, for instance, the h1 node:

mininet> h1 sysctl -a | grep mptcp

I dont get any output. In other words, the system parameters can't be found by sysctl -a from within a Mininet host.

Expected/Desired Behavior:

mininet> h1 sysctl -a | grep mptcp

=>

net.mptcp.mptcp_checksum = 1
net.mptcp.mptcp_debug = 0
net.mptcp.mptcp_enabled = 1
net.mptcp.mptcp_path_manager = fullmesh
net.mptcp.mptcp_scheduler = default
net.mptcp.mptcp_syn_retries = 3
net.mptcp.mptcp_version = 0

 

Actual Behavior:

mininet> h1 sysctl -a | grep mptcp

=>

[No output]

 

Detailed Steps to Reproduce the Behavior

1. Install "Ubuntu 20.04.5 LTS" and the "5.4.230.mptcp" kernel.
2. Run sudo mn
3. Inside the Mininet console execute sysctl -a | grep mptcp
4. Compare output when running it outside the host

Additional Information

The following people seems to have had the same problem:

• TR : proc/sys/net/mptcp missing
• mptcp inside lxc container can't access /proc/sys/net/mptcp_enabled

---

Best regards, Rasmus

 

 

 

Member

matttbe commented on Mar 16, 2023

Hi Rasmus,

I don't know how mininet works these days: does it create network namespaces per host or is it a VM?

Because (out of tree) MPTCP's sysctl knobs are not net namespace aware: you set them on the "host" (the initial netns) but not per netns: they are not visible from the netns. Like other sysctls.

If you don't know how to check, maybe try this from a "host" and check if it is not empty

# ip netns identify $$

 

Or simply check if you have MPTCP this file from a "host": /proc/net/mptcp_net/mptcp

(or check nsenter, compare readlink /proc/self/ns/net, check if qemu is running, etc.)

 

 

Author

mrconter1 commented on Mar 17, 2023 • 

edited 

Thanks for your reply. Mininet works by running the hosts in different namespaces and they aren't separate VM's. Here are some outputs from both the host system and from within a Mininet host. It seems to be the same output regardless of if sudo or not is used in both cases.

Command	Result in Ubuntu (outside a Mininet host)	Result from within a Mininet host
ip netns identify $$	[No output]	[No output]
ls /proc/net/mptcp_net/	mptcp snmp	mptcp snmp
readlink /proc/self/ns/net	net:[4026532008]	net:[4026533846] (differs between Mininet hosts)
ls /proc/sys/net/mptcp	mptcp_checksum mptcp_debug mptcp_enabled mptcp_path_manager mptcp_scheduler mptcp_syn_retries mptcp_version	[No output]

I think it's relevant to note that creating hosts with inNamespace=False:

h1 = net.addHost('h1', inNamespace=False) # don't spawn in net namespaces
h2 = net.addHost('h2', inNamespace=False) # don't spawn in net namespaces
r1 = net.addHost('r1', inNamespace=False) # don't spawn in net namespaces

 

results in me being able to see the system parameters from inside a Mininet host by running sysctl -a | grep mptcp. While this solves the access problem it seems to result in the network parameters that you can set for each Mininet host to be ignored.

I think I somehow need to figure out how I can mount the namespace or perhaps the "/proc/sys/net/mptcp" folder. Running mount --bind /proc/sys/net/mptcp from within a Mininet host just results it:

mount: /proc/sys/net/mptcp: mount point does not exist.

 

 

 

Member

matttbe commented on Mar 17, 2023

OK, good to know.

Why do you need to access the sysctl entries from the namespaces? If you change them, they will change the behaviour in all the netns.

 

 

Author

mrconter1 commented on Mar 17, 2023

Each Mininet host needs to be able to access the system parameters in order to work properly.

 

 

Member

matttbe commented on Mar 17, 2023

I don't think you can make them appearing in /proc/sys/net/mptcp without breaking stuff (or changing the kernel code to expose the parameters to all netns or make them netns aware).

Can you not dump the parameters in a file and read this file?

Or maybe you can try something like as root from a netns:

mkdir -p /var/run/netns/
ln -sfT /proc/1/ns/net /var/run/netns/host

 

then from the netns, you might be able to do:

ip netns exec host sysctl net.mptcp

 

 

 

Member

matttbe commented on Mar 17, 2023

or make them netns aware

Probably too late to do that for this out-of-tree kernel. Maybe time to switch to the upstream one? https://mptcp.dev

 

 

Author

mrconter1 commented on Mar 17, 2023

I don't think you can make them appearing in /proc/sys/net/mptcp without breaking stuff (or changing the kernel code to expose the parameters to all netns or make them netns aware).

Can you not dump the parameters in a file and read this file?

Or maybe you can try something like as root from a netns:

mkdir -p /var/run/netns/
ln -sfT /proc/1/ns/net /var/run/netns/host

 

then from the netns, you might be able to do:

ip netns exec host sysctl net.mptcp

 

When I do the following from within a Mininet host:

sudo mkdir -p /var/run/netns/
sudo ln -sfT /proc/1/ns/net /var/run/netns/host
sudo ip netns exec host sysctl net.mptcp

 

The last command fails with:

mount of /sys failed: Device or resource busy

 

 

Author

mrconter1 commented on Mar 17, 2023

or make them netns aware

Probably too late to do that for this out-of-tree kernel. Maybe time to switch to the upstream one? https://mptcp.dev

That might be the way forward. I am also thinking of trying this out from Debian. That might work better but I am just speculating...

 

 

Member

matttbe commented on Mar 17, 2023

Yes, there might be a way but I never had to do that (and it is not specific to MPTCP).

I don't think using Debian will change anything but why not trying.
Maybe also look at nsenter, I guess mininet uses that.

(or maybe they still have a way to use different VMs? I thought it was like that in the past, a long time ago)

 

 

Member

matttbe commented on Mar 17, 2023

Maybe this? (probably you will get the same error)

nsenter -t 1 sysctl net.mptcp

 

 

 

Author

mrconter1 commented on Mar 17, 2023

Unfortunately it doesn't seem to work either:

mininet> h1 nsenter -t 1 sysctl net.mptcp
sysctl: cannot stat /proc/sys/net/mptcp: No such file or directory

 

But anyways. Thanks for the help! I will try my luck with Debian now.