Building the nest

[CwF]

I have been testing a handful of things exploring the performance and consistency of double nested virtual machines.

Mixed news on the networking. Yes, a macvtap on on unconfigured vfio eths work many layers down without any virt bridging and yes, the host one layer above can cause issues with 'persistent naming'. I'll keep going on that...

Yes, lower level VM's are 'nice' to the cpus and do not needlessly clock beyond duty rating (non-turbo), while performing as expected.

Yes, pipewire is a mess in this use case and can't deliver smooth audio. I already knew that, so I did revert to pulse and tried again. Using a remote machine for access to a L3 VM is needlessly pushing the extremes, but... Pulse did clean it up and deadbeef could even do visualizations cleanly. But, I concluded the scenario sub-par.

Then, as I sipped and listened to my Dove's 'whoo' in the snow filled trees I realized the L1 nested hypervisor image I used is also on pipewire...

I'll need to try again.

[Linuxgaming1824]

I have been testing a handful of things exploring the performance and consistency of double nested virtual machines.

Have you ever tried easy-os it uses a virtualized file system on top of a virtualized file system on top of a virtualized file system, that makes it seem perceptually to the end user like there is great performance!

As opposed to virtual machines in other words, there are operating systems designed around similar principles

[CwF]

You kind of missed it there buddy.

[Linuxgaming1824]

Yes there are nested operating systems as opposed to nested operating systems in virtual machines.

I think for a variety of projects based on these virtualization solutions it is cheating actually,

because they ignore the root causes of performance/security degradation.(or rather build on top of them)

I think it's something we have to try and get away from

[CwF]

virtualization solutions it is cheating actually,

How exactly is it cheating?

because they ignore the root causes of performance/security degradation.(or rather build on top of them)

What are those exactly?

I think it's something we have to try and get away from

Why, exactly?
The virtualization path was obvious decades ago and is ingrained in all the computing you experience.

[Linuxgaming1824]

They are subject to discovery aren't they! I'll give you an example when we are using a computer and loading processes' into memory aside from the basic task of loading the processes' into memory there are extraneous functions that cost us processing time, in the name of, in this case, security! The basic task is, load the process into memory, whereas now, it is load the process into memory, and also randomize the bits, and/or alter it's location, so they cannot be exploited. That's essentially what virtualization does broadly, it alters the basic goal or task of computing and transforms it into something entirely different. So when we have a file system operation that on a standard debian operating system with ext4 takes 60 seconds, and a standard debian operating system performing the same file system operation with ext4(or rather the virtual machine image) inside a virtual machine where it takes 30 seconds, what are we actually observing? What is wrong with the standard non-virtualized file system operation?

[CwF]

what are we actually observing?

cache
..and not really on point.

[Linuxgaming1824]

and this contrast is apparent to me, testing normal systems versus non-virtual systems and their basic operations. I understand that the context is different which alters how different things will perform. Sometimes it's dramatic, as with file system operations. I attempt to customize the standard system to get the best performance it can get, by reducing the 'virtualization' built into our operating systems actually.

but, ultimately, since it's not a NEW system, and it's just customizing the old system, I guess people can't sell that for a profit after all.

; )

[Linuxgaming1824]

One thing I've started testing recently(not to mention hundreds of things) is working with smaller partitions and alternate overall partitioning schemes to alter systems performance

What's the difference between /boot /efi and / compared to just /

?

Or, a 100 GB / and a 1 TB / or a 1/2 TB /

What's the difference between UEFI and BIOS partitioning schemes for a root file system?

[Linuxgaming1824]

If the system is change over time, than we can measure these differences in terms of performance, which has broad all-encompassing effects

With a highly customized linux system we can get down to 1ms response times from applications, and using the system broadly, virtualization solutions can't do that, they're moving backwards, but subprocesses' to using the system broadly do perform better.

[CwF]

Alright then, reverted that middle hypervisor to pulse and it did help.

That middle layer still gets confused with the network naming .links. Different every time. The layer underneath is consistent. Strange.
So,
Bare metal eth without configuration and declared as vfio and macvtap'ed via virtio for the guest hypervisor is slow, yet consistently renamed.
Bare metal eth with host configuration and macvtap'ed via virtio for the guest hypervisor is fine or dead, inconsistently renamed.
Two bare metal eth without configuration and declared as vfio and also vfio in the guest hypervisor and again without configuration, then macvtap'ed for a double nested vm are both consistently renamed and work fine.

Interesting...a tad confusing. The point here is actually exploring a batch (dozen+) link files in /etc/systemd/network that can parse correctly, without ambiguous overlap, where the image is booted as bare metal(L0), guest(L1), or nested guest(L3), and consistently name the 2 network segments.

and for you @Linuxgaming1824, when the electromagnetic storm comes I want all these images to be able to jump into your OnStar computer in your car and hide until it's all over.

[CwF]

Bugs in the nest!
notice the available net interfaces:

Code: Select all

$ ls /sys/class/net
enp10s0  enp1s0  enp8s0  enp9s0  Glan  lo

and when asking to explain itself:

Code: Select all

$  udevadm test-builtin net_setup_link /sys/class/net/enp10s0
.....
Created link configuration context.
enp10s0: Device has name_assign_type=4
enp10s0: Device has addr_assign_type=0
ID_NET_DRIVER=virtio_net
enp10s0: Config file /etc/systemd/network/10-nested-Wlan.link is applied
enp10s0: Using static MAC address.
enp10s0: Policies didn't yield a name, using specified Name=Wlan.
ID_NET_LINK_FILE=/etc/systemd/network/10-nested-Wlan.link
ID_NET_NAME=Wlan

and the desktop panel indicator says Wlan is dead, even though it is working...not as Wlan, but working

Let's ask someone else:

Code: Select all

# vnstat --add -i Wlan
Error: Unable to get interface "Wlan" statistics.
Only available interfaces can be added for monitoring.

The following interfaces are currently available:
    enp1s0 Glan enp10s0 enp8s0 enp9s0 

The interface is working because of a redundant declaration in /etc/network/interfaces.d/ defining enp10s0. Without it the interface does not work while the name is still confused - it was the last addition.

Initial boot is always wrong, restarts often rename correctly.
This is a middle layer VM, using libvirt and virt-manager.
enp10s0 is a macvtap of the host MB port, also Wlan, and serving multiple vm's, all working
enp1s0, enp8s0, enp9s0 are not configured in the host and vfio passed to this guest,
declared as vfio in this guest, they pass again to the next layer and work fine.
Glan is the 4th of the quad vfio passed to this vm, however it is not passed but rather a macvtap.

Yes, I still like systemd, and will keep trying.

[cds60601]

Can't you just use .link files?

[CwF]

Can't you just use .link files?

Yes, I am. It does show reading and respecting a link file, it's just lying!

Code: Select all

ID_NET_LINK_FILE=/etc/systemd/network/10-nested-Wlan.link

Code: Select all

enp10s0: Policies didn't yield a name, using specified Name=Wlan.

...and I just found it, a slight typo

Code: Select all

Wlan: Device has name_assign_type=4
Wlan: Device has addr_assign_type=0
ID_NET_DRIVER=virtio_net
Wlan: Config file /etc/systemd/network/42-virtio-Wlan.link is applied
Wlan: Using static MAC address.
Wlan: Policies didn't yield a name, using specified Name=Wlan.
ID_NET_LINK_FILE=/etc/systemd/network/42-virtio-Wlan.link
ID_NET_NAME=Wlan

This is the second time my mistake has caused systemd to lie!
Using the 10- rule forces a working connection, and wrong. The typo in the 42 link now corrected.
It correctly ignored many other rules...done!