Fine-Tuning Memory Allocation on Virtual Machines

I often end up using Virtual Machines as a part of my research. Every operating system has some pros and cons. Working on different operating systems sometimes helps create better PoCs since some of them have good built-in tools required for specific tasks, which on the contrary might require a manual set-up from scratch on a generic one. An example of this can be the difference between different home versions of Windows and Open Source Linux Distros. Thus, virtual machines help. However, setting up multiple VMs also creates performance issues on the host OS. This is something I have been frustrated with for quite some time. Over a while, I have discovered a methodology to fine-tune this scenario, and here is what I have learned:

The urge to end this article here by saying: “Just dual-boot”

But for whatever reason if you still want to install VM over the host OS, read up ahead

NOTE: It has been assumed that the user is using a type 2 or Hosted Hypervisor for this process like Oracle VirtualBox, VMware Workstation, and Parallels Desktop.

1. Memory Allocation Setting: When you create a new virtual machine in VirtualBox, you can specify the amount of memory (RAM) to allocate to it. Let’s say you’re setting up a new Windows 10 virtual machine. During the creation process, let’s say you allocate 4 GB of RAM to this VM.

Fig 1: Editing System Settings on a Ubuntu VM set-up on a Windows 11 host

2. Dynamic vs. Fixed Allocation: For the Windows 10 virtual machine, you can choose dynamic allocation. This means that although you’ve assigned 4 GB of RAM, the VM will use only what it needs initially. As you open applications and perform tasks within the VM, it can use more RAM, up to a maximum of 4 GB. This allows efficient use of resources, as the VM isn’t locked into using the entire allocated amount unless required.

Below the “Base Memory” setting, there might be an option labeled “Enable dynamic memory.” Check this option to enable dynamic memory allocation.

You’ll see a slider or an input field labeled “Base Memory” in Fig 1. This is the initial memory allocation for the VM. To enable dynamic memory allocation, make sure the “Enable I/O APIC” option is checked (if available).

3. Overcommitment: Let’s say you have 16 GB of physical RAM on your host machine. With overcommitment, you could create two virtual machines, both allocated 8 GB of RAM each, totaling 16 GB. However, if both VMs start using their full allocated amounts simultaneously, the host’s performance might suffer due to excessive swapping.

Regularly monitor VM performance and adjust CPU allocations based on actual usage patterns. Fine-tuning can optimize resource utilization.

4. Ballooning: Imagine you’re running multiple VMs, and one of them is a Linux server with 6 GB of allocated RAM. If this VM isn’t actively using all 6 GB, VirtualBox’s memory ballooning can help. The host OS communicates with the guest OS through VirtualBox tools, and if the host needs more memory for other tasks, it can request the guest OS to release some of its unused memory back to the host.

5. Resource Management: Suppose you have a scenario where you’re running a resource-intensive application on one VM and a less critical application on another VM. By adjusting VirtualBox’s priority settings, you can ensure that the VM running the critical application gets higher priority access to CPU and memory resources.

Base the number of CPUs on the workload of the VM. Some workloads, like web servers or lightweight applications, might not require many CPUs, while others like database servers might benefit from more CPUs.

The execution cap specifies how much of the host’s physical CPU capacity a VM can utilize. It’s often expressed as a percentage. Here’s how it affects performance:

• Unlimited (100%): If you set no execution cap, the VM can use up to 100% of the host’s CPU capacity when needed. This provides full access to the CPU but can lead to resource contention if multiple VMs are running on the same host.
• Capped (Less than 100%): Setting a lower execution cap limits the VM’s CPU usage. This can help prevent a single VM from monopolizing resources but might also impact performance if the cap is set too low.
• Reservation: A CPU reservation ensures that a certain amount of CPU capacity is always available to the VM. This can provide consistent performance but might lead to underutilization of resources if set too high.

6. Configuration Adjustments: As your needs change, you can modify the memory allocation of your VMs. For instance, if you find that your Windows 10 VM is running slowly due to memory constraints, you can power off the VM and increase its allocated memory from 4 GB to 6 GB. Keep in mind that you should have enough physical RAM on your host to accommodate these changes effectively.

A few other things which you might be curious about while setting this up:

Selecting Chipset, TPM, and boot order

The chipset selection affects how the virtual machine interacts with the virtualized hardware components, such as disk controllers, USB controllers, and more. For instance:

If you choose PIIX3, the virtual machine will use an older emulation of hardware components. This might be useful for compatibility with legacy systems but could limit access to advanced features available on newer hardware.

If you choose ICH9, the virtual machine will use a more modern emulation that supports newer hardware features. This is generally recommended for running modern operating systems and applications that expect advanced hardware capabilities.

ICH9 is a safer and more feature-rich choice. However, if you’re dealing with older software or specific compatibility requirements, you might opt for PIIX3.

TPM stands for “Trusted Platform Module.” TPM emulation or passthrough refers to VirtualBox’s capability to simulate or provide access to a virtual TPM device within the virtualized environment. It enhances security by providing features like secure boot, encryption, and key management.

Boot-Order:

For a general-purpose setup, consider this boot order:

1. Optical Drive (CD/DVD)
2. Hard Disk
3. Network (PXE)
4. Floppy (if Applicable)

However, remember that these recommendations can change based on your use case. For example:

• If you’re setting up a VM for a production environment, you’d likely want to prioritize the hard disk for faster and more reliable boots after the initial installation.
• If you’re working in a network environment where you deploy virtual machines automatically, network boot might be more relevant.
• If you’re installing from an ISO image and want the VM to boot directly from it, the optical drive should be the first option.

Paravirtualization Interface: A paravirtualization interface is a specialized communication channel or set of APIs (Application Programming Interfaces) that allows a virtual machine (VM) to interact directly with the underlying virtualization platform or hypervisor. This reduces the overhead associated with traditional virtualization methods, where the hypervisor has to emulate hardware devices for the guest OS.

Paravirtualization generally requires installing specific virtualization software or components on both the host and guest systems.

Selecting between different paravirtualization interfaces

Legacy virtualization technologies like VMware Workstation, Oracle VirtualBox, or Parallels Desktop are often used for desktop virtualization, development, testing, and running multiple operating systems on personal computers. Not designed for large-scale production environments.

Minimal OS installations refer to lightweight operating system installations that provide only essential components.

KVM is a popular open-source virtualization solution commonly used in Linux environments. It’s suitable for a wide range of scenarios from small to large-scale environments.

Hyper-V is a virtualization solution developed by Microsoft primarily for Windows environments. It’s suitable for businesses that heavily rely on Microsoft technologies.

Both the hypervisor and the guest OS need to support the paravirtualization interface for it to work.

When you enable nested paging, you allow the virtualization software (such as a hypervisor) to use the hardware’s built-in paging mechanisms for both the outer VM (host VM) and the inner VM (guest VM). This helps reduce the need for the hypervisor to translate memory addresses between the host and guest VMs. Without nested paging, the hypervisor would need to perform additional translations, leading to increased overhead and potentially slower performance.

PAE/NX: Enabling PAE/NX involves configuring specific hardware features in your virtualization platform or BIOS/UEFI settings to enhance the memory management capabilities of your virtual machines. Here’s what PAE/NX means and how to enable it:

Physical Address Extension (PAE):
PAE is a memory management feature that allows 32-bit processors to access more than 4 GB of physical memory. Without PAE, 32-bit systems are limited to addressing up to 4 GB of RAM. With PAE enabled, they can access more memory, although individual applications may still be limited by their architecture.

No-eXecute (NX):
NX, also known as Execute Disable (XD) or Data Execution Prevention (DEP), is a security feature that prevents the execution of malicious code in certain memory areas. It marks memory regions as non-executable to reduce the risk of buffer overflow attacks.

Enabling nested VT-x, also known as Nested Virtualization or Nested Virtualization Extensions, involves enabling the hardware virtualization support within a virtual machine (VM) as mentioned earlier in this write-up.

In summary, adjusting settings like memory allocation, dynamic vs. fixed allocation, overcommitment, ballooning, and resource prioritization can have a significant impact on the performance and stability of your virtualized environment. Regularly evaluating and fine-tuning these configurations based on your VM usage patterns and host hardware capabilities will help you achieve optimal results.