Based on Debian Bullseye (11.6)
Latest 5.15 Kernel as stable default
Newer 6.2 kernel as opt-in
QEMU 7.2
LXC 5.0.2
ZFS 2.1.9
Ceph Quincy 17.2.5
Ceph Pacific 16.2.11
Highlights
Proxmox VE now provides a dark theme for the web interface.
Guests in resource tree can now be sorted by their name, not only VMID.
The HA Cluster Resource Scheduler (CRS) stack was expanded to rebalance VMs & CTs automatically on start, not only recovery.
Added CRM command to the HA manager to switch an online node manually into maintenance mode (without reboot.
Changelog Overview
Enhancements in the web interface (GUI)
Add a fully-integrated "Proxmox Dark" color theme variant of the long-time Crisp light theme.
By default, the prefers-color-scheme media query from the Browser/OS will be used to decide the default color scheme.
Users can override the theme via a newly added Color Theme menu in the user menu.
Add "Proxmox Dark" color theme to the Proxmox VE reference documentation.
The prefers-color-scheme media query from the Browser/OS will be used to decide if the light or dark color scheme should be used.
The new dark theme is also available in the Proxmox VE API Viewer.
Local storage types that are located on other cluster nodes can be added.
A node selector was added to the Add Storage wizard for the ZFS, LVM, and LVM-Thin storage types.
Automatically redirect HTTP requests to HTTPS for convenience.
This avoids "Connection reset" browser errors that can be confusing, especially after setting up a Proxmox VE host the first time.
Task logs can now be downloaded directly as text files for further inspection.
It is now possible to choose the sort-order of the resource tree and to sort guests by name.
Fix loading of changelogs in case additional package repositories are configured.
Improve editing of backup jobs:
Add a filter to the columns of the guest selector.
Show selected, but non-existing, guests.
Remove the "Storage View" mode from the resource tree panel.
This mode only showed the storage of a cluster and did not provide additional information over the folder or server views.
The Proxmox Backup Server specific columns for verification and encryption status can now be used for sorting in the backup content view of a storage.
Polish the user experience of the backup schedule simulator by splitting the date and time into two columns and better check the validity of the input fields.
Improve accessibility for screens with our minimal required display resolution of 720p
add scrolling overflow handler for the toolbar of the backup job view
rework the layout of the backup job info window for better space usage and reduce its default size
Fix search in "Guests without backup" window.
Node and Datacenter resource summary panels now show the guest tag column by default.
Show role privileges when adding permissions.
Allow the use of the `-` character in snapshot names, as the backend has supported this for some time.
Update the noVNC guest viewer to upstream version 1.4.0.
Fix overly-strict permission check that prevented users with only the VM.Console privilege from accessing the noVNC console.
Align permission check for bulk actions with the ones enforced by the API.
Switch the check from the Sys.PowerMgmt privilege to the correct VM.PowerMgmt one.
Invalid entries in advanced fields now cause the advanced panel to unfold, providing direct feedback.
HTML-encode API results before rendering as additional hardening against XSS.
Fix preselection of tree elements based on the URL after login.
Fix race condition when switching between the content panel of two storage before one of them hasn't finished loading.
Metric server: Expose setting the verify-certificate option for InfluxDB as advanced setting
Replace non-clickable checkbox with icons for backup jobs, APT repositories, and replication jobs.
Fix error when editing LDAP sync setting and only a single parameter is not set to a non-default value.
Add missing online-help references for various panels and edit windows.
Improved translations, among others:
Arabic
French
German
Italian
Japanese
Russian
Slovenian
Simplified Chinese
Virtual Machines (KVM/QEMU)
New QEMU Version 7.2:
QEMU 7.2 fixes issues with Windows Guests, installed from a German ISO, during installation of the VirtIO drivers.
Fix crash of VMs with iSCSI disks on a busy target.
Fix rare hang of VMs with IDE/SATA during disk-related operations like backup and resize.
Many more changes, see the upstream changelog for details.
Taking a snapshot of a VM with large disks following a PBS backup occasionally was very slow. This has been fixed (issue #4476).
Running fsfreeze/fsthaw before starting a backup can now optionally be disabled in the QEMU guest agent options.
Note: Disabling this option can potentially lead to backups with inconsistent filesystems and should therefore only be disabled if you know what you are doing.
Cloning or moving a disk of an offline VM now also takes the configured bandwidth limits into consideration (issue #4249).
Fix an issue with EFI disks on ARM 64 VMs.
Add safeguards preventing the moving of disks of a VM using io_uring to storage types that have problems with io_uring in some kernel versions.
General improvements to error reporting. For example, the error messages from query-migrate are added when a migration fails and a configured, but non-existing physical CD-ROM drive, results in a descriptive error message.
Allow users to destroy a VM even if it's suspended.
Fix a race-condition when migrating VMs on highly loaded or slower clusters, where the move of the guest's config file to the target node directory might not have been propagated to the target node.
Rolling back a VM to a snapshot with state (memory) and still selecting to start the VM after the rollback does not cause an error anymore (rollbacks with state result in a running VM).
Deleting snapshots of running VMs, with a configured TPM on Ceph storages with krbd enabled, is now possible.
Fix command execution via pvesh and QEMU guest agent in VMs on other cluster nodes.
Update Linux OS version description to include 6.x kernels.
Containers (LXC)
Update to LXC 5.0.2 and lxcfs 5.0.3.
Allow riscv32 and riscv64 container architectures through the binfmt_misc kernel capability.
After installing the qemu-user-static and binfmt-support packages one can use a RISC-V based rootfs image to run as container directly on an x86_64/amd64 Proxmox VE host.
Create /etc/hostname file on Alma Linux, CentOS, and Rocky Linux containers. With this, DHCP requests sent by the container now include its hostname.
Add option to disconnect network interfaces of containers, similarly to network interfaces of VMs.
Make container start more resilient after OOM or node crash (empty AppArmor profile files do not cause a crash).
Improve cleanup upon failed restores (remove the container configuration if restore fails due to an invalid source archive, remove firewall configuration).
Ignore bind or read-only mount points when running pct fstrim.
During container shutdown, wait with a timeout in case lxc-stop fails. This prevents the shutdown task from running indefinitely and having to be aborted manually.
Templates:
Updated Debian Bullseye template from 11.3 to 11.6.
Updated Proxmox Mail Gateway template from 7.0 to 7.2.
General improvements for virtual guests
The "Bulk Stop" action was renamed to "Bulk Shutdown" to better describe its behavior.
Allow overriding timeout and force-stop settings for bulk shutdowns.
Allow bulk actions even if the user does not have the required privileges for all guests but has the privileges for each guest involved in the bulk action.
HA Manager
Add CRM command to switch an online node manually into maintenance (without reboot).
When a node goes into maintenance mode all active HA services will be moved to other nodes, but automatically migrate them back once the maintenance mode is disabled again.
The HA Cluster Resource Scheduler (CRS) stack was expanded to rebalance VMs & CTs automatically on start, not only recovery.
One can now enable the ha-rebalance-on-start option in the datacenter.cfg or via the web UI to use Proxmox CRS to balance on service start up.
A new intermediate state request_started has been added for the stop -> start transitions of services.
Improve scheduling algorithm for some cases.
make CPU load matter more if there is no memory load at all
avoids boosting tiny relative differences over higher absolute loads.
use a non-linear averaging algorithm when comparing loads.
The previous algorithm was blind in cases where the static node stats are the same and there is (at least) one node that is over committed when compared to the others.
Improved management for Proxmox VE clusters
Ensure that the current working directory is not in /etc/pve when you set up the cluster using the pvecm CLI tool.
Since pmxcfs, which provides the mount point for /etc/pve, is restarted when you set up the cluster, a confusing "Transport endpoint is not connected" error message would be reported otherwise.
The proxmox-offline-mirror tool now supports fetching data through an HTTP proxy.
Fetching the changelog of package updates has been improved:
The correct changelog will be downloaded if repositories from multiple Proxmox projects are configured, for example if one has Proxmox VE and Proxmox Backup Server installed on the same host.
Support getting the for packages coming from a Debian Backports repository.
You can now configure if you want to receive a notification mail for new available package updates.
The wrapper for acme.sh DNS-validation plugins received fixes for 2 small issues:
a renaming of parameters for the acmedns plugin was pulled from upstream.
a missing method was added to fix an issue with the dns_cf.sh plugin.
Improved pvereport: In order to provide a better status overview, add the following information:
/etc/pve/datacenter.cfg.
ceph health detail.
OpenSSL errors are now reported in full to ease troubleshooting when managing the nodes certificate.
Add missing or newly added/split-out packages to the Proxmox VE apt version API, also used for the pveversion -v call:
proxmox-mail-forward
proxmox-kernel-helper
libpve-rs-perl
Backup/Restore
Suppress harmless but confusing "storing login ticket failed" errors when backing up to Proxmox Backup Server.
Storage
It is now possible to override the specific subdirectories for content (ISOs, container templates, backups, guest disks) to custom values through the content-dirs option.
The CIFS storage type can now also directly mount a specific subdirectory of a share, thus better integrating into already existing environments.
The availability check for the NFSv4 storage type was reworked in order to work with setups running without rpcbind.
Fix ISO upload via HTTP in a few edge cases (newlines in filenames, additional headers, not sent by common browsers).
Fix caching volume information for systems which both have a local ZFS pool storage and a ZFS over iSCSI storage configured during guest disk rescan.
Storage Replication
Extend support for online migration of replicated VM guests.
One can now also migrate VMs if they included snapshots, as long as those are only on replicated volumes.
Disk Management
Improve showing the SMART values for the correct NVMe devices.
Ceph
Expose more detailed OSD information through the API and use that to add an OSD Detail window in the web interface.
You can now check the backing device, logical volume info, front- and back- network addresses and more using the new OSD detail window.
Show placement groups per OSD in the web interface.
Improve schema description for various Ceph-related API endpoints.
This also improves the api-viewer and pvesh tool for various Ceph-related API endpoints.
Fix broken cmd-safety endpoint that made it impossible for non-root users to stop/destroy OSDs and monitors.
Allow admins to easily set up multiple MDS per node to increase redundancy if more than one CephFS is configured.
Access Control
ACL computation was refactored causing a significant performance improvement (up to a factor of 450) on setups with thousands of entries.
It is now possible to override the remove-vanished settings for a realm when actively syncing it in the GUI.
Allow quoted values in LDAP DN attributes when setting up an LDAP realm.
Firewall & Software Defined Networking
ipsets can be added even with set host-bits. For example, 192.0.2.5/24 is now a valid input. Host-bits get cleared upon parsing (resulting in 192.0.2.0/24 in the example).
Firewall logs can be restricted to a timeframe with the since and until parameters to the API call
The conditional loading of nf_conntrack_helpers was dropped for compatibility with kernel 6.1.
Not adding link-local IPv6 addresses on the internal guest-communication devices was fixed in a corner-case.
The MTU is now set to the value of the parent bridge on the automatically generated VLAN-bridge devices for non-VLAN-aware bridges.
The EVPN plugin now also merges a defined prefix-list from /etc/frr/frr.conf.local.
Installation ISO
the version of BusyBox shipped with the ISO was updated to version 1.36.0.
The EFI System Partition (ESP) defaults to 1 GiB of size if the root disk partition (hdsize) is bigger than 100 GB.
UTC can now be selected as timezone during installation.
Notable bug fixes
An issue with OVS network configuration where the node would lose connectivity when upgrading Open vSwitch (see
https://bugs.debian.org/1008684).
A race condition in the API servers causing failed tasks when running a lot of concurrent API requests was fixed.
Known Issues & Breaking Changes
In QEMU 7.2, it is a hard error if audio initialization fails rather than a warning.
This can happen, for example, if you have an audio device with SPICE driver configured but are not using SPICE display. To avoid the issue, make sure the configuration is valid.
With pve-edk2-firmware >= 3.20221111-1 we know of two issues affecting specific set ups:
virtual machines using OVMF/EFI with very little memory (< 1 GiB) and certain CPU types (e.g. host) might no longer boot.
Possible workarounds are to assign more memory or to use kvm64 as the CPU type.
The background for this problem is that OVMF << 3.20221111-1 used to guess the address (bit) width only from the available memory, but now there is more accurate detection that better matches what the configured CPU type provides. The more accurate address-width can lead to a larger space requirement for page tables.
The (non-default) PVSCSI disk controller might result in SCSI disk not being detected inside the guess in regressions.
We're still investigating this, until then you might either evaluate if your VM really requires the non-standard PVSCSI controller, use the SATA bus instead, or keep using the older pve-edk2-firmware package.
