Proxmox VE 9 architecture: the architect's guide

A virtualisation platform is not designed by picking servers: it is designed by answering five questions in order, and accepting what each answer rules out afterwards. Here are those questions, the rules that follow from them, and the figure each one puts on your annual bill.

What you are leaving, and what you take with you #

VMware's licensing change moved whole estates towards open source. The shift is real, but it turns on a difference of kind, not of price: with VMware you buy a licence counted per core, with a billed minimum per processor; with Proxmox you buy a subscription counted per socket, and the software runs without it. Without a subscription you lose access to the validated update repository and to support, not the platform.

This guide assumes the decision in principle is made. If it is not, the comparison of the options after the takeover and the migration handbook each cover their own moment.

The five questions that decide everything else #

An architect who starts from the server catalogue has the order wrong. These five answers constrain everything that follows, and configuration will not fix them once the hardware is delivered.

The node: you count sockets, not servers #

A Proxmox subscription is paid per physical CPU socket, per year. That sentence decides your operating budget, and it has a consequence many discover after ordering.

A single-socket server with a 96-core processor costs one subscription. Two 48-core processors in the same chassis cost two, for the same compute. The market routinely sells single-socket machines from 64 to 192 cores with twelve memory channels: at equal performance, the annual bill can double depending on how you fill the chassis.

• Never plan a node beyond 80 % of its CPU or 80 % of its memory. That margin is not comfort: it is what absorbs the load of a failed node's machines.
• Overcommitting the CPU is normal, overcommitting memory is not. Virtual machines do not all use their processor at the same moment; they do hold their memory permanently.
• The overcommit ratios we apply: 3 vCPU per physical core for a mixed load (web, apps, services), 1.5 for databases or any latency-sensitive load, 4 for development and testing.

What storage takes before the first virtual machine #

This is the most expensive sizing mistake, because it only shows up in the room. Distributed storage reserves memory and cores on every node, before a single virtual machine starts, and that need does not depend on the machines' memory: it depends on the number of disks and the raw capacity.

The plan that held on screen and not in the room #

A measured example, because an order of magnitude convinces better than a rule. Forty machines of 2 vCPU, 4 GB and 2 TB, an ordinary file server. A model that covered storage with a flat 20 % margin on the machines' memory proposed three 128 GB nodes. That cluster's storage alone demands roughly 366 GB of memory per node.

The flat margin could not work: it is proportional to the machines' memory, while the real need depends on the number of disks and the capacity. The two quantities vary independently, and the gap becomes absurd as soon as you have a lot of disk and little memory.

Calculate this item separately, always, before choosing memory module sizes.

Choosing storage: local, replicated, distributed or shared #

Four families, four trade-offs. The choice turns on acceptable data loss, not on performance: all four are fast with NVMe disks.

The 66 % rule on Ceph #

Do not fill a Ceph cluster beyond two thirds of its usable capacity. Past roughly 70 %, rebuilding after a lost disk becomes risky: the cluster has to copy the missing data somewhere, and if it runs out of room while repairing itself, it stops.

A full Ceph cluster is not slow, it is blocked. Budget that reserve when you order the disks, not when the alert arrives.

The networks that never share a cable #

Three flows, three natures, three requirements. What is decided here cannot be fixed by configuration once the cards are ordered.

Why the cluster network refuses a bond #

Nodes talk constantly to know who is alive. That conversation manages its own redundancy: give it two independent links, on two different cards, and it fails over by itself.

A bond only protects against a cable or port failure. It does not protect against a switch misconfiguration, which takes both strands at once because they are seen as one logical link. That is exactly the scenario that reboots nodes with no explanation.

Two links, two cards, two switches. One gigabit per link is enough: this traffic is tiny, but it is latency-sensitive. Never let it share a cable with storage.

Cabling, and the switch it implies #

The cable follows from distance and speed, and it changes the price of the port as much as the price of the link.

• Passive copper (DAC): up to 3 to 5 metres, cheapest, no optics. For nodes in the same rack or up to the top-of-rack switch, this is the default.
• Active optical cable (AOC): up to a few tens of metres, module built into both ends, nothing to assemble.
• Fibre with separate modules: beyond that, and whenever the link crosses a room or a building. The module then has to be accepted by the switch, which is not a given: see the guide to vendor coding.
• Jumbo frames (MTU 9000) on the storage zone, enabled end to end. A jumbo frame configured on one side only does not produce a clear error, it produces intermittent loss.
• Link aggregation across two switches so that a node's bond survives losing a whole chassis.

The case where the switch disappears #

Below four nodes, you can wire the servers directly to each other and skip the fast switch, whose price often exceeds that of the cards. This topology has limits, and one of them is a classic trap: direct mesh without a switch explains how far it holds and why the ring costs dearly.

The hypervisor firewall #

Proxmox ships a three-tier firewall: the whole cluster, each node, each virtual machine. Rules flow down from the upper tier, and each tier can be enabled independently.

• Aliases and address sets are defined once and reused. A rule holding a hardcoded address goes stale without warning.
• Security groups bundle a reusable set of rules, to be applied to machines of the same role rather than copied out.
• Connection logging is what makes a rule verifiable. Without it you will know a flow is blocked, never why.
• ⚠️ Enabling the firewall at cluster level without first writing the rule that allows your own admin network locks you out. Write the rule, check it, then enable.

Hardening the node before it goes to the room #

A compromised hypervisor takes every machine it hosts with it. That is why it is not protected like an ordinary server.

• Management lives in its own network zone, separate from the virtual machines. If an attacker takes a machine, they must not find the hypervisor's admin interface at the end of the same cable.
• Create a separate admin account. The installer creates none and leaves access to the root account: fix that on day one.
• Restrict the web interface and remote access to named networks. Not to the whole world, even behind a long password.
• Apply processor microcode updates. They fix hardware flaws the operating system cannot work around alone.

GPU: three ways to share a card, three things given up #

The most common need today is training and running artificial intelligence models. Three mechanisms exist, and each forbids something. All of them require I/O virtualisation (IOMMU, VT-d on Intel, AMD-Vi on AMD) to be enabled in the BIOS or UEFI.

What card sharing changes in the sizing #

A card with sixteen compute units and 64 GB of memory splits, for instance, into four virtual cards of four units and 16 GB. The same principle exists on network cards, where one physical card exposes several virtual functions, each assignable to a machine.

These cards have two consequences the node arithmetic often ignores: they draw a lot of current, and they impose a cooling regime not every room can hold. Have the power and the cooling priced at the same time as the nodes, not afterwards. The density constraint, air or liquid, is covered in the GPU hosting guide.

The five levels of resilience, from RPO to RTO #

Two words settle this section. RPO is how much data you accept losing, measured in time: an RPO of fifteen minutes means an incident can erase the last quarter hour of work. RTO is how long you accept being down.

These two figures are not technical, they are contractual. Have management write them down before choosing storage: they decide, not the other way round. The general ladder, in hosting vocabulary, is covered in the continuity plan guide. The table below translates it into Proxmox mechanisms, with the orders of magnitude each one holds.

The witness site, and why it is not optional #

A cluster decides by majority. With two sites of equal weight, losing the link between them creates two halves that each believe they are the survivor and each keep writing. The data diverges, and reconciliation is done by hand.

The third decision point, which hosts nothing, exists to break the tie. It can be a tiny machine elsewhere. On an even number of nodes, the same role is held by a dedicated lightweight component whose only job is to vote.

That detail is the difference between a continuity plan and two platforms contradicting each other. Cluster sizing covers it in detail.

Backup lives outside the cluster #

A backup stored in the cluster it protects is not a backup. The dedicated backup server deduplicates and compresses: budget roughly twice the useful data to keep about thirty days, a deliberately cautious assumption.

• Three copies, two media, one off site. The rule is old and it still holds.
• ⚠️ The direction in which data moves changes the nature of the protection. In the usual direction the site pushes to the remote copy, so it holds the remote copy's credentials. An attacker who takes the site finds them, and destroys the backup along with the rest.
• In the opposite direction, the remote server pulls. The local site stores no credentials for the remote server. An attacker who takes the entire local site has nothing to reach the copy with. That is the real argument against ransomware, and it costs nothing but a change of direction.
• Tape and object storage remain useful for long retention and for a genuinely disconnected copy.

Software-defined networking when a flat network stops being enough #

As long as every machine shares one addressing plan, a bridge and VLANs will do. Three situations cross that line: several tenants to isolate from one another, a layer 2 network to stretch between nodes without touching the physical switch, and address management to centralise.

Proxmox answers with a software-defined networking layer built on three objects: the zone, which carries the technology, the virtual network that lives inside it, and the subnet that carries the addressing. Technologies range from a simple bridge to VLAN, VXLAN and EVPN with a routing controller.

This layer requires the physical network to carry larger frames, since it encapsulates. Plan for it when ordering switches, not afterwards.

Subscriptions: the constraint that costs money when ignored #

Four levels exist, from the validated update repository without tickets up to support answering within two business hours. The price is counted per socket per year, on the vendor's public grid.

⚠️ Inside a cluster, every node must carry the same level. You cannot put a high subscription on one node and leave the others with nothing. As soon as one node has one, all of them must have one, at the same level. Otherwise all of them become invalid, and you lose access to the validated repository until it is put right.

This is the budget line most often underestimated in a comparison against the incumbent. Count the sockets across all nodes, multiply, then compare.

Where to put the platform: on your premises, in a data centre, or both #

The hardware is the same in all three cases. What changes is who holds the power, the cooling, the physical security and the link, and at what price.

The most useful hybrid is not the one people picture #

The hybrid that gets sold is usually half the production at the customer's site and half in a data centre. The one that actually protects is simpler: production on one side, the backup copy on the other, in the direction where the remote site does the pulling.

That arrangement costs a part rack and a link. It covers the incident that takes a building, which is the scenario no internal redundancy can do anything about. And it goes in without touching the existing production.

If you only do one thing after reading this guide, do that one.

Two clusters sometimes beat one stretched cluster #

A cluster stretched across two buildings inherits both: the link's latency becomes the latency of the conversation between nodes, and a fibre cut becomes a partition. From two sites upwards it is often healthier to run two independent clusters rather than one stretched one.

Proxmox provides a separate manager for this, built explicitly to drive several clusters and machines belonging to none. It centralises the view, the accounts and the permissions, instead of granting access node by node.

⚠️ Migrating a machine from one cluster to another is still announced as experimental. On Proxmox VE itself it exists only on the command line and not in the interface; it does appear in the multi-cluster manager's interface. Treat it as a tool for planned transfers, not as an automatic failover mechanism: your recovery plan must rest on backup and replication, not on it.

Running the numbers on your estate #

All these rules can be computed, and the calculation is right below. Give your number of machines and their average size: you get the number of nodes, what storage reserves on each one before the first virtual machine, the subscriptions on the vendor's public grid, and the height to host.

The result lives in the page address: you can paste it into a ticket or send it to your integrator, and the calculation is open without an account. Our Digital Architect runs the same engine and goes further, with the continuity levels, the mapping from VMware and the pre-order checklist; that dossier requires opening one.

Finding the hands that will build it #

A priced plan does not rack a server. Three delivery models exist, and a serious integrator handles all three: turnkey in a data centre, where they order, install, test and hand you the credentials; on your premises, working in your room and with your constraints; hybrid, production on one side and the backup copy on the other.

Ask for all three quotes rather than one. The gap between them will tell you what you are really paying for the room, and it is often the opposite of what people assume.

Providers who work in this area declare themselves in their own account: they tick Integration and private cloud under their activity, and the declaration then goes through validation. You will find them in the provider directory.

⚠️ A skill is ticked, never inferred. Nobody is presented here as doing virtualisation because they do managed hosting. If the directory shows no one in your area yet, it is because no one has declared themselves there, and we would rather say so than paper over it.

FAQ #

Written on 15 September 2026.

From reading to comparing: relevant data centers

TG

Telehouse - Paris 2 (Voltaire - Léon Frot)

Paris · 356 networks on site · certified ISO 27001, PCI DSS

See the exact price →

DR

Digital Realty Marseille MRS1/2/3/4

Marseille · 218 networks on site

View the listing →

EI

Equinix PA2 - Paris, Saint-Denis

Paris · 146 networks on site · certified HDS, ISO 22301

View the listing →

EI

Equinix PA3 - Paris, Saint-Denis

Paris · 122 networks on site · certified HDS, ISO 22301

View the listing →

UL

UltraEdge Lyon-Venissieux

Vénissieux · 83 networks on site · certified ISO 27001, ISO 50001

View the listing →

Estimate my cost →   Compare data centers

Other guides

• Colocation vs cloud vs dedicated: the real 3-year TCO
• After the VMware/Broadcom price hikes: cloud, on-prem, or colocation?
• Why your colocation bill doubles at renewal (and how to avoid it)
• Where to host your own hardware: the colocation guide

Guides · Expert answers →