Configure Proxmox server with OpenTofu for VM provisioning

Saturday, January 4th 2025  — 
 proxmox, opentofu, vm, deployment

This post is the first of my serie about how to configure and manage own dedicated server with the usage of open source tools:

Steps of this guide are inspired by the great Stéphane Robert's post (in french) and some other internet documentation resources.

Prerequisite

So before we start you must acquire a dedicated server (or run our own at home) and proceed with the Proxmox OS initial installation. You can easily find an offer from a cloud provider of your choice. For example, OVH have some interesting and pretty affordable servers with their Kimsufi / ECO products range.

Configure Proxmox server

Some initial configuration must be done on Proxmox system before deployment.

So let's securise a bit our host:

nano /etc/ssh/sshd_config

#In the config file change the default SSH port:
Port 12365 #for example

#And restart the SSH agent:
service ssh restart

#Change the default root password:
passwd root

Configure our network's host:

#Ensure that /etc/network/interfaces file have this line:
source /etc/network/interfaces.d/*

#Restart network service:
systemctl restart networking.service

More info about Proxmox SDN here.

In order to get an IP address for our VMs we will setup a simple zone with SNAT and DHCP documented here.

Optionally we can add an extra layer of security on the Proxmox firewall to authorise the access of SSH or the web dashboard only from specific whitelisted IPs. Leave me a comment if your are interested in such customizations.

Ok, let's install some required packages:

apt install console-setup
apt install libguestfs-tools -y

Create a dedicated role that will be used by opentofu service account (role_name to replace with the name of your choice):

pveum role add <role_name> -privs "Datastore.Allocate Datastore.AllocateSpace Datastore.Audit Pool.Allocate Sys.Audit Sys.Console Sys.Modify VM.Allocate VM.Audit VM.Clone VM.Config.CDROM VM.Config.Cloudinit VM.Config.CPU VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Console VM.Migrate VM.Monitor VM.PowerMgmt SDN.Use"

We can list all roles with this command:

pveum role list

Now, create a service account (user) that will be used for deployment with username and password of your choice:

pveum user add <username>@pve --password <password>

We can list all users with this command:

pveum user list

Assign the role to the user:

pveum aclmod / -user <username>@pve -role <role_name>

Notice: To ensure that everything works fine you can test the service account user by login on Proxmox web dashboard.

We must create a token that will be used to securely interact between Proxmox and Opentofu. To do so, type this command with the names of your choice:

pveum user token add <username>@pve <token_name> -expire 0 -privsep 0 -comment "<token_comment>"

We can list all tokens for a specific user with this command:

pveum user token list <username>@pve

Let's create our first VM template. The idea here is to download a server image on our Proxmox host and to pre-configure some parameters that we will use later:

cd /var/lib/vz/template/iso
wget https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img
sudo virt-customize -a ubuntu-22.04-server-cloudimg-amd64.img --install qemu-guest-agent
sudo virt-customize -a ubuntu-22.04-server-cloudimg-amd64.img --root-password password:changeme
sudo qm create 9999 --name "ubuntu.server.local" --memory 2048 --cores 2 --net0 virtio,bridge=vnet1
sudo sudo qm importdisk 9999 ubuntu-22.04-server-cloudimg-amd64.img local
qm set 9999 --tags "template,ubuntu"
sudo qm set 9999 --scsihw virtio-scsi-pci --scsi0 local:9999/vm-9999-disk-0.raw
sudo qm set 9999 --boot c --bootdisk scsi0
sudo qm set 9999 --ide2 local:cloudinit
sudo qm set 9999 --serial0 socket --vga serial0
sudo qm set 9999 --agent enabled=1
sudo qm set 9999 --ipconfig0 ip=dhcp
sudo qm template 9999

Our Proxmox environment is now ready ;-) You can see the created template under the Datacenter > server node menu on the web dashboard.

Configure our deployment environment

Now let's configure our operator for deployment. For this example a basic Ubuntu desktop distribution will be used.

Install Opentofu as a Terraform alternative:

sudo snap install --classic opentofu

Notice: We will use Opentofu instead of Terraform because of a change in license operated by HashiCorp. The Opentofu fork will remain open source and can be modifed freely. More info here.

For simplicity we can add an alias to type commands faster later:

echo "alias tf='tofu'" >> ~/.bash_aliases

Configure the SSH agent to work with Opentofu properly:

eval `ssh-agent -s`
ssh-add
cat ~/.ssh/authorized_keys
ssh-copy-id -i ~/.ssh/id_rsa.pub root@<proxmox_server_public_ip>

Opentofu like Terraform need some environment variables to work. For the sake of simplicity let's create _secret.sh file in home directory (avoid this on production environments):

export TF_VAR_endpoint="<proxmox_endpoint>"
export TF_VAR_api_token="<username>@pve!<token_name>=<token>"
export TF_VAR_tf_username="<username>"
export TF_VAR_tf_password="<password>"
export TF_VAR_ssh_agent_username="<ssh_agent_username>"
export TF_VAR_ssh_agent_password="<ssh_agent_password>"
export TF_VAR_target_node_name="<target_node_name>"
export TF_VAR_target_node_ip="<target_node_ip>"
export TF_VAR_target_port="<target_node_port>"

So now we can quickly load all our variables:

source ./_secret.sh

Write our deployment code

So here we are at the most interesting point. The power of IaC (Infrastructure as Code) is to keep things repeatable and bring us the possibility to manage infrastructure versioning in the same way as we do in software development.

Create a file called main.tf. This file will contains the most of our configuration with the Proxmox provider, the template to use and resources specifications.

terraform {
  required_providers {
    proxmox = {
      source = "bpg/proxmox"
      version = "0.51.1"
    }
  }
}

provider "proxmox" {
  endpoint = var.endpoint
  #api_token = var.api_token
  username = var.tf_username
  password = var.tf_password
  insecure = true # TODO: import server self-signed cert to disable it
  ssh {
    agent    = true
    username = var.ssh_agent_username
    password = var.ssh_agent_password
    #private_key = file("~/.ssh/id_rsa")
    node {
      name    = var.target_node_name
      address = var.target_node_ip
      port    = var.target_node_port
    }
  }
}

data "proxmox_virtual_environment_vms" "template" {
  node_name = var.target_node_name
  tags      = ["template", var.template_tag]
}

resource "proxmox_virtual_environment_file" "cloud_user_config" {
  content_type = "snippets"
  datastore_id = "local"
  node_name    = var.target_node_name

  source_raw {
    data = file("cloud-init/user_data")

    file_name = "${var.vm_hostname}.${var.domain}-ci-user.yml"
  }
}

resource "proxmox_virtual_environment_file" "cloud_meta_config" {
  content_type = "snippets"
  datastore_id = "local"
  node_name    = var.target_node_name

  source_raw {
    data = templatefile("cloud-init/meta_data",
      {
        instance_id    = sha1(var.vm_hostname)
        local_hostname = var.vm_hostname
      }
    )

    file_name = "${var.vm_hostname}.${var.domain}-ci-meta_data.yml"
  }
}

resource "proxmox_virtual_environment_vm" "vm" {
  name      = "${var.vm_hostname}.${var.domain}"
  node_name = var.target_node_name

  on_boot = var.onboot


  agent {
    enabled = true
  }

  tags = var.vm_tags

  cpu {
    type    = "host"
    cores   = var.cores
    sockets = var.sockets
    flags   = []
  }

  memory {
    dedicated = var.memory
  }

  network_device {
    bridge  = "vnet0"
    model   = "virtio"
  }

  # Ignore changes to the network
  ## MAC address is generated on every apply, causing
  ## TF to think this needs to be rebuilt on every apply
  lifecycle {
    ignore_changes = [
      network_device,
    ]
  }

  boot_order    = ["scsi0"]
  scsi_hardware = "virtio-scsi-single"

  disk {
    interface    = "scsi0"
    iothread     = true
    datastore_id = "${var.disk.storage}"
    size         = var.disk.size
    discard      = "ignore"
  }

  dynamic "disk" {
    for_each = var.additionnal_disks
    content {
      interface    = "scsi${1 + disk.key}"
      iothread     = true
      datastore_id = "${disk.value.storage}"
      size         = disk.value.size
      discard      = "ignore"
      file_format  = "raw"
    }
  }

  clone {
    vm_id = data.proxmox_virtual_environment_vms.template.vms[0].vm_id
  }

  initialization {
    # ip_config {
    #   ipv4 {
    #     address = "dhcp"
    #   }
    # }

    datastore_id         = "local"
    interface            = "ide2"
    user_data_file_id    = proxmox_virtual_environment_file.cloud_user_config.id
    meta_data_file_id    = proxmox_virtual_environment_file.cloud_meta_config.id
  }


}

As you can see, there are many values defined from variables to get a more maintainable code. Let's create a variables.tf file:

variable "endpoint" {
  description = "URL of Proxmox web interface"
  type = string
}

variable "api_token" {
  description = "Token to connect Proxmox API"
  type = string
}

variable "tf_username" {
  description = "Proxmox TF username"
  type = string
}

variable "tf_password" {
  description = "Proxmox TF password"
  type = string
}

variable "ssh_agent_username" {
  description = "Proxmox SSH agent username"
  type = string
}

variable "ssh_agent_password" {
  description = "Proxmox SSH agent password"
  type = string
}

variable "target_node_name" {
  description = "Proxmox node name"
  type        = string
}

variable "target_node_ip" {
  description = "Proxmox node IP"
  type        = string
}

variable "target_node_port" {
  description = "Proxmox node port"
  type        = string
}

variable "onboot" {
  description = "Auto start VM when node is start"
  type        = bool
  default     = true
}

variable "target_node_domain" {
  description = "Proxmox node domain"
  type        = string
  default = ""
}

variable "vm_hostname" {
  description = "VM hostname"
  type        = string
  default = "ourserver"
}

variable "domain" {
  description = "VM domain"
  type        = string
  default = "local"
}

variable "vm_tags" {
  description = "VM tags"
  type        = list(string)
  default = [ "ubuntu" ]
}

variable "template_tag" {
  description = "Template tag"
  type        = string
  default = "ubuntu"
}

variable "sockets" {
  description = "Number of sockets"
  type        = number
  default     = 1
}

variable "cores" {
  description = "Number of cores"
  type        = number
  default     = 6
}

variable "memory" {
  description = "Number of memory in MB"
  type        = number
  default     = 20480
}

variable "vm_user" {
  description = "User"
  type        = string
  sensitive   = true
  default = "sysadmin"
}

variable "disk" {
  description = "Disk (size in Gb)"
  type = object({
    storage = string
    size    = number
  })
  default = {
    storage = "local"
    size = 20
  }
}

variable "additionnal_disks" {
  description = "Additionnal disks"
  type = list(object({
    storage = string
    size    = number
  }))
  default = [
    {
      storage = "local"
      size = 180
    }
  ]
}

To finish our code we need to create a folder called cloud-init. Files inside will be used for the VM first start configuration. Create an empty file meta_data and another one called user_data with this content:

#cloud-config
hostname: "ourhost.me"
manage_etc_hosts: false
write_files:
  - path: /etc/hosts
    content: |
      127.0.1.1 ourhost.me
      127.0.0.1 localhost
      <proxmox_server_public_ip> ourhost.me
package_upgrade: true
users:
  - default
  - name: ubuntu
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ssh-rsa AAAAB...
ssh_pwauth: true ## This line enables ssh password authentication
timezone: Europe/Paris

Notice: More information about Opentofu can be found here.

Operate our server

So now we have (if our configuration is correct) the possibility to perform operations regarding our IaC configuration code:

# Initialise our configuration (dependencies downloading and global check)
tf plan

# Apply the desired state of ressources regarding our configuration (this command will show changes that will be made before approve)
tf apply

# Delete this configuration (add -auto-approve argument to avoid to do confirmations)
tf destroy

Here is the full repository with the discussed source code.

Conclusion

That's all folks! With this guide we have an overview of how to configure Proxmox with Opentofu and get a VM up and running in no time. Stay tuned for my next posts ;-)

Comments