Podman in Incus container

Got this error:-

ERRO[0000] running `/usr/bin/newuidmap 738 0 1000 1 1 524288 65536`: newuidmap: write to uid_map failed: Operation not permitted Error: cannot set up namespace using "/usr/bin/newuidmap": should have setuid or have filecaps setuid: exit status 1

This is after setting the nesting config to true:-

incus config set <container_name> security.nesting "true"

We also need to set subid/subgid mapping inside the container:-

incus exec <container_name> -- sh -c "echo 'ubuntu:100000:65536' > /etc/subuid"
incus exec <container_name> -- sh -c "echo 'ubuntu:100000:65536' > /etc/subgid"

The newuidmap and newgidmap binaries must have explicit permissions to manipulate user and group namespaces. This fixes the Operation not permitted: should have setuid/setgid error.

incus exec <container_name> -- setcap cap_setuid+ep /usr/bin/newuidmap
incus exec <container_name> -- setcap cap_setgid+ep /usr/bin/newgidmap

To make the config easier, I put this in a cloud-init config to properly set up the container.

#cloud-config
users:
  - name: ubuntu
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: wheel
    shell: /usr/bin/zsh
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINEWBrd0A8TxQMU464x0DpU7q5rTkOJr0v/IWiub5h56 kamal@x1

packages:
  - zsh
  - git
  - vim
  - curl
  - openssh-server
  - util-linux-user       # for the `chsh` command
  - fira-code-fonts       # optional Nerd Font (remove if you don’t want it)
  - shadow-utils
  - podman
  - libcap

runcmd:
  - chown -R ubuntu:ubuntu /home/ubuntu
  - chmod 700 /home/ubuntu/.ssh
  - chmod 600 /home/ubuntu/.ssh/authorized_keys
  # 1. Force a temporary password ("temp") to initialize the account state.
  # This inserts the required hash into /etc/shadow.
  - echo "ubuntu:temp" | chpasswd

  # 2. Immediately remove the password hash (delete the password).
  # This makes the account passwordless while preserving the 'activated' state.
  - passwd -d ubuntu
  - systemctl enable sshd
  - systemctl start sshd
  - su - ubuntu -c 'sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" -- --unattended'
  - su - ubuntu -c "git clone https://github.com/zsh-users/zsh-autosuggestions ~/.oh-my-zsh/custom/plugins/zsh-autosuggestions"
  - su - ubuntu -c "git clone https://github.com/zsh-users/zsh-completions ~/.oh-my-zsh/custom/plugins/zsh-completions"
  - su - ubuntu -c "git clone https://github.com/zsh-users/zsh-syntax-highlighting ~/.oh-my-zsh/custom/plugins/zsh-syntax-highlighting"
  - su - ubuntu -c 'echo "export PATH=\"\$HOME/.local/bin:\$PATH\"" >> /home/ubuntu/.zshrc'
  - su - ubuntu -c "mkdir -p /home/ubuntu/.local/bin"
  - su - ubuntu -c "curl -sS https://starship.rs/install.sh | sh -s -- -y --bin-dir /home/ubuntu/.local/bin"
  - su - ubuntu -c 'echo "eval \"\$(starship init zsh)\"" >> /home/ubuntu/.zshrc'
  - su - ubuntu -c "sed -i 's/^plugins=(git)/plugins=(git zsh-autosuggestions zsh-completions zsh-syntax-highlighting)/g' /home/ubuntu/.zshrc"
  - su - ubuntu -c "curl -LsSf https://astral.sh/uv/install.sh | sh"
  - su - ubuntu -c "curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash"
  # Define variables for clarity
  - USERNAME="ubuntu"
  - START_ID="100000"
  - COUNT="65536"

  # --- Subordinate ID Mapping ---
  # Create/overwrite the required subordinate UID/GID entries for the default user.
  - sh -c "echo \"$USERNAME:$START_ID:$COUNT\" >> /etc/subuid"
  - sh -c "echo \"$USERNAME:$START_ID:$COUNT\" >> /etc/subgid"

  # --- Binary Capabilities Fix ---
  # Fix for 'newuidmap': grants capability to set UIDs.
  - sh -c "setcap cap_setuid+ep /usr/bin/newuidmap"

  # Fix for 'newgidmap': grants capability to set GIDs.
  - sh -c "setcap cap_setgid+ep /usr/bin/newgidmap"

  # --- Incus Nesting Dependencies (Optional but recommended for Fedora/Podman) ---
  # Ensure the necessary kernel modules are available for unprivileged overlay mounts
  - modprobe overlay
  - modprobe fuse

Now we can launch the container:-

incus launch images:fedora/42/cloud tmp --config=user.user-data="$(cat fedora-42.yml)" --config security.nesting=true

And test running podman:-

incus exec <container_name> -- su - ubuntu -c "podman run hello-world"

Additional notes

To check the status of cloud-init execution inside the container:-

cloud-init status
tail -f /var/log/cloud-init-output.log