Refactor Run class.

When using capture=True together with subprocess.run, interactive
session will not work, as all the output will be captured. Perfectly
well seen with `boxpy ssh`. This commit will fix this issue.

README.rst

@@ -2,8 +2,8 @@
 box.py
 ======
 
-Box.py is a simple automation tool meant to run Ubuntu, Fedora or Centos Stream
-cloud images on top of VirtualBox.
+Box.py is a simple automation tool meant to run Ubuntu, Fedora, Centos Stream
+or Debian cloud images on top of VirtualBox.
 
 What it does is simply download official cloud image, set up VM, tweak it up
 and do the initial pre-configuration using generated config drive.
@@ -16,7 +16,7 @@ weird named options for ``vboxmanage`` ;P)
 Requirements
 ------------
 
-- Python >=3.7
+- Python >=3.8
 
   - `pyyaml`_
   - `requests`_
@@ -24,30 +24,61 @@ Requirements
 - Virtualbox (obviously)
 - ``mkisofs`` or ``genisoimage`` command for generating ISO image
 - ``wget`` command for fetching images
-- ``sha256sum`` command for checksum check
+- ``sha256sum`` and ``sha512sum`` commands for checksum check
 - ``qemu-img`` from *qemu-utils* package command for converting between images
   formats
 
 
+Tested distros
+--------------
+
+- Ubuntu
+  - 18.04
+  - 20.04
+  - 22.04
+  - 24.04
+- Fedora
+  - 37
+  - 38
+  - 39
+  - 40
+  - 41
+- Centos Stream
+  - 8
+  - 9
+- Debian
+  - 10 (buster)
+  - 11 (bullseye)
+  - 12 (bookworm)
+  - 13 (trixie) - prerelease
+
+There is possibility to use whatever OS image which supports cloud-init. Use
+the ``--image`` param for ``create`` command to pass image filename, although
+it's wise to at least discover (or not, but it may be easier in certain
+distributions) what username is supposed to be used as a default user and pass
+it with ``--username`` param.
+
+
 How to run it
 -------------
 
 First, make sure you fulfill the requirements; either by using packages from
-your operating system, or by using virtualenv for Python requirements, i.e.:
+your operating system, or by using virtualenv, i.e.:
 
 .. code:: shell-session
 
    $ python -m virtualenv .venv
    $ . .venv/bin/activate
-   (.venv) $ pip install requirements.txt
+   (.venv) $ pip install .
 
-then you can issue:
+You'll have ``boxpy`` command created for you as well.
 
 .. code:: shell-session
 
-   $ alias boxpy='python /path/to/box.py'
+   $ boxpy -V
+   boxpy 1.9.2
 
-or simply link it somewhere in the path:
+Other option is simply link it somewhere in the path:
 
 .. code:: shell-session
 
@@ -55,25 +86,25 @@ or simply link it somewhere in the path:
    $ chmod +x ~/bin/boxpy
 
 and now you can issue some command. For example, to spin up a VM with Ubuntu
-18.04 with one CPU, 2GB of memory and 10GB of disk:
+20.04 with one CPU, 1GB of memory and 6GB of disk:
 
 .. code:: shell-session
 
-   $ boxpy create --version 18.04 myvm
+   $ boxpy create --version 20.04 myvm
 
 note, that Ubuntu is default distribution you don't need to specify
 ``--distro`` nor ``--version`` it will pick up latest LTS version. Now, let's
-recreate it with 20.04:
+recreate it with 22.04:
 
 .. code:: shell-session
 
-   $ boxpy rebuild --version 20.04 myvm
+   $ boxpy rebuild --version 22.04 myvm
 
 or recreate it with Fedora and add additional CPU:
 
 .. code:: shell-session
 
-   $ boxpy rebuild --distro fedora --version 34 --cpu 2 myvm
+   $ boxpy rebuild --distro fedora --version 39 --cpu 2 myvm
 
 now, let's connect to the VM using either ssh command, which is printed out at
 as last ``boxpy`` output line, or simply by using ssh boxpy command:
@@ -91,17 +122,23 @@ use it ad-hoc, or place on your ``.bashrc`` or whatever:
 
 Currently, following commands are available:
 
-- ``list`` - for quickly listing all/running VMs
-- ``info`` - to get summary about VM
-- ``destroy`` - that is probably obvious one
+- ``completion`` - as described above
 - ``create`` - create new VM
+- ``destroy`` - that is probably obvious one
+- ``info`` - to get summary about VM
+- ``list`` - for quickly listing all/running VMs
 - ``rebuild`` - recreate specified VM
 - ``ssh`` - connect to the VM using ssh
-- ``completion`` - as described above
+- ``start`` - stop the running VM
+- ``stop`` - start stopped VM
 
 All of the commands have a range of options, and can be examined by using
 ``--help`` option.
 
+
+YAML Configuration
+------------------
+
 What is more interesting though, is the fact, that you can pass your own
 `cloud-init`_ yaml file, so that VM can be provisioned in easy way.
 
@@ -203,6 +240,19 @@ configuration additional NIC for virtual machine, i.e:
      advanced:
        nic2: intnet
 
+To select image from local file system, it is possible to set one by providing
+it under ``boxpy_data.image`` key:
+
+.. code:: yaml
+
+   …
+   boxpy_data:
+     image: /path/to/the/qcow2/image
+     default_user: cloud-user
+
+Note, that default_user is also needed to be provided, as there is no guess,
+what is the default username for cloud-init configured within provided image.
+
 
 License
 -------

box.py

@@ -18,14 +18,13 @@ import requests
 import yaml
 
 
-__version__ = "1.3"
+__version__ = "1.11.1"
 
 CACHE_DIR = os.environ.get('XDG_CACHE_HOME', os.path.expanduser('~/.cache'))
 CLOUD_IMAGE = "ci.iso"
-FEDORA_RELEASE_MAP = {'32': '1.6', '33': '1.2', '34': '1.2'}
 TYPE_MAP = {'HardDisk': 'disk', 'DVD': 'dvd', 'Floppy': 'floppy'}
 DISTRO_MAP = {'ubuntu': 'Ubuntu', 'fedora': 'Fedora',
-              'centos': 'Centos Stream'}
+              'centos': 'Centos Stream', 'debian': 'Debian'}
 META_DATA_TPL = string.Template('''\
 instance-id: $instance_id
 local-hostname: $vmhostname
@@ -46,9 +45,9 @@ ssh:
   emit_keys_to_console: false
 boxpy_data:
   cpus: 1
-  disk_size: 10240
+  disk_size: 6144
   key: ~/.ssh/id_rsa
-  memory: 2048
+  memory: 1024
 '''
 COMPLETIONS = {'bash': '''\
 _boxpy() {
@@ -125,7 +124,7 @@ _boxpy() {
         fi
     fi
 
-    opts="create destroy rebuild info list completion ssh"
+    opts="create destroy rebuild info list completion ssh start stop"
     if [[ ${cur} == "-q" || ${cur} == "-v" || ${COMP_CWORD} -eq 1 ]] ; then
         COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
         return 0
@@ -137,11 +136,36 @@ _boxpy() {
                 COMPREPLY=( $(compgen -W "bash" -- ${cur}) )
             fi
             ;;
-        create|rebuild)
-            items=(--cpus --disable-nested --disk-size --distro --forwarding
-                --key --memory --hostname --port --config --version --type)
+        start)
+            items=(--type)
             if [[ ${prev} == ${cmd} ]]; then
-                if [[ ${cmd} = "rebuild" ]]; then
+                if [[ ${cmd} = "start" ]]; then
+                    _vms_comp vms
+                else
+                    COMPREPLY=( $(compgen -W "${items[*]}" -- ${cur}) )
+                fi
+            else
+                _get_excluded_items "${items[@]}"
+                COMPREPLY=( $(compgen -W "$result" -- ${cur}) )
+
+                case "${prev}" in
+                    --type)
+                        COMPREPLY=( $(compgen -W "gui headless sdl separate" \
+                            -- ${cur}) )
+                        ;;
+                    --*)
+                        COMPREPLY=( )
+                        ;;
+                esac
+            fi
+
+            ;;
+        create|rebuild)
+            items=(--cpus --disable-nested --disk-size --default-user --distro
+                --forwarding --image --key --memory --hostname --port --config
+                --version --type)
+            if [[ ${prev} == ${cmd} ]]; then
+                if [[ ${cmd} = "rebuild" || ${cmd} == "start" ]]; then
                     _vms_comp vms
                 else
                     COMPREPLY=( $(compgen -W "${items[*]}" -- ${cur}) )
@@ -159,8 +183,8 @@ _boxpy() {
                         _ssh_identityfile
                         ;;
                     --distro)
-                        COMPREPLY=( $(compgen -W "ubuntu fedora centos" \
-                                -- ${cur}) )
+                        COMPREPLY=( $(compgen -W "ubuntu fedora centos
+                            debian" -- ${cur}) )
                         ;;
                     --type)
                         COMPREPLY=( $(compgen -W "gui headless sdl separate" \
@@ -193,6 +217,25 @@ _boxpy() {
                 _vms_comp runningvms
             fi
             ;;
+        stop)
+            items=(--poweroff)
+            if [[ ${prev} == ${cmd} ]]; then
+                if [[ ${cmd} = "stop" ]]; then
+                    _vms_comp runningvms
+                else
+                    COMPREPLY=( $(compgen -W "${items[*]}" -- ${cur}) )
+                fi
+            else
+                _get_excluded_items "${items[@]}"
+                COMPREPLY=( $(compgen -W "$result" -- ${cur}) )
+
+                case "${prev}" in
+                    --*)
+                        COMPREPLY=( )
+                        ;;
+                esac
+            fi
+            ;;
     esac
 
 }
@@ -232,28 +275,33 @@ class Run:
     Helper class on subprocess.run()
     command is a list with command and its params to execute
     """
-    def __init__(self, command, capture_output=True):
+    def __init__(self, command, capture=True):
         result = subprocess.run(command, encoding='utf-8',
-                                capture_output=capture_output)
-        if result.stdout:
-            LOG.debug2(result.stdout)
-        if result.stderr:
-            LOG.debug2(result.stderr)
+                                capture_output=capture)
 
         self.returncode = result.returncode
         self.stdout = result.stdout.strip() if result.stdout else ''
         self.stderr = result.stderr.strip() if result.stderr else ''
 
+        if self.stdout:
+            LOG.debug2(self.stdout)
+        if self.stderr:
+            LOG.debug2(self.stderr)
+
 
 class BoxError(Exception):
     pass
 
 
-class BoxNotFound(BoxError):
+class BoxNotFoundError(BoxError):
     pass
 
 
-class BoxVBoxFailure(BoxError):
+class BoxVBoxError(BoxError):
+    pass
+
+
+class BoxConfError(BoxError):
     pass
 
 
@@ -348,23 +396,26 @@ class FakeLogger:
 
 class Config:
     ATTRS = ('cpus', 'config', 'creator', 'disable_nested', 'disk_size',
-             'distro', 'forwarding', 'hostname', 'key', 'memory', 'name',
-             'port', 'version')
+             'distro', 'default_user', 'forwarding', 'hostname', 'image',
+             'key', 'memory', 'name', 'port', 'version', 'username')
 
     def __init__(self, args, vbox=None):
         self.advanced = None
         self.distro = None
+        self.default_user = None
         self.cpus = None
         self.creator = None
         self.disable_nested = 'False'
         self.disk_size = None
         self.forwarding = {}
         self.hostname = None
+        self.image = None
         self.key = None
         self.memory = None
         self.name = args.name  # this one is not stored anywhere
         self.port = None       # at least is not even tried to be retrieved
         self.version = None
+        self.username = None
         self._conf = {}
 
         # set defaults stored in hard coded yaml
@@ -401,6 +452,10 @@ class Config:
                 continue
             setattr(self, attr, str(val))
 
+        # sort out case, where there is image/default-user provided
+        if self.image:
+            self._update_distros_with_custom_image()
+
         # set distribution and version if not specified by user
         if not self.distro:
             self.distro = 'ubuntu'
@@ -470,7 +525,7 @@ class Config:
     def _read_filename(self, fname):
         fullpath = os.path.expanduser(os.path.expandvars(fname))
         if not os.path.exists(fullpath):
-            return
+            return None
 
         with open(fname) as fobj:
             return fobj.read()
@@ -487,7 +542,7 @@ class Config:
             self.ssh_key_path = os.path.join(os.path.expanduser("~/.ssh"),
                                              self.ssh_key_path)
         if not os.path.exists(self.ssh_key_path):
-            raise BoxNotFound(f'Cannot find ssh public key: {self.key}')
+            raise BoxConfError(f'Cannot find ssh public key: {self.key}')
 
     def _set_defaults(self):
         conf = yaml.safe_load(USER_DATA)
@@ -542,6 +597,18 @@ class Config:
 
         self._conf = conf
 
+    def _update_distros_with_custom_image(self):
+        self.image = os.path.abspath(self.image)
+        self.distro = 'custom'
+        if not self.username:
+            self.username = self.default_user
+        DISTROS['custom'] = {'username': self.default_user,
+                             'realname': 'custom os',
+                             'img_class': CustomImage,
+                             'amd64': 'x86_64',
+                             'image': self.image,
+                             'default_version': '0'}
+
     def _update(self, source, update):
         for key, val in update.items():
             if isinstance(val, collections.abc.Mapping):
@@ -551,6 +618,50 @@ class Config:
         return source
 
 
+class OsTypes:
+    def __init__(self, conf):
+        self._conf = conf
+        self._ostypes = []
+
+        self._gather_os_types()
+
+    def _gather_os_types(self):
+        out = Run(['vboxmanage', 'list', 'ostypes']).stdout
+        for line in out.split('\n'):
+            if not line.startswith('ID:'):
+                continue
+            self._ostypes.append(line.split(':')[1].strip())
+
+    def ubuntu(self):
+        lts = ''
+        major, minor = (int(x) for x in self._conf.version.split('.'))
+
+        if major % 2 == 0 and minor == 4:
+            lts = '_LTS'
+        name = f"Ubuntu{major}{lts}_64"
+
+        if name not in self._ostypes:
+            return 'Ubuntu_64'
+
+        return name
+
+    def fedora(self):
+        return "Fedora_64"
+
+    def debian(self):
+        name = f"Debian{self._conf.version}_64"
+        if name not in self._ostypes:
+            return 'Debian_64'
+
+        return name
+
+    def get(self):
+        if not hasattr(self, self._conf.distro):
+            return "Linux_64"
+
+        return getattr(self, self._conf.distro)()
+
+
 class VBoxManage:
     """
     Class for dealing with vboxmanage commands
@@ -559,11 +670,12 @@ class VBoxManage:
         self.name_or_uuid = name_or_uuid
         self.vm_info = {}
         self.uuid = None
+        self.running = False
 
     def get_vm_base_path(self):
         path = self._get_vm_config()
         if not path:
-            return
+            return None
 
         return os.path.dirname(path)
 
@@ -571,12 +683,12 @@ class VBoxManage:
         path = self._get_vm_config()
         if not path:
             LOG.warning('Configuration for "%s" not found', self.name_or_uuid)
-            return
+            return None
 
         dom = xml.dom.minidom.parse(path)
         if len(dom.getElementsByTagName('HardDisk')) != 1:
             # don't know what to do with multiple discs
-            raise BoxError()
+            raise BoxError
 
         disk = dom.getElementsByTagName('HardDisk')[0]
         location = disk.getAttribute('location')
@@ -598,6 +710,7 @@ class VBoxManage:
                     return line
 
                 return line.split(' ')[0].strip()
+        return None
 
     def get_vm_info(self):
         out = Run(['vboxmanage', 'showvminfo', self.name_or_uuid])
@@ -610,6 +723,9 @@ class VBoxManage:
             if line.startswith('Config file:'):
                 self.vm_info['config_file'] = line.split('Config '
                                                          'file:')[1].strip()
+
+            if line.startswith('State:'):
+                self.running = line.split(':')[1].strip().startswith('running')
                 break
 
         dom = xml.dom.minidom.parse(self.vm_info['config_file'])
@@ -653,6 +769,9 @@ class VBoxManage:
     def poweroff(self):
         Run(['vboxmanage', 'controlvm', self.name_or_uuid, 'poweroff'])
 
+    def acpipowerbutton(self):
+        Run(['vboxmanage', 'controlvm', self.name_or_uuid, 'acpipowerbutton'])
+
     def vmlist(self, only_running=False, long_list=False, only_boxpy=False):
         subcommand = 'runningvms' if only_running else 'vms'
         machines = {}
@@ -692,6 +811,7 @@ class VBoxManage:
                 '--delete']).returncode != 0:
             LOG.fatal('Removing VM "%s" failed', self.name_or_uuid)
             return 7
+        return None
 
     def create(self, conf):
         memory = convert_to_mega(conf.memory)
@@ -702,12 +822,19 @@ class VBoxManage:
             LOG.fatal('Failed to create VM:\n%s', out.stderr)
             return None
 
+        if out.stdout.startswith('WARNING:'):
+            LOG.fatal('Created crippled VM:\n%s\nFix the issue with '
+                      'VirtualBox, remove the dead VM and start over.',
+                      out.stdout)
+            return None
+
         for line in out.stdout.split('\n'):
             if line.startswith('UUID:'):
                 self.uuid = line.split('UUID:')[1].strip()
 
         if not self.uuid:
-            raise BoxVBoxFailure(f'Cannot create VM "{self.name_or_uuid}".')
+            msg = f'Cannot create VM "{self.name_or_uuid}".'
+            raise BoxVBoxError(msg)
 
         port = conf.port if conf.port else self._find_unused_port()
 
@@ -718,7 +845,11 @@ class VBoxManage:
                '--acpi', 'on',
                '--audio', 'none',
                '--nic1', 'nat',
-               '--natpf1', f'boxpyssh,tcp,,{port},,22']
+               '--natpf1', f'boxpyssh,tcp,,{port},,22',
+               '--graphicscontroller', 'vmsvga',
+               '--vram', '16',
+               '--ostype', OsTypes(conf).get()]
+
         for count, (hostport, vmport) in enumerate(conf.forwarding.items(),
                                                    start=1):
             cmd.extend(['--natpf1', f'custom-pf-{count},tcp,,{hostport},'
@@ -726,14 +857,15 @@ class VBoxManage:
 
         if Run(cmd).returncode != 0:
             LOG.fatal(f'Cannot modify VM "{self.name_or_uuid}"')
-            raise BoxVBoxFailure()
+            raise BoxVBoxError
 
-        if conf.disable_nested == 'False':
-            if Run(['vboxmanage', 'modifyvm', self.name_or_uuid,
-                    '--nested-hw-virt', 'on']).returncode != 0:
-                LOG.fatal(f'Cannot set nested virtualization for VM '
-                          f'"{self.name_or_uuid}"')
-                raise BoxVBoxFailure()
+        if conf.disable_nested == 'False' and Run(['vboxmanage', 'modifyvm',
+                                                   self.name_or_uuid,
+                                                   '--nested-hw-virt',
+                                                   'on']).returncode != 0:
+            LOG.fatal(f'Cannot set nested virtualization for VM '
+                      f'"{self.name_or_uuid}"')
+            raise BoxVBoxError
 
         return self.uuid
 
@@ -769,7 +901,7 @@ class VBoxManage:
         if Run(['vboxmanage', 'modifymedium', 'disk', src, '--resize',
                 str(size), '--move', fullpath]).returncode != 0:
             LOG.fatal('Resizing and moving image %s has failed', dst)
-            raise BoxVBoxFailure()
+            raise BoxVBoxError
         return fullpath
 
     def storageattach(self, controller_name, port, type_, image):
@@ -793,7 +925,7 @@ class VBoxManage:
         if Run(['vboxmanage', 'startvm', self.name_or_uuid, '--type',
                 type_]).returncode != 0:
             LOG.fatal('Failed to start: %s', self.name_or_uuid)
-            raise BoxVBoxFailure()
+            raise BoxVBoxError
 
     def setextradata(self, key, val):
         res = Run(['vboxmanage', 'setextradata', self.name_or_uuid, key, val])
@@ -807,7 +939,7 @@ class VBoxManage:
         if Run(['vboxmanage', 'modifyvm', self.name_or_uuid, f'--{nic}',
                 kind]).returncode != 0:
             LOG.fatal('Cannot modify VM "%s"', self.name_or_uuid)
-            raise BoxVBoxFailure()
+            raise BoxVBoxError
 
     def is_port_in_use(self, port):
         used_ports = self._get_defined_ports()
@@ -867,11 +999,12 @@ class VBoxManage:
 class Image:
     URL = ""
     IMG = ""
+    CHECKSUMTOOL = 'sha256sum'
 
-    def __init__(self, vbox, version, arch, release):
+    def __init__(self, vbox, version, arch, fname=None):
         self.vbox = vbox
         self._tmp = tempfile.mkdtemp(prefix='boxpy_')
-        self._img_fname = None
+        self._img_fname = fname
 
     def convert_to_vdi(self, disk_img, size):
         LOG.info('Converting and resizing "%s", new size: %s', disk_img, size)
@@ -909,6 +1042,7 @@ class Image:
             return False
 
         LOG.info('Calculating checksum for "%s"', self._img_fname)
+        LOG.debug('Checksum file: "%s"', self._checksum_file)
         fname = os.path.join(self._tmp, self._checksum_file)
         expected_sum = self._get_checksum(fname)
 
@@ -917,7 +1051,7 @@ class Image:
             return False
 
         if os.path.exists(os.path.join(CACHE_DIR, self._img_fname)):
-            cmd = ['sha256sum', os.path.join(CACHE_DIR, self._img_fname)]
+            cmd = [self.CHECKSUMTOOL, os.path.join(CACHE_DIR, self._img_fname)]
             calulated_sum = Run(cmd).stdout.split(' ')[0]
             LOG.details('Checksum for image: %s, expected: %s', calulated_sum,
                         expected_sum)
@@ -931,8 +1065,11 @@ class Image:
             return True
 
         fname = os.path.join(CACHE_DIR, self._img_fname)
-        LOG.header('Downloading image %s', self._img_fname)
-        Run(['wget', '-q', self._img_url, '-O', fname])
+        LOG.header('Downloading image %s', self._img_url)
+        result = Run(['wget', '-q', self._img_url, '-O', fname])
+        if result.returncode != 0:
+            LOG.fatal("Error downloading image %s", self._img_url)
+            return False
 
         if not self._checksum():
             # TODO: make some retry mechanism?
@@ -943,15 +1080,15 @@ class Image:
         return True
 
     def _get_checksum(self, fname):
-        raise NotImplementedError()
+        raise NotImplementedError
 
 
 class Ubuntu(Image):
     URL = "https://cloud-images.ubuntu.com/releases/%s/release/%s"
     IMG = "ubuntu-%s-server-cloudimg-%s.img"
 
-    def __init__(self, vbox, version, arch, release):
-        super().__init__(vbox, version, arch, release)
+    def __init__(self, vbox, version, arch, fname=None):
+        super().__init__(vbox, version, arch)
         self._img_fname = self.IMG % (version, arch)
         self._img_url = self.URL % (version, self._img_fname)
         self._checksum_file = 'SHA256SUMS'
@@ -969,17 +1106,64 @@ class Ubuntu(Image):
         return expected_sum
 
 
+class Debian(Image):
+    URL = "https://cloud.debian.org/images/cloud/%s/daily/latest/%s"
+    IMG = "debian-%s-generic-%s-daily.qcow2"
+    CHECKSUMTOOL = 'sha512sum'
+    CODENAME_MAP = {'13': 'trixie',
+                    '12': 'bookworm',
+                    '11': 'bullseye',
+                    '10': 'buster'}
+
+    def __init__(self, vbox, version, arch, fname=None):
+        super().__init__(vbox, version, arch)
+        codename = self.CODENAME_MAP[version]
+        self._img_fname = self.IMG % (version, arch)
+        self._img_url = self.URL % (codename, self._img_fname)
+        self._checksum_file = 'SHA512SUMS'
+        self._checksum_url = self.URL % (codename, self._checksum_file)
+
+    def _get_checksum(self, fname):
+        expected_sum = None
+        Run(['wget', self._checksum_url, '-q', '-O', fname])
+        with open(fname) as fobj:
+            for line in fobj.readlines():
+                if self._img_fname in line:
+                    expected_sum = line.split(' ')[0]
+                    break
+
+        return expected_sum
+
+
 class Fedora(Image):
     URL = ("https://download.fedoraproject.org/pub/fedora/linux/releases/%s/"
            "Cloud/%s/images/%s")
     IMG = "Fedora-Cloud-Base-%s-%s.%s.qcow2"
     CHKS = "Fedora-Cloud-%s-%s-%s-CHECKSUM"
+    REVISION = {'37': '1.7',
+                '38': '1.6',
+                '39': '1.5',
+                '40': '1.14',
+                '41': '1.4'}
 
-    def __init__(self, vbox, version, arch, release):
-        super().__init__(vbox, version, arch, release)
-        self._img_fname = self.IMG % (version, release, arch)
+    def __init__(self, vbox, version, arch, fname=None):
+        super().__init__(vbox, version, arch)
+        revision = self.REVISION[version]
+        if int(version) >= 40:
+            if int(version) == 40:
+                # Started from Fedora 40 there is "Generic" in the image names.
+                self.IMG = "Fedora-Cloud-Base-Generic.%s-%s-%s.qcow2"
+                # Fedora 40 have messed up position of the items in filename.
+                self._img_fname = self.IMG % (arch, version, revision)
+            else:
+                # But in Fedora 41 there is no dot between Generic and
+                # version, but between version and arch.
+                self.IMG = "Fedora-Cloud-Base-Generic-%s-%s.%s.qcow2"
+                self._img_fname = self.IMG % (version, revision, arch)
+        else:
+            self._img_fname = self.IMG % (version, revision, arch)
         self._img_url = self.URL % (version, arch, self._img_fname)
-        self._checksum_file = self.CHKS % (version, release, arch)
+        self._checksum_file = self.CHKS % (version, revision, arch)
         self._checksum_url = self.URL % (version, arch, self._checksum_file)
 
     def _get_checksum(self, fname):
@@ -998,13 +1182,12 @@ class Fedora(Image):
 
 class CentosStream(Image):
     URL = "https://cloud.centos.org/centos/%s-stream/%s/images/%s"
-    IMG = '.*(CentOS-Stream-GenericCloud-%s-[0-9]+\.[0-9].%s.qcow2).*'
+    IMG = '.*(CentOS-Stream-GenericCloud-%s-[0-9]+.[0-9].%s.qcow2).*'
     CHKS = "CHECKSUM"
 
-    def __init__(self, vbox, version, arch, release):
+    def __init__(self, vbox, version, arch, release, fname=None):
         super().__init__(vbox, version, arch, release)
-        self._checksum_file = '%s-centos-stream-%s-%s' % (self.CHKS, version,
-                                                          arch)
+        self._checksum_file = f'{self.CHKS}-centos-stream-{version}-{arch}'
         self._checksum_url = self.URL % (version, arch, self.CHKS)
         # there is assumption, that we always need latest relese for specific
         # version and architecture.
@@ -1012,12 +1195,13 @@ class CentosStream(Image):
         self._img_url = self.URL % (version, arch, self._img_fname)
 
     def _get_image_name(self, version, arch):
-        Run(['wget', self._checksum_url, '-q', '-O', self._checksum_file])
+        fname = os.path.join(self._tmp, self._checksum_file)
+        Run(['wget', self._checksum_url, '-q', '-O', fname])
 
         pat = re.compile(self.IMG % (version, arch))
 
         images = []
-        with open(self._checksum_file) as fobj:
+        with open(fname) as fobj:
             for line in fobj.read().strip().split('\n'):
                 line = line.strip()
                 if line.startswith('#'):
@@ -1026,9 +1210,11 @@ class CentosStream(Image):
                 if match and match.groups():
                     images.append(match.groups()[0])
 
+        Run(['rm', fname])
         images.reverse()
         if images:
             return images[0]
+        return None
 
     def _get_checksum(self, fname):
         expected_sum = None
@@ -1044,29 +1230,38 @@ class CentosStream(Image):
         return expected_sum
 
 
+class CustomImage(Image):
+
+    def _download_image(self):
+        # just use provided image
+        return True
+
+
 DISTROS = {'ubuntu': {'username': 'ubuntu',
                       'realname': 'ubuntu',
                       'img_class': Ubuntu,
                       'amd64': 'amd64',
-                      'default_version': '20.04'},
+                      'default_version': '24.04'},
            'fedora': {'username': 'fedora',
                       'realname': 'fedora',
                       'img_class': Fedora,
                       'amd64': 'x86_64',
-                      'default_version': '34'},
+                      'default_version': '41'},
            'centos': {'username': 'centos',
                       'realname': 'centos',
                       'img_class': CentosStream,
                       'amd64': 'x86_64',
-                      'default_version': '8'}}
+                      'default_version': '9'},
+           'debian': {'username': 'debian',
+                      'realname': 'debian',
+                      'img_class': Debian,
+                      'amd64': 'amd64',
+                      'default_version': '12'}}
 
 
 def get_image_object(vbox, version, image='ubuntu', arch='amd64'):
-    release = None
-    if image == 'fedora':
-        release = FEDORA_RELEASE_MAP[version]
     return DISTROS[image]['img_class'](vbox, version, DISTROS[image]['amd64'],
-                                       release)
+                                       DISTROS[image].get('image'))
 
 
 class IsoImage:
@@ -1117,7 +1312,8 @@ def vmcreate(args, conf=None):
     if not conf:
         try:
             conf = Config(args)
-        except BoxNotFound:
+        except BoxConfError as err:
+            LOG.fatal(f'Configuration error: {err.args[0]}.')
             return 7
         except yaml.YAMLError:
             LOG.fatal(f'Cannot read or parse file `{args.config}` as YAML '
@@ -1141,13 +1337,14 @@ def vmcreate(args, conf=None):
     if not vbox.create_controller('SATA', 'sata'):
         return 4
 
-    for key in ('distro', 'hostname', 'key', 'version'):
+    for key in ('distro', 'hostname', 'key', 'version', 'image', 'username'):
+        if getattr(conf, key) is None:
+            continue
         if not vbox.setextradata(key, getattr(conf, key)):
             return 5
 
-    if conf.user_data:
-        if not vbox.setextradata('user_data', conf.user_data):
-            return 6
+    if conf.user_data and not vbox.setextradata('user_data', conf.user_data):
+        return 6
 
     if not vbox.setextradata('creator', 'boxpy'):
         return 13
@@ -1179,34 +1376,69 @@ def vmcreate(args, conf=None):
 
     # than, let's try to see if boostraping process has finished
     LOG.info('Waiting for cloud init to finish ', end='')
+    username = DISTROS[conf.distro]["username"]
     cmd = ['ssh', '-o', 'StrictHostKeyChecking=no',
            '-o', 'UserKnownHostsFile=/dev/null',
            '-o', 'ConnectTimeout=2',
            '-i', conf.ssh_key_path[:-4],
-           f'ssh://{DISTROS[conf.distro]["username"]}'
-           f'@localhost:{vbox.vm_info["port"]}', 'sudo cloud-init status']
+           f'ssh://{username}@localhost:{vbox.vm_info["port"]}',
+           'sudo cloud-init status']
     try:
+        counter = 0
         while True:
-            out = Run(cmd).stdout
-            LOG.debug('Out: %s', out)
+            out = Run(cmd)
+            LOG.debug('Out: %s', out.stdout)
 
-            if (not out) or ('status' in out and 'running' in out):
+            if (not out.stdout) or ('status' in out.stdout and
+                                    'running' in out.stdout):
                 LOG.info('.', end='')
                 sys.stdout.flush()
+                if 'Permission denied (publickey)' in out.stderr:
+                    if conf.username and conf.username != username:
+                        username = conf.username
+                        vbox.setextradata('username', username)
+                        cmd[9] = (f'ssh://{username}'
+                                  f'@localhost:{vbox.vm_info["port"]}')
+                        continue
+                    raise PermissionError(f'There is an issue with accessing '
+                                          f'VM with ssh for user {username}. '
+                                          f'Check output in debug mode.')
                 time.sleep(3)
+                counter += 1
+
+                # TODO: there is something odd with debian cloud images prior
+                # to 12 (bookworm), as on first run system crashes. In that
+                # case after ~20 seconds there should already be panic, reset
+                # machine as a workaround. Remove this after debian 12
+                # stabilization later this year.
+                if (counter == 8 and conf.distro == 'debian'
+                        and conf.version != '12'):
+                    LOG.debug('Resetting `%s`, due to the issue with kernel '
+                              'panic on Debian %s the first run', conf.name,
+                              conf.version)
+                    counter += 1
+                    vbox.poweroff()
+                    time.sleep(3)
+                    vbox.poweron(args.type)
                 continue
 
             LOG.info(' done')
             break
-        out = out.split(':')[1].strip()
+        out = out.stdout.split(':')[1].strip()
         if out != 'done':
             cmd = cmd[:-1]
             cmd.append('cloud-init status -l')
             LOG.warning('Cloud init finished with "%s" status:\n%s', out,
                         Run(cmd).stdout)
 
+    except PermissionError:
+        LOG.info('\n')
+        iso.cleanup()
+        image.cleanup()
+        vbox.destroy()
+        raise
     except KeyboardInterrupt:
-        LOG.warning('\nIterrupted, cleaning up')
+        LOG.warning('\nInterrupted, cleaning up')
         iso.cleanup()
         image.cleanup()
         vbox.destroy()
@@ -1218,16 +1450,23 @@ def vmcreate(args, conf=None):
 
     # reread config to update fields
     conf = Config(args, vbox)
+    if conf.username and conf.username != username:
+        username = conf.username
+    else:
+        username = DISTROS[conf.distro]["username"]
     LOG.info('You can access your VM by issuing:')
-    LOG.info(f'ssh -p {conf.port} -i {conf.ssh_key_path[:-4]} '
-             f'{DISTROS[conf.distro]["username"]}@localhost')
+    LOG.info('ssh -p %s -i %s %s@localhost', conf.port, conf.ssh_key_path[:-4],
+             username)
     LOG.info('or simply:')
-    LOG.info(f'boxpy ssh {conf.name}')
+    LOG.info('boxpy ssh %s', conf.name)
     return 0
 
 
 def vmdestroy(args):
-    for name in args.name:
+
+    vm_names = args.name if isinstance(args.name, list) else [args.name]
+
+    for name in vm_names:
         vbox = VBoxManage(name)
         if not vbox.get_vm_info():
             LOG.fatal(f'Cannot remove VM "{name}" - it doesn\'t exists.')
@@ -1319,7 +1558,8 @@ def vminfo(args):
             LOG.info(line)
 
     if 'user_data' in info:
-        LOG.info(f'User data file path:\t{info["user_data"]}')
+        LOG.info('User data file path:\t%s', info['user_data'])
+    return 0
 
 
 def vmrebuild(args):
@@ -1327,12 +1567,13 @@ def vmrebuild(args):
     if not vbox.get_vm_info():
         LOG.fatal(f'Cannot rebuild VM "{args.name}" - it doesn\'t exists.')
         return 20
-    else:
-        LOG.header('Rebuilding VM: %s', args.name)
+
+    LOG.header('Rebuilding VM: %s', args.name)
 
     try:
         conf = Config(args, vbox)
-    except BoxNotFound:
+    except BoxNotFoundError as ex:
+        LOG.fatal(f'Error with parsing config: {ex}')
         return 8
     except yaml.YAMLError:
         LOG.fatal(f'Cannot read or parse file `{args.config}` as YAML '
@@ -1356,6 +1597,13 @@ def vmrebuild(args):
         conf.disk_size = vbox.get_media_size(disk_path)
 
     vmdestroy(args)
+
+    # Wait till VM is gone
+    while True:
+        vbox = VBoxManage(args.name)
+        if not vbox.get_vm_info():
+            break
+
     vmcreate(args, conf)
     return 0
 
@@ -1373,18 +1621,54 @@ def connect(args):
 
     try:
         conf = Config(args, vbox)
-    except BoxNotFound:
+    except BoxNotFoundError:
         return 11
     except yaml.YAMLError:
         LOG.fatal(f'Cannot read or parse file `{args.config}` as YAML '
                   f'file.')
         return 16
 
-    return Run(['ssh', '-o', 'StrictHostKeyChecking=no',
-                '-o', 'UserKnownHostsFile=/dev/null',
-                '-i', conf.ssh_key_path[:-4],
-                f'ssh://{DISTROS[conf.distro]["username"]}'
-                f'@localhost:{conf.port}'], False).returncode
+    username = conf.username or DISTROS[conf.distro]["username"]
+    cmd = ['ssh', '-o', 'StrictHostKeyChecking=no',
+           '-o', 'UserKnownHostsFile=/dev/null',
+           '-i', conf.ssh_key_path[:-4],
+           f'ssh://{username}'
+           f'@localhost:{conf.port}']
+    LOG.debug('Connecting to vm `%s` using command:\n%s', args.name,
+              ' '.join(cmd))
+    return Run(cmd, capture=False).returncode
+
+
+def _set_vmstate(name, state, guitype=None, poweroff=False):
+
+    vbox = VBoxManage(name)
+    if not vbox.get_vm_info():
+        LOG.fatal(f'No machine has been found with a name `{name}`.')
+        return 20
+
+    if vbox.running and state == "start":
+        LOG.info('VM "%s" is already running.', name)
+        return 1
+
+    if not vbox.running and state == "stop":
+        LOG.info('VM "%s" is already stopped.', name)
+        return 1
+
+    if state == "start":
+        vbox.poweron(guitype)
+    elif poweroff:
+        vbox.poweroff()
+    else:
+        vbox.acpipowerbutton()
+    return 0
+
+
+def vmstart(args):
+    _set_vmstate(args.name, 'start', guitype=args.type)
+
+
+def vmstop(args):
+    _set_vmstate(args.name, 'stop', poweroff=args.poweroff)
 
 
 def main():
@@ -1414,10 +1698,16 @@ def main():
                         help="Alternative user-data template filepath")
     create.add_argument('-d', '--distro', help="Image name. 'ubuntu' is "
                         "default")
+    create.add_argument('-e', '--default-user', help="Default cloud-init user "
+                        "to be used with custom image (--image param). "
+                        "Without image it will make no effect.")
     create.add_argument('-f', '--forwarding', action='append', help="expose "
                         "port from VM to the host. It should be in format "
                         "'hostport:vmport'. this option can be used multiple "
                         "times for multiple ports.")
+    create.add_argument('-i', '--image', help="custom qcow2 image filepath. "
+                        "Note, that it requires to provide --default-user as "
+                        "well.")
     create.add_argument('-k', '--key', help="SSH key to be add to the config "
                         "drive. Default ~/.ssh/id_rsa")
     create.add_argument('-m', '--memory', help="amount of memory in "
@@ -1452,17 +1742,23 @@ def main():
                           help='show only running VMs')
     list_vms.set_defaults(func=vmlist)
 
-    rebuild = subparsers.add_parser('rebuild', help='Rebuild VM, all options '
+    rebuild = subparsers.add_parser('rebuild', help='rebuild VM, all options '
                                     'besides vm name are optional, and their '
                                     'values will be taken from vm definition.')
     rebuild.add_argument('name', help='name or UUID of the VM')
     rebuild.add_argument('-c', '--config',
                          help="Alternative user-data template filepath")
     rebuild.add_argument('-d', '--distro', help="Image name.")
+    rebuild.add_argument('-e', '--default-user', help="Default cloud-init "
+                         "user to be used with custom image (--image param). "
+                         "Without image it will make no effect.")
     rebuild.add_argument('-f', '--forwarding', action='append', help="expose "
                          "port from VM to the host. It should be in format "
                          "'hostport:vmport'. this option can be used multiple "
                          "times for multiple ports.")
+    rebuild.add_argument('-i', '--image', help="custom qcow2 image filepath. "
+                         "Note, that it requires to provide --default-user as "
+                         "well.")
     rebuild.add_argument('-k', '--key',
                          help='SSH key to be add to the config drive')
     rebuild.add_argument('-m', '--memory', help='amount of memory in '
@@ -1481,13 +1777,26 @@ def main():
     rebuild.add_argument('-v', '--version', help='distribution version')
     rebuild.set_defaults(func=vmrebuild)
 
+    start = subparsers.add_parser('start', help='start VM')
+    start.add_argument('name', help='name or UUID of the VM')
+    start.add_argument('-t', '--type', default='headless',
+                       help="VM run type, headless by default.",
+                       choices=['gui', 'headless', 'sdl', 'separate'])
+    start.set_defaults(func=vmstart)
+
+    stop = subparsers.add_parser('stop', help='stop VM')
+    stop.add_argument('name', help='name or UUID of the VM')
+    stop.add_argument('-p', '--poweroff', action='store_true', help='poweroff '
+                      'machine instead of using ACPI power signal')
+    stop.set_defaults(func=vmstop)
+
     completion = subparsers.add_parser('completion', help='generate shell '
                                        'completion')
     completion.add_argument('shell', choices=['bash'],
                             help="pick shell to generate completions for")
     completion.set_defaults(func=shell_completion)
 
-    ssh = subparsers.add_parser('ssh', help='Connect to the machine via SSH')
+    ssh = subparsers.add_parser('ssh', help='connect to the machine via SSH')
     ssh.add_argument('name', help='name or UUID of the VM')
     ssh.set_defaults(func=connect)
 
@@ -1497,10 +1806,14 @@ def main():
 
     args = parser.parse_args()
 
+    if 'image' in args and 'default_user' not in args:
+        parser.error('Parameter --image requires --default-user')
+        return 22
+
     LOG.set_verbose(args.verbose, args.quiet)
 
     if 'func' not in args and args.version:
-        LOG.info(f'boxpy {__version__}')
+        LOG.info('boxpy %s', __version__)
         parser.exit()
 
     if hasattr(args, 'func'):
@@ -1508,6 +1821,7 @@ def main():
 
     parser.print_help()
     parser.exit()
+    return 23
 
 
 if __name__ == '__main__':

examples/devstack-fedora36.yaml

@@ -1,9 +1,7 @@
-# Note, that cloud init will fail, due to old cloudinit package, which module
-# cc_keys_to_console doesn't recognize skipping option. Just ignore this error.
 package_update: true
 packages:
   - bash-completion
-  - ctags-etags
+  - ctags
   - git
   - git-review
   - htop
@@ -25,13 +23,13 @@ write_files:
       disable_service n-vnc
       disable_service dstat
 runcmd:
-  - [su, -, fedora, -c, "git clone https://opendev.org/openstack/devstack ~/devstack"]
-  - [su, -, fedora, -c, "cp /tmp/local.conf /home/fedora/devstack/"]
-  - [su, -, fedora, -c, "echo 'export HOST_IP=10.0.2.15' >> .bashrc"]
+  - su - fedora -c "git clone https://opendev.org/openstack/devstack ~/devstack"
+  - su - fedora -c "cp /tmp/local.conf /home/fedora/devstack/"
+  - su - fedora -c "echo 'export HOST_IP=10.0.2.15' >> .bashrc"
 boxpy_data:
   key: vm
   cpus: 2
   memory: 4GB
   disk_size: 10GB
   distro: fedora
-  version: 34
+  version: 36

examples/devstack-ubuntu-22.04.yaml

@@ -7,7 +7,7 @@ packages:
   - ipython3
   - jq
   - mc
-  - python-apsw
+  - python3-apsw
   - python3-flake8
   - python3-jedi
   - python3-pip
@@ -26,17 +26,15 @@ write_files:
       RABBIT_PASSWORD=$$ADMIN_PASSWORD
       SERVICE_PASSWORD=$$ADMIN_PASSWORD
 runcmd:
-  - [apt, purge, '-y', python3-pyasn1-modules]
-  - [apt, purge, '-y', python3-simplejson]
-  - [su, -, ubuntu, -c, "git clone https://github.com/gryf/vmstrap"]
-  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh"]
-  - [rm, -fr, /home/ubuntu/vmstrap]
-  - [su, -, ubuntu, -c, "cp /tmp/local.conf /home/ubuntu/devstack/"]
-  - [su, -, ubuntu, -c, "echo 'export HOST_IP=10.0.2.15' >> .bashrc"]
+  - apt purge -y python3-pyasn1-modules
+  - apt purge -y python3-simplejson
+  - su - ubuntu -c "git clone https://opendev.org/openstack/devstack ~/devstack"
+  - su - ubuntu -c "cp /tmp/local.conf /home/ubuntu/devstack/"
+  - su - ubuntu -c "echo 'export HOST_IP=10.0.2.15' >> .bashrc"
 boxpy_data:
   key: vm
   cpus: 2
   memory: 6GB
   disk_size: 10GB
   distro: ubuntu
-  version: 20.04
+  version: 22.04

examples/kuryr_multinode_master.yaml

@@ -7,7 +7,7 @@ packages:
   - ipython3
   - jq
   - mc
-  - python-apsw
+  - python3-apsw
   - python3-flake8
   - python3-jedi
   - python3-pip
@@ -34,55 +34,55 @@ write_files:
     content: |
       [[local|localrc]]
       disable_all_services
-      
+
       # Cinder
       disable_service c-api
       disable_service c-bak
       disable_service c-sch
       disable_service c-vol
       disable_service cinder
-      
+
       # Coredns
       disable_service coredns
-      
-      # Dstat 
+
+      # Dstat
       disable_service dstat
-      
+
       # ETCD
       enable_service etcd3
-      
+
       # Glance
       enable_service g-api
       enable_service g-reg
-      
+
       # Horizon
       disable_service horizon
-      
+
       # Keystone
       enable_service key
-      
+
       # kubernetes
       enable_service kubernetes-master
       disable_service kubernetes-worker
-      
+
       # kuryr
       enable_service kuryr-daemon
       enable_service kuryr-kubernetes
-      
+
       # mysql.
       enable_service mysql
-      
+
       # Nova services
       enable_service n-api
       enable_service n-api-meta
       enable_service n-cond
       enable_service n-cpu
       enable_service n-sch
-      
+
       # Neutron
       enable_service neutron
       enable_service neutron-tag-ports-during-bulk-creation
-      
+
       # Octavia
       enable_service o-api
       enable_service o-cw
@@ -90,74 +90,77 @@ write_files:
       enable_service o-hk
       enable_service o-hm
       enable_service octavia
-      
+
       # Neutron ovn services
       enable_service ovn-controller
       enable_service ovn-northd
       enable_service ovs-vswitchd
       enable_service ovsdb-server
-      
+
       # Placement API
       enable_service placement-api
       enable_service placement-client
-      
+
       # Neutron services
       enable_service q-ovn-metadata-agent
       enable_service q-qos
       enable_service q-svc
       enable_service q-trunk
       enable_service rabbit
-      
+
       # Swift
       disable_service s-account
       disable_service s-container
       disable_service s-object
       disable_service s-proxy
-      
+
       # Tempest
       enable_service tempest
-      
+
       # TLS
       disable_service tls-proxy
-      
+
       # Vars
       ADMIN_PASSWORD="secretadmin"
+      CONTAINER_ENGINE="crio"
       DATABASE_PASSWORD="secretdatabase"
       ETCD_USE_RAMDISK="True"
-      
+
       KURYR_ENABLED_HANDLERS="vif,endpoints,service,namespace,pod_label,policy,kuryrnetworkpolicy,kuryrnetwork,kuryrport,kuryrloadbalancer"
       KURYR_ENFORCE_SG_RULES="False"
       KURYR_EP_DRIVER_OCTAVIA_PROVIDER="ovn"
       KURYR_K8S_MULTI_WORKER_TESTS="True"
       KURYR_K8S_OCTAVIA_MEMBER_MODE="L2"
       KURYR_LB_ALGORITHM="SOURCE_IP_PORT"
-      KURYR_NEUTRON_DEFAULT_ROUTER="router1"
+      KURYR_NEUTRON_DEFAULT_ROUTER="kuryr-router"
       KURYR_SG_DRIVER="policy"
       KURYR_SUBNET_DRIVER="namespace"
-      
+      KURYR_SUPPORT_POD_SECURITY="True"
+
       LOGFILE="/opt/stack/logs/devstacklog.txt"
       LOG_COLOR="False"
       ML2_L3_PLUGIN="ovn-router,trunk,qos"
       OCTAVIA_AMP_IMAGE_FILE="/tmp/test-only-amphora-x64-haproxy-ubuntu-bionic.qcow2"
       OCTAVIA_AMP_IMAGE_NAME="test-only-amphora-x64-haproxy-ubuntu-bionic"
       OCTAVIA_AMP_IMAGE_SIZE="3"
-      
-      OVN_BRANCH="v20.06.2"
+
+      OVN_BRANCH="v21.06.0"
       OVN_BUILD_FROM_SOURCE="True"
       OVN_DBS_LOG_LEVEL="dbg"
       OVN_L3_CREATE_PUBLIC_NETWORK="True"
-      
+      OVS_BRANCH="a4b04276ab5934d087669ff2d191a23931335c87"
+
       VAR_RUN_PATH="/usr/local/var/run"
-      
+
       RABBIT_PASSWORD="secretrabbit"
       RECLONE="no"
       SERVICE_PASSWORD="secretservice"
       SERVICE_TOKEN="password"
-      
+
       LIBS_FROM_GIT=cinder,devstack,devstack-gate,devstack-plugin-container,glance,keystone,kuryr-kubernetes,kuryr-tempest-plugin,neutron,nova,octavia,octavia-tempest-plugin,ovn-octavia-provider,placement,python-octaviaclient,requirements,swift,tempest
-      
+
       TEMPEST_PLUGINS="/opt/stack/kuryr-tempest-plugin /opt/stack/octavia-tempest-plugin"
-      
+
       # enabled plugins
       enable_plugin devstack-plugin-container https://opendev.org/openstack/devstack-plugin-container
       enable_plugin kuryr-kubernetes https://opendev.org/openstack/kuryr-kubernetes
@@ -166,23 +169,26 @@ write_files:
       enable_plugin octavia https://opendev.org/openstack/octavia
       enable_plugin octavia-tempest-plugin https://opendev.org/openstack/octavia-tempest-plugin
       enable_plugin ovn-octavia-provider https://opendev.org/openstack/ovn-octavia-provider
-      
-      [[post-config|$OCTAVIA_CONF]]
+
+      [[post-config|$$OCTAVIA_CONF]]
       [api_settings]
       enabled_provider_drivers = amphora:'Octavia Amphora driver',ovn:'Octavia OVN driver'
 runcmd:
-  - [apt, purge, '-y', python3-pyasn1-modules]
-  - [apt, purge, '-y', python3-simplejson]
-  - [su, -, ubuntu, -c, "git clone https://github.com/gryf/vmstrap"]
-  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh"]
-  - [rm, -fr, /home/ubuntu/vmstrap]
-  - [su, -, ubuntu, -c, "echo 'export HOST_IP=192.168.10.10' >> .bashrc"]
-  - [su, -, ubuntu, -c, "cp /tmp/local.conf /home/ubuntu/devstack/"]
-  - [systemctl, restart, systemd-networkd]
+  - systemctl stop systemd-resolved.service
+  - systemctl disable systemd-resolved.service
+  - echo "nameserver 1.1.1.1" > /etc/resolv.conf
+  - apt purge -y snapd
+  - apt purge -y python3-pyasn1-modules
+  - apt purge -y python3-simplejson
+  - su - ubuntu -c "git clone https://opendev.org/openstack/devstack"
+  - su - ubuntu -c "cp /tmp/local.conf /home/ubuntu/devstack/"
+  - su - ubuntu -c "echo 'export HOST_IP=192.168.10.10' >> .bashrc"
+  - systemctl restart systemd-networkd
 boxpy_data:
   key: vm
   cpus: 4
   memory: 16GB
   disk_size: 50GB
+  version: 20.04
   advanced:
     nic2: intnet

examples/kuryr_multinode_worker.yaml

@@ -7,7 +7,7 @@ packages:
   - ipython3
   - jq
   - mc
-  - python-apsw
+  - python3-apsw
   - python3-flake8
   - python3-jedi
   - python3-pip
@@ -34,28 +34,28 @@ write_files:
     content: |
       [[local|localrc]]
       disable_all_services
-      
+
       # Cinder
       disable_service c-bak
       disable_service c-vol
-      
+
       # Dstat (started to fail recently)
       disable_service dstat
-      
+
       # Horizon
       disable_service horizon
-      
+
       # kubernetes
       disable_service kubernetes-master
       enable_service kubernetes-worker
-      
+
       # kuryr
       enable_service kuryr-daemon
       disable_service kuryr-kubernetes
-      
+
       # Nova
       enable_service n-cpu
-      
+
       # Neutron
       enable_service neutron
       enable_service ovn-controller
@@ -63,46 +63,48 @@ write_files:
       enable_service ovn-octavia-provider
       enable_service ovs-vswitchd
       enable_service ovsdb-server
-      
+
       # Placement API
       enable_service placement-client
-      
+
       # Neutron services cd
       enable_service q-ovn-metadata-agent
       disable_service q-svc
-      
+
       # tempest
       disable_service tempest
-      
+
       # tls
       disable_service tls-proxy
-      
+
       # Vars
       ADMIN_PASSWORD="secretadmin"
+      CONTAINER_ENGINE="crio"
       DATABASE_HOST="192.168.10.10"
       DATABASE_PASSWORD="secretdatabase"
       ENABLE_CHASSIS_AS_GW="False"
       GLANCE_HOSTPORT="192.168.10.10:9292"
-     
+
       # turn on ovn-provider
       KURYR_ENFORCE_SG_RULES="False"
       KURYR_EP_DRIVER_OCTAVIA_PROVIDER="ovn"
       KURYR_K8S_OCTAVIA_MEMBER_MODE="L2"
       KURYR_LB_ALGORITHM="SOURCE_IP_PORT"
-      KURYR_NEUTRON_DEFAULT_ROUTER="router1"
+      KURYR_NEUTRON_DEFAULT_ROUTER="kuryr-router"
       VAR_RUN_PATH="/usr/local/var/run"
-      
+
       KURYR_ENABLED_HANDLERS="vif,endpoints,service,namespace,pod_label,policy,kuryrnetworkpolicy,kuryrnetwork,kuryrport,kuryrloadbalancer"
       KURYR_SG_DRIVER="policy"
       KURYR_SUBNET_DRIVER="namespace"
-      
-      
-      OVN_BRANCH="v20.06.2"
+
+
+      OVN_BRANCH="v21.06.0"
       OVN_BUILD_FROM_SOURCE="True"
       OVN_DBS_LOG_LEVEL="dbg"
       OVN_L3_CREATE_PUBLIC_NETWORK="True"
-      
-      
+      OVS_BRANCH="a4b04276ab5934d087669ff2d191a23931335c87"
+
+
       LIBVIRT_TYPE="qemu"
       LOGFILE="/opt/stack/logs/devstacklog.txt"
       LOG_COLOR="False"
@@ -113,25 +115,28 @@ write_files:
       SERVICE_HOST="192.168.10.10"
       SERVICE_PASSWORD="secretservice"
       SERVICE_TOKEN="password"
-      
+
       LIBS_FROM_GIT=cinder,devstack,devstack-gate,devstack-plugin-container,glance,keystone,kuryr-kubernetes,kuryr-tempest-plugin,neutron,nova,octavia,placement,python-octaviaclient,requirements,swift,tempest
-      
+
       # enabled plugins
       enable_plugin devstack-plugin-container https://opendev.org/openstack/devstack-plugin-container
       enable_plugin kuryr-kubernetes https://opendev.org/openstack/kuryr
 runcmd:
-  - [apt, purge, '-y', python3-pyasn1-modules]
-  - [apt, purge, '-y', python3-simplejson]
-  - [su, -, ubuntu, -c, "git clone https://github.com/gryf/vmstrap"]
-  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh"]
-  - [rm, -fr, /home/ubuntu/vmstrap]
-  - [su, -, ubuntu, -c, "echo 'export HOST_IP=192.168.10.11' >> .bashrc"]
-  - [su, -, ubuntu, -c, "cp /tmp/local.conf /home/ubuntu/devstack/"]
-  - [systemctl, restart, systemd-networkd]
+  - systemctl stop systemd-resolved.service
+  - systemctl disable systemd-resolved.service
+  - echo "nameserver 1.1.1.1" > /etc/resolv.conf
+  - apt purge -y snapd
+  - apt purge -y python3-pyasn1-modules
+  - apt purge -y python3-simplejson
+  - su - ubuntu -c "git clone https://opendev.org/openstack/devstack"
+  - su - ubuntu -c "cp /tmp/local.conf /home/ubuntu/devstack/"
+  - su - ubuntu -c "echo 'export HOST_IP=192.168.10.11' >> .bashrc"
+  - systemctl restart systemd-networkd
 boxpy_data:
   key: vm
   cpus: 4
   memory: 16GB
   disk_size: 50GB
+  version: 20.04
   advanced:
     nic2: intnet

examples/wmaker.yaml

@@ -1,5 +1,6 @@
 packages:
   - build-essential
+  - exuberant-ctags
   - gettext
   - libfontconfig1-dev
   - libgif-dev
@@ -21,13 +22,17 @@ packages:
   - libxrender-dev
   - libxt-dev
   - make
+  - mc
   - sharutils
+  - silversearcher-ag
+  - tmux
+  - vim-nox
   - xinit
 runcmd:
-  - [su, -, ubuntu, -c, "git clone https://github.com/gryf/wmaker -b experimental"]
-  - [su, -, ubuntu, -c, "git clone https://github.com/gryf/vmstrap"]
-  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh"]
-  - [rm, -fr, /home/ubuntu/vmstrap]
+  - su - ubuntu -c "git clone https://github.com/gryf/wmaker -b experimental"
+  - su - ubuntu -c "git clone https://github.com/gryf/vmstrap"
+  - su - ubuntu -c "vmstrap/bootstrap.sh -c"
+  - rm -fr /home/ubuntu/vmstrap
 boxpy_data:
   key: vm
   cpus: 4

pyproject.toml

@@ -0,0 +1,66 @@
+[build-system]
+requires = ["setuptools >= 77.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "boxpy"
+dynamic = ["version"]
+authors = [
+    {name = "Roman Dobosz", email = "gryf73@gmail.com"}
+]
+license = "GPL-3.0-or-later"
+description = "Run Linux cloud image on top of VirtualBox using commandline tool"
+readme = "README.rst"
+requires-python = ">=3.8"
+keywords = ["vboxmanage", "virtualbox", "vm", "virtual machine", "automation"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: End Users/Desktop",
+    "Topic :: Terminals",
+    "Topic :: Utilities",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3 :: Only"
+]
+dependencies = [
+    "pyyaml>=5.4.1",
+    "requests>=2.26.0"
+]
+
+[project.urls]
+Homepage = "https://github.com/gryf/boxpy"
+
+[project.scripts]
+boxpy = "box:main"
+
+[tool.setuptools]
+py-modules = ["box"]
+
+[tool.setuptools.dynamic]
+version = {attr = "box.__version__"}
+
+[tool.ruff]
+line-length = 79
+indent-width = 4
+
+[tool.ruff.lint]
+extend-select = [
+    "B",  # flake8-bugbear
+    "C4",  # flake8-comprehensions
+    "E",  # pycodestyle
+    "F",  # pyflakes
+    "FA",  # flake8-future-annotations
+    "G",  # flake8-logging-format
+    "N",  # pep8-naming
+    "PGH",  # pygrep-hooks
+    "PIE",  # flake8-pie
+    "RET",  # flake8-return
+    "SIM",  # flake8-simplify
+    "UP",  # pyupgrade
+    "W",  # pycodestyle
+    "YTT",  # flake8-2020
+]

requirements.txt

@@ -1,2 +0,0 @@
-pyyaml>=5.4.1
-requests>=2.26.0