Fix nasty bug for debian images

Issue/glitch with kernel panic on first run seems to be fixed on Debian
Bookworm (12). Do not reset VM for that version (and hopefully
upcoming).

README.rst

@@ -2,8 +2,8 @@
 box.py
 ======
 
-Box.py is a simple automation tool meant to run Ubuntu or Fedora cloud images
+Box.py is a simple automation tool meant to run Ubuntu, Fedora, Centos Stream
-on top of VirtualBox.
+or Debian cloud images on top of VirtualBox.
 
 What it does is simply download official cloud image, set up VM, tweak it up
 and do the initial pre-configuration using generated config drive.
@@ -16,14 +16,15 @@ weird named options for ``vboxmanage`` ;P)
 Requirements
 ------------
 
-- Python 3.x
+- Python >=3.8
 
   - `pyyaml`_
+  - `requests`_
 
 - Virtualbox (obviously)
-- ``mkisofs`` or ``genisoimage`` command for generating iso image
+- ``mkisofs`` or ``genisoimage`` command for generating ISO image
 - ``wget`` command for fetching images
-- ``sha256sum`` command for checksum check
+- ``sha256sum`` and ``sha512sum`` commands for checksum check
 - ``qemu-img`` from *qemu-utils* package command for converting between images
   formats
 
@@ -38,7 +39,7 @@ your operating system, or by using virtualenv for Python requirements, i.e.:
 
    $ python -m virtualenv .venv
    $ . .venv/bin/activate
-   (.venv) $ pip install requirements.txt
+   (.venv) $ pip install -r requirements.txt
 
 then you can issue:
 
@@ -54,7 +55,7 @@ or simply link it somewhere in the path:
    $ chmod +x ~/bin/boxpy
 
 and now you can issue some command. For example, to spin up a VM with Ubuntu
-18.04 with one CPU, 2GB of memory and 10GB of disk:
+18.04 with one CPU, 1GB of memory and 6GB of disk:
 
 .. code:: shell-session
 
@@ -90,17 +91,23 @@ use it ad-hoc, or place on your ``.bashrc`` or whatever:
 
 Currently, following commands are available:
 
-- ``list`` - for quickly listing all/running VMs
+- ``completion`` - as described above
-- ``info`` - to get summary about VM
-- ``destroy`` - that is probably obvious one
 - ``create`` - create new VM
+- ``destroy`` - that is probably obvious one
+- ``info`` - to get summary about VM
+- ``list`` - for quickly listing all/running VMs
 - ``rebuild`` - recreate specified VM
 - ``ssh`` - connect to the VM using ssh
-- ``completion`` - as described above
+- ``start`` - stop the running VM
+- ``stop`` - start stopped VM
 
 All of the commands have a range of options, and can be examined by using
 ``--help`` option.
 
+
+YAML Configuration
+------------------
+
 What is more interesting though, is the fact, that you can pass your own
 `cloud-init`_ yaml file, so that VM can be provisioned in easy way.
 
@@ -140,11 +147,20 @@ pass filenames to the custom config, instead of filling up
        permissions: '0644'
        filename: /path/to/local/file.txt
 
-during processing this file, boxpy will look for ``filename`` key in the yaml
+or
-file for the ``write_files`` sections, and it will remove that key, read the
+
-file and put its contents under ``content`` key. What is more important, that
+.. code:: yaml
-will be done after template processing, so there will be no interference for
+
-possible ``$`` characters.
+   write_files:
+     - path: /opt/somefile.txt
+       permissions: '0644'
+       url: https://some.url/content
+
+during processing this file, boxpy will look for ``filename`` or ``url`` keys
+in the yaml file for the ``write_files`` sections, and it will remove that key,
+read the file and put its contents under ``content`` key. What is more
+important, that will be done after template processing, so there will be no
+interference for possible ``$`` characters.
 
 What is more interesting is the fact, that you could use whatever cloud-init
 accepts, and a special section, for keeping configuration, so that you don't
@@ -193,6 +209,19 @@ configuration additional NIC for virtual machine, i.e:
      advanced:
        nic2: intnet
 
+To select image from local file system, it is possible to set one by providing
+it under ``boxpy_data.image`` key:
+
+.. code:: yaml
+
+   …
+   boxpy_data:
+     image: /path/to/the/qcow2/image
+     default_user: cloud-user
+
+Note, that default_user is also needed to be provided, as there is no guess,
+what is the default username for cloud-init configured within provided image.
+
 
 License
 -------
@@ -202,3 +231,4 @@ This work is licensed under GPL-3.
 
 .. _pyyaml: https://github.com/yaml/pyyaml
 .. _cloud-init: https://cloudinit.readthedocs.io
+.. _requests: https://docs.python-requests.org

examples/devstack-fedora34.yaml

@@ -1,3 +1,5 @@
+# Note, that cloud init will fail, due to old cloudinit package, which module
+# cc_keys_to_console doesn't recognize skipping option. Just ignore this error.
 package_update: true
 packages:
   - bash-completion
@@ -12,7 +14,6 @@ packages:
 write_files:
   - path: /tmp/local.conf
     permissions: '0644'
-    owner: fedora:fedora
     content: |
       [[local|localrc]]
       ADMIN_PASSWORD=pass
@@ -33,4 +34,4 @@ boxpy_data:
   memory: 4GB
   disk_size: 10GB
   distro: fedora
-  version: 32
+  version: 34

examples/devstack-ubuntu-20.04.yaml

@@ -19,7 +19,6 @@ packages:
 write_files:
   - path: /tmp/local.conf
     permissions: '0644'
-    owner: ubuntu:ubuntu
     content: |
       [[local|localrc]]
       ADMIN_PASSWORD=pass
@@ -27,8 +26,10 @@ write_files:
       RABBIT_PASSWORD=$$ADMIN_PASSWORD
       SERVICE_PASSWORD=$$ADMIN_PASSWORD
 runcmd:
+  - [apt, purge, '-y', python3-pyasn1-modules]
+  - [apt, purge, '-y', python3-simplejson]
   - [su, -, ubuntu, -c, "git clone https://github.com/gryf/vmstrap"]
-  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh"]
+  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh -c"]
   - [rm, -fr, /home/ubuntu/vmstrap]
   - [su, -, ubuntu, -c, "cp /tmp/local.conf /home/ubuntu/devstack/"]
   - [su, -, ubuntu, -c, "echo 'export HOST_IP=10.0.2.15' >> .bashrc"]

examples/kuryr_multinode_master.yaml

@@ -31,129 +31,154 @@ write_files:
               - 192.168.10.10/24
   - path: /tmp/local.conf
     permissions: '0644'
-    owner: ubuntu:ubuntu
     content: |
       [[local|localrc]]
       disable_all_services
-      
+
-      disable_service tls-proxy
-      
       # Cinder
       disable_service c-api
       disable_service c-bak
       disable_service c-sch
       disable_service c-vol
       disable_service cinder
-      
+
       # Coredns
       disable_service coredns
-      
+
-      # Dstat 
+      # Dstat
       disable_service dstat
-      
+
       # ETCD
       enable_service etcd3
-      
+
       # Glance
       enable_service g-api
       enable_service g-reg
-      
+
       # Horizon
       disable_service horizon
-      
+
       # Keystone
       enable_service key
-      
+
       # kubernetes
       enable_service kubernetes-master
       disable_service kubernetes-worker
-      
+
       # kuryr
       enable_service kuryr-daemon
       enable_service kuryr-kubernetes
-      
+
       # mysql.
       enable_service mysql
-      
+
       # Nova services
       enable_service n-api
       enable_service n-api-meta
       enable_service n-cond
       enable_service n-cpu
       enable_service n-sch
+
+      # Neutron
       enable_service neutron
       enable_service neutron-tag-ports-during-bulk-creation
+
+      # Octavia
       enable_service o-api
       enable_service o-cw
+      enable_service o-da
       enable_service o-hk
       enable_service o-hm
       enable_service octavia
-      
+
-      # OVN
+      # Neutron ovn services
       enable_service ovn-controller
       enable_service ovn-northd
       enable_service ovs-vswitchd
       enable_service ovsdb-server
-      
+
       # Placement API
       enable_service placement-api
       enable_service placement-client
-      
+
       # Neutron services
-      enable_service q-agt
-      enable_service q-dhcp
-      enable_service q-l3
-      enable_service q-meta
       enable_service q-ovn-metadata-agent
+      enable_service q-qos
       enable_service q-svc
+      enable_service q-trunk
       enable_service rabbit
+
+      # Swift
       disable_service s-account
       disable_service s-container
       disable_service s-object
       disable_service s-proxy
+
+      # Tempest
       enable_service tempest
-      enable_service tls-proxy
+
-      
+      # TLS
+      disable_service tls-proxy
+
       # Vars
       ADMIN_PASSWORD="secretadmin"
       DATABASE_PASSWORD="secretdatabase"
       ETCD_USE_RAMDISK="True"
+
       KURYR_ENABLED_HANDLERS="vif,endpoints,service,namespace,pod_label,policy,kuryrnetworkpolicy,kuryrnetwork,kuryrport,kuryrloadbalancer"
-      KURYR_EP_DRIVER_OCTAVIA_PROVIDER="amphora"
+      KURYR_ENFORCE_SG_RULES="False"
-      KURYR_K8S_API_PORT="6443"
+      KURYR_EP_DRIVER_OCTAVIA_PROVIDER="ovn"
-      KURYR_K8S_CLOUD_PROVIDER="False"
-      KURYR_K8S_CONTAINERIZED_DEPLOYMENT="True"
       KURYR_K8S_MULTI_WORKER_TESTS="True"
+      KURYR_K8S_OCTAVIA_MEMBER_MODE="L2"
+      KURYR_LB_ALGORITHM="SOURCE_IP_PORT"
+      KURYR_NEUTRON_DEFAULT_ROUTER="router1"
+      KURYR_SG_DRIVER="policy"
+      KURYR_SUBNET_DRIVER="namespace"
+
       LOGFILE="/opt/stack/logs/devstacklog.txt"
       LOG_COLOR="False"
-      ML2_L3_PLUGIN="router"
+      ML2_L3_PLUGIN="ovn-router,trunk,qos"
       OCTAVIA_AMP_IMAGE_FILE="/tmp/test-only-amphora-x64-haproxy-ubuntu-bionic.qcow2"
       OCTAVIA_AMP_IMAGE_NAME="test-only-amphora-x64-haproxy-ubuntu-bionic"
       OCTAVIA_AMP_IMAGE_SIZE="3"
-      Q_AGENT="openvswitch"
+
-      Q_ML2_TENANT_NETWORK_TYPE="vxlan"
+      OVN_BRANCH="v20.06.2"
-      Q_ML2_PLUGIN_MECHANISM_DRIVERS="openvswitch,linuxbridge"
+      OVN_BUILD_FROM_SOURCE="True"
+      OVN_DBS_LOG_LEVEL="dbg"
+      OVN_L3_CREATE_PUBLIC_NETWORK="True"
+
+      VAR_RUN_PATH="/usr/local/var/run"
+
       RABBIT_PASSWORD="secretrabbit"
       RECLONE="no"
       SERVICE_PASSWORD="secretservice"
       SERVICE_TOKEN="password"
-      TEMPEST_PLUGINS="/opt/stack/kuryr-tempest-plugin"
+
-      USE_PYTHON3="True"
+      LIBS_FROM_GIT=cinder,devstack,devstack-gate,devstack-plugin-container,glance,keystone,kuryr-kubernetes,kuryr-tempest-plugin,neutron,nova,octavia,octavia-tempest-plugin,ovn-octavia-provider,placement,python-octaviaclient,requirements,swift,tempest
-      
+
-      LIBS_FROM_GIT=cinder,devstack,devstack-gate,devstack-plugin-container,glance,keystone,kuryr-kubernetes,kuryr-tempest-plugin,neutron,nova,octavia,placement,python-octaviaclient,requirements,swift,tempest
+      TEMPEST_PLUGINS="/opt/stack/kuryr-tempest-plugin /opt/stack/octavia-tempest-plugin"
-      
+
       # enabled plugins
       enable_plugin devstack-plugin-container https://opendev.org/openstack/devstack-plugin-container
-      enable_plugin kuryr-kubernetes https://github.com/gryf/kuryr-kubernetes
+      enable_plugin kuryr-kubernetes https://opendev.org/openstack/kuryr-kubernetes
       enable_plugin kuryr-tempest-plugin https://opendev.org/openstack/kuryr-tempest-plugin
       enable_plugin neutron https://opendev.org/openstack/neutron
       enable_plugin octavia https://opendev.org/openstack/octavia
+      enable_plugin octavia-tempest-plugin https://opendev.org/openstack/octavia-tempest-plugin
+      enable_plugin ovn-octavia-provider https://opendev.org/openstack/ovn-octavia-provider
+
+      [[post-config|$OCTAVIA_CONF]]
+      [api_settings]
+      enabled_provider_drivers = amphora:'Octavia Amphora driver',ovn:'Octavia OVN driver'
 runcmd:
+  - [apt, purge, '-y', python3-pyasn1-modules]
+  - [apt, purge, '-y', python3-simplejson]
   - [su, -, ubuntu, -c, "git clone https://github.com/gryf/vmstrap"]
-  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh"]
+  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh -c"]
   - [rm, -fr, /home/ubuntu/vmstrap]
   - [su, -, ubuntu, -c, "echo 'export HOST_IP=192.168.10.10' >> .bashrc"]
   - [su, -, ubuntu, -c, "cp /tmp/local.conf /home/ubuntu/devstack/"]
+  - [systemctl, restart, systemd-networkd]
 boxpy_data:
   key: vm
   cpus: 4
@@ -161,3 +186,4 @@ boxpy_data:
   disk_size: 50GB
   advanced:
     nic2: intnet
+  version: 20.04

examples/kuryr_multinode_worker.yaml

@@ -31,83 +31,103 @@ write_files:
               - 192.168.10.11/24
   - path: /tmp/local.conf
     permissions: '0644'
-    owner: ubuntu:ubuntu
     content: |
       [[local|localrc]]
       disable_all_services
-     
+
-      disable_service tls-proxy
-      
       # Cinder
       disable_service c-bak
       disable_service c-vol
-      
+
       # Dstat (started to fail recently)
       disable_service dstat
-      
+
       # Horizon
       disable_service horizon
-      
+
       # kubernetes
       disable_service kubernetes-master
       enable_service kubernetes-worker
-      
+
       # kuryr
       enable_service kuryr-daemon
       disable_service kuryr-kubernetes
-      
+
+      # Nova
       enable_service n-cpu
-      
+
-      # OVN
+      # Neutron
+      enable_service neutron
       enable_service ovn-controller
+      disable_service ovn-northd
+      enable_service ovn-octavia-provider
       enable_service ovs-vswitchd
       enable_service ovsdb-server
-      
+
       # Placement API
       enable_service placement-client
-      
+
-      # Neutron services
+      # Neutron services cd
       enable_service q-ovn-metadata-agent
+      disable_service q-svc
+
+      # tempest
       disable_service tempest
+
+      # tls
       disable_service tls-proxy
-      
+
       # Vars
       ADMIN_PASSWORD="secretadmin"
       DATABASE_HOST="192.168.10.10"
       DATABASE_PASSWORD="secretdatabase"
+      ENABLE_CHASSIS_AS_GW="False"
       GLANCE_HOSTPORT="192.168.10.10:9292"
-      ML2_L3_PLUGIN="router"
+
+      # turn on ovn-provider
+      KURYR_ENFORCE_SG_RULES="False"
+      KURYR_EP_DRIVER_OCTAVIA_PROVIDER="ovn"
+      KURYR_K8S_OCTAVIA_MEMBER_MODE="L2"
+      KURYR_LB_ALGORITHM="SOURCE_IP_PORT"
+      KURYR_NEUTRON_DEFAULT_ROUTER="router1"
+      VAR_RUN_PATH="/usr/local/var/run"
+
       KURYR_ENABLED_HANDLERS="vif,endpoints,service,namespace,pod_label,policy,kuryrnetworkpolicy,kuryrnetwork,kuryrport,kuryrloadbalancer"
-      KURYR_FORCE_IMAGE_BUILD="True"
+      KURYR_SG_DRIVER="policy"
-      KURYR_EP_DRIVER_OCTAVIA_PROVIDER="amphora"
+      KURYR_SUBNET_DRIVER="namespace"
-      KURYR_K8S_API_PORT="6443"
+
-      KURYR_K8S_CLOUD_PROVIDER="False"
+
-      KURYR_K8S_CONTAINERIZED_DEPLOYMENT="True"
+      OVN_BRANCH="v20.06.2"
+      OVN_BUILD_FROM_SOURCE="True"
+      OVN_DBS_LOG_LEVEL="dbg"
+      OVN_L3_CREATE_PUBLIC_NETWORK="True"
+
+
+      LIBVIRT_TYPE="qemu"
       LOGFILE="/opt/stack/logs/devstacklog.txt"
       LOG_COLOR="False"
-      Q_AGENT="openvswitch"
+      Q_HOST="192.168.10.10"
-      Q_ML2_TENANT_NETWORK_TYPE="vxlan"
-      Q_ML2_PLUGIN_MECHANISM_DRIVERS="openvswitch,linuxbridge"
       RABBIT_HOST="192.168.10.10"
       RABBIT_PASSWORD="secretrabbit"
       RECLONE="no"
       SERVICE_HOST="192.168.10.10"
       SERVICE_PASSWORD="secretservice"
       SERVICE_TOKEN="password"
-      TEMPEST_PLUGINS="/opt/stack/kuryr-tempest-plugin"
+
-      USE_PYTHON3="True"
-      
       LIBS_FROM_GIT=cinder,devstack,devstack-gate,devstack-plugin-container,glance,keystone,kuryr-kubernetes,kuryr-tempest-plugin,neutron,nova,octavia,placement,python-octaviaclient,requirements,swift,tempest
-      
+
       # enabled plugins
       enable_plugin devstack-plugin-container https://opendev.org/openstack/devstack-plugin-container
       enable_plugin kuryr-kubernetes https://opendev.org/openstack/kuryr
 runcmd:
+  - [apt, purge, '-y', python3-pyasn1-modules]
+  - [apt, purge, '-y', python3-simplejson]
   - [su, -, ubuntu, -c, "git clone https://github.com/gryf/vmstrap"]
-  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh"]
+  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh -c"]
   - [rm, -fr, /home/ubuntu/vmstrap]
   - [su, -, ubuntu, -c, "echo 'export HOST_IP=192.168.10.11' >> .bashrc"]
   - [su, -, ubuntu, -c, "cp /tmp/local.conf /home/ubuntu/devstack/"]
+  - [systemctl, restart, systemd-networkd]
 boxpy_data:
   key: vm
   cpus: 4
@@ -115,4 +135,4 @@ boxpy_data:
   disk_size: 50GB
   advanced:
     nic2: intnet
-  port: 2223
+  version: 20.04

examples/wmaker.yaml

@@ -1,5 +1,6 @@
 packages:
   - build-essential
+  - exuberant-ctags
   - gettext
   - libfontconfig1-dev
   - libgif-dev
@@ -21,12 +22,16 @@ packages:
   - libxrender-dev
   - libxt-dev
   - make
+  - mc
   - sharutils
+  - silversearcher-ag
+  - tmux
+  - vim-nox
   - xinit
 runcmd:
   - [su, -, ubuntu, -c, "git clone https://github.com/gryf/wmaker -b experimental"]
   - [su, -, ubuntu, -c, "git clone https://github.com/gryf/vmstrap"]
-  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh"]
+  - [su, -, ubuntu, -c, "vmstrap/bootstrap.sh -c"]
   - [rm, -fr, /home/ubuntu/vmstrap]
 boxpy_data:
   key: vm

requirements.txt

@@ -1 +1,2 @@
 pyyaml>=5.4.1
+requests>=2.26.0