Added WHDLoad support

README.rst

@@ -314,6 +314,155 @@ The steps would be as follows:
 - optionally create archive with save state (if save state directory place is
   *not* a global one)
 
+whdload
+-------
+
+Options used:
+
+* ``wrapper`` (required) with ``whdload`` as an value
+* ``wrapper_whdload_base`` (required) path to the whdload base system. Usually
+  it's minimal system containing at least whdload executables in C, and config
+  in S. Read on below for further details.
+* ``wrapper_archive`` (optional) path to the whdload archive, defaults to same
+  name as configuration file with some detected archive extension. Note, that
+  name is case sensitive
+* ``wrapper_archiver`` (optional) archiver to use for storage save state -
+  default ``7z``.
+
+This module is solely used with whdload distributed games (not just whdload
+slave files, but whole games, which can be found on several places on the
+internet).
+
+Base image
+~~~~~~~~~~
+
+To make it work, first the minimal system archive need to be prepared. There
+are few dependences to be included in such small system:
+
+- `WHDLoad`_ 18.9
+- `uaequit`_
+- `SetPatch`_ 43.6
+- ``Excecute``, ``Assign`` and whatever commands you'll be use in scripts from
+  your copy of Workbench
+- `kgiconload`_ - tool for reading icon and executing *default tool* with
+  optionally defined tool types as parameters (in this case: WHDLoad)
+- `SKick`_ optionally - for kickstart relocations. Also images of corresponding
+  kickstart ROM images will be needed.
+
+Now, the tree for the minimal image could look like that:
+
+.. code::
+   .
+   ├── C
+   │   ├── Assign
+   │   ├── DIC
+   │   ├── Execute
+   │   ├── kgiconload
+   │   ├── Patcher
+   │   ├── RawDIC
+   │   ├── SetPatch
+   │   ├── UAEquit
+   │   ├── WHDLoad
+   │   └── WHDLoadCD32
+   └── S
+       ├── startup-sequence
+       └── WHDLoad.prefs
+
+to use relocation tables you'll need to place ``Kickstarts`` drawer into Devs
+drawer, so it'll looks like this:
+
+.. code::
+   .
+   ├── C
+   │   ├── Assign
+   │   ├── …
+   │   └── WHDLoadCD32
+   ├── Devs
+   │   └── Kickstarts
+   │       ├── 39115_ROMKick.PAT
+   │       ├── 39115_ROMKick.RTB
+   │       ├── …
+   │       ├── kick40068.A4000.PAT
+   │       └── kick40068.A4000.RTB
+   └── S
+       ├── startup-sequence
+       └── WHDLoad.prefs
+
+Important: You'll need to prepare archive with base OS without top directory,
+i.e. suppose you have prepared all the files in ``/tmp/baseos``:
+
+.. code:: shell-session
+
+   $ pwd
+   /tmp
+   $ cd baseos
+   $ pwd
+   /tmp/basos
+   $ ls
+   C    S
+   $ zip -r /tmp/base.zip .
+     adding: C/ (stored 0%)
+     adding: C/Assign (deflated 31%)
+     adding: C/WHDLoadCD32 (deflated 26%)
+     adding: C/RawDIC (deflated 46%)
+     adding: C/UAEquit (deflated 39%)
+     adding: C/Execute (deflated 42%)
+     adding: C/Patcher (deflated 56%)
+     adding: C/DIC (deflated 33%)
+     adding: C/kgiconload (deflated 49%)
+     adding: C/SetPatch (deflated 39%)
+     adding: C/WHDLoad (deflated 23%)
+     adding: S/ (stored 0%)
+     adding: S/startup-sequence (deflated 36%)
+     adding: S/WHDLoad.prefs (deflated 51%)
+
+You can do it with other archivers as well, like 7z: ``7z a /tmp/base.7z .`` or
+tar with different compressions: ``tar Jcf /tmp/base.tar.xz .``, ``tar zcf
+/tmp/base.tgz .``, ``tar jcf /tmp/base.tar.bz2 .``. It should work with all
+mentioned at the beginning of this document archivers.
+
+Starting point is in ``S/startup-sequence`` file, where eventually 
+``S/whdload-startup`` is executed, which will be created by fs-uae-warpper
+before execution by fs-uae.
+
+
+Configuration
+~~~~~~~~~~~~~
+
+Now, to use whdload module with any of the WHDLoad game, you'll need to prepare
+configuration for the wrapper.
+
+Example configuration:
+
+.. code:: ini
+
+   [config]
+   wrapper = whdload
+   wrapper_whdload_base = $CONFIG/whdload_base.7z
+   # ...
+
+And execution is as usual:
+
+.. code:: shell-session
+
+   $ fs-uae-wrapper ChaosEngine_v1.2_0106.fs-uae
+
+Now, similar to the archive module, it will create temporary directory, unpack
+base image there, unpack WHDLoad game archive, search for slave file, and
+preapre ``s:whdload-startup``, and finally pass all the configuration to
+fs-uae.
+
+
+Limitations
+===========
+
+There is one limitation when using save ``wrapper_save_state`` option. In case
+of floppies it should work without any issues, although save state for running
+Workbench or WHDLoad games may or may not work. In the past there was an issue
+with `fs-uae`_ where saving state was causing data corruption on the emulated
+system. Use it with caution!
+
+
 License
 =======
 
@@ -328,3 +477,8 @@ This work is licensed on 3-clause BSD license. See LICENSE file for details.
 .. _tar: https://www.gnu.org/software/tar/
 .. _zip: http://www.info-zip.org
 .. _CheeseShop: https://pypi.python.org/pypi/fs-/fs-uae-wrapperuae-wrapper
+.. _WHDLoad: https://www.whdload.de
+.. _uaequit: https://aminet.net/package/misc/emu/UAEquit
+.. _SKick: https://aminet.net/package/util/boot/skick346
+.. _SetPatch: https://aminet.net/package/util/boot/SetPatch_43.6b
+.. _kgiconload: https://eab.abime.net/showpost.php?p=733614&postcount=92

fs_uae_wrapper/base.py

@@ -177,13 +177,14 @@ class Base(object):
         Configuration file will be placed in new directory, therefore it is
         needed to calculate new paths so that emulator can find assets.
         """
-        options = ['wrapper_archive', 'accelerator_rom', 'base_dir',
+        logging.debug("_normalize_options")
-                   'cdrom_drive_0', 'cdroms_dir', 'controllers_dir',
+        options = ['wrapper_archive', 'wrapper_whdload_base',
-                   'cpuboard_flash_ext_file', 'cpuboard_flash_file',
+                   'accelerator_rom', 'base_dir', 'cdrom_drive_0',
-                   'floppies_dir', 'floppy_overlays_dir', 'fmv_rom',
+                   'cdroms_dir', 'controllers_dir', 'cpuboard_flash_ext_file',
-                   'graphics_card_rom', 'hard_drives_dir', 'kickstart_file',
+                   'cpuboard_flash_file', 'floppies_dir',
-                   'kickstarts_dir', 'logs_dir', 'save_states_dir',
+                   'floppy_overlays_dir', 'fmv_rom', 'graphics_card_rom',
-                   'screenshots_output_dir']
+                   'hard_drives_dir', 'kickstart_file', 'kickstarts_dir',
+                   'logs_dir', 'save_states_dir', 'screenshots_output_dir']
 
         for num in range(20):
             options.append('cdrom_image_%d' % num)

fs_uae_wrapper/whdload.py

@@ -0,0 +1,115 @@
+"""
+Run fs-uae with WHDLoad games
+
+It will use compressed base image and compressed directories.
+"""
+import logging
+import os
+import shutil
+
+from fs_uae_wrapper import base
+from fs_uae_wrapper import utils
+
+
+class Wrapper(base.ArchiveBase):
+    """
+    Class for performing extracting archive, copying emulator files, and
+    cleaning it back again
+    """
+    def __init__(self, conf_file, fsuae_options, configuration):
+        super(Wrapper, self).__init__(conf_file, fsuae_options, configuration)
+        self.archive_type = None
+
+    def run(self):
+        """
+        Main function which accepts configuration file for FS-UAE
+        It will do as follows:
+            - extract base image and archive file
+            - copy configuration
+            - run the emulation
+        """
+        logging.debug("run")
+        if not super().run():
+            return False
+
+        if not self._extract():
+            return False
+
+        if not self._copy_conf():
+            return False
+
+        return self._run_emulator()
+
+    def _validate_options(self):
+        """
+        Do the validation for the options, additionally check if there is
+        mandatory WHDLoad base OS images set.
+        """
+        if not super()._validate_options():
+            return False
+
+        if not self.all_options.get('wrapper_whdload_base'):
+            logging.error("wrapper_whdload_base is not set in configuration, "
+                          "exiting.")
+            return False
+        return True
+
+    def _extract(self):
+        """Extract base image and then WHDLoad archive"""
+        base_image = self.fsuae_options['wrapper_whdload_base']
+        if not os.path.exists(base_image):
+            logging.error("Base image `%s` does't exists in provided "
+                          "location.", base_image)
+            return False
+
+        title = self._get_title()
+        curdir = os.path.abspath('.')
+        os.chdir(self.dir)
+        result = utils.extract_archive(base_image)
+        os.chdir(curdir)
+        if not result:
+            return False
+
+        if not super()._extract():
+            return False
+
+        return self._find_slave()
+
+    def _find_slave(self):
+        """Find Slave file and create apropriate entry in S:whdload-startup"""
+        curdir = os.path.abspath('.')
+        os.chdir(self.dir)
+
+        # find slave name
+        slave_fname = None
+        slave_path = None
+
+        for root, dirs, fnames in os.walk('.'):
+            for fname in fnames:
+                if fname.lower().endswith('.slave'):
+                    slave_path, slave_fname = os.path.normpath(root), fname
+                    break
+        if slave_fname is None:
+            logging.error("Cannot find .slave file in archive.")
+            return False
+
+        # find corresponfing info (an icon) fname
+        icon_fname = None
+        for fname in os.listdir(slave_path):
+            if (fname.lower().endswith('.info') and
+                os.path.splitext(slave_fname)[0].lower() ==
+                os.path.splitext(fname)[0].lower()):
+                icon_fname = fname
+                break
+        if icon_fname is None:
+            logging.error("Cannot find .info file corresponding to %s in "
+                          "archive.", slave_fname)
+            return False
+
+        # Write startup file
+        with open("S/whdload-startup", "w") as fobj:
+            fobj.write(f"cd {slave_path}\n")
+            fobj.write(f"C:kgiconload {icon_fname}\n")
+
+        os.chdir(curdir)
+        return True

pyproject.toml

@@ -10,7 +10,7 @@ description = "Automate archives support and state saves for fs-uae"
 readme = "README.rst"
 requires-python = ">=3.8"
 keywords = ["uae", "fs-uae", "amiga", "emulator", "wrapper"]
-version = "0.9.1"
+version = "0.10.0"
 classifiers = [
     "Development Status :: 5 - Production/Stable",
     "Environment :: Console",

tests/test_whdload.py

@@ -0,0 +1,192 @@
+import os
+import shutil
+from tempfile import mkdtemp
+from unittest import TestCase
+from unittest import mock
+
+from fs_uae_wrapper import whdload
+from fs_uae_wrapper import utils
+
+
+class TestWHDLoad(TestCase):
+
+    def setUp(self):
+        self.dirname = mkdtemp()
+        self.curdir = os.path.abspath(os.curdir)
+        os.chdir(self.dirname)
+
+    def tearDown(self):
+        os.chdir(self.curdir)
+        try:
+            shutil.rmtree(self.dirname)
+        except OSError:
+            pass
+
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._validate_options')
+    def test_validate_options_arch_validation_fail(self, base_valid):
+
+        base_valid.return_value = False
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertFalse(wrapper._validate_options())
+
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._validate_options')
+    def test_validate_options_no_base_image(self, base_valid):
+
+        base_valid.return_value = True
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertFalse(wrapper._validate_options())
+
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._validate_options')
+    def test_validate_options_with_base_image_set(self, base_valid):
+
+        base_valid.return_value = True
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        wrapper.all_options['wrapper_whdload_base'] = 'fake_base_fname.7z'
+        self.assertTrue(wrapper._validate_options())
+
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase.run')
+    def test_run_base_run_fail(self, run):
+
+        run.return_value = False
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertFalse(wrapper.run())
+
+    @mock.patch('fs_uae_wrapper.whdload.Wrapper._extract')
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase.run')
+    def test_run_extract_fail(self, run, extract):
+
+        run.return_value = True
+        extract.return_value = False
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        wrapper.all_options = {'wrapper': 'whdload',
+                               'wrapper_archive': 'fake.tgz',
+                               'wrapper_archiver': 'rar'}
+        self.assertFalse(wrapper.run())
+
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._copy_conf')
+    @mock.patch('fs_uae_wrapper.whdload.Wrapper._extract')
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase.run')
+    def test_run_copy_conf_fail(self, run, extract, copy_conf):
+
+        run.return_value = True
+        extract.return_value = True
+        copy_conf.return_value = False
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertFalse(wrapper.run())
+
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._run_emulator')
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._copy_conf')
+    @mock.patch('fs_uae_wrapper.whdload.Wrapper._extract')
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase.run')
+    def test_run_emulator_fail(self, run, extract, copy_conf, run_emulator):
+
+        run.return_value = True
+        extract.return_value = True
+        copy_conf.return_value = True
+        run_emulator.return_value = False
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertFalse(wrapper.run())
+
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._run_emulator')
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._copy_conf')
+    @mock.patch('fs_uae_wrapper.whdload.Wrapper._extract')
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase.run')
+    def test_run_success(self, run, extract, copy_conf, run_emulator):
+
+        run.return_value = True
+        extract.return_value = True
+        copy_conf.return_value = True
+        run_emulator.return_value = True
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertTrue(wrapper.run())
+
+    @mock.patch('os.path.exists')
+    def test_extract_nonexistent_image(self, exists):
+        exists.return_value = False
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        wrapper.fsuae_options['wrapper_whdload_base'] = 'fakefilename'
+        self.assertFalse(wrapper._extract())
+
+    @mock.patch('os.chdir')
+    @mock.patch('os.path.exists')
+    def test_extract_extraction_failed(self, exists, chdir):
+        exists.return_value = True
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        wrapper.fsuae_options['wrapper_whdload_base'] = 'fakefilename.7z'
+        self.assertFalse(wrapper._extract())
+
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._extract')
+    @mock.patch('fs_uae_wrapper.utils.extract_archive')
+    @mock.patch('os.chdir')
+    @mock.patch('os.path.exists')
+    def test_extract_extraction_of_whdload_arch_failed(self, exists, chdir,
+                                                       image_extract,
+                                                       arch_extract):
+        exists.return_value = True
+        image_extract.return_value = True
+        arch_extract.return_value = False
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        wrapper.fsuae_options['wrapper_whdload_base'] = 'fakefilename'
+        self.assertFalse(wrapper._extract())
+
+    @mock.patch('fs_uae_wrapper.whdload.Wrapper._find_slave')
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._extract')
+    @mock.patch('fs_uae_wrapper.utils.extract_archive')
+    @mock.patch('os.chdir')
+    @mock.patch('os.path.exists')
+    def test_extract_slave_not_found(self, exists, chdir, image_extract,
+                                     arch_extract, find_slave):
+        exists.return_value = True
+        image_extract.return_value = True
+        arch_extract.return_value = True
+        find_slave.return_value = False
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        wrapper.fsuae_options['wrapper_whdload_base'] = 'fakefilename'
+        self.assertFalse(wrapper._extract())
+
+    @mock.patch('fs_uae_wrapper.whdload.Wrapper._find_slave')
+    @mock.patch('fs_uae_wrapper.base.ArchiveBase._extract')
+    @mock.patch('fs_uae_wrapper.utils.extract_archive')
+    @mock.patch('os.chdir')
+    @mock.patch('os.path.exists')
+    def test_extract_success(self, exists, chdir, image_extract, arch_extract,
+                             find_slave):
+        exists.return_value = True
+        image_extract.return_value = True
+        arch_extract.return_value = True
+        find_slave.return_value = True
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        wrapper.fsuae_options['wrapper_whdload_base'] = 'fakefilename'
+        self.assertTrue(wrapper._extract())
+
+    @mock.patch('os.walk')
+    @mock.patch('os.chdir')
+    def test_find_slave_no_slave_file(self, chdir, walk):
+        walk.return_value = [(".", ('game'), ()),
+                            ('./game', (), ('foo', 'bar', 'baz'))]
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertFalse(wrapper._find_slave())
+
+    @mock.patch('os.listdir')
+    @mock.patch('os.walk')
+    @mock.patch('os.chdir')
+    def test_find_slave_no_corresponding_icon(self, chdir, walk, listdir):
+        contents = ('foo', 'bar', 'baz.slave')
+        walk.return_value = [(".", ('game'), ()),
+                            ('./game', (), contents)]
+        listdir.return_value = contents
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertFalse(wrapper._find_slave())
+
+    @mock.patch('builtins.open')
+    @mock.patch('os.listdir')
+    @mock.patch('os.walk')
+    @mock.patch('os.chdir')
+    def test_find_slave_success(self, chdir, walk, listdir, bopen):
+        contents = ('foo', 'bar', 'baz.slave', 'baz.info')
+        walk.return_value = [(".", ('game'), ()),
+                            ('./game', (), contents)]
+        listdir.return_value = contents
+        wrapper = whdload.Wrapper('Config.fs-uae', utils.CmdOption(), {})
+        self.assertTrue(wrapper._find_slave())
+        bopen.assert_called_once()