Removed tilda mapping, it's handled by c1541 now

Starting from beginning of 2020 Python 2.7 is no longer supported[1],
hence the change of shebang for uc1541 script and removal of _ord
function.

[1] https://devguide.python.org/versions/

README.rst

@@ -1,5 +1,5 @@
 ===================================================
-Midnight Commander virtual filesystem for d64 files
+Midnight Commander virtual filesystem for C64 disks
 ===================================================
 
 This extfs provides an access to disk image files for the Commodore
@@ -10,17 +10,17 @@ made by Commodore.
 Features
 ========
 
-* view, remove, copy files in and out of the image
+* View, remove, copy files in and out of the image,
-* support for read from and write to d64 images
+* Support for read from and write to d64, d71 and d81 images,
-* gzipped disk images support
+* Gzipped disk images support,
-* onfly characters remapping (see below)
+* On fly characters remapping (see below).
 
 Remarks
 =======
 
-Due to different way of representing file entries on regular D64 disk images,
+Due to different way of representing file entries on regular Commodore disk
-there could be issues with filenames that are transfered from/to the image.
+images, there could be issues with filenames that are transfered from/to the
-Following rules was applied to represent a single file entry:
+image. Following rules was applied to represent a single file entry:
 
 1. An extension is attached to the end of a filename depending on a file type.
    Possible extensions are: ``prg``, ``del``, ``seq``, ``usr`` and ``rel``.
@@ -30,21 +30,20 @@ Following rules was applied to represent a single file entry:
 3. Every slash character (``/``) will be replaced by pipe character (``|``).
 4. Leading space will be replaced by tilda (``~``).
 
-While copying from D64 image to filesystem, filenames will be stored as they
+While copying from disk image to filesystem, filenames will be stored as they
 are seen on a listing.
 
-While copying from filesystem to D64 image, filename conversion will be done:
+While copying from filesystem to disk image, filename conversion will be done:
 
-1. Every ``$`` and ``*`` characters will be replaced by question mark (``?``)
+1. Every ``$`` and ``*`` characters will be replaced by question mark (``?``).
-2. Every pipe (``|``) and backslash (``\``) characters will be replaced by
+2. Every pipe (``|``) and backslash (``\\``) characters will be replaced by
-   slash (``/``)
+   slash (``/``).
-3. Every tilda (``~``) will be replaced by a space
+3. ``prg`` extension will be truncated.
-4. ``prg`` extension will be truncated
 
 Representation of a directory can be sometimes confusing - in case when one
 copied file without extension it stays there in such form, till next access
-(after flushing VFS). Also file sizes are not accurate, since D64 directory
+(after flushing VFS). Also file sizes are not accurate, since directory entries
-entries have sizes stored as 256 bytes blocks.
+in CBM format have sizes stored as 256 bytes blocks.
 
 Screens below shows how it looks like on real C64 machine and its representation
 on Midnight Commander listing.
@@ -57,20 +56,30 @@ file.
 Rquirements
 ===========
 
-* Python 2.7
+* Python 3.6 or higher (checked recently with 3.11)
 * Vice installation (c1541 program in path)
 
 Installation
 ============
 
 * copy ``uc1541`` to ``~/.local/share/mc/extfs.d/``
-* add or change entry for files handle in ``~/.config/mc/mc.ext``::
+* add or change entry for files handle in ``~/.config/mc/mc.ext``:
+
+.. code:: ini
 
    # Disk images for Commodore computers (VIC20, C64, C128)
-    regex/\.(d64|D64|d6z|D6z|d6Z|D6Z)$
+   [d64]
+   Regex=\.(d64|d71|d81)$
+   RegexIgnoreCase=true
    Open=%cd %p/uc1541://
    View=%view{ascii} c1541 %f -list
-        Extract=c1541 %f -extract
+
+   [d6z]
+   Regex=\.(d[678]z)$
+   RegexIgnoreCase=true
+   Open=%cd %p/uc1541://
+   View=%view{ascii} t=$(mktemp); zcat %f > ${t}; c1541 ${t} -list 2>/dev/null; rm ${t}
+   
 
 Configuration
 =============
@@ -87,6 +96,15 @@ script behaviour:
 Changelog
 =========
 
+* **3.6** Fixed line matcher for directory entries.
+* **3.5** Drop Python2 support.
+* **3.4** Code cleanup. Removed dummy logger class and sys.args based argument
+  parsing.
+* **3.3** Added support for .d71 and .d81 disk images.
+* **3.2** Changed shebang to ``python`` executable instead of ``python3``
+* **3.1** Argparse on Python3 have different behaviour, where if no subcommand
+  is provided, it will pass anyway. Fixed.
+* **3.0** Added beta quality Python3 support
 * **2.8** Treat non standard discs a bit better
 * **2.7** Added support for gzipped disk images
 * **2.6** Added mkdir and run handling (or rather lack of handling :). Minor

uc1541

@@ -1,24 +1,25 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 """
 UC1541 Virtual filesystem
 
 Author: Roman 'gryf' Dobosz <gryf73@gmail.com>
-Date: 2014-01-04
+Date: 2023-10-04
-Version: 2.8
+Version: 3.6
 Licence: BSD
-source: https://bitbucket.org/gryf/uc1541
+source: https://gihub.com/gryf/uc1541
 mirror: https://github.com/gryf/uc1541
 """
-
+import argparse
-import sys
-import re
-import os
 import gzip
-from subprocess import Popen, PIPE
-
-if os.getenv('UC1541_DEBUG'):
 import logging
+import os
+import re
+import subprocess
+import sys
+
+
 LOG = logging.getLogger('UC1541')
+if os.getenv('UC1541_DEBUG'):
     LOG.setLevel(logging.DEBUG)
     FILE_HANDLER = logging.FileHandler("/tmp/uc1541.log")
     FILE_FORMATTER = logging.Formatter("%(asctime)s %(levelname)-8s "
@@ -26,35 +27,55 @@ if os.getenv('UC1541_DEBUG'):
     FILE_HANDLER.setFormatter(FILE_FORMATTER)
     FILE_HANDLER.setLevel(logging.DEBUG)
     LOG.addHandler(FILE_HANDLER)
-else:
+
-    class LOG(object):
+
+SECLEN = 256
+
+
+def _get_raw(dimage):
     """
-        Dummy logger object. Does nothing.
+    Try to get contents of the D64 image either it's gzip compressed or not.
     """
-        @classmethod
+    raw = None
-        def debug(*args, **kwargs):
+    with gzip.open(dimage, 'rb') as fobj:
+        # Although the common approach with gzipped files is to check the
+        # magic number, in this case there is no guarantee that first track
+        # does not contain exactly the same byte sequence as the magic number.
+        # So the only way left is to actually try to uncompress the file.
+        try:
+            raw = fobj.read()
+        except (IOError, OSError):
             pass
+    if not raw:
+        with open(dimage, 'rb') as fobj:
+            raw = fobj.read()
 
-        @classmethod
+    return raw
-        def info(*args, **kwargs):
-            pass
-
-        @classmethod
-        def warning(*args, **kwargs):
-            pass
-
-        @classmethod
-        def error(*args, **kwargs):
-            pass
-
-        @classmethod
-        def critical(*args, **kwargs):
-            pass
 
 
-class D64(object):
+def _get_implementation(disk):
     """
-    Implement d64 directory reader
+    Check the file under fname and return right class for creating an object
+    corresponding for the file
+    """
+    len_map = {822400: D81,  # 80 tracks
+               819200: D81,  # 80 tracks, 3200 error bytes
+               349696: D71,  # 70 tracks
+               351062: D71,  # 70 tracks, 1366 error bytes
+               174848: D64,  # usual d64 disc image, 35 tracks, no errors
+               175531: D64,  # 35 track, 683 error bytes
+               196608: D64,  # 40 track, no errors
+               197376: D64}  # 40 track, 768 error bytes
+
+    if disk[:32].startswith(b'C64'):
+        return  # T64
+
+    return len_map.get(len(disk))(disk)
+
+
+class Disk(object):
+    """
+    Represent common disk interface
     """
     CHAR_MAP = {32: ' ', 33: '!', 34: '"', 35: '#', 37: '%', 38: '&', 39: "'",
                 40: '(', 41: ')', 42: '*', 43: '+', 44: ',', 45: '-', 46: '.',
@@ -81,36 +102,20 @@ class D64(object):
                   0b011: 'usr',
                   0b100: 'rel'}
 
-    def __init__(self, dimage):
+    DIR_TRACK = 18
+    DIR_SECTOR = 1
+
+    def __init__(self, raw):
         """
         Init
         """
-        LOG.debug('image: %s', dimage)
+        self.raw = raw
-        self.raw = None
         self.current_sector_data = None
         self.next_sector = 0
         self.next_track = None
         self._dir_contents = []
         self._already_done = []
 
-        self._get_raw(dimage)
-
-    def _get_raw(self, dimage):
-        """Try to get contents of the D64 image either it's gzip compressed or
-        not."""
-        fobj = gzip.open(dimage)
-        # Although the common approach with gzipped files is to check the
-        # magic number, in this case there is no guarantee that first track
-        # does not contain exactly the same byte sequence as the magic number.
-        # So the only way left is to actually try to uncompress the file.
-        try:
-            self.raw = fobj.read()
-        except IOError:
-            fobj.close()
-            fobj = open(dimage)
-            self.raw = fobj.read()
-            fobj.close()
-
     def _map_filename(self, string):
         """
         Transcode filename to ASCII compatible. Replace not supported
@@ -120,10 +125,10 @@ class D64(object):
         filename = list()
 
         for chr_ in string:
-            if ord(chr_) == 160:  # shift+space character; $a0
+            if chr_ == 160:  # shift+space character; $a0
                 break
 
-            character = D64.CHAR_MAP.get(ord(chr_), '?')
+            character = D64.CHAR_MAP.get(chr_, '?')
             filename.append(character)
 
         # special cases
@@ -149,22 +154,23 @@ class D64(object):
             return False
 
         if self.next_track is None:
-            LOG.debug("Going to the track: 18,1")
+            LOG.debug("Going to the track: %s, %s", self.DIR_TRACK,
-            offset = self._get_d64_offset(18, 1)
+                      self.DIR_SECTOR)
+            offset = self._get_offset(self.DIR_TRACK, self.DIR_SECTOR)
         else:
-            offset = self._get_d64_offset(self.next_track, self.next_sector)
+            offset = self._get_offset(self.next_track, self.next_sector)
             LOG.debug("Going to the track: %s,%s", self.next_track,
                       self.next_sector)
 
-        self.current_sector_data = self.raw[offset:offset + 256]
+        self.current_sector_data = self.raw[offset:offset + SECLEN]
 
         # Guard for reading data out of bound - that happened for discs which
-        # store only raw data, even on 18 track
+        # store only raw data, even on directory track
         if not self.current_sector_data:
             return False
 
-        self.next_track = ord(self.current_sector_data[0])
+        self.next_track = self.current_sector_data[0]
-        self.next_sector = ord(self.current_sector_data[1])
+        self.next_sector = self.current_sector_data[1]
 
         if (self.next_track, self.next_sector) in self._already_done:
             # Just a failsafe. Endless loop is not what is expected.
@@ -185,30 +191,11 @@ class D64(object):
                                                   num & 2 and 1,
                                                   num & 1), 2), '???')
 
-    def _get_d64_offset(self, track, sector):
+    def _get_offset(self, track, sector):
         """
         Return offset (in bytes) for specified track and sector.
         """
-
+        return 0
-        offset = 0
-        truncate_track = 0
-
-        if track > 17:
-            offset = 17 * 21 * 256
-            truncate_track = 17
-
-        if track > 24:
-            offset += 6 * 19 * 256
-            truncate_track = 24
-
-        if track > 30:
-            offset += 5 * 18 * 256
-            truncate_track = 30
-
-        track = track - truncate_track
-        offset += track * sector * 256
-
-        return offset
 
     def _harvest_entries(self):
         """
@@ -217,7 +204,7 @@ class D64(object):
         sector = self.current_sector_data
         for dummy in range(8):
             entry = sector[:32]
-            ftype = ord(entry[2])
+            ftype = entry[2]
 
             if ftype == 0:  # deleted
                 sector = sector[32:]
@@ -225,12 +212,12 @@ class D64(object):
 
             type_verbose = self._get_ftype(ftype)
 
-            protect = ord(entry[2]) & 64 and "<" or " "
+            protect = entry[2] & 64 and "<" or " "
             fname = entry[5:21]
             if ftype == 'rel':
-                size = ord(entry[23])
+                size = entry[23]
             else:
-                size = ord(entry[30]) + ord(entry[31]) * 226
+                size = entry[30] + entry[31] * 226
 
             self._dir_contents.append({'fname': self._map_filename(fname),
                                        'ftype': type_verbose,
@@ -249,11 +236,128 @@ class D64(object):
         return self._dir_contents
 
 
+class D64(Disk):
+    """
+    Implement d64 directory reader
+    """
+
+    def _get_offset(self, track, sector):
+        """
+        Return offset (in bytes) for specified track and sector.
+
+        Track   Sectors/track   # Tracks
+        -----   -------------   ---------
+         1-17        21            17
+        18-24        19             7
+        25-30        18             6
+        31-40        17            10
+        """
+        offset = 0
+        truncate_track = 0
+
+        if track > 17:
+            offset = 17 * 21 * SECLEN
+            truncate_track = 17
+
+        if track > 24:
+            offset += 7 * 19 * SECLEN
+            truncate_track = 24
+
+        if track > 30:
+            offset += 6 * 18 * SECLEN
+            truncate_track = 30
+
+        track = track - truncate_track
+        offset += track * sector * SECLEN
+
+        return offset
+
+
+class D71(Disk):
+    """
+    Implement d71 directory reader
+    """
+
+    def _get_offset(self, track, sector):
+        """
+        Return offset (in bytes) for specified track and sector.
+
+            Track       Sec/trk   # Tracks
+        --------------  -------   ---------
+         1-17 (side 0)    21         17
+        18-24 (side 0)    19          7
+        25-30 (side 0)    18          6
+        31-35 (side 0)    17          5
+        36-52 (side 1)    21         17
+        53-59 (side 1)    19          7
+        60-65 (side 1)    18          6
+        66-70 (side 1)    17          5
+        """
+        offset = 0
+        truncate_track = 0
+
+        if track > 17:
+            offset = 17 * 21 * SECLEN
+            truncate_track = 17
+
+        if track > 24:
+            offset += 7 * 19 * SECLEN
+            truncate_track = 24
+
+        if track > 30:
+            offset += 6 * 18 * SECLEN
+            truncate_track = 30
+
+        if track > 35:
+            offset += 5 * 17 * SECLEN
+            truncate_track = 35
+
+        if track > 52:
+            offset = 17 * 21 * SECLEN
+            truncate_track = 17
+
+        if track > 59:
+            offset += 7 * 19 * SECLEN
+            truncate_track = 24
+
+        if track > 65:
+            offset += 6 * 18 * SECLEN
+            truncate_track = 30
+
+        track = track - truncate_track
+        offset += track * sector * SECLEN
+
+        return offset
+
+
+class D81(Disk):
+    """
+    Implement d81 directory reader
+    """
+    DIR_TRACK = 40
+    DIR_SECTOR = 3
+    FILE_TYPES = {0b000: 'del',
+                  0b001: 'seq',
+                  0b010: 'prg',
+                  0b011: 'usr',
+                  0b100: 'rel',
+                  0b101: 'cbm'}
+
+    def _get_offset(self, track, sector):
+        """
+        Return offset (in bytes) for specified track and sector. In d81 is
+        easy, since we have 80 tracks with 40 sectors for 256 bytes each.
+        """
+        # we wan to go to the beginning (first sector) of the track, not it's
+        # max, so that we need to extract its amount.
+        return (track * 40 - 40) * SECLEN + sector * SECLEN
+
+
 class Uc1541(object):
     """
     Class for interact with c1541 program and MC
     """
-    PRG = re.compile(r'(\d+)\s+"([^"]*)".+?\s(del|prg|rel|seq|usr)([\s<])')
+    PRG = re.compile(r'(\d+)\s+"([^"]*)".+?(del|prg|rel|seq|usr)([\s<])')
 
     def __init__(self, archname):
         self.arch = archname
@@ -262,7 +366,7 @@ class Uc1541(object):
         self._verbose = os.getenv("UC1541_VERBOSE", False)
         self._hide_del = os.getenv("UC1541_HIDE_DEL", False)
 
-        self.pyd64 = D64(archname).list_dir()
+        self.dirlist = _get_implementation(_get_raw(archname)).list_dir()
         self.file_map = {}
         self.directory = []
 
@@ -291,19 +395,7 @@ class Uc1541(object):
         """
         LOG.info("Removing file %s", dst)
         dst = self._get_masked_fname(dst)
-
+        return self._call_command('delete', dst=dst)
-        if not self._call_command('delete', dst=dst):
-            return self._show_error()
-
-        # During removing, a message containing ERRORCODE is sent to stdout
-        # instead of stderr. Everything other than 'ERRORCODE 1' (which means:
-        # 'everything fine') is actually a failure. In case of verbose error
-        # output it is needed to copy self.out to self.err.
-        if '\nERRORCODE 1\n' not in self.out:
-            self.err = self.out
-            return self._show_error()
-
-        return 0
 
     def copyin(self, dst, src):
         """
@@ -311,11 +403,7 @@ class Uc1541(object):
         """
         LOG.info("Copy into D64 %s as %s", src, dst)
         dst = self._correct_fname(dst)
-
+        return self._call_command('write', src=src, dst=dst)
-        if not self._call_command('write', src=src, dst=dst):
-            return self._show_error()
-
-        return 0
 
     def copyout(self, src, dst):
         """
@@ -328,10 +416,7 @@ class Uc1541(object):
 
         src = self._get_masked_fname(src)
 
-        if not self._call_command('read', src=src, dst=dst):
+        return self._call_command('read', src=src, dst=dst)
-            return self._show_error()
-
-        return 0
 
     def mkdir(self, dirname):
         """Not supported"""
@@ -350,9 +435,9 @@ class Uc1541(object):
         program seem to have issues with it while writing, so it will also be
         replaced.
         """
+        # TODO: revisit those shitty mappings, when are which done.
         char_map = {'|': "/",
                     "\\": "/",
-                    "~": " ",
                     "$": "?",
                     "*": "?"}
 
@@ -392,8 +477,9 @@ class Uc1541(object):
         uid = os.getuid()
         gid = os.getgid()
 
-        if not self._call_command('list'):
+        res = self._call_command('list')
-            return self._show_error()
+        if res != 0:
+            return res
 
         idx = 0
         for line in self.out.split("\n"):
@@ -404,18 +490,14 @@ class Uc1541(object):
                     continue
 
                 display_name = ".".join([fname, ext])
-                pattern_name = self.pyd64[idx]['fname']
+                pattern_name = self.dirlist[idx]['fname']
 
                 if '/' in display_name:
                     display_name = display_name.replace('/', '|')
 
-                # workaround for space and dash at the beggining of the
+                # workaround for space at the beggining of the filename
-                # filename
+                if display_name.startswith(' '):
-                char_map = {' ': '~',
+                    display_name = '_' + display_name[1:]
-                            '-': '_'}
-                display_name = "".join([char_map.get(display_name[0],
-                                                     display_name[0]),
-                                        display_name[1:]])
 
                 if ext == 'del':
                     perms = "----------"
@@ -426,7 +508,7 @@ class Uc1541(object):
                                   'display_name': display_name,
                                   'uid': uid,
                                   'gid': gid,
-                                  'size': int(blocks) * 256,
+                                  'size': int(blocks) * SECLEN,
                                   'perms': perms})
                 idx += 1
         return directory
@@ -454,9 +536,28 @@ class Uc1541(object):
         if dst:
             command.append(dst)
 
-        self.out, self.err = Popen(command, stdout=PIPE,
+        LOG.debug('executing command: %s', ' '.join(command))
-                                   stderr=PIPE).communicate()
+        # For some reason using write and delete commands and reading output
-        return not self.err
+        # confuses Python3 beneath MC and as a consequence MC report an
+        # error...therefore for those commands let's not use
+        # universal_newlines...
+        universal_newlines = True
+        if cmd in ['delete', 'write']:
+            universal_newlines = False
+        #(self.out,
+        # self.err) = subprocess.Popen(command,
+        #                              universal_newlines=universal_newlines,
+        #                              stdout=subprocess.PIPE,
+        #                              stderr=subprocess.PIPE).communicate()
+        out = subprocess.run(command, capture_output=True,
+                             universal_newlines=universal_newlines)
+
+        self.out, self.err = out.stdout, out.stderr
+
+        if self.err:
+            LOG.debug('an err: %s', self.err)
+            return self._show_error()
+        return 0
 
 
 CALL_MAP = {'list': lambda a: Uc1541(a.arch).list(),
@@ -469,8 +570,10 @@ CALL_MAP = {'list': lambda a: Uc1541(a.arch).list(),
 
 def parse_args():
     """Use ArgumentParser to check for script arguments and execute."""
-    parser = ArgumentParser()
+    parser = argparse.ArgumentParser()
-    subparsers = parser.add_subparsers(help='supported commands')
+    subparsers = parser.add_subparsers(help='supported commands',
+                                       dest='subcommand')
+    subparsers.required = True
     parser_list = subparsers.add_parser('list', help="List contents of D64 "
                                         "image")
     parser_copyin = subparsers.add_parser('copyin', help="Copy file into D64 "
@@ -514,43 +617,6 @@ def parse_args():
     return args.func(args)
 
 
-def no_parse():
-    """Failsafe argument "parsing". Note, that it blindly takes positional
-    arguments without checking them. In case of wrong arguments it will
-    silently exit"""
-    try:
-        if sys.argv[1] not in ('list', 'copyin', 'copyout', 'rm', 'mkdir',
-                               "run"):
-            sys.exit(2)
-    except IndexError:
-        sys.exit(2)
-
-    class Arg(object):
-        """Mimic argparse object"""
-        dst = None
-        src = None
-        arch = None
-
-    arg = Arg()
-
-    try:
-        arg.arch = sys.argv[2]
-        if sys.argv[1] in ('copyin', 'copyout'):
-            arg.src = sys.argv[3]
-            arg.dst = sys.argv[4]
-        elif sys.argv[1] in ('rm', 'run', 'mkdir'):
-            arg.dst = sys.argv[3]
-    except IndexError:
-        sys.exit(2)
-
-    return CALL_MAP[sys.argv[1]](arg)
-
 if __name__ == "__main__":
     LOG.debug("Script params: %s", str(sys.argv))
-    try:
+    sys.exit(parse_args())
-        from argparse import ArgumentParser
-        PARSE_FUNC = parse_args
-    except ImportError:
-        PARSE_FUNC = no_parse
-
-    sys.exit(PARSE_FUNC())