### Navigation

- index
- modules |
- next |
- previous |
- PyFFI 2.2.2 documentation »
- `pyffi` — Interfacing block structured files »

# `pyffi.spells` — High level file operations

Note

This module is based on wz’s NifTester module, although nothing of wz’s original code is left in this module.

A toaster, implemented by subclasses of the abstract `Toaster` class, walks over all files in a folder, and applies one or more transformations on each file. Such transformations are called spells, and are implemented by subclasses of the abstract `Spell` class.

A spell can also run independently of a toaster and be applied on a branch directly. The recommended way of doing this is via the `Spell.recurse()` method.

## Supported spells

For format specific spells, refer to the corresponding module.

- `pyffi.spells.cgf` — Crytek Geometry/Animation (.cgf/.cga) spells
- `pyffi.spells.kfm` — NetImmerse/Gamebryo Keyframe Motion (.kfm) spells
- `pyffi.spells.nif` — NetImmerse/Gamebryo File/Keyframe (.nif/.kf/.kfa) spells
  - `pyffi.spells.nif.fix` — spells to fix errors
  - `pyffi.spells.nif.modify` — spells to make modifications

Some spells are applicable on every file format, and those are documented here.

class `pyffi.spells.``SpellApplyPatch`(toaster=None, data=None, stream=None)

  Bases: `pyffi.spells.Spell`

  A spell for applying a patch on files.

  `datainspect`()

    There is no need to read the whole file, so we apply the patch already at inspection stage, and stop the spell process by returning `False`.

    Returns: | `False`
    Return type: | `bool`

## Adding new spells

To create new spells, derive your custom spells from the `Spell` class, and include them in the `Toaster.SPELLS` attribute of your toaster.

class `pyffi.spells.``Spell`(toaster=None, data=None, stream=None)

  Bases: `builtins.object`

  Spell base class. A spell takes a data file and then does something useful with it. The main entry point for spells is `recurse()`, so if you are writing new spells, start with reading the documentation with `recurse()`.

  `READONLY` = True

    A `bool` which determines whether the spell is read only or not. Default value is `True`. Override this class attribute, and set to `False`, when subclassing a spell that must write files back to the disk.

  `SPELLNAME` = None

    A `str` describing how to refer to the spell from the command line. Override this class attribute when subclassing.

  `data` = None

    The `Data` instance this spell acts on.

  `stream` = None

    The current `file` being processed.

  `toaster` = None

    The `Toaster` instance this spell is called from.

  `__init__`(toaster=None, data=None, stream=None)

    Initialize the spell data.

    Parameters: | data (`Data`) – The file `data`. stream (`file`) – The file `stream`. toaster (`Toaster`) – The `toaster` this spell is called from (optional).

  `recurse`(branch=None)

    Helper function which calls `_branchinspect()` and `branchinspect()` on the branch, if both successful then `branchentry()` on the branch, and if this is succesful it calls `recurse()` on the branch’s children, and once all children are done, it calls `branchexit()`.

    Note that `_branchinspect()` and `branchinspect()` are not called upon first entry of this function, that is, when called with `data` as branch argument. Use `datainspect()` to stop recursion into this branch.

    Do not override this function.

    Parameters: | branch (`GlobalNode`) – The branch to start the recursion from, or `None` to recurse the whole tree.

  `_datainspect`()

    This is called after `pyffi.object_models.FileFormat.Data.inspect()` has been called, and before `pyffi.object_models.FileFormat.Data.read()` is called.

    Returns: | `True` if the file must be processed, `False` otherwise.
    Return type: | `bool`

  `datainspect`()

    This is called after `pyffi.object_models.FileFormat.Data.inspect()` has been called, and before `pyffi.object_models.FileFormat.Data.read()` is called. Override this function for customization.

    Returns: | `True` if the file must be processed, `False` otherwise.
    Return type: | `bool`

  `_branchinspect`(branch)

    Check if spell should be cast on this branch or not, based on exclude and include options passed on the command line. You should not need to override this function: if you need additional checks on whether a branch must be parsed or not, override the `branchinspect()` method.

    Parameters: | branch (`GlobalNode`) – The branch to check.
    Returns: | `True` if the branch must be processed, `False` otherwise.
    Return type: | `bool`

  `branchinspect`(branch)

    Like `_branchinspect()`, but for customization: can be overridden to perform an extra inspection (the default implementation always returns `True`).

    Parameters: | branch (`GlobalNode`) – The branch to check.
    Returns: | `True` if the branch must be processed, `False` otherwise.
    Return type: | `bool`

  `dataentry`()

    Called before all blocks are recursed. The default implementation simply returns `True`. You can access the data via `data`, and unlike in the `datainspect()` method, the full file has been processed at this stage.

    Typically, you will override this function to perform a global operation on the file data.

    Returns: | `True` if the children must be processed, `False` otherwise.
    Return type: | `bool`

  `dataexit`()

    Called after all blocks have been processed, if `dataentry()` returned `True`.

    Typically, you will override this function to perform a final spell operation, such as writing back the file in a special way, or making a summary log.

  `branchentry`(branch)

    Cast the spell on the given branch. First called with branch equal to `data`‘s children, then the grandchildren, and so on. The default implementation simply returns `True`.

    Typically, you will override this function to perform an operation on a particular block type and/or to stop recursion at particular block types.

    Parameters: | branch (`GlobalNode`) – The branch to cast the spell on.
    Returns: | `True` if the children must be processed, `False` otherwise.
    Return type: | `bool`

  `branchexit`(branch)

    Cast a spell on the given branch, after all its children, grandchildren, have been processed, if `branchentry()` returned `True` on the given branch.

    Typically, you will override this function to perform a particular operation on a block type, but you rely on the fact that the children must have been processed first.

    Parameters: | branch (`GlobalNode`) – The branch to cast the spell on.

  classmethod `toastentry`(toaster)

    Called just before the toaster starts processing all files. If it returns `False`, then the spell is not used. The default implementation simply returns `True`.

    For example, if the spell only acts on a particular block type, but that block type is excluded, then you can use this function to flag that this spell can be skipped. You can also use this function to initialize statistics data to be aggregated from files, to initialize a log file, and so.

    Parameters: | toaster (`Toaster`) – The toaster this spell is called from.
    Returns: | `True` if the spell applies, `False` otherwise.
    Return type: | `bool`

  classmethod `toastexit`(toaster)

    Called when the toaster has finished processing all files.

    Parameters: | toaster (`Toaster`) – The toaster this spell is called from.

## Grouping spells together

It is also possible to create composite spells, that is, spells that simply execute other spells. The following functions and classes can be used for this purpose.

`pyffi.spells.``SpellGroupParallel`(*args)

  Class factory for grouping spells in parallel.

`pyffi.spells.``SpellGroupSeries`(*args)

  Class factory for grouping spells in series.

class `pyffi.spells.``SpellGroupBase`(toaster=None, data=None, stream=None)

  Bases: `pyffi.spells.Spell`

  Base class for grouping spells. This implements all the spell grouping functions that fall outside of the actual recursing (`__init__()`, `toastentry()`, `_datainspect()`, `datainspect()`, and `toastexit()`).

  `ACTIVESPELLCLASSES` = []

    List of active spells of this group (not instantiated). This list is automatically built when `toastentry()` is called.

  `SPELLCLASSES` = []

    List of `Spell`s of this group (not instantiated).

  `datainspect`()

    Inspect every spell with L{Spell.datainspect} and keep those spells that must be cast.

  `spells` = []

    List of active spell instances.

  classmethod `toastentry`(toaster)
  classmethod `toastexit`(toaster)

class `pyffi.spells.``SpellGroupParallelBase`(toaster=None, data=None, stream=None)

  Bases: `pyffi.spells.SpellGroupBase`

  Base class for running spells in parallel (that is, with only a single recursion in the tree).

  `branchentry`(branch)

    Run all spells.

  `branchexit`(branch)
  `branchinspect`(branch)

    Inspect spells with `Spell.branchinspect()` (not all checks are executed, only keeps going until a spell inspection returns `True`).

  `changed`
  `dataentry`()

    Look into every spell with `Spell.dataentry()`.

  `dataexit`()

    Look into every spell with `Spell.dataexit()`.

class `pyffi.spells.``SpellGroupSeriesBase`(toaster=None, data=None, stream=None)

  Bases: `pyffi.spells.SpellGroupBase`

  Base class for running spells in series.

  `branchentry`(branch)
  `branchinspect`(branch)
  `changed`
  `dataentry`()
  `dataexit`()
  `recurse`(branch=None)

    Recurse spells in series.

## Creating toaster scripts

To create a new toaster script, derive your toaster from the `Toaster` class, and set the `Toaster.FILEFORMAT` attribute of your toaster to the file format class of the files it can toast.

class `pyffi.spells.``Toaster`(spellclass=None, options=None, spellnames=None, logger=None)

  Bases: `builtins.object`

  Toaster base class. Toasters run spells on large quantities of files. They load each file and pass the data structure to any number of spells.

  `ALIASDICT` = {}

    Dictionary with aliases for spells.

  `DEFAULT_OPTIONS` = {'patchcmd': '', 'dryrun': False, 'sourcedir': '', 'resume': False, 'skip': [], 'inifile': '', 'helpspell': False, 'destdir': '', 'prefix': '', 'examples': False, 'arg': '', 'jobs': 8, 'exclude': [], 'createpatch': False, 'suffix': '', 'pause': False, 'verbose': 1, 'gccollect': False, 'raisetesterror': False, 'applypatch': False, 'refresh': 32, 'diffcmd': '', 'include': [], 'only': [], 'archives': False, 'series': False, 'spells': False, 'interactive': True}

    List of spell classes of the particular `Toaster` instance.

  `EXAMPLES` = ''

    Some examples which describe typical use of the toaster.

  `FILEFORMAT`

    The file format class (a subclass of `FileFormat`).

    alias of `FileFormat`

  `SPELLS` = []

    List of all available `Spell` classes.

  `cli`()

    Command line interface: initializes spell classes and options from the command line, and run the `toast()` method.

  `exclude_types` = []

    Tuple of types corresponding to the exclude key of `options`.

  `get_toast_head_root_ext`(filename)

    Get the name of where the input file filename would be written to by the toaster: head, root, and extension.

    Parameters: | filename (`str`) – The name of the hypothetical file to be toasted.
    Returns: | The head, root, and extension of the destination, or `(None, None, None)` if `--dry-run` is specified.
    Return type: | `tuple` of three `str`s

  `get_toast_stream`(filename, test_exists=False)

    Calls `get_toast_head_root_ext(filename)()` to determine the name of the toast file, and return a stream for writing accordingly.

    Then return a stream where result can be written to, or in case test_exists is True, test if result would overwrite a file. More specifically, if test_exists is True, then no streams are created, and True is returned if the file already exists, and False is returned otherwise.

  `include_types` = []

    Tuple of types corresponding to the include key of `options`.

  `indent` = 0

    An `int` which describes the current indentation level for messages.

  `inspect_filename`(filename)

    Returns whether to toast a filename or not, based on skip_regexs and only_regexs.

  `is_admissible_branch_class`(branchtype)

    Helper function which checks whether a given branch type should have spells cast on it or not, based in exclude and include options.

        >>> from pyffi.formats.nif import NifFormat
        >>> class MyToaster(Toaster):
        ...     FILEFORMAT = NifFormat
        >>> toaster = MyToaster() # no include or exclude: all admissible
        >>> toaster.is_admissible_branch_class(NifFormat.NiProperty)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiNode)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiAVObject)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiLODNode)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiMaterialProperty)
        True
        >>> toaster = MyToaster(options={"exclude": ["NiProperty", "NiNode"]})
        >>> toaster.is_admissible_branch_class(NifFormat.NiProperty)
        False
        >>> toaster.is_admissible_branch_class(NifFormat.NiNode)
        False
        >>> toaster.is_admissible_branch_class(NifFormat.NiAVObject)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiLODNode)
        False
        >>> toaster.is_admissible_branch_class(NifFormat.NiMaterialProperty)
        False
        >>> toaster = MyToaster(options={"include": ["NiProperty", "NiNode"]})
        >>> toaster.is_admissible_branch_class(NifFormat.NiProperty)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiNode)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiAVObject)
        False
        >>> toaster.is_admissible_branch_class(NifFormat.NiLODNode) # NiNodes are!
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiMaterialProperty) # NiProperties are!
        True
        >>> toaster = MyToaster(options={"include": ["NiProperty", "NiNode"], "exclude": ["NiMaterialProperty", "NiLODNode"]})
        >>> toaster.is_admissible_branch_class(NifFormat.NiProperty)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiNode)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiAVObject)
        False
        >>> toaster.is_admissible_branch_class(NifFormat.NiLODNode)
        False
        >>> toaster.is_admissible_branch_class(NifFormat.NiSwitchNode)
        True
        >>> toaster.is_admissible_branch_class(NifFormat.NiMaterialProperty)
        False
        >>> toaster.is_admissible_branch_class(NifFormat.NiAlphaProperty)
        True

  `logger` = <logging.Logger object at 0xb13b090>

    A `logging.Logger` for toaster log messages.

  `msg`(message)

    Write log message with `logger.info()`, taking into account `indent`.

    Parameters: | message (`str`) – The message to write.

  `msgblockbegin`(message)

    Acts like `msg()`, but also increases `indent` after writing the message.

  `msgblockend`(message=None)

    Acts like `msg()`, but also decreases `indent` before writing the message, but if the message argument is `None`, then no message is printed.

  `only_regexs` = []

    Tuple of regular expressions corresponding to the only key of `options`.

  `options` = {}

    The options of the toaster, as `dict`.

  static `parse_inifile`(option, opt, value, parser, toaster=None)

    Initializes spell classes and options from an ini file.

        >>> import pyffi.spells.nif
        >>> import pyffi.spells.nif.modify
        >>> class NifToaster(pyffi.spells.nif.NifToaster):
        ...     SPELLS = [pyffi.spells.nif.modify.SpellDelBranches]
        >>> import tempfile
        >>> cfg = tempfile.NamedTemporaryFile(delete=False)
        >>> _ = cfg.write(b"[main]\n")
        >>> _ = cfg.write(b"spell = modify_delbranches\n")
        >>> _ = cfg.write(b"folder = tests/nif/test_vertexcolor.nif\n")
        >>> _ = cfg.write(b"[options]\n")
        >>> _ = cfg.write(b"source-dir = tests/\n")
        >>> _ = cfg.write(b"dest-dir = _tests/\n")
        >>> _ = cfg.write(b"exclude = NiVertexColorProperty NiStencilProperty\n")
        >>> _ = cfg.write(b"skip = 'testing quoted string'    normal_string\n")
        >>> cfg.close()
        >>> toaster = NifToaster(logger=fake_logger)
        >>> import sys
        >>> sys.argv = [
        ...     "niftoaster.py",
        ...     "--ini-file=utilities/toaster/default.ini",
        ...     "--ini-file=%s" % cfg.name,
        ...     "--noninteractive", "--jobs=1"]
        >>> toaster.cli()
        pyffi.toaster:INFO:=== tests/nif/test_vertexcolor.nif ===
        pyffi.toaster:INFO:  --- modify_delbranches ---
        pyffi.toaster:INFO:    ~~~ NiNode [Scene Root] ~~~
        pyffi.toaster:INFO:      ~~~ NiTriStrips [Cube] ~~~
        pyffi.toaster:INFO:        ~~~ NiStencilProperty [] ~~~
        pyffi.toaster:INFO:          stripping this branch
        pyffi.toaster:INFO:        ~~~ NiSpecularProperty [] ~~~
        pyffi.toaster:INFO:        ~~~ NiMaterialProperty [Material] ~~~
        pyffi.toaster:INFO:        ~~~ NiVertexColorProperty [] ~~~
        pyffi.toaster:INFO:          stripping this branch
        pyffi.toaster:INFO:        ~~~ NiTriStripsData [] ~~~
        pyffi.toaster:INFO:creating destination path _tests/nif
        pyffi.toaster:INFO:  writing _tests/nif/test_vertexcolor.nif
        pyffi.toaster:INFO:Finished.
        >>> import os
        >>> os.remove(cfg.name)
        >>> os.remove("_tests/nif/test_vertexcolor.nif")
        >>> os.rmdir("_tests/nif/")
        >>> os.rmdir("_tests/")
        >>> for name, value in sorted(toaster.options.items()):
        ...     print("%s: %s" % (name, value))
        applypatch: False
        archives: False
        arg:
        createpatch: False
        destdir: _tests/
        diffcmd:
        dryrun: False
        examples: False
        exclude: ['NiVertexColorProperty', 'NiStencilProperty']
        gccollect: False
        helpspell: False
        include: []
        inifile:
        interactive: False
        jobs: 1
        only: []
        patchcmd:
        pause: True
        prefix:
        raisetesterror: False
        refresh: 32
        resume: True
        series: False
        skip: ['testing quoted string', 'normal_string']
        sourcedir: tests/
        spells: False
        suffix:
        verbose: 1

  `skip_regexs` = []

    Tuple of regular expressions corresponding to the skip key of `options`.

  `spellnames` = []

    A list of the names of the spells.

  `toast`(top)

    Walk over all files in a directory tree and cast spells on every file.

    Parameters: | top (str) – The directory or file to toast.

  `toast_archives`(top)

    Toast all files in all archives.

  `top` = ''

    Name of the top folder to toast.

  `write`(stream, data)

    Writes the data to data and raises an exception if the write fails, but restores file if fails on overwrite.

  `writepatch`(stream, data)

    Creates a binary patch for the updated file.

### Navigation

- index
- modules |
- next |
- previous |
- PyFFI 2.2.2 documentation »
- `pyffi` — Interfacing block structured files »