# dbprocessing

# dbprocessing.dbprocessing.ProcessQueue

class dbprocessing.dbprocessing.ProcessQueue(mission, dryrun=False, echo=False)

  Main code used to process the Queue.

  Looks in incoming and builds all possible files

  Warning

  As this object holds a reference to `DButils`, that database should be closed before the program terminates. Deleting this object will ordinarily suffice.

  __init__(mission, dryrun=False, echo=False)

    Initializes the process queue

    Parameters
      mission str

        Mission database, as in `DButils`. May also be an existing instance of `DButils`.

      dryrun bool, default False

        Treat all functionality as “dry run”, make no changes.

      echo bool, default False

        Log all SQL statements (as in `DButils`).

  Methods

  `buildChildren`(file_id[, debug, skip_run, ...]) | go through and all the runMe's and add to the runme_list variable
  `checkIncoming`([glb]) | Goes out to incoming and grabs all files there adding them to self.queue
  `diskfileToDB`(df) | given a diskfile go through and do all the steps to add it into the db
  `figureProduct`([filename]) | Imports inspectors and figures out which inspectors claim the file
  `importFromIncoming`() | Import a file from incoming into the database
  `mk_tempdir`([suffix]) | create a secure temp directory
  `moveToError`(fname) | Moves a file from incoming to error
  `onStartup`() | Processes can be defined as output timebase "STARTUP" which means to run them each time to processing chain is run
  `reprocessByCode`(id_in[, startDate, endDate, ...]) | Add files made by a code to the queue for reprocessing.
  `reprocessByDate`([startDate, endDate, ...]) | Add files to the queue for reprocessing, by file date.
  `reprocessByInstrument`(id_in[, level, ...]) | Add files for an instrument to the queue for reprocessing
  `reprocessByProduct`(id_in[, startDate, ...]) | Add files of a particular product to the queue for reprocessing.
  `rm_tempdir`() | remove the temp directory
  `set_filename`(filename) | Setter for filename, this is cleaner than just random sets

  Attributes

  `filename`
  buildChildren(file_id, debug=False, skip_run=False, run_procs=None)

    go through and all the runMe’s and add to the runme_list variable

    Parameters
      file_id `int`

        file ID of the file for which children will be built

      skip_run `bool`, default False

        Skip RUN timebase processes if True

      run_procs `str`, optional

        If provided, comma-separated list of process IDs or process names to run; other processes are ignored. (Default: all possible processes).

  checkIncoming(glb='*')

    Goes out to incoming and grabs all files there adding them to self.queue

    Parameters
      glb `str`, optional

        Glob pattern that files must match.

  diskfileToDB(df)

    given a diskfile go through and do all the steps to add it into the db

    Parameters
      df `Diskfile`

        File to add to database.

  figureProduct(filename=None)

    Imports inspectors and figures out which inspectors claim the file

    Parameters
      filename `str`, optional

        Full path to file to check, default `filename`.

  importFromIncoming()

    Import a file from incoming into the database

  mk_tempdir(suffix='_dbprocessing_28973')

    create a secure temp directory

  moveToError(fname)

    Moves a file from incoming to error

    Parameters
      fname `str`

        Full path to file to move to error.

  onStartup()

    Processes can be defined as output timebase “STARTUP” which means to run them each time to processing chain is run

  reprocessByCode(id_in, startDate=None, endDate=None, incVersion=None)

    Add files made by a code to the queue for reprocessing.

    Parameters
      id_in `str` or `int`

        ID or filename of code to reprocess

      startDate `datetime`, optional

        First date to reprocess (default all)

      endDate `datetime`, optional

        Last date to reprocess (default all)

      incVersion `int` {0, 1, 2}, optional

        Which version number to increment: major (0), minor (1), subminor (2). Forces reprocessing. (default do not force).

  reprocessByDate(startDate=None, endDate=None, incVersion=None, level=None)

    Add files to the queue for reprocessing, by file date.

    Parameters
      startDate `datetime`, optional

        First date to reprocess (default all)

      endDate `datetime`, optional

        Last date to reprocess (default all)

      incVersion `int` {0, 1, 2}, optional

        Which version number to increment: major (0), minor (1), subminor (2). Forces reprocessing (default do not force).

      level `float`, optional

        Only reprocess files of this level (default all)

  reprocessByInstrument(id_in, level=None, startDate=None, endDate=None, incVersion=None)

    Add files for an instrument to the queue for reprocessing

    Parameters
      id_in `str` or `int`

        ID or name of instrument to reprocess

      level `int`, optional

        Only reprocess files of this level (default all)

      startDate `datetime`, optional

        First date to reprocess (default all)

      endDate `datetime`, optional

        Last date to reprocess (default all)

      incVersion `int` {0, 1, 2}, optional

        Which version number to increment: major (0), minor (1), subminor (2). Forces reprocessing (default do not force).

  reprocessByProduct(id_in, startDate=None, endDate=None, incVersion=None)

    Add files of a particular product to the queue for reprocessing.

    Parameters
      id_in `str` or `int`

        ID or name of code to reprocess

      startDate `datetime`, optional

        First date to reprocess (default all)

      endDate `datetime`, optional

        Last date to reprocess (default all)

      incVersion `int` {0, 1, 2}, optional

        Which version number to increment: major (0), minor (1), subminor (2). Forces reprocessing (default do not force).

  rm_tempdir()

    remove the temp directory

  set_filename(filename)

    Setter for filename, this is cleaner than just random sets

    Parameters
      filename `str`

        filename to set to `filename`

  filename

    Full path to file currently being processed (`str`)

Release: 0.1.0 Doc generation date: Feb 10, 2022