# dbprocessing

# dbprocessing.Utils

Various utilities of use throughout this code.

Functions

`chunker`(seq, size) | Return a long iterable in a tuple of shorter lists.
`dateForPrinting`([dt, microseconds, brackets]) | Return a string of the date format for printing on the screen
`daterange_to_dates`(daterange) | Given a daterange return the date objects for all days in the range
`datetimeToDate`(dt) | Given an input datetime.datetime or datetime.date return a datetime.date
`dirSubs`(path, filename, utc_file_date, ...) | Do any substitutions that are needed to put thing in the right place
`expandDates`(start_time, stop_time) | Given a start and a stop date make all the dates in between
`flatten`(l) | Flatten an irregularly nested list of lists
`parseDate`(inval) | Given a date of the for yyyy-mm-dd parse to a datetime.
`parseVersion`(inval) | Given a format of the form x.y.z parse to a Version
`processRunning`(pid) | Given a PID see if it is currently running.
`progressbar`(count, blocksize, totalsize[, text]) | Print a progress bar with urllib.urlretrieve reporthook functionality
`readconfig`(config_filepath) | Read a database definition config file.
`split_code_args`(args) | Split a string with a bunch of command line arguments into a list
`strargs_to_args`(strargs) | Read in the arguments string from the db and change to a dict
`toBool`(value) | Returns true if passed 'True', 'true', True, 1, 'Yes', 'yes', 'Y', or 'y'
`toDatetime`(dt[, end]) | Convert a date, date string, or datetime to a datetime
`toNone`(value) | Returns None if passed '', 'None', 'none', or 'NONE'
`unique`(seq) | Take a list and return only the unique elements in the same order

dbprocessing.Utils.chunker(seq, size)

  Return a long iterable in a tuple of shorter lists.

  Taken from https://stackoverflow.com/questions/434287/what-is-the-most-pythonic-way-to-iterate-over-a-list-in-chunks

  Parameters
    seq `Iterable`

      Iterable to split up

    size `int`

      Size of each split in the output, last one has the remaining elements of `seq`

  Returns
    `tuple`

      tuple of lists of the iterable `seq` split into `len(seq)/size` segments

dbprocessing.Utils.dateForPrinting(dt=None, microseconds=False, brackets='[]')

  Return a string of the date format for printing on the screen

  If dt is `None` return `now()`.

  Parameters
    dt `datetime`, default `now()`

      object to format

    microseconds `bool`, default False

      Include microseconds

    brackets `str`, optional

      Which brackets to encase the time in, default `('[', ']')`

  Returns
    str

      Iso formatted string

  Examples

      >>> from dbprocessing.Utils import dateForPrinting
      >>> print("{0} Something occurred".format(dateForPrinting()))
      [2016-03-22T10:51:45]  Something occurred

dbprocessing.Utils.daterange_to_dates(daterange)

  Given a daterange return the date objects for all days in the range

  Parameters
    daterange `Sequence` of `datetime`

      Start and stop dates

  Returns
    `list`

      All the dates between `daterange[0]` and `daterange[1]`

dbprocessing.Utils.datetimeToDate(dt)

  Given an input datetime.datetime or datetime.date return a datetime.date

  Parameters
    dt `datetime` or `date`

      input to convert

  Returns
    `date`

      Same date as in `dt`, but as a `date` object.

dbprocessing.Utils.dirSubs(path, filename, utc_file_date, utc_start_time, version, dbu=None)

  Do any substitutions that are needed to put thing in the right place

  Todo

  This may be useless/could be made more useful

  Honored substitutions used as {Y}{PRODUCT}{DATE}

  Parameters
    path str

      Path to the file

    filename str

      Name of the time

    utc_file_date `datetime`

      File’s date

    utc_start_time `datetime`

      File’s start time

    version `Version`

      Version to substitute

    dbu `DButils`, optional

      Current database connection. If not specified, creates a new connection.

  Returns
    `str`

  Notes

  Valid subsitutions are:

    - Y: 4 digit year

    - m: 2 digit month

    - b: 3 character month (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)

    - d: 2 digit day

    - y: 2 digit year

    - j: 3 digit day of year

    - H: 2 digit hour (24-hour time)

    - M: 2 digit minute

    - S: 2 digit second

    - VERSION: version string, interface.quality.revision

    - DATE: the UTC date from a file, same as Ymd

    - MISSION: the mission name from the db

    - SPACECRAFT: the spacecraft name from the db

    - PRODUCT: the product name from the db

dbprocessing.Utils.expandDates(start_time, stop_time)

  Given a start and a stop date make all the dates in between

  Inclusive on the ends

  Parameters
    start_time `datetime`

      Date to start the list

    stop_time `datetime`

      Date to end the list, inclusive

  Returns
    :class:`list` of class:~datetime.datetime

      All the dates between start_time and stop_time

dbprocessing.Utils.flatten(l)

  Flatten an irregularly nested list of lists

  Taken from https://stackoverflow.com/questions/2158395/flatten-an-irregular-list-of-lists

  Parameters
    l `list`

      Nested list of lists to flatten

  Returns
    `list`

      Flattened list

dbprocessing.Utils.parseDate(inval)

  Given a date of the for yyyy-mm-dd parse to a datetime.

  This is just a wrapper around `strptime()` If the format is wrong ValueError is raised.

  Parameters
    inval `str`

      String date representation of the form YYYY-MM-DD

  Returns
    `datetime`

      datetime object parsed from the string

dbprocessing.Utils.parseVersion(inval)

  Given a format of the form x.y.z parse to a Version

  This is a wrapper around `fromString()`.

  Parameters
    inval `str`

      String Version representation of the form xx.yy.zz

  Returns
    `Version`

      Version object parsed from the string

dbprocessing.Utils.processRunning(pid)

  Given a PID see if it is currently running.

  Taken from from https://stackoverflow.com/questions/568271/how-to-check-if-there-exists-a-process-with-a-given-pid-in-python

  Parameters
    pid `int`

      Process ID

  Returns
    `bool`

      True if `pid` is running, False otherwise

dbprocessing.Utils.progressbar(count, blocksize, totalsize, text='Download Progress')

  Print a progress bar with urllib.urlretrieve reporthook functionality

  Taken from spacepy

  Parameters
    count `float`

      The current count of the progressbar

    blocksize `float`

      The size of each block (mostly useful for file downloads)

    totalsize `float`

      The total size of the job, progress is `count*blocksize*100/totalsize`

    text `str`, optional

      The text to print in the progressbar

  Examples

      >>> import spacepy.toolbox as tb
      >>> import urllib
      >>> urllib.urlretrieve(config['psddata_url'], PSDdata_fname, reporthook=tb.progressbar)

dbprocessing.Utils.readconfig(config_filepath)

  Read a database definition config file.

  Parameters
    config_filepath `str`

      full path to the config file

  Returns
    `dict`

      Dictionary of key value pairs from config files

  See also

  addFromConfig.py
  addFromConfig.py

dbprocessing.Utils.split_code_args(args)

  Split a string with a bunch of command line arguments into a list

  As needed by Popen

  This is different than just split() since we have to keep options together with the flags.

  Parameters
    args `str`
  Returns
    :class:`list` of class:str

  See also

  `shlex`

  Examples

      >>> split_code_args("code -n hello outfile")
      [code, -n hello, outfile]

dbprocessing.Utils.strargs_to_args(strargs)

  Read in the arguments string from the db and change to a dict

  Parameters
    strargs `str`

      A string of arguments(“foo=bar baz=qux”)

  Returns
    `dict`

      A dictionary of the arguments are their values

dbprocessing.Utils.toBool(value)

  Returns true if passed ‘True’, ‘true’, True, 1, ‘Yes’, ‘yes’, ‘Y’, or ‘y’

  Parameters
    value

      Value to evaluate if true

  Returns
    `bool`

dbprocessing.Utils.toDatetime(dt, end=False)

  Convert a date, date string, or datetime to a datetime

  If a time is provided, passed through; otherwise set to start/end of day.

  Parameters
    dt `datetime`, `date`, or `str`

      input to convert

    end `bool`, default False

      If input has no time, set to end of day (default to start of day)

  Returns
    `datetime`

      Input converted to `datetime`

dbprocessing.Utils.toNone(value)

  Returns None if passed ‘’, ‘None’, ‘none’, or ‘NONE’

  Parameters
    value

      Value to evaluate if none

  Returns
    any

      `None` or same input value

dbprocessing.Utils.unique(seq)

  Take a list and return only the unique elements in the same order

  Parameters
    seq `list`

      List to return the unique elements of

  Returns
    `list`

      List with only the unique elements

Release: 0.1.0 Doc generation date: Feb 10, 2022