add option for prefix_chars in silcam and holo load classes

[emlynjdavies]

add option for prefix_chars in silcam and holo load classes. This allows load functions to have an optional number of characters to exclude in filename prefixes.

Example toml pipeline step with optional prefix_chars defined:

[steps.load]
pipeline_class = 'pyopia.instrument.silcam.Load'
prefix_chars = 5

prefix_chars defaults to 1 if not defined (to match the standard 'D' at the start of "normal" silcam and LISST-Holo filenames.

[nordam]

How about reviving my elegant and robust suggested regex from yonder, which will match the standard date and time format, irrespective of the number of characters in the prefix?

#291

[emlynjdavies]

How about reviving my elegant and robust suggested regex from yonder, which will match the standard date and time format, irrespective of the number of characters in the prefix?

#291

so, instead of parsing a single integer in the config settings, you suggest parsing a regex that describes the filename format?

maybe like this:

def timestamp_from_filename(filename, datetime_format='D%Y%m%dT%H%M%S.%f'):
    '''get a pandas timestamp from a silcam filename

    Parameters
    ----------
    filename : string
        silcam filename (.silc)

    datetime_format : string, optional
        Format string for parsing the timestamp from the filename,
        by default 'D%Y%m%dT%H%M%S.%f'

    Returns
    -------
    timestamp : pandas.Timestamp
        timestamp from pandas.to_datetime()
    '''

    # get the timestamp of the image (in this case from the filename)
    timestamp = pd.to_datetime(os.path.splitext(os.path.basename(filename))[0],
                               format=datetime_format)
    return timestamp

[emlynjdavies]

Since the to_datetime function cannot accept wildcards, this means that any prefix must have known and constant characters if we were to use the format string as the only input. The approach of using the prefix_chars integer allows for any content of those prefix characters (so they could be something changing within one dataset as long as their length is constant).

We can perhaps do both of these, by having optional inputs for prefix_chars and a datetime_format, where datetime format defaults to 'None' (which should keep the current default automatic parsing)