opsimsummary.opsim_out module

This module deals with representing the data in an OpSim output (to the extent we will care about it). A description of the OpSim output can be found at (opsim description)[https://www.lsst.org/scientists/simulations/opsim/summary-table-column-descriptions-v335]

In brief, we will use two tables from the OpSim output:

• A Summary Table which has the desired information

• A Proposal Table which contains a dictionary to interpreting the propID column

  of Summary.

class opsimsummary.opsim_out.OpSimOutput(summary, propIDDict=None, proposalTable=None, subset=None, propIDs=None, zeroDDFDithers=True, opsimversion='lsstv3')

Bases: object

Class representing a subset of the output of the OpSim including information from the Summary and Proposal Tables with the subset taken over the proposals

Attributes:

opsimversion: {‘lsstv3’|’sstf’|’lsstv4’}

version of OpSim corresponding to the output format.

summary: `pd.DataFrame`

selected records from the Summary Table of pointings

propIDDict: dict

dictionary with strings as keys and integers used in the Summary Table to denote these proposals

proposalTable: `pd.DataFrame`

the propsal table in the output

subset: string

subset of proposals included in this class

propIDs : list of integers

integers corresponding to the subset selected through proposals

zeroDDFDithers : bool, defaults to True

if True, set dithers in DDF to 0, by setting ditheredRA, ditheredDec to fieldRA, fieldDec. This should only be used for opsimversion=’lsstv3’. For opsimversion=’sstf’ or ‘lsstv4’, this will be set to False despite inputs, since this is already done, and cannot be done with the inputs.

Methods

dropDuplicates(df, propIDDict, opsimversion)	drop duplicates ensuring keeping identity of ddf visits
fromOpSimDB(dbname[, subset, opsimversion, …])	Convenience method to instantitate the OpSimOutput class directly from an OpSim output rather than providing the elementary inputs in the class constructor.
get_allowed_subsets()	Provide a sequence of implemented subset values
get_dithercolumns(summary, opsimversion[, …])	Use a method prescription to obtain dithered values of pointings starting from a fixed pointing.
get_opsimVariablesForVersion([opsimversion])	Static method to returns a dictionary for the opsim version where the keys are names of quantities used in this codebase, and the values are the names of quantities in the OpSim output database
get_propIDDict(proposalDF[, opsimversion])	Return a dictionary with keys ‘ddf’, ad ‘wfd’ with the proposal IDs corresponding to deep drilling fields (ddf) and universal cadence (wfd)
propIDVals(subset, propIDDict, proposalTable)	Parameters:
validate_pointings(summary[, opsimVars, …])	Validate a dataframe of pointings for further use.

static dropDuplicates(df, propIDDict, opsimversion)

drop duplicates ensuring keeping identity of ddf visits

Parameters:

df : pd.DataFrame

propIDDict : dict

Returns:

`pd.DataFrame` with the correct propID and duplicates dropped

classmethod fromOpSimDB(dbname, subset='combined', opsimversion='lsstv3', zeroDDFDithers=True, user_propIDs=None, dithercolumns=None, add_dithers=False, tableNames=('Summary', 'Proposal'), filterNull=False, **kwargs)

Convenience method to instantitate the OpSimOutput class directly from an OpSim output rather than providing the elementary inputs in the class constructor.

Parameters:

dbname : string

absolute path to database

subset : string, optional, defaults to ‘combined’

one of {‘_all’, ‘unique_all’, ‘wfd’, ‘ddf’, ‘combined’} determines a sequence of propIDs for selecting observations appropriate for the OpSim database in use

opsimversion : {‘lsstv3’|’sstf’|’lsstv4’}

version of OpSim corresponding to the output format.

zeroDDFDithers : bool, defaults to True

if True, set dithers in DDF to 0, by setting ditheredRA, ditheredDec to fieldRA, fieldDec

dithercolumns: `pd.DataFrame`, defaults to `None`

a pandas dataframe with the columns ditheredRA, ditheredDec and index obsHistID, when not None this is used to create opsimVars[pointingRA] and opsimVars[pointingDec] deleting the these columns if they existed.

add_dithers : Bool, defaults to False

if True add dithers by generate ourselves by invoking cls.get_dithers and options through **kwargs. Even if False, becomes True if opsimVars[‘pointingRA’] is not in the list of `summary[columns] so that it needs to be created, and dithercolumns is None.

user_propIDs : sequence of integers, defaults to None

proposal ID values. If not None, overrides the use of subset

tableNames : tuple of strings, defaults to (‘Summary’, ‘Proposal’)

names of tables read from the OpSim database

filterNull : Bool, defaults to False

if True, the summary table should be filtered to rows that do not contain NULL values in the fiveSigmaDepth column.

kwargs: dict

of options relating to changing the methods of adding dithers. keywords are rng of type np.random.RandomState, ddf_ditherscale, wfd_ditherscale, method. If not provided, the parameters take default values.

static get_allowed_subsets()

Provide a sequence of implemented subset values

static get_dithercolumns(summary, opsimversion, method='default', ddfId=5, rng=<mtrand.RandomState object>, wfd_ditherscale=1.75, ddf_ditherscale=0.2)

Use a method prescription to obtain dithered values of pointings starting from a fixed pointing.

Parameters:

summary : pd.DataFrame

indexed by obsHistID and having the columns fieldRA, fieldDec

opsimversion : string, defaults to lsstv3

version of the OpSim producing the database.

method : string

{‘default|FlatSky’} only implemented

rng : randomState

kwargs :

static get_opsimVariablesForVersion(opsimversion='lsstv3')

Static method to returns a dictionary for the opsim version where the keys are names of quantities used in this codebase, and the values are the names of quantities in the OpSim output database

Parameters:

opsimversion: string, defaults to `lsstv3`

can be {lsstv3`|`lsstv4`|`sstf}

Returns:

dictionary: key, value pairs where keys are variable names used in `OpSimSummary`

and values are variable names used in the OpSim database with the given version.

Examples

>>> from opsimsummary import OpSimOutput
>>> OpSimOutput.get_opsimVariablesForVersion('lsstv4')
{'summaryTableName': 'SummaryAllProps', 'obsHistID': 'observationId',
 'propName': 'propName', 'propIDName': 'propId',
 'propIDNameInSummary': 'proposalId', 'ops_wfdname': 'WideFastDeep',
 'ops_ddfname': 'DeepDrillingCosmology1',
 'expMJD': 'observationStartMJD', 'FWHMeff': 'seeingFwhmEff',
 'pointingRA': 'ditheredRA', 'pointingDec': 'ditheredDec',
 'filtSkyBrightness': 'skyBrightness', 'angleUnit': 'degrees'}

static get_propIDDict(proposalDF, opsimversion='lsstv3')

Return a dictionary with keys ‘ddf’, ad ‘wfd’ with the proposal IDs corresponding to deep drilling fields (ddf) and universal cadence (wfd)

Parameters:

proposalDF : pd.DataFrame, mandatory

a dataframe with the Proposal Table of the OpSim Run.

opsimversion: {‘lsstv3’|’sstf’|’lsstv4’}, defaults to ‘lsstv3’

version of opsim from which output is drawn

Returns

——-

dictionary with keys ‘wfd’ and ‘ddf’ with values given by integers

corresponding to propIDs for these proposals

opsimVars

Dictionary where the keys are names of quantities used in OpSimSummary, and the values are the names of quantities in the OpSim output database used.

static propIDVals(subset, propIDDict, proposalTable)

subset : string

must be member of OpSimOutput.allowed_subsets()

propIDDict : dictionary, mandatory

must have subset as a key, and an integer or seq of ints as values

proposalTable : pd.DataFrame

Dataframe representing the proposal table in the OpSim datbase output

list of propID values (integers) associated with the subset

propIds

list of values in propID Column of the Summary Table of OpSim to be considered for this class, either because they were directly provided or through the subset argument.

static validate_pointings(summary, opsimVars=None, check_anycols=False)

Validate a dataframe of pointings for further use. If opsimVars is None then only check that there are no no.nan`s, else check that the table of pointings has the necessary format and units by checking that required columns indicated by `opsimVars exist and have sensible values.

Parameters:

summary: `pd.DataFrame` of pointings

opsimVars: dictionary, defaults to `None`

should be dictionary for each supported OpSim version availble from OpSimOutput.get_opsimVariablesForVersion(opsimversion)

check_anycols: Bool, defaults to False

if True, this will check all columns rather than fiveSigmaDepth for nans

Returns:

Bool (True|False) But exits on False.