streamobs.surveys module

streamobs.surveys module#

class streamobs.surveys.Survey(name: str, release: str | None = None, bands: list = None, maglim_maps: dict = None, coeff_extinc: dict = None, saturation: dict = None, sys_error: dict = None, completeness: Callable | None = None, completeness_band: str | None = None, delta_saturation: float | None = None, log_photo_error_catalog: Callable | None = None, log_photo_error_sample: Callable | None = None, gal_misclassification: Callable | None = None, gal_misclassification_detection: Callable | None = None, ebv_map: ndarray | None = None, coverage: ndarray | None = None)[source]#

Bases: object

Container for survey properties and data.

This class stores all survey-specific data including HEALPix maps, efficiency functions, and photometric error models. Data is loaded once and reused to avoid redundant file I/O.

Band-specific properties (magnitude limits, extinction, errors, etc.) are stored in dictionaries keyed by band name (e.g., ‘g’, ‘r’, ‘i’).

name#

Survey identifier (e.g., ‘lsst’).

Type:: str

release#

Survey data release version (e.g., ‘yr1’, ‘yr5’).

Type:: str, optional

bands#

List of photometric bands available (e.g., [‘g’, ‘r’, ‘i’]). Default is an empty list.

Type:: list of str, optional

maglim_maps#

HEALPix maps of magnitude limits, keyed by band. Example: {‘g’: array(…), ‘r’: array(…)}. Default is an empty dictionary.

Type:: dict, optional

coeff_extinc#

Extinction coefficients A_band/E(B-V), keyed by band. Example: {‘g’: 3.303, ‘r’: 2.285}. Default uses LSST values for standard bands.

Type:: dict, optional

saturation#

Bright magnitude limits where detector saturates, keyed by band. Example: {‘g’: 16.0, ‘r’: 16.0}. Default is 16.0 mag for all bands.

Type:: dict, optional

sys_error#

Systematic photometric errors (mag), keyed by band. Example: {‘g’: 0.01, ‘r’: 0.01}. Default is 0.0 for all bands.

Type:: dict, optional

ebv_map#

HEALPix map of E(B-V) extinction values (band-independent).

Type:: np.ndarray, optional

coverage#

HEALPix map of survey coverage (1=observed, 0=not observed).

Type:: np.ndarray, optional

completeness#

Efficiency function f(magnitude) -> efficiency [0, 1]. Same function used for all bands, obtained from r band.

Type:: callable, optional

delta_saturation#

Magnitude difference for saturation threshold in the initial functions.

Type:: float, optional

completeness_band#

Band used to derive completeness function (e.g., ‘r’).

Type:: str, optional

log_photo_error_catalog#

Catalog (reported) photometric error model f(delta_mag) -> log10(mag_error), where delta_mag = mag - maglim. This is the survey’s reported error curve (e.g. photoerror_r.csv); it is written as magerr and drives the S/N cut. Loaded from the log_photo_error_catalog config key, or the legacy log_photo_error key. Always present for a configured survey.

Type:: callable, optional

log_photo_error_sample#

Sample photometric error model f(delta_mag) -> log10(mag_error): the true scatter of observed-minus-true magnitudes, used to draw the observed magnitudes. Optional second curve (config key log_photo_error_sample). If not provided, the noise draw falls back to the catalog model, which reproduces the previous single-curve behaviour exactly.

Type:: callable, optional

Examples

Load a survey:

>>> survey = Survey.load('lsst', release='dc2')
>>> print(survey.bands)
['g', 'r']

Access magnitude limit for g-band at pixel 10000:

>>> maglim_g = survey.get_maglim('g', pixel=10000)
>>> print(maglim_g)

Calculate extinction in r-band at pixel 10000:

>>> extinction_r = survey.get_extinction('r', pixel=10000)
>>> print(extinction_r)

Get detection efficiency for magnitude 24.0 in g-band:

>>> maglim_r = survey.get_maglim('r', pixel=10000)
>>> efficiency_g = survey.get_completeness('r', 24.0, maglim_r)
>>> print(efficiency_g)

Calculate photometric error for magnitude 24.0 in r-band:

>>> maglim_r = survey.get_maglim('r', pixel=10000)
>>> photo_error_r = survey.get_photo_error('r', 24.0, maglim_r)
>>> print(photo_error_r)

get_classification_efficiency(band: str, magnitude: float, maglim: float, **kwargs) → float[source]#

Get classification-only efficiency (star/galaxy separation).

This is a convenience wrapper around get_efficiency() with type="classification_efficiency".

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’).
magnitude (float or np.ndarray) – True apparent magnitude(s).
maglim (float or np.ndarray) – Magnitude limit(s) at the position(s).
**kwargs – Additional keyword arguments passed to get_efficiency().

Returns:

Classification efficiency in range [0, 1].

Return type:

float or np.ndarray

See also

get_efficiency: Full documentation of efficiency calculation.
get_completeness: Combined detection and classification efficiency.
get_detection_efficiency: Detection-only efficiency.

get_completeness(band: str, magnitude: float, maglim: float, **kwargs) → float[source]#

Get detection completeness (combined detection and classification efficiency).

This is a convenience wrapper around get_efficiency() with type="completeness".

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’).
magnitude (float or np.ndarray) – True apparent magnitude(s).
maglim (float or np.ndarray) – Magnitude limit(s) at the position(s).
**kwargs – Additional keyword arguments passed to get_efficiency().

Returns:

Detection efficiency in range [0, 1].

Return type:

float or np.ndarray

See also

get_efficiency: Full documentation of efficiency calculation.
get_detection_efficiency: Detection-only efficiency.
get_classification_efficiency: Classification-only efficiency.

get_detection_efficiency(band: str, magnitude: float, maglim: float, **kwargs) → float[source]#

Get detection-only efficiency (ignoring classification).

This is a convenience wrapper around get_efficiency() with type="detection_efficiency".

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’).
magnitude (float or np.ndarray) – True apparent magnitude(s).
maglim (float or np.ndarray) – Magnitude limit(s) at the position(s).
**kwargs – Additional keyword arguments passed to get_efficiency().

Returns:

Detection efficiency in range [0, 1].

Return type:

float or np.ndarray

See also

get_efficiency: Full documentation of efficiency calculation.
get_completeness: Combined detection and classification efficiency.
get_classification_efficiency: Classification-only efficiency.

get_efficiency(band: str, magnitude: float, maglim: float, type='completeness', **kwargs) → float[source]#

Get efficiency function values.

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’). Currently unused since same function is used for all bands.
magnitude (float or np.ndarray) – True apparent magnitude(s).
maglim (float or np.ndarray) – Magnitude limit(s) at the position(s).
type (str, optional) – Type of efficiency function to use. Options are "completeness", "detection_efficiency", or "classification_efficiency". Default is "completeness".
**kwargs –
Additional keyword arguments. Currently:

delta_saturationfloat, optional
Magnitude difference for saturation threshold in the initial completeness function. Default is -10.4.

Returns:

Detection efficiency in range [0, 1]. Value of 0 means no detection, 1 means certain detection.

Return type:

float or np.ndarray

Raises:

ValueError – If the efficiency function for the requested type is not loaded, or if type is not one of the recognized values.

Notes

The efficiency is set to 0 in the following cases:

Source is brighter than saturation limit (magnitude < saturation)
Survey does not cover the position (maglim < saturation or maglim is NaN)
Source is too faint for reliable detection

Examples

Get completeness for a single source:

>>> maglim = survey.get_maglim('r', pixel=10000)
>>> eff = survey.get_efficiency('r', 24.0, maglim)

Get detection efficiency for an array of magnitudes:

>>> mags = np.linspace(18, 28, 100)
>>> eff = survey.get_efficiency('r', mags, maglim, type="detection_efficiency")

See also

get_completeness: Convenience wrapper for combined efficiency.
get_detection_efficiency: Convenience wrapper for detection-only.
get_classification_efficiency: Convenience wrapper for classification-only.

get_extinction(band: str, pixel: int = None) → float[source]#

Get extinction (A_band) for a specific band.

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’).
pixel (int, optional) – HEALPix pixel index. If None, returns extinction map for entire sky.

Returns:

Extinction in magnitudes. If pixel is None, returns full HEALPix map array.

Return type:

float or np.ndarray

Raises:

ValueError – If the specified band is not available or E(B-V) map not loaded.

get_gal_misclassification(band: str, magnitude: float, maglim: float, **kwargs) → float[source]#

Get galaxy misclassification efficiency (probability a galaxy passes stellar selection).

Mirrors get_efficiency() for the "completeness" type, but without 1-padding at the bright end: bright galaxies are not forced to efficiency = 1 when their delta_mag falls below the saturation threshold. This reproduces the behaviour of custom_get_completeness used for galaxies in external background-generation scripts.

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’).
magnitude (float or np.ndarray) – True apparent magnitude(s) including extinction.
maglim (float or np.ndarray) – Magnitude limit(s) at the source position(s).
**kwargs –

delta_saturationfloat, optional
Override the survey’s default saturation threshold.

Returns:

Misclassification probability in [0, 1].

Return type:

float or np.ndarray

Raises:

ValueError – If gal_misclassification has not been loaded.

get_gal_misclassification_detection(band: str, magnitude: float, maglim: float, **kwargs) → float[source]#

Get galaxy misclassification × detection efficiency.

Mirrors get_efficiency() for the "detected_missclassified" type, but without 1-padding at the bright end: bright galaxies are not forced to efficiency = 1 when their delta_mag falls below the saturation threshold. This reproduces the behaviour of custom_get_completeness used for galaxies in external background-generation scripts.

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’).
magnitude (float or np.ndarray) – True apparent magnitude(s) including extinction.
maglim (float or np.ndarray) – Magnitude limit(s) at the source position(s).
**kwargs –

delta_saturationfloat, optional
Override the survey’s default saturation threshold.

Returns:

Misclassification × detection probability in [0, 1].

Return type:

float or np.ndarray

Raises:

ValueError – If gal_misclassification_detection has not been loaded.

get_maglim(band: str, pixel: int = None) → float[source]#

Get magnitude limit for a specific band.

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’).
pixel (int, optional) – HEALPix pixel index. If None, returns entire map.

Returns:

Magnitude limit(s). If pixel is None, returns full HEALPix map array.

Return type:

float or np.ndarray

Raises:

ValueError – If the specified band is not available in the survey.

get_photo_error(band: str, magnitude: float, maglim: float, kind: str = 'catalog', **kwargs) → float[source]#

Get photometric error estimate.

Parameters:

band (str) – Band identifier (e.g., ‘g’, ‘r’). Used to get band-specific sys_error.
magnitude (float or np.ndarray) – True apparent magnitude(s).
maglim (float or np.ndarray) – Magnitude limit(s) at the position(s).
kind (str, optional) – Which error model to evaluate: "catalog" (reported error, written as magerr and used for S/N cuts) or "sample" (true scatter, used to draw the observed magnitudes). "sample" falls back to the catalog model if no sample curve is loaded. Default is "catalog".
**kwargs –
Additional keyword arguments:

delta_saturationfloat, optional
Magnitude difference for saturation threshold in the initial error function. Default is -10.4.

Returns:

Total photometric error (including systematic component) in magnitudes.

Return type:

float or np.ndarray

Raises:

ValueError – If the requested photo error model is not loaded.

classmethod load(survey: str, release: str | None = None, config_file: dict | None = None, **kwargs) → Survey[source]#

Load survey data and return Survey instance.

This is a convenience method that calls SurveyFactory.create_survey().

Parameters:

survey (str) – Survey name (e.g., ‘lsst’).
release (str, optional) – Survey release/version (e.g., ‘yr1’, ‘yr5’).
config_file (dict, optional) – Custom configuration dictionary.
**kwargs – Additional keyword arguments passed to SurveyFactory.create_survey() to override config values (including verbose: bool = True)

Returns:

Loaded Survey instance.

Return type:

Survey

Examples

Load LSST year 5 survey:

>>> lsst_yr5 = Survey.load('lsst', release='yr5')

Load with uniform survey variant:

>>> lsst_yr4 = Survey.load('lsst', release='yr4', uniform_survey=True)
>>> lsst_yr4_uniform = SurveyFactory._cached_surveys['lsst_yr4_uniform']

property log_photo_error: Callable | None#: Backward-compatible alias for the catalog (reported) error model.

property namespace: str#

Column-prefix namespace for this survey.

"{name}_{release}" when a release is set (e.g. "lsst_yr5", "roman_dc2"), otherwise just "{name}". This is the prefix used for every injected column (<namespace>_<band>_obs etc.), so the same survey at two releases produces distinct, non-colliding columns.

class streamobs.surveys.SurveyFactory[source]#

Bases: object

Factory for creating and caching Survey instances.

This class handles:

Loading survey configurations from YAML/JSON files
Caching loaded surveys to avoid redundant file I/O
Validating survey configurations
Populating Survey objects with all necessary data

_cached_surveys#

Cache of previously loaded Survey objects, keyed by “survey_release”.

Type:: dict

classmethod clear_cache(survey: str | None = None, release: str | None = None, **kwargs)[source]#

Clear cached survey data to free memory or force reload.

Parameters:

survey (str, optional) – Survey name to clear. If None, clears all cached surveys.
release (str, optional) – Survey release to clear. Only used if survey is specified.
**kwargs –

verbosebool, optional
Whether to print progress messages. Default is True.

Examples

Clear all cached surveys:

>>> SurveyFactory.clear_cache()

Clear all LSST surveys:

>>> SurveyFactory.clear_cache('lsst')

Clear only LSST year 1:

>>> SurveyFactory.clear_cache('lsst', 'yr1')

classmethod create_survey(survey: str, release: str | None = None, config_file: dict | None = None, **kwargs) → Survey[source]#

Create or retrieve a cached Survey instance.

Parameters:

survey (str) – Survey name (e.g., ‘lsst’).
release (str, optional) – Survey release/data version (e.g., ‘yr1’, ‘yr5’). If None, loads the base survey configuration.
config_file (dict, optional) – Custom configuration dictionary to use instead of loading from file. If provided, bypasses the standard config file loading.
**kwargs –
Additional keyword arguments. Recognized:

uniform_surveybool, optional
If True, also creates and caches a uniform version of the survey with constant magnitude limits. The uniform survey can be accessed via SurveyFactory._cached_surveys['{survey}_{release}_uniform']. Default is False.

verbosebool, optional
Whether to print progress messages. Default is True.

Any other keyword arguments override config values.

Returns:

Fully loaded Survey instance (from cache if available).

Return type:

Survey

Notes

First call: Loads all data files and caches the result.
Subsequent calls: Returns cached instance (much faster).
Use clear_cache() to force reloading from files.
When uniform_survey=True, the original survey is returned, but a uniform variant is also cached for later retrieval.

See also

_save_uniform_survey: Creates the uniform survey variant.
clear_cache: Clears cached surveys.
list_cached_surveys: Lists all cached survey keys.

classmethod list_cached_surveys() → list[source]#

Get list of currently cached survey names.

Returns:: Names of cached surveys in format “survey_release” or “survey”.
Return type:: list of str

static open_map_sparse(file_path, **kwargs)[source]#

Open a sparse HEALPix map and return it as a standard HEALPix map.

Parameters:

file_path (str) – Full path to the sparse HEALPix file (.hsp).
**kwargs – Additional keyword arguments (currently unused).

Returns:

Standard HEALPix map with NaN for unseen pixels.

Return type:

np.ndarray

static set_completeness(filename, delta_saturation=-10.4, selection='both')[source]#

Load and interpolate completeness/efficiency function from file.

Parameters:

filename (str) –
Path to CSV file with columns:
- ’delta_mag’ : Magnitude difference from limit (mag - maglim)
- ’detection_eff’ : Detection efficiency only
- ’classification_eff’ : Classification efficiency only (the legacy misspelled header ‘classifiction_eff’ is also accepted)
- ’classification_detection_eff’ : Combined efficiency
- ’missclassification_eff’ : Galaxy misclassification efficiency (probability a galaxy passes stellar selection)
delta_saturation (float, optional) – Magnitude difference threshold for saturation. Default is -10.4.
selection (str, optional) –
Which efficiency to use:
- ’detected’ : Detection efficiency only (column ‘detection_eff’)
- ’classified’ : Classification efficiency only (column ‘classification_eff’, or legacy ‘classifiction_eff’)
- ’both’ : Combined detection and classification (column ‘classification_detection_eff’)
- ’missclassified’ : Galaxy misclassification efficiency (column ‘missclassification_eff’)
Default is ‘both’.

Returns:

Interpolation function f(delta_mag) -> efficiency [0, 1].

Return type:

callable

Raises:

ValueError – If selection is not one of the recognised values.

Notes

Bright stars (delta_mag <= delta_saturation): Efficiency forced to 0.
Faint stars (beyond data): Returns efficiency = 0.0.
The saturation parameter is automatically passed from the survey object.

static set_photo_error(filename, delta_saturation=-10.4)[source]#

Load photometric error model from file.

This creates an interpolation function that estimates photometric uncertainty as a function of the magnitude difference from the survey limit (delta_mag).

Parameters:

filename (str) –
Path to CSV file with columns:
- ’delta_mag’ : Magnitude difference from limit (mag - maglim)
- ’log_mag_err’ : Logarithm (base 10) of magnitude error
delta_saturation (float, optional) – Bright magnitude difference threshold. Used to determine the bright-end extension point. Default is -10.4.

Returns:

Interpolation function f(delta_mag) -> log10(magnitude_error).

Return type:

callable

Notes

Bright stars (delta_mag < delta_saturation): Extended with constant error.
Faint stars (beyond data): Returns log10(error) = 1.0 (error = 10 mag).
The saturation parameter is automatically passed from the survey object.

streamobs.surveys module

Contents

streamobs.surveys module#