| | |
- __builtin__.property(__builtin__.object)
-
- QCFlag
- astro.main.OnTheFly.OnTheFly
-
- ProcessTarget(common.database.DBMeta.DBMixin, astro.main.OnTheFly.OnTheFly)
- common.database.DBMeta.DBMixin(__builtin__.object)
-
- ProcessTarget(common.database.DBMeta.DBMixin, astro.main.OnTheFly.OnTheFly)
class ProcessTarget(common.database.DBMeta.DBMixin, astro.main.OnTheFly.OnTheFly) |
| |
The ProcessTarget mixin encapsulates the notion of a make-able object.
This mixin is useable when a persistent object implements special methods
make(), verify(), compare(), and inspect(), First of all, this mixin
provides special methods to record and inspect the persistent attribute
process_status. For example:
>>> class MyClass(DBObject, ProcessTarget):
... def make(self):
... # process stuff to make a MyClass object
... set_made()
... def verify(self):
... # process stuff to verify results
... set_verified()
...
>>> m = MyClass()
>>> m.make()
>>> m.is_made()
1
>>> m.is_verified()
0
>>> m.verify()
>>> m.is_verified()
1
It is assumed that quality control methods (verify(), compare(), inspect())
will record their results in the persistent attribute quality_flags.
Internally we use bit-masking to set and inspect bits in this attribute.
Hence, each flag cooresponds with the index of a bit in quality_flags.
Possible flags must be QCFlag properties.
For example:
>>> class MyClass(DBObject, ProcessTarget):
... BAD_GALAXY_COUNT = QCFlag(0)
... BAD_STAR_COUNT = QCFlag(1)
... def verify(self):
... if 1:
... self.BAD_GALAXY_COUNT = 1
...
>>> m = MyClass()
>>> m.verify()
>>> m.BAD_GALAXY_COUNT
1
>>> "BAD_GALAXY_COUNT" in m.get_qcflags_set()
1
>>> m.get_qcflags_set()
["BAD_GALAXY_COUNT"]
Note the different methods that are available to find out which
quality control flags have been set.
This class has the following important attributes :
1) process_status : a flag marking the processing status of the object.
This flag can be inspected by using the functions
is_made(), is_verified() and is_compared(). The
corresponding values can be set by calling the
functions set_made(), set_verified(),set_compared()
2) quality_flags : a generic parameter used to record results of
quality control operations (verify(), compare(),
inspect()).
These operations will set flags to mark a failure to
pass some quality control criterium.
3) is_valid : a boolean switch that enables users, after
processing, to declare the target invalid for
further processing purposes.
|
| |
- Method resolution order:
- ProcessTarget
- common.database.DBMeta.DBMixin
- __builtin__.object
- astro.main.OnTheFly.OnTheFly
Methods defined here:
- check_mandatory_dependencies(self)
- check if all the mandatory dependencies are set
- check_observing_blocks(self, childs)
- check if all childs have the same observing_block
- check_templates(self, childs)
- check if all childs have the same template
- compare(self)
- Quality Control -- basic trend analysis
The compare() method is used for default trend analysis. This
is done by comparing with a previous version of the same
object (last weeks bias, for example). This may be as simple
as comparing attribute values, but may also involve more
complex computations (e.g., subtracting the two images, and
analysing the residuals)
- derive_timestamp(self)
- Set the creation_date attribute
- get_dependencies(self)
- Return a list of names of attributes that are required for make()
- get_qcflags_set(self)
- Return a list of names of flags that have been set.
- get_qcflags_set_dict(self)
- Return a dictionary of flags that have been set. The key is the
name, the value is a tuple of (index, docstring).
- inspect(self)
- Quality Control -- visual inspection
Visual inspection of the data remains a powerful tool in
quality control. The inspect() method provides the mechanism
to record the results of visual inspection for posterity.
- is_compared(self)
- Return true if the object has been compared
- is_inspected(self)
- Return true if the object has been inspected
- is_made(self)
- Return true if the object has been made
- is_ok(self)
- Return true if no quality control flags have been set.
- is_verified(self)
- Return true if the object has been verified
- make(self)
- Construct this object from the provided data.
The make method is expected to produce all data and compute
all values associated with this object (i.e.: reduce the
data). It is assumed that the object will have a number of
dependencies which have to be satisfied. A dependency is
assumed to be satisfied if the dependency is not None. In the
case that the dependency is itself a ProcessTarget, the
dependency should be made.
Use the method get_dependencies() to get a list of attributes
on whose values this target depends.
- set_compared(self)
- Set the process status to indicate that the object has been
compared.
- set_inspected(self)
- Set the process status to indicate that the object has been
inspected.
- set_made(self)
- Set the process tatus to indicate that the object has been made
- set_process_parameters_from_dict(self, pars={})
- pars is a dictionary of the type e.g.:
{'BiasFrame.process_params.SIGMA_CLIP':8}
- set_user_config(self, user_config_dict)
- Store the user specified configuration (as produced e.g. by the
util.Pars tool) in a transient attribute.
- set_verified(self)
- Set the process status to indicate that the object has been
verified.
- verify(self)
- Quality Control -- internal consistency
This method inspects the values of various attributes of the
object to see if these are within the expected range for that
object. The purpose of this method is mostly to perform
sanity-checks on measured results. It is assumed that the
required measurements (for example image statistics) are done
during data reduction (i.e. while executing make()), and stored
in persistent attributes.
Class methods defined here:
- get_qcflags(cls) from __builtin__.type
- Return a list of attribute names of QCflag() objects.
- is_cal(cls) from __builtin__.type
- Test for being cal, derived raw classes should set _IS_CAL to 1
- is_config(cls) from __builtin__.type
- Test for being config, derived raw classes should set _IS_CONFIG
to 1
- is_raw(cls) from __builtin__.type
- Test for being raw, derived raw classes should set _IS_RAW to 1
- is_science(cls) from __builtin__.type
- Test for being science, derived raw classes should set _IS_SCIENCE
to 1
- is_seq(cls) from __builtin__.type
- Test for being seq, derived raw classes should set _IS_SEQ to 1
- is_support(cls) from __builtin__.type
- Test for being config, derived raw classes should set _IS_SUPPORT
to 1
- select(cls, **searchterms) from __builtin__.type
- Class method to select RawFrames, Calfiles and ReducedScienceFrames
from the database.
Syntax example:
q = RawScienceFrame.select(instrument='WFI', filter='#842',
chip='ccd50', time_from='2000-01-02 04:45:46',
time_to='2000-01-02 05:03:00')
Possible search terms:
----------------------
check_quality - (Default 1) Exclude invalidated data (quality_flags!=0)
check_validity - (Default 1) Exclude invalidated data (is_valid=0)
chip - select of the same CCD ('ccd50', 'ccd51', etc.)
date - select of the same date (i.e. date at the start of
observing night, in yyyy-mm-dd format)
exptime - select frames with similar exposure time
(EXPTIME-0.8sec to EXPTIME+0.8 sec)
extension - select (raw) frames for a certain extension of its
RawFitsData object
filename - select a frame(!) by its filename
filter - select of the same filter ('#842', '#843', etc.)
instrument - select of the same instrument ('WFI', 'WFC', 'OCAM')
object - select for OBJECT header keyword, uses "like"
functionality, which allows wildcards "*" and "?"
time_from - precise form of date, in yyyy-mm-dd hh:mm:ss format
time_to - required when using time_from
- select_for_raw(cls, raw, overscan=None, check_quality=1, check_validity=1) from __builtin__.type
- Class method to select the most recent valid calfile for the
provided raw frame from the database.
- select_for_reduced(cls, reduced, overscan=None, check_quality=1, check_validity=1) from __builtin__.type
- Synonym for the select_for_raw method; the implementation should
be identical.
Data and other attributes defined here:
- PROCESS_TIME = 0
- STATUS_COMPARE = 2
- STATUS_INSPECT = 3
- STATUS_MAKE = 0
- STATUS_VERIFY = 1
- creation_date = <common.database.DBMeta.persistent object at 0x2aaab8d15c90>
- is_valid = <common.database.DBMeta.persistent object at 0x2aaab8d15cd0>
- mandatory_dependencies = ()
- process_status = <common.database.DBMeta.persistent object at 0x2aaab8d15c10>
- quality_flags = <common.database.DBMeta.persistent object at 0x2aaab8d15c50>
Class methods inherited from common.database.DBMeta.DBMixin:
- get_persistent_attributes(cls) from __builtin__.type
- return a list of persistent attributes
Data descriptors inherited from common.database.DBMeta.DBMixin:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Methods inherited from astro.main.OnTheFly.OnTheFly:
- Flagged(self)
- Check if any flag is set
return 1 for flag is set
return 0 no flag
- after_do_make(self, switches=None)
- extra make statements, this method can be overridden by OnTheFly classes to
implement extra make statements which will be executed after the make of self
- after_set_onthefly_dependencies(self, switches, advanced)
- this method is called after the onthefly method set_onthefly_dependencies
derived classes can implement this method to customize the dependencies
setting after this has been done automatically by onthefly
- after_uptodate_object(self, dependencies_missing, dependencies_new, dependencies_obsolete)
- The uptodate_object method determines if the object is uptodate, and will result in three listings :
dependencies_missing - missing mandatory dependencies
dependencies_new - new dependencies
dependencies_obsolete - current dependencies that are obsolete
Derived classes can override this method to tweak the listings
- check_for_mandatory(self, dep_str, dep_new)
- check if dependency is mandatory and present
dep_str is the name of the attribute, dep_new is the (attribute) object
- get_onthefly_dependencies(self, advanced=None)
- Method retrieves a list of all dependencies used for
on-the-fly processing
Return: [dependency, dependency class, dependency name]
- uptodate(self, date_time=None, template=None, switches=None, advanced=None)
- This method checks if the object is up-to-date, for the given datetime.
OnTheFly_uptodate class has the functionality for checking uptodate
Class methods inherited from astro.main.OnTheFly.OnTheFly:
- exist(cls, date_time=None, chip=None, filter=None, object_id=None, parent=None, parent_attr=None, template=None, extra=[], advanced=None, switches=None) from __builtin__.type
- for the given parameters and class return the most
recent object(s), or None
- get_onthefly(cls, date_time=None, filter=None, chip=None, object_id=None, advanced=None, switches=None, template=None, parent=None, parent_attr=None) from __builtin__.type
- For the given parameters and class get the object,
if the object does not exist -> make it.
This is done recursively for all dependencies.
The object can only be made if there are raws
|
class QCFlag(__builtin__.property) |
| |
A quality control flag.
ProcessTarget objects have quality control flags. These flags are defined
at class scope using the QCFlag property. The property can also be used to
document the flag. For example:
>>> class MyClass(DBObject, ProcessTarget):
... TOO_FEW_OBJECTS = QCFlag(0, "There should be more...")
... TOO_MANY_OBJECTS = QCFlag(1, "If there are this many...")
...
>>> m = MyClass()
>>> m.TOO_MANY_OBJECTS = 1
>>> m.TOO_MANY_OBJECTS
1
QCFlags assumes that the containing class has a (persistent) attribute
quality_flags, which should be either an integer or a long. The
ProcessTarget class has this attribute. QCFlag uses bitmasking (using index
to create a mask) to register in the quality_flags attribute the flags set.
Example :
>>> m.TOO_MANY_OBJECTS = 1
>>> m.quality_flags |= 1 << MyClass.TOO_MANY_OBJECTS.index
are equivalent.
|
| |
- Method resolution order:
- QCFlag
- __builtin__.property
- __builtin__.object
Methods defined here:
- __init__(self, index, doc='')
- A quality control flag indexes the quality_flags attribute.
Instantiate with an unused index 0<=index<31.
Data descriptors defined here:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Methods inherited from __builtin__.property:
- __delete__(...)
- descr.__delete__(obj)
- __get__(...)
- descr.__get__(obj[, type]) -> value
- __getattribute__(...)
- x.__getattribute__('name') <==> x.name
- __set__(...)
- descr.__set__(obj, value)
Data descriptors inherited from __builtin__.property:
- fdel
- fget
- fset
Data and other attributes inherited from __builtin__.property:
- __new__ = <built-in method __new__ of type object at 0x73f060>
- T.__new__(S, ...) -> a new object with type S, a subtype of T
| |