| | |
- astro.main.OnTheFly.OnTheFly
-
- AssociateList(common.database.DataObject.DataObject, astro.main.OnTheFly.OnTheFly)
- common.database.DBMain.DBObject(__builtin__.object)
-
- AssociateListParameters
- common.database.DataObject.DataObject(common.database.DBMain.DBObject)
-
- AssociateList(common.database.DataObject.DataObject, astro.main.OnTheFly.OnTheFly)
- exceptions.Exception(exceptions.BaseException)
-
- AssociateListError
class AssociateList(common.database.DataObject.DataObject, astro.main.OnTheFly.OnTheFly) |
| |
This class represents the result of the association between two or more sourcelists.
|
| |
- Method resolution order:
- AssociateList
- common.database.DataObject.DataObject
- common.database.DBMain.DBObject
- __builtin__.object
- astro.main.OnTheFly.OnTheFly
Methods defined here:
- __init__(self, debug=0)
- __str__(self)
- after_set_onthefly_dependencies(self, switches, advanced)
- customized version for setting the associate type,
before the actual make is called
- associate_lists(self, list1=None, list2=None)
- associate_sourcelists(self)
- check_master(self)
- check_number_of_associates(self)
- check_preconditions(self)
- commit(self)
- consistency_check(self, silent=False)
- copy_attributes(self)
- count_from_flag_on_associates(self, mask=None, mode='ANY', count=None, countmode='EQ')
- count_from_flag[_on_associates] returns the number of associations which contain
sources from specified sourcelists
mask mask to filter which associations are to be counted. The mask works as a
bitmask, each bit set representing a sourcelist which was input to the
associatelist. For example bit 2 set means that only the wanted associations
which contain a source from sourcelist number 2 will be counted (depends on
mode).
Default: all associations.
mode Determines how the masking works. Mode can be:
ALL Use only the associations which contain sources from exact
the specified sourcelists.
INTERSECT Use only the associations which contain sources from at least
the specified sourcelists.
ANY Use only the associations which contain sources from at least
one of the specified sourcelists.
Default (if mask is specified): ANY.
count determines the number of different sourcelists which participate in an
association, i.e. if count = 3 then only associations are counted which
have sources from 3 different sourcelists (depends on countmode).
Actually count is the bitcount of the number of participating sourcelists.
countmode Mode can be:
LT returns number of associations with less than the specified
number of participating sourcelists.
LE returns number of associations with less than or equal the
specified number of participating sourcelists.
EQ returns number of associations with exact the specified number
of participating sourcelists.
GE returns number of associations with greater than or equal the
specified number of participating sourcelists.
GT returns number of associations with greater than the specified
number of participating sourcelists.
Default: EQ.
Only works when count is defined.
- count_from_members_on_associates(self, members=0, mode='EQ')
- count_from_members[_on_associates] counts the number of associations which
have a specified number of groupmembers.
members Number of members to look for (depends on mode)
mode Mode can be:
LT returns number of associations with less than the specified
number of members.
LE returns number of associations with less than or equal the
specified number of members
EQ returns number of associations with exact the specified number
of members
GE returns number of associations with greater than or equal the
specified number of members
GT returns number of associations with greater than the specified
number of members.
Default: EQ.
- derive_timestamp(self)
- get_attributes_of_pairs(self, dict1=None, dict2=None)
- get_attributes_of_singles(self, sourcelistnumber, dict=None)
- get_attributes_on_associates(self)
- get_attributes[_on_associates] returns all available attributes of all
participating SourceLists.
- get_data_from_area_on_associates(self, Area=None, depth=20, attrlist=[], mask=None, mode='ANY', count=None, countmode='EQ')
- get_data[_on_associates] returns data associated with requested attributes of the
associated sources.
The function returns a dictionary with as key the AID (Association IDentifier) and as
value the attribute values according to the (modified) list of specified attributes,
for each requested sourcelist a list.
Area specifies the wanted area [coverage of AssociateList].
depth the HTM trixel index depth on which the area is defined [20].
attrlist list containing the wanted attributes. Note that SLID and SID will always
be returned and that SLID is the first and SID the second item. On
return attrlist will be modified accordingly.
Default: [].
mask mask to filter which associations are to be returned. The mask works as a
bitmask, each bit set representing a sourcelist which was input to the
associatelist. For example bit 2 set means that only the wanted attributes
of sourcelist number 2 will be returned (depends on mode). The mask can
also be a list of SLIDs (SourceList IDentifiers) or SourceList Instances.
Default: all associations.
mode Determines how the masking works. Mode can be:
ALL Use only the associations which contain sources from exact
the specified sourcelists.
INTERSECT Use only the associations which contain sources from at least
the specified sourcelists.
ANY Use only the associations which contain sources from at least
one of the specified sourcelists.
Default (if mask is specified): ANY.
count determines the number of different sourcelists which participate in an
association, i.e. if count = 3 then only associations are counted which
have sources from 3 different sourcelists (depends on countmode).
Actually count is the bitcount of the number of participating sourcelists.
countmode Mode can be:
LT returns number of associations with less than the specified
number of participating sourcelists.
LE returns number of associations with less than or equal the
specified number of participating sourcelists.
EQ returns number of associations with exact the specified number
of participating sourcelists.
GE returns number of associations with greater than or equal the
specified number of participating sourcelists.
GT returns number of associations with greater than the specified
number of participating sourcelists.
Default: EQ.
Only works when count is defined.
- get_data_on_associates(self, attrlist=[], mask=None, mode='ANY', count=None, countmode='EQ')
- get_data[_on_associates] returns data associated with requested attributes of the
associated sources.
The function returns a dictionary with as key the AID (Association IDentifier) and as
value the attribute values according to the (modified) list of specified attributes,
for each requested sourcelist a list.
attrlist list containing the wanted attributes. Note that SLID and SID will always
be returned and that SLID is the first and SID the second item. On
return attrlist will be modified accordingly.
Default: [].
mask mask to filter which associations are to be returned. The mask works as a
bitmask, each bit set representing a sourcelist which was input to the
associatelist. For example bit 2 set means that only the wanted attributes
of sourcelist number 2 will be returned (depends on mode). The mask can
also be a list of SLIDs (SourceList IDentifiers) or SourceList Instances.
Default: all associations.
mode Determines how the masking works. Mode can be:
ALL Use only the associations which contain sources from exact
the specified sourcelists.
INTERSECT Use only the associations which contain sources from at least
the specified sourcelists.
ANY Use only the associations which contain sources from at least
one of the specified sourcelists.
Default (if mask is specified): ANY.
count determines the number of different sourcelists which participate in an
association, i.e. if count = 3 then only associations are counted which
have sources from 3 different sourcelists (depends on countmode).
Actually count is the bitcount of the number of participating sourcelists.
countmode Mode can be:
LT returns number of associations with less than the specified
number of participating sourcelists.
LE returns number of associations with less than or equal the
specified number of participating sourcelists.
EQ returns number of associations with exact the specified number
of participating sourcelists.
GE returns number of associations with greater than or equal the
specified number of participating sourcelists.
GT returns number of associations with greater than the specified
number of participating sourcelists.
Default: EQ.
Only works when count is defined.
- get_distances_on_associates(self)
- get_distances calculates the distances between all possible pairs in an
association. The output is a dictionary with as key the associate ID
(AID) and as value a list containing for each pair a tuple which
consists of three items: ((SLID1, SID1), (SLID2, SID2), DISTANCE).
SLID1 is always <= SLID2 and when SLID1=SLID2 SID1 < SID2.
- get_number_of_associations(self)
- get_number_of_pairs(self)
- get_number_of_singles(self)
- get_onthefly_dependencies(self, advanced=None)
- The dependencies that will be onthefly processable
This overrides the method in OnTheFly because input_lists must be filled in
and not sourcelists
- get_pairs(self)
- get_singles(self)
- get_statistics_of_pairs(self)
- getitem_on_associates(self, index)
- len_on_associates(self)
- make(self)
- make_from_dictionary_on_associates(self, dictionary=None)
- make_from_dictionary[_on_associates] creates an AssociateList where the
associations are defined by the user.
If the input_lists contain two Sourcelists, the dictionary should contain
as keys the AIDs, as value a tuple containing two lists, the first list
containing the SIDS of the first SourceList, the second list containing
those of the second SourceList.
If the input_lists contains an AssociateList, the dictionary should then
contain as keys the AIDs, as value a list of SIDs.
- make_from_list_on_associates(self, inputlist)
- make_from_list[_on_associates] creates an AssociateList where the
associations are defined by the user.
The inputlist should contain the lists of (SLID, SID) tuples which
make up one association. For xample the input list containing
[[(1,0), (2,0), (3,1)], [(1,1), (2,1)], [(3,2)]] describes three associations,
the first with three sources, the second with2 and the last with only 1
source.
The associatelisttype will be set to 0, the inputsourcelists will be calculated
from the inputlist.
The user is free to specify name, process_params, llRA etc and OBJECT.
If the AssociateList already exists, the associations are append to the
associatelist. The should however contain the same Sourclists as the existing
AssociateList.
- make_image_dict_on_associates(self, aids, mode='sky')
- make_image_dict[_on_associates] creates an image dict which can be input to
the image server software.
aids An aid (or a list of aids) for which the input is generated.
mode Produce sky (default) or grid coordinates.
- make_mask(self, mask)
- Converts a list or tuple of SLIDs or SourceLists into a bitmask.
- make_matched_associatelist(self)
- make_skycat_on_associates(self, slid, mask=None, mode='ALL')
- Make a skycat catalog to visualize associated sources.
If mask and mode are not specified the catalog will be made for the
associates which have a member in all input SourceLists.
Possible arguments:
slid: SourceList identifier, catalog is made for this SourceList
mask: See AssociateList.get_data_on_associates() method
mode: See AssociateList.get_data_on_associates() method
AssociateList.sourcelists can be used to find back the SLIDs taking
part in the AssociateList.
- migrate_flag(self)
- set_associatelist_identifier(self)
- set_assoclist_name(self)
- Generate and set the unique filename for this AssociateList.
- set_coordinate_ranges(self, L1, L2)
- set_made(self)
- set_process_parameters_from_dict(self, pars={})
- pars is a dictionary of the type e.g.:
{'BiasFrame.process_params.SIGMA_CLIP':8}
- set_search_area(self, llRA, llDEC, lrRA, lrDEC, urRA, urDEC, ulRA, ulDEC)
- set_search_distance(self, search_distance)
- set_sextractor_flag_mask(self, mask)
- set_single_out_closest_pairs(self, single_out_closest_pairs)
- set_user_config(self, user_config_dict)
- Store the user specified configuration (as produced e.g. by the
util.Pars tool) in a transient attribute.
- sql_query_on_associates(self, attrlist=[], mask=None, mode='ANY')
- update_number_of_associates(self)
Data descriptors defined here:
- ALID
- AssociateList ID [None]
- OBJECT
- Object name [None]
- associatelisttype
- Type of AssociateList [None]
- associates
- Definition of base-type of AssociateList [None]
- creation_date
- Date this object was created [None]
- filename
- The name of the associated file [None]
- globalname
- The name used to store and retrieve file to and from Storage [None]
- inputassociatelist
- Input AssociateList [None]
- is_valid
- Manual/external flag to disqualify bad data (SuperFlag) [None]
- llDEC
- Declination of lower left containing box [deg]
- llRA
- Right Ascension of lower left containing box [deg]
- lrDEC
- Declination of lower right containing box [deg]
- lrRA
- Right Ascension of lowe r right containing box [deg]
- name
- Name of the AssociateList [None]
- number_of_associates
- Number of associates [None]
- object_id
- The object identifier
The object identifier is an attribute shared by all persistent
instances. It is the prime key, by which object identity is established
- process_params
- Processing parameters [None]
- sourcelists
- Input list of SourceLists [None]
- ulDEC
- Declination of upper left containing box [deg]
- ulRA
- Right Ascension of upper left containing box [deg]
- urDEC
- Declination of upper right containing box [deg]
- urRA
- Right Ascension of upper right containing box [deg]
Data and other attributes defined here:
- PROCESS_TIME = 100
Methods inherited from common.database.DataObject.DataObject:
- FileName(self)
- Return the pathame of the object
- Name(self)
- Return the name of the object
- check_mandatory_dependencies(self)
- check if all the mandatory dependencies are set
- derive_hash(self)
- Apply hashing function to the contents of the file that is part of
this object.
The SHA hashing function is used to calculate a hash value of the
contents of this DataObject. The hash value is returned as a
hexadecimal string.
- distinct(self, param_list)
- This method returns the distinct values from an input list. The
current implementation is kinda cheesy.
- exists(self)
- Test is the file existsts
- get_canonical_name(self, processlevel='')
- The canonical name of a DataObject.
- get_pathname(self)
- Getter for pathname
- is_on_dataserver(self)
- Checks if file is accessable on the dataserver.
NOTE: A return of False does NOT imply the file is not on any
dataserver! It is simply not on any accessable
dataserver.
- is_stored(self)
- locate(self)
- See common/net/dataserver_client.py (Data_IO.locate).
Returns a list of strings describing all dataservers where the file is
located as well as a few properties of the file, i.e.
['ip=...,port=...,path=...,size=...,mdate=...,
'ip=...,port=...,path=?,size=?,mdate=?']
Question marks indicate that this dataserver is not reachable.
- name_insert_suffix(self, suffix)
- name_replace_suffix(self, suffix)
- name_with_new_suffix(self, suffix)
- release(self)
- retrieve(self)
- set_filename(self, pathname=None, processlevel='')
- Specify a filename for this DataObject or use the default filename.
This method is used to set the filename attribute of a DataObject.
If called without a filename, the filename is set to the canonical name.
Classes that are derived from DataObject are expected to define a
method get_canonical_name() that returns the name for an instance
of that class. This is mandatory for CalFiles (or files that are
store()d on the data server) and optional for other files.
- set_pathname(self, pathname)
- Setter for pathname
- set_stored(self)
- store(self)
- store_with_hash_as_name(self, prefix='', suffix='fits')
- Name the file after the hash value of the file and store it.
First the hash value of the file is calculated. The pathname of the
dataobject is renamed to the hash value with the given suffix appended.
Then the file is stored.
Class methods inherited from common.database.DataObject.DataObject:
- is_cal(cls) from common.database.DBMeta.DBObjectMeta
- Test for being cal, derived raw classes should set _IS_CAL to 1
- is_config(cls) from common.database.DBMeta.DBObjectMeta
- Test for being config, derived raw classes should set _IS_CONFIG to 1
- is_raw(cls) from common.database.DBMeta.DBObjectMeta
- Test for being raw, derived raw classes should set _IS_RAW to 1
- is_science(cls) from common.database.DBMeta.DBObjectMeta
- Test for being science, derived raw classes should set _IS_SCIENCE to 1
- is_seq(cls) from common.database.DBMeta.DBObjectMeta
- Test for being seq, derived raw classes should set _IS_SEQ to 1
- is_support(cls) from common.database.DBMeta.DBObjectMeta
- Test for being config, derived raw classes should set _IS_SUPPORT to 1
Data descriptors inherited from common.database.DataObject.DataObject:
- pathname
Data and other attributes inherited from common.database.DataObject.DataObject:
- filepath = './'
- localname = ''
- mandatory_dependencies = ()
- storage = <common.net.dataserver_client.Storage at 0x167ec20>
Methods inherited from common.database.DBMain.DBObject:
- __del__(self)
- Destructor for the DBObject
This destructor is called by the garbage collector for the transient
part of the DBObject. When it is called it calls the gc_transient()
method of the DBProxy implementation. The implementation of the
gc_transient() method can then decide what to do with the persistent
counterpart.
E.g. the database can be synchronized with its transient counterpart.
Another possibility is that the gc_transient() implementation checks
whether any attributes of a read-only object have changed since an
object was fetched from the database. If there are differences an
exception can be raised.
- __reduce__(self)
- Method necessary to create picklable objects
This mehod return a tuple containg a function object used for
unpickling and a picklable state tuple, to be used as argument
to the unpickle function. If necessary this object is registered
in the pickle cache
- as_dict(self, seqnr=1, dependency_name='', dictionary={}, allow_lazy_typed_list=False)
- Method much like the info method, but returns a dictionary.
Dictionary looks like:
{'<classname>.<property1_name>.<property2_name>': value,
'<classname>.<property2_name>.<property3_name>': value}
Problem: infinite loops for cold/flat dependencies. It seems a maximum
recursion depth limits the problem.
- get_creator(self)
- Returns the name of the user who created this object.
- get_persistent(self)
- get_project(self)
- Returns the name of the project to which this object belongs.
- info(self, level=0, doc=False, _indent=0, _printclass=True)
- Print the values of all persistent properties of the object.
level : level of introspection
doc : display docstrings for attributes
_indent : internal parameter used for formatting
_printclass : internal parameter used for formatting
- inverse_objects(self, max_results=100)
- Iterate through and return all objects that use the current object
max_results The maximum number of objects returned per query
- inverse_query(self)
- go through all inverse properties to see if this object is used
- persists(self)
- Returns whether an object is already made persistent or transient.
- recommit(self)
- Commits changes to an object, without following links.
Class methods inherited from common.database.DBMain.DBObject:
- get_inverse_properties(cls, props=None) from common.database.DBMeta.DBObjectMeta
- return all inverse properties
that are all the properties that point to this class
Data descriptors inherited from common.database.DBMain.DBObject:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Data and other attributes inherited from common.database.DBMain.DBObject:
- __metaclass__ = <class 'common.database.DBMeta.DBObjectMeta'>
- This is the metaclass for persistent classes
Provides:
__new__ -- manages class construction
__call__ -- manages object instantiation
- database = <common.database.DBOracle.DBProxy instance at 0xfab0e0>
- pickle_id = None
Methods inherited from astro.main.OnTheFly.OnTheFly:
- Flagged(self)
- Check if any flag is set
return 1 for flag is set
return 0 no flag
- after_do_make(self, switches=None)
- extra make statements, this method can be overridden by OnTheFly classes to
implement extra make statements which will be executed after the make of self
- after_uptodate_object(self, dependencies_missing, dependencies_new, dependencies_obsolete)
- The uptodate_object method determines if the object is uptodate, and will result in three listings :
dependencies_missing - missing mandatory dependencies
dependencies_new - new dependencies
dependencies_obsolete - current dependencies that are obsolete
Derived classes can override this method to tweak the listings
- check_for_mandatory(self, dep_str, dep_new)
- check if dependency is mandatory and present
dep_str is the name of the attribute, dep_new is the (attribute) object
- uptodate(self, date_time=None, template=None, switches=None, advanced=None)
- This method checks if the object is up-to-date, for the given datetime.
OnTheFly_uptodate class has the functionality for checking uptodate
Class methods inherited from astro.main.OnTheFly.OnTheFly:
- exist(cls, date_time=None, chip=None, filter=None, object_id=None, parent=None, parent_attr=None, template=None, extra=[], advanced=None, switches=None) from common.database.DBMeta.DBObjectMeta
- for the given parameters and class return the most
recent object(s), or None
- get_onthefly(cls, date_time=None, filter=None, chip=None, object_id=None, advanced=None, switches=None, template=None, parent=None, parent_attr=None) from common.database.DBMeta.DBObjectMeta
- For the given parameters and class get the object,
if the object does not exist -> make it.
This is done recursively for all dependencies.
The object can only be made if there are raws
|
class AssociateListParameters(common.database.DBMain.DBObject) |
| | |
- Method resolution order:
- AssociateListParameters
- common.database.DBMain.DBObject
- __builtin__.object
Data descriptors defined here:
- SEARCH_DISTANCE
- Radius of search for associates [arcsec]
- SEXTRACTOR_FLAG_MASK
- SExtractor flag mask value to use to filter sources [None]
- SINGLE_OUT_CLOSEST_PAIRS
- Single out closest pairs [None]
- object_id
- The object identifier
The object identifier is an attribute shared by all persistent
instances. It is the prime key, by which object identity is established
Methods inherited from common.database.DBMain.DBObject:
- __del__(self)
- Destructor for the DBObject
This destructor is called by the garbage collector for the transient
part of the DBObject. When it is called it calls the gc_transient()
method of the DBProxy implementation. The implementation of the
gc_transient() method can then decide what to do with the persistent
counterpart.
E.g. the database can be synchronized with its transient counterpart.
Another possibility is that the gc_transient() implementation checks
whether any attributes of a read-only object have changed since an
object was fetched from the database. If there are differences an
exception can be raised.
- __reduce__(self)
- Method necessary to create picklable objects
This mehod return a tuple containg a function object used for
unpickling and a picklable state tuple, to be used as argument
to the unpickle function. If necessary this object is registered
in the pickle cache
- as_dict(self, seqnr=1, dependency_name='', dictionary={}, allow_lazy_typed_list=False)
- Method much like the info method, but returns a dictionary.
Dictionary looks like:
{'<classname>.<property1_name>.<property2_name>': value,
'<classname>.<property2_name>.<property3_name>': value}
Problem: infinite loops for cold/flat dependencies. It seems a maximum
recursion depth limits the problem.
- commit(self)
- Commits object including objects that are referenced and have been changed.
If this object refers to other persistent object a commit() ensures that
1. ALL changes to the object and referred objects are made persistent.
2. NONE of the changed object and referred objects are made persistent.
Under normal circumstances only the first thing should ever happen.
If the second case occurs it is almost certainly because of some bug in
the routine that implements the DBProxy interface.
- get_creator(self)
- Returns the name of the user who created this object.
- get_persistent(self)
- get_project(self)
- Returns the name of the project to which this object belongs.
- info(self, level=0, doc=False, _indent=0, _printclass=True)
- Print the values of all persistent properties of the object.
level : level of introspection
doc : display docstrings for attributes
_indent : internal parameter used for formatting
_printclass : internal parameter used for formatting
- inverse_objects(self, max_results=100)
- Iterate through and return all objects that use the current object
max_results The maximum number of objects returned per query
- inverse_query(self)
- go through all inverse properties to see if this object is used
- persists(self)
- Returns whether an object is already made persistent or transient.
- recommit(self)
- Commits changes to an object, without following links.
Class methods inherited from common.database.DBMain.DBObject:
- get_inverse_properties(cls, props=None) from common.database.DBMeta.DBObjectMeta
- return all inverse properties
that are all the properties that point to this class
Data descriptors inherited from common.database.DBMain.DBObject:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Data and other attributes inherited from common.database.DBMain.DBObject:
- __metaclass__ = <class 'common.database.DBMeta.DBObjectMeta'>
- This is the metaclass for persistent classes
Provides:
__new__ -- manages class construction
__call__ -- manages object instantiation
- database = <common.database.DBOracle.DBProxy instance at 0xfab0e0>
- pickle_id = None
| |