Skip to content

Create OCA rules (and some DM changes)

Hugo Buddelmeijer requested to merge hb/ocadmchanges into standardcado

This is initial code to create OCA rules automatically from the data lineage.

The goal is not so much to create the OCA rules, but to use the creation of OCA rules as a way to ensure we have designed the pipeline properly.

This only includes the relations between the data products. Things that are missing:

  • where-clauses (e.g. select on filter etc.)
  • group-by clauses
  • cardinality (minRet and maxRet)

It contains some data model changes:

  • All templates now have INS.MODE, DPR.CATG, DPR.TECH and DPR.TYPE defined.
  • Persistence now has its own recipe. This is still preliminary, but is a good use-case for the OCA rules.
  • There is now a distinction between Raw data and Sky. Sky inherits from Raw and is on-sky data.
  • The reduced images and catalog classes have been refactored, so they are easier to use in OCA rules.
  • The photometric and astrometric calibration is now done on fully reduced frames (that is, background subtraction and illumination correction is applied).

Next is adding where and group-by clauses. But this is already a large change, so let's do this first.

@verdoes can you look at the data model changes in particular? That is, micado.vodsl and associationmatrix.ditaa / .png.

Edited by Hugo Buddelmeijer

Merge request reports