Skip to content

Data model changes as requested by FDR

Hugo Buddelmeijer requested to merge develop_datamodel into develop

This Merge Request contains most of the changes requested in RIXes at FDR.

This MR contains a large number of data model changes. In particular:

  • The persistence correction is included in the other recipes, so no separate recipe is required.
  • The DepersistedDark is kept, but only as a temporary test for the CPL integration; it can be removed later.

These changes were These changes were agreed upon between the NOVA team, MPE team, and ESO in July 2022, see #262. Future changes that are foreseen for (mostly) this summer but not yet incorporated:

  • Some minor further modifications were agreed upon in #262 which will be dealt separately; these should not require data model changes.
  • There will be some name changes, but for now we plan to only change them in the ESO representation; perhaps we can update the data model later.
  • Consensus from the spectroscopy team still needs to be achieved, which due to vacations will probably happen only in August; agreement is expected.
  • The classes related to depersistence will need to be removed. This is not urgent.

@wjvriend, this MR contains a huge number of data model changes, how shall we proceed to merge this?

We agreed in #262 that the MPE team currently does not rely on MicadoWISE. This gives us some flexibility.

My proposal:

  • You review the changes in micado.vodml and the src/micado/toolbox directory as much as you like. (The could is also a little bit cleaner now.)
  • We merge this into develop.
  • We throw away all the data. (I expect the data model changes are too expansive to modify the schema; feel free to decide otherwise though.)
  • We update the scripts in the simulations directory to produce simulated data to fill the archive. (We could run those scripts in the CI with a small detector, and with the full detector for the archive.)

Other points:

  • I've merged develop into master and created release 2022.07.1, which will be the last release with the data model as submitted for FDR.
  • I can branch my feature branches from this branch (develop_datamodel), so we can merge this when you have the time @wjvriend.
  • Our primary goal w.r.t. to the upcoming changes is to make the develop and master branches match with the documents that are updated for FDR.
Edited by Hugo Buddelmeijer

Merge request reports