MICADO-WISE issueshttps://gitlab.astro-wise.org/micado/micadowise/-/issues2023-02-22T09:00:13Zhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/271Coordinate incorporation HIGHCONTRAST_HDR data item into MicadoWISE2023-02-22T09:00:13ZGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlCoordinate incorporation HIGHCONTRAST_HDR data item into MicadoWISEAdded it to section 7.1 of [overleaf version of Calibration Plan](https://www.overleaf.com/project/5f15776fe3de1300011e6285) to resolve https://jira.eso.org/browse/MIC-2163 but did not propagate this data item into MicadoWISE [vodsl](htt...Added it to section 7.1 of [overleaf version of Calibration Plan](https://www.overleaf.com/project/5f15776fe3de1300011e6285) to resolve https://jira.eso.org/browse/MIC-2163 but did not propagate this data item into MicadoWISE [vodsl](https://gitlab.astro-wise.org/micado/micadowise/-/blob/develop/src/vodml/micado.vodsl).Gijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/268Reintroduce job that runs build.sh nightly2022-08-26T19:50:45ZHugo Buddelmeijerhugo@buddelmeijer.nlReintroduce job that runs build.sh nightlyThe `build.sh` script is currently only run on Merge Requests, because it creates an assist merge request. However, that means that it is not ran during the nightly jobs.
This has lead to us missing several problems, so it might be usef...The `build.sh` script is currently only run on Merge Requests, because it creates an assist merge request. However, that means that it is not ran during the nightly jobs.
This has lead to us missing several problems, so it might be useful to somehow run `build.sh` as part of the nightly schedule too.https://gitlab.astro-wise.org/micado/micadowise/-/issues/265NOVA-A*V meeting August 17-19 20222023-02-22T09:03:17ZHugo Buddelmeijerhugo@buddelmeijer.nlNOVA-A*V meeting August 17-19 2022# NOVA-A*V meeting August 17/18/19 2022
@kdleschinski from the A*Vienna team will visit @buddel, @wjvriend and @verdoes from the NOVA team on August 17,18 and 19 2022. This issue is used to track that meeting.
# Goals:
At the end of t...# NOVA-A*V meeting August 17/18/19 2022
@kdleschinski from the A*Vienna team will visit @buddel, @wjvriend and @verdoes from the NOVA team on August 17,18 and 19 2022. This issue is used to track that meeting.
# Goals:
At the end of the meeting,
- We have formalized the interface between MicadoWISE and ScopeSim (and ScopeSim_templates and the IRDB), in particular we will tackle
- the propagation of IRDB information into MicadoWISE,
- a way to develop our packages without causing headaches to the other team,
- storing DID files in the IRDB,
- the FITS header keywords in the simulated data,
- the procedure to use ScopeSim_Templates in MicadoWISE (e.g. how to store them in headers).
- We have have a prioritized list of features that the NOVA teams wants to have added ScopeSIM in the future, e.g.
- a better way to set the RNG seed to achieve reproducibility
- a way to simulate templates.
- We have a plan to create pypi based irdb packages for MICADO (and other instruments) to ensure "proper" version control of the instrument packages.
- The A*Vienna team is comfortable with using MicadoWISE properly, both standalone and with the archive and distributed processing.
Optional goals:
- We have a plan on how to get the MPE / spectroscopy / ICS teams involved, in particular with the IRDB.
- We have a plan to create conda-packages of ScopeSim.
- We have a roadmap of activities for ScopeSIM and MicadoWISE for the Gaia astrometry use case
- We have a way forward for all the issues linked below (mostly part of the above already).
# Agenda
Thu 10:30-13:00 discuss Gaia astrometry use case and its roadmap of activities for ScopeSIM and MicadoWISE (lead Gijs).
# Links
Primary MicadoWISE-ScopeSIM issue:
- #226 Coordinate ScopeSIM matters
MicadoWISE issues related to ScopeSIM:
- #264 Ensure that MW simulated data is equivalent to simulated ESO delivery
- #254 Harmonize Headers with ScopeSIM again
- #246 Be test user of MicadoWISE on JupyterHub
- #245 First Simulation Delivery
- #235 Ensure download directory for scopesim is writable
- #211 Make headers (especially of raw data) fully reproducible
- #175 Create realistic raw darks (with some structure)
- #125 Review if integration of IRDB, ScopeSIM and MicadoWISE is ok for calibration through forward modeling of instrument
- #53 Create header keywords per observation template for ICS and ScopeSIM
- #15 Support Time and Space dependent Information
- #13 Create interface between MicadoWISE and ScopeSIM for Templates and ObservingBlocks
ScopeSIM issues related to MicadoWISE
- https://github.com/AstarVienna/ScopeSim/issues/23 coordinate MICADO-SCAO-GaiaRef science use case
- https://github.com/AstarVienna/ScopeSim/issues/31 Dithered observations should be possible
- https://github.com/AstarVienna/ScopeSim/issues/68 Introduce more realistic detector features in simulated MICADO data
- https://github.com/AstarVienna/ScopeSim/issues/30 Calibration data should have persistent structure
- https://github.com/AstarVienna/ScopeSim/issues/33 Persistence should be simulated
Gijs-to-self: [mailthread](https://mail.google.com/mail/u/0/?tab=cm#search/from%3Abuddelmeijer/QgrcJHrnxTPkChZQhSKCXWnbTGGcSWqgwtb)Handover HB2022-08-17https://gitlab.astro-wise.org/micado/micadowise/-/issues/264Ensure that MW simulated data is equivalent to simulated ESO delivery2023-04-18T06:46:43ZHugo Buddelmeijerhugo@buddelmeijer.nlEnsure that MW simulated data is equivalent to simulated ESO deliveryWe delivered simulated data to ESO that was produced directly with ScopeSIM: https://nextcloud.mpe.mpg.de/nextcloud/s/HDyiysQiQZji4Yq
We should ensure that the data that can be created through MicadoWISE is equivalent as the directly-th...We delivered simulated data to ESO that was produced directly with ScopeSIM: https://nextcloud.mpe.mpg.de/nextcloud/s/HDyiysQiQZji4Yq
We should ensure that the data that can be created through MicadoWISE is equivalent as the directly-through-scopesim data. Perfect identity is perhaps not possible due to random number seeds, depending on how the simulations where made.
TODO:
* Ensure we can create data with the same settings, e.g. through https://gitlab.astro-wise.org/micado/micadowise/-/issues/257
* Create a test from the notebooks on nextcloud to ensure that we can create the exact same pixels by forcing the RNG.
Not strictly necessary, but related:
* Ensure the headers match, so we can also ingest the ScopeSIM data, see https://gitlab.astro-wise.org/micado/micadowise/-/issues/254
Handy links:
* Gijs-to-self: [mailthread](https://mail.google.com/mail/u/0/?tab=mm#search/to%3Aleschinski/KtbxLwGgCtDgJrqgvGpSMHRVDxwGTLRTgB)Handover HBWillem-Jan VriendWillem-Jan Vriendhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/262MPE-NOVA data flow face-to-face meeting 11+12jul20222023-02-22T09:01:26ZGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlMPE-NOVA data flow face-to-face meeting 11+12jul2022MPE+NOVA data flow teams face-to-face meeting 11+12 July 2022
Participants: Yixian, Erich Wiezorrek, Gijs, Yves (part), Eckhard (part), Willem-Jan (part and only remote). Ric is not participating live, but shall be kept in loop about r...MPE+NOVA data flow teams face-to-face meeting 11+12 July 2022
Participants: Yixian, Erich Wiezorrek, Gijs, Yves (part), Eckhard (part), Willem-Jan (part and only remote). Ric is not participating live, but shall be kept in loop about results and proposed decisions and Ric has final say as data flow team lead.
# Objectives of the visit
By end of meeting we have accomplished the following:
1. Relinquish the Past: the hand-over of WP Imaging from NOVA to MPE is completed. The MPE WP Imaging team shall have no more dependence on NOVA team and no need for NOVA input to proceed with their work.
2. Conduct the Present: there is a procedure in place how WP Imaging keeps MicadoWISE team informed about updates to imaging pipeline (e.g., changes in data items, workflows, recipe inputs/outputs) so that the MicadoWISE team can update BasicMicadoWISE to continue supporting the WP Imaging team with:
a. a web-based queryable data archive with raw and processed simulated and lab data for the 3 imaging modes
b. a JupyterHub python environment in which the ESO imaging recipes can be embedded for interactive scientific data quality assessment.
3. Open the Future: the NOVA MicadoWISE team has gathered MPE's list of desirable functionalities for an interactive scientific data quality analysis system that complements the EDPS data processing system. The duty of that complementary system is to support the MICADO instrument team through commissioning, shake down phase of ELT and MICADO and then the early science harvest. This list of desirable functionalities will be the basis for the NOVA MicadoWISE team to seek funding opportunities (proposal submission anticipated not to happen before 1jan 2024) for building a system that has those functionalities. Starting the development from the current basic version of MicadoWISE.
# Agenda
Mon morning:
* Discuss already planned workflow / data model changes
* In particular: Persistence, Non-linearity, Data item namings (proposal will be shared by Hugo and Gijs before the meeting)
* See https://gitlab.astro-wise.org/micado/datareductionlibrarydesigndrs/-/merge_requests/121 and https://gitlab.astro-wise.org/micado/datareductionlibrarydesigndrs/-/jobs/49620/artifacts/raw/ELT-SPE-MCD-56305-0019_DataReductionLibraryDesignDRS.pdf
* Discussion on how to share design and how to propagate design changes.
* Guided tour by NOVA through current Digital Design and its proposed maintenance plan as described in [post-FDR memo](https://docs.google.com/document/d/1wOMjXqZDb_z0c7U_9yn-Row2zgTJo11Z6BsaaowwOeE/edit?usp=sharing)
* How far can we get with existing (semi-)deliveries, e.g. IRDB + DID + EDPS + C-code + ICS-code?
* Which data products are used in the pipeline? That is, which data items exist, what FITS extensions do they have, what FITS headers keywords do they have? This is not described in any ESO delivery, but essential for MicadoWISE because this corresponds to database tables and columns, so we need to define this ourselves.
* Decide a collaborative plan on who is responsible for what in Digital Design (e.g., includes IRDB + DID + EDPS) and how to propagate information. That is, which location is the master-copy of each bit of information, and how/when/by who is this copied to the rest?
* In particular: how do we coordinate large workflow / data model updates; e.g. separate branches, separate databases?
Mon afternoon:
* Tackle remaining topics mentioned in [RIXs](https://docs.google.com/presentation/d/1a0NAG0Xm3TWICnNlhnk9zLhRE2PfZQ852CEDFFqo_Y4/edit?usp=sharing), that are not yet discussed before
* Check per AI for a RIX that how to address it is now covered: change assignee to MPE person ([resolved RIX-AIs from Dec2021](https://docs.google.com/presentation/d/1a0NAG0Xm3TWICnNlhnk9zLhRE2PfZQ852CEDFFqo_Y4/edit#slide=id.g11c609d5810_0_0), [RIX-AIs with deadline 01Nov2022](https://docs.google.com/presentation/d/1a0NAG0Xm3TWICnNlhnk9zLhRE2PfZQ852CEDFFqo_Y4/edit#slide=id.g11c609d5810_0_5), [RIX-AIs with deadline MidTermReview](https://docs.google.com/presentation/d/1a0NAG0Xm3TWICnNlhnk9zLhRE2PfZQ852CEDFFqo_Y4/edit#slide=id.g11c609d5810_0_10))
* Hand-over NOVA docs in [MICADO data flow documentation portal](https://bscw.mpe.mpg.de/bscw/bscw.cgi/5787509?op=preview&back_url=2745070) from NOVA to MPE:
* Calibration Plan ([overleaf](https://www.overleaf.com/project/5f15776fe3de1300011e6285), [gitrepo](https://gitlab.astro-wise.org/micado/calibrationplan)),
* DRLD ([overleaf](https://www.overleaf.com/project/5f157739e3de1300011e61aa), [gitrepo](https://gitlab.astro-wise.org/micado/datareductionlibrarydesigndrs)) and
* DRLVT ([overleaf](https://www.overleaf.com/project/5f157758e3de1300011e6274), [gitrepo](https://gitlab.astro-wise.org/micado/drlvalidationtestplandrs))
Tue morning (with Willem-Jan remotely):
* Guided tour by NOVA through MicadoWISE imaging data archive (DBview).
* Delivery by NOVA to MPE of a MicadoWISE JupyterNotebook running ScopeSIM to create raw darks & flats that are then processed using Yixian's masterdark and masterflat ESO recipes using the Groningen compute cluster and the MicadoWISE imaging archive for its input and output. This JupyterNotebook will continue to be technically supported by MicadoWISE team.
Tue afternoon:
* 13:30 Meeting with Yves
* Tick off open FDR AI for RIXes:
* Discuss RIX [MIC-2144](https://jira.eso.org/browse/MIC-2144) about naming convention (part of #256)
* Discuss DPR keyword hierarchy (RIXs [MIC-1672](https://jira.eso.org/browse/MIC-1672))
* [MIC-2157](https://jira.eso.org/browse/MIC-2157)
* [MIC-2142](https://jira.eso.org/browse/MIC-2142)
* [MIC-1639](https://jira.eso.org/browse/MIC-1639) the hierarchy of DPR.TECH and DPR.TYPE is a bit arbitrary now, a bit of guidance could be useful. (see also !321)
* More RIXes:
* RIX [MIC-2166](https://jira.eso.org/browse/MIC-2166) Persistence and non-linearity
* and RIX [MIC-2146](https://jira.eso.org/browse/MIC-2146) Unify IMG and SPEC persistence
* RIX [MIC-1652](https://jira.eso.org/browse/MIC-1652) "Using 'image" for being 9 frames (i.e. 9 pixel 'images') is misleading." Not sure it is now correct.
* RIX [MIC-2145](https://jira.eso.org/browse/MIC-2145) Update the document with all info regarding the catalogues formats and size
* Discuss [EDPS-rules](https://docs.google.com/document/d/1E8uk2D5ZXjOniXcak9GgC1cwsXR85beuN4tygd1VXm8/edit#)
* Discuss 'data item dictionary'
* Discuss granularity of recipes (see [memo explaining the choice for fine-grained recipes in the FDR pipeline](https://docs.google.com/document/d/1DApb7tsXwrh65GWoFiMkl9i_OCOBwJQqUyJ6ffhQ0cQ/edit#heading=h.c3bffeho5jge))
* if time permits: depersist strategy, the merging with PSF-R software, and the regression tests
* 16:00: Meet with Eckhard: debrief on meeting results.
* 16:30 Contingency (but Gijs has Euclid project videocon until 18:00)
# Conclusions
<!-- Output copied to clipboard! -->
<!-----
Yay, no errors, warnings, or alerts!
Conversion time: 0.834 seconds.
Using this Markdown file:
1. Paste this output into your source file.
2. See the notes and action items below regarding this conversion run.
3. Check the rendered output (headings, lists, code blocks, tables) for proper
formatting and use a linkchecker before you publish this page.
Conversion notes:
* Docs to Markdown version 1.0β33
* Fri Jul 15 2022 06:24:26 GMT-0700 (PDT)
* Source doc: 20220711.MPE
----->
Below are the agreements made between the NOVA team (Hugo and Gijs), the MPE team (Yixian), and to some extent ESO (Yves). This list primarily includes the agreements between the NOVA Imaging team and the MPE Imaging team. Agreements internal to NOVA or MPE, or between ESO (Yves) and either team, are only included where relevant. The agreements are ordered based on the order of the "goals of the meeting" in the agenda.
* The past: Handover to MPE of the documents (DRLD, CP, DRLVT).
* For all [RIXes](https://docs.google.com/presentation/d/1a0NAG0Xm3TWICnNlhnk9zLhRE2PfZQ852CEDFFqo_Y4/edit?usp=sharing) there is a plan to get them resolved, and discussed with Yves when necessary. Some are still to be discussed with the spectroscopy team; agreement is expected. 3 RIXes are handed over to YC, to be resolved at a later stage:
* [https://jira.eso.org/browse/MIC-2131](https://jira.eso.org/browse/MIC-2131) Image Analysis Library or DPS-ICS libraries
* [https://jira.eso.org/browse/MIC-2200](https://jira.eso.org/browse/MIC-2200) The trigger of the pipeline recipe is a template, not an OB.
* [https://jira.eso.org/browse/MIC-2138](https://jira.eso.org/browse/MIC-2138) The detailed description of the tests will be developed in future updates of the document
* The NOVA team will ensure all other RIXes will get the status of “Resolved” before the handover. (Hugo will do [these](https://gitlab.astro-wise.org/micado/micadowise/-/issues/197#note_36291) (TBC), Gijs will do rest.)
* Still to be discussed with spectroscopy is the naming of the data items / CPL recipes / DRL functions, and the shared parts of our pipeline. We expect that to go well.
* The documents will be updated to make them useful as a reference document for the MPE team. In particular, this means that
* The DRLD will be updated to reflect the changes we agreed upon (see below).
* E.g. the naming conventions are explained; this is more important than actually using the correct names.
* It will be indicated in the text where it is already anticipated that MPE might adapt the design, for example the PRO.CATG keywords.
* There is no need to handover the scripts for the automatic generation of the information in the DRLD. The MPE team can still use them if they want.
* MPE will first focus on the User Manual, which has to be delivered before the next version of the DRLD, and will then later update the DRLD based on the User Manual.
* The present: Propagating pipeline changes back from the MPE team to the NOVA team, as needed for BasicMicadoWISE (the archive and distributed processing):
* The desire of the NOVA team is to have a machine readable “Single Source of Truth” for the various pieces of information that is shared between the teams. (Different information can have a different source though.) This will make it possible to
* at least automatically verify that BasicMicadoWISE uses the correct truth, in case the information has to be copied by hand. Or even better,
* If possible, directly use the correct information automatically; e.g. through code generation.
* We determined that almost all information needed for BasicMicadoWISE can be derived from ESO-deliveries, although the details on how to best do that need to be investigated:
* EDPS rules: the NOVA team anticipates that they can parse/use EDPS rules to get
* a list of data items that exist,
* the workflow orchestration, and
* some of the header keywords,
* CPL recipes: from the documentation, or otherwise from calling esorex, it will be possible to infer:
* which recipes exists,
* what process parameters they have,
* what input / output they have, which should overlap with the EDPS rules
* QC parameters list (ESO-ELT-DIC.MICADO_QC), for which
* the "Context" property of each parameter will be used to indicate which data items have that QC parameter, and
* we need to decide where this file will be stored and shared (currently in MicadoWISE git repository); perhaps it can be included with the pipeline code or with the IRDB.
* To remain feasible a minimal amount of FITS header keywords & values will be stored in the queryable database of the archive:
* Keywords from the template descriptions, which is a small enough set to share manually if necessary (see below).
* Keywords from the IRDB (that is, hardware description), which will be agreed upon between the NOVA team and the simulator team.
* Keywords relevant for constructing the workflow, which can be derived from the EDPS rules.
* QC parameters; from ESO-ELT-DIC.MICADO_QC.
* Process parameters; from the CPL recipes.
* Provenance; from the EDPS rules and/or the CPL recipes.
* Any other header keywords can optionally be included; but are not essential.
* The Instrument Reference Database (IRDB) will also provide relevant information, of which the Simulator team is the custodian, because it is also needed for ScopeSIM.
* The NOVA team will rely on the IRDB. How information from the MPE team is shared with the Simulator team (if any) is between them.
* Of particular interest is information about templates, for which we do not yet have a (new) single source of truth location.
* The template information is small enough to be shared informally as a fallback if necessary.
* ScopeSIM has the desire to process full templates (currently it can do only single exposures); that would require the template information to be in the IRDB, MicadoWISE would then automatically be able to use it.
* There will (at some point) be a machine readable representation of the templates by the ICS team, so using that might also be a possibility.
* The current digital design files (e.g. the VO-DML representations, Python files, yaml files etc in the MicadoWISE gitlab repository) will not be used by the MPE team, with the exception of ESO-ELT-DIC.MICADO_QC. These files will therefore be maintained by the NOVA team, or discarded if they are not needed anymore (their primary goal was to aid the design at a time when there was no other representation of that information).
* The future: The future of (Basic)MicadoWISE (the archive and distributed processing).
* In the short term (~the next year):
* The archive can be used to create and/or store simulated data. For this, the NOVA team will ensure that a) data simulated directly through from ScopeSIM can be ingested, and b) data simulated through MicadoWISE will be equivalent to data simulated through ScopeSIM directly.
* The development and testing of the recipes can be done locally by MPE and does not require MicadoWISE. It is therefore not imminent for MicadoWISE to support running of the latest CPL recipes, or vice-versa to ensure the CPL recipes are always runnable by MicadoWISE. This will allow the teams to work at their own pace; e.g. it is not directly problematic if the EPDS rules and CPL recipes are out of sync, with each other or with MicadoWISE.
* Nevertheless, the NOVA team will frequently attempt integration of the CPL recipes into BasicMicadoWISE, for example when the MPE team makes an internal development release. This will ensure we detect integration problems early and will strengthen our collaboration.
* Medium term (~after a year):
* The validation of the (already tested) recipes would benefit from the archive and distributed processing; especially for the critical algorithms (mainly for astrometric imaging). For example MicadoWISE would make it easy to run the recipes on large amounts of observations with a range of observing conditions to test robustness of recipes against "crappy" observation quality.
* The recipes and workflow should be relatively stable at that point, so fully supporting them in MicadoWISE should then be feasible.
* Long term (~commissioning and beyond):
* The prime reason to include the long term perspective in the meeting is because it provided a framework for points 2 and (the rest of) 3 above. We agreed that it would be useful to be able to store the real data in the archive, and did not discuss it much further.
* The NOVA team will attempt to also include lab data in the archive where this would be useful.
* The NOVA and MPE teams will work out further plans for the (Full)MicadoWISE Consortium Information System, including funding proposals.
We also made some agreements with respect to the design itself. Some of them were informational only, some require changes that will be incorporated by NOVA in the DRLD before handover, and some are changes that will be made by MPE.
* Informational:
* Flatfielding: The flatfielding in the current design is split in two (multiplicative) corrections “MasterFlat” and “Illumination Correction”. The split is designed this way, because the sky Background subtraction (an additive effect) should be applied after the MasterFlat, but before the Illumination Correction (at least in the FDR version of the design). This lead to the following design:
* The MasterFlat corrects the pixel-to-pixel differences of the quantum efficiency, assuming a fully uniform calibration lamp. This MasterFlat is created by median stacking without any smoothing, and then normalizing. Applying the MasterFlat will not actually lead to a ‘flat’ image because the illumination pattern of the calibration lamp is not actually uniform.
* The Illumination Correction corrects for the incorrect assumption of a uniform illumination. The assumption at FDR was that the illumination pattern is smooth and can be fully parameterized. The Illumination Correction is therefore not an image but a table (containing the coefficients of this parametrization).
* Distortions: The design of the distortion correction at FDR is twofold:
* The DistortionELT and DistortionWAM will allow a low-order correction of the distortion. The DistortionELT and DistortionWAM are both derived by the calibration team and provided as a static calibration data product. (During the meeting, we assumed that the DistortionWAM is created by the user. However, in the FDR design, this is not the case, see the Calibration Plan.)
* The workflow for Astrometric Imaging includes a higher-order correction that is derived from the science data itself.
* Before handover:
* The persistence correction will not require a separate recipe anymore. This will make the imaging pipeline equivalent to the spectroscopy pipeline.
* The nonlinearity correction will be the first correction in the workflow.
* The naming convention will be clarified, in particular of the data items and DRL functions. The actual names will be updated on a best-effort basis, because the actual data items and DRL functions will depend on implementation choices from MPE.
* The PRO.CATG values will be kept as-is, because they are an integral part of the current pipeline design and changing them would require a full overhaul of the DRLD which is undesirable.
* The granularity of the recipes will be kept as is. In particular, recipes will run on single exposures whenever feasible. (Still triggered by a full template though.)
* The creation of the DistortionELT (and DistortionWAM?) data product will be removed from the Astrometric Imaging association matrix, because it is produced by the calibrations team and not by individual scientists.
* After handover:
* The derivation of the nonlinearity will use the det_mon software from ESO.
* The PRO.CATG (and therefore DO.CATG) values will be changed to the liking of MPE, probably to be identical to the name of the data items.
* The MPE team will decide whether to update the OCA rules to reflect the new PRO.CATG values or whether to directly create EDPS rules.
* The granularity of the recipes will be discussed further between MPE and ESO.
* MPE will create smaller utility recipes where useful, or at least structure the processing in DRL functions such that it would be easy to create such smaller utility recipes (e.g. by the NOVA team).
* MPE will consider adding association matrices for data products produced by the calibration team, e.g. Nonlinearity, Persistence, DistortionELT, etc.
# Handy links
* GijsToSelf: [mailthreadYixian](https://mail.google.com/mail/u/0/?tab=rm#search/from%3Ayixian/QgrcJHrhwzsqrPmTHmNDKtCVsNzXHFMxhFq), [mailthreadYves](https://mail.google.com/mail/u/0/?tab=rm#search/from%3Ajung/FMfcgzGpGdnzDhPgZhNxBJKwqxHlfbzp), [RIXs](https://docs.google.com/presentation/d/1a0NAG0Xm3TWICnNlhnk9zLhRE2PfZQ852CEDFFqo_Y4/edit?usp=sharing)Gijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/260Use EDPS as backward chaining?2023-02-22T09:00:58ZHugo Buddelmeijerhugo@buddelmeijer.nlUse EDPS as backward chaining?Currently the backward chaining is done through a very ad-hoc https://gitlab.astro-wise.org/micado/micadowise/-/blob/develop/src/micado/toolbox/augment_backward_chaining.py . The purpose of `augment_backward_chaining.py` is twofold:
* C...Currently the backward chaining is done through a very ad-hoc https://gitlab.astro-wise.org/micado/micadowise/-/blob/develop/src/micado/toolbox/augment_backward_chaining.py . The purpose of `augment_backward_chaining.py` is twofold:
* Contain the information required to generate the OCA rules in https://gitlab.astro-wise.org/micado/micadowise/-/blob/develop/reflex/micado.oca
* Be a stepping stone for a real backward chaining implementation later.
The OCA rules are going away though. There will probably never be OCA rules for MICADO. The OCA rules are being replaced by EDPS rules. !314 creates draft EDPS rules.
We should investigate whether the opposite is possible. That is, that YC creates EDPS rules manually for the official ESO MICADO Pipeline, and that we subsequently use those EDPS rules to create the backward chaining in WISE.
As in, can we generate `exists()`, `get_onthefly()`, and `get_onthefly_dependencies()` and such from these EDPS rules? Or can we integrate the EDPS into MicadoWISE and simply call that instead of using our own backward chaining?
Tentatively assigned to @wjvriend because it will probably not be possible for me to work on this.Handover HBhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/257Use IRDB for defaults, e.g. for DetectorArray2023-02-22T09:00:50ZHugo Buddelmeijerhugo@buddelmeijer.nlUse IRDB for defaults, e.g. for DetectorArrayThe values from the IRDB should be used as defaults, also for persistent attributes that are instances, e.g. for the DetectorArray.
Current IRDB integration:
* IRDB is a git submodule of MicadoWISE.
* Defaults from the IRDB are manuall...The values from the IRDB should be used as defaults, also for persistent attributes that are instances, e.g. for the DetectorArray.
Current IRDB integration:
* IRDB is a git submodule of MicadoWISE.
* Defaults from the IRDB are manually copied into micado.vodsl (Hardware section), but only for simple attributes (ints, floats, strings), not objects themselves.
* ìrdb_test.py` verifies whether these defaults are correct.
* Nothing is done for objects themselves. It is therefore only possible to e.g. use the true detector array by manually typing it out.
Next step:
* In `python_from_vodml.py`, also generate default objects, querying the IRDB for values.
* Then use these default objects as defaults for the persistent attributes.
Future step (should be relatively easy, depending on success of previous step):
* Remove Hardware section from micado.vodsl, because everything is retrieved from the IRDB.
* Perhaps also refactor `python_from_vodml.py`, because the hardware section is not created from vodml anymore but from the IRDB.Pipeline Skeleton Runninghttps://gitlab.astro-wise.org/micado/micadowise/-/issues/256Remaining RIXes2023-02-22T09:00:45ZHugo Buddelmeijerhugo@buddelmeijer.nlRemaining RIXesSee https://docs.google.com/presentation/d/1a0NAG0Xm3TWICnNlhnk9zLhRE2PfZQ852CEDFFqo_Y4/edit#slide=id.g11c609d5810_0_5
Status of RIXes done as far as NOVA team concerned
- 🏁 Closed by ESO.
- 🏳️🌈 Submitted to ESO and marked as Resolved....See https://docs.google.com/presentation/d/1a0NAG0Xm3TWICnNlhnk9zLhRE2PfZQ852CEDFFqo_Y4/edit#slide=id.g11c609d5810_0_5
Status of RIXes done as far as NOVA team concerned
- 🏁 Closed by ESO.
- 🏳️🌈 Submitted to ESO and marked as Resolved.
- 🇩🇪 Handed over to MPE.
Status of in progress RIXes.
- 💌 In DRLD / CP / DRLVT on overleaf, and ready to be submitted to ESO.
- 💌🇦🇹 In DRLD / CP / DRLVT on overleaf, to be discussed with Innsbruck team before submitting.
- 💠 In MicadoWISE develop branch.
Next person to work on things:
- 🇭 HB
- 🇬 GVK
- 🇾 YC (handed over to MPE)
- 🇼 WK (send to Innsbruck for discussion)
More
- ✒ Requires text in DRLD / CP / DRLVT to update
- 🇦🇹 Anything that still needs to be agreed upon by Innsbruck team.
RIXes in Group 2, highlighted as important:
* 🇼✒🇦🇹 https://jira.eso.org/browse/MIC-2144 important: imaging & spectro: (i) correct & uniform function/product naming conventions (Hugo makes proposal), need to add
* Description on what naming convention is for DRL functions
* Perhaps also update DRL functions
* Do the same for data products? (Does not seem to be part of this RIX)
* Discuss with WK
* 🇼✒💌🇦🇹 https://jira.eso.org/browse/MIC-2144 (ii) uniform persistence workflow (Hugo makes proposal first to Gijs and later to cc. Wolfgang)
* 🇬✒ https://jira.eso.org/browse/MIC-2181 important: astrometry: decide on FITS WorldCoordinateSystem parameterisation (Gjis)
* 🏳️🌈 https://jira.eso.org/browse/MIC-2142 important: ESO-compliant (hierarchical) naming convention for raw data classification FITS keyword needed for workflow orchestration (!321 acceptable to YJ, but RIX raised by Adam Dobrzycki)
* 🏳️🌈 https://jira.eso.org/browse/MIC-2220 important: shorten classification FITS keyword values processed data (but keep string descriptive) (Hugo, to be submitted)
* 🏳️🌈 https://jira.eso.org/browse/MIC-2157 Done important: erroneous values in some raw data classification FITS keyword needed for workflow orchestration. Correct all DPR.TECH in ELT SPE MCD 56305 0019 Data Reduction Library Design. (Hugo, to be submitted) ( https://gitlab.astro-wise.org/micado/micadowise/-/merge_requests/320 )
Other RIXes in GROUP 2: requiring implementation choices, deadline 1 Nov 2022 (first main post-FDR delivery: Data Reduction System Chain Skeleton)
* 🇬✒ https://jira.eso.org/browse/MIC-2145 Update the document with all info regarding the catalogues formats and size (Gijs)
* 🏳️🌈 https://jira.eso.org/browse/MIC-2129 more efficient OCA rules https://gitlab.astro-wise.org/micado/micadowise/-/merge_requests/320 (MR in MicadoWISE repository either merged or submitted)
* 🏁 https://jira.eso.org/browse/MIC-2123 raw skyframes classification FITS keyword: resolved by Wolfgang Kausch
* 🇩🇪 https://jira.eso.org/browse/MIC-2131 Image Analysis Library or DPS-ICS libraries
* 🏳️🌈 https://jira.eso.org/browse/MIC-2220 Done important: shorten classification FITS keyword values processed data (but keep string descriptive) (Hugo)
* 🏳️🌈 https://jira.eso.org/browse/MIC-2166 Correct workflow so that correction of non-linearity is the first action to carry out (done)
* 🏳️🌈 https://jira.eso.org/browse/MIC-2165 Remove the persistence map from the linearity coefficient determination. (done)
* 🏳️🌈 https://jira.eso.org/browse/MIC-2164 Complete the list of QC parameters, in particular, to include parameters that estimate the accuracy of the quantities provided.
* 🇬 https://jira.eso.org/browse/MIC-2163 Properly define the output of the micado_coro_healthcheck recipe. In particular, regarding the creation of a FITS file.
* 🇬 https://jira.eso.org/browse/MIC-2140 how to handle the windowing resetting
GROUP 3: AIs requiring implementation and testing activities towards Mid Term Review, deadline Mid Term Review.
* 🇩🇪 https://jira.eso.org/browse/MIC-2200 The trigger of the pipeline recipe is a template, not an OB.
* 🇼💌🇦🇹 https://jira.eso.org/browse/MIC-2146 Unifying as much as possible the workflows handling the persistence correction
* 🇬 https://jira.eso.org/browse/MIC-2143 important: nvestigate and describe how to propagate errors for the astrometric case
* 🇬 🇦🇹 https://jira.eso.org/browse/MIC-2141 Investigate whether the data resampling (especially for the spectroscopic mode) is really needed.
* 🇩🇪 https://jira.eso.org/browse/MIC-2138 The detailed description of the tests will be developed in future updates of the documentGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/252Coordinate ingestion & release of MicadoWISE archive with simulated data for ESO2023-02-22T09:12:12ZGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlCoordinate ingestion & release of MicadoWISE archive with simulated data for ESOGijs-to-self: get clarity on MicadoWISE headersGijs-to-self: get clarity on MicadoWISE headersGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/249Improve dbviewer2024-02-27T19:56:31ZHugo Buddelmeijerhugo@buddelmeijer.nlImprove dbviewerThe MicadoaWISE dbviewier is located at http://micado-dbview.hpc.rug.nl . E.g. [all MasterDarks](http://micado-dbview.hpc.rug.nl/DbView?project=SIM&set_cookie=&QSort=&2ndQSort=&mainpref_numrows=100&mainpref_fcontext_project_only=yes&main...The MicadoaWISE dbviewier is located at http://micado-dbview.hpc.rug.nl . E.g. [all MasterDarks](http://micado-dbview.hpc.rug.nl/DbView?project=SIM&set_cookie=&QSort=&2ndQSort=&mainpref_numrows=100&mainpref_fcontext_project_only=yes&mainpref_show_expanded=yes&mainpref_show_empty_joins=no&Exportselect=NoDownload&class_str=MasterDark&mode=table_res&tree_state=MasterDark)
It can be improved:
- [ ] The dbviewer should be mentioned in the documentation.
- [ ] All `dummy_*` columns should be hidden from view.
- [x] The download links should go to the MicadoWISE dataserver (now they go to http://localhost:8008/).
- [ ] There seems to be something weird with following uplinks. See the [uplinks of a specific DepersistedDark](http://micado-dbview.hpc.rug.nl/DbView?mode=object_uplinks&class_str=DepersistedDark&object_id=D5EC889AD1F9649CE053184A17AC424F); there is only 1. However following that links [shows 10 times the same MasterDark](http://micado-dbview.hpc.rug.nl/DbView?numrows=1000&class_str=MasterDark&mode=table_res&project=SIM&set_cookie=1&fcontext_project_only=no&MasterDark.darks.object_id=HEXTORAW%28%27D5EC889AD1F9649CE053184A17AC424F%27%29&MasterDark.darks.object_id.op=%3D&tree_state=MasterDark,MasterDark.darks%23DepersistedDark&mainpref_show_expanded=yes&mainpref_show_empty_joins=no).https://gitlab.astro-wise.org/micado/micadowise/-/issues/246Be test user of MicadoWISE on JupyterHub2023-02-22T09:01:48ZGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlBe test user of MicadoWISE on JupyterHubObjective: realise via a MicadoWISE Python Notebook on the OmegaCEN JupyterHub the science use case called “MICADO-SCAO-GaiaRef”. It assesses the achievable accuracy in absolute astrometry, absolute photometry and spectroscopy for celest...Objective: realise via a MicadoWISE Python Notebook on the OmegaCEN JupyterHub the science use case called “MICADO-SCAO-GaiaRef”. It assesses the achievable accuracy in absolute astrometry, absolute photometry and spectroscopy for celestial objects observed with MICADO at the ELT in SCAO mode using a Gaia star as the Natural Guide Star (NGS). The NGS is centered in the central detector and the celestial object is inside the central detector as well. The celestial object can be a star, a galaxy or a solar system object.
Handy links:
* [MICADO-SCAO-GaiaRef on ScopeSIM github](https://github.com/AstarVienna/ScopeSim/issues/23)
* [BasicMicadoWISE notes](https://docs.google.com/document/d/1zL5QzuWH5tSSPhYI2_W-kSJ8zJL1pmv9qjaQyArlh9c/edit#bookmark=id.5e5bw6eo36v7)
* [MICADO-SCAO-GaiaRef draft paper](https://docs.google.com/document/d/1zEkTaA47ZEbJLvH9gTdKunU1cyDM66PtlXszMbElP0A/edit?usp=sharing)Gijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/245First Simulation Delivery2023-02-22T09:04:24ZHugo Buddelmeijerhugo@buddelmeijer.nlFirst Simulation Delivery6 months after FDR we have to deliver our first simulated data to ESO. The exact date for this is uncertain, we set April 27 2022 for ourselves, as that is the FDR2 board report + 6 months. We are in pretty good shape to achieve that wit...6 months after FDR we have to deliver our first simulated data to ESO. The exact date for this is uncertain, we set April 27 2022 for ourselves, as that is the FDR2 board report + 6 months. We are in pretty good shape to achieve that with ScopeSIM (v1.4) and MicadoWISE as they currently are.
I (HB) have made this an issue in MicadoWISE because that makes it easy divide this up into smaller issues. It is labeled with "To review" because then it shows up in the kanban board and is therefore reviewed every week by me.
(@ycao and @kdleschinski I tag you in this issue, I'll send a follow-up email as well.)
Plan
====
Proposed plan by HB, detailed below:
* Limit our scope to data required for Standard Imaging pipeline skeleton.
* Use MicadoWISE to simulate the data, through a script in the `simulations` directory.
* (Optionally) store the data in the MicadoWISE archive.
The reason to simulate through MicadoWISE is because:
* This creates the headers compliant with our design and archive (see #53 to get that in ScopeSIM directly),
* Allows full reproducibility (or almost, see #211 ),
* Makes it easy to script everything,
* Allows direct ingestion into the archive.
In the (near) future it should be possible to simulate the data directly through ScopeSIM as well; in my opinion this is not necessary for our April 26th delivery.
Scope
=====
We should limit our scope to what is both useful and achievable. Ultimately it should probably be @ycao who has the final say in what we deliver.
Within Scope
------------
Proposal by HB for what is within scope of our first data delivery:
* Standard Imaging Raw
* DARK_RAW (ScopeSIM Template source: empty_sky)
* FLAT_RAW (ScopeSIM Template source: flatlamp)
* SCIENCE_RAW (ScopeSIM Template source: cluster)
* Astrometric Imaging
* Nothing
* High Contrast Imaging
* Nothing
We can already simulate all this right now.
For now we should simulate the effects as they are required for the skeleton, as in we don't care about the same things that scientists care about. E.g. we don't care about the Atmospheric Dispersion, because there is nothing that we do in the pipeline with the ADC (currently, see #49).
Unsure about Scope
------------------
Data items that HB is unsure about whether we should or should not simulate them:
* Standard Imaging Raw
* ILLUM_RAW (doable if we use the same setup as for SCIENCE_RAW)
* NONLIN_RAW (doable if we use the same setup as for FLAT_RAW)
* STDFIELD_RAW (doable if we use the same setup as for SCIENCE_RAW)
* Intermediate / Processed data from Standard Imaging Pipeline:
* It is unclear to me (HB) what is required of us w.r.t. delivering intermediate and final data products.
* Output from other pipelines (Astrometric Imaging) that is used in Standard Imaging (so required to run skeleton):
* NONLINEARITY_IMG (this is quite a complicated structure that we have not properly defined yet)
* ILLUMCORR_HDR (might be doable)
* DISTORTIONWAM_HDR and DISTORTIONELT_HDR (I expect our design of these will change based on the work of the Astrometry Working Group)
* External Data (that is required to run the pipeline, but should somehow be tied to the simulations):
* REF_ASTROM_CAT
* REF_PHOTOM_CAT
My proposal is to not simulate any of them now and add what we can.
E.g. we can probably add ILLUM_RAW and STDFIELD_RAW by just simulating the same cluster as for SCIENCE_RAW. That would technically work to test our skeleton, but doesn't make sense from a science perspective.
I (HB) am not so sure about the *_HDR data items (FITS files with all information in the headers, no pixels/tables) because we have not properly defined what these should look like. I mean, there is a data structure defined in the DRLD, but these will certainly change once we start implementing it. OTOH, it should be pretty easy to create these data items so maybe I should.
I (HB) am even less sure about the *_CAT data items (FITS files with tables, no pixels), because a) we have not very well defined what these contents should be, and b) I have no idea how to ensure that the contents would actually match the simulations. (Maybe it should be the other way around, @ycao and @buddel define there reference catalogs and use those as input to ScopeSIM when simulating STDFIELD_RAW and ILLUM_RAW.)
Out of Scope
------------
Proposal by HB for what is out of scope for our first data delivery:
* Standard Imaging Processed
* Persistence correction frames
* Astrometric Imaging
* Everything
* High Contrast Imaging
* Everything
Process / Actions
=================
Proposed process to use and associated actions:
(I've added names to this, but I think it should be a collaborative effort.)
* @ycao decides on the exact scope (maybe by editing the simulation script directly?):
* Which data to simulate and process?
* How many, which DIT and NDIT etc (e.g. for flats and darks)?
* What dither pattern (e.g. for the SCIENCE_RAW)?
* What sources / effects.
* @buddel:
* Create the simulation script.
* Run the simulations and store in the archive.
* @kdleschinski
* Fix anything that is broken / not supported in ScopeSIM (nothing essential in v0.1.4 to run the skeleton I think).
Note that at some point HB should get out of the loop, so we can use this first-simulated-data-delivery as a project to facilitate this handover.https://gitlab.astro-wise.org/micado/micadowise/-/issues/242Improve build.sh2023-02-22T09:00:38ZHugo Buddelmeijerhugo@buddelmeijer.nlImprove build.shbuild.sh needs to be improved
- [ ] document the build.sh test
- [ ] automatically use database engine = filebased for build.sh
- [x] make auto deletion an option
- [ ] if build.sh generates new files, is that picked up by the git diff ...build.sh needs to be improved
- [ ] document the build.sh test
- [ ] automatically use database engine = filebased for build.sh
- [x] make auto deletion an option
- [ ] if build.sh generates new files, is that picked up by the git diff script? does it cause problems?
- [ ] perhaps use `parallel` for the generation of the imageshttps://gitlab.astro-wise.org/micado/micadowise/-/issues/241Propagate attributes2023-02-22T09:00:35ZHugo Buddelmeijerhugo@buddelmeijer.nlPropagate attributesSome attributes should be propagated:
- INSTNAME == MICADO
- FILTNAME
- date obs / MJD-OBS / tpl startSome attributes should be propagated:
- INSTNAME == MICADO
- FILTNAME
- date obs / MJD-OBS / tpl starthttps://gitlab.astro-wise.org/micado/micadowise/-/issues/239Action plan for MicadoWISE usage on Jupyter Hub2023-02-22T09:04:19ZHugo Buddelmeijerhugo@buddelmeijer.nlAction plan for MicadoWISE usage on Jupyter HubThis is an umbrella issue to track what is necessary to make most use of our OmegaCEN Jupyter Hub, in particular w.r.t. MICADO.
* Jupyter Hub: https://practicum.astro.rug.nl/
* Instructions to add a user: https://gitlab.astro-wise.org/m...This is an umbrella issue to track what is necessary to make most use of our OmegaCEN Jupyter Hub, in particular w.r.t. MICADO.
* Jupyter Hub: https://practicum.astro.rug.nl/
* Instructions to add a user: https://gitlab.astro-wise.org/micado/micadowise/-/wikis/New-user
The main goal is that the MICADO DFS team (e.g. Yixian, Ric) can use the Jupyter Hub to
* simulate data
* commit and store data into the archive
* search and retrieve data from the archive
* process data
* do some simple analysis, like quick look and trend plots
To make this possible we have a multistage approach:
- [X] Run what we have now: simulate data and process it.
- [ ] Retrieve data from the archive #238 , and process that.
- [ ] Store data into the archive. This should work and mainly requires
- testing,
- writing down how to set credentials
- actually include it in the notebook, at least optionally
- [ ] Create WISE-recipes for MICADO
- preferably this will be a single recipe that can create any data product, like the ProcessTargetRecipe, that just sets dependencies, retrieves data, and calls `make()`
- first run this locally
- then on lpu
- then on dpu
- [ ] Run actual CPL recipes within these recipes.
- [ ] Create notebooks for some actual use cases of the MICADO use cases document.https://gitlab.astro-wise.org/micado/micadowise/-/issues/237Make NWA-MICADO jupyter notebook2023-02-22T08:58:35ZGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlMake NWA-MICADO jupyter notebookThis is a placeholder. See https://github.com/AstarVienna/ScopeSim/issues/23 for everything.This is a placeholder. See https://github.com/AstarVienna/ScopeSim/issues/23 for everything.Gijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/230Make it easy to create simulation archive.2023-02-22T09:00:29ZHugo Buddelmeijerhugo@buddelmeijer.nlMake it easy to create simulation archive.There now is a simulation directory (#202) but more is necessary.
- [ ] The installation instructions should add `cx_oracle` `instantclient`
- [ ] Apparently it is necessary to have a `micado/config/startup.py` ?
- [ ] currently `contex...There now is a simulation directory (#202) but more is necessary.
- [ ] The installation instructions should add `cx_oracle` `instantclient`
- [ ] Apparently it is necessary to have a `micado/config/startup.py` ?
- [ ] currently `context` needs to be imported before a dataitem otherwise it is filebased
- [ ] default project and privileges should be set (`SIM` and 2 or so), which can be overruled
- [ ] simulated data should be tagged, see #221
- [ ] there should be instructions on how to setup Environment.cfg with database credentials
- [ ] downloads do not work on the dbviewer #231https://gitlab.astro-wise.org/micado/micadowise/-/issues/226Coordinate ScopeSIM interface to MicadoWISE and for absolute astrometry usecase2023-02-22T09:02:02ZGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlCoordinate ScopeSIM interface to MicadoWISE and for absolute astrometry usecaseMaintained here: https://github.com/AstarVienna/ScopeSim/issues/23 and here https://github.com/AstarVienna/ScopeSim/pull/165
Topics to discuss with Kieran:
- ScopeSim ToDo list from AWG (add my NWA-MICADO ScopeSIM wishes to that)
- simu...Maintained here: https://github.com/AstarVienna/ScopeSim/issues/23 and here https://github.com/AstarVienna/ScopeSim/pull/165
Topics to discuss with Kieran:
- ScopeSim ToDo list from AWG (add my NWA-MICADO ScopeSIM wishes to that)
- simulations for NWA-MICADO project
- consolidate Digital Design part in hands of Kieran (see DPS-ICS todo list with task for Hugo)
- consolidate MicadoWISE - ScopeSIM interface upgrades (talk to Hugo about this)
- #13+
- #15+
- #211+
- Dithered observations should be possible https://github.com/AstarVienna/ScopeSim/issues/31
- #175+
- https://github.com/AstarVienna/ScopeSim/issues/68
- #115+
- #53+
- Ensure FPA parameters are up to date https://github.com/AstarVienna/irdb/issues/28
- Allow HCI simulations https://github.com/AstarVienna/ScopeSim/issues/32
- Persistence should be simulated https://github.com/AstarVienna/ScopeSim/issues/33
- from Carmelo?: Just to list a few issues we were facing using SimCADO:
- 1.Sub-pixel and interpolation issues
- 2.SimCADO model vs GalFit model difference at the r=0
- 3.Single-Multi Threading pythonGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlGijs Verdoes Kleijng.a.verdoes.kleijn@rug.nlhttps://gitlab.astro-wise.org/micado/micadowise/-/issues/223RecipeSteps / DRL Functions perhaps should not be a DBObject2022-01-21T16:40:01ZHugo Buddelmeijerhugo@buddelmeijer.nlRecipeSteps / DRL Functions perhaps should not be a DBObjectIn retrospect it was probably not so useful to make RecipeSteps a DBObject. It doesn't do that much harm, but it creates a large number of tables that are never filled.In retrospect it was probably not so useful to make RecipeSteps a DBObject. It doesn't do that much harm, but it creates a large number of tables that are never filled.https://gitlab.astro-wise.org/micado/micadowise/-/issues/215QC parameters should usually be per detector2022-01-11T12:20:42ZHugo Buddelmeijerhugo@buddelmeijer.nlQC parameters should usually be per detectorMany QC parameters are currently designed for the full image, but they usually make more sense for each detector. See also #193Many QC parameters are currently designed for the full image, but they usually make more sense for each detector. See also #193