HLA-1150: MVM failure due to HAPProduct constructor mismatch #1693
Conversation
Codecov ReportAttention:
📢 Thoughts on this report? Let us know!. |
|
As of now, the code is creating MVMs with filenames e.g. hst_skycell-p1749x18y06_wfpc2_pc_empty_aperture_f502n_all_drz.fits Looking for ways to remove the empty_aperture keyword |
…//github.com/s-goldman/drizzlepac into HLA-1150_MVM-Failure_from_log_level_11_01_23
|
It looks like the MVM output drz filename is based on the SkyCellProduct.info string. While we had previously added aperture to this string, it isn't needed for MVMs and so it has been removed from only the info string for SkyCellProduct objects. This appears to have fixed the naming issue. |
s-goldman
changed the title
| ) | ||
| input_table.add_column(add_col, index=7) | ||
| poller_dtype += ("str",) | ||
| cols["aperture"] = ["empty_aperture"] * len(usable_datasets) |
There was a problem hiding this comment.
It is possible for cols["aperture"] Line 1331 to have more rows than are remaining in usable_datasets which will ultimately cause a failure.
There was a problem hiding this comment.
It shoudn't? Are tests still coming back with uneven indices?
There was a problem hiding this comment.
I was able to find the edge case that you were alluding to: Full poller table with datasets that are not usable. I altered the analyze.py module to return indices (as well as usable_datasets) to retrieve the corresponding aperture values from the input_table/poller_file.
|
Code appears to run as expected and is passing jenkins regression tests |
…1695) Co-authored-by: Steve Goldman <[email protected]>
Resolves HLA-1150
Closes #1692
This PR addresses a failure in dlhstdmcv2 HST Auto-run for 10/31/23 where the following error was thrown:
TypeError: init() missing 1 required positional argument: 'log_level'This appears linked HLA-1110 (#1683) likely due to aperture not being added to the Product Objects for MVM processing. A test should be created to pick up these kinds of errors in the future.
The first commit shows the relevant changes for this PR, he second commit is the result of blacking poller_utils.py
Additional issues were found in the process of fixing these bugs. The code reduces the number of files to process using the analyze module and produces a new list called usable_datasets. This resulted in a uneven match of the new aperture column and the list that is passed through the code: cols. The new code now returns indices along with the usable_dataset, in order to select the correct aperture values to pass.