Uneven detection threshold in HAP catalogs

[stscijgbot-hstdp]

Issue HLA-1407 was created on JIRA by Rick White:

When there are shifts between images that result in uneven exposure times across the image, the HAP segment and point catalogs both look incorrect.

The HAP segment catalogs appear to use a deeper detection threshold on the edges than in the center of the images, where there are more exposures. The opposite ought to be true: since the exposure time is greater in the center, there should be more sources detected there.

On the other hand, the HAP point catalogs have no sources at all in the edge regions. The point catalogs only include sources in the region where all the exposures overlap.

A test confirms that the problem still exists in the current version of the HAP pipeline.

There must be one or more bugs in the calculation of the noise in different image regions. And apparently the point catalog only finds sources in a restricted region.

[stscijgbot-hstdp]

Comment by Rick White on JIRA:

A more detailed discussion of the issue is on an innerspace page. Some information from that page is included here.

A good example visit is hst_12939_3j_wfc3_ir_iby03j. This WFC3/IR visit has 2 exposures each in the f110w and f160w filters. There are significant offsets among the 4 exposures (presumably to help with removing bad pixels in the detector). Here is a link to the interactive display using the total image (which is the sum of the two filters).

Note that there are no cosmic rays (CRs) in this image because this is wfc3/ir. That makes this a good example because there is no question about additional CRs in the edge regions. (I do see this problem for other instruments too.)

The other reason this is a good example is that the shifts between the exposures are quite large and are easily visible in the image and catalogs (see figures below).

The 3 images below show the total image, the point catalog, and the segment catalog.
!hst_12939_3j_wfc3_ir_total_iby03j.png|width=31%! !hst_12939_3j_wfc3_ir_total_iby03j-pt.png|width=31%! !hst_12939_3j_wfc3_ir_total_iby03j-seg.png|width=31%!

The large shifts between the exposures produce the stair-steps at the corners. The point catalog distribution avoids the edges completely. Some gaps near the edges are expected since the catalog software flags sources that are close to the edge. But the edge region is clearly much wider than necessary.

The segment catalog on the other hand is very sparse in the center (it finds many fewer sources than the point catalog in the same region), and instead finds many more sources on the edges. That is certainly incorrect.

Michele De La Pena processed this visit with the current version of the HAP software and found that the resulting catalogs look pretty similar to these. More info will be posted when it is available.

[stscijgbot-hstdp]

Comment by Rick White on JIRA:

Here are images of the newly processed catalogs from Michele De La Pena using the latest release, CALDP-2024.10.2-4+enchilada.

!hst_12939_3j_wfc3_ir_total_iby03j-pt-new.png|width=48%! !hst_12939_3j_wfc3_ir_total_iby03j-seg-new.png|width=48%!

The number of sources has increased in the new catalog (due to overall changes in the detection threshold). But the patterns for both the point and segment catalogs are the same as the older software version.

[stscijgbot-hstdp]

Comment by Rick White on JIRA:

Michele De La Pena
I just made updates to my innerspace page. I have figured out what causes this problem and include code to fix them. There are actually two bugs.

I think this should make big improvements in both the point and segment catalogs. As a bonus, I believe this will lead to much better segment catalogs for ACS/SBC images. I had noticed some issues with those, and think they trace back to one of these bugs.

Let me know if you have any questions!