Skip to content

Add qFSP boundary mask to RSLC#246

Draft
bhawkins wants to merge 12 commits intoisce-framework:developfrom
bhawkins:rslc_add_qfsp_mask
Draft

Add qFSP boundary mask to RSLC#246
bhawkins wants to merge 12 commits intoisce-framework:developfrom
bhawkins:rslc_add_qfsp_mask

Conversation

@bhawkins
Copy link
Copy Markdown
Contributor

This PR implements the inputDataExceptionMask in RSLC products for the LSAR qFSP-related exception codes. It handles all six possible qFSP exception codes. For now, the strategy to is locate the peaks of the AC table weights in the EL angle domain and use the EL-angle LUT to generate the mask. This mask is in the raw data domain, so it's resampled to the RSLC grid using reskew LUTs. (These could be used to reskew other tables in some future PR, too). I added a vectorized LUT2d::eval overload to make this faster.

Eyeballing the plots for the case I tested on, I'm not sure the mask covers everything where there's obvious residual phase. So maybe instead of defining the overlap region as peak-to-peak, we should be more conservative and use a -6 dB threshold or something. I hope folks can do some InSAR tests to help figure out the best approach.

Note that the mask is populated based on the contents of the input L0B hasInputDataException flag, so that must be populated to see any effect.

Another question is what to do about mixed-mode frames where one L0B has a qFSP slip and its neighbor doesn't. For now, the qFSP overlap region is marked invalid for the entirety of the output product.

@bhawkins bhawkins requested review from hfattahi and rad-eng-59 March 30, 2026 19:51
Copy link
Copy Markdown
Contributor Author

@bhawkins bhawkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only got part of the way through this PR in our meeting today. Comments so far:

Dictionary specifying convergence parameters for geo2rdr solver.
Keys among {"tol_height", "time_start", "time_start"}
See isce3.geometry.geo2rdr_bracket
method: str
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
method: str
method: str, optional

See isce3.geometry.geo2rdr_bracket
method: str
LUT2d interpolation method
b_error: bool
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
b_error: bool
b_error: bool, optional

Comment on lines +1871 to +1872
log.warning("Internal calibration (INT_CAL) file was not provided "
"so unable to populate inputDataExceptionMask")
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to think about the interaction with EAP on/off.

})
.def("eval", py::overload_cast<const double, const double>(&LUT2d<T>::eval, py::const_))
.def("eval", py::overload_cast<double,const Eigen::Ref<const Eigen::VectorXd>&>(&LUT2d<T>::eval, py::const_))
.def("eval", py::overload_cast<double,const const_vec_t&>(&LUT2d<T>::eval, py::const_))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicated const?

Suggested change
.def("eval", py::overload_cast<double,const const_vec_t&>(&LUT2d<T>::eval, py::const_))
.def("eval", py::overload_cast<double, const_vec_t&>(&LUT2d<T>::eval, py::const_))

method: str
LUT2d interpolation method
b_error: bool
Enable LUT2d bounds error checking
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Enable LUT2d bounds error checking
Enable LUT2d bounds error checking. Defaults to True.

Keys among {"tol_height", "time_start", "time_start"}
See isce3.geometry.geo2rdr_bracket
method: str
LUT2d interpolation method
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
LUT2d interpolation method
LUT2d interpolation method. Defaults to "bilinear".

"""
NISAR data anomaly codes (bit flags)
"""
NO_ANOMALY = 0
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, L0B processor is using the same integer values for the same purpose for the first 6 fields but with more concise names (see below). Given our L0B process does not process S-SAR, it does not need to cover new fields. However, for the sake of consistency between names, we should perhaps request a change to L-SAR L0B to use the same dataclass. Here is the similar class in the current L0B process:

class QfspOutlierId(Flag):
    """NISAR L-band qFSP outlier identifier w/ bitwise operation"""
    NA = 0
    H0 = auto()
    H1 = auto()
    H2 = auto()
    V0 = auto()
    V1 = auto()
    V2 = auto()

Btw, for bitwise enum.Flag, the auto() will automatically generates the power of 2 values in the ascending order starting from 2^0.

# There can be multiple intervals associated with each anomaly.
Boundaries = dict[AnomalyCode, Sequence[ELAngleInterval]]

def get_qfsp_mask_boundaries(anomaly_code: Union[AnomalyCode, int],
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the meeting, you can use Antenna pattern HDF5 product whom used to generate AC table in instrument HDF5. The respective antenna parser will have some very useful methods. For instance, to get EL angle of the peak for all beams and their common AZ boresight angle, AntennaParser has this method handy:

el_peaks, az_boresight = AntennaParser(hdf5file).locate_beams_peak(pol)

FYI, regardless of the EL angle resolution of the patterns, the actual peak locations are locally and precisely estimated in this antenna method. AC table have a EL angle resolution of 50 mdeg.

SLIP_QFSP_V0 = 1 << 3
SLIP_QFSP_V1 = 1 << 4
SLIP_QFSP_V2 = 1 << 5
SLIP_SSAR = 1 << 6
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why we need to use S-SARor even RESERVED given they are not properly being handled.

Comment on lines +133 to +139
if len(boundaries) == 0:
for block_start in range(0, nt, block_size):
block = slice(block_start, min(block_start + block_size, nt))
nb = block.stop - block.start
buf = np.zeros((nb, dataset.shape[1]), dataset.dtype)
buf[...] = AnomalyCode.NO_ANOMALY.value
write_block(buf, np.s_[block, :])
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be any of three possibilities rather than NO_ANOMALY.
I would use anomaly_code as it is being passed by L0B process which for L-SAR it will automatically be 0 when there is no qFSP issue while it preserves the value for any other issue down the line to be handled accordingly in the future. Otherwise, I don't see the point of defining any other anomalies than qFSP in the respective enum.
That being said:

Suggested change
if len(boundaries) == 0:
for block_start in range(0, nt, block_size):
block = slice(block_start, min(block_start + block_size, nt))
nb = block.stop - block.start
buf = np.zeros((nb, dataset.shape[1]), dataset.dtype)
buf[...] = AnomalyCode.NO_ANOMALY.value
write_block(buf, np.s_[block, :])
if len(boundaries) == 0:
for block_start in range(0, nt, block_size):
block = slice(block_start, min(block_start + block_size, nt))
nb = block.stop - block.start
buf = np.zeros((nb, dataset.shape[1]), dataset.dtype)
buf[...] = AnomalyCode(anomaly_code).value
write_block(buf, np.s_[block, :])

(time, range).
int_cal : InstrumentParser
NISAR LSAR INT_CAL file containing the angle-to-coefficient (AC) tables.
"""
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""
Notes
-------
This function generates invalid mask simply for 12-channel L-SAR product with qFSP sample slip anomaly.
"""

def get_qfsp_mask_boundaries(anomaly_code: Union[AnomalyCode, int],
int_cal: InstrumentParser) -> Boundaries:
"""
Determine EL angle intervals associated with NISAR anomaly codes.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Determine EL angle intervals associated with NISAR anomaly codes.
Determine EL angle intervals covering qFSP transition regions of 12-channel (three qFSPs) L-band NISAR associated with qFSP sample slip anomaly codes.

peak_angles[rxpol] = np.array([angles[i, j] for (i, j) in
enumerate(peak_indices)])

# (start, end) EL angles between beam x and y peaks
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it is worth checking the size of the arrays prior to slicing assuming 12 channels.

Suggested change
# (start, end) EL angles between beam x and y peaks
# (start, end) EL angles between beam x and y peaks
if (peak_angles['H'].size != 12 or peak_angles['V'].size != 12):
raise ValueError(f'Number of channels for H/V is {peak_angles['H'].size}/{peak_angles['V'].size} instead of 12 expected for L-SAR NISAR!')

del opts["dtype"]
mask = slc.create_anomaly_mask(frequency, shape=og.shape, **opts)
if instparser is not None:
log.info(f"Writing inputDataExceptionMask for frequency{frequency}")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps one can check for EAP and if it is OFF issue a warning that the mask values can be invalid due to EAP=OFF.
I suggest we simply generate a qFSP sample-slip mask when EAP=True which also guarantees that both antenna and instrument HDF5 are available. In case of EAP=False, a warning is issued w/ empty dataset.

block_end = min(block_start + block_size, nt)
nb = block_end - block_start
# Allocate each chunk to avoid HDF5 I/O as much as possible.
mask_chunk = np.zeros((nb, nr), dataset.dtype)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
mask_chunk = np.zeros((nb, nr), dataset.dtype)
mask_chunk = np.full(fill_value=AnomalyCode.NO_ANOMLAY.value, shape=(nb, nr), dtype=dataset.dtype)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants