Add qFSP boundary mask to RSLC#246
Conversation
bhawkins
left a comment
There was a problem hiding this comment.
Only got part of the way through this PR in our meeting today. Comments so far:
| Dictionary specifying convergence parameters for geo2rdr solver. | ||
| Keys among {"tol_height", "time_start", "time_start"} | ||
| See isce3.geometry.geo2rdr_bracket | ||
| method: str |
There was a problem hiding this comment.
| method: str | |
| method: str, optional |
| See isce3.geometry.geo2rdr_bracket | ||
| method: str | ||
| LUT2d interpolation method | ||
| b_error: bool |
There was a problem hiding this comment.
| b_error: bool | |
| b_error: bool, optional |
| log.warning("Internal calibration (INT_CAL) file was not provided " | ||
| "so unable to populate inputDataExceptionMask") |
There was a problem hiding this comment.
Need to think about the interaction with EAP on/off.
| }) | ||
| .def("eval", py::overload_cast<const double, const double>(&LUT2d<T>::eval, py::const_)) | ||
| .def("eval", py::overload_cast<double,const Eigen::Ref<const Eigen::VectorXd>&>(&LUT2d<T>::eval, py::const_)) | ||
| .def("eval", py::overload_cast<double,const const_vec_t&>(&LUT2d<T>::eval, py::const_)) |
There was a problem hiding this comment.
duplicated const?
| .def("eval", py::overload_cast<double,const const_vec_t&>(&LUT2d<T>::eval, py::const_)) | |
| .def("eval", py::overload_cast<double, const_vec_t&>(&LUT2d<T>::eval, py::const_)) |
| method: str | ||
| LUT2d interpolation method | ||
| b_error: bool | ||
| Enable LUT2d bounds error checking |
There was a problem hiding this comment.
| Enable LUT2d bounds error checking | |
| Enable LUT2d bounds error checking. Defaults to True. |
| Keys among {"tol_height", "time_start", "time_start"} | ||
| See isce3.geometry.geo2rdr_bracket | ||
| method: str | ||
| LUT2d interpolation method |
There was a problem hiding this comment.
| LUT2d interpolation method | |
| LUT2d interpolation method. Defaults to "bilinear". |
| """ | ||
| NISAR data anomaly codes (bit flags) | ||
| """ | ||
| NO_ANOMALY = 0 |
There was a problem hiding this comment.
FYI, L0B processor is using the same integer values for the same purpose for the first 6 fields but with more concise names (see below). Given our L0B process does not process S-SAR, it does not need to cover new fields. However, for the sake of consistency between names, we should perhaps request a change to L-SAR L0B to use the same dataclass. Here is the similar class in the current L0B process:
class QfspOutlierId(Flag):
"""NISAR L-band qFSP outlier identifier w/ bitwise operation"""
NA = 0
H0 = auto()
H1 = auto()
H2 = auto()
V0 = auto()
V1 = auto()
V2 = auto()Btw, for bitwise enum.Flag, the auto() will automatically generates the power of 2 values in the ascending order starting from 2^0.
| # There can be multiple intervals associated with each anomaly. | ||
| Boundaries = dict[AnomalyCode, Sequence[ELAngleInterval]] | ||
|
|
||
| def get_qfsp_mask_boundaries(anomaly_code: Union[AnomalyCode, int], |
There was a problem hiding this comment.
As discussed in the meeting, you can use Antenna pattern HDF5 product whom used to generate AC table in instrument HDF5. The respective antenna parser will have some very useful methods. For instance, to get EL angle of the peak for all beams and their common AZ boresight angle, AntennaParser has this method handy:
el_peaks, az_boresight = AntennaParser(hdf5file).locate_beams_peak(pol)FYI, regardless of the EL angle resolution of the patterns, the actual peak locations are locally and precisely estimated in this antenna method. AC table have a EL angle resolution of 50 mdeg.
| SLIP_QFSP_V0 = 1 << 3 | ||
| SLIP_QFSP_V1 = 1 << 4 | ||
| SLIP_QFSP_V2 = 1 << 5 | ||
| SLIP_SSAR = 1 << 6 |
There was a problem hiding this comment.
Not sure why we need to use S-SARor even RESERVED given they are not properly being handled.
| if len(boundaries) == 0: | ||
| for block_start in range(0, nt, block_size): | ||
| block = slice(block_start, min(block_start + block_size, nt)) | ||
| nb = block.stop - block.start | ||
| buf = np.zeros((nb, dataset.shape[1]), dataset.dtype) | ||
| buf[...] = AnomalyCode.NO_ANOMALY.value | ||
| write_block(buf, np.s_[block, :]) |
There was a problem hiding this comment.
This can be any of three possibilities rather than NO_ANOMALY.
I would use anomaly_code as it is being passed by L0B process which for L-SAR it will automatically be 0 when there is no qFSP issue while it preserves the value for any other issue down the line to be handled accordingly in the future. Otherwise, I don't see the point of defining any other anomalies than qFSP in the respective enum.
That being said:
| if len(boundaries) == 0: | |
| for block_start in range(0, nt, block_size): | |
| block = slice(block_start, min(block_start + block_size, nt)) | |
| nb = block.stop - block.start | |
| buf = np.zeros((nb, dataset.shape[1]), dataset.dtype) | |
| buf[...] = AnomalyCode.NO_ANOMALY.value | |
| write_block(buf, np.s_[block, :]) | |
| if len(boundaries) == 0: | |
| for block_start in range(0, nt, block_size): | |
| block = slice(block_start, min(block_start + block_size, nt)) | |
| nb = block.stop - block.start | |
| buf = np.zeros((nb, dataset.shape[1]), dataset.dtype) | |
| buf[...] = AnomalyCode(anomaly_code).value | |
| write_block(buf, np.s_[block, :]) |
| (time, range). | ||
| int_cal : InstrumentParser | ||
| NISAR LSAR INT_CAL file containing the angle-to-coefficient (AC) tables. | ||
| """ |
There was a problem hiding this comment.
| """ | |
| Notes | |
| ------- | |
| This function generates invalid mask simply for 12-channel L-SAR product with qFSP sample slip anomaly. | |
| """ |
| def get_qfsp_mask_boundaries(anomaly_code: Union[AnomalyCode, int], | ||
| int_cal: InstrumentParser) -> Boundaries: | ||
| """ | ||
| Determine EL angle intervals associated with NISAR anomaly codes. |
There was a problem hiding this comment.
| Determine EL angle intervals associated with NISAR anomaly codes. | |
| Determine EL angle intervals covering qFSP transition regions of 12-channel (three qFSPs) L-band NISAR associated with qFSP sample slip anomaly codes. |
| peak_angles[rxpol] = np.array([angles[i, j] for (i, j) in | ||
| enumerate(peak_indices)]) | ||
|
|
||
| # (start, end) EL angles between beam x and y peaks |
There was a problem hiding this comment.
Perhaps it is worth checking the size of the arrays prior to slicing assuming 12 channels.
| # (start, end) EL angles between beam x and y peaks | |
| # (start, end) EL angles between beam x and y peaks | |
| if (peak_angles['H'].size != 12 or peak_angles['V'].size != 12): | |
| raise ValueError(f'Number of channels for H/V is {peak_angles['H'].size}/{peak_angles['V'].size} instead of 12 expected for L-SAR NISAR!') |
| del opts["dtype"] | ||
| mask = slc.create_anomaly_mask(frequency, shape=og.shape, **opts) | ||
| if instparser is not None: | ||
| log.info(f"Writing inputDataExceptionMask for frequency{frequency}") |
There was a problem hiding this comment.
Perhaps one can check for EAP and if it is OFF issue a warning that the mask values can be invalid due to EAP=OFF.
I suggest we simply generate a qFSP sample-slip mask when EAP=True which also guarantees that both antenna and instrument HDF5 are available. In case of EAP=False, a warning is issued w/ empty dataset.
| block_end = min(block_start + block_size, nt) | ||
| nb = block_end - block_start | ||
| # Allocate each chunk to avoid HDF5 I/O as much as possible. | ||
| mask_chunk = np.zeros((nb, nr), dataset.dtype) |
There was a problem hiding this comment.
| mask_chunk = np.zeros((nb, nr), dataset.dtype) | |
| mask_chunk = np.full(fill_value=AnomalyCode.NO_ANOMLAY.value, shape=(nb, nr), dtype=dataset.dtype) |
This PR implements the
inputDataExceptionMaskin RSLC products for the LSAR qFSP-related exception codes. It handles all six possible qFSP exception codes. For now, the strategy to is locate the peaks of the AC table weights in the EL angle domain and use the EL-angle LUT to generate the mask. This mask is in the raw data domain, so it's resampled to the RSLC grid using reskew LUTs. (These could be used to reskew other tables in some future PR, too). I added a vectorizedLUT2d::evaloverload to make this faster.Eyeballing the plots for the case I tested on, I'm not sure the mask covers everything where there's obvious residual phase. So maybe instead of defining the overlap region as peak-to-peak, we should be more conservative and use a -6 dB threshold or something. I hope folks can do some InSAR tests to help figure out the best approach.
Note that the mask is populated based on the contents of the input L0B
hasInputDataExceptionflag, so that must be populated to see any effect.Another question is what to do about mixed-mode frames where one L0B has a qFSP slip and its neighbor doesn't. For now, the qFSP overlap region is marked invalid for the entirety of the output product.