ISCE with data from the olden days

Processing legacy data (we're talking ERS, Envisat, JERS, Radarsat) with ISCE is possible, although not well documented. Since the developers are (rightly) more focused on getting the code running well on contemporary data, and developing the new versions that will one day handle data from the NISAR mission, you might need to do a lot of the legwork yourself.

So it proved the other day, when I tried to process some very old JERS data over southern California using stripmapApp. In theory, there are good reasons for wanting to do so – to make use of the improved topographic registration, split-band processing and other goodies built into the stripmapApp workflow, for instance. Plus of course, the software is still supported (unlike legacy codes like ROI_PAC), so if it can be coaxed into working, you would have a working set-up for the next few years at least!

1) The ISCE JERS parser was incomplete

Once upon a time, in a version of ISCE a long way away, there might have been an effort to write a parser for JERS data, Looking in the guts of the Python code, it seems that someone started writing a script to do it, maybe based on the scripts from ROI_PAC, and then gave up halfway through. I tried to run it once, just to see if it would work, and I got an error. Digging into the code a little (at $ISCE_HOME/components/isceobj/Sensor/JERS.py), I found the following:

def extractImage(self): raise NotImplementedError()

...in other words, at the part where the script is supposed to extract the image, it just raises an error that it is not implemented. Oof! Never mind...

2) ISCE didn't read ROI_PAC format metadata properly

There is often more than one way to do something in a code as versatile as ISCE. Since I had processed this pair of images from raw in ROI_PAC, I had the raw data in ROI_PAC format. So all I would have to do was tell it I had the files in that format. Easy!

My stripmapApp.xml file looks a bit like this:

<stripmapApp> <component name="insar"> <property name="sensor name">ROI_PAC</property> <component name="reference"> <property name="RAWFILE"> <value>/path_to_files/19930430/19930430.raw</value> </property> <property name="HDRFILE"> <value>/path_to_files/hdr_data_points_19930430.rsc</value> </property> <property name="OUTPUT"> <value>reference</value> </property> </component> ...

The key, really, is that the ROI_PAC raw file is accompanied by a raw.rsc file that contains the metadata. In this case, the metadata contained the vital information that there was a fairly large Doppler centroid bias in both scenes (about -1.2 PRF, or -1900 Hz in each case). However, it became obvious that this vital information was being ignored, as the interferogram that I ended up with was fuzzy, as if it were processed with the wrong Doppler.

A fuzzy mess

Doing a little detective work, it seems that ISCE had re-estimated the Doppler centroid bias all on its own, as when I consulted the log files, they proudly cited an incorrect value:

reference.frame.doppler_vs_pixel = [-284.0383895212319, -0.08412333288059079, 2.1196022751847877e-06] 

reference.instrument.range_pixel_size = 8.76586134502924

reference.sensor.DOPPLER_RANGE0 = -1.2416044171086 

reference.sensor.DOPPLER_RANGE1 = -4.42229572736922e-05 

reference.sensor.DOPPLER_RANGE2 = 0

reference.sensor.PRF = 1555.2000000

Note that the Doppler value of -284 Hz is approximately 1 PRF away from the 'correct' Doppler of approximately -1900 Hz. And, digging deeper, it seems that ISCE tried to estimate the Doppler itself (using the ROI_PAC-era 'DopIQ' code), rather than use the values from the metadata. Aha!

3) Force ISCE to use the metadata values (and scale them properly)

The 'default Doppler' approach in ISCE uses information from the metadata for its Doppler needs. But it seems that the ROI_PAC data type defaults to something else! You can force it to use the metadata by including the following in your stripmapApp.xml file:

<property name="reference doppler method">useDEFAULT</property> <property name="secondary doppler method">useDEFAULT</property>

This change made, I was intrigued to find that the reported Doppler values increase to very large numbers!

reference.frame.doppler_vs_pixel = [-3003002.848290641, 0.0, 0.0, 0.0] secondary.frame.doppler_vs_pixel = [-2882779.5730339526, 0.0, 0.0, 0.0]

-300,000 Hz is a very big shift! About 1555 times too big, I would say. And would you know it, 1555 Hz is the value of the PRF! So the fix to the problem was to figure out where these values were coming from (in $ISCE_HOME/components/isceobj/Sensor/ROI_PAC.py) and scaling them appropriately by the PRF, up or down. I submitted these changes as a pull request to the ISCE repo on GitHub, so hopefully these changes will propagate through to everyone in due course...

4) Fix alignment and timing issues

One more issue to overcome is the alignment of the two images. After making the necessary changes to focus them properly, it still seems that there were still issues with making an interferogram out of them. Namely, that they were not aligned. Whoops!

This failed interferogram is made from two focused images that were not properly aligned. If you look carefully, you can see double features throughout, suggesting that the misalignment is in the azimuth (y) direction.

Poor alignment is not entirely surprising – it is a matter of timing and orbit information, neither of which was as accurate 30 years ago as more recent missions provide. (We've been spoiled by Sentinel-1 for sure!) It turns out that there is a fine alignment that is done within ISCE that is ultimately used to correct for relative timing biases, and this was not doing the job with this interferogram. The likely reason? The timing bias is probably too large for the standard tolerances in ISCE. We need more 'slop' in the process so that these larger misalignments can be fixed.

It turns out that the place where these things are hard-coded (!) is in a script: $ISCE_HOME/components/isceobj/StripmapProc/runRefineSecondaryTiming.py

By default, the search window size is 40 lines in azimuth. I changed it to 80, which I figured would be big enough to capture the misalignment in radar coordinates (this was a guess, but if you are really keen, you could look at the focused slcs and measure some offsets between the images manually). Having updates and saved the script, I reran the interferogram formation, And, lo and behold! It worked!

Giving the two images a taller search window enabled them to be properly aligned. And now the amplitudes are crisp and the phase is coherent. And there are fringes!

While it is great that it is fixable, and that I know where to make such changes now, I do wonder if this should be an input parameter, rather than something you have to edit in a script. But that might be an issue to fix on another day...

5) I get by with a little help from my friends

I didn't just magically intuit what to do and where to fix things, as much as I would like to think I could. No, I got this far by asking for help, namely by opening an issue on the ISCE repository on Github. Luckily, one of the developers helped me by pointing to some of the places where the problems might arise, and this was enough to inspire a deeper dive into the places where things might be going awry (and then I could grep for the rest!)

So yes, if you have a problem, and nobody else can help, perhaps opening an issue might be the solution. (Assuming that the A-Team can't help you, of course...)