Fun with StaMPS and Doris

Stamps, yesterday. (Picture from blogandcollectibles.com)

Although the title may make it sound like I am indulging in philately with an elderly neighbour, this is actually high-tech stuff. For those who are down with the lingo, this is the versatile and freely-available multi-temporal InSAR processing package StaMPS, written by the inestimable Andy Hooper of the University of Leeds. [Doris, on the other hand, is an inscrutable interferogram processor from the Netherlands that I have trouble understanding.] For those who are not down with the lingo, move swiftly along.

With the end of the teaching quarter (wahoo!) I find myself with a little time to indulge in doing science, involving actually looking at some data. [Of course, it shouldn't feel like an indulgence – it is, after all, my job – but after ten weeks of teaching, it seems like a real treat!] And, despite being a long-standing user of the code (one of the original users in fact), I am a little rusty. And so, for posterity's sake, I have documented here the steps it took to get me to where it might actually produce something... [update, 8/6/2016: Forgot to post this at the time, and now my memory of what I actually did is a little hazy. So the second half of this is a little more brief than you might like, blame the me from sometime last summer.]

tl;dr

If you are running StaMPS on a Mac, the FreeBSD version of the 'date' command is sufficiently different that it breaks the script that parses the timing information from the ROI_PAC metadata and writes it out in Doris format. And that messes up everything. See item 11 below for the fix.

Dorises, yesterday. (From Google Image Search)

1) The setup

The hardware: My trusty (but aging) 'cheesegrater' 12-core Mac Pro running OSX 10.10, with 64Gb of RAM and just enough disk space. Of course, the code was developed with the Linux operating system in mind...

The software: StaMPS v.3.3b1 (freshly installed) and Doris 4.02. ROI_PAC 3.0.1. Libraries installed via fink. MATLAB R2014b.

The data: 20 ALOS PALSAR scenes from New Zealand, from 2007–2011. (I have never processed ALOS data before in StaMPS...) A rather excessive SRTM DEM of the whole of South Island with holes filled in by the CGIAR consortium.

2) The phony war

I had to swap in my own existing modification of 'make_raw_alos.pl' that gives files named by their 8-digit dates by default, as indeed StaMPS likes. The regular version didn't seem to want to do that, and so the 'step_slcs_alos' step threw up a load of errors in passing. Once the switch was made, all went swimmingly – a master SLC image was generated, I picked out my area of interest (I used the mdx image viewer that comes with ROI_PAC to find the line and pixel numbers for the bounds), and was able to crop the master SLC image down to the bare essentials.

The next step, 'make_slcs_alos', also ran without complaints. I now had a processed SLC image for every date in my dataset. I was feeling so productive!

3) step_master_timing crashes with an error

FFS.

4) Manually fix the master date and time

After manually editing the date in the 'master.res' file to what I thought was the time it was after (I used the time listed as 'FIRST_CENTER_xxxx' for the hour, minute and second, and the fraction of the second that actually was given in 'master.res'), I actually got 'step_master_timing' to run. This was already a great improvement.

5) Hack the roipac2doris script to use the 'first center' time

I got times in the right format, at least!

6) No images matched

Bah.

7) Make a manual match and re-run Doris

Doesn't help.

8) Try again without step_master_timing

The images don't exactly match well, but they do match.

9) Now a manual timing correction

Crashed, so I ran step_master_timing.

Followed the instructions, but I couldn't see a match.

10) Looked at the DEM match in detail

The shift had hit a bound. Maybe the timing problem wasn't so well fixed after all.

11) Hack the roipac2doris script again (properly, this time)

Right. After some hacking around, swearing at the monitor, reading some man pages, swearing some more, and after a period of intense concentration, finally getting somewhere, I found that two lines containing the BSD 'date' command needed to be edited to account for the new format:

set UTCday = `date -juf "%Y %m %d %H %M %S" "$YEAR $MONTH $DAY 0 0 0" +"%s"` 

set UTC1 =  `date -juf "%s" "$UTCtime" +"%d-%m-%Y %T"`$AZIT1_DS

So, find 'em and edit away.

12) Delete all the SLCs and start again

When in doubt, purge.

13) Find that MATLAB has changed the default save convention

One last bonus crash! You need to make the following changes to account for the fact that new(ish) MATLAB doesn't save files like it used to...

ps_load_initial.m, line 193:

save(phsavename,'ph','-v7.3');

ps_est_gamma_quick.m, line 285:

save(pmname, 'ph_patch', 'K_ps', 'C_ps', 'coh_ps', 'N_opt', 'ph_res', 'step_number', 'ph_grid', 'n_trial_wraps', 'grid_ij', 'grid_size', 'low_pass', 'i_loop', 'ph_weight', 'Nr', 'Nr_max_nz_ix', 'coh_bins', 'coh_ps_save', 'gamma_change_save', '-v7.3') 

And hopefully now it should work. [Although, there are no guarantees that you will see any exciting signals in your data, of course...]