One of the new features of IDL 8.4 that I've been digging
is the Lambda() function. While
there are some good examples in the help docs, I thought I'd share a few of my
real world examples and some lessons learned.
One of the simpler examples I had is to take a Hash that I
know has all string keys and create a List containing them all in uppercase
form. There are a couple ways to accomplish this. The pre-8.4 way to do this
requires a few lines of code:
h = Hash('foo', 1, 'bar', 2, 'baz', 3)
keys = h.Keys()
upperArray = StrArr(keys.Count())
foreach key, keys, i do upperArray[i] =
StrUpCase(key)
upperKeys = List(upperArray,
/EXTRACT)
Not too bad,
but with the static methods introduced in IDL 8.4 we can do better, sorta:
h = Hash('foo', 1, 'bar', 2, 'baz', 3)
keys = h.Keys()
keyArray = keys.ToArray()
upperArray
= keyArray.ToUpper()
upperKeys
= List(upperArray, /EXTRACT)
At least
this is using the ToUpper()
static method to vectorize the transform of the entire array in one line
instead of using the foreach loop on each element. But with Lambda functions
and the List::Map () method we can do even better:
h = Hash('foo', 1, 'bar', 2, 'baz', 3)
keys = h.Keys()
upperKeys
= keys.Map(Lambda(str:
str.ToUpper()))
The Lambda
function is applied to each element of the List, and its output is used as the
corresponding element in the output List. In this case the Lambda function
takes the input List element and calls the ToUpper() static method on it. Care
does have to be taken that the source List contains only strings, or bad things
ensue:
IDL> h = Hash('foo', 1, 2, 'bar', 'baz', 3)
IDL> keys = h.Keys()
IDL> upperKeys = keys.Map(Lambda(str: str.ToUpper()))
% Attempt to call undefined method: 'IDL_INT::TOUPPER'.
%
Execution halted at: $MAIN$
Here the
Hash has an integer key and the Lambda function tried to call ToUpper() on an
IDL_Int variable, not an IDL_String variable.
Let's move
on to a pair of Lambda functions I created when working on ENVITask and the
ability of SetProperty and GetProperty to disambiguate _REF_EXTRA keywords
against the list of ENVITask class properties and parameters owned by the
task. The first Lambda function is used to identify all List elements that
start with a given string, and the second is used to identify all List elements
that are initial substrings of a given string. I'll start with a specific input
string, and then generalize it to work with any string. Here is the Lambda
function that can be used to find all strings in a List that start with the
substring 'ENVI':
IDL> substringMatch = Lambda(listElem: listElem.StartsWith('ENVI'))
IDL> l = List('ENVIRaster', 'Gaussian', 'ENVIVector', 'Laplacian')
IDL> l2 = l.Filter(substringMatch)
IDL> l2
[
"ENVIRaster",
"ENVIVector"
]
The Lambda
function uses the new static method StartsWith()
to return a Boolean state of whether the current List element does or does not
start with the string 'ENVI'. We could have used the /FOLD_CASE keyword in the
StartsWith() call to make the string compare case insensitive.
Here is the
Lambda function that can be used to find all strings in a List that are
substrings of the string 'ENVIRaster':
IDL> startsWithSubstring = Lambda(listElem: ('ENVIRaster').StartsWith(listElem))
IDL> l = List('EN', 'ENVIRas', 'ENVIVector', 'ENVIRaster', 'ENVIRasterSpatialRef')
IDL> l2 = l.Filter(startsWithSubstring)
IDL> l2
[
"EN",
"ENVIRas",
"ENVIRaster"
]
Here the
Lambda function uses the trick of putting the string literal inside parentheses
so that it can invoke the static method on it.
This is all
well and good, but if I had a number of different strings I wanted to compare
against I'd have to write new Lambda function definitions for each one. Instead I want to have a parameterized Lambda function that uses a variable
instead of a string literal for the search string. We can't just add a second
parameter to the Lambda function definition however:
IDL> substringMatch = Lambda(listElem, substring: listElem.StartsWith(substring))
IDL> l = List('ENVIRaster', 'Gaussian', 'ENVIVector', 'Laplacian')
IDL> l2 = l.Filter(substringMatch('ENVI'))
% IDL_STRING::STARTSWITH: String expression required in this
context: SUBSTRING.
% Execution halted at: IDL$LAMBDAF4 1 <Command Input
Line>
%
$MAIN$
The Lambda()
function will create a valid lambda function for you, but it doesn't work as
expected, and definitely doesn't work in the List::Filter() method. The
problem is that the occurrence of 'substring' before the semicolon is treated
as an input variable, so it expects the variable to be defined when it invokes
the StartsWith() static method. Also you can't treat the substringMatch
variable like a function of one variable, passing in the string value you want,
it was defined with two parameters. But there is hope, as the Lambda function
help demonstrates in its "More Examples" section you can create a
Lambda function that in turn generates another Lambda function with the string
literals you want plugged in. Here is the generalized substring matching
Lambda function in action:
IDL> substringMatch = Lambda('subString: Lambda("listElem:
listElem.StartsWith(''"+subString+"'')")')
IDL> l = List('ENVIRaster', 'Gaussian', 'ENVIVector', 'Laplacian')
IDL> l2 = l.Filter(substringMatch('ENVI'))
IDL> l2
[
"ENVIRaster",
"ENVIVector"
]
This new
Lambda looks messy, but it can be broken down and analyzed. The first thing we
notice is that that the outer Lambda is passed in a single string literal, not
unquoted code. This is needed for the nested Lambda syntax to work. Now let's
look at what that string is when printed to the console:
IDL> print, 'subString: Lambda("listElem:
listElem.StartsWith(''"+subString+"'')")'
subString:
Lambda("listElem: listElem.StartsWith('"+subString+"')")
The outer
Lambda function takes its input parameter, which I named 'subString', and
passes that into another call to Lambda(). This inner Lambda function
concatenates three strings, two string literals that use double quotes and the
subString value. We'll also note that the two string literals include a single
quote so that the value of the subString variable is turned into a string
literal when the inner Lambda function is evaluated. Looking back at the
original declaration of the outer Lambda, we see a single quoted string literal
with some double quoted string literals inside, each of which include the
single quote character, which can be escaped as a pair of single quotes. A
little messy, but it works when sit down and pull it apart.
Here then is
the generalized superstring matching Lambda function in action:
IDL> startsWithSubstring = Lambda('superString:
Lambda("listElem:
(''"+superString+"'').StartsWith(listElem)")')
IDL> l = List('EN', 'ENVIRas', 'ENVIVector', 'ENVIRaster', 'ENVIRasterSpatialRef')
IDL> l2 = l.Filter(startsWithSubstring('ENVIRaster'))
IDL> l2
[
"EN",
"ENVIRas",
"ENVIRaster"
]
A similar
analysis as above will prove how this nested Lambda function works.
There are
two big caveats in using this nested Lambda function concept in your code. First, if you want to play with them in the console window, you're initially going
to get weird errors like this:
IDL> substringMatch = Lambda('subString: Lambda("listElem:
listElem.StartsWith(''"+subString+"'')")')
IDL> l = List('ENVIRaster', 'Gaussian', 'ENVIVector', 'Laplacian')
IDL> l2 = l.Filter(substringMatch('ENVI'))
% Type conversion error: Unable to convert given STRING to Long64.
%
Detected at: $MAIN$
The reason
for this is that the interpreter will think that the substringMatch variable is
a string, not a function, so it thinks you are using the old school parenthesis
array indexing syntax. The error message is rather cryptic in this regard, but
that is what it means. You can fix this by simply typing compile_opt idl2 in
the console window, then rerunning the l2 line of code. This shows how the
compile_opt syntax can be used to change the behavior of the main interpreter
session, not just inside routines. It also demonstrates that you need to make sure
you include compile_opt in any routines where you use Lambda functions.
The second
caveat is that if you want to use nested Lambda functions like this in code
that you are compiling into save files, then you need to do a little extra
work. If you try to use the code above, compile it and then call RESOLVE_ALL
before calling SAVE, then you'll get error messages like this:
Attempt to call undefined function: 'STARTSWITHSUBSTRING'.
Attempt
to call undefined function: 'SUBSTRINGMATCH'.
The solution
to this is to add any variable names like this that you use to hold the nested
Lambda function values to an array that you pass into the SKIP_ROUTINES keyword
in RESOLVE_ALL. Once you do this, the save file will build and it should work
properly when the code is executed.