Functional Scala: Closures

Welcome to the fourth episode of Functional Scala!

This time we’ll take a closer look at those all-around highly discussed Closures and try to clear some misunderstandings about it, since there are so many discussions out there, confusing simple Functions with Closures. In the first place, Closures are simply Functions featuring some special characteristics (which we’ll gonna talking about) turning them into, well … Closures.

As we’ve all gathered some profound knowledge about the definition and some of the characteristics of common Functions within the first three episodes, let’s not waste any more time and take a look at another example of a Function, which is intended to filter a list of given integer values by the first element within that list: all values less than the first element should be included within the result. Admitted, this Function isn’t that meaningful (and only works properly for non-empty lists), but it will serve as a good entrance into the world of Closures:

val belowFirst  =  ( xs : List[Int] )  =>  {

    val first = xs( 0 )

    val isBelow  =  ( y : Int )  =>   y < first

    for(  x <- xs;  if(  isBelow( x )  )  yield x
}

...
belowFirst( List( 5, 1, 7, 4, 9, 11, 3 ) )      // => List( 1, 4, 3 )

Wow, hold on, chap! What’s going on here? In fact, there are some new features within that example, so let’s try to understand the Functions inner machinery line by line:

Nothing new should turn up at line 3: it’s simply an expression where we refer to the first element of the given list by introducing an appropriate value declaration first. The type of that value must be an Int, since its an element out of a List of integer values (we leave that thrilling job to the compilers type inference capabilities). We’re now able to refer to the first element by using that alias name as the story continues …

Ok, one value declaration under the belt, another one at line 5. And now take a closer look. Can you spot the type of that value? Yep, it’s simply another (local) Function. We shouldn’t have any problems with that, since we already know, that a Function is an ordinary (first class) value and therefore nothing more, but also nothing less than any other values (like our value first of type Int). If you’re familiar with Haskell, you may think of that nested Function as defined within a where clause.  Well, it turns out, that this local Function qualifies as our Closure, we’ll take a closer look in a moment.

In order to bring our inspection to an proper end, let’s take a look at line 7. What we have here is a so called list comprehension (we’re going to focus within another episode). For now, think of it as visiting every element within the ingoing list xs and decide which one may go into an outgoing list (as the resulting value of the whole comprehension expression), if it gets past that if-guard. Since that list comprehension represents the last expression within our function, it represents also the result of the Function.

Open Terms

You may became somewhat suspicious while you inspected that nested local Function. Let’s take it out of its context and take a closer look in isolation:

...
val isBelow  =  ( y : Int )  =>   y < first
...

Didn’t we say, that a Function calculates its result solely in terms of its arguments? If so, what’s about that variable first (you know, not that kind of mutable variable as we think of it in an imperative language)? The only declared argument is y, so the use of y inside the Functions body is bound to that argument. And while there isn’t any other one, the use of first within the Function body is clearly not bound to an argument! Well, whenever you detect such a variable that isn’t declared as one of the Functions arguments nor introduced locally – congrats,  you’ve found what it’s said to be a free variable.

And the story goes on: A Function containing at least one free variable is called an Open Term. In order to get to a fully functional Function, all free variables have to be bound (turning that open term into a closed term). For this to be done, the compiler reaches out to the so called lexical environment, in which that Function was defined and tries to find a binding. This process is called closing over which results in a closed expression or – for short – a Closure.

In the above example, the scope in which the local Function was defined is the nearest local part of the lexical environment. It’s the body of the surrounding Function. And within that scope, the free variable can be successfully bound to our value declaration of first, refering to the first element within the given list. But what if the free variable couldn’t have been bound to a value within the body of the surrounding Function? Well, than the Binding process wouldn’t have been completed. In this case, the compiler needs to expand the search for an appropriate Binding. But where to look for next?  The arguments of the surrounding Function belong to the lexical environment, too. So a free variable within a closure could also be bound to an argument (which gives rise to some powerful abstractions, as we will see in some future posts. I have to mention the power of Closures especially in conjunction with higher order Functions here, since i’ve always mentioned higher order Functions so far). If there’s still no successful Binding available, the compiler closes over to the scope, in which the surrounding function was defined itself. So the following scenario would be completely legal:

val offset = 3
...
val below  =  ( barrier :Int, xs :List[Int] )   =>  {

    val isBelow  =  ( y : Int )  =>   y  <  barrier + offset

    for(  x <- xs;  if(  isBelow( x )  )  yield x
}

...
below( 5, List( 5, 1, 7, 4, 9, 11, 3 ) )      // => List( 5, 1, 7, 4, 3 )

Ok, here we have two free variables barrier and offset within Function isBelow. And while barrier is bound to the first argument of the surrounding Function, the compiler has to close over to the outer scope in order to find a valid Binding for offset. All in all, not to scary any more – just a simple ordinary Closure!

Binding – values vs. variables

All is fine in functional world if a free variable binds to an immutable value. Since that value can’t change between multiple calls of the Closure, the Function can’t result into different values (given the same arguments), at least for the part which depends on the bound variable. And of course the other way round also holds: the Closure itself can’t change the bound variable and therefore can’t produce any side effect. So all in all, even a Closure is a pure Function if all of the values it operates on are immutable (or if there’s generally no idea of value assignment within the language).

But what about Scala? Since Scala offers mutable variables and therefore also value (re-)assignment of such variables, it depends on whether Scala provides a kind of static Binding, where free variables bind directly to an (immutable) value or rather a kind of dynamic Typing, where free variables bind to relative locations in memory that can store values (and the values in the bound location might change). Well, we can bring Scalas behaviour to light by starting a simple experiment:

var minAge = 18

val isAdult  =  ( age :Int )  =>   age >= minAge

val isGermanAdult = isAdult( 20 )   // => true

minAge = 21

val isUsAdult = isAdult( 20 )       // => false

I think you’re already in the position to clearly see what’s going on! Function isAdult seems to be a Closure since it contains a free variable minAge. The free variable is bound to the variable within the surrounding environment. Note, that minAge is declared as a mutable variable instead of an immutable value! And now take a closer look at line 7, where the value of that variable gets changed. It’s exactly between the two calls to our Closure isAdult which is applied to the same argumends twice.  Rats! Our Function results into different values, based on the value change of the bound variable between the Function calls.

Now guess the kind of Binding. It must be dynamic, since a Closure behaves differently due to variable changes. The Binding doesn’t refer to the value, given at the time of Function definition but instead to the underlying location in memory. Again, Scala bite us due to the fact that it’s an hybrid object (imperative) functional language and therefore also allows for imperative constructs. Pertaining to the construction and usage of Closures, this means that we might walk into the trap again and produce some impure Functions.

Quintessence

So what’s the quintessence this time?
We saw that a Closure is some special kind of Function which refers to some free variables, bound to values within the surrounding lexical sope of the Function. ‘Well, then?’ you might wonder and ask: ‘Beside the fact that i now understand that most discussions essentially revolve around Functions in general, only partly about Closures’ what’s the big deal? I can’t give you a fully statisfying answer at this time. As you will see in some further episodes, Closures play a fundamental part in many different areas, ranging from simple state representation up to Monads (now you should wake up and pick notice, since Monads are so fancy these days and you might still impress your team members and friends).

And there’s another one, we discovered several times now: while Scala provides great freedom to go either an imperative or functional way, you might get into problems if you mix them up. As soon as you introduce state to your Functions or the environment in which your Functions are living in,  as soon there’s a risk to end up with impure Functions. That’s what we should keep in mind: Of course, you might take advantage of all the options which may come with the blend of object oriented and functional concepts, but with greater freedom, there’s always also demand for greater discipline …