Functional Scala: Closures

Welcome to the fourth episode of Functional Scala!

This time we’ll take a closer look at those all-around highly discussed Closures and try to clear some misunderstandings about it, since there are so many discussions out there, confusing simple Functions with Closures. In the first place, Closures are simply Functions featuring some special characteristics (which we’ll gonna talking about) turning them into, well … Closures.

As we’ve all gathered some profound knowledge about the definition and some of the characteristics of common Functions within the first three episodes, let’s not waste any more time and take a look at another example of a Function, which is intended to filter a list of given integer values by the first element within that list: all values less than the first element should be included within the result. Admitted, this Function isn’t that meaningful (and only works properly for non-empty lists), but it will serve as a good entrance into the world of Closures:

val belowFirst  =  ( xs : List[Int] )  =>  {

    val first = xs( 0 )

    val isBelow  =  ( y : Int )  =>   y < first

    for(  x <- xs;  if(  isBelow( x )  )  yield x
}

...
belowFirst( List( 5, 1, 7, 4, 9, 11, 3 ) )      // => List( 1, 4, 3 )

Wow, hold on, chap! What’s going on here? In fact, there are some new features within that example, so let’s try to understand the Functions inner machinery line by line:

Nothing new should turn up at line 3: it’s simply an expression where we refer to the first element of the given list by introducing an appropriate value declaration first. The type of that value must be an Int, since its an element out of a List of integer values (we leave that thrilling job to the compilers type inference capabilities). We’re now able to refer to the first element by using that alias name as the story continues …

Ok, one value declaration under the belt, another one at line 5. And now take a closer look. Can you spot the type of that value? Yep, it’s simply another (local) Function. We shouldn’t have any problems with that, since we already know, that a Function is an ordinary (first class) value and therefore nothing more, but also nothing less than any other values (like our value first of type Int). If you’re familiar with Haskell, you may think of that nested Function as defined within a where clause.  Well, it turns out, that this local Function qualifies as our Closure, we’ll take a closer look in a moment.

In order to bring our inspection to an proper end, let’s take a look at line 7. What we have here is a so called list comprehension (we’re going to focus within another episode). For now, think of it as visiting every element within the ingoing list xs and decide which one may go into an outgoing list (as the resulting value of the whole comprehension expression), if it gets past that if-guard. Since that list comprehension represents the last expression within our function, it represents also the result of the Function.

Open Terms

You may became somewhat suspicious while you inspected that nested local Function. Let’s take it out of its context and take a closer look in isolation:

...
val isBelow  =  ( y : Int )  =>   y < first
...

Didn’t we say, that a Function calculates its result solely in terms of its arguments? If so, what’s about that variable first (you know, not that kind of mutable variable as we think of it in an imperative language)? The only declared argument is y, so the use of y inside the Functions body is bound to that argument. And while there isn’t any other one, the use of first within the Function body is clearly not bound to an argument! Well, whenever you detect such a variable that isn’t declared as one of the Functions arguments nor introduced locally – congrats,  you’ve found what it’s said to be a free variable.

And the story goes on: A Function containing at least one free variable is called an Open Term. In order to get to a fully functional Function, all free variables have to be bound (turning that open term into a closed term). For this to be done, the compiler reaches out to the so called lexical environment, in which that Function was defined and tries to find a binding. This process is called closing over which results in a closed expression or – for short – a Closure.

In the above example, the scope in which the local Function was defined is the nearest local part of the lexical environment. It’s the body of the surrounding Function. And within that scope, the free variable can be successfully bound to our value declaration of first, refering to the first element within the given list. But what if the free variable couldn’t have been bound to a value within the body of the surrounding Function? Well, than the Binding process wouldn’t have been completed. In this case, the compiler needs to expand the search for an appropriate Binding. But where to look for next?  The arguments of the surrounding Function belong to the lexical environment, too. So a free variable within a closure could also be bound to an argument (which gives rise to some powerful abstractions, as we will see in some future posts. I have to mention the power of Closures especially in conjunction with higher order Functions here, since i’ve always mentioned higher order Functions so far). If there’s still no successful Binding available, the compiler closes over to the scope, in which the surrounding function was defined itself. So the following scenario would be completely legal:

val offset = 3
...
val below  =  ( barrier :Int, xs :List[Int] )   =>  {

    val isBelow  =  ( y : Int )  =>   y  <  barrier + offset

    for(  x <- xs;  if(  isBelow( x )  )  yield x
}

...
below( 5, List( 5, 1, 7, 4, 9, 11, 3 ) )      // => List( 5, 1, 7, 4, 3 )

Ok, here we have two free variables barrier and offset within Function isBelow. And while barrier is bound to the first argument of the surrounding Function, the compiler has to close over to the outer scope in order to find a valid Binding for offset. All in all, not to scary any more – just a simple ordinary Closure!

Binding – values vs. variables

All is fine in functional world if a free variable binds to an immutable value. Since that value can’t change between multiple calls of the Closure, the Function can’t result into different values (given the same arguments), at least for the part which depends on the bound variable. And of course the other way round also holds: the Closure itself can’t change the bound variable and therefore can’t produce any side effect. So all in all, even a Closure is a pure Function if all of the values it operates on are immutable (or if there’s generally no idea of value assignment within the language).

But what about Scala? Since Scala offers mutable variables and therefore also value (re-)assignment of such variables, it depends on whether Scala provides a kind of static Binding, where free variables bind directly to an (immutable) value or rather a kind of dynamic Typing, where free variables bind to relative locations in memory that can store values (and the values in the bound location might change). Well, we can bring Scalas behaviour to light by starting a simple experiment:

var minAge = 18

val isAdult  =  ( age :Int )  =>   age >= minAge

val isGermanAdult = isAdult( 20 )   // => true

minAge = 21

val isUsAdult = isAdult( 20 )       // => false

I think you’re already in the position to clearly see what’s going on! Function isAdult seems to be a Closure since it contains a free variable minAge. The free variable is bound to the variable within the surrounding environment. Note, that minAge is declared as a mutable variable instead of an immutable value! And now take a closer look at line 7, where the value of that variable gets changed. It’s exactly between the two calls to our Closure isAdult which is applied to the same argumends twice.  Rats! Our Function results into different values, based on the value change of the bound variable between the Function calls.

Now guess the kind of Binding. It must be dynamic, since a Closure behaves differently due to variable changes. The Binding doesn’t refer to the value, given at the time of Function definition but instead to the underlying location in memory. Again, Scala bite us due to the fact that it’s an hybrid object (imperative) functional language and therefore also allows for imperative constructs. Pertaining to the construction and usage of Closures, this means that we might walk into the trap again and produce some impure Functions.

Quintessence

So what’s the quintessence this time?
We saw that a Closure is some special kind of Function which refers to some free variables, bound to values within the surrounding lexical sope of the Function. ‘Well, then?’ you might wonder and ask: ‘Beside the fact that i now understand that most discussions essentially revolve around Functions in general, only partly about Closures’ what’s the big deal? I can’t give you a fully statisfying answer at this time. As you will see in some further episodes, Closures play a fundamental part in many different areas, ranging from simple state representation up to Monads (now you should wake up and pick notice, since Monads are so fancy these days and you might still impress your team members and friends).

And there’s another one, we discovered several times now: while Scala provides great freedom to go either an imperative or functional way, you might get into problems if you mix them up. As soon as you introduce state to your Functions or the environment in which your Functions are living in,  as soon there’s a risk to end up with impure Functions. That’s what we should keep in mind: Of course, you might take advantage of all the options which may come with the blend of object oriented and functional concepts, but with greater freedom, there’s always also demand for greater discipline …

About these ads
Posted in Scala. 24 Comments »

24 Responses to “Functional Scala: Closures”

  1. Heiko Says:

    In line three of the first listing xs(0) will throw an exception for the empty list; better add a comment or use headOption.

  2. Lutz Hankewitz Says:

    Nice clarification of closures. I also like the deeper view into the compiler’s search for the binding of free variables.

    May I suggest to switch age and barrier in line 3 of the second listing?

  3. Lutz Hankewitz Says:

    sorry I meant the forth listing.

  4. JHerber Says:

    This illustrates an issue with “hybrid” languages. If the language is missing features necessary to state compile-time proofs about that what we believe to be pure, then the hybrid nature of the language actually harms our ability to confidently engineer the benefits of FP into the software with that language.

  5. Twitted by scalaBin Says:

    [...] This post was Twitted by scalaBin [...]

  6. tech blog Says:

    thanks for this post really helps me a lot buddy

  7. bet365 Says:

    hello I was luck to search your blog in digg
    your topic is exceptional
    I get much in your website really thanks very much
    btw the theme of you website is really splendid
    where can find it

  8. Alexander Semenov Says:

    Mario, thank you for the awesome Scala articles series. One question: what if a function belongs to an object and accesses that object’s state variables (vals or vars). Will this kind of function be a closure?

    • Mario Gleichmann Says:

      Alexander,

      first of all thanks for your kind and encouraging words!

      Yep, that function would still be called a closure since it still includes a free variable, leaving the function as an open term. Hence, also in this case ‘Closing over’ takes place where the nearest surroundig scope of that function is the object it’s defined within.

      The tricky thing in Scala is, that you’re free to declare variables beside of values. In the case that a closure refers to a mutable value it’s not free of side effects and therefore isn’t pure any more.

      Greetings

      Mario

      • Alexander Semenov Says:

        Great! So that just means that a simple Java getter is a kind of closure?

      • Alexander Semenov Says:

        Replying to myself. No: according to “Programming in Scala”, function must be created at runtime to be a closure.

  9. High, Higher, Higher Order Functions « brain driven development Says:

    [...] it within a local function (we’ve already seen local functions while we’ve focussed on Closures), so we can refer to that local function afterwards within the comprehension’s guard. Again, [...]

  10. World Spinner Says:

    Functional Scala: Closures « brain driven development…

    Here at World Spinner we are debating the same thing……

  11. Ken Shih Says:

    Just a note… there appears to be a missing end-parenthesis in your 1st example on line 7. That is,
    “for( x <- xs; if( isBelow( x ) ) yield x"
    should be
    "for( x <- xs; if( isBelow( x ) )) yield x"

    Thank you for a great series of posts!

  12. Functional Scala: Curried Functions and spicy Methods « brain driven development Says:

    [...] But wait, where come those values like x, y or even radSquare? Since those values aren’t defined as arguments of our resulting function, we have some free variables which only got bound by closing over into the lexical scope where the function is created (that is the body of our function maker, were we’ve calculated the square of the circles radius only once). So the delivered function must be a … tataaa … closure! [...]

  13. Significant Software Development Developments of 2011 | Software Says:

    [...] been one of the touted advantages of Scala (see honorable mention section of this post for more on Scala‘s big year). Programming languages such as Haskell (or Jaskell on the JVM) and LISP (invented [...]

  14. Significant Software Development Developments of 2011 | Says:

    [...] been one of the touted advantages of Scala (see honorable mention section of this post for more on Scala‘s big year). Programming languages such as Haskell (or Jaskell on the JVM) and LISP (invented [...]

  15. Functional Wrappers for legacy APIs | Martin's Blog Says:

    […] reading: Functional Scala: Closures […]

  16. Anonymous Says:

    Great post!!! Thanks,


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 43 other followers

%d bloggers like this: