Swift Data Pipelines - Jason Larsen

May 23, 2015 ☼ Swift

The forward pipe operator is an infix operator taken from languages like F# and Elixir. I’ve been working on a Swift implementation that will help clean up data processing pipelines.

In order to understand the motivation for the forward pipe operator, let’s first look at some regular looking code.

let students = Database.allStudents()
let grades = Grades.forStudents(students)
let curvedGrades = curve(grades, average: 0.6)
let finalGrades = prepareGrades(curvedGrades)

This code is simple and has clear variable naming. However, even with clear variable names, it may still be difficult for a reader to immediately realize that each variable declaration’s real purpose is only to hold a value to be used in the next line. These temporary variables introduce noise that may obfuscate the simple flow of data. So why all the temporary variables? Well, the alternative is this:

prepareGrades(curve(Grades.forStudents(Database.allStudents()), average: 0.6))

Clearly, the temporary variables are the lesser evil.

The . operator allows us to chain class and instance methods together in a clean an consistent manner, but it does not help when we want to chain together method calls between distinct entities or stand-alone functions.

Meet the Forward Pipe Operator

The forward pipe simply takes the left hand side of the expression and applies it as the first argument of the function on the right.

increment(1) // becomes
1 |> increment

divide(5, 10) // becomes
5 |> divide(10)

map(1...100, increment) // becomes
1...100 |> map(increment)

That’s nothing special in and of itself—all it has done is provide a new way to call methods and functions. But look at how it lets us rewrite our original example:

let finalGrades = Database.allStudents()
                  |> Grades.forStudents
                  |> curve(average: 0.6)
                  |> prepareGrades

This is much cleaner. The forward pipe expresses the series of function calls as a clear data pipeline; it reads like a sequential list of operations to be performed.

It does’t matter if you’re chaining instance methods, class methods, or functions—the forward pipe chains them all together.

Optional Pipelines

We’ve seen that the forward pipe can clean up our data pipelines, but one of the stumbling blocks in Swift is learning to deal with one of it’s best features: optionals.

Optionals show up almost everywhere in your swift code. Want to use a Dictionary? You’re going to have to deal with optionals.

let animalNoiseMap = ["cow":"moo", "dog":"woof", "cat":"meow"]

animalNoiseMap["dog"]?.uppercaseString // .Some("WOOF")
animalNoiseMap["fox"]?.uppercaseString // .None

The ? operator is great for chaining together instance methods, but once again as programmers we work with more than just instance methods. Want to count the number of vowels in the animal sounds above?

let numberVowelsFox: Int?
if let noise = animalNoiseMap["fox"] {
    numberVowelsFox = count(filter(noise, isVowel))
}
else {
    numberVowelsFox = nil
}

Yikes. That if-let is a lot of work. We could eliminate the else if we wanted to use var insted of let, but then we’d lose some safety. If you understand map you can clean up your code significantly:

map(animalNoiseMap["fox"]) { count(filter($0, isVowel)) }

But not everyone groks functors.

Luckily, Pipes makes dealing with optionals in your pipelines easy. In fact, you don’t need to anything—just write the same code you would have written without Optional.

animalNoiseMap["fox"]
|> filter(isVowel)
|> count // whole expression evaluates to Int?

I might be biased, but that’s pretty wizard.

Result Pipelines

But the forward pipe operator doesn’t stop there, folks! Enjoy using Result? Just like Optionals, Pipes supports Result. (specifically antitypical/Result) Don’t care about Result? That’s ok, you don’t have to link it in for Pipes to work.

func escapeInput(string: String) -> String { ... }

func readFile(fileName: String) -> Result<String> { ... }

func processText(string: String) -> String { ... }

let processedText = inputFileName
                    |> escapeInput
                    |> readFile
                    |> processText

Once again, you don’t even have to think about Results as you chain these functions together—if result shows up along the way, the whole epxression will evaluate to a result. And if something fails along the way, it will short circuit and you’ll get your .Failure with the appropriate error message. Neato.

Keep Your Pipes Clean

Pipes lends itself very well to functional programming ideas. Ideally, keep all the functions and methods in the data processing pipeline pure—i.e., without side effects. Pipes has a bunch of pure implementations and helper functions for Array and Dictionary so you can perform pure versions of append, extend, remove and the the likes.

OK. I Lied a Little.

I’m actually using some currying to add a little readability to my examples. Au naturel, Pipes expects functions that take more than one argument to provide a tuple on the right hand side where the first item is the function, and the rest are the ordered arguments.

// with the trick
5 |> divide(10)
// without the trick
5 |> (divide, 10)

When you use your own functions with more than one argument you’ll need to either use a tuple on the right hand side, or create a function signature where the first argument is curried on the end of the function signature

func divide(numerator: Double, denominator: Double) -> Double {
    return numerator / denominator
}

// the trick
func divide(denominator: Double)(numerator: Double) -> Double {
    return numerator / denominator
}

Essentially currying like this turns our functions into functions that take only one argument, so they don’t need to be put inside a tuple on the right hand side.

Pipes has such curried versions of many standard functions bundled in the framework for your convenience.

Conclusion