Blog Post |
Functional Programming at LeapYear
Haskell at LeapYear
LeapYear’s technology sits as a core component of our customer’s analytics stack, enabling the use and exchange of the most sensitive data assets in the world. As a result, our customers have an expectation for bullet-proof software. Haskell, the foundational language for core programs at LeapYear, is up to the task, and delivers fundamental correctness guarantees, security, and efficiency.
Haskell is a highly expressive and capable language. Using its advanced type system and rich extensions, a programmer can wield Haskell to precisely specify programs, reducing both the set of possible programs and the chances of specifying an incorrect program. The synthesis of features in Haskell allows us to implement high-complexity code in an efficient manner.
Five years after we made the critical choices on programming languages and technologies, Haskell has proven itself as a language suited for the demanding environment in which we operate. Further, we’ve gained striking benefits from our ability to refactor highly complex code. More fundamentally, Haskell has impacted our thinking as an engineering organization, from the languages and tooling we choose to work with, to the practices we employ outside of Haskell.
In the rest of this piece, I’ll explore both the strengths of the language, and those we learned through experience using Haskell as a primary development language. In a later post I’ll talk about some of the hard-earned lessons about best practices using Haskell.
What is Haskell?
Haskell is a programming language originally designed and developed by a body of functional programming researchers and released in 1990. At the time it served as a basis for programming languages research, mainly for the academic community. Its usage and support has grown considerably since, and Haskell is currently standardized and maintained as the Glasgow Haskell Compiler (GHC) and developed by a body of contributors from both industry and the open-source community.
Haskell is defined by its emphasis on strong, static typing with support for type polymorphism, type classes and families, and higher-kinded types. These capabilities are provided by the type inference engine that makes these features easy to use, usually providing helpful error messages and guidance. Non-strictness and referential transparency are also key to Haskell, which allow developers to ignore aspects of the execution model when it’s convenient, only re-examining execution when meaningful performance benefits are to be gained. Additionally, Haskell makes heavy use of monads which are an essential tool in managing state and side-effects within an application in a consistent way.
Combined, these features make it relatively quick to prototype and develop strongly-typed programs which have fewer bugs, allowing developers to focus on the essential aspects of the program and letting the compiler assist. This provides developers with confidence that they can start and expand projects without worrying about unintended bugs showing up.
How LeapYear Uses Haskell
Haskell sits in the core of our software—a differentially private relational algebra, statistics and ML engine. Correctness and security are essential to our software, and Haskell’s type system allows us to verify program properties even before testing (We believe that type checking is only a supplement and never a substitute for testing). In this way, wide swaths of common bugs can be eliminated because of the strong typing features that Haskell provides.
At the heart of LeapYear are two related types that form the basis of the query language we support. We have an untyped abstract syntax tree (AST, essentially the grammar map of an expression in our language), which encodes incoming queries without respect to correct typing, privacy usage or permissioning. The core system’s job is to take this query, resolve the referents, check the permissions, and ultimately convert this AST into a typed AST that is written using generalized algebraic data types (GADTs) and a few heavier language extensions. Structuring the types this way allows us to have the compiler verify statically the runtime type checking code of our execution engine. Additional AST and typing structure allows us to track privacy properties essential to the LeapYear product. (I gave a talk at LambdaConf ’17 on the subject if you are interested in learning how to write such a type checking system in Haskell).
Additionally, Haskell allows us to take on complex refactoring with confidence. Code-complexity presents high risks to modifiability of code, and managing what is essentially a multi-pass compiler is arguably one of the most complex sorts of programs to manage. The type system has been remarkably helpful as we’ve had to refactor our execution engine to accomplish new features we are adding to the system, whether we are including additional transformations or new statistical algorithms and functions. The central piece of querying logic has been in place and used in LeapYear for over four years, supporting our development roadmap and feature releases.
Perhaps the most transformative, and unintended consequence of selecting Haskell is the impact it has had on our behavior as a team. I consider Haskell to be a mindset. And it’s clear this mindset affects all of the work we do, even when we’re not specifically using Haskell as the programming language. We work on many different front-end interfaces, from python libraries (the most common ML programming environment) to administrative capabilities, and we often choose to implement stronger typing solutions (TypeScript, MyPy) and get many of the basic benefits of Haskell across all of our work. The necessary stress on “correctness” has allowed us to produce high-quality software that our customers feel secure using, knowing it will function as we’ve intended.
Conclusion
While I may not have predicted it at the time, the selection of Haskell five years ago has been critical to our success. Haskell has functioned as a secret weapon for our team, giving us efficiency, core correctness guarantees and most importantly impacting the way we think about programming in every situation. The features of the language have given us supreme confidence in our ability to support and progress our codebase and offer a high-functioning, high-stability, privacy-preserving data analysis platform for our customers.
We are always on the lookout for Haskellers with deep, production code experience. You can refer to our open positions here. If you don’t see a job that applies, but you think you have the right Haskell skills to join our team, email us at careers@leapyear.io.
Related For Developers Posts
VIEW ALL
