Feature Flags: Your Tool for Anxiety-Free Production Releases

This blog post delves into the world of feature flags. We talk about what feature flags are, their implementation at Enova SMB and the benefits of using them.

What is a Feature Flag?

Feature flags (also known as feature toggles or feature switches) are a software development technique that turns certain functionality on and off during runtime, without deploying new code. This allows for better control and more experimentation over the full lifecycle of features.

The idea behind feature flags is to build conditional feature branches into code in order to allow teams to modify system behavior without changing code. If the flag is “on,” new code is executed.  If the flag is “off,” the code is skipped. As such, it can make both very small (e.g. new page color) or very large changes (e.g. new backend process) available to users.

There are two types of feature flags:

Static Feature Flag: A static feature flag gets hardcoded into the app at build time.  You decide during compilation what will be “on” or “off”. Most of the time, it’s part of the config file which can change based on the environment.

Dynamic Feature Flag: A dynamic feature flag, on the other hand, can be changed at runtime. This happens either via a secret settings screen in the debug variant or via a remote tool (e.g. Firebase Remote Config) when the app is in production.

Benefits

There are several benefits of using feature flags.

1. You can test in Production:  Feature flags allow engineering teams to test new features in production, while mitigating the risk of a bad release by providing a way to quickly roll back the feature if necessary via a kill switch. Because it is often impossible to completely simulate the production environment in staging, feature flags allow you to validate the functionality of new feature releases in the real world while minimizing the risk.
2. Faster Release Cycles: Feature flags decouple feature lifecycle management from code deployment, freeing up the engineering team to work on other tasks. Enabling features can be as easy as toggling an on/off switch or editing a config file vs doing a lengthy code push.
3. Canary Releases: Another use case for Feature flags is allowing a team to test a new feature on a subgroup of end users to see how it performs before rolling it out to a broader audience. Canary testing helps to limit the risk of launching a feature to your full user base, and allows you to quickly roll back a feature simply by toggling it on or off, rather than having to go through another deploy cycle.
4. Continuous Deployment:  Another major benefit is their ability to allow quick learning. Feature flagging allows companies to continuously deliver and deploy software to their users in a faster way. The use of feature flags allows companies to perform gradual feature rollouts, fix bugs in the code without redeploying, experience a more streamlined development cycle and do rollbacks of code more easily.
5. A/B Testing: Feature flags allow marketers and other non-engineers to A/B test various features within products or systems. With no need for a code deployment, business and non-technical teams can hypothesize, observe and track feature performance and analytics.

Feature Flags & Continuous Delivery

Advocated by several of the best names in Software Engineering (such as Martin Fowler and Uncle Bob), Continuous Delivery is a software development discipline in which software can be released to production at any time. It has been adopted by leading software companies around the world whose teams are now expected to quickly deliver software that is stable and bug-free to their user base. It’s an idea that has been embraced and popularized by tech companies such as Facebook and LinkedIn, among others. Facebook Gatekeeper and LinkedIn’s Lix Engine are two of the prominent systems that use feature flags. How Facebook tames Complexity with Reversibility can be found here. To know more about LinkedIns Lix Engine, check out this blog post

Feature flagging enhances CI/CD by making “continuous” more achievable. The feature flags and feature toggles are a key component of the implementation of continuous delivery, allowing the separation of feature rollout from code deployment. Incomplete features can be merged into the production codebase but hidden behind feature flags.

Now that we know the usefulness of feature flags and how awesome they can be, let’s see what makes a good feature flag:

1. First and foremost, it must be incredibly easy to add a new feature flag.
2. In terms of feature flag values, we are going to restrict ourselves to just boolean flags. Having binary values keeps things simple, both from a development perspective as well as conceptually: something is either on or off.
3. Finally, we should be able to easily toggle feature flags on/off in automated tests and we don’t want to lock ourselves into a particular framework.

Feature Flags and Enova

CI/CD (Continuous Integration / Continuous Delivery) is the name of the game and we at Enova wholeheartedly embrace innovation. Enova SMB uses feature flags to test and deploy its code. We are able to push features into production/live yet have the feature disabled/toggled until it’s completed. We often use dark releases to show incomplete features to just the dev team. This way we can test partially completed work in production (we ain’t scared!!) over multiple releases/deployments before we ‘turn ON the Flag’ (completed) and it becomes visible to the public.  One of the projects where feature flags were impactful was our recent integration with an online OCR vendor. We modified the process of receiving and sending traditional PDF documents to the OCR service for parsing. This project had multiple moving parts across different applications . We were able to reap the full benefits of feature flags by making continuous deliveries, having multiple dark releases and the ability to test our changes in production with the flexibility of reverting back with just a config change. This was a game changer for us as we were able to test our entire updated Underwriting flow (component-wise and end-to-end) even before we officially released in production. Currently we are using Static Feature Flags. We set our flags in a config file and our code takes the configuration and executes the flow accordingly. Using this approach we were able to improve our Underwriting process by ~8% with efficiency improvements predicted to double up to ~15% as we iterate over our learnings. With Continuous Delivery and feature flag management, our team can launch, control, and measure their features at scale.