The Spice of Life: Enova’s Tech Exchange Program
The day-to-day of software development can get a bit repetitive. Personally I tend to get bored easily — we probably all got into engineering because we like to learn and try new things. At Enova I try to go out of my way to take on work that’s outside of my “normal” duties, and in my experience those tangentially related tasks can be some of the most valuable. For starters, having some variety helps keep you interested in the core functions of your own team.
But more than just keeping me engaged, taking an interest in Enova’s tech stack beyond just my own responsibilities has really helped me grow as an engineer. Working closely with other teams helps me understand the wide variety of technical problems we deal with, and how to solve them. Not to mention that working with those teams also helps them understand my team’s viewpoint, and that shared understanding helps build a stronger engineering team overall.
Of course it can be hard to find enough tangential interactions to build a strong relationship with the wide variety of teams at Enova, but luckily we have a program built specifically for engineers to get that kind of experience.
The Tech Exchange Program
The program is called the “Tech Exchange” in which anyone in Technology gets to spend 4 weeks embedded in a different team of their choice from somewhere in the tech organization. Engineers plan in advance with their manager to be absent, so short of an emergency they won’t do any work with their normal team.
Instead, for the duration of the Tech Exchange, you are a member of a new team, and you join in any work that team would normally do. The program was started organically a couple of years ago when one of our engineers wanted to take some time to help one of our infrastructure teams modernize a platform she was a subject matter expert on. So from day one the goal has been not just for the exchanging engineer to learn from a new team, but for the new team to gain from the perspective of the exchanging engineer.
Exchange #1: Network Engineering
I’ve been lucky enough to do two different tech exchanges at Enova. I normally work in our Software Engineering department, dealing directly with application development, but my first exchange was to one of the infrastructure focused teams in our IT department. I spent four weeks with our Network Engineering team, which manages everything from our load balancers and firewalls all the way down to our routing and network access control configurations.
When I was with them I focused on helping build out programmatic tooling for our production load balancers. On the application side, proper configuration of the load balancers is obviously important to ensure that our APIs serve requests properly, but many of the engineers in SE know very little about how the load balancers actually work. On the infrastructure side, our load balancers have a configuration API but the team didn’t have a lot of experience working with APIs directly, and hadn’t had much time to spend learning, instead focusing mostly on manually making changes as needed. In the time I was there, I was able to work with the Network Engineering team to build out a simple application using the load balancer’s API to monitor the status of nodes in a single traffic pool. This was only a single use case, but the Network Engineering team was able to use it as a starting point for developing further integrations with the load balancer’s API.
And of course, by the end of four weeks I knew a lot more about the design and configuration of our networks than I knew before.
Exchange #2: Problem Management
My second tech exchange was a little different. I decided to spend some time learning about the daily life of one of the teams in our Technology Operations department. For four weeks, I immersed myself in the day to day of the Problem Management team, who deal largely with the the ability of our applications to perform appropriately under a production level work load. A lot of my exchange was spent helping with the capacity planning for our production nodes. In order to ensure that our applications don’t encounter issues in production, we need to check that our web servers and background processes are configured to serve an appropriate level of traffic without overwhelming the resources of the nodes they’re deployed to. The Problem Management team regularly scans our production logs for signs of issues. Queueing requests or background jobs can indicate an underprovisioned application while idle processes being killed means we’re exceeding our node’s capacity for no reason.
I got to investigate some poorly performing applications (including some owned by my team in SE!) and figure out an appropriate configuration for their workload. I learned about some of the tools and techniques for assessing production performance, which I’ve been able to apply to some of the incidents since that time. On top of that, the Problem Management team also got to understand the usage patterns and design motivations of some of our more finicky services, and they’ve been able to adjust their future capacity planning to take those particular needs into account.
Program Best Practices
If you’re interested in reaping the rewards of periodic rotations, I have a few pieces of advice on how to implement such a program effectively.
I think the most important part of the success of the Tech Exchange at Enova is having the exchanging engineers themselves pick a team to rotate to based on genuine interest. While it may be tempting to place engineers on teams based on the perceived benefits of particular cross training, we find the program is most successful when the engineers in question get to select the team and experiences that seems most intriguing to them. The time allocated is short, and willing volunteers are likely to make the most of it.
Speaking of the short time involved, another key part of a successful rotation is planning the move in advance. Everything from the focus and projects of the exchange time, to the winding down of existing projects, to things as mundane as application access and login credentials needs to be planned and in place before the exchange actually starts. Without good advance planning, engineers can easily lose the first week just to getting their plan for the exchange itself in order.
My last recommendation is to ensure that the exchanging engineers are fully immersed in the team they’re rotating to. They should move to a desk in that team’s area, attend that team’s normal meetings (standups, planning meetings, etc.), pair with engineers on that team, and even attend that team’s extra-curricular activities (a team lunch or a happy hour can be a great way to build a personal relationship with a team that you don’t normally work with). Tell your engineers to avoid the temptation of checking in on their normal projects, they’ll have plenty of time for that when the exchange is over.
Working with different teams at Enova has kept me engaged, made me a better engineer, and made our engineering team stronger overall. I’d encourage anyone who can to spend some time in someone else’s shoes. You’ll be surprised how valuable an experience it can be.