These last two years at Feeld have been quite intense for our tech team. After several rewrites of the app, I joined as CTO with one clear goal: To make Feeld a world-class app. My first step was to migrate the entire codebase to React Native (which probably warrants another blog post), which took around six months and helped us unify two small client teams (Android and iOS) into an entire frontend team, enabling us to move fast.
Still, with just one backend developer, our server infrastructure was beginning to crack under pressure. Thus, we decided to completely revamp our infrastructure and move to Kubernetes. After hiring three new backend developers, we contacted our friends at Jetstack to give us a two-day intensive (and intense) workshop on Kubernetes, and then we began the migration.
Our API used to live in Heroku – a perfect choice for starting up our infrastructure. However, our node.js monolithic codebase had been growing in size, and it was becoming challenging to develop and onboard new developers into it. Furthermore, we were hitting performance concerns, especially at top traffic times. We needed to take action.
We decided to port our entire infrastructure to Kubernetes, and followed a simple approach:
- Move all existing monolithic code to run in Kubernetes.
- Release new clients that could be configured to point to either Heroku or Kubernetes as their API.
- Start sending traffic gradually, and monitor the behaviour.
- Start chipping away at the code by creating new microservices to take on the tasks done initially by the monolith.
We have currently completed the first three points, and if you're using Feeld, you have most likely been hitting our new infrastructure since around March 2019. The transition was hard work, but the results paid off – we are now able to handle more users concurrently, scale the infrastructure as demand grows, and our developers can work in parallel on different microservices.
Our first task was to build a new authentication service, one that did not depend on Facebook. Our users had been asking for an alternative login method a long time, so it was the perfect candidate to be Feeld's first microservice.
Another huge decision we have taken is to move away from node.js as our primary backend language. We settled on Haskell due to its incredible expressivity and bomb-proof type system. We have written our new authentication service entirely in Haskell, and while it has been a steep learning curve for our team, we are confident that we will be able to provide a better service, with fewer errors and outages.
Speaking of outages, it hasn't been all smooth sailing with our new systems. We have experienced a few blackouts, most prominently the one that brought down half the internet. While that one was not our fault, and we were able to quickly redirect traffic to our old servers (the wonders of redundancy), there have been several others that were due to a misconfiguration of our Kubernetes cluster.
But with each bump in the road, our team gets wiser and quicker at solving problems. We have had 99.651% uptime since we started using our new infrastructure, and we have begun using Monzo's incident response system to communicate, solve and log incidents as they occur.
I am proud of the tremendous effort that my team has put in to ensure that we can provide our growing community with a faster and more reliable service, and enable the features they want quicker and more safely.
By the way, did I mention we are hiring?