Backend For Frontend — Making Hotstar Zip!
Client applications that are faster to load are a delight. At Hotstar, we’re using a pattern we call Backend For Frontend (BFF) to improve our customer journey. This is our journey and the decisions we made along the way.
Hotstar is the home of Indian cricket and this means that we must align our platform to serve the millions of cricket fans who throng to our platform to catch the action. As our services mature, we became acutely aware of all the aggregation that our clients had to do, times the number of our clients, today we’re around 13 clients and adding more all the time.
All this amounts to duplicating behaviour, resiliency and metrics repeatedly, which is a lot of wasted cycles. This was one of the prime drivers when we started to review what we could do differently in our stack. This also lined up well with our goal of improving client load times.
The negative aspects of higher load times(perceived and actual) are well documented. For example: BBC found out that 1sec increase in page load time led to 10% drop in users.
The following graphic shows a very simplified view of the architecture that our clients use to render content to our customers.
Challenges
- One size does not fit all : Irrespective of the hardware device and its capabilities, all client apps would receive the same response payload from the backend services. We felt that the payload for our mobile clients could be trimmed down when compared to say a Web App or a SmartTV app. This became important to control size on the wire and optimize for data needed for specific screen formats.
- Fragmented UI processing on the clients : Most of the UI processing logic resided on the clients. Each client processed the service response according to custom logic. Often times, this would lead to a drift in the way content information was derived at and rendered on the UI. This would at times become a source of mismatch for the information rendered, leading to broken user experience across devices. Also, with a majority of business logic processing occurring on the clients we felt that the perceived page load time could be improved, especially for low end devices, by migrating this processing to a backend service.
- Very chatty clients : Our Personalisation service is a standalone micro-service, separate from the Content micro-service. The personalisation service returns unique identifiers to identify contents that reflect the users preference. The clients would then request the content service for the metadata corresponding to these identifiers. This layout made the client app’s chatty and also contributed significantly to the page load time.
In order to solve the above listed problems, created a middle layer between our clients and our micro-services that would meet the following requirements:
- This layer should make all the multi service calls and compose the entire metadata for the client to consume. This should be done at a very high throughput and at very low latency.
- The technology or framework required to build this layer should facilitate high inclusion of client teams. The client teams should also be able to easily make changes without little to no assistance from the backend services team
- The services built at this layer should be highly scalable.
This layer is our implementation of the Backend For FrontEnd(BFF) pattern. The high level architecture resembles the image below.
One of the key decisions for the implementation was the technology or framework to choose. We wanted that our client team developers be able to contribute to the development and maintenance of this layer without requiring them to invest a lot on learning a new technology or framework.
Also, we wanted to quickly validate our hypotheses of improved page load and app load times by implementing this solution. So, the technology/framework we had to chose needed to be fast, and also easy for anyone to pick-up. To this end, we decided to use NodeJS as we already had in-house expertise on NodeJS. Also, several of the other client app developers had worked with JS earlier and they could also start contributing quickly.
However, the NodeJS ecosystem has a bunch of frameworks — some of the widely used being Hapi, Express, Restify . These all serve as a general purpose web-framework but one of the frameworks that caught the attention of our team, was Fastify. Fastify was blazingly fast compared to the other frameworks available. A benchmark comparison is shown below.
Fastify has the added advantage of providing software re-use and extensibility thanks to its plugin model. Also, the plugin system was quite rich and would meet our requirements of Swagger documentation, CORS enabling, logging , and response compressing.
One of the other offerings of Fastify that we found useful was Validation and Serialization, which was offered out of the box. This allowed us to quickly validate any incoming requests for appropriate headers and query params and also with the fast-json-stringify plugin used internally by Fastify, we noticed a significant improvement while using the JSON serialisation.
In order to avoid the network delay and improve the performance we used DNS Caching.
We wanted to use containerization right from the start. On Android alone, our App is installed on 100Million+ devices. Thus, it was important for the services to be load tested rigorously before pushing to production. We leveraged our load testing infrastructure that sits on Gatling and Flood.io to prove out the service.
Thanks to this exhaustive load testing exercise, we reached at a configuration for both the socket timeouts and the heap memory of the service instance(we currently have a service instance running per container), that would allow each service instance to handle 1k requests per minutes.
We have currently rolled out the BFF service to be used by 10% of our Android app’s. Early results have been fairly encouraging. Page load times have been reduced by 50% compared to the previous implementation.
The Road Ahead
We feel that we can bring additional reduction in the page load times on our client applications by trying out the following:
- Currently the middle-layer BFF uses HTTP 1.1 for all service communication. We believe that HTTP 2, with its support for multiplexing different network requests on the same connection will offer us significant improvements in the speed of aggregating response from multiple different services.
- In our aim to launch the middle layer quickly, we haven’t leveraged the benefits of using node streams. Using node streams would allow each service instance to be more memory efficient and also time efficient.
- Finally, all our communication with the backend micro-services is via CDN. Some time delay is incurred with DNS resolution, rate limiting, and WAF protection. In the future, we could directly communicate with the Backend MicroServices ELB.
- Our implementation is not an exact implementation of the traditional BFF pattern. Instead it functions more as an API Gateway. Also, the client app’s still have the logic for generating the view layer from the metadata our service returns. We want to evolve our service in a direction it transitions more towards the traditional BFF pattern and generates the view for each of the client app’s.
We believe that we have made significant progress and have validated our idea of improving end user experience. As we enter a critical period for Hotstar, with IPL-2019, the Indian General Elections and the Cricket World Cup coming up, there are potential opportunities for us to get more users to have a delightful and hassle free viewing experience on Hotstar — all powered by the new service we built.
We’re hiring! If you are interested in building technological solutions or products, working in cross-functional teams, then do check out tech.hotstar.com. Given the scale at we operate and the impact we have in the Indian OTT domain, there surely is a role that will spike your interest!
A lot of credit for this implementation goes to the synergy and initiative shown by our Product Manager(Aditya Katare), VP Engineering(Kshitij), Client Team(Ankit Agarwal, Ashwini Kumar, Gautham Ballal, Manpreet Singh, Prachi Sharma), CMS Team(Akash Roy Choudhury), Personalisation Team(Jayesh Sidhwani, Sanjay Singh), and DevOps Team(Kaushik Chandrashekhar, Tejhas B, Prakhar Joshi)
(If you are interested in why Fastify is so fast compared to the other frameworks, do checkout the below talk)
