TAGGING AT SCALE
At Musiio, one of the fundamental tasks of our technology is to automate the herculean task of analyzing massive amounts of music rapidly in order to provide catalog owners with metadata for all of their songs (Genre, BPM, Mood, Instrumentation, and so on).
In this blog, we’ll give a broad overview of how a startup may achieve a similar scale of operations using stateless servers, microservices, and Kubernetes, in order to outsource peak capacity and enable reliable horizontal scaling of our infrastructure.
MANUAL VS AUTOMATION
When done manually, an expert might tag 100 songs per day at 90% accuracy (5 minutes per track in a 100% productive 8-hour day – we’re being generous). However, increasing that speed would drastically lower the accuracy, rendering the ‘increased productivity’ essentially cosmetic.
Recently, we scaled our infrastructure to handle the task of tagging 5 million tracks per day reliably, with a consistent average accuracy of 90% and higher across all classifiers.
That’s at least 25 Terabytes of MP3s (on average 5mb each) passing through our infrastructure on a single day, with an output of 75 million metadata tags!
Let’s explore some of the methods we used.
PROBLEM SOLVING WORKFLOW
We did not jump immediately from manual tagging to our scalable cloud-based architecture. Here are the steps we took:
Section 1: SERVER HOSTED APPLICATION
Hosting an application on the cloud reduces the burden of infrastructure maintenance. A small tech team can then focus its expertise on optimising said application.
Section 2: STATELESS SERVERS
The cost of cloud environments can easily leave you with a high monthly bill. It is crucial to understand the usage pattern of your applications and run your applications only when required.
Enter stateless servers. Stateless servers enable you to scale up your applications only when there is traffic, and then scale it down to zero when there is none. This can massively reduce your cloud costs and is one of the primary things to consider after migrating your applications to the cloud.
Section 3: MICROSERVICES
As time goes on, we keep adding features to our applications. At some stage our applications get so large it’s extremely hard to debug any issues that might arise. This is the perfect time to transition from a monolith application to microservice architecture.
Microservices can be thought of as small components of a larger application that can be isolated from each other. These components can then be independently developed, scaled and maintained. The major prerequisite would be to define the communication pattern between these services. This enables developers in a small team to work independently on the different components flexibly, keeping your projects chugging along without bottlenecks.
Section 4: KUBERNETES
Kubernetes is the most popular way to implement microservice architecture in the cloud. It lets you easily define the infrastructure requirements of applications and deploy them to the cloud. Other important features include autoscaling in the event of traffic pattern changes, load balancing requests to a set of applications, and attaching shared volumes to our applications.
Section 5: OUTSOURCING PEAK CAPACITY
Let’s say this happens: Your company grows bigger, and you have a few big clients whose requirements in terms of server load are far more resource-intensive.
In an on-premise infrastructure set up, dynamic scaling is hard to achieve as you would need to purchase new hardware to deal with each large increase in traffic. In contrast, on the cloud, a service provider will increase your resource limits within one or two working days after you request it. It is much faster and you will not have to worry about the costs of setting up and maintaining the new hardware.
Section 6: HORIZONTAL SCALING VS VERTICAL SCALING
There are two ways to scale a service, horizontal and vertical scaling. Horizontal scaling involves increasing the number of instances of your application to handle additional workload.
For example, if 2 instances of your application can handle 4 requests, you would scale it to 4 instances to handle 8 requests. Kubernetes enables you to do this automatically by allowing you to determine and set scaling criteria. This removes the need to create custom scaling services for simpler backend architectures.
Vertical scaling involves increasing the resources given to an application to handle more workload. For example, in a multiprocessing application, you can handle processing 5 songs at a time with 3 cpus. By increasing the cpus to 9 you would be able to process 15 songs in parallel. One of the reasons you would use this method over horizontal scaling is slow start up time of adding another instance to your workload.
Generally, a mixture of both horizontal and vertical scaling is used to handle the desired workload. Start with vertical scaling to give a particular instance enough resources(eg:. cpu/memory) to achieve the desired performance. Then add horizontal scaling so you don’t have a single point of failure if that application instance dies.
CONCLUSION
For startups who need an infrastructure that flexibly scales in size and cost with the demands of revenue-generating work, a cloud-based solution is essential. This overview should acquaint you with the key concepts needed to build a scalable system.