Neonlabs

Amazon RDS provides Relational Database Scalability

// November 3rd, 2009 // amazon-ec2, cloud computing, mysql, sql

I was very excited to hear that Amazon added a service for Relational Databases for the Cloud. It does make my life easier. One of the most important properties of the Cloud is scalability, and it is one of the most sought properties for companies with huge data flow. Using Amazon EC2 IaaS, makes it easy and flexible to scale your application, only if it is built or architected to be scalable.

There is no free lunch when it comes to scalability in the cloud. In order to take advantage of the flexibility and scalability of the Cloud, you need to architect your application in a way that it can be scaled. What Amazon provides is the infrastructure, but the application must be able to take advantage of that.

Now, this is not simple task, but it is achievable and most EC2 users are doing it now. But scaling the database is not as simple, even if you have the source code, like in the case of MySQL.

For example, let’s think about a SaaS application that is built with fine grained services which can all be scaled using Amazon’s Load Balancer. The application depends on an MySQL Server tuned for high loads sitting on an image in Amazon EC2. The services works fine most of the year, but in December, the load pumps up enough to make MySQL suffer.

Now this app is already in the cloud, so our fine grained services are scaled as needed, but the database, has become our bottleneck. What can we do? Well, there are several common options, which are not specific to the cloud and can be combined:

• Use Sharding or Federation.
• Use Database Replication.
• Use MemCache to improve memory access to data.

Now, everybody knows this works, but it comes with a cost in maintenance;  if it works in an on-premise data center it should work in the cloud. The problem with this approach, is that it doesn’t take full advantage of Cloud Scalability in Amazon EC2. The only way to take advantage of EC2’s scalability would be to actually make MySQL natively scalable, like we did with our application. Which is a huge effort, and sometimes not worth it.

Of course, Amazon reps will tell you to use SimpleDB, which I wouldn’t recommend for any mission-critical work due to the consistency issue. SimpleDB relaxes consistency for the sake of  (limited) performance, and also it doesn’t have great performance with huge datasets, which is the norm in enterprise applications. (For more information on SimpleDB’s pros and cons look here.)

Now with the new Amazon RDS, we can use a relational database that is supported by popular languages/frameworks and that can take advantage of the scalability in the cloud (specifically Amazon EC2). We just create our database using the Amazon RDS API, and point our applciation to it. If we need to scale, we can do it using the API, taking full advantage of EC2. When the data flow lowers again, we can easily release the resources using the API.

How did Amazon accomplish this? Well, it is too early to know. There are not a lot of sources of information regarding Amazon’s RDS architecture. But as it looks right now it seems like an API in front of a set of mysql databases sitting in Amazon’s EC2 images, maybe using a combination of the techniques explained before. What Amazon is aiming to provide with this new service is a database server that is easy to use, maintain and scale. Which I think is a pretty cool.

I will post more information regarding RDS after I play around with it more.

Tweet This Post