Devlog

Just a quick post today around the topic of scalability. This is really a broad topic, but I wanted to give sortof a "starting point" or overview of how to create very scalable websites. The newest of my projects requires the system to scale infinitely, so I had to do a lot of research. I found enough articles on the subject but I never really found a checklist of sorts that pointed me in the right direction, everything was so open-ended.

About Scalability

A scalable system is a system that is easily maintainable and can easily grow to accommodate increased usage.

Scalability is not about performance. Though you should aim towards a good performing system, performance and scalability are completely separate topics. Scalability is about maintaining the same level of performance as the usage grows.

Scalability is not about specific technologies (PHP, Ruby, Python, .NET, Java etc). Any language can be used to create a scalable system. In the realm of web applications, a website that is written in a scripting language like Python can be just as fast and scalable as one written in C. Large parts of Yahoo! for example are written in PHP -- and we know that scales. The language makes no difference because the speed of execution is not the trouble with scalable systems, it's the data. We'll come back to this idea in a bit.

Vertical Scaling or "scaling up"

Vertical scaling is achieving better performance by simply adding better hardware. If your system runs well with 1000 users but not so well at 5000 users, then you can simply upgrade your server with the fastest hardware and you're set.

Vertical scaling is easy. You don't need to think about scaling at all when writing your applications because it's essentially always running in one environment, on one server. If things get too busy, you just upgrade your server.

The problem with scaling up is that it is expensive. Low-end hardware is very cheap, but high end hardware is exponentially more expensive. There comes a time where the cost of the hardware is simply too much, or, that the hardware with the required resources simply does not exist.

So, vertical scaling is a type of scaling that you usually get "for free" without any work. But it is not practical if you want to build a site that serves millions.

Summary:

• Better hardware = better performance
• Pros: Easy
• Cons: Expensive; there will always be a technological limit to how powerful a machine you can possibly buy.

Horizontal Scaling or "scaling out"

Scaling out is achieving better performance by adding more servers. Instead of constantly upgrading to more powerful machines, we can buy a bunch of less-powerful machines cheaply. It is much cheaper to buy many commodity servers than it is to buy one really powerful server. And unlike vertical scaling which has a "limit" to how powerful of a machine you can possibly buy, horizontal scaling can go on forever.

Basically instead of 1 really powerful machine with 32GB of memory and 4 processors, you have 8 commodity machines with 1 processor and 4GB of memory. Each machine handles a portion of the load instead of one machine handling it all. When you need more resources, it's as simple as adding a new (and cheap) commodity machine.

The cost of scaling out comes in two new flavors. The first is the cost of administering these new machines. Instead of say 1 monster server you now have to babysit 8 smaller servers. The real-world cost of doing this though is (fortunately) relatively low. Each machine you add to your system should be identical, and so, system administration tasks can be largely automated.

The second cost comes in design difficulty. You have to think a lot more while designing your system to make sure it can scale out. That is, how does your system work accross 8 separate servers?

Summary:

• More hardware = better performance
• Pros: Cheaper; infinitely scalable
• Cons: Harder to design, a bit harder to manage because you have more machines to watch

Database Replication

Database replication is about automatically copying data from one database server to another database server. There is a master database that sends out data to it's slaves who copy it to their own database. The idea is that you can spread load to all of the slaves instead of relying on a single database server to handle the load of every user.

There are few types of database replication, but the most popular is called master-slave replication. This is where you have a single master database, and any number of slave databases. All write queries (queries that modify the database) go to the master. All read queries (SELECT type queries) are spread amongst the slaves. This works well in sites where the number of reads far outweigh the number of writes (which is just about every website online!).

Designing for Database Replication

For master-slave replication, your application must know which server to send queries to. This is fairly simple because it is easy to interpret plain-text SQL and know which kinds of queries are writes (UPDATE, INSERT, REPLACE) and which are reads (SELECT). For example, in your database abstraction layer you could modify your query method to read something like:

public function query($sql)
{
    if (preg_match('#^\s*SELECT#i', $sql)) {
        $conn = $this->slave_connection;
    } else {
        $conn = $this->master_connection;
    }

    // Send query to $conn database
    // ...
}

The other thing you need to do is decide how you will load-balance the slave servers. There are hardware solutions or software solutions. The simplest method, and the one I personally like to use, is to simply choose one at random when your application starts up.

Replication Lag

Replication lag is the time it takes for all slaves to execute the write queries that happened on the master. This is usually very very fast (milliseconds) and thus not usually something to worry about. But if you are overloading your cluster, then the time it takes may increase and your users will start to see the affects. Things like saved changes not appearing when they reload the page.

One potential cause is that the slaves themselves are being overloaded -- they are serving too many read queries so they are falling behind. This can be fixed by adding new slaves to help spread the load some more.

Database Engines

One thing worth noting is that a slave can use a different database engine than its master. For example, your master can use MySQL InnoDB tables and your slaves can use MySQL MyISAM tables.

I mention this because InnoDB is required for transactions, but is much slower than MyISAM. So it seems natural to use InnoDB for the master (write) database and MyISAM for the slave (read) database. It means your slaves can handle more load.

Database Partitioning

Database partitioning is about splitting up your database so that all of the data is spread across multiple databases.

Clustering

Clustering is splitting your database up at the table level. So you have your users, usergroups and profiles on one database, and then you have your forums, topics and replies on another database.

This method is pretty straightforward but it is severely limited. It means you can't join on tables used in a different database, and it adds a certain overhead because each page now requires connections for each cluster. So if you have split your database into 4, one request now needs 4 connections.

Also, each cluster will likely have different requirements. For example, there will likely be many many more replies than there are user profiles. So this makes it more difficult to manage.

Sharding

Sharding is splitting data up at the row level. This means you have a bunch of databases with identical tables, but the data they contain is split up. For example, users 0-10000 are stored on shard 1, 10001-20000 are stored on shard 2 etc.

This method is infinitely scalable because when one database becomes overworked, you just add another shard.

The difficulty comes when you need to join with data that exists on different shards. The best way is to simply ensure that all related data exists together in a single shard -- but this can be pretty difficult. The other way is to denormalize your database so that you don't need to join with other tables.

Database Denormalization

Going against all best-practices with database design -- denormalization basically means copying data from one table into another to eliminate the need for table joins. Joins are fairly expensive, so it is sometimes worth the horrible icky feeling in your stomach when you denormalize.

So as an example, you might have a 'threads' table that has a field 'last_poster_id' that links to a 'users' record. If you wanted to get the username for the last poster, you'd need to join with the 'users' table. But if you denormalize your schema, the 'threads' table would have a new field called 'last_poster_username'.

The downside of denormalization is that you need to make sure all records stay consistent. With the example above, it means that if we change the username for the user we must also change every 'last_poster_username' in the 'threads' table. In situations where data is read many more times than it is written, this is usually okay.

Caching

The single most important aspect to scalable websites is caching. Every website, when you get right down to it, reads data more than it writes data. So, instead of reading data from a database -- which means reading from disk -- you should read data from a memory cache. With very popular websites, reading from disk is complete and utter disaster! You must cache to memory if you want to become the next YouTube.

Cache what? How to cache?

You should, in short, cache everything. The database is only there for persistent storage, and to refresh caches when the caches become stale. So that means your User::getUserinfo() method looks something like:

public function getUserinfo($user_id)
{
    if (cache_exists("user:$user_id")) {
        return cache_get("user:$user_id");
    }

    // Else, you need to fetch the data from the database
    // ...

    // And then cache the result for subsequent requests
    cache_save("user:$user_id", $userinfo);

    return $userinfo;
}

There are many different technologies that you can use to cache data. But the best of them (or at least most widely used) is called memcached. PHP even has a memcached module.

Handling stale data

When you are caching data, you need to be careful that the data in the cache is updated when the data in your database is updated. So this might mean a simple update to your User::save() method for example. Other times, you might add some logic so that cached data is automatically removed after a certain period (so that the next time its requested, it will be fetched and then recached once again).

Where to cache

Above, I mentioned that you should cache to memory. But I should clarify that memory is not the only way to cache things; and might be superfluous in many situations. Indeed, before you even need to think about memory caching you should identify a disk IO bottleneck so you know for sure that a memory cache will even improve the situation.

Before you move to a memory cache, the easiest way to implement a caching system is through files in the filesystem. You might even cache data in the database. The whole point in caching is to reduce load. So, for example, if you cached an entire page in the database all you need to do is fetch one row and you can present it to the user quickly (versus say, 10 queries and HTML-building routines you might normally need to do).

The point I'm trying to make is that caching isn't always about memory. It's about storing data (possibly pre-processed data) in a location that is inexpensive to access -- or at least inexpensive relative to the original source.

It just so happens that memory is the only source for a very popular site to read cache from, because all other disk-bound caches (including files and databases) become too slow.

Separate Components

You should separate components where you can so you can create specialized machines that are very good at certain tasks. For example, Apache is great for serving PHP pages but is really overkill if you need to serve up static CSS/Javascript files. Try adding a new server that runs Lighttpd to serve up those kinds of things. Additionally, don't put your MySQL server on the machine as your memcached server. You get the idea.

In Closing...

So there's a very brief overview of the topics you need to read about. I hope you now understand the sorts of things that go into a scalable website. My goal was to demystify the ideas of scalability so you can better research specifics; and of course in coming days I'll try and talk about specific techniques.