1. High Coupling

All software you write depends on some other software. And parts of your own software depends on other parts of your software. Coupling is how much one part of your software depends on another part. In a highly coupled system, lots of parts all depend on each other. In a lowly coupled system, parts are independent from each other as much as possible.

Why It’s Bad

• Changes made to one part of your system affect other parts of your system. This means making changes becomes difficult and error prone.
• It becomes difficult or impossible to use even functionally unrelated modules separately because they depend on each other so heavily.
• Due to the previous point, it is nearly impossible to write unit tests. This compounds the problem of making changes because you can’t properly test to see if your changes break things in other parts of your system.

Solution

Stop coupling! Write classes that are as separate from each other as you can. This may mean building utility classes, or classes that help “connect” bits of your system. The point is that Part A of your system should be completely separate from Part B of your system.

Further Reading

• Strive for low coupling and high cohesion: What does that even mean?

2. Premature Optimization

We’ve all heard the famous quote by Donald Knuth, the author of The Art of Computer Programming: Premature optimization is the root of all evil.

Developers often worry about the performance of their software before performance should be considered at all.

Why It’s Bad

• A developer who writes code with performance as a first priority will produce code that is hard to use and maintain.
• Code that is “optimized” prematurely rarely ever results in software that actually runs faster. Without proper testing/benchmarking, it is often hard to know which parts of your system is the bottleneck. Saving 15 milliseconds on a calculation routine does nothing for the data-fetching routine that takes 1500 milliseconds.
• Due to the lack of proper testing, developers will spend too much time on menial performance optimizations while missing the important ones.

Solution

Write your code with performance in your mind — but don’t sacrifice usability and maintainability. Write your code first, and then only once you have identified real bottlenecks you can start optimizing.

3. Abusing Inheritance

Inheritance is great, but some developers try to force their software to fit into rigid inheritance trees. Inheritance is a tool, but it’s not the only way to write reusable software.

Why It’s Bad

• Abusing inheritance results in classes that have functionality that they shouldn’t.
• Subclasses make assumptions about their parent classes. This couples children to their parents. As we know, coupling is bad. The deeper our class tree gets, the more coupling, and the worse the design is.
• You end up with huge families of classes. For example: A parent Animal class. Then a Dog child. Then a BlackDog, and a BlackBigDog, and BlackSmallDog, and BlackBigLoudDog and BlackSmallLoudDog… You can see how it quickly gets out of control!
• You tie functionality/algorithms to a specific family of classes, making it harder to reuse. The only way to access such functionality is through inheritance — even when inheritance doesn’t exactly fit the bill.

Solution

The old maxim: Favor composition over inheritance.

Composition adds functionality at runtime — giving you much flexibility. Inheritance forces you to define all functionality at code-time, limiting possibilities.

Composition means your classes are built up (composed) of other classes for additional functionality. This reduces needless children, and decouples functionality into separate reusable classes of their own. Using the example mentioned previously, a Dog class might be made up of separate classes to define it’s appearance (AnimalSize class?) and loudness of it’s bark (AnimalSound class?). An example calls might then be, $dog->getSize()->display() and $dog->getSound()->makeSound().

In other words, you want “HAS A” (composition) relationships rather than “IS A” (inheritance) relationships.

4. Failure to separate responsibilities

Some developers fall into the trap of creating classes that do way too much — so called God Objects. Classes should do one thing only (see the Single responsibility principle).

Why It’s Bad

• Other parts of your system depend too much on a single class, creating very high coupling. Any changes made will ripple out to all it’s dependants.
• It is hard to reuse code written in these classes because they come with so much “baggage”. Instantiating such a monster object wastes resources with all of the unneeded functionality that has been added.

Solution

Refactor. The God Object needs to be properly deconstructed into smaller, more specialized classes that all do a single job.

5. Failure to refactor

Refactoring is the activity of switching around your softwares internal design, while keeping it’s functionality the same. For example, renaming a method is a small example of refactoring. A more intense example might be splitting one class into two separate classes (maybe you’re tearing apart a God Object?).

Why It’s Bad

• As your product grows older and you add new features, classes will start to gain functionality they did not before. Over time, classes may become too big resulting in monster or God Objects.
• As your product grows and you add features, some classes may start to become complex. Without refactoring to simplify the inner workings of your system, this complexity may start to get out of control. For example, a class constructor that takes 20 arguments.
• The longer refactoring is put off, the worse the problem becomes and the harder the task will be in the future.

Solution

Refactor as soon as you encounter smelly code. The earlier you refactor, the easier it will be.

6. Complexity

Code should be simple to read and maintain. Functions that take 20 arguments, or classes that require too many helper objects are examples of complex designs that can be fixed.

Why It’s Bad

• Complex designs are hard to understand and thus hard to maintain.
• Complex designs will result in hard to find bugs.

Solutions

Simplify. If a function takes 20 arguments, maybe some of those arguments can be encapsulated into a class. Or maybe the one function can be replaced by an easier to use class. If a class requires lots of setup, consider using the Builder pattern. If you have a system of classes that all work together towards some goal, consider simplifying the interface by using a Facade.

7. Reinventing The Wheel

Some developers, especially amateurs, like to create everything from scratch and ignore all of the available free library code that exists on the internet.

Why It’s Bad

• You waste time writing components when you could be actually building your product.
• You get none of the free testing done against a popular library. Anything you write from scratch will contain bugs.
• Not always true (but may especially apply to amateurs): Many libraries are written by developers who may be more experienced than you. These coders produce better organized code with more features.
• Not always true: Developers of libraries are often experts in whatever task the library is meant to solve. For example, database abstraction. Because of this, the library developers will produce a better product than you could.

Solutions

Realize that writing everything in-house is almost always a bad thing. There will of course be times where there exists no suitable library. But almost everything has been written before, and written well. A good developer knows the tools at his disposal, and uses them.

Every time you consider writing something yourself, you should search for a good library first. Common difficult tasks such as database abstraction, web services, image manipulation, validation and parsing are examples of components where an abundance of free library code exists. Less well-known problems, or new problems, may not have many pre-built solutions.

8. Failure to apply known design patterns

Design patterns are well known solutions to architectural problems in software. I’ve mentioned a couple in this article already (Builder, Facade).

Why It’s bad

• A developer who doesn’t know design patterns is more likely to design his software incorrectly.
• Not knowing about design patterns will decrease your efficiency. Every problem has been solved before. By knowing which pattern to apply to a problem, half of your work is done for you. Otherwise, you will go through many revisions before you end up at the same solution anyway.
• Design patterns help a developer communicate with other developers. It’s the language of the profession. You will not be able to convey ideas effectively if you don’t understand the terminology.

Solution

Read all material you can find on the topic of design patterns (see next section). Amateur programmers may not appreciate the value of these patterns at first — or, indeed, may not fully understand them — but as time goes on, this knowledge is priceless.

Further Reading

• Head First Design Patterns
• Design Patterns: Elements of Reusable Object-Oriented Software
• Patterns of Enterprise Application Architecture

About Scalability

A scalable system is a system that is easily maintainable and can easily grow to accommodate increased usage.

Scalability is not about performance. Though you should aim towards a good performing system, performance and scalability are completely separate topics. Scalability is about maintaining the same level of performance as the usage grows.

Scalability is not about specific technologies (PHP, Ruby, Python, .NET, Java etc). Any language can be used to create a scalable system. In the realm of web applications, a website that is written in a scripting language like Python can be just as fast and scalable as one written in C. Large parts of Yahoo! for example are written in PHP -- and we know that scales. The language makes no difference because the speed of execution is not the trouble with scalable systems, it's the data. We'll come back to this idea in a bit.

Vertical Scaling or "scaling up"

Vertical scaling is achieving better performance by simply adding better hardware. If your system runs well with 1000 users but not so well at 5000 users, then you can simply upgrade your server with the fastest hardware and you're set.

Vertical scaling is easy. You don't need to think about scaling at all when writing your applications because it's essentially always running in one environment, on one server. If things get too busy, you just upgrade your server.

The problem with scaling up is that it is expensive. Low-end hardware is very cheap, but high end hardware is exponentially more expensive. There comes a time where the cost of the hardware is simply too much, or, that the hardware with the required resources simply does not exist.

So, vertical scaling is a type of scaling that you usually get "for free" without any work. But it is not practical if you want to build a site that serves millions.

Summary:

• Better hardware = better performance
• Pros: Easy
• Cons: Expensive; there will always be a technological limit to how powerful a machine you can possibly buy.

Horizontal Scaling or "scaling out"

Scaling out is achieving better performance by adding more servers. Instead of constantly upgrading to more powerful machines, we can buy a bunch of less-powerful machines cheaply. It is much cheaper to buy many commodity servers than it is to buy one really powerful server. And unlike vertical scaling which has a "limit" to how powerful of a machine you can possibly buy, horizontal scaling can go on forever.

Basically instead of 1 really powerful machine with 32GB of memory and 4 processors, you have 8 commodity machines with 1 processor and 4GB of memory. Each machine handles a portion of the load instead of one machine handling it all. When you need more resources, it's as simple as adding a new (and cheap) commodity machine.

The cost of scaling out comes in two new flavors. The first is the cost of administering these new machines. Instead of say 1 monster server you now have to babysit 8 smaller servers. The real-world cost of doing this though is (fortunately) relatively low. Each machine you add to your system should be identical, and so, system administration tasks can be largely automated.

The second cost comes in design difficulty. You have to think a lot more while designing your system to make sure it can scale out. That is, how does your system work accross 8 separate servers?

Summary:

• More hardware = better performance
• Pros: Cheaper; infinitely scalable
• Cons: Harder to design, a bit harder to manage because you have more machines to watch

Database Replication

Database replication is about automatically copying data from one database server to another database server. There is a master database that sends out data to it's slaves who copy it to their own database. The idea is that you can spread load to all of the slaves instead of relying on a single database server to handle the load of every user.

There are few types of database replication, but the most popular is called master-slave replication. This is where you have a single master database, and any number of slave databases. All write queries (queries that modify the database) go to the master. All read queries (SELECT type queries) are spread amongst the slaves. This works well in sites where the number of reads far outweigh the number of writes (which is just about every website online!).

Designing for Database Replication

For master-slave replication, your application must know which server to send queries to. This is fairly simple because it is easy to interpret plain-text SQL and know which kinds of queries are writes (UPDATE, INSERT, REPLACE) and which are reads (SELECT). For example, in your database abstraction layer you could modify your query method to read something like:

public function query($sql)
{
    if (preg_match('#^\s*SELECT#i', $sql)) {
        $conn = $this->slave_connection;
    } else {
        $conn = $this->master_connection;
    }

    // Send query to $conn database
    // ...
}

The other thing you need to do is decide how you will load-balance the slave servers. There are hardware solutions or software solutions. The simplest method, and the one I personally like to use, is to simply choose one at random when your application starts up.

Replication Lag

Replication lag is the time it takes for all slaves to execute the write queries that happened on the master. This is usually very very fast (milliseconds) and thus not usually something to worry about. But if you are overloading your cluster, then the time it takes may increase and your users will start to see the affects. Things like saved changes not appearing when they reload the page.

One potential cause is that the slaves themselves are being overloaded -- they are serving too many read queries so they are falling behind. This can be fixed by adding new slaves to help spread the load some more.

Database Engines

One thing worth noting is that a slave can use a different database engine than its master. For example, your master can use MySQL InnoDB tables and your slaves can use MySQL MyISAM tables.

I mention this because InnoDB is required for transactions, but is much slower than MyISAM. So it seems natural to use InnoDB for the master (write) database and MyISAM for the slave (read) database. It means your slaves can handle more load.

Database Partitioning

Database partitioning is about splitting up your database so that all of the data is spread across multiple databases.

Clustering

Clustering is splitting your database up at the table level. So you have your users, usergroups and profiles on one database, and then you have your forums, topics and replies on another database.

This method is pretty straightforward but it is severely limited. It means you can't join on tables used in a different database, and it adds a certain overhead because each page now requires connections for each cluster. So if you have split your database into 4, one request now needs 4 connections.

Also, each cluster will likely have different requirements. For example, there will likely be many many more replies than there are user profiles. So this makes it more difficult to manage.

Sharding

Sharding is splitting data up at the row level. This means you have a bunch of databases with identical tables, but the data they contain is split up. For example, users 0-10000 are stored on shard 1, 10001-20000 are stored on shard 2 etc.

This method is infinitely scalable because when one database becomes overworked, you just add another shard.

The difficulty comes when you need to join with data that exists on different shards. The best way is to simply ensure that all related data exists together in a single shard -- but this can be pretty difficult. The other way is to denormalize your database so that you don't need to join with other tables.

Database Denormalization

Going against all best-practices with database design -- denormalization basically means copying data from one table into another to eliminate the need for table joins. Joins are fairly expensive, so it is sometimes worth the horrible icky feeling in your stomach when you denormalize.

So as an example, you might have a 'threads' table that has a field 'last_poster_id' that links to a 'users' record. If you wanted to get the username for the last poster, you'd need to join with the 'users' table. But if you denormalize your schema, the 'threads' table would have a new field called 'last_poster_username'.

The downside of denormalization is that you need to make sure all records stay consistent. With the example above, it means that if we change the username for the user we must also change every 'last_poster_username' in the 'threads' table. In situations where data is read many more times than it is written, this is usually okay.

Caching

The single most important aspect to scalable websites is caching. Every website, when you get right down to it, reads data more than it writes data. So, instead of reading data from a database -- which means reading from disk -- you should read data from a memory cache. With very popular websites, reading from disk is complete and utter disaster! You must cache to memory if you want to become the next YouTube.

Cache what? How to cache?

You should, in short, cache everything. The database is only there for persistent storage, and to refresh caches when the caches become stale. So that means your User::getUserinfo() method looks something like:

public function getUserinfo($user_id)
{
    if (cache_exists("user:$user_id")) {
        return cache_get("user:$user_id");
    }

    // Else, you need to fetch the data from the database
    // ...

    // And then cache the result for subsequent requests
    cache_save("user:$user_id", $userinfo);

    return $userinfo;
}

There are many different technologies that you can use to cache data. But the best of them (or at least most widely used) is called memcached. PHP even has a memcached module.

Handling stale data

When you are caching data, you need to be careful that the data in the cache is updated when the data in your database is updated. So this might mean a simple update to your User::save() method for example. Other times, you might add some logic so that cached data is automatically removed after a certain period (so that the next time its requested, it will be fetched and then recached once again).

Where to cache

Above, I mentioned that you should cache to memory. But I should clarify that memory is not the only way to cache things; and might be superfluous in many situations. Indeed, before you even need to think about memory caching you should identify a disk IO bottleneck so you know for sure that a memory cache will even improve the situation.

Before you move to a memory cache, the easiest way to implement a caching system is through files in the filesystem. You might even cache data in the database. The whole point in caching is to reduce load. So, for example, if you cached an entire page in the database all you need to do is fetch one row and you can present it to the user quickly (versus say, 10 queries and HTML-building routines you might normally need to do).

The point I'm trying to make is that caching isn't always about memory. It's about storing data (possibly pre-processed data) in a location that is inexpensive to access -- or at least inexpensive relative to the original source.

It just so happens that memory is the only source for a very popular site to read cache from, because all other disk-bound caches (including files and databases) become too slow.

Separate Components

You should separate components where you can so you can create specialized machines that are very good at certain tasks. For example, Apache is great for serving PHP pages but is really overkill if you need to serve up static CSS/Javascript files. Try adding a new server that runs Lighttpd to serve up those kinds of things. Additionally, don't put your MySQL server on the machine as your memcached server. You get the idea.

In Closing...

So there's a very brief overview of the topics you need to read about. I hope you now understand the sorts of things that go into a scalable website. My goal was to demystify the ideas of scalability so you can better research specifics; and of course in coming days I'll try and talk about specific techniques.

Why use Unicode

I covered this in my previous article on Unicode, but let's recap. The reason to use Unicode is so you can create multilingual applications.

Without Unicode you basically have two ways to serve a multilingual site. I'll briefly explain each here.

Encoded HTML Entities

The first way is to store all data in one character set (for example, iso-8895-1 for English) but encode all other foreign characters as HTML entities. Most web developers have probably done this at one time or another -- for example, using © for the copyright symbol. But you can also encode any character this way by specifying the Unicode code point like so: π for π.

Common Uses:

• When you have a page in one language, but sometimes need to use a character from another (perhaps you use the word über a lot ;-)).

Advantages

• Easy to use.
• Works with any character set you already have in place.

Disadvantages

• It's not "correct". Instead of solving the problem, you're just patching it up.
• Building systems that do not know about HTML or entities becomes troublesome. For example, sending emails or building a search engine.
• As just normal text (not rendered HTML), the entity codes mean nothing. This becomes a problem if you have some way for users to consume your content other than through a web-browser (as already mentioned, email is a big one).

Multiple Encodings

The second way is to store data in multiple encodings. Store English in ISO-8859-1, store Russian in ISO-8859-5 etc.

Common Use:

• These days you see this kind of thing in places where systems are not ready to handle Unicode. For example, years ago some email providers or email clients did not handle Unicode at all. So a reason to mix encodings is for compatibility with legacy systems.
• Another reason this is in use today is due to legacy code. It might be a big job to switch to using Unicode.

Advantages:

• Legacy systems usually have no problem working with any of the popular character sets.
• Content is portable -- unlike using HTML entities which requires a parser to convert the entities.

Disadvantages:

• Impossible to mix characters from two different languages. For example, imagine you have a site that contains both Japanese and Russian articles. If you were to have some global site-map that lists all articles, the titles would be mangled because you can't represent both Cyrillic and Japanese characters with any single character set.
• You introduce some complexity. Along with each piece of content, you need to store which character set was used. You need to worry about outputting the correct character set header when serving web pages. If you have some sort of service-based application, you need to worry about which character set requests are sent in and which one to send results back in.

Using Unicode

Using Unicode suffers from none of the disadvantages describes above. Indeed, the very reason Unicode was created was to overcome those problems.

The most often used encoding for Unicode is UTF-8. This variable-length encoding stores Unicode characters in 1 to 4 bytes. There are other encodings, like UTF-16 or the depreciated UTF-7 -- but we'll only be talking about UTF-8 today. In fact, UTF-8 will probably be all you ever need.

Before we continue, I want to at least mention a few "problems" associated with using Unicode.

• Unicode will sometimes use more storage space. While normal ASCII characters will use the same amount of storage (1 byte), most other characters will use 2-4 bytes. Storage is cheap nowadays so this often isn't a problem.
• Unicode is "new". The idea isn't new, but the widespread adoption is. (At least, relative to older character sets like ISO-8895-1 et all). This means that if your app interacts with very old systems a lot, then you might have some trouble with Unicode.

  You'll likely have no problems, but I mention this just to make sure you test any systems and devices your app needs to work with. For example, cell phones are notorious for their slow Unicode adoption rate.

PHP and UTF-8

Declaring UTF-8

To start serving UTF-8 web pages, you need to send a content-type header. Web servers will automatically send a default header if you don't (which is why you may never have had to do this before). Most of the time the default header says the content is HTML, and is encoded in ISO-8895-1. Since you output HTML and your web pages are English -- this is almost always okay.

To start using Unicode, however, you need to either change the server config and change the default encoding or output it yourself from within PHP:

header('Content-Type: text/html; charset=utf-8');

Also note that your HTML content might also be spouting a 'http-equiv' meta tag that features the encoding. You should make sure it has UTF-8 there, too:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>

A browser will always take the HTTP header over the one in the meta tag, but it doesn't hurt to be consistent.

Handling UTF-8

It's all good to declare your pages as UTF-8, but that's useless if you're data isn't actually encoded in UTF-8. So let's talk about how to work with UTF-8 strings.

First of all, let me mention that browsers will submit forms in the same encoding as the page. So since you declared you pages as being encoded in UTF-8, all user data will be in UTF-8 too. The real work comes with using PHP to validate and manipulate user input.

The Problem

PHP is pretty much completely unaware of UTF-8 except in some specific circumstances. Most of the time, PHP treats all text as a string of 1 bytes -- that is, ASCII. This has a number of consequences due to the way UTF-8 uses a variable-length encoding scheme, and PHP's lack of functionality to properly handle it.

PHP's string functions are made to work on 1 byte characters. So counting the length of a string is as simple as checking how many bytes the string consumes, or getting a specific character by providing an offset is as simple as returning the byte at the offset. But stick in a UTF-8 encoded string, and now you have single characters that take up 2 (or 3 or 4) bytes. The result: PHP's string functions don't see a 2 byte character, they see 2 separate 1 byte characters (which is, of course, incorrect).

The solution is to write your own string functions that know all about UTF-8. You need to think about which functions will require characters to be 1 byte -- those are the dangerous ones. So things like strlen() or strpos(). Others, like explode(), only search for a series of bytes (which can be UTF-8) and use it -- so they are okay.

Here's a list of some unsafe UTF-8 functions:

• ord
• str_ireplace
• str_pad
• str_split
• strcasecmp
• strcspn
• stristr
• strlen
• strpos
• strrpos
• strrev
• strspn
• strtolower
• strtoupper
• substr_replace
• substr_replace
• [l|r]trim - Only unsafe when using the second $charlist argument, otherwise it's safe
• ucfirst
• ucwords

Libraries and Extensions

You're pretty insane if you want to go ahead and re-write all of the string functions yourself! Instead, there are a few ways you can safely handle UTF-8 strings without all the headbanging.

• iconv which is default in PHP5, is mostly used when converting various character sets to UTF-8. However, there are a few string functions like iconv_strlen() that are UTF-8 aware. Not the holy grail..
• mbstring is another PHP extension that is intended to be the holy grail. But it's not in PHP by default, which can be a deal-breaker if you produce mass-market products that'll be used on shared servers. This extension has redefined all string functions so they are UTF-8 aware, and also offers some other features.
• phputf8 (download) is a rewrite of the string functions written in PHP. It will auto-detect if iconv or mbstring are available and use those string functions where possible (since compiled string functions will be much faster than PHP string functions).

I like to the phputf8 library in all my projects. It's simple and clean and it works!

Well Formedness

It is possible that UTF-8 input can be ill-formed. This is because of the variable-length encoding. To remove any ambiguity when parsing UTF-8 there are certain bytes that are invalid. These invalid ranges are a design feature, they make it impossible for the computer to "mix-up" a two-byte character for two separate one byte characters (or visa-versa).

You don't need to really understand what I just said since it has to do with how UTF-8 is represented on your computer (read the Wikipedia article if you want to know more). All you need to understand is that UFT-8 text can be malformed, and thus, it's something you need to validate.

So when you are accepting form input, you should always make sure it's valid or else there's a possibility of various attacks. For example, imagine a UTF-F byte sequence is "AB", but an attacker modifies the request so only the first byte "A" is sent. Due to the way UTF-8 is encoded, your computer knows there should be a second byte after "A". So when you blindly output the single "A" into your HTML page, it's possible the next byte (whatever it may be) is "eaten up". Imagine if the next character is a double-quote intended to close off an HTML attribute or something like that. Possibility for XSS is introduced.

One very simple way to check for well formedness is to use preg_match() with the 'u' modifier:

if (strlen($str) AND !preg_match('/^.{1}/us', $str)) {
    die('Invalid UTF-8');
}

Since any invalid UTF-8 string will result in the match failing, all you need to do is match a single character.

If you're using phputf8, you can use the utf8_compliant() or utf8_is_valid() functions. utf8_compliant() uses the method described above, but will also pass 5 and 6 byte sequences which aren't technically valid UTF-8 (5 and 6 bytes are not unsafe, however). utf8_is_valid() will make sure the string is actually valid UTF-8 but will take longer.

Regular Expressions

When using regex with UTF-8, you need to use the 'u' modifier. That's all it takes!

if (preg_match('/myregex/u', $str)) {

}

Stick with ASCII when...

When possible. There are still a lot of places where Latin letters and numbers are the only valid input. For example, URL's and email addresses.

Also, working with ASCII will often be better performing than using UTF-8. Especially if you are using a PHP-based library like phputf8 which will be slower still then compiled libraries like mbstring at certain operations.

UTF-8 and MySQL

If you use MySQL out of the box, and PHP's mysql_* functions, then you can INSERT and SELECT data from MySQL and you're safe almost all the time. That is, you can INSERT a string of UTF-8 characters into a table that is marked as ISO-8895-1. This is because MySQL simply sees a string of 1 byte characters -- it doesn't actually know that your text is UTF-8.

You'll immediately see a problem if you use a tool like phpMyAdmin, though. Because phpMyAdmin will output the results to you in the same charset as the database table. You'll end up seeing garbled text.

Two other problems will quickly become apparent:

1. Collation. Collation is how a language is sorted. For example, in English we go from A-Z.
2. String functions. Using MySQL's string functions may garble your strings if the table charset and the data charset are different.

What you should do is make sure your tables are encoded in UTF-8, and that the default collation is UTF-8, and that your MySQL client connection is being interpreted as UTF-8.

To set the global character set and collation you can edit the MySQL configuration file:

default-character-set=utf8
default-collation=utf8_general_ci

This sets the charset to UTF-8 and the collation to case-insensitive UTF-8.

When creating a database or table, you can also supply these values:

CREATE DATABASE `test` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci
CREATE TABLE `test` (...) CHARACTER SET utf8 COLLATE utf8_general_ci

Note that specific fields can also have their own charset and collation using the same syntax.

Once you connect to the MySQL server, you should issue a query to indicate that your queries will be in UTF-8:

mysql_query("SET NAMES 'utf8'");

After that -- you're good to go.

Converting content to UTF-8

First, a note: Make sure you have enough storage for the new strings. As explained earlier, UTF-8 strings can possibly use up to 4 times the amount of storage space. This usually isn't much of a problem with content saved to disk. However, if you are storing content in a database, your fields like VARCHAR(255) may be much too small.

If you need to convert simple strings on the go, you should use iconv. For example, perhaps you need to convert an email message from Windows-1251 (common with Russian) to UTF-8:

$content = iconv('Windows-1251', 'UTF-8', $old_content);

If you are using MySQL, you can convert entire tables of content quite easily by issuing ALTER TABLE queries. Note that your existing charset and collation set on the tables need to be correct or else the translation will garble your text! (Going back to my note above about how MySQL won't know what charset you INSERT text in).

ALTER TABLE `table` CHARACTER SET utf8 COLLATE utf8_general_ci
ALTER TABLE `table` CHANGE `table` `table` [TYPE] CHARACTER SET utf8 COLLATE utf8_general_ci

Converting UTF-8 content into another character set

Sometimes you might need to convert content back into another set. For example, if you've decided to switch to using UTF-8 but your app still communicates with a legacy system that only understands ISO-8895-1.

Again, you should use iconv which is perfectly suited for this kind of thing:

$content = iconv('UTF-8', 'ISO-8895-1', $old_content);

Note that the characters represented in UTF-8 may not be able to be represented in the character set you want to convert to. For example, if you're content contained the copyright symbol, ISO-8895-1 won't have that symbol.

You have two options: You can either transliterate the characters that can't be converted or simply discard them.

Transliteration is the process of replacing the character with one that looks sort-of like it. For example, "é" might be replaced with "e". To achieve this, you append "//TRANSLIT" to the out-charset:

$content = iconv('UTF-8', 'ISO-8895-1//TRANSLIT', $old_content);

On the other hand if you just want to get rid of the offending characters, simply append "//IGNORE" to the out-charset.

Conclusion

I hope you've learned a bit about using UTF-8 with PHP. It's really not all that bad once you take in all of the information -- definitely not as scary as you thought it'd be, I bet!

Additional reading:

• WACT: Character Sets / Character Encoding Issues
• UTF-8: The Secret of Character Encoding
• Wikipedia on UTF-8 and Unicode

What are character sets?

Let's start with the basics. Every character of a string is stored as some binary number in your computer. A character set is a map that maps the number in the computer with an actual glyph that you can recognize. For example, in the ASCII character set, the code for 'a' is 97.

Back in the day there was ASCII (well there were more before that, but let's not go too far back in history). ASCII used numbers from 0-127 to map certain control characters (like newlines) and English letters and punctuation (a-z, 0-9, quotes etc). If you spoke English, then life was good.

But if you spoke a different language that used different characters (for example, accented characters), you were pretty much screwed.

To solve this problem, engineers in different parts of the world started to come up with different character sets. That is, they created character sets so they could represent glyphs relevant to their language.

In a pre-internet era, this wasn't all too bad. But once people started sharing documents, the problem became all too clear. An American reading a report drafted in Russia would see a page of garbled text because the character set used on his American-bought computer was completely different from the character set used on the Russian-bought computer.

Every common character set agrees on the characters from 0-127 (thankfully!). But any character represented from 128 and up differ greatly. For example, on some early PC's the code 219 represents an opaque block (used to create boxes and such in old apps). But on other computers that code represented a U with a circumflex instead.

Further complicating the problems, the size of the code was sometimes different. In Asian alphabets, there are thousands of characters. That means there are too many codes to fit in the usual 1 byte (0-255) that most computers were using. To overcome this problem, special character sets were created in which some characters took 1 byte, and other characters took 2 bytes.

This might not seem so bad, but think about it for a second. The most common tasks like finding the length of a string now becomes more difficult. You can't simply count the number of bytes the string occupies because that would give you an erroneous number (you can't know how many characters are 1 bytes and how many are 2 bytes). And even things like iterating through a string a letter at a time is not a simple matter of moving a memory pointer up by 1 byte.

Unicode

Now that the problem is sufficiently explained, let's get to the solution.

Unicode was created as the one character set to rule them all. In Unicode, every possible glyph imaginable is mapped to a unique number.

Unicode determines which number, called a code point, represents which character. The standard way to represent a code point is like this: U+0097 for the letter 'a'. That is 'U+[hex]'.

But here's where it gets a bit tricky. The Unicode standard only defines which numbers map to which characters. The actual means of storing the number is still up in the air.

Most people think that Unicode is limited to a maximum of 16 bits. But this is untrue: Unicode is limitless. It's limitless because Unicode itself doesn't care about storage, it only cares about mapping code points to characters. So Unicode can theoretically represent an unlimited amount of languages.

Encodings

So if Unicode only cares about code points, how does a computer store the actual data? This done done with a specific encoding.

So far we've been using the term 'character set' to mean two things: Which characters are in the set and how they are represented. Basically by representing text in a computer there are two components:

1. A so called character repertoire which defines which characters can be represented. ASCII's character repertoire says we can represent English letters, numbers and punctuation (amongst others). Unicode on the other hand can represent everything.
2. A character encoding states how the computer actually stores these characters. It defines how a bunch of binary bits can be converted into the actual characters. With ASCII, we know that a character is always stored in 1 byte, and that 'a' is 97 and 'b' is 98 etc. With Unicode, there are multiple encodings.

The most popular encoding you hear about today is UTF-8. UTF-8 uses 1 to 4 bytes to store each code point. For code points 0-127, 1 byte is used. As the code points increase, the bytes it takes to represent them increases up to a maximum of 4 bytes.

Since Unicode defines code points 0-127 the same as ASCII (for example, 'a' is code point 97, just like 'a' in ASCII is 97), and UTF-8 encodes code points 0-127 as a single byte -- UTF-8 is directly backwards compatible with ASCII. For most English systems, this makes converting to UTF-8 very very easy. For example, if you have a bunch of English web pages, you could switch the character encoding header on your web server to 'UTF-8' without any work. If your systems contain different languages (and thus, non-ASCII characters), then converting to UTF-8 will require other tools.

When character sets are important?

In short: ALL THE TIME

There is no such thing as text without a character set. There always needs to be a way for a computer to convert the random bits and bytes in a file to characters that humans can understand. Most of the time when you read about "plain text", it means a file using ASCII code-points 0-127.

How programs decode characters in files depends. For example, some browsers might guess by looking at common patterns of code points. But most web sites these days will specify the character set right in the response headers, so a browser doesn't need to guess.

If you have ever opened up a web page or a file and seen a bunch of ???'s, then it means the application is trying to decode the text data using an incorrect character set.

Conclusion

I hope you learned something about character sets today. In an upcoming article I will write about the role character sets and Unicode play in our applications, specifically with PHP.

About Zend Framework

The Zend Framework is a free (BSD-like license) PHP framework. It contains components to tackle many common problems including authentication, cacheing, mail, sessions, and of course MVC.

ZF is simple. It is a "use at will" framework, meaning you can take specific components and use them wherever you want. You can, for example, take just the Mail component and stick it into your already existing PHP application. This decoupled architecture is contrary to some other frameworks (including symfony or CakePHP to certain extents) where you are forced into using a single paradigm.

This framework is written for PHP5 only, so it is not encumbered by the limitations of PHP4. It is written using the best OOP principals, is thoroughly tested, and has a marvelous community of maintainers that keep it up to date.

Zend Framework versus other frameworks (symfony, CakePHP etc)

Typically when you think of a framework, you envision a completely developed codebase that includes its own of configuration, its own conventions, and specific programming paradigms. Many frameworks including symfony, CakePHP, Code Igniter and Ruby's Rails, follow this kind of thinking.

Zend Framework is a collection of loosely coupled components. As mentioned earlier, you can use any one of the components separately without running into any problems. Even so, the components still work very well together (for example, Zend_Auth can seamlessly use Zend_Session for persistence).

You need to do a bit more work to use ZF initially. ZF is not all tied together, so if you want to use MVC (for example) then you have to create your own bootstrap, and create your own directory structure etc. When you download ZF it's simply a directory that contains the library files, you're the one who has to initialize and use everything. Compare this to symfony (or many others) where everything is ready to go after you edit a couple configuration lines.

I prefer ZF because I feel like I have more control. I am not tied into any one paradigm, I don't need to play around with configuration files and I can mix and match my code to my hearts content.

Please note that I am not belittling the other frameworks, I am just trying to explain what sets ZF apart. If you read some of my previous posts, you'll see I'm also an advocate of symfony.

The Parts

MVC with Zend Framework is quite easy. Here's a brief overview of each part and how they relate to each other.

1. Front Controller

Most people will use a Front Controller. Using a Front Controller means that all requests come through a central starting point. Usually a single index.php will instantiate the front controller object and handle all incoming requests to your site. The file is referred to as a "bootstrap" file, and will contain code to initialize other things in your system too (for example, loading configuration settings).

If you've been developing on the web for a while, you've probably seen lots of scripts that have many entry points. For example, you might have index.php, login.php, article.php etc. Most of the time you also have a global.php file that is included at the top of each script. When you have all these entry points, it can quickly become hard to manage and you also start to get a bunch of code duplication.

A Front Controller solves this. Instead of physical files on the disk defining how a request should be handled, the request itself is entered through the Front Controller and then analyzed to see what should be done with it.

2. The Request Object

Each request made has a corresponding Request object. This object wraps up the entire request environment. For example, the default HTTP environment contains things like request parameters from GET and POST. The Request object also contains which controller and action should be called upon.

3. The Router

The Router is what actually inspects the incoming request and decides which Controller and Action should be called up. For example, it will look at a request of /login and decide that the Login Controller should handle the request, and the Index Action should be called.

ZF comes with a default router, so you don't need to change anything unless you want special behavior. By default, the incoming URL is analyzed and the parts are mapped to specific controllers and actions: /:controller/:action. If they are not provided at all, then they are assumed to be 'index'.

3. The Dispatcher

Dispatching is the process of looking at the requested controller and action, and actually including the correct Controller file, instantiating it, and calling on the action. The Dispatcher is in a loop. That means you can run as many controllers and actions as you like.

Again, ZF comes with a default dispatcher that sets down some naming conventions that we'll explore in a bit.

4. The Controller

Finally there is your actual controller. A controller handles a specific part of your app (for example, working with articles) and can have multiple actions (for example, listing articles, viewing a single article, viewing a printable version etc).

The Process

A request is handled like so:

1. A web request is made to a single entry point (usually an index.php file)
2. The entry point creates a Front Controller and runs it
3. The Front Controller creates a Request object that wraps up the environment. At this point there is no controller or action set.
4. The Front Controller uses a Router to inspect the request to figure out which controller and action should be called upon. It sets these values in the Request object.
5. The Front Controller starts the Dispatcher loop which includes the correct Controller source file, instantiates it, and runs the correct Action.
6. A Controller does what it does. It can handle forms, get database info, render HTML etc.
7. Any Controller can also modify the Request object and set a different Controller or Action, so the Dispatch loop is run again. If no change is made, then the Dispatch loop ends and the result is served to the user

You can get a visual representation of the process by viewing this image.

Let's Get Started

The File Structure

I like to separate my files into two parts. One part for the actual application files (controllers, libs, views, functions etc) and then public files that the user needs access to (javascript, css, images etc).

For the sake of simplicity, I also like to package ZF with my app. If you have a shared directory where you place libraries (PEAR, for example), then you of course don't need to include it with your own files.

Here's the structure I use:

• root
  • /client
    • /css
    • /images
    • /js
  • /appfiles
    • /application
      • /controllers
      • /views
        • /scripts
    • /lib
      • /Zend
  • /.htaccess
  • /index.php

This also makes it easy to split up the appfiles from the public files. For example, may people like to place the application source files outside of the document root on the webserver.

The /appfiles/application directory contains all of your controller and view files. We'll talk about the naming convention the default Dispatcher expects in a sec.

The /appfiles/lib directory is where I personally like to keep the Zend Framwork files.

You can of course have any other directories you'd like. On one of my recent projects I had /appfiles/functions and /appfiles/classes for some various custom code files.

Rewrite URLs

To create friendly URL's like /login or /article/view, you need a way to rewrite all URL's to go through the main index.php file. If you are using an Apache webserver and htaccess is a viable option, then the following simple code snippet should work fine for you:

RewriteEngine on
RewriteRule !\.(js|ico|gif|jpg|png|css)$ index.php

If you are on a different server, there are other ways to rewrite URLs.

Note that you can still use ZF's MVC components without rewriting URL's. Refer to the documentation for info on how to do this.

Create Bootstrap

The bootstrap file is the "single entry point" I've been talking about. It contains the code that launches the Front Controller. In this case, our bootstrap file is index.php.

Here's the code for a simple bootstrap file:

<?php

#------------------------------
# Get the paths set
#------------------------------

define('ROOT', realpath(dirname(__FILE__) . '/appfiles'));
set_include_path('.' . PATH_SEPARATOR . ROOT . PATH_SEPARATOR . ROOT.'/lib');

#------------------------------
# Register the Zend autolaoder
#------------------------------

require_once(ROOT . '/lib/Zend/Loader.php');
Zend_Loader::registerAutoload();

#------------------------------
# Dispatch
#------------------------------

Zend_Controller_Front::run(ROOT . '/application/controllers');

Line 7 defines the ROOT constant to be the full path to the appfiles/ directory. If you were to move the appfiles/ directory somewhere else, you'd want to edit this line so the path stays correct.

Line 8 sets the PHP include path so that the Zend files are all included without any problems. We need this line because we are distributing ZF with the project, we need to tell PHP where to look for the files when we include them.

Line 15 and 16 include Zend_Loader and register it's autoload functionality. PHP5 introduced the ability to autoload classes. Whenever a class is used but has not been defined, PHP will try and find the file and include it for you. This saves you from having to manually write all of the require_once()'s you would normally write under PHP4.

Line 23 fires off the Front Controller which starts up your whole application. As a parameter you need to pass the path to your controller files which in our case is /appfiles/application/controllers.

If you run this now (http://yoursite.com/path/to/project/), you'll see an exception because ZF can't find your controller file to handle the request. So let's write it.

Controllers and Actions

What are Controllers? Actions?

Lets get this question out of the way for those who are new to this kind of website architecture. A Controller is a class that handles the business logic of a given part of your website. It fetches data from a datasource, interacts with output, and then passes any calculations to the View to be displayed to the user (usually as HTML). Some examples of Controllers might be: LoginController, ArticleController and SearchController. Before you might have had files like login.php, article.php and search.php -- now you have classes that control all of this behavior.

Each Controller has a set of Actions, which is just a method you define within the Controller class. An Action is something specific the Controller needs to do. Using our examples listed before, this might be processing a login, displaying an article or performing a search. Before you might have used a variable in the URL to determine what needed to be done. For example, "article.php?do=display" now becomes the ArticleController with the 'display' Action.

Naming Conventions

So you know that controllers are classes, and actions are methods in the controllers. Now the only thing left to cover is how ZF knows which file to load, and which method to call.

You defined the location of the controller files when you called Zend_Controller_Front::run() in the bootstrap. In our example, it is /appfiles/application/controllers.

The Dispatcher (explained above) is responsible for loading the appropriate files, instantiating the correct class, and then calling the correct method. The default Dispatcher that comes with ZF uses a naming convention so you can add controllers and actions easily (instead of say, creating a configuration file that defines it all).

There are two very simple rules:

• All controllers need to be called SomethingController where "Something" can be whatever you want. The class source code needs to be in a file called "SomethingController.php".
• All actions need to be called someAction() where "some" is anything you want.

The default Router and Dispatcher will route a URL /something/some to the SomethingController and the someAction() action. (Meaning: Remember that the default URLs are /:controller/:action).

If a controller is not specified, then the 'index' controller (IndexController class) will be assumed. If an action is not specified, then the 'index' action is assumed (indexAction() method).

Just to hammer it into your head, let's have a few examples of how some possible URLs might be loaded using the default Router and Dispatcher:

• /: IndexController class, indexAction() method
• /login: LoginController class, indexAction() method
• /login/lostpass: LoginController class, lostpassAction() method
• /article/new: ArticleController class, newAction() method

You can also use hyphens in your URL's if you use camelCase in the naming of controllers or actions:

• /download/message-attachment: DownloadController class, messageAttachmentAction() method
• /sign-up/confirm-email: SignUpController class, confirmEmailAction() method

Views

What are Views?

A View is the thing that you give back to the user after they request something. Usually this is an HTML page, but it can be other things like XML or RSS feeds, file downloads etc.

File Locations

Controllers will try to render a view script automatically. It will search for view scripts in /views/scripts/ in the directory above the controller directory (for us that means /appfiles/applications/views/scripts/). Each controller is expected to have a directory of its own, and each action is expected to have a .phtml file that is the actual PHP script to render. For example, LoginController's indexAction() will have a view in views/scripts/login/index.phtml.

View Files

The default view scripts in ZF are plain old PHP. You can add a different templating system if you'd like (like Smarty), but for simplicity, this article will use the default.

In the controller you can set variables to be available to the view, so you can do all of the usual stuff you'd expect like looping, formatting etc.

An Example Page

Time for a really simple example page that demonstrates everything you've just read.

First step is creating a new controller. This controller will be located at: /appfiles/application/controllers/IndexController.php with the following code:

<?php

class IndexController extends Zend_Controller_Action {
    public function indexAction() {

        $this->view->name = "Christopher";

    }
}

Next we need to create a view script to render the HTML page. The view script will be located at: /appfiles/application/views/scripts/index/index.phtml with the following code:

<h1>Hi, <?php echo $this->name; ?></h1>

Now if you run this in your browser, you'll get a simple page that displays a name. So what is going on here?

The Controller

The IndexController is a class the extends Zend_Controller_Action. All of your controllers must extend this base class (or a child of this base class -- you can create your own if you needed to). And on line 4 we have defined the indexAction() method to handle the default index action.

By defining both the index controller with an index action, we now have a default page do display whenever there was no URL supplied (remember that the dispatcher assumes 'index' if no controller or index was supplied).

On line 6 you'll notice we are assigning a name to a variable $this->view->name. This is how you pass values to your view script. You can assign any values. The view script is just PHP remember, so it doesn't matter if you assign arrays, objects, integers or strings.

The View

The view in our case is very simple. I just wanted to highlight how we use the variables assigned in the controller. All values assigned in the controller are available as $this->varname.

What about Models?

I haven't discussed anything about the models in this article. That's because the model in ZF's MVC can be just about anything.

A model is an object, or a system of objects, that represent some sort of data. Most often models are objects that represent specific database records, or database tables. For example, if you have a "user" table then you'd also have a "User" object to represent records from the table. You might also have a "UserTable" object to make finding records easier (sometimes called "peer objects"). So you might have classes with methods like User::setPassword() and UserTable::findUserByEmail(). This technique of mapping a table to objects is the ActiveRecord pattern.

You can code these objects yourself, but usually some sort of ORM (Object-relational mapping) tool/library is used. Two of the most popular are Doctrine and Popel (I suggest Doctrine). With these libraries, you define the structure of your database and the rest of the work is done for you. That is, the means of satisfying relationships and filling data objects with actual data form the database is done by the library rather then by hand. In the case of Doctrine, actual SQL is abstracted into so-called "DQL" (Doctrine Query Language) to make your programs 100% portable.

The point of a model is to create a separation between the data-tier and the logic-tier in your application. For example, your application's "change password" controller shouldn't need to know the low-level details of how a password is changed (the hashing, salting etc). That kind of thing should be coded into the model.

Finished

That's all there is to it. Click here to download the sample files used in this article (note that you need to add your own Zend library to the /appfiles/lib directory).

Next article I will go over some more advanced techniques including subclassing the front controller, action controllers, creating plugins and creating helpers.

There is no formal way to actually define member visibility in Javascript (no "public" and "private" keywords like you might see in Java). Instead we just use the language features in such a way that it creates the same public/private situation as we'd expect in other languages (or nearly the same).

Public

A member is public if "the world" can access it. For methods this means anyone can call on it, and for properties this means anyone can use its value or change it. If you are creating object-oriented programs, having nothing but public members breaks a lot of the ideology. In many languages, like PHP before version 5, programmers used conventions to denote members were private and not to be used. Sometimes this works, but true encapsulation is much better.

There are two ways you create public members:

Constructor

function Person() {
    this.name = 'Christopher';
}

var me = new Person();
alert(me.name);

Using 'this' binds the variable to the object, so anyone can use it.

Prototype

function Person() {

}

Person.prototype.name = 'Christopher';

var me = new Person();
alert(me.name);

Adding a method or property to the prototype also makes it public.

Private

A member is private if only the object itself can access it.

To create private members, you define them within the constructor:

function Person() {
    var first_name = 'Christopher';
    var last_name = 'Nadeau';

    function fullname() {
        return first_name + ' ' + last_name;
    }
}

var me = new Person();
alert(me.name); // can't do it!

In this example there are two private properties for first and last name, and one private method to return a full name.

Some Issues...

There are a couple of issues you have to be aware of when using private members.

Only methods defined in the constructor can use private members. That is to say, we cannot do something like this:

function Person() {
    var first_name = 'Christopher';
    var last_name = 'Nadeau';

    function fullname() {
        return first_name + ' ' + last_name;
    }
}

Person.prototype.alertName = function() {
    // Can't do this. 'this.first_name' does not actually exist!
    alert(this.first_name);
}

The scope of the private members are in the constructor only. To get around this problem, you must use what is called privileged methods.

Privileged

As stated before, only methods defined within the constructor have access to private members. Given that many objects are built around using 'prototype', this is problematic. The solution is to create privileged methods.

A privileged method is a method defined in the constructor but assigned to a public variable. Since it is defined within the scope of any other private members, it has access to them. Here's an example:

function Person() {
    var first_name = 'Christopher';
    var last_name = 'Nadeau';

    function fullname() {
        return first_name + ' ' + last_name;
    }

    this.alertName = function() {
        alert(first_name);
    }
}

You can guess that you will be able to retrieve the 'alertName' method or reset it. You might think that it would be possible to redefine the method to take control of some other private member. Not so! The method is a closure and retains the scope in which it was defined. Since it was defined in the scope of the other private members, it has access to them. There is no way to recreate this scope, so even reassigning a completely new function will not allow you to expose any private members.

The 'this' reference

Another thing to be aware of is that functions defined within the constructor have "this" pointing to themselves, and not the object itself. Thus if you need to call other public methods, you cannot. For example, let's make the fullname() method privileged and modify alertName() to call it:

function Person() {
    var first_name = 'Christopher';
    var last_name = 'Nadeau';

    // This is now a privileged method
    this.fullname = function() {
        return first_name + ' ' + last_name;
    }

    this.alertName = function() {
        alert(this.fullname()); // wont work
    }
}

To get around this problem, Javascript programmers assign another private property with the value of 'this' inside the constructor, and use that variable as a reference to the object. The two most common names are 'self' and 'that'; I prefer to use 'self'. Here is the final object, working as expected:

function Person() {
    var first_name = 'Christopher';
    var last_name = 'Nadeau';

    // Create a reference to this object so closures
    // can access it.
    var self = this;

    this.fullname = function() {
        return first_name + ' ' + last_name;
    }

    this.alertName = function() {
        alert(self.fullname()); // Using self to reference the object
    }
}

What is What

The MVC pattern is made up of three separate parts -- the Model, View and Controller. In this section we'll explore each part and what they are supposed to do.

The Model

The Model is the thing that takes care of the data in your application. It offers ways to find and retrieve data, ways to save data, etcetera. In a lot of cases your model might be a simple class that is just a map of a database table. Many of the web frameworks like symfony give you tools to automatically generate a PHP class that has all of the properties and methods for any particular database schema you have. For example, a users table might have a corresponding User class with methods like User::findByUsername() and User::setEmail().

The important thing is that you encapsulate the domain logic. Sometimes all the model needs to do is retrieve rows from a database or read data from a file. Other times additional processing may be needed, like calculating the users age based on a birth date.

The View

The View takes the data from the Model and displays it in a useful way. In web applications this usually means that the data from the Model is displayed in an HTML page. Some other Views might be for generating an RSS feed or data for a web service. By separating the Model and the View, you can easily switch things up to create new interfaces to the data without much work at all.

Note that keeping the Model and the View separate does not mean that you can't get direct access to the Model itself. In many cases you will be using the Model directly. To output a users username you might use the Model and call User::getUsername() and output it. The idea is only that you separate the domain logic from the presentation. For example, you might have some sort of subscription service. If you wanted to output a message when the users subscription is expired it might seem logical to do something like this within the View:

Welcome back, <?php echo $user->getUsername(); ?>!

<?php if ($user->getSubscriptionExpiry() <time()): ?>
    Your subscription has expired. Please renew.
<?php endif; ?>

This is mixing domain logic with the presentation. You are hard-coding what "expired" means directly into the View. If a time came when you needed to change the criteria for when a user is considered expired, then you'd need to go back into every file and change this hard-coded definition. Instead, you should create a method in the Model:

class User {
    // ....

    public function isExpired() {
        return ($this->getSubscriptionExpiry() <time());
    }

    // ...
}

The Controller

The Controller is the part that connects the Model and the View so they can work together. Each part should be quite separate. That is, the Model should not have any code in it for outputting HTML (or any presentation) and the View shouldn't worry about things like fetching results and calculating user expiration dates. Since the two are completely separate for the most part, you need a Controller in the middle that connects them and makes the whole shebang work.

The Controller takes data such as input and works with it. In web apps this usually means taking information from the URL or forms, processing it, and then outputting a web page as a result. For example, if a user wants to update their email address then the the Controller will take the new email address from the form input, use the Model to update the record in the database, and then show a success page to the user using the View.

Here's how a request might be handled:

1. A request comes in
2. The controller looks at the incoming data like the query string or form data
3. The controller might need to work with the model to fetch the requested data or to update it. For example, using User::findByUsername() to search for a particular user or User::setEmail() to update an email address.
4. Then the controller invokes the view to output a result. This might be a page with all the information for the requested user, or a success page after the email address was saved.

What Is Each Part

We've explored what each part should do, but you still might be confused as to what each part actually is. Is it a file? A class? What?!

The thing is, the MVC is only a patten. There is no single right way to implement it. A Model is can be a single class that encapsulates all the work needed. But by no means does it need to be a single class. It is not uncommon for a Model to consist of multiple classes. For example, one class that is responsible for encapsulating the data and working with it, and another class that is responsible for actually fetching the data. A lot of the time one Model may depend on another Model to satisfy related data like connecting a web order to a user account that made the order.

The Controller is just a single class at it's most basic form. You might have a single class and each method is defined to handle one type of request (list users, edit user, search users etcetera). But again, it is not uncommon for a Controller to be made up of several different parts.

The View can be almost anything. In web applications it is usually a template that is evaluated with data gathered by the controller. You can use simple PHP files like demonstrated earlier, or a template system like Smarty.

The important part of using the MVC pattern is just the way that all these parts are separated form each other.

Why

Why use this pattern? At first glance it seems like it's a lot of extra work. If you wanted to make a list of users for example, then a simple list_users.php script that does a database query and outputs some HTML seems like it should take 15 minutes. If you use the MVC pattern, you'll have to think about creating the model first, and then the separate template files for the view, and then you have to worry about the controller to glue them together.

All valid points. It is true that the MVC pattern requires a bit more work. But as soon as you have laid down the foundation, it will make your life so much easier. While some changes may require edits in each part, like a dramatic change to the model might need changes made to the controller and view, you'll also get away with other changes more easily (remember our user expiration example?). More then that though, by keeping these three parts separate you allow yourself the possibility to interchange them. For example, using just one single model you can easily create a number of different interfaces to the data. Imagine how easy it would be to create an HTML page, an RSS feed and an XML data provider. In fact, all three of these things might even share a common Controller and only use separate Views. Adding a web service to interact with the data is as simple as creating a new controller to handle requests.

Try the MVC for your next project, you'll be glad you did.

Interface

The actual interface in any application is extremely important. Getting the interface wrong can totally kill a product before a customer even gets the opportunity to try everything out.

An interface has to include everything it needs to, but at the same time stay clean and simple. You want as little of a learning curve as possible. By placing things in logical positions, and using well-known widgets, you can easily fit everything together in an intuitive way.

I would recommend the use of Javascript to simplify your interfaces, and only officially supporting the latest browsers (that is, Firefox, Safari 2, Opera, IE6+). By setting such a requirement, you can use lots of Javascript widgets and finally throw out all of the old "web 1.0" style pages, which just do not work with a lot of todays advanced applications. Obviously if you have a front-end where normal end-users use, you need to support as many browsers and platforms as possible. But I think you can afford higher requirements when you design an administrator interface.

I'm going to compare two web applications to make a couple of points. The first product is vBulletin, a commercial message board application that is regarded to be one of the best available on the market. The second is DeskPRO, a commercial helpdesk and ticket system. Here are screenshots from the User Search form from each application. I think each app is excellent, these just happen to be screens that illustrate the points I want to make.

User searching in vBulletin

User searching in DeskPRO

If you ask me, the DeskPRO interface is much easier to use compared to the vBulletin interface. When you come to the vBulletin interface for the first time, you get a long page of options. It is a daunting task to go through all of those options and find the ones that are relevant to you. While the two search forms contain different criteria, DeskPRO still displays everything in a manner that is just easier to use.

Logical Separation

You can see the vBulletin page separates search critera from display options and ordering by simply putting those options into different tables. While this works in a way, it still does not let the user quickly scan the page and find the thing he or she is looking for. Even a reference at the top of the page that has anchors to each section would increase the usability of the vBulletin user search page.

The first step on the road to better interfaces is the ability to logically separate chunks of related objects and make getting to each section easy and intuitive. In the case of DeskPRO, you see the use of tabs. I love tabs for interfaces, they're the perfect tool. Everyone knows what tabs are, what they do, and how to work them. That means you get the value of organization with zero learning curve. Instead of scanning a big long page for the section you want, you can simply click the tab and be presented with the options you want.

Input Overload

The second part is to simplify the options you present to the user as much as possible. Too many options is a bad thing. You may want to offer as much flexibility as possible, but it is easy to go over the line of "just right" and into the zone of "too much" (more on this in a bit).

Two perfect examples of what I mean can be found when comparing the DeskPRO and vBulletin interfaces. The first is the date field. If you want a date, then you should provide an easier way of inputting it then a simple text box. Select boxes seem to be quite popular as it ensures the format of the date isn't a problem and the input is valid. But in todays world of rich interfaces, there's no excuse not to use a Javascritpt widget to popup a mini-calendar for the user to select from.

The second example can be found in the number ranges. In DeskPRO you see the user ID range, and in vBulletin you see ranges for many properties from join date all the way down to infraction points. Why does each range in vBulletin span two inputs on two separate rows? This is redundant. If vBulletin were to combine each input box into a single row it would eliminate nine table rows. Nine! How much simpler would the form look at a glance if nine rows were removed?

Help

One thing both applications have in common is the ease at which help can be obtained (see the little help icons?). This is a very important thing to have. While it is always best to strive for a self-explanatory system, there will always be a user who wants some clarification on some aspect. By providing help at the click of a button, you make it that much easier to use your application.

Features

Every developer wants their app to be as flexible as possible. The more flexible the thing is, the happier the customer is because he won't need to come back as often to request changes. But there comes a point where an application starts to have too many features. At that point the application has become bloated. We've all heard that word before, but what does it really mean and how does one avoid it?

To me, bloated software is software that contains too many features I never use and probably never will. Knowing that the features are there can be a comfort, but in the long run, it's a development nightmare when there's just so many. A prime example is the newest vBulletin release introduced Podcasting support. Which percentage of the customer base will use these features, do you think?

Features always come at a cost. Every feature you add makes the application more difficult to maintain, it adds to the "size", it adds more stuff the user needs to look at, and it usually has at least a small hit on performance. To prevent your application from becoming bloated you should carefully consider every single feature before actually adding it in. If there is a large demand for a big feature, consider making it an addon instead of an actual built-in feature. The fact that you aren't "hard coding" something new into your application will mean the new feature costs virtually nothing unless a customer chooses to activate it.

Besides just bloated software, you run the risk of giving the user too much to think about. Have you ever been shopping for a product where there are thousands to choose from? How hard was it to narrow down your choice to ten? To five? And how hard was it to actually choose the one you finally purchased? Instead of offering ten different things, it's actually better to offer three. The last thing you want to do is overload the user with options.

Implementation

How to implement a feature may seem a silly thing to talk about, but it's something that still trips even me up every once and a while. We are developers, remember, so we can't exactly say how a real end user might interact or interpret our applications.

For instance, we as developers know all about ID's. To the end user an ID is just a number. A user ID, a forum ID, a ticket ID -- all numbers. A username, a forum title or ticket subject -- these are the things the average end user thinks about. While providing an ID as additional information may be useful, I still see many applications that list form fields requesting ID's as a way to identify something. For example, "Enter which forum ID's you want to apply X to" is not a very usable form. For the average user, a select box that lists the forums is much more usable then an input box.

You always need to be on lookout. As developers, we are constantly thinking about how things work, how the gears are working behind the scenes. You always need to keep in mind that the user likely has no idea about the how things work, they just want the thing to do what they expect.

Developing a good application that does all sorts of nifty things is great, but by the time it comes to adding an interface around it, it's like trying to proof-read your own work. While you can probably do a pretty good job if you put your mind to it, you need a fresh pair of eyes that hold no assumptions or knowledge of your work. So if you can, grab a friend to look over things and find out if you've done things in a clear and concise manner.

The Road To Great Applications: Become The User

I've gone over three major aspects of a good application: the interface, the features, and the implementation of those features. The interface needs to be easy to use and navigate, and needs to display everything important in such a way the user can find what he or she needs very quickly. For any application to be useful, it needs to have the features the users want and be flexible enough to fit their needs. As the developer you need to decide which features the majority of your customers actually want as to not introduce bloat. Finally, you need to implement the features in a way that the user can understand easily and use without much thought. To achieve this goal, you have to think like a user coming to your application for the first time. Do these things and I'm sure your products will fly off the shelves.