February 29, 2008
...search was one of most interesting problems I worked on at Audiogalaxy. It was one of the core functions of the site, and somewhere between 50 to 70 million searches were performed every day. At peak times, the search engine needed to handle 1500-2000 searches every second against a MySQL database with about 200 million rows. It was frequently hard to design for more than 10 or 100x our current traffic (and our growth was so dramatic that there wasn't really ever time to spend more than a few weeks on the problem), so it wasn't until the 4th major iteration on the cluster design that I really considered the scale problems to be solved. Here is a diagram of the architecture I ended up with (component explanations are below)..."