Web developers the world over are always terminally worried about whether or not their application will scale orders of magnitude without keeling over.
It’s a funny thing, because plenty of those applications would have stood a fighting chance if their builders were not that worried. Worries like these lead to premature optimization, and premature optimization is a huge timesink. It also makes it harder to change your application once you become aware of what it was that you should have been building (as opposed to what you actually built). That’s because optimized code is usually larger and harder to understand than just ‘good enough’ code.
It works like concrete shoes, they fit perfectly but they slow you down and are very hard to change for another pair, and forget about reshaping them without breaking them.
If premature optimization would be the only bad effect from all this worrying that would be bad enough. But there is a much bigger problem underlying all this.
Working at scale means that you are already successful. But success has many dimensions and getting a lot of visitors to your website is in and of itself not enough to define success (at least, not for me).
If you want to be commercially successful chances are that you won’t need all that many visitors at all. You just need the right visitors.
If you’re not making a media play for eyeballs and advertising dollars (a losing proposition in my opinion, unless you really know what you are doing) and you’re not trying to be the next big thing in social or search the only thing that matters is conversion, and the ‘funnel’ starts well before the visitors hit your website. And that’s exactly where the problem lies with a great many websites that have scaling issues. They have millions (or even tens of millions) of freeloaders but hardly any paying customers.
Now in the media world that may be acceptable, after all, eyeballs can be exposed to ads and that way you could possibly get to monetize them (no guarantees on that one). But if your game is not eyeballs, if it is e-commerce then the most important things that you should concentrate on are:
end-to-end tracking of your customers
first impressions
page load time
ease of use
the bottom line
Sure, there is a distant relationship between scaling and page load time, if you do things in a terrible way then you’ll lose your customers before they get the chance to become customers.
But in general, a single well configured machine running something as terrible as PHP and mysql (just to name a few popular scapegoats) can make you a fortune in sales. And if your margins are good you’ll be cleaning up. With maybe a few thousand to a few tens of thousands of visitors per day you could be making well in the millions per year or more.
And you’ll be wealthy enough to buy yourself a big fat server in case you decide that you need a bit more speed to cover your growth.
After that - and only then - if you plan on growing further you may have to add a load balancer or some other means of making your server park larger/faster without a complete re-write. This is known as a Ferrari problem. Until then a well configured server (with a proper back-up mechanism), or maybe two is pretty much all that you need to make a go of it.
As long as you make sure you get the right traffic.
If you need to worry about scaling issues in an early stage the first thing that you will want to investigate is whether or not the business is viable to begin with. If it really takes millions of visitors to make a go of it, you may simply be in the wrong business.
Content/eyeball plays are hard to do right, they require a lot of knowledge and tons of hardware or a service like AWS, app engine, engine yard, heroku etc. Now those services make deploying something ‘scalable’ easier than it would be otherwise. But the devil is in the details, once you start to rely on them for your scaling you can’t easily back out anymore and by then you’ll realize that what seemed cheap to begin with is now horrendously expensive compared to running your own hardware (the only permanent benefit is that you don’t have to administer your servers, but one competent admin can administer 300+ servers (hi Rob!)).
Even lousy e-commerce sites make money hand over fist, and there is nothing wrong in identifying a niche and working it for all it is worth. You won’t make the cover of Time Magazine and you won’t be ordering a private jet but that Ferrari is a definite possibility, if that’s the thing you are hurting for. (college education for your kids is probably a better idea ;) ).
I’ve seen a (legal) business that fit in a shoebox and that makes > 0.5M annually with extremely little work, and fewer than 1000 visitors per day. You could run that on a linode node (their smallest version) without a problem.
Don’t worry too much about scaling. Make sure you get the right visitors rather than just any visitors and keep an eye on the funnel. An advertising ploy is a monetization strategy for businesses that have no product worth selling, there is lots of fruit hanging a lot lower than that one.