I figured it was time to attempt to create a content site for my business. After all, just doing all these stuff for everyone and never promoting it seemed a little silly. So, I present you with the new DigitalCrowd site. I had been given the opportunity to mess around with Joomla, and while not a large fan of CMS systems (yes, I am using WordPress here) it makes it easy for sites which are pretty much static with little modifications of text now and then, just like this site.

I finally did it. It is what my idle thoughts at times where consumed with. A low-cost, yet highly available hot fail over solution for my web sites. Until the release of MySQL 5.x came along, the perfect setup, just wasn’t possible.

Today, I brought online a server located in Europe. It provides everything I imagined. Geo-distant replication of both web and database. Talk about sweet. Not only do I get the ideal fail-over solution should my US based server become unavailable, but I also provide a server on the other side of the globe for my users closer to that server to connect too. Which means, much faster and in some cases, 2-3 times faster access to data from my sites, improving overall user experience all over the globe.

In addition, since my “fail-over” is really a secondary live site, it fails over to the US based server should it become unavailable. If that wasn’t enough, the entire setup provides a sudo-load balancing situation. While, replication between the dual-master setups means that data from both servers is being written to both servers, it does spread out the read loads on the databases and the web servers themselves. During high traffic times, both servers can be put into DNS to distribute traffic evenly, independent of IP origin.

Last, but not least… I am working an a static mirror or representation of my sites, should the database go down (on both active servers) then traffic will either be sent to another server with a static representation of the site, or to the active servers themselves with this data. I haven’t decided on that entire detail yet.

UPDATE: (11/21/06) – This whole setup works better than I ever dreamed. It has kicked in on numerous occasions (mainly self induced issues) and works like a champ.

I started the upgrade on a good note. I backed up the databases “as-is” before the upgrade. That is very important. Failure to heed the backup advice always leads to Murphy’s Law taking effect. The quick and dirty way is to shut down the mysql server and copy the data files directly to another location. This is much faster than doing mysqldumps and is a ton faster if the worst case happens and you need to go back to those files. What would take hours upon hours with a dump takes only a few minutes with a copy. The bad news is, the mysql server is down and well, you get nasty error messages. But, only for those few minutes then your back in business.

Next, I copy over the primary web folder. Although I have a backup of this, it is nice to have the latest at your finger tips, especially when doing major upgrades like this round where everything finds a new home.

Now… The FUN BEGINS…

When it comes to backing up servers with lots of data, especially servers that are not within reaching distance, it is a horribly painful process. You must weigh numerous factors to figure out what is best. Even then, your options are very limited. Add in nightly backups, geo-distant requirements and the options get even more narrow. Now, figure in that you want to keep the process to a minimal (after all, servers are busy you know) and you want it all automated. Next, add in 40GB of data – not all of which changes nightly. Last but not least, it must be at the lowest cost possible, if not free (aside from current costs).

Oh and one other minor detail… the backup pipe must be equal in each direction. After all, you’d want your restores to be as fast as your backups. Right?

Because of my inability to settle for something “less than” I have terminated plans to take macosx.com down for 4-5 hours to go through the lengthy upgrade process on the site and introduce lots of new code and new layouts. Not something I am terribly happy with, but in light of the current issues that remain, I don’t feel that I will be able to nail them all out tomorrow following the upgrade and have time to address any new issues that come up. For some crazy reason, people tend to use websites different than the programmer uses them which always introduces glitches, bugs or obvious suggestions which seemed to have went unnoticed by the programmer.

At this time, I am pushing toward next Thursday evening which is Thanksgiving Day in the USA. I still have my work cut out for me to meet that date. Without dates (even if flexible) things sometime never come to completion.

I had hoped to get more completed this evening so that I can meet my goal date of Nov 17th (evening) to start rolling out the changes. However, I decided at the last minute to revise the entire location of all the different files, images and what not which I had not coded to do easily. As such, it has taken some time to implement my latest “time saver”. It didn’t save time now, but will be very helpful in the future and allow the entire site to support themes. I am not fond of sites with different themes, but some people want to view macosx.com at 1024×768 and not fill the entire screen. At some point, I will roll out a theme just for them.

The more time I spend here, the less I spend on the code changes.