We are already in 2007 and Sampa databases are running on SQL Server 2000, a 7 year old product!!! The rationale is the one that every business should use: It works, so why bother upgrading?
The answer came in two batches. First, about 6 months ago while working on our Geo-location database, I hit some walls with the size of the database (25 GB). But the final blow came last week with our stats database.
We collect about a gazillion datapoints from each website and each visitor of each website. That generates almost 1 GB of data everyday. Of couse this is raw data, useless unless processed. We have our own batch processing system that parse all this data, aggregates and save a portion of it to files and another on the database.
Every once in a while, the processed data either gets corrupted (because the logs were not copied properly overnight), or we decide to change our definitions of Unique User, Page View, etc. In both bases we need to reprocess all data back to May/2006, when we started collecting data. That is about 25GB of raw logs.
All the logs must be processed in less than 24 hours otherwise the day rollover causes data corruption. Currently, reprocessing all stats is taking about 10 hours. That is a pain because if anything goes wrong, I pretty much have to wait until the next day to apply the fix and try again, since the job cannot start between 7AM and 5PM because it will hit the UTC midnight (5PM PDT).
All that to say that we attacked the problem in three fronts:
- Optimize the code to use multi-threading to process;
- Bought a new server w/ 2-procs (dual core) and RAID 1 (mirroring) that hopefully will improve read speed.
- Upgraded to SQL Server 2005
SQL 2000 is great, but is old. The UI is old, the functionality is old, the SQL Query Analyzer is the Notepad of database management. I'm hoping that Microsoft did a great job on the UI and perf of SQL 2005.
But most of all, I hopde the migration is smooth. This would be a really bad time for a painful upgrade experience.
I'm the Co-founder & CTO of 
