Save $45k Per Month!
Running a web crawler is costly. Bandwidth, hardware, support, continued development - it all adds up.
|Item||Description||Cost Per Month|
|100Mbit per month of high quality bandwidth||Bandwidth isn't cheap. Spinn3r is constantly scouring the web looking for updated content. RSS, Atom, and newer specifications make indexing the web less expensive than in the past, but these protocols are still inherently wasteful.||$1,500|
|30 servers @ $750 per month in a state of the art datacenter||Spinn3r indexes 150GB of content per month. We maintain 18 months of archives. Three copies of every piece of data. This runs across thirty servers performing a range of crawl frontier management, database indexing, and robot content processing.||$22,500|
|Administration costs||Once deployed these machines don't admin themselves. Failed network cards. Crashed hard drives. Melted motherboards. All of this can and does happen in production. You need a team standing by ready to fix these issues so they don't knock your site offline. Spinn3r handles all this for you.||$10,000|
|Geographically distributed backups||Freeze your databases, take them out of replication, upload them to S3. This costs bandwidth and storage and it's certainly not free. Spinn3r maintains external backups in case of total disaster. This allows you to sleep safe knowing the data you need for your mission critical application is safe and sound.||$1,000|
|Continued development costs||The live web isn't static. It's constantly evolving with new protocols and changes needed to support new hosting services. Continued development is going to be necessary. You're going to need to hire one to two developers just to stay in the same place.||$10,000|
Total saved per month: $45,000
Features: Technical Specification | Spam Prevention | Content Extract | Public Statistics | Screenshots
Copyright © 2005-2012 Tailrank Inc (Spinn3r). All rights reserved.