Spinn3r

Screenshots

In Spinn3r 3.0 we introduced a new admin console which provides more detail about our internal operation. From statistics about our crawler to web based versions of some of our core operations, the admin console is your ticket to an easy to use crawler.

monitor-crawler

Crawler performance: We expose raw crawler performance to our customers including number of feeds, weblogs, etc. indexed per hour.

monitor-content3

Content rate: We include statistics on the full firehose API. Spinn3r indexes about 100-200k posts per hour and stats on our current throughput is available in the console.

monitor-comments

Comment throughput: Numbers around the number of comments being indexed by Spinn3r including breakdown by hosting provider.

monitor-growth

Growth rate: Stats around our growth rate including number of sources added.

monitor-hosting-providers3-feed

Hosting providers: Hosting provider breakdown including percentage of each blog CMS hosting share over 24 hours and week including graphs for each CMS broken out over 24 hour and 1 week periods.

monitor-hosting-providers3-permalink

Statistics: All Spinn3r are exposed.

monitor-lang3

Language: Spinn3r includes a robust language classifier that supports more than forty languages.

monitor-links

Link API and link growth: Spinn3r also indexes links within our service so you can view outbound links, inbound links, and links by source.

monitor-pings.png

Pings: View statistics on pings (weblog updates) that we see from the blogosphere.

monitor-publisher-type

Publisher type: View stats on distribution of weblogs, mainstream media sources, forums, etc.

source-validate

Statistics: All Spinn3r are exposed.

permalink-history

Statistics: All Spinn3r are exposed.

link-history

Statistics: All Spinn3r are exposed.