ralphm's blog

Monday, 3 May 2004

Will RSS Readers Clog The Web?

Polling all day long...

I was sent a link by herb to a story on Wired News: Will RSS Readers Clog the Web?. Like, me, some people wonder if RSS will scale when an RSS aggregator would be embedded in for example Internet Explorer. Dave Winer comments that if implemented properly, the check for new content is an "infinitesimal" burden. That is, using the appropriate HTTP 1.1 headers.

Still, Winer said he would love to work on a solution system when this turns out to be a real problem. He goes on about having a group of aggregator developers to talk about these issues. Well, my gutt feeling is that polling the original content provider every half an hour is inefficient by any standard.

On the other hand, Tim Bray talks about RSS By The Numbers, in which a Dave morse testifies that the traffic burden on his network went down significantly when he started syndicating information in RSS feeds. His information hungry customers can poll the feed for updates every 2.5 minutes, and still use less bandwidth than fetching the web page once. Provided that they use the right tools, probably.

I believe that whether you are going to use Jabber pubsub like I said earlier, or some other way of spreading the load, something should be done. Jabber enabled desktop aggregators will probably not be in wide spread use any time soon, but what about feeding RSS mirrors using pubsub?