ralphm's blog

Wednesday, 28 April 2004

TYPO3 & Jabber Publish-Subscribe

Stop pulling!

A while ago, herb contacted me because he was writing his thesis in Brazil. He said he was very interested in my work on Publish-Subscribe using Jabber. So, I said: Ask away. Just now he contacted me again, stating that he was very interested in testing out Mimír and that he was busy working on an RSS component for the TYPO3 Content Management System.

I don't know any CMS systems that well, but I am told that TYPO3 is a great, powerful, Open Source CMS. Herb's plan is to hook up TYPO3 up to Jabber, using pubsub to publish items to interested parties. Very nice! I've written about Mimír on a few occasions, and initiatives like this might bring us further to the goal of having news pushed to you instead of having to pull it yourself.

I've set up a pubsub node (mimir/news/typo3_test) for him for testing, and had the Mimír news bot subscribe to it for its accociated channel. The only thing he has to implement is some magic to send a snippet like this to the pubsub component:

<iq to="pubsub.ik.nu" type="set">
  <pubsub xmlns="http://jabber.org/protocol/pubsub">
    <publish node="mimir/news/typo3_test">
      <item id="{$url}">
        <news xmlns="mimir:news">

Of course, the stuff in curly brackets needs to be filled in. At the moment Mimír uses a simple proprietary namespace for holding the data, but I can imagine using a format like Atom for this instead. Anyway, herb has not coded anything so far, but I'll keep you posted of his progress. If anyone is interested, let me know.

By coincidence, I had been thinking about the future of Mimír again this morning (under the shower, good thinking place). Ideally, when systems like TYPO3 have pubsub capabilities, you would want to optimize the distribution of notifications. For now, the pubsub JEP only talks about direct subscribers, but it would be far more efficient in terms of bandwidth and speed to have a distribution tree, where the root of the tree is the original content provider, the leaves are the end users (human or not), and the nodes in between pubsub repeaters. Compare with mirrors of websites and the way real world gossiping works.

I think I'll be working on that next. Maybe we need a JEP on that. For verifying the origin of a notifications, you could use either trusted repeaters, or signed notifications.

On the BOFH channel

Using the eyes...

Giving the BOFH page another look, I noticed a little orange button on the bottom of the page. I could have saved myself the trouble of fixing the scraper, since they have a nice RSS 2.0 feed now. They have probably moved to a new CMS, which explains why all the URLs have changed, and got feeds along the way. Good for The Register.

Updating Mimír feeds

Missing out on things...

A few of my Mimír channels had been very quiet recently. One of them was the channel for the BOFH stories over at The Register. This channel is being screenscraped by a little Perl script that uses LibXML and LibXSLT to turn the HTML into a nice RSS file. Then this RSS file is simply aggregated by the modified Janchor aggregator that feeds Mimír. Aparently, they've also changed the URLs to the stories, so a lot of old stories came up as being new.

While checking if Janchor now did get the new BOFH feed, I saw a error warning for offline offlineDizzyd's blog, Gradual Epiphany. Turns out he's moved to Movable Type for holding his blog, and the URL for the RSS feed also changed. Seems he did not migrate his archive (yet?), so that is too bad. Need to ping him on that. Also, it seems he has not sent presence to my site's bot for over a week. Hmm.