ralphm's blog

Sunday, 31 May 2009

PubSubHubbub

Pushing it with HTTP...

PubSubHubbub is a protocol and reference implementation for doing publish-subscribe using web hooks, polling in feeds triggered by a ping from the publisher, and POSTing Atom entries to notify subscribers. The notification part is similar to what I've been working on for the publish-subscribe stuff at Mediamatic Lab, where we spiced up Idavoll with an HTTP interface to bridge the gap between XMPP Publish-Subscribe and HTTP speaking entities.

Although I spend a lot of time working on XMPP based publish-subscribe, I understand the reasons for going for a full HTTP-based approach. XMPP can be intimidating for developers of web applications. While the differences between XMPP and HTTP are important (stateful connections, asynchronous processing, etc), the fact that it is different is reason often enough. Hosting facilities don't always offer ways to do XMPP, and there is not nearly enough running code out there to make it easier for people to play with these technologies to spice up their web application with non-IM XMPP functionality. Having platforms like Google App Engine provide sending and handling raw XMPP stanzas as part of the API would surely help.

That said, PubSubHubbub has two separate sides to it, the publishing part and the notification part. There's nothing that prevents a hub to do the publishing part using regular XMPP publish-subscribe. Instead of fetching the Atom Feed over HTTP every time, it could use autodiscovery to find out the publish-subscribe node and upgrade by subscribing to it instead. Similarly, the notification part could send out XMPP notifications. Combined with existing HTTP aggregator, that combination is very similar to how the aggregator for Mimír works.

I'm still not convinced that PubSubHubbub is the answer to the efficient exchange of updates on social objects, but I do think it is a good way to make smaller entities be part of a federation of social networking sites. Likely, we'll see a hybrid approach, to begin with.

Social Web FooCamp 2009

The stuff we did besides throwing Frisbees...

Last month I was fortunate enough to attend Social Web FooCamp at O'Reilly HQ in Sebastopol, CA, a follow up to Social Graph FooCamp in 2008. I can't express how inspiring such events are, being able to have a continuous, in-depth conversation with so many bright minds about so many topics that keep you busy on regular days, and more. I'll give a quick overview of the whole trip, and then go into depth in a series of posts.

My trip started with a visit to friend and former Jaiku colleague Andy Smith, who was kind enough to take me in at Houseku. As soon as I landed on SFO, I got an SMS from him to make a detour to his office. Besides meeting a bunch of Andy's fellow googlers, I got to spend some time with Brett Slatkin talking about PubSubHubbub.

The next day I got a ride to Sebastopol from Edwin Aoki. After a trip full of interesting conversation, we arrived at the O'Reilly offices. Sebastopol was a lot warmer than San Francisco, perfect for camping. Lots of familiar faces, but also a lot of new ones. During the Friday evening, apart from the general introduction, I didn't get to any sessions, but instead spent talking to a bunch of people on XMPP, Publish-Subscribe and the work I am doing on federating social networks under that name Open-CI at Mediamatic Lab.

The next two days were filled with sessions and hallway talk on OpenID, OAuth, different approaches to Publish-Subscribe and inter-site communication, resource and service discovery and service scalability. While most of the topics were similar to last year, I was glad to share what we've done at Mediamatic Lab over the past year, while learning how others have fared. We used these technologies to make a true federation of social networking sites where you can make cross-site relations between people and their social objects. Some of our discoveries there we're shared among the participants, while others had interesting other approaches.

Especially interesting to me was a session on OAuth and OpenID where I could explain how we tried to improve upon the user experience. Both technologies have a bad reputation in this area. With some smart defaults and trust between sites, we could eliminate some of the screens. There was talk about using pop-ups in some situations, either as lightboxes or as new (small) windows. In our experience the former can't be used if you want to do SSL (since you can't validate the address and certificate). The latter was deemed confusing in our user tests. Research is still ongoing, I suppose. The other issue had to do with presenting OpenID providers. We currently use a drop down, but that doesn't scale up very nicely. Logos might work, but in the end has the same issue.

I also got to show Blaine Cook the code I wrote recently to make it easier to write XMPP publish-subscribe enabled services (code-as-a-node), that has been included in the recent Wokkel release. In turn, Blaine shared his thoughts on simple addressing on the web and we got to hash it out with a bunch of people like Brad Fitzpatrick, who also organized the pubsub shootout session. Finally, Eran Hammer-Lahav showed his work on XRD.

I'm pretty sure I forgot to mention a lot of things, but when it comes back to me, I'll write about it some other time.