Progress on tcp.im  ... Progress on tcp.im  [thanks, Scripting News] WS (SOAP, XMLRPC etc) protocols over IM has enormous potential not least because it can indirectly address any machine out there, neatly bypassing our NAT and firewall problems. This is going to open up all sorts of possibilities as finally we'll be able to more or less guarantee that when we need a call back, the other machine will be able to talk to us. It's great to see several initiatives working on this around Userland, Jabber and AIM.

As always there's some downsides. First the IM market is fragmented, which means we really need support for MSN, AIM, ICQ as a minimum. Just a SMOP right? Then we have a bigger issue about async, non blocking code and some assumptions built into the RPC view of WS. IM generally assumes that both parties are online and that messages flow from one to the other more or less instantly. But this is not necessarily true. Apart from inherent latency, some IM systems support a store and forward model more like SMTP. If we go into this with the RPC mindset then IM will be treated just like http and we'll write lots of code that assumes an immediate response. The code we write will be blocking and will sit there waiting for the response until timeout. Wouldn't it be better to take notice of the "messaging" part of IM and deliberately build this as a message queue architecture instead of an RPC architecture? There's some good work being done on SOAP over SMTP and IM is enough like SMTP that we should be able to use a lot of common code and effectively bring on both at once with common entry points. 

Now the SOAP SMTP spec suggests using the existing SMTP message IDs and in-reply-to headers to provide the control over which message refers to which call. But of course IM doesn't implicitly have any of this with different implementations across different IM systems. It's going to look more and more unfortunate that IM is a proprietary protocol mess with competing standards rather than IETF RFC driven sanity like smtp, pop3, irc, nntp, http et al. 

WS over IM is indeed a mind bomb. But watch out for the gotchas.

EU to tax e-commerce with the US As a good EU VAT payer, I laughed at this until I realized that this would affect me. Voidstar is hosted with a US hosting company and paid via credit card. From July next year, I'll either be chucked off because they haven't registered in an EU country or have to pay VAT. I can understand the reasoning but I also can't imagine how this is going to work.

05/02/02 21:07 CEST A new blogging company.As covered in Microcontent News, Nick Denton, co-founder and former CEO of Moreover (not to mention venerable British blogger) has announced his new venture, which is going to address, in his words, the "need to make weblogs more accessible: to turn them from a cult phenomenon into mainstream media, without corrupting the form." Of course, it'll only be an 80% company. [thanks, EVHEAD [via News Is Free]]

Go live is early 2003. Come on dude! What happened to Internet time? Or is the wait because he only wants to work a 4 day week?

Hell hath no fury like a FriendReunited. Well it made me laugh anyway.

O'Reilly Network: How I Learned To Stop Worrying and Love the Decepticons [May 01, 2002] : How I Learned To Stop Worrying and Love the Decepticons

Alan Cox attacks the European DMCA Wake up call [thanks, The Register USA] If you thought the DMCA was bad, just look at what the EU is cooking up to try and keep us in line.

Ben Hammersley is writing the O'Reilly book on RSS. Content Syndication with XML and RSS

The Internet is for everyone. RFC3271 Even Martians.

Need To Know 2002-04-26 presents Michael Greene's Grammy speech "The Insidious Virus of Illegal Music Downloading" turned into a rockin' Drum and Bass track. RIAA, We Love You That's a big shout to DJ C0ntaX for that one.

The England World Cup footie saga is a laugh isn't it? First it's Beckham's foot, now it's Sven's dick! Simon Barnes in the Times had it right. Ulrika-ka-ka-ka Johnsson "If she was a tennis tournament she'd be the Swedish Open"

There's a debate going on about programming approaches to Web Services that allow them to be extensible and not brittle. There's one solution that I whole heartedly support. "Use named params. Ignore parameters you don't recognise. Default parameters that are not passed". This is what we did with CGI and in that environment it just works. It allows WS developers to add extra parameters to server code later without breaking existing clients. It allows servers to drop parameters when they're no longer needed. It allows clients to call the WS when they don't have all the data required. 

But unfortunately several server toolkit implementations break if a full set of parameters are not supplied. And it forces a particular style of encoding in XML-RPC that is supported but isn't obvious in the spec. There seems to be a mindset here that equates WS programming with local object programming in compiled languages. It may even be inherent in the "Obect Access" initials part of SOAP. Dynamic Scripting languages like PHP and perl have understood this for years.

If you hard code the WS call as a fixed list of required parameters and break when they're not all there or an extra one is present, you're storing up problems for the future.

Great speech from Bruce Sterling. Viridian Note 00309: CFP 2002 Speech : "Where do you want to go today, Mr. and Mrs. America?" "Hey, I want to cruise in Steve the Dell Dude's borrowed convertible, playing borrowed MP3s!" "But no no NO, that's not what we meant! We meant, where do you want to go today, to GIVE US SOME MONEY."

Brent Sleeper writes "the core web services standards must not be subject to patent restrictions." Absolutely.

I've been trying to puzzle out an equivalent of Google's PageRank but for people instead of web pages.

Something between "6 degrees of separation", Advogato's trust metrics, and PGP's web of trust.

I know and trust these 3 people and you know and trust me so you can trust these three people.

And then tying it to a personal networking app. "Find me the top 10 people who specialize in web services strategy."

One key issue is trying to find or create an activity that is equivalent to the web page link to base the metric on.

Whew! That was close. My wife forgot our wedding anniversary as well... 

So Google have a SOAP API. This was just the excuse I needed to start playing with SOAP in PHP. And thus the game began.

First I had to choose a SOAP implementation. It turns out that php-PEAR has one that seems to be based on the well respected SOAPX4 I also wanted to start experimenting with some other libraries in PEAR and the code has a sort of quasi-stamp of approval from PHP as it's now in the main distributions. But for some reason PEAR wasn't installed on my hosting machine probably because I did it.

So the next stage is to update PHP from 1.0 to 1.2 and thereby hangs a tale. Now I host with Interland with their cheapest hosting package that gives me more or less full control. I get a virtual root in a sandbox but can access httpd.conf and such like. Interland told me that I couldn't update php because I needed full su root access. Hah! What do they know? Download php, untar it, work out what config parameters I need from phpinfo(), run ./configure etc, run make, run make install and let it fail. Sure enough, there is libphp4.so in the libs directory. Copy it to my own lib directory and then adjust httpd.conf so that the LoadModule php4_module line points at my copy instead of libexec. Restart apache and Bingo! php 4.12.

So now the next stage is to work out what happened to PEAR. It ought to go into /usr/libs but I don't have write access there. But I can see all the PEAR libraries in the php source tree. If I put it in usr/local/dah/dah/dah/libs then maybe I can get php to find it. Sure enough the php.ini file needs an entry include_path = ".:/usr/local/dah/dah/dah/lib/php/pear" Restart apache, run a short web page test for the pear DB library and it works. Bingo 2!

Now of course, so far I've glossed over several hours of trial and error and inspired guesswork, but the plan is coming together.

So now for SOAP. PEAR SOAP isn't in the php distribution but a quick Google search does turn up some minimal documentation and evidence of it being in the PEAR CVS. There's supposed to be a PEAR package manager that makes getting a new package easy. This is the point where you hit a web page on CVS PEAR install that says "This page not written yet". So finally I find the installed pear.in script that has to be moved and renamed pear and the instruction that says you need php to run it. Oh well, back to step 2 while I build a stand alone copy of php. Ok so I can see the executable. It's got execute rights. I type php, hit enter and get command not found. huh? It's right there. It takes me several minutes to realize that in Unix that should be ./php (stop laughing at the back) Makes complete sense. I have to tell it to run this php right here, not any old php it might find in the path.

So now php works, I can change the #! line at the top of the pear script to my copy, run ./pear So. list-available looks like a good option. Bang! xmlrpc support not loaded, error, error, error. ok, so we'll come back to that.

Then I remembered a note on a page about accessing Google API from php. "To use this code you will need both PEAR and the PEAR SOAP package installed somewhere on your php include path. You can get the SOAP package from CVS:" Have we got CVS? Yes. Type in the suggested lines. Bang! Some stupid CVS error that I can't be bothered to debug. OK. type the same lines into WinCVS locally. It all works, there's SOAP/, ftp it into the server's PEAR tree, fire up the sample php file, BINGO! Google search results! Try another search, Bang! No results. Ah-Hah! So when the note in the top of the example file said some returns don't work due to a bug in PEAR/SOAP parser it meant it.

So the next day, I have another look and the file now says "The PEAR SOAP package bug which caused some search terms to fail has now been fixed." So WinCVS, update, ftp, try again and Bingo!

So now I've got Google search results coming out of php code. Amazing.

But what was all that stuff about xmlrpc, package managers and such like. Well it appears that php now includes support for xmlrpc using Dan Libby's epi library. And because of that the PEAR guys assume it's there and use it in the package manager. And the php install docs say just use --xmlrpc(=DIR) as a config parameter. Well of course, I'd tidied up all the source, so download and untar the php source again. Add the parameter, check ./configure. Except that some of the xmlrpc responses look incomplete. Run make anyway, Bang! iconv.h missing. What? What happened then? Maybe I'm supposed to install xmlrpc-epi.c first. Download that, untar, configure, make, Bang! The Makefile borks with some impenetrable message. So it's off to the sourceforge pages, where I find a whole lot of equally impenetrable messages about how maybe FreeBSD is odd. Or maybe this or maybe that. But nothing that says "Do it like this and it'll work"

Now core PHP is really good. The docs are great, the code works, the install works. But step off the beaten track and we're back into Nerdland where it's assumed that you "Have a Clue", can debug other people's makefiles and can just know what to do by some weird telepathic osmosis.

Postscript: What I really wanted to do is put some cool Googley SOAP code into Theecademy.com But I have even less control there and they're running php 4.0.6 with no hope of getting an upgrade and no PEAR. But I copied all the useful files across, used a line in .htaccess to set the include_path. The test code said "requires PHP 4.1 or higher". Oh really? Comment out that check and it says array_key_exists() not found. And the php manual says "The name of this function is key_exists() in PHP version 4.0.6" So global search and replace through all of PEAR/SOAP and Bingo! Googley search results.

Netmaster 10baseT is in the house It's the BOFH Rap!

Yah gotta love this. Automatic blog entry creation with .:Autolog:.. For those days when you feel uninspired...

A dystopian vision of Life On the Net in 2004

Have you noticed how people have started referring to Yahoo! as Y!. Should this be pronounced Why Shriek? or Why Bang?

And what about the characters often found at the top of perl and shell scripts #! Is that Hash Bang ?