I keep thinking about Last.FM's radio player and it's three button "Love, Skip, Ban" control. I want to put these buttons all over the place because they look to me like the bare minimum UI to build a rating system. So how about:-
- A Firefox status bar extension to the last.fm radio that makes these buttons always available
- The three buttons on Google AdSense displays so that the publisher can express preferences for particular ads.
- Put the buttons on a TV remote and link them to Tivo.

Which then got me thinking about Amazon and their recommendation system. To make it work, you have to go through and manually rate items you already have. But I only have a 3 state reaction to a lot of things, not a 5 way grade. I love it, it's OK or I never want to hear/see it again.

Big Champagne track P2P downloads and produce a BillBoard-esque top 10. As you'd expect the top 10 mirrors CD shipments. It would be interesting to do some long tail analysis of their data.

Key questions:-
- Total number of unique files in a week
- Total number of files that are of music that has been deleted from the music Biz catalogues or are just not in commercial catalogues and so are unavailable anywhere else.
- Total number of files that are unavailable on commercial downloading sites like iTMS, etc.

I'm sure you could think of more.

I went to San Diego last week and while I was out there bought a Fujitsu 80Gb 5400rpm drive for the laptop. I also got a USB2 laptop drive enclosure. So over the last 24 hours I've been swapping and upgrading disks across 3 laptops. Just to recap, I was starting with a recent Toshiba laptop with a 30Gb disk configured as FAT32. The disk was pretty much full with about 1.5Gb free.

I started with my own laptop. The new drive goes into the enclosure and is connected via USB. I opted to start with Norton Ghost. Two nightly attempts at this failed with a memory error after a good 5 hours of running. Not good. It seems that Ghost does a pretty good job of copying volumes across as long as you don't take the option to extend the partition to use the extra space. So what I finally did was use Ghost to copy the main volume into completely unallocated space on the new drive and not attempt to extend it. I then swapped the drives so the new one is in the laptop. Then I ran convert.exe to change it from FAT32 to NTFS. The final trick to extend the partition across all the available space was to boot the machine from a Knoppix CD and use QTParted to adjust the partition. It may seem strange to use Linux to modify a Windows machine, but I didn't want to buy Norton Partition Manager as well as Ghost. In retrospect, there probably is a way to use Linux utilities to do the initial copy but QTParted doesn't do partition copies and Partition Manager on Knoppix is a command line utility and not at all obvious. And I haven't got my head fully around using Knoppix and windows/samba networks.

So then it's on to my son's Fujitsu Lifebook which apparently has a 20gb disk. Firing up Computer Management, Disk Management, I was amazed to see that it had actually had a 60Gb disk and the person who installed XP had partitioned it as 20Gb NTFS and 40Gb of unallocated space. duh! Again, 10 minutes of booting into Knoppix and QTParted had a single 60Gb partition. Result!

The third machine is a quite old Dell Inspiron 4000. This has a 20Gb disk and out of all the work above, I had a spare 30Gb in the USB enclosure from my machine. Unfortunately as well as being quite slow, the Dell only had a single USB1.1 port. Trying to do volume copies across this was going to take half a day. So this time I used the network. Ghost did a backup across the network to space on my main laptop. I then used Ghost to restore this (without changing the partition size) onto the USB enclosure using my main laptop. Now I've got a 20Gb bootable disk with 10Gb spare. Next, swap the disk with the Dell. Convert to NTFS. Finally, boot the dell into Knoppix and adjust the 20Gb partition to 30Gb.

Finally, I've got the Dell's 20Gb disk in the USB enclosure so that gets cleaned and reformatted and can be used for backups or portable storage.

So here's some lessons in all this.
- 2.5" HD USB2 enclosures are cheap and damn useful. In the UK they are about £20 ($20 in the USA)
- 2.5" 5400rpm laptop drives are now pretty cheap (< £100, < $150) and available in up to 80Gb sizes. 100Gb drives are just appearing but all seem to be 4800rpm.
- Norton Ghost will do volume copies from one disk to another or from the internal boot disk to a USB disk but seems to have problems with adjusting partition sizes afterwards.
- This may be because adjusting a FAT32 partition seems to involve moving and adjusting every file, whereas adjusting an NTFS partition only involves changing the partition table (as long as the partition doesn't move).
- So, if you're on FAT32, switch to NTFS as early as possible.
- If you get a new machine with Win XP installed, don't just assume the disk has been partitioned properly. At least have a look!
- Knoppix is an incredibly powerful rescue CD for Windows machines. It's free. You've got to find you're way round a whole new OS. But it's ways of doing things are not totally alien. And the utilities included are really powerful. I'm sure there's a way of doing partition copies across disks but I haven't worked it out yet. Assuming that's possible there's really no point in buying Ghost for this task. Though you may still want Ghost for it's backup abilities.
- Make sure you've always got a good copy of whatever you're doing before doing anything that could potentially destroy all your work.
- If you're not comfortable with hacking around like this, don't even start. Take the laptop and new drive to a shop and pay them to do it.

I think I end up where I was when I started this. I'm somewhat surprised and disappointed that XP doesn't have the tools built in to do these sort of tasks. Pretty much anything you do with Fdisk or Disk management is destructive and you lose data. There's no utility to copy partitions. And backup (at least in XP Home) is pretty limited.

Delicious Linkbacks
How very neat. Go to a web page, hit the bookmarklet and get a list of the del.icio.us pointers to it. [from: del.icio.us]

Yahoo! Search for Creative Commons licensed content. You can search for content that is free for commercial use and/or where you are free to produce derivative works.

New set of Search APIs so you can develop programs that interact with Yahoo's search.

Blogging and social networking.

Acquires Flickr. for photo sharing and publishing.

Upgrades mail service to 1Gb

Introduces a virtual market for ideas Yahoo Buzz is a notional market where you can track, buy and sell, and create futures in notional ideas with notional money. Tech Buzz is the same thing for technical products and services.

If it wasn't that Google search and news were still better than Yahoo's, I'd go long on Yahoo and short on Google. And then there's all those really annoying ads on Yahoo. [from: JB Ecademy]

Tom Coates has another big idea. plasticbag.org | weblog | Social Software for Set-Top boxes...

This is TV meets last.fm

drmblog DRM - Digital Rights Management
Just Say No To DRM [from: del.icio.us]

When I was at school, we did a business game course to try and get us to understand what manufacturing was all about. My group decided we were going to liquidate everything and go out of business. So in the last 3 rounds we stopped manufacturing anything. Our production costs went to zero, our warehousing costs wound down, while income remained the same. We didn't win but we came second for money in the bank.

So what happens when the Music biz realises they are screwed and decides to go out in a blaze of glory and with money in the bank? The plan (as explained here before) is to digitise everything they've got in the archives that is mastered or at least in more or less final form. Put it all up on an AllofMp3.com style site where you can download it in your choice of encoding and with no DRM. Set the price at around 5 cents a song. And go for broke. See if they can get at least 1 billion downloads in a year. this would bring in huge amounts of money but it would also as a side effect flood the P2P networks with high quality, properly tagged MP3s with no DRM.

So they'd be using their entire history as seed capital for whatever the next big idea is and in the process take down the whole current distribution chain of hardware CDs.

Isn't a supernova better than going out with a whimper?

Had an idea last night for a web site called ThisSucks or NoClothes or such like. It would be for people to expose the emperor with no clothes. To be able to open the window and shout "I don't care what everyone else says, This Sucks".

The plan is for people to post to del.icio.us with the tag ThisSucks/their_tag. The site would then pick these up and republish them via RSS and probably via a lazyweb style trackback as well.

I'm probably not going to do this, but maybe by the power of lazyweb somebody else will.

And here's my first.

Google Adsense Sucks for Bloggers!

National Speed Offence Database
[from: del.icio.us]

Clay Shirky
What we think we know about categorization is wrong. Because we're holding onto old outmoded techniques for categorization.

Q: what is Ontology. A: It depends on what the meaning of "is" is. The study of what exists in a domain and how do these elements relate.

The parable of the Travel Agent.
Travel agents exist to distribute the interface between a handful of airlines and a large number of consumers. The web replaces this so the TAs claim they add value. What's surprising is that the internet plays tried to use the same argument. They tried to recapitulate the old order rather than undermine it. It took some time for people to realise the problem had changed.

Classification schemes. Periodic table. Best classification scheme ever. Almost perfect. Context shifts where a whole column were labelled "gasses" where that's only true at some temporary ranges.

Libraries are the commonest classification system. And have huge fundamental mistakes. eg Dewey scheme category religion is all Christian. Library of congress treats Asia and Switzerland as equivalent in size. The essence is actually "number of books" about this topic. Optimises linear shelf space. Not reality. Unfortunately librarians now are using the same approach in the digital domain where shelf space is irrelevant. The argument like travel agents is that they are recapitulating what went before instead of undermining it.

Yahoo grew into a hierarchy of categories. So they hired a professional ontologist. Who built a huge tree. They said "we understand this better than you". They felt they couldn't organise the world without the shelf so they added the shelf back in. And so we get a tree structure. But the world isn't tree structured. So add a few cross links. So let's have a hierarchy with lots and lots of links. But the ontologists said "get outta here" and limited them to a maximum of 3.

In reality, there are lots of links and no tree. And Google took over because there is no filing system. There's only links. Google bought DMOZ, but nobody used it so they downgraded it.

When does ontological organisation work well? Small corpus, formal categories, stable entities, clear edges, coordinated users, expert users, expert cataloguers, authoritative source. Note: ontologists often claim the users don't understand the categories. And see this as a user's problems.

Turn it around and you have where it works badly. And that is a perfect description of the web. Huge scale, uncoordinated users, no authority.

Voodoo categorisation. Act on the model and it changes the world. Classify an SUV as a small truck and it becomes popular. Signal Loss. Ontologists claim that synonyms fail. But actually synonyms refer to different things.

Predicting the future is hard. A. This is a book about Dresden. B. This book is about Dresden, and goes into the category "East Germany". Ooops. Countries are radically different to cities. One is an idea, the other is physical. But we can't change it because we don't have the staff to move the books. Absolutely key. Categorization requires predicting the future.

"My God, it's full of links". Adventures in scale pt.1 Don't merge categories, merge the GUIDs.

Great minds don't think alike. Adventures in scale pt.2 del.icio.us. power law distribution of people and numbers of tags they've done. Long tail. classic sign of an unconstrained population behaviour. Look at number of entries for tags for one person, and it's another power law. 10% of the tags have 90% of the entries. Now look at 2 URLs and study the tags used against them. A lot of entries have very clear convergence. Some URLs have classic power law curves with less consensus. Which gives us a measure of the certainty of the popular tags.

Organic Categorization
- Market logic: individual motivation but group value.
- Merged from URLs (links), not categories
- Merges create overlap, not sync
- Merges are probabilistic not binary
- User and time are core attributes
- Signal loss comes from expression not compression
- One off categories are ignored, rather than deflected. Filtering is after the publishing. (very deep idea here).
- The semantics are in the users, not in the system. Does the world make sense or do we make sense of the world. Objective vs subjective. Recognises that there are alternate views.

(note: If you don't understand Unix, you are doomed to re-invent it. There is only World, Group and User)

In a primary school that had no server. How it works.
- The teacher runs Instiwiki on her iBook.
- Students find the wiki on the lan via rendezvous
- Security is simple. Turn the laptop off.
- Students and teachers can easily find their work.
- The app is very responsive
- This solution is not supported by the local T department!
- Teacher and laptop need to be present. Which is tricky when she's sick and their's a temp.

The whole thing started as a weblog post. Trying to create an OSS school administration platform. All REST and built on Zope.

Clay Shirky. Teaches a course for computers and art. Aimed at Artists and creatives who aren't afraid of the machines. Looking for things which ought to exist and trying to bring them into reality

The students are running point and they started to involve phones into what they were doing. Now more formal.

PacManhatten. Big Games Class. Tried using GPS but in an urban environment it doesn't work. So fell back on two way voice, over phones. A control room controls a runner.

ConQwest. Joint venture with Qwest. Combines 2d barcodes (semacodes) with cameraphones. Automated server interpreting the semacodes and sending data back to phone.

Dodgeball. First mobile social software. Problem is that you have to tell the system where you are. But it is SoSo that becomes part of real life. First problem was the ex-girlfriend bug. Your ex is still a friend of a friend so you get messages suggesting you meet!

Mobjects/Hearbeat. Bluetooth huggable piece of soft plastic. Send a hug to someone and their Mobject glows and demands a hug back.

Phone hardly used as a device.
- Standard connectivity beats local flexibility.
- Only the minimum latform is widespread.
- SMS had in USA (doh!)
- Develops lack experience and tools
- Device manufacturers unfamiliar with hackishness
- OWNERZERD by the US carriers. (same in UK).

- Server infrastructure is key. Pry data out of carriers.
- Out of band (eg Flickr) allows complimentary value.
- supports CPU-intensive post-processing
- Phone # is universal primary key

Underuse of Voice
- VoIP.

Mesh is coming but not soon.
- Dodgeball is social mesh on point to point links.
- Multi-network coming. Bluetooth, Wifi

Note here the differences with the UK. And the recent announcement that Broadreach WiFi hotspots would be free for Skype only. So Skype + Skype in and out + wifi + PDA + Broadreach = free encrypted phone calls with presence.

I was just walking through the lobby and came across a Brit I didn't know. He's got his Apple iBook open facing outwards with a camera on the top running iChat and the remote video on full screen. On the screen is Suw Charman in the UK (a regular on IRC #joiito) talking and listening as he walks round the conference. Hi Suw!

In the hall is a big Apple screen. It's running the feed from a chat session. The chat is full of bots that are scanning the crowd for Bluetooth devices and then looking up the IDs on Google and matching them with people's websites. It's also getting a feed from Technorati of all the blogs and photos people are using to document the conference. You should be able to see this at http://etech.inroomchat.org/chatlogs/

The Brits are a little subculture within the conference. They're constantly giggling and cracking jokes about obscure UK TV programs. During the last corporate presentation from Nokia there was a mass IRC coordinated walkout.

There seem to be very few Bluetooth headsets here compared with the UK. But everyone has a cellphone pressed to their ear.

Out on the streets, the Californians are all trim and lightly tanned. Very few *big* people compared with the rest of the US.

Clay Shirky (commentator), Stewart Butterfield (Flickr), Joshua Schachter(del.icio.us), Jimmy Wales(wikipedia)

Wikipedia categorization started last summer. English was chaos for some weeks. It self organised quickly but took a while to rationalise.

Stewart. Tags are not necessarily a replacement for categories. 200,000 tags.

Joshua. del.icio.us started with a personal text file with 20,000 url entries. Then he started adding #tag on the end so that he could do search and replace. Then it became a web site. Then multi-user. What's interesting is community behaviour where people group round a common tag that means nothing in itself. JB: I love this! This is how open source software gets written. It starts with a personal itch that you can't stop scratching. And often because the simplest possible tool you're using doesn't quite cut it any more.

Flickr: People using the comments attached to a photo or tag to have a conversation. So the tag or photo becomes a placemark for an on the fly discussion board.

Q from Marc Canter: Can we share tags across systems? Technorati already doing that. (Incidentally, arc keeps asking this and I don't get what he's asking for)

There are no bad tags. As long as they are useful for the user and there is feedback they will tend to be good enough. There is still a UI problem with finding things tagged with say Java when people used JDK. making it useable relies on clever UI around "Related" tags. "the point wasn’t to let you find all and only pictures of elephants, it was to give people better tools for organizing their own pictures, it was a happy accident that it worked across users. "

James Wales.

English >500,000 articles. In 350,000 categories. Higher Alexa than NY Times. 500 million page views per month.

Wikicities.com Social computing successor to home pages. 170 communities formed in 3 months. Growing faster than Wikipedia did. Users free to license content however they like.

Wikis Solve Author Fatigue: anyone can take over. Quality control: Everything can be peer reviewed and improved.

Organisation by the community: Software does not force how the community works.

Wikipedia is a social innovation not a technical innovation.

Justin F. Chapweske Founder of Swarmcast.

The trend is towards larger and larger files. We're now at the stage of moving DVDs around over the net. But http doesn't work for very large data files. Probability of failure tends to one as file size grows. So is there an equivalent to RAID for moving large files? Fault tolerance doesn't help. Load balancing still has a single transaction happening on a single server. Akamai costs 20-30 times normal bandwidth. The cheap version of using mirrors introduces security risks and the user experience sucks.

Hence Swarming. Ad hoc CDN (Content Distribution Network). Transparent: Looks no different from normal http. Opens up the possibility of swarm streaming. This should become ubiquitous. Uses simple additional http headers.

Best Practices for web design to support this.
- Static elements

Public test edition available. http://onionnetworks.com/products/swarmstream/

JB: I have my doubts about this. I'm not sure why people will install this in their browser. Maybe it needs to partner with the pr0n industry like peercast.

Cory Doctorow. He's reading a speech which will be posted on the net. I'll find the URL.

http://www.boingboing.net/2005/03/16/corys_talk_from_etec.html

read it. This one's important.

Day 2
Neil Gershenfeld. Director of MIT Centre for bits and atoms.
State of the Art in Fabrication. $10 billion chip fab. Just spreading stuff around and baking it like we've been doing for thousands of years. Just with finer detail. But it's nothing compared with Ribosome fabs and DNA. Passes all the tests of computing. undamentally digital process.

So they're working on hardware fab where the materials and process are a computer. The spec for the design is implicit in the materials. Working downwards from cm to micrometers to nanometres.

Back to communications theory. Shanno showed that there is a threshold. With some error checking, there is a threshold. Below that you can get zero errors despite finite amounts of noise. Now the ribosome-DNA process has error checking built in and so can replicate with zero errors. This shows it's possible to have hardware fabrication with zero errors if you take a computational approach. The challenge is to do it at all size levels.

So MIT started a course, "How to make almost anything". Some projects: A scream body (scream anywhere in silence into a back pack and then play back the scream later). A browser for parrots. Defensive clothing to protect your personal space. It turned out personal fab means you can produce anything for a single individual. Laser Cutter+Sign cutter+small micron milling machine+automated electronics assembly. for $20K. An approximation of a personal fab plant. Labs like this now all over the world including the 3rd world. He's showing Ghana street kids using a lab to build stacking bots. In India building milk analysis machines for rural farmers for $1 of parts. Norway building mesh network sensors to track reindeer. India-Pakistan border putting up internet enabled qounset huts. Gives people something to do instead of shooting each other. Now using the labs to make the huts self-reproducing.

What's needed is distributed VC to help fund distributed fab design.

Personal Fab is now at the PDP11 era from the 70s but it's moving really fast.

Bits to atoms discussion
JB: This stuff reminds me of SF novels. Imagine a garbage can in the corner. Pour grey goo into it, download a program to the can and out pops a microwave.

Tim O'Reilly , Dale Dougherty, Bran Ferren, Neil Gershenfeld, Saul Griffith

Squidlabs. A startup to build personal fab plants in SF.
Applied Minds. High pressure rapid hardware prototyping. Building tools to support this. Merging creativity wth engineering by starting at the model, rather than the spec. (JB: extreme engineering to parallel extreme programming. There's a note there that the first time an engineer uses a CNC machine, they can't believe how dumb they are.)

The ideal workshop is one which lets you quickly build the tool you need to build the thing you actually want. eg Using a computer controlled water jet cutter or ink jet printer to feed cad designs straight through to complex physical shapes. I get the feeling that when Apple gets bored with music and video, we'll see them produce iFactory! See also ifabricate from last night. A community web site to break down a big hardware project into sub-routines that people can describe individually; When building a bicycle, somebody else can design the wheels. So a project becomes assembling designs from other people.

An aside "Just in time education instead of just in case education".

iFabricate.com
Really rather cool bunch of Australians doing OS hardware fabrication. Like Bicycles. [from: del.icio.us]