Terry Jones

I’m sitting in a hotel in Berlin, the Hotel Ibis Berlin Mitte. They’ve done a deal with Vodafone to provide wifi access for their guests.

Here’s the price list:

• 30 minutes – 5.95 euros, or $8.66
• 2 hours – 12.95 euros, or $18.85
• 24 hours – 29.95 euros, or $43.59

There’s no option to connect/disconnect and use your time bit by bit. You have to take it all at once, making the 24 hour option particularly attractive.

Way to go Vodafone! You idiots. With bargain basement rates like these I will certainly keep coming back. Same goes for you Ibis Hotel. Typical phone company strategy – maximally fuck your customers in the short term.

I don’t mean to appear cold-hearted. I have a heart. Really. But news of a shakeup at Powerset given release delays doesn’t come as a surprise at all.

What is surprising is to read that Powerset has “been hampered by limited resources.” Oh puhleease. Since when has $12.5M (minimum) in funding qualified as having limited resources?

Delays in getting hold of the Xerox NLP API caused fundamental problems? I used that API (ten years ago, admittedly) and, sorry to say, it’s not the key to unlocking the natural language understanding puzzle. But it was widely trumpeted as the key to Powerset knocking off Google. The mysterious all-powerful NLP API from the mysterious all-fumbling Xerox PARC finally lands in the hands of a commercial company poised to Make Good! Powerset had snatched the NLP crown jewels out from under Google’s nose!

It wouldn’t surprise me if PARC were glad to get rid of the rusty old thing. “Psssst, buddy. You over there… wanna buy an antique NLP API owned by former royalty? S’good fer what ails ya.”

OK, I’m being a bit sarcastic and silly. I guess I just have limited patience for these projects and especially for the breathless hype that surrounds them.

I’ve often wondered about Powerset (and Metaweb) hitting the wall. Lots of hype, pressure, and funding. Lots of people. High burn rate. And revenue coming from…… where exactly? And that’s not to mention the blow to our confidence that Powerset were really onto something deep when they let a genius programmer drink and get away from his handlers at a dotcom-style bash.

I’d say the real reason Powerset are “hampered” is the fact that they’re trying to solve something that’s practically impossible.

If you look at it that way, then I suppose having only $12.5M to achieve the impossible really is a case of having limited resources.

Stay tuned. There’s a long, nasty and heartless blog posting locked up inside me about people and companies that chase words like “understanding”, “meaning” and “intelligence”.

First of all, let me just say that Jason Calacanis is a genius.

Having said that, I most humbly submit that his posting today on Facebook’s WORST two features is a little off in one regard.

I also hate the way Facebook tries to pull me into its world when they could so easily just deliver my message in email. But I don’t think they’re doing it for the page views.

I think they’re doing it because they want to take over the world.

Once there were PCs. Then came the mass migration towards online apps that run in one’s browser. Microsoft got that one right, though they were a little early in calling it (culling it?), and then ironically were present at the conception and birth of what really kicked it into high gear, XMLHttpRequest. But that’s another story.

So what’s next? Or, what would Facebook like to be next? Well, the obvious next step in the progression: mass migration to a particular platform running inside your browser. It just makes sense.

Except it doesn’t.

But that’s what I think Facebook will be going after. They want us all in there. That’s where we should be sending and receiving messages, IMs, poking each other, and, of course, throwing expensive virtual food. Why not twitter in Facebook? Why not Tabblo in Facebook? Why not watch videos inside of Facebook? Why not everything in Facebook?

There’s no way it can work, but I bet that’s what they’re going after – to as great an extent as possible. A bundle of cash will make for some nice acquisitions, as would an IPO. And everything they buy is going to wind up inside Facebook.

I’ll save my reasons for why it can’t work for another posting.

I like arms races. But in an interesting arms race there’s frequent movement on both sides.

So I’m often surprised that the measures people and programs take to obscure email addresses haven’t changed much over the last 5(?) years.

There are still many software packages and web sites that do the bare minimum to obscure email addresses. For example, here’s a recent interesting posting from Vaughan Pratt on Simple Turing machines, Universality, Encodings, etc. The mailing list software is the extremely popular Mailman system. Vaughan’s email is “obscured” as pratt at cs.stanford.edu. That approach is so old it can hardly be counted as more challenging for a spammer to harvest than if mailman had simply included the actual address.

And mailman is just one example. People do it too, using extremely transparent and repetitive schemes, like joe AT xyz DOT com.

Given how much people dislike spam, how easy the above examples are to extract, and how creative humans can be, I find it amazing that the practice of obscuring emails addresses has barely moved in the last years. Do you suppose the spammers are standing still? Well, maybe they are, given the lack of advance on the obscuring side.

There is some ingenuity, like using terryblah@flahmydomain.com accompanied by an instruction (in English) to remove all instances of blah and flah to get the real address. Given that humans are so creative with language and that NLP doesn’t stand a snowball’s chance in hell, you’d think the humans would have little trouble staying ahead in this race. But right now I expect the address harvesters have the upper hand.

Here’s another example.

To get my personal email address, join the second to the last four letters of strawberry, add an at sign, add the tenth letter, then put on “on”, then a period. You get the final part by dropping the last letter of the acronym for Eastern Standard Time.

I.e., that’s “terry” plus “@” plus “j” plus “on” plus “.” plus “es”. Yes, this is overkill, but it illustrates how easy it is to create highly personalized but simple instructions for a human to follow that no program is ever going to handle. Even if an attack on the above could be automated, it’s clearly not worth the cost just to get one email address.

Surely it’s time to move on.

My 7-year-old daughter was here the other day and needed to make a call. She decided to use my landline, but found the old-fashioned phone (the one Telefonica installed, new, about 18 months ago) very confusing.

The handset had no numeric keypad on it. Plus, very weird, it had a cable. Then, when you picked up the phone unit, it had a cable on it too. So she couldn’t bring it to me to explain her difficulty. So I gave her instructions. Put the handpiece to your ear. Do you hear a high-pitched noise? (No, a woman’s voice. OK, hang up, now pick it up again.) Now push the buttons on the main unit.

All so old fashioned, though at least there were buttons involved, unlike when she ran into an old rotary phone.

There’s also a nice symmetry. When I first got a mobile phone, in 1999, Ana noticed that I always stopped when I was making a call. She had to point out to me that you can just keep walking, no need to stop. Now I have my daughter here trying to make a call and getting into trouble when she tries to walk across the room holding the fixed phone – to her great surprise she found that you can’t just walk around.

Which reminds me of the time she pointed to a bulky (switched off) monitor on my desk and asked what it was. I was surprised – she’d seen me using a computer thousands of times, but only ever a laptop. Now that flat screens are everywhere, she may never really run into an old computer with a CRT monitor. But she will probably remember having seen one, just like I remember having to use punch cards for one programming task (which they made us do at Sydney University, just so we’d know what it was like)

OK, nothing deep today. But I must feed the blog.

[This taken from my comment on Fred Wilson‘s posting Andreessen on Platforms, in which he discussed Marc Andreessen‘s posting The three kinds of platforms you meet on the Internet.]

I think Marc’s posting has two flaws. The first, which is serious, is that he didn’t put enough thought into it. The second, less of a problem, is that in several places it comes across as biased and a bit of a Level 3 sales pitch. I may be guilty of the former in what follows. Certainly my reply is a bit piecemeal – but there are only so many hours in the day.

In what follows, when I talk about “you”, I mean you the humble individual programmer.

Firstly, things become clearer if we categorize Marc’s Levels 1, 2 and 3 differently. Level 1 and 2 are two sides of the same coin:

• Level 1: You write an app, and you call out to an API (a library of functions) that someone else has written.
• Level 2: You write functions, and an app that someone else has written calls you (treats your code as a library function it can call).

To me these things are opposites. Within Level 2, there are two classes:

• Level 2a: You write functions. An app that someone else has written calls your code, which runs on your server.
• Level 2b: You write functions. An app that someone else has written calls your code, which runs on their server.

My Level 2b is what Marc calls Level 3. I’ll continue to use his terms.

Note that only in Level 1 are you really writing a full app. In level 2 and 3 you’re writing functions that are called from an existing application (like facebook or photoshop) that you almost certainly didn’t write. To make you feel better, they give your functions pleasing names like “plug in” (photoshop), “extension” (firefox), and even “app” (facebook).

To me that’s a more logical division of the 3 classes. I see no reason at all to call Level 1 a “platform”. You are writing an app. You’re calling someone else’s libraries – some of them will be local, some will be on the network. You’re not writing a platform. The only platform here is in the local OS of the machine your app is running on.

If we stop calling Level 1 a platform, it makes that word much less cloudy. That means that things like Photoshop, Firefox, and Facebook (Level 2), and Ning, Salesforce.com, and 2nd life (Level 3) all provide platforms for you. But Flickr, delicious, the Google maps API, etc., are not platforms and calling them that is just confusing. They’re just APIs or libraries that other apps can call (across the network, in these cases).

—

Next, virtually ALL applications in operation today are running in Level 3 platforms. Most of them run in the environment provided by operating systems.

Once you look at things that way, you see that the thing which is important is the runtime environment provided by the Level 3 platform you are already running on. Is it fast, secure, scalable, flexible, etc.? Can you write the kinds of things you want to write with it? Should you try something else?

I think Marc didn’t look at his Level 3 this way, or at least not clearly.

Now, traditionally in the field of computing, there has been a single main way of providing a platform. You provided a computer system — a mainframe, a PC operating system, a database, or even an ERP system or a game — that contained a programming environment that let people create and run code, plus an API that let them hook into the core system in various ways and do things.

The Internet — as a massive distributed system of many millions of internetworked computers running many different kinds of software — complicates things, and gives rise to three new models of platform that you see playing out in the Internet industry today.

I don’t think they’re all platforms, and I don’t think any of them are new :-)

But let me say up front — they’re all good. In no way to I intend to cast aspersions on what anyone I discuss is doing. Having a platform is always better than not having a platform, period. Platforms are good, period.

Hey, all platforms are great. But some are greater than others…

Level 1 is what I call an “Access API”.

…

This is undoubtedly a very useful thing and has now been proven effective on a widespread basis. However, the fact that this is also what most people think of when they think of “Internet platform” has been seriously confusing, as this is a sharply limited approach to the idea of providing a platform.

Do most people think of things like the Flickr API as being internet platforms? If it’s sharply limited (I agree), then please let’s not call it a platform.

What’s the problem? The entire burden of building and running the application itself is left entirely to the developer. The developer needs to provide her own runtime system, programming language, database, servers, storage, networking, bandwidth, and security, and needs to take responsibility for running all of the above — and then exposing the application to users. This is a very high bar in terms of both technical expertise and financial resources.

This is painting an overly bleak picture. Almost every application programmer on earth uses an off-the-shelf runtime system (e.g., an OS or a Java sandbox), off-the-shelf databases, servers, networking, etc. Yes they choose a programming language (as they do if they choose to use a Level 3 system). It’s work to pick these things out and combine them but that’s a very far cry from shouldering the _entire_ burden.

This is an example of what feels like salesmanship in Marc’s article. He’s right in general, but the way he puts it feels slanted.

As a consequence, you don’t see that many applications get built relative to what you’d think would be possible with these APIs — in fact, uptake of web services APIs has been nothing close to what you saw with previous widespread platforms such as Windows or the Mac.

And this isn’t a good comparison. It’s comparing use of a Level 1 API to use of what Marc later tells us is a Level 3 system (a traditional OS).

Because of this and because Level 1 platforms are still highly useful, notwithstanding their limitations, I believe we will see a lot more of them in the future — which is great. And in fact, as we will see, Level 2 and Level 3 platforms will typically all incorporate an Level 1-style access API as well.

Right. In fact Level 1 platforms (aka APIs) underpin all of Marc’s levels. Which is to say that even if he’s right, the Level 1 “platform” isn’t going away or lessening in importance – that’s because it’s not a platform at all. It’s a API, and libraries of functions exposed as APIs are useful things to have around. Likewise, APIs on the local OS aren’t about to go away either – in fact they’re crucial to the operation of the OS, just as they are to the operation of a level 3 platform (which is also running in a Level 3 OS).

So Level 1 isn’t going anywhere, or getting less important.

When you develop a Facebook app, you are not developing an app that simply draws on data or services from Facebook, as you would with a Level 1 platform. Instead, you are building an app that acts like a “plug-in” into Facebook — your app literally shows up within the Facebook user experience, often as a box in the middle of a page that Facebook otherwise defines, such as a user profile page.

Here (as with Photoshop or Firefox), your code is like a library function you write that is called by another app. In this case, your code runs on your server, and the calling app (usually on another server, if it’s a web app) takes your results and displays them (often to a web browser).

Level 3 is what I call a “Runtime Environment”.

In a Level 3 platform, the huge difference is that the third-party application code actually runs inside the platform — developer code is uploaded and runs online, inside the core system. For this reason, in casual conversation I refer to Level 3 platforms as “online platforms”.

And here, your code is like a library function you write that is called by another app. In this case, your code runs on the platform’s server, and the calling app (on their server) takes your results and displays them (often to a web browser).

Obviously this is a huge difference from Level 2. And this difference — and what makes it possible — is why I think Level 3 platforms are the future.

And the past.

There follow a number of breathless paragraphs that describe exactly why it’s hard to build an OS, and what the advantages are once you manage it.

Then it’s acknowledged that yes, this is all… just like having an OS!

So those long paragraphs feel like Marc is either completely blind to an _extremely_ obvious and almost perfect analogy, or, like he’s a salesman trying out a snow job on just how incredibly amazing these totally new Level 3 platforms will be. It’s impossible to think #1, so I’m left feeling #2.

The Level 3 Internet platform approach is ironically much more like the computer industry’s typical platform model than Levels 2 or 1.

Back to basics: with a traditional platform, you take a computer, say a PC, with an operating system like Windows. You create an application. The application code runs right there, on the computer. It doesn’t run elsewhere — off the platform somewhere — it just runs right there — technically, within a runtime environment provided by the platform. For example, an application written in C# runs within Microsoft’s Common Language Runtime, which is part of Windows, which is running on your computer.

At which point you note that basically all programs already run in a Level 3 platform:

I say this is ironic because I’m not entirely sure where the idea came from that an application built to run on an Internet platform would logically run off the platform, as with Level 1 (Flickr-style) or Level 2 (Facebook-style) Internet platforms. That is, I’m not sure why people haven’t been building Level 3 Internet platforms all along — apart from the technological complexity involved.

But nothing is running “off platform”. It’s all already Level 3. Yes, there are differences in environment… coming up.

So who’s building Level 3 Internet platforms now?

First, I am — Ning has been built from the start to be a Level 3 platform.

…

Second, in a completely different domain, Salesforce.com is also taking a Level 3 platform approach

…

Third, and again in a completely different domain, Second Life is a Level 3 platform.

Fourth, Amazon is — I would say — “sort of” building a Level 3 Internet platform with EC2 and S3. I say “sort of” because EC2 is more focused on providing a generic runtime environment for any kind of code than it is for building any specific kind of application — and because of that, there are no real APIs in EC2 that you wouldn’t just have on your own PC or server.

Ah, there’s a very interesting bias…

The generic traditional PC OS is a Level 3 platform, despite the fact that it’s not specifically geared towards any particular use. But EC2/S3 are somehow only sort of Level 3 precisely because they have the exact same property???

By this, I mean: Ning within our platform provides a whole suite of APIs for easily building social networking applications; Salesforce within its platform provides a whole suite of APIs for easily building enterprise applications; Second Life within its platform provides a whole suite of APIs for easy building objects that live and interact within Second Life. EC2, at least for now, has no such ambitions, and is content to be more of a generic hosting environment.

However, add S3 and some of Amazon’s other web services efforts to the mix, and you clearly have at least the foundation of a Level 3 Internet platform.

I might argue this the other way round. Things like Ning and 2nd life and Facebook are trying to be real Level 3 platforms to allow people to build a wide range of apps (i.e., 3rd party functions that they call), but they’re only “sort of” true Level 3 because they’re built for a specific purpose and so are only useful for that purpose – even if the purpose is broad, like “the” social network.

Things that are more generic, like EC2 and S3, are more like the generic computational environment provided by a traditional OS. And for that reason, one can expect them to be used for a wider range of applications (including standalone applications, not just code that lives within the Facebook or Ning world). For that reason you might expect that applications written against them will be longer-lived, as they will not die as fashion and coolness moves its fickle hand from MySpace to Facebook to Ning to…?

Would you buy a used Level 3 platform from this man?

Fifth and last, Akamai, coming from a completely different angle, is tackling a lot of the technical requirements of a Level 3 Internet platform in their “EdgeComputing” service — which lets their customers upload Java code into Akamai’s systems. The Java code then runs on the “edge” of the network on Akamai’s servers, and is distributed, managed, and secured so that it runs at scale and without stepping on other customers’ applications.

This is not a full Level 3 Internet platform, nor do I think Akamai would argue that it is, but there are significant similarities in the technical challenges, and it’s certainly worth watching what they do with their approach over time.

Why is it not a full Level 3 platform? Because it doesn’t have a particular focus?

I believe that in the long run, all credible large-scale Internet companies will provide Level 3 platforms. Those that don’t won’t be competitive with those that do, because those that do will give their users the ability to so easily customize and program as to unleash supernovas of creativity.

Oh my!

But having already said that Level 3 platforms will need underlying Level 2 and Level 1, it doesn’t seem like the Level 3 providers are driving the lesser levels out of the marketplace.

One might instead argue that it’s the Level 3 providers who are most likely to disappear. We’ve seen exactly that happen in the traditional Level 3 world (operating systems), while some applications and many great libraries hop happily from one Level 3 environment to the next.

I think there will also be a generational shift here. Level 3 platforms are “develop in the browser” — or, more properly, “develop in the cloud”. Just like Internet applications are “run in the browser” — or, more properly, “run in the cloud”. The cloud being large-scale Internet services run on behalf of users by large Internet companies and other entities. I think that kids coming out of college over the next several years are going to wonder why anyone ever built apps for anything other than “the cloud” — the Internet — and, ultimately, why they did so with anything other than the kinds of Level 3 platforms that we as an industry are going to build over the next several years — just like they already wonder why anyone runs any software that you can’t get to through a browser. Granted, I’m overstating the point but I’m doing so for clarity, and I’m quite confident the point will hold.

But everything _already_ runs “in the cloud” on a Level 3 platform. Your local OS has far more functionality, more speed, more libraries, more space, more flexibility, etc., for you to run your applications in. OK, I’m being a bit difficult, and understating the point. Maybe.

Now to the main point, which I think is valid, but which Marc doesn’t answer.

Before we had operating systems with all their benefits (see the long list of benefits Marc tells us will accrue from his Level 3 – ease of use! open source! buying and selling code that just runs!) a forward-looking person could have looked ahead and predicted the rise of the operating system. What sorts of programs, what supernovas of creativity might they have predicted?

Marc looks ahead…

A new platform typically enables a new set of applications that were not previously possible. Why else would there be a need for a new platform?

But: keep this in mind; look for the new applications that a new platform makes possible, as opposed to evaluating the new platform on the basis of whether or not you see older classes of applications show up on it right away.

But give us no examples at all.

I’m extremely interested in this. What will these applications be?

Is it true that what we can build with these future systems is not “possible” without them? Or just not feasible? Where does their extra power come from? I think it’s NOT principally from the great diversity of apps that can be written to run on these platforms, but from what you gain by having a large number of apps running in the _same environment_ – be it in an OS with a file system, a process subsystem and communicating processes, or a Level 3 internet platform with whatever it provides.

In the fullness of time, whenever that is, we may see the rise of truly open internet Level 3 platforms that will challenge the well-funded closed commercial ones. Meanwhile, I’m happy to _only_ be working away at Level 1.

[This pulled from my comments and questions on Fred Wilson‘s posting Every Product Is A Platform on September 10, 2007]

My question to VCs and others is where you see value in having others build on an API. I can see some arguments – visibilty and branding, pushing maturity of the API, giving you an under-the-radar tap with which you can experiment with increasing traffic, maybe giving you ideas for products (if you’re the kind to take that route), finding (and then hiring) good hackers who love your product. These are all indirect benefits. I’m curious about why, from an investor’s POV, there’s value in having others build on the API. There are 250+ things built on the del.icio.us API. Were they of value? Did they increase revenue in any direct way? If you argue that there’s great direct value, can I therefore walk into your office, claim that thousands of people will write apps using my API and argue for a massive valuation? :-)

Do any of the companies offering an API have a strategy for monetizing it, or simply recouping costs for bandwidth, servers, etc.? Sure, the exposure is great. But, as I was once taught, you can die from over-exposure.

Here’s another way of looking at my question: if API traffic is 10x bigger than interactive web traffic, then just 1/11th of Twitter’s computing resources are being used to support their (arguably) most important customers. Maybe the site could have been many times faster if they had opened up API usage slower. I found the Twitter web interface unusably slow in the first 6 months after I heard about it – a feeling that many shared. Is that because they were actually using 90% of their resources supporting apps they didn’t write and didn’t benefit (directly, financially) from? That’s a very delicate line to choose to walk. At that level of diverting resources from normal users, there’s a huge risk blowing it. Hence my question about value. Sure, the 3rd party apps are cool and exciting – but are they so important that it makes sense to give you front-line customers a miserable time, making your service extremely slow.

To go to another extreme, imagine releasing an API that was so powerful that thousands of people wrote to it, but which had no user-facing component. How is that going to make you money unless you charge for it? E.g., Amazon’s S3. If you charge, like Amazon, I understand the model. If you don’t charge and the API is eating 90% of your resources, you may be shooting yourself in the foot rather severely.

It’s an interesting problem. As I said earlier, I agree with you that if you can do it, product should drive platform. Twitter could have followed that route, but apparently went the other way round. Or maybe things were just totally out of control and they unexpectedly found themselves in this 10:1 situation.

One thing’s for sure, if you’re using 10/11ths of your resources on your (non-paying) API customers, you should definitely make sure the rest of the world knows about it :-)

Russell and I met with Esther Dyson (a Fluidinfo investor) recently. After she’d listened to our presentation and seen the latest demo, she said that we’d “given the towel another half twist” and that we should carry on twisting.

She was referring to the process of tightening up and focusing company vision, strategy, business plan, etc.

I liked the analogy a lot. Twisting a wet towel is fun. It’s hard work, and it gets harder. But it’s surprising and satisfying to see just how much water you can get out of the thing before you let nature take its course and finish the job.

It also applies to writing documents. I spent most of 2005 writing a proposal to start a research institute for the computational study of infectious diseases (still in the works, though I’m no longer directly involved). Thanks to the repeated insistence of Derek Smith in Zoology at Cambridge, the document went through about 5 iterations, each more painful and difficult than the previous. It drove me nuts. But it was amazing how much better the thing became at each round, and the end result was hugely satisfying.

I’m going through the same process now with Fluidinfo as we prepare to raise our first round of outside financing. Putting together a slide show, executive summary, and demo is a ton of work. I’ve been round the loop a few times already. Earlier tonight I gave a presentation to Vicente López, general manager of the Barcelona Media Centre for Innovation. He poked holes in the presentation from start to finish. I took notes.

So I just spent the last 6 hours slowly twisting the towel. As a result the presentation is much improved. I figure we still have a couple of half twists left to do.

Meanwhile, I’ve paused to reward myself by knocking off today’s blog entry.

Shared nothing architectures are all the rage. And so, a little geeky joke to lighten the mood around here:

Q: What’s the opposite of shared nothing?

A: Shard nothing.

Look it up.

Usually when I hear about the thinking behind new web technology I dismiss it pretty quickly. That’s not because I don’t like what people are doing or find it interesting, I just find that almost everything is some kind of application built on an old framework. I’m much more interested in trying to change the framework itself.

I’ve been aware of Radar Networks for some time. I talked to Tim O’Reilly about Fluidinfo in March 2007, and he compared what I was saying to Nova’s claims for Radar. Now that Radar have released Twine, I’ve gone and read some of Nova’s blog postings. I probably should have done that ages ago.

It turns out we agree on many things. Here’s one in particular, in an article entitled Understanding The Semantic Web: A Response to Tim O’Reilly’s Recent Defense of Web 2.0, he has a section entitled “THE SEMANTIC WEB IS THE DATA WEB” which corresponds nicely to my why data (information representation) is the key to the coming semantic web posting.

That’s pretty refreshing. And there’s more, including well-aligned and practical thinking about the word “semantic” and various other words.

I may say more in another posting.

Entertainment retailers are not happy that Prince is giving away his upcoming album, via a deal with the Mail on Sunday newspaper. Their reaction is one of abject fear with a sprinkling of nonsense:

It would be an insult to all those record stores who have supported Prince throughout his career

All those stores making all that money, colluding to fix prices, over all those years, and they were just doing it to support the artists! My heart bleeds for them.

You can almost smell the fear.

A weird thing happened to me this morning.

I needed to call someone in Portugal. I reached for the trusty land line, checked for a dial tone (so old fashioned), grabbed the number, and went to dial. Then I realized I didn’t remember the prefix to dial to get out of the country!

That’s pretty amazing. I’ve been living “overseas” (whatever that means) for over 20 years, and I’ve made plenty of international calls in that time.

I’ve been using Skype for international calls almost exclusively for at least 2 years.

Concepts like “dial tone” and “international dialing prefix” are soon going to appear extremely quaint.

I took my kids to a flea market a couple of months ago. We ran across a rotary phone. Although they knew it was a phone, they couldn’t figure out how you were supposed to dial. Why not just push a button? Dial tone? Access code? Why not just push a (mouse) button?

I like O’Reilly technical books. Back in 1987 I put together some notes to write a book on the vi editor, and later considered submitting the idea to O’Reilly. I used to think I knew just about everything there was to know about vi, at least as a user, and I spent a small amount of time fiddling with its code to fix some limitations. Of course now being a hardened emacs user, it’s a good thing I didn’t blot my career early by writing a book on a crappy editor like vi.

I just did a quick count of the O’Reilly titles on my shelves: I have fifty five.

And you?

Amazon, intentionally or not, have done a great job with their special offer feature that suggests a second book to you and offers you both at the same time for a discount.

One could argue that it’s not in their interests to offer you a second book that you would buy later anyway at its normal price. (Yes, you can argue that it’s implicitly in their interest because it creates goodwill.)

At least in this customer’s experience, they do a great job of offering me things that I might want but never offering me anything I already know that I want. You might think that that’s because I always immediately buy everything I want, but that’s not true.

Today they slipped up and offered me something I knew in advance that I also wanted. I went to look at Glut: Mastering Information Through the Ages, and after I clicked to see the book, I wondered if they might just maybe offer me Everything Is Miscellaneous: The Power of the New Digital Disorder. And… they did.

That’s a first for me. I buy lots of books on Amazon, and I’ve never been offered something I knew I wanted.

Of course it’s also in their interests to occasionally slip up like this. Then people write blog posts praising them and saying how good their algorithms are.

At least for me, Amazon’s “better together” is almost pitch perfect. They consistently land tempting titles just outside the small ring of books I’ve already decided I’m going to buy at some later point. (Note that making special offers like this is very different from the far simpler “customers who bought X also bought Y” – which is just a lookup.) It’s easy to imagine Amazon’s algorithms trying to figure out what I’m almost certainly going to buy anyway, and what I might well buy but probably wont, and picking something tantalizing and just over the edge, just out of reach. What a great way to push readers’ boundaries while making more sales and not leaving money on the table.

Whatever’s going on, and whatever you think might be going on, it’s clearly not simple to keep customers happy and enthusiastic via special offers that do not sacrifice money the customer would in fact spend anyway.

I’ve spent many hours reading about Amazon’s S3 and EC2 services since they were announced. They’re certainly very attractive, and they are being put to heavy use by many companies. There’s a list of examples over on O’Reilly Radar. Don MacAskill of SmugMug gave a great talk at ETech about SmugMug’s use of S3. SmugMug have something like 200TB in storage at S3.

I think S3 and EC2 are fantastic and innovative offerings from Amazon. I’d love to use them for my own project.

But if you read the Web Services Licensing Agreement, it’s quite worrying. Or at least it should be worrying for anyone whose potentially S3/EC2-reliant service may one day rub Amazon the wrong way.

Here are a few extracts:

5. You agree to provide such additional information and/or other materials related to your Application as reasonably requested by us or our affiliates to verify your compliance with this Agreement.

What does “other materials” include? Source code?

If your Application is available as an online solution, you acknowledge and agree that we (and/or our affiliates) may crawl or otherwise monitor your Application for the purpose of verifying your compliance with this Agreement, and that you will not seek to block or otherwise interfere with such crawling or monitoring (and that we and/or our affiliates may use technical means to overcome any methods used on your Application to block or interfere with such crawling or monitoring).

“Otherwise monitor” is pretty creepy and all-encompassing. I’m supposed to give Amazon blanket permission to monitor my service in any way they choose? I think it’s fair enough for them to reserve the right and means to verify that I’m in accordance with the agreed T&C, but the above language is…. well, see below.

If your Application is a desktop solution, you agree to furnish a copy of your Application upon request for the purpose of verifying your compliance with this Agreement.

What does this mean? Source code?

And then we get to the real kicker:

8) If your Application is determined (for any reason or no reason at all, in our sole discretion) to be unsuitable for Amazon Web Services, we may suspend your access to Amazon Web Services or terminate this Agreement at any time, without notice.

Wow.

But big net-and-web-friendly Amazon, they wouldn’t just pull the plug on something they didn’t like. Would they? The experience of Zlio might make you wonder, as might the experience of Alexaholic Statsaholic.

From what little I know of those two cases, I don’t see a reason to condemn Amazon. But they do give pause, and section 8 of the T&C is frightening. There’s more in the agreement that I find vague (just what is an Amazon Property?), but that’s enough examples for now.

IANAL, but I’ve worked on and negotiated dozens of contracts. What we have here is a contract for services drawn up by the lawyers of just one party. This is the kind of shot across the bows you can take when your side gets to draft the contract, and it inevitably comes back with Unacceptable or Rejected all over the place, especially when you’ve egregriously over-reached. You know you’re over-reaching, of course. You get to frame the terms of the contract, which is why it’s so nice to do the first draft.

And yes, OK, Amazon is offering a service, they can define the price and the T&C as they see fit, and you can like it or lump it. But there’s another way, which is to push back a little.

S3 and EC2, and most likely future Amazon offerings, are important. They change a lot and they deserve to be widely used. It’s worth fighting about because they’re so great, because the T&C could be fixed, and because drafters of contractual terms like these expect you to push back.

Potential customers shouldn’t have to worry that Amazon might cut them off without warning and without reason. We should instead speak up and push for a better deal. Because right now the terms of the deal are totally one-sided. Amazon are big enough and mature enough and smart enough to know that it’s in their interests to make S3, EC2 and the rest of their web services as big as possible, and of course they know that their T&C are over-reaching.

If you’re building something that Amazon may one day decide they don’t like, or that they want to compete with, I’d be careful about using S3 or EC2. What if Amazon come along one day and offer to buy you for a deliberately lowball price—or else? What if [insert evil villain] calls up Jeff Bezos one day and makes a deal to have your service cut off? That’s going to be totally opaque to you, and you have no recourse. What if Amazon is bought by XXX, who then decide to cut you off? This may all sound farfetched, but these sorts of things do happen.

Comment #2 on the Zlio RW/W page I referenced above makes an important point. Amazon’s platform is akin to an operating system on which services can be built. Amazon promotes it like a platform. But they reserve the right to dump you unceremoniously, without notice, and without reason. Come on Amazon! We may be fragile startups dying to use your services, but we’re not idiots. If you want to build a platform and have people use it, do it properly. Otherwise, you’re just reserving the right to act like Microsoft after they finally woke up and realized that they could write applications for their OS too, and proceeded to use ugly means to wipe out competitors – to their ongoing and deserved detriment. But even Microsoft didn’t have an EULA that said they could take the OS away from you any time they felt like it.

Given a choice between Amazon cutting the price on S3 again and having them revise their T&C, I’d much rather the latter. But if we all silently accept their T&C, there’s no reason for them to revisit.

A few small changes could make Amazon’s web services irresistible.

Just after I started messing around with Python, my friend Nelson posted about writing some simple Python to speed up the UNIX sort | uniq -c | sort -nr idiom.

I played with it a bit trying to speed it up, and wrote several versions in Python and Perl. This was actually just my second Python program.

The other night I was re-reading some newer Python (2.5) docs and decided to try applying the latest and greatest Python tools to the problem. I came up with this:

from sys import stdin
from operator import itemgetter
from collections import defaultdict

total = 0
data = defaultdict(int)
freqCache = {}

for line in stdin:
    data[line] += 1
    total += 1

for line, count in sorted(data.iteritems(), key=itemgetter(1), reverse=True):
    frac = freqCache.setdefault(count, float(count) / total)
    print "%7d %f %s" % (count, frac, line),

In trying out various options, I found that defaultdict(int) is hard to beat, though using defaultdict with an inline lambda: 0 or a simple def x(): return 0 are competitive.

In the solution I sent to Nelson, I simply made a list of the data keys and sorted it, passing lambda a, b: -cmp(data[a], data[b]) as a sort comparator. Nelson pointed out that this was a newbie error, as it stops Python from taking full advantage of its blazingly fast internal sort algorithm. But…. overall the code was quite a bit faster than Nelson’s approach which sorted a list of tuples.

So this time round I was pretty sure I’d see a good improvement. The code above just sorts on the counts, and it lets sort use its own internal comparator. Plus it just runs through the data dictionary once to sort and pull out all results – no need to fish into data each time around the print loop. So it seemed like the best of both worlds.

But, this code turns out to be about 10% slower (on my small set of inputs, each of 200-300K lines) than the naive version which extracts data.keys, sorts it using the above lambda, and then digs back into data when printing the results.

It looks nice though.

Here’s another thing I’ve had enough of: The graphic design bandwagon of which this image is a perfect example:

This technique is like a rash all over the web. It’s one thing to jump on the bandwagon and make your site look all cool and Web2.0-esque, but there’s another thing about these images that bugs me.

I don’t understand them.

There’s something about them that just doesn’t work for me. When I look at an image like the above, it somehow doesn’t sit right in my mind. I mean, where’s the light coming from? That’s not a shadow, it’s a reflection. It’s bouncing off that nice shiny black highly-reflective surface. So I guess the solution is that there is a bright light somewhere behind me and above my head. Is that it?

Images that have a shadow next to them or behind them are so much easier to deal with. But that was the bandwagon 10 years ago. Now we have the Web2.0 effect in full color, not boring gray. It’s romantic, it’s engaging, and it’s coming right at you, like, like, yes like a perfect reflection on a cool and glassy alpine lake.

And it’s….. everywhere.

I was going to write a much longer set of thoughts on moving to Python, but I don’t have time. Instead I’ll summarize by saying that I programmed for 28 years in various languages before switching to Python nearly 2 years ago.

I like Python. A lot. And there are multiple reasons, which I may go into another time.

One thing that has struck me as very interesting is my use of regular expressions. I came to Python after doing a lot of work in Perl (about 8 years). In Perl I used regular expressions all the time. And I mean every single day, many times a day. I like regular expressions. I understand pretty well how they work. I found multiple errors in the 2nd edition of Mastering Regular Expressions. I made a 20% speedup to version 4.72 of Grepmail with a trivial change to a regex. I put both GNU and Henry Spencer regex support into strsed. I use them in emacs lisp programming and in general day-to-day emacs usage, and in their limited form on the shell command line and in grep.

So given that regular expressions are so powerful, that I well know how to wield them, and that I did so perhaps ten thousand times during those 8 years of Perl, you might expect that I’d use them frequently in Python.

But that’s not the case.

In two years of writing Python almost every day, I think I’ve probably only used regular expressions about 10 times!

I’m not going to speculate now on why that might be the case. I’m writing this partly to see if others (in my huge circle of readers) have experienced something similar. I was prompted to write by an svn check in message of Daniel’s last night. He said:

You know things are bad when you find yourself resorting to regular expressions

And I knew exactly what he meant. When I find myself reaching for the Python pocket guide to refresh my memory on using Python regular expressions, it’s such an unusual event (especially given the contrast mentioned above) that I find myself wondering if maybe I’m doing something really inefficient and unPythonic.

Antonio over at the Onda has a post up about Productivity and being Always-On. He’s got comments turned off, so I’m going to make a few here.

First of all, I really enjoy Antonio’s writings. That’s why I read his blog. But today I just need to push back a little :-) I think all four of Antonio’s points about what you can expect to go wrong are rather weak and/or misleading.

Let’s go through them.

Power (this one was on me for being unprepared). Between Spain and England, I discovered 3 different plug types. What is more, if you travel with a laptop and a phone (more than one device to plug in) and check in late, good luck getting the hotels to have anything to lend you to plug your American appliances in.

You could substitute the U.S. for Spain or the UK in this sentence and it would remain true. There’s actually a good deal of standardized plug size across Europe. Yes, the UK and the US (and some other countries) do things differently. But Spain is part of a large swathe of countries that follow a standard. I could mention the use of 110 volt devices, but I wont. But I do suggest, just for fun, going to the reception of some US hotels and asking them if they have a European plug converter they could lend you. Or try asking for two. I’ve lived 10 years in the US and 10 years in Europe and I have a fairly strong opinion about where you’re more likely to find accommodating help for stuff that requires regular employees of a company to even be aware of the existence of other countries.

Consistent SMS/data on your cellphone. Having just switched to a GSM network, I was really excited by the prospect of 3G networks and zippy-fast mobile data. While voice worked everywhere, SMS and data did not. In fact, SMS was the flakiest of all of the services that I’ve come to rely on— I could receive messages almost everywhere, but I had at best 50% odds of being able to send them.

I’d put this down to (probably) having a mixture of Europe and US carriers involved. I also spent nearly 5 years working in the cell phone industry and know first hand from various carriers that passing SMS between their networks is (or was a few years back) hugely flaky. Someone from a US carrier (I don’t remember which), told me that, officially, US-Euro SMS was not supported by their network but that messages did sometimes “leak” through, but they weren’t sure how! In Spain I find SMS extremely reliable, and I send probably 200/month. When in the US I also have not-infrequent problems, in both directions.

And as far as the wi-fi is concerned, it does seem to be fairly ubiquitous, but in 100% of the cases it was expensive and encumbered by either its billing mechanism or by some lame proxy server setup that blocked most of the useful Internet services you’d want to get access to.

The same could easily be said of the US, and probably every other country. This is too general a complaint – I’ve encountered expensive brain-dead wifi all over the place. One pleasant exception is the airport at Las Vegas, with free wifi. Plus see below.

Overall Internet speed. Finally, the speed of “broadband” connections (especially in Spain) is painful. In this new world of rich Internet applications, it’s easy to forget that we’ve only just been able to get to the point where we can use them in the US and that this is far from a given for other parts of the world. For instance, in Spain Tabblo.com was completely unusable, and even Gmail was severely hobbled by the dearth of bandwidth.

This is also very weak. Who was the ISP? In what city? What sort of bandwidth was the contract? How many different places, ISPs, did you try out? It’s like saying “I went to the US and my broadband connection sucked, so therefore broadband connections suck in the US”. FWIW, I’ve had an ADSL connection with a fixed IP address in Barcelona for about 7 years. I had the connection for several years, at a cost of about US$30/month during which the CEO of the company I worked for in Manhattan couldn’t even get any DSL connection to his Manhattan apartment. I mean nothing. He was using a modem for years while I had a much zippier always-on connection. These days I have a theoretical max of 1Mb up and 20Mb down, and the last time I tested it it was running at about 6Mb. A connection at that speed can be had from Ya for just US$26/month. I ssh into servers and the connections stay up until I close them (often many days). I can even work with Tabblo. I know dozens of people here who use GMail as their only mail source, and I’ve seen it working just fine, without noticeable delay.

That’s it for now I guess. While I’m sure Antonio’s experiences happened, they read like someone comparing their comfortable home setup with what they experienced as a foreign tourist. Of course those experiences will be very different, even if the underlying services are identical. You see the same thing when tourists complain about how expensive a country is. Yes, you can pay 12 euros (US$16!) for a large (and I mean beer stein large) Fanta on the Ramblas. But that says more about you than it does about Spain :-)

I’m trying to work. But right now I’m not working, I’m writing this.

Q: Why am I writing this instead of working?

A: Because my MacBook Pro has decided to spend 2 minutes showing the spinning color wheel in the application I was trying to use.

I used to have a 17″ G4 Powerbook with 2GB of RAM. When people asked me what I thought of it I was always very positive and enthusiastic in my reply. Then I got a 17″ MacBook Pro, and I hate it and wouldn’t recommend it to anyone.

Every single day, probably half a dozen times, I’ll be using some application (in this case Aquamacs, a version of Emacs for the Mac) when, for no apparent reason, the mouse cursor will change to the spinning pizza wheel of death, and remain that way for up to a couple of minutes. This happens to me in Firefox too, so it’s definitely not an Emacs thing. It probably happens in other apps too, I haven’t paid as much attention as I probably should have.

I’m too busy swearing at the machine.

Anyway….. what the hell!? I wish I could just throw this piece of junk away and get something that just works. I’m planning to head back to Linux next time around. I can’t believe that this machine could have made it out the door. But I need it too much to be able to just send it back, so I’m stuck, and I resort to cursing and blogging.

I have other complaints too. I run a make clean that removes a bunch of files (around 20M of stuff, not that many files). Sometimes the rm command will just sit there for 15 or 20 seconds before completing. What’s it thinking about? I run unit tests on Python code all the time. Sometimes the python command to run the tests will just sit there, the other day it was taking almost a minute to launch python. Yes, I know, I can go look at what’s going on on my machine, and I do, and sometimes, yes, it’s busy. But this is just ridiculous. I’ve been using UNIX for 25 years and I haven’t had to wait for things like this since the early 80’s (and then on a machine I was sharing with up to 128 others). It’s infuriating to have the expensive latest and greatest laptop and then have it perform like a dog.

FWIW, the machine is probably a year old. Probably these seem like small problems, but I think Apple really screwed up, and there’s plenty of complaints online about problems with these laptops. Blast them to hell.

Now back to work – if I can manage to get a cursor in Emacs, that is.