Archive for October, 2009

Digital hobgoblins

Sunday, October 4th, 2009

hobgoblin-hallMuch of the thinking behind FluidDB comes from thinking how we work with information in the real world, and comparing that to how we do so in the computational world (aka Hobgoblin Hall). The differences are striking.

Over the years, I’ve often asked myself why things are so bizarre in the computational world and why we don’t do something about it. Without going into the answers to those questions, I’ll just say that I think we’ve all grown up in Hobgoblin Hall and despite the fact that we’re all perfectly familiar with the freedoms of the outside world, we take it for granted that things are deeply weird in our computational homes.

Here I’ll quickly outline a few of the more glaring oddities. (BTW, I’d be remiss not to point out that FluidDB has none of the following restrictions.)

Things must be named, and have one name. In the real world we have plenty of things that don’t have names. As I look around my desk right now, I can see dozens of things that don’t have names. We also often give things many names – first names, surnames, nicknames, abbreviated names, English/Spanish/Chinese/etc names. Flexibility in naming (no names, one name, multiple names, private names, etc) is obviously of great utility. Yet in computational systems we’re usually compelled to name things, and we’re restricted to a single name. These are just a couple of the problems I have with file systems – I have about 10 others, but will spare you.

Inconsistency and ambiguity are common in the real world. While they’re obviously often not helpful, there are times when it is very useful to have both. Things may become clearer over time. Systems evolve. In the natural world we use representations that allow high degrees of inconsistency and ambiguity, and it’s very useful to be able to do so – how else would we learn or get anything accomplished if not? Yet if you suggest a computational system that explicitly allows for any level of inconsistency and ambiguity in information, people start to get nervous or even upset. They’ll begin to argue with you, and suggest ways to “fix” the system to get rid of the undesirable qualities. Why is that?

Multiple organizations of the same information are very common in the natural world. We do it all the time. Computationally it’s rare that systems allow us to multiply organize things. That’s changing, thankfully, with the rise of tagging and with music collection software that allows multiple simultaneous playlists (or “smart” dynamic playlists) of the same underlying sound files. But those systems are the exception rather than the rule.

There’s an obsession with “meaning” and pinning down what things are “about” in the computational world. In the real world we don’t seem to care that much – we’re more concerned with utility. What’s a book for? Something to read? Something to stop other objects from blowing away? Something to be hollowed out to hold a gun? Something to create an intellectual impression? A decoration? Something to hold up other books? A hiding place? A book can be all these things, and we can move seamlessly between them. What’s a glass for? Is it a weapon? Something we can hold to the wall to hear a conversation? Something to use as an insect trap? A fingerprint capturing device? A musical instrument? Maybe even something to drink out of? We don’t really know, and it’s not important to know. We don’t obsess over the “meaning” of a glass or try to determine what single thing it might be “about” etc. We just use it as we see fit. Similarly we can’t anticipate how people will want to use information – and our storage architecture shouldn’t try. (You could counter by pointing out the FluidDB about tag. But usage of the about tag is entirely optional. It’s a convenience. You can make your own, or use none at all. And a FluidDB object can be about whatever you think it’s about and used for whatever you want to use it for – even if others have completely different interpretations of and uses for the same object. No problem.)

Later (meta?) data is often most usefully put with the original data. That’s what we commonly do in the real world. It’s convenient, easy, useful, and natural. For example, when you’re reading a book and you want to remember what page you’re up to, you can simpy dog-ear the page or insert a bookmark. The extra information travels with the book. In the computational world you can’t do things like that unless a programmer has anticipated that you might want to and made provision for the extra information in the underlying data structure (or database). So we’re very often forced to put extra unanticipated information elsewhere – e.g., in a file, in our heads. Unfortunately, that later information is often the most important – because it’s generated by individuals who are trying to customize or personalize their computational world. I’ll have much more to say about that another time. For now: a writable architecture like FluidDB does not have this limitation because you can always put the new (meta?) information with the old (content?). And search on it, etc. That offers a fundamental change in how we work with information. I’ll blog about it at length one of these days. You get to think about it in the meantime :-)

Fluidinfo as a universal metadata engine

Saturday, October 3rd, 2009

Image: Jin Wicked

Image: Jin Wicked

One way to use Fluidinfo, among many, is as a universal engine for metadata.

I’ll have to explain what I mean by that, especially seeing as some people got the impression from the earlier post on data vs metadata that we don’t think metadata is important, or that it doesn’t exist, or similar. I tried to make it clear in the post, and in responding to the comments that followed, that that’s not what was meant: In fact that’s one of the major initial goals of Fluidinfo – to be a metadata engine for everything. So that’s how important we think metadata is! The way to support metadata on anything is to have an underlying architecture that’s flexible enough to allow that to happen – without someone setting the thing up with an a priori determination of what’s meta- and what’s not. True support for metadata is too important for that – to do it properly you need the architecture to be neutral.

The question is: how can Fluidinfo be used as a universal metadata engine?

Metadata can be loosely defined as data that’s about other data. So to provide a universal metadata engine, any time any application wants to store some metadata (M) about anything (A), Fluidinfo should always have a place to put M. Moreover, the application shouldn’t have to stop to ask if it’s ok to store the metadata, and its needs shouldn’t have to be anticipated.

The key word in the above paragraph is about. Fluidinfo has an about tag that can (optionally) be added to objects to indicate what they’re about. There’s a lot that could be said about the about tag – in fact, the person who pushed for its inclusion in Fluidinfo, Nicholas Radcliffe even started a blog of that name. The main thing to know for now is that the about tag on an object (if any), is immutable and its value (always a string) is unique across all Fluidinfo objects.

To give some simple examples, there might be objects in Fluidinfo with about tags that have values such as isbn:140679239X or or US:ZIP:90210 or or IP: or….. anything you like. That’s the point.

So when an application – any application – wants to store information about A, it just asks Fluidinfo for the object whose about tag has the value A. If the object already exists, Fluidinfo returns it. If not, Fluidinfo creates a new object, sets its about value to A, and returns it.

That’s the first part of being a universal metadata engine: if you want to store information about something, Fluidinfo gives you an obvious place to put that information (provided you can convert your particular A into a string of some kind). In this regard, Fluidinfo is like a wiki. When you use a wiki, you can ask it for the page on any subject, and if it doesn’t exist it will be created. As with a wiki, you can think of Fluidinfo as already having objects about everything; just like a wiki, Fluidinfo doesn’t actually create any particular object until someone asks for it.

The second crucial component is Fluidinfo’s model of control. As mentioned in the Information. Naturally. post, Fluidinfo objects do not have owners. That means that all applications are guaranteed that they can store information onto the Fluidinfo object about A.

Putting these two together, you get something that starts to look very much like a universal metadata engine. Got some metadata to store about something? Fluidinfo gives you an obvious place to put it and a guarantee, in advance, that you’ll be able to do so. This is what we mean when we say Fluidinfo makes the world more writable.

To give a couple of quick examples, Emanuel Carnevale has written two Javascript programs for the Firefox 3.5 Jetpack extension. These are just quick proofs of concept for now, but they will mature. One is fluidy-hood that offers functionality along the lines of Google’s Sidewiki (though more general), and the second is BRB, the Borthwick Remember Button, in honor of John Borthwick of Betaworks who asked for one. These are very simple pieces of code that use Fluidinfo as a universal metadata engine, in both cases putting information onto the object that’s about the URL you’re currently looking at in Firefox.

A final comment about the creation of value: These tiny apps have limited and unremarkable value to their individual users. Things get much more interesting though we you consider that these applications are creating truly social data. It is directly searchable via the Fluidinfo query language. It can be combined with other information—homogenous (created by the same app being run by someone else) and heterogenous (related but different information about the same thing created by other apps). It can be accessed, augmented, and mashed up by others. And the person who created the information can continue to control it: share it, protect it, edit it, delete it, etc.

When you look at data and applications like this, you begin to see why we’re so excited about the kinds of changes in how we work with information that we think Fluidinfo can help to introduce.

There’s a lot more that can be said regarding the about tag, about how all this affects customization, personalization, and information organization in general, about ambiguity and its resolution, and about the creation of value via putting information into context. Those things will have to wait for later blog postings, though.

Stay tuned.