Fluidinfo

January 4, 2012

Next-generation tagging

Filed under: Essence,Tagging — Terry Jones @ 7:10 pm

Image: rosweed

In 2003, Joshua Schachter released Delicious, and brought tagging to the web. Delicious gave normal users a place to store and share their information about any web page. By January 2007, a Pew Internet & American Life Project study had found that 28% of online Americans had used tagging. As offered by Delicious, and popularized by sites such as Flickr and Technorati, basic tagging has three fundamental properties:

  1. You can tag any URL.
  2. You don’t have to ask permission.
  3. Shared public tagging creates social value

Most fundamentally, tagging made the web more writeable. This changed our perception of the web itself, from something that was almost entirely read-only to something where we were suddenly invited to contribute. The change was a cornerstone of what Tim O’Reilly dubbed “Web 2.0”. It’s what Fred Wilson describes as “giving every person a voice“. Fred quotes Ev Williams, a Twitter founder, who says “society has not fully realized what this means”. One view of Twitter is that it offers a form of simple self-tagging. Viewing tagging as low-friction publication of small pieces of information about things, Twitter has done for people what Delicious did for URLs.

Tagging with Fluidinfo

The model of information in Fluidinfo takes tagging to a new level. Over the last three months we’ve been hard at work building a user interface (UI) to make this possible. The two main additional properties of tagging with Fluidinfo are:

  1. You can tag anything at all, not just URLs.
  2. Tags can optionally be given values.

Tagging is too powerful an idea to be restricted to just URLs or people. It should be possible to tag anything. And, as you’ll see below, tags with values are powerful and natural. We tag with values in the non-digital world all the time.

Tag anything at all

Every day each of us encounters thousands of things that are not URLs. With Fluidinfo you can tag anything you can name. You can tag songs, movies, and books – just put the name into the box at the top of the Fluidinfo page, hit RETURN, and you’ll be looking at the page for that thing in Fluidinfo. If you’re logged in, you can add tags. Look at what pops up on the left to see other information about the object. You can also tag a product name, an email address, a person’s name, an IP address, a license plate number, a place name, flight numbers, a word or phrase, a zip code, a stock symbol, a latitude/longitude pair, a Twitter @name or #hashtag, taxi medallion numbers, time/date combinations, phone numbers, a DNA sequence, a chess position or a Sudoku puzzle, etc. To get to the Fluidinfo object for something, just visit http://fluidinfo.com/about/something. The easiest way to jump to the Fluidinfo page for something you see on the web is via a browser extension (Chrome, Firefox, Safari). Just right-click on a link, an image, on selected text, or on the page itself, and jump straight to that thing in Fluidinfo.

Tag with values

With Fluidinfo, you can provide a value when tagging. Tagging with values will seem new to many people, but it’s actually not new at all. Look at the luggage tag image above. It has (implicitly) a ‘destination’ tag with a value of ‘JFK’, it has a tag named ‘Flight number’ with a value of 222, a tag named ‘Row’ with a value of 3, a ‘Sorting Symbol’ with value ‘B’ and something else whose value is 821474. Those tag names and values are all put on the same physical tag because they’re related, and it makes sense that they travel together, attached to the luggage. So, far from being new, tagging with values simply provides us with something familiar which has always existed in the physical world of tagging.

To give some examples in Fluidinfo, I have put a ‘rating’ tag onto Gone With the Wind with a value of 4. A tag value can be anything at all: a comment, a set of keywords, ‘longitude’ and ‘latitude’ tags with numeric values, even arbitrary data (e.g., some HTML, an image, a PDF file).

Most interesting, a tag value can be a URL (or list of URLs). You might tag an image with a value that is the URL of another image. You might put a ‘for-dad’ tag onto something with a value pointing to a video. In the Fluidinfo UI, tag values that are URLs are shown as simplified embedded pages or images.

Examples of things in Fluidinfo

Here are links to a variety of things in Fluidinfo. Take a look at the list of links on the left of each page. These are ‘views’ of the data, either from different sources or displayed in different ways. If you log in, you’ll be able to add your own tags.

June 15, 2011

A shared writable object for everything: Sudoku problems

Filed under: Essence — Tags: , , — Terry Jones @ 11:34 am

Image: wikipedia

Fluidinfo is composed of objects that have tags with values. One of the tags is special, the about tag (its full name is actually fluiddb/about). One of its properties is that its values are unique across Fluidinfo. In other words, there can only be one object with any particular about value. It’s a bit like Wikipedia, which you can think of as having a single shared writable page for every topic imaginable. Fluidinfo offers a shared writable object for everything imaginable. The last part of a Wikipedia URL is like a Fluidinfo about tag value. That’s very wiki-like, but other properties of Fluidinfo make it more useful for applications than a wiki is. For example, it has a permissions system, its data is typed, and it has a query language. If you’d like to learn more about the about tag, there’s an entire blog named after it, run by Nicholas Radcliffe.

This morning I was having breakfast with Esteve Fernández and Jamu Kakar and Jamu mentioned a friend who’s heavily into Sudoku. I mentioned that there are several mobile apps that let you take a picture of a Sudoku puzzle and then solve it for you. I also said “Fluidinfo has an object for every Sudoku puzzle”.

I thought I’d write a very short blog post to illustrate this. So I took the Sudoku puzzle from the Wikipedia entry and made a Fluidinfo object for it. I read the cells of the puzzle left-to-right, top-to-bottom, and used a hyphen to indicate an empty cell in the starting configuration. Written as a single string of characters, that’s 53--7----6--195----98----6-8---6---34--8-3--17---2---6-6----28----419--5----8--79. I used the Fluidinfo Explorer to create an object in Fluidinfo with that as the about value. Then I put a tag called terrycojones/solution onto that object, with value 534678912672195348198342567859761423426853791713924856961537284287419635345286179, which is the correct solution to the puzzle, again read left-to-right, top-to-bottom.

This illustrates a few things. Firstly, Fluidinfo really does have an object for all Sudoku puzzles (created as needed, of course). Second, I’ve established a convention for the about tag value to represent those things. I could have done this in many different ways, and the solution I chose may not be the best. If I were intending to add information about lots of Sudoku puzzles, I would publish my choice and encourage others to follow it (which anyone could do, since any Fluidinfo user may create an object with any about tag – if the object already exists it’s no problem, you just get the already existing object). Third, the terrycojones/solution tag I put onto the object may not be of much use to the wider world. But, I could give other people (or applications) that I trust write permission for that tag so they could tag the objects too. Fourth, if I thought those solutions were worth something, I could make the tag unreadable by default and try to sell access to it (i.e., allow only those who paid me to have read access).

Finally, this illustrates the simple way in which isolated activities, like individually solving a puzzle, can be made social through shared writable storage. If I used a Sudoku application that tagged the shared object with some subset of terrycojones/solved or terrycojones/working-on-it or terrycojones/too-hard or terrycojones/solution-time tags, and others used that application too, solving Sudoku puzzles would instantly be social. Any Sudoku application could use Fluidinfo to allow you to see puzzles your friends couldn’t do or were working on, it could show you the amount of time your friends took, give you hints, find errors, etc. It’s easy to think of a ton of possible social extensions to the Sudoku world, and these include collaborative efforts.

This is a nice example of how shared writable storage with an object for everything allows formerly isolated actions to easily be made social. I’m planning to write up some more simple examples. There are many of them.

June 1, 2011

Personalized filtering of friend requests in social networks

Filed under: APIs,Data,Essence — Terry Jones @ 4:42 pm

In an O’Reilly Radar article titled Getting closer to the web2.0 address book in October 2010, I described how a set of applications might coordinate via shared storage to solve Tim O’Reilly‘s question:

Given that knowledge about who I know is in my phone, in the O’Reilly org chart, and in the set of authors of O’Reilly books, why do I still have to manually approve friend requests from Facebook, LinkedIn, etc.?

In that article I argued that this aspect of what Tim calls the web 2.0 address book should be solved not by an application, or even by applications making API calls to one another, but by a set of applications that communicate asynchronously via shared writable storage. As well as shared storage, also needed are a permissions system, a query language, and conventions used by the cooperating applications. No API calls are needed between applications. The interaction between the cooperating applications takes place entirely via shared storage. API calls are only used to add, retrieve, and query data in the shared storage.

At Gluecon in Denver last week, I gave a talk titled The Real Promise of Cloud Storage in which I argued that shared storage gives two big advantages. The second was that it allows the evolution of “asynchronous data protocols”. The final slides of the presentation showed the series of steps that would be taken to build what Tim was asking for. I wasn’t able to go into the detail of how this would work in the time available. So in this post I’ll give the detail of how this can be implemented using Fluidinfo. The post is long because I want to spell out the steps, the assumptions, the Fluidinfo tag permissions, the queries, the security aspects, and so on.

The business case

I’m going to pretend LinkedIn is the company that would like its users to have more flexibility in approving friend requests. Additional flexibility could include personalized friend resolvers (for example based on the O’Reilly org chart) and also applications that could assist in various way, and if desired even automate some responses (e.g., people you have called more than twice in the last month, or people you have put on a particular list on Twitter).

LinkedIn is a good candidate because they have traditionally placed less emphasis on being a burgeoning social network, and more on link quality, by requiring a higher level of proof to create more meaningful reciprocal connections. That seems to be changing slowly now, though. You used to need to know another user’s email to connect to them, but that has been relaxed over the last couple of years. It’s obvious why LinkedIn would do this: knowing connections between their members is valuable. LinkedIn will now show me suggested people I might know, and let me reach out to connect with them with a single click. No need to go look up email addresses, etc.

I suspect many people have valid unanswered friend connection requests sitting in their LinkedIn inboxes. I do, and I almost never make time to deal with them. Those requests, if accepted, would add value to LinkedIn. The (valid) connection requests that sit unanswered for whatever reason represent unrealized value. So it’s very much in LinkedIn’s interest to help people connect. They should be interested in a solution to Tim’s dilemma, as should Facebook and any social network whose friending mechanism is reciprocal.

Using the shared writable storage of Fluidinfo to solve this problem

Here is how this could be achieved with Fluidinfo.

1. LinkedIn announces that it will support personalized friend resolution via Fluidinfo. Users will have control over what kinds of friend resolution they want to enable. LinkedIn will use the linkedin.com user name in Fluidinfo. (Fluidinfo makes it possible to put domain names on data. Every Fluidinfo user name that corresponds to a domain is reserved for its internet domain owner.)

2. Suppose Tim, a LinkedIn user, decides to try the new functionality. LinkedIn directs him to a page where he can choose a Fluidinfo user name (let’s say "tim") and create a Fluidinfo tag called tim/friend-request/linkedin. Tim sets the permissions so that the Fluidinfo linkedin.com user can both add and delete the tag on Fluidinfo objects. He then tells LinkedIn his Fluidinfo user name. This is necessary because LinkedIn will later create data in Fluidinfo on Tim’s behalf. Note that LinkedIn only knows Tim’s Fluidinfo user name, not his password.

3. Suppose a user Alice now tells LinkedIn that she is a friend of Tim’s. LinkedIn creates a new object in Fluidinfo and puts an instance of tim/friend-request/linkedin onto it with a value (perhaps in JSON format) holding the information Alice is willing to supply as evidence that she knows Tim. This could include things like her Twitter user name, her phone number, the name of a company she used to work at with Tim, etc. LinkedIn also adds a random request identifier (more on which below).

The object in Fluidinfo looks like this (the tag value is in the rectangle under the tag name):


Note that the object does not have an owner. That’s a fundamental feature of Fluidinfo: objects don’t have owners. Instead, tags on objects have permissions. That’s crucial in what follows, because it allows any application to add more information to the object representing the friend request.

A friend resolver based on Twitter friends: TwitResolve

4. Let’s suppose a keen-eyed developer notices the LinkedIn announcement and decides to write a Twitter-based friend resolver called TwitResolve. The developer grabs the internet domain twitresolve.com and then gets the twitresolve.com user name on Fluidinfo (as the domain owner, only they can do this, as above). TwitResolve will use two tags in Fluidinfo, twitresolve.com/accepted and twitresolve.com/examined. It sets the permissions on the tag twitresolve.com/accepted so that only the Fluidinfo user called linkedin.com can read it.

5. Let’s suppose Tim hears of TwitResolve and decides to use it. The TwitResolve sign-up page asks Tim to change the permissions on the Fluidinfo tag tim/friend-request/linkedin so that the Fluidinfo user twitresolve.com can read the tag. (The setting of the permission could be done in various ways, which we’ll ignore for now.) By giving TwitResolve permission to read the tag that LinkedIn puts onto objects, Tim enables TwitResolve’s participation in helping to resolve friend requests for him. TwitResolve also asks Tim for his Twitter username (and gets Tim to prove that).

6. TwitResolve runs a process periodically that asks Fluidinfo for the outstanding requests for Tim that it hasn’t already examined. The relevant query in Fluidinfo’s query language is has tim/friend-request/linkedin except has twitresolve.com/examined.

7. In our case, this query finds the object shown above and TwitResolve examines the value of the tim/friend-request/linkedin tag. It talks to Twitter to discover whether Tim is following Alice (it can do this because Alice’s Twitter user name is in the tag value, and Tim provided his Twitter user name to TwitResolve). Suppose Tim is not following Alice. TwitResolve cannot conclude anything, so it simply puts a twitresolve.com/examined tag (with no value) onto the request object in order to avoid re-examining this request. At this point the Fluidinfo object looks as below and the friend request is still unfulfilled.


A friend resolver based on the O’Reilly org chart

8. News gets out quickly to the digerati and someone inside O’Reilly decides to write a resolver based on the O’Reilly org chart. They create two tags, similar to those created by TwitResolve, called oreilly.com/accepted and oreilly.com/examined, again setting the permissions on the former so that only the Fluidinfo user called linkedin.com can read that tag. Tim decides to use the new resolver, so he gives the oreilly.com user permission to read the tim/friend-request/linkedin tag.

9. The O’Reilly resolver might be started by cron each night. When it runs, it sends a Fluidinfo query, just like TwitResolve does. Suppose Alice does not work at O’Reilly. The details in the request cannot confirm her as a friend of Tim’s. Just as with TwitResolve, the O’Reilly program puts an oreilly.com/examined tag onto the object, and we arrive at the following:


A friend resolver based on Amazon books

10. To give a third example (to fulfill Tim’s requirements), another resolver could be written based on authors of books. Suppose it’s written by someone at Amazon. (The details of how this might successfully match Alice to Tim aren’t important here.) Suppose it is also unsuccessful in matching this friend request. It adds an amazon.com/examined tag to the object:


An iPhone friend resolver: iPhoneFriender

11. Next, someone decides to write a resolver that can match based on phone numbers, called iPhoneFriender. They get the iphonefriender.com domain name and user name in Fluidinfo. They will use the tags (iphonefriender.com/accepted and iphonefriender.com/examined), and set the permissions on the iphonefriender.com/accepted tag so that only the linkedin.com Fluidinfo user can read it.

When the iPhoneFriender application runs, it sends the Fluidinfo query has tim/friend-request/linkedin except has iphonefriender.com/examined and finds our object. It reads the request details. Suppose it finds Alice’s phone number in Tim’s phone address book, and can see whether Tim has called Alice or vice-versa, whether Tim has chosen not to accept her calls, etc. iPhoneFriender has preference settings to allow Tim to specify what kinds of requests it will ask him to confirm, and which to automatically accept. Suppose that one way or another the friend request is accepted.

12. The iPhoneFriender application needs to tell LinkedIn that Alice is recognized as a friend of Tim’s. So it puts an iphonefriender.com/accepted tag with value 9871261721498793 onto the original object. This is the unique request_id value from the original request. Remember that the iphonefriender.com/accepted tag has its permissions set so that only linkedin.com can read it. Hiding the identifier in this way prevents rogue applications from falsely claiming to have satisfied friend requests. Only an application that Tim has given permission to read the tim/friend-request/linkedin tag to could know the request identifier. Only LinkedIn can read the identifier value out of the iphonefriender.com/accepted tag attached by iPhoneFriender. iPhoneFriender also adds an iphonefriender.com/examined tag to the object to avoid repeating work.

The Fluidinfo object now looks as follows (tag values, when present, are shown in rectangles under the tag names):


Friendship, requited!

13. Later, LinkedIn sends Fluidinfo the query has tim/friend-request/linkedin. The search matches our object and LinkedIn then gets a list of its tags. For tags named */accepted (where * matches any user name) it tries to read the value of the tag. If it finds a tag whose value matches the identifier in the request tag on the object, LinkedIn adds the friendship link inside its own site. It also deletes the tim/friend-request/linkedin tag from the object, resulting in:


At this point we’re basically done. There are many possible variations on the above. For example, using timestamps to retry friend resolution, using timestamps to only examine recent requests, using separate tags to hold pieces of friend request information, using an extra linkedin.com tag to reduce the number of queries it must make to find outstanding requests, etc. Applications are not forced to use an examined tag to avoid repeating work; if they do they can name it whatever they please. It’s also easy to imagine more exotic resolvers, e.g., Tim giving people a secret random number and looking for that in the request (LinkedIn would have to allow Alice to add it to the request, obviously), etc. Participating applications could also clean up by finding objects with their examined tag but that no longer have a tim/friend-request/linkedin tag, and removing their examined and accepted tags (if present).

Why this is nice

The above dance is nice for several reasons:

  • This is an open, convention-based, extensible, and validated application ecosystem. LinkedIn just writes data to Fluidinfo and periodically checks for resolution. In effect it is giving Tim the power to use any application he wants to resolve the request in any way he wants. Participating applications just follow the established tag naming convention. LinkedIn knows that if any application attaches a tag of the form */accepted with value 9871261721498793 to the object, that it must represent a validated acceptance by Tim.
  • As a result, anyone can play. An idiosyncratic resolver such as that based on the O’Reilly org chart is as legitimate a contributor as any other. None of the resolvers needed to ask for permission (from LinkedIn) to participate, or needed to be anticipated by LinkedIn.
  • There are no API calls between the applications involved. This is significant because APIs have to be designed in advance and you need permission to use them. In our scenario, all communication between applications is done via a data protocol: adding to and retrieving from shared storage.
  • LinkedIn is free to ignore the */accepted tags from any application if it chooses.
  • Tim can withdraw permission for a resolver to work on his behalf, simply by taking away that application’s permission to read tim/friend-request/linkedin tags.
  • Tim can stop LinkedIn from creating friend resolution tags by removing the linkedin.com user’s permission to add the tim/friend-request/linkedin tag to objects. That would be somewhat extreme, seeing as LinkedIn is likely to offer its users a simpler way for people to turn off the feature, but it’s worth pointing out that Tim has control.
  • To those who don’t have permission to read tim/friend-request/linkedin tags, there is no way to see who has made the friend request. (The fact that Tim is the target could also be easily obscured, if wanted.)
  • All communication is convention-based and asynchronous. This resembles the way we (and other organisms), often communicate in natural systems. I suspect most information communication between living organisms is asynchronous, though I have no way to quantify this. Asynchronous communication via conventions in shared storage (e.g., those seen in Twitter with hashtags and @addressing) is so powerful because it is open-ended and evolutionary. Fit conventions (in the biological sense) will flourish. Conventions can be extended by any player, without harm. I wrote more on this in Dancing out of time: Thoughts on asynchronous communication.

Note that the above is just an example of how applications can communicate indirectly and asynchronously through shared storage using evolving conventions instead of using direct, synchronous, predefined API calls between one another. We have seen a solution to a difficult address book problem that has not involved writing an address book application. Instead, the problem is solved by a set of lightweight and loosely coupled cooperating applications communicating through data. I have (very slowly!) come to realize that this form of inter-application communication is an important part of what Fluidinfo makes possible. This is all enabled by the simple move to shared writable storage, coupled with a flexible permissions model and a query language.

Thanks for reading. I really hope you’ll find this as interesting as I do. Thanks to Nicholas Radcliffe, Tim O’Reilly, and Bar Shirtcliff for comments that greatly improved the above.

May 14, 2011

Speaking at Gluecon 2011 in Denver

Filed under: Events — Terry Jones @ 7:36 am

Gluecon 2011I’ll be speaking at Gluecon in Denver on May 25/26. The conference looks fantastic and there are lots of people going that I’m looking forward to catching up with. My talk (Thu May 26, 9:30am) is titled Evolution of inter-application data protocols via shared writable storage, with the following rather wordy abstract:

Cloud storage offers a variety of potential advantages: greater capacity, ease of scaling, lower cost of ownership, fewer operations staff, less hardware build-out, reduced responsibility for backups, etc. These are all straightforward and in a sense linear changes. There is another advantage though that is more interesting: the inherent value created when applications use shared storage. Shared storage holds the potential for unanticipated valuable operations, including search and mash-ups, that go beyond the linear value of isolated per-application storage. Shared storage also allows asynchronous inter-application communication based on data protocols that emphasize emergent agreed conventions rather than a priori rules. This is a sharp departure from inter-application communication via pre-specified synchronous remote procedure calls. In this talk I will elaborate on this point of view, with examples from evolutionary systems and discussion of Fluidinfo. I’ll argue that shared writable storage is the real promise of cloud storage, and show how it offers an approach to a class of problems which includes Tim O’Reilly’s oft-asked question “Where is the Web 2.0 address book?”

For a simpler description of what I mean by all this, read the pair of articles on the O’Reilly Radar site: Dancing out of time: Thoughts on asynchronous communication and Getting closer to the Web 2.0 address book. If you’re going to Gluecon, please say hi! If you’re not going and you’d like to, you can register here and use the discount code spkr12 to get 15% off.

March 21, 2011

Announcing a writable API for O’Reilly books and authors

Filed under: Awesomeness,Events,Writable APIs — Terry Jones @ 9:41 am

Today we’re excited to announce the release of a writable API for O’Reilly books and authors. There’s far too much news and information around this release to pack into a single blog post. Here’s a summary of what’s new today and where to find out more.

Here’s an extract from the press release:

General manager and publisher Joe Wikert is excited by the opportunities that a writable API provides to O’Reilly and other publishers. “It’s like LEGOs for publishing,” he says of the new malleability in his industry. “It’s as though we’ve been selling plastic children’s toys and the pieces were all glued together so customers could only use them the way we intended them to be used,” he adds. “Now we’ve decided to break the pieces into their component parts and let customers build whatever they want.”

Last but not least: if you want a modern, writable API for your data, drop us a line at info at fluidinfo com, and let’s talk.

March 15, 2011

Fluidinfo named as Tim O’Reilly’s favorite startup

Filed under: Awesomeness,Events,People — Terry Jones @ 4:11 pm

Image credit: The Guardian

Tim O’Reilly was interviewed by Jason Calacanis on stage on the opening day of SXSW last week. Jason asked Tim to name his favorite start-up and Tim nominated Fluidinfo! Thanks Tim 🙂

You can read excerpts from the interview in A SXSW fireside chat with Tim O’Reilly and Jason Calacanis on the TechChi blog:

O’Reilly’s favorite startup is Terry Jones’ Fluidinfo “because I’m not sure it’s going to work. He’s got his teeth into something that is bigger than he is. He may be overwhelmed and he may not get it,” O’Reilly said.” That passion it’s kind of like the Wright Brothers that wanted to fly or Thomas Edison and the light bulb… It’s not the entrepreneur chasing the million bucks, it’s the entrepreneur chasing the big idea.”

The interview was also picked up in a Business Insider article: Tim O’Reilly’s Favorite Startup.

March 5, 2011

Indicating (shared) interest in things without disclosing what they are

Filed under: Awesomeness,Essence,Howto,Programming — Terry Jones @ 2:21 pm

Imagine you want wanted to tell the world you were interested in something, for example an email address or a phone number, without telling the world what that thing was. That may not sound so interesting, but if several people were doing the same thing, it would be a mechanism for discovery of private things you had in common, without telling anyone else what those things were.

Russell Manley and I just thought of a simple way to do this using Fluidinfo. Here’s how we did it for the email addresses we know.

For each email address, compute its MD5 sum. Then, put a rustlem/knows or terrycojones/knows tag onto the object whose fluiddb/about value is the MD5 sum. The MD5 algorithm is essentially one-way, so even if someone finds a Fluidinfo object with either of our tags on it (which is trivial) they cannot recover the original email address.

This is pretty nice. We’re independently indicating things of interest, but neither of us is publicly saying what those things are. Because we’re putting our information onto the same objects in Fluidinfo, we can then easily discover things we have in common with each other (and with others), without the world knowing what. We can do the same thing for phone numbers, or anything else.

Getting the data into Fluidinfo was trivial. Here’s code I used to put a terrycojones/knows tag (with value True) onto the appropriate objects:

import sys, hashlib
from fom.session import Fluid

fdb = Fluid()
fdb.login('terrycojones', 'PASSWORD')

for thing in sys.stdin.readlines():
    about = hashlib.md5(thing[:-1]).hexdigest()
    fdb.about[about]['terrycojones/knows'].put(True)

You pass a list of email addresses to this script on standard input.

Russell and I each had about a thousand email addresses in our address books. A first question is how many addresses we know in common. You can get the answer to this with the simple Fluidinfo query has terrycojones/knows and has rustlem/knows. It turns out there are 53 common addresses. But the results don’t tell us which addresses those are, which is also interesting.

We also wrote a small script to print any tags ending in /knows for a set of email addresses given on the command line.

import sys, hashlib
from fom.session import Fluid
from fom.errors import Fluid404Error
fdb = Fluid()

for thing in sys.argv[1:]:
    about = hashlib.md5(thing).hexdigest()
    print thing, about
    try:
        for tag in fdb.about[about].get().value['tagPaths']:
            if tag.endswith('/knows'):
                print '\t', tag
    except Fluid404Error:
        print '\tunknown'

So given an email address, we can run the above and see who else knows (or claims to) that email address.

We find all this quite thought provoking. Without going into details of the social side of this, it’s worth pointing out that Fluidinfo makes this kind of information sharing very easy because it has a guaranteed writable object for everything, including all MD5 sums. Because the fluiddb/about tag is unique and isn’t owned by anyone, any user can add their knows tag to the object for any MD5 sum. The ability for users and applications to work independently and yet to share information by just following a fluiddb/about convention is one of the coolest things about Fluidinfo.

Finally, note that this system does not guarantee privacy. If someone already knows an email address or phone number (etc) they can compute its MD5 sum and examine the Fluidinfo tags on the corresponding object. Doing so they might see a rustlem/knows tag and would then be free to draw their own conclusion.

You can play too. All you need is a Fluidinfo account and the above code. Please let us know how you get on. For example, you can freely tweet any MD5 sums we have in common. We’re going to use the hashtag #incommon, like this.

February 25, 2011

Fluidinfo voted Top Technology Company at LAUNCH in San Francisco

Filed under: Events,Happiness — Terry Jones @ 12:43 am

Wow…. Fluidinfo was just voted “the clear winner” as Top Technology Company out of 100 start-ups in the LaunchPad at the LAUNCH conference in San Francisco!

Thanks to all the judges, especially to Robert Scoble, Marshall Kirkpatrick, Brian Alvey, Naval Ravikant and Mark Pesce. Mark said “Fluidinfo is totally crazy, but it’s the kind of crazy I love.” 🙂

It’s weird, because 3 weeks ago I told Jason Calacanis, who suggested in email that we enter, that I didn’t think we should go up on stage at LAUNCH. We don’t (yet) do UI, and start-up events are heavily oriented towards sexy UI and ideas that can be explained in a couple of minutes. We were so busy already, it seemed like a recipe to do something mediocre if we threw together a demo. Instead I asked if we could just hang out in the LaunchPad with the 99 other start-ups and talk to people passing by. Today the judges went through the LaunchPad and talked to all the start-ups. Several told me we should go up on stage to do a 3 minute presentation, so I thought “why not?”, threw together some Keynote slides, opened some browser tabs (on the quick WeMet.At app written by Nicholas Tollervey in a few hours, and the increasingly great Fluidinfo Explorer (that link points at the ReadWriteWeb top-level namespace in Fluidinfo) written by Pier-Andre Parent, who we’ve never even met) and went for it.

The whole Fluidinfo team has been working really hard towards LAUNCH for the last 3 weeks. Everything at LAUNCH and recently announced on our blog has been built by all of us. It’s nice to win a prize, because after being funded we decided to be quiet, keep our heads down, build and train a team, and not even try to get people to use Fluidinfo until 2011. In January we began our first outward-facing efforts, building and releasing quite a few writable APIs. It’s still early days yet, and we have something very cool right around the corner.

Thanks to all the other great startups and the organizers at LAUNCH, especially Jason Calacanis, Tyler Crowley, and Jason Krute. Standing up for 20 hours and talking for about 30 was never so much fun!

February 23, 2011

Putting domain names onto data with Fluidinfo

Filed under: Data,Essence,Writable APIs — Terry Jones @ 11:15 am

Internet domain names can be thought of as a mechanism for attaching trust and reputation to digital information. We do this in two major ways: (1) by using domain names in the URLs of web pages, and (2) by putting them in the sender’s “From” address of email messages.

To give a concrete example, suppose you see some shoes for sale on a web page. If you look at the page URL and see the amazon.com or zappos.com domain name, trust and reputation knowledge springs instantly to mind. You know the quality is probably good, the price competitive, and that if the shoes are lost in shipping you’ll be sent another pair for no charge. On the other hand, if you see ebay.com in the URL, a different matrix of trust and reputation knowledge will spring to mind. A similar thing happens if you get email from someone you’ve never met. If you see stanford.edu or forbes.com in the email “From” line, reputation information springs to mind.

Looked at in this way, domain names are small tokens that we send alongside other pieces of content such as web pages and emails. The domain name carries vital trust and reputation information. Recognition and trust in domain names is globally distributed, spread variously through the brains of most of the people on the planet, with its integrity guaranteed by DNS. Domain names make the internet useful. Without them, digital information online would be almost useless as we could not confidently trust any 3rd party data.

Question: given that we can attach domain names to web pages and email messages, can we find a way to attach them to other things?

Domain names on data

We’re excited to announce that Fluidinfo now makes it possible to put domain names onto individual pieces of data.

To illustrate, the image on the right shows a fanciful example book object in Fluidinfo (large version). The tag names on the object are colored. You’ll see that some of them contain domain names: amazon.com/price, barnesandnoble.com/price and vintage.com/epub. Tags in Fluidinfo can have values, as illustrated by the amazon.com/price tag whose value for this book is $19.

The combination of a Fluidinfo tag name containing a domain and an associated tag value is exactly like a URL containing a domain name and an associated HTML value (i.e., a web page) or an email message with a domain name in its From line.

Because Fluidinfo objects don’t have owners (their tags do, though), any number of domain owners are free to put their information, branded with their domain name, onto any Fluidinfo object.

A killer combination: writable APIs with domain-branded data

Fluidinfo automatically provides a writable API for all its data. By allowing for domain names on data, domain holders who want to publish information about their products can now do so with an API that has three major advantages:

  • Your data is branded with your domain name.
  • Your data lives in a writable ecology of related data, collecting on the same Fluidinfo objects. This allows for search across data from different users and domains, put there by different applications. It allows for additional data of all kinds, for mashups, and for customization, personalization, and filtering.
  • Fluidinfo has a flexible permissions system at the level of its tags, so you maintain full control of your own data. You can make it public or private, or can allow or disallow access for specific others.

Because Fluidinfo objects are fine grained, composed simply of tags with values as in the image above, applications can fetch, search on, or combine specific pieces (or combinations) of data provided by different trusted sources with single requests. There is a general principle here: information becomes more useful and valuable when it is stored in context. This is illustrated vividly by Google, which collects web pages into one place to enable search, and by Wikipedia, which allows people to pool related information. Although these examples have very different models of trust and reputation, they both illustrate the underlying principle.

Getting your domain name in Fluidinfo

To start using your domain in Fluidinfo, first sign up, using your domain name as your user name. Our sign-up system will recognize that the username is a domain and will send you an email telling you how to prove that you control the domain. Once that’s done, you can begin using Fluidinfo to upload information branded with your domain and to provide an API for others (or for your own company) to find your products or otherwise use the information you make available.

In other words, all Fluidinfo usernames that correspond to actual internet domains are automatically reserved for their owners. Besides preventing a chaotic land grab, this is how we can guarantee to people seeing information in Fluidinfo that the value of a Fluidinfo tag whose name includes a domain name can be trusted exactly as it would be if that domain appeared in a web page URL or email From address.

So there you have it… domain names on data. We’re very excited to see where this will lead and we’re actively building out some writable APIs with domain-branded data. You can too. Claim your domain name in Fluidinfo right now.

February 15, 2011

How I made a writable API for Union Square Ventures in an hour

Filed under: APIs,Essence,Howto,Programming,Writable APIs — Terry Jones @ 9:11 am

Image: Eric Archivell

I was mailing Fred Wilson and Albert Wenger of Union Square Ventures late last year, talking about Fred’s article Giving every person a voice. Fred said

I hadn’t really thought that we are all about shrinking the minimal viable publishing object, but that may well be true in hindsight.

I wanted to illustrate Fluidinfo as doing both: providing a minimal viable way to publish data (with an API), and also giving everyone a voice. So I decided to build Union Square Ventures a minimal API, and to then add my voice. In an hour.

A minimal viable API for USV

USV currently has 30 investments. If you want to get a list of the 30 company URLs, how would you do it? A non-programmer would have no choice but to go to the USV portfolio page, and click on each company in turn, then right-click on the link to each company’s home page and copy the link address, and then add that URL to your list. That process is boring and error prone.

If you’re a programmer though, you’d find this ridiculously manual. You’d much rather do that in one command, for example if you’re collecting information on VC company portfolios, perhaps for research or to get funded. Or if you were building an application, perhaps to do what Jason Calacanis is doing as part of the collecting who’s funding whom on Twitter and Facebook. You want your application to be able to fetch the list of USV company URLs in one simple call.

So I made a unionsquareventures.com user in Fluidinfo (sign up here), did the repetitive but one-time work of getting their portfolio companies’ URLs out of their HTML (so you wouldn’t have to), and added it to Fluidinfo. I put a unionsquareventures.com/portfolio tag onto the Fluidinfo object about each of those URLs. In other words, because Fluidinfo has an object for everything (including all URLs), I asked it to tag that object.

That was just 7 lines of code using the elegant and simple Python FOM library for Fluidinfo written by Ali Afshar:

import sys
from fom.session import Fluid

fdb = Fluid()
fdb.login('unionsquareventures.com', 'password')
urls = [i[:-1] for i in sys.stdin.readlines()] # Read portfolio URLs from stdin

for url in urls:
    fdb.about[url]['unionsquareventures.com/portfolio'].put(True)

As a result, using the jsongrep script I wrote to get neater output from JSON, I can now use curl and the Fluidinfo /values method to get the list of USV portfolio companies in the blink of an eye:

curl 'http://fluiddb.fluidinfo.com/values?query=has%20unionsquareventures.com/portfolio&tag=fluiddb/about' |
jsongrep.py results . . fluiddb/about value | sort
u'http://amee.cc'
u'http://getglue.com'
u'http://stackoverflow.com'
u'http://tumblr.com'
u'http://www.10gen.com'
u'http://www.boxee.tv'
u'http://www.buglabs.net'
u'http://www.clickable.com'
u'http://www.cv.im'
u'http://www.disqus.com'
u'http://www.edmodo.com'
u'http://www.etsy.com'
u'http://www.flurry.com'
u'http://www.foursquare.com'
u'http://www.hashable.com'
u'http://www.heyzap.com'
u'http://www.indeed.com'
u'http://www.meetup.com'
u'http://www.oddcast.com'
u'http://www.outside.in'
u'http://www.returnpath.net'
u'http://www.shapeways.com'
u'http://www.simulmedia.com'
u'http://www.soundcloud.com'
u'http://www.targetspot.com'
u'http://www.twilio.com'
u'http://www.twitter.com'
u'http://www.workmarket.com'
u'http://www.zemanta.com'
u'http://zynga.com'

There you have it, a sorted list of all Union Square Ventures portfolio companies’ URLs, from the command line. I can do it, you can do it, and any application can do it.

The jsongrep.py program can also be used to pull out selective pieces of the output. For example, which of the companies have “ee” in their URL?

curl 'http://fluiddb.fluidinfo.com/values?query=has%20unionsquareventures.com/portfolio&tag=fluiddb/about' |
jsongrep.py results . . fluiddb/about value '.*ee' | sort
u'http://www.meetup.com'
u'http://amee.cc'
u'http://www.indeed.com'
u'http://www.boxee.tv'

So maybe, in order to be funded by USV, it helps to have “ee” in your URL? 🙂

What about USV companies that don’t have “.com” URLs?

curl 'http://fluiddb.fluidinfo.com/values?query=has%20unionsquareventures.com/portfolio&tag=fluiddb/about' |
jsongrep.py results . . fluiddb/about value '.*(?

OK, these things are geeky, but that's part of the point of an API: to enable applications to do things. We've made the portfolio available programmatically, and you can immediately see how to do fun things with it that you couldn't easily do before. In fact, it's quite a bit more interesting than that. As a result of doing this work, I can tell you that there was a company listed a couple of months ago on the portfolio page that is no longer there. And there's a company that's been invested in that's not yet listed. That's a different subject, but it does illustrate the power of doing things programmatically.


This is a minimal viable API for USV because there's only one piece of information being made available (so far). But an API it is, and it's already useful.


It's also writable.


Giving everyone a voice


In a sense we've just seen that everyone has a voice. USV put a tag onto the Fluidinfo objects that correspond to the URLs of their portfolio companies and they didn't have to ask permission to do so.


But what about me? I'm a person too. I've met the founders of some of those companies, so I'm going to put a terrycojones/met-a-founder-of tag onto the same objects. Fluidinfo lets me do that because its objects don't have owners, its permission system is instead based at the level of the tags on the objects.


So I wrote another 7 line program, like the one above, and added those tags.  I also added another USV tag, called unionsquareventures.com/company-name. Let's pull back just the names of the companies whose founders I've met:


curl 'http://fluiddb.fluidinfo.com/values?query=has%20unionsquareventures.com/portfolio%20and%20has%20terrycojones/met-a-founder-of&tag=unionsquareventures.com'/company-name |
jsongrep.py results . . . value | sort
u'Bug Labs'
u'Foursquare'
u'GetGlue'
u'Meetup'
u'Shapeways'
u'Stack Overflow'
u'Tumblr'
u'Twitter'
u'Zemanta'


Isn't that cool?  I do indeed have a voice!


You have one too. If you sign up for a Fluidinfo account you can add your own tags and values to anything in Fludinfo. And you can use Fluidinfo, just as I've illustrated above, to make your own writable API. See also: our post from yesterday, What is a writable API?
« Newer Posts — Older Posts »

Powered by WordPress