Below is an interview I did yesterday with Mac Slocum at the O’Reilly TOC conference in New York. We discuss writable book APIs and why they matter, as well as talking about what that might mean for publishers, readers, and the publishing process in general. (You can also see the interview on YouTube.)
When we released the Fluidinfo API for Boing Boing two weeks ago, Simon Willison noted on his blog:
“Fluidinfo really is a fascinating piece of software.” …. “Writable APIs are much less common than read-only APIs – Fluidinfo instantly provides both.”
If you search online to try to discover what people mean by a “writable API”, it’s hard to find anything that merits the name. So what did Simon mean? What is a writable API?
Both Simon and the team at Fluidinfo think “writable API” should be a kind of shorthand for an API that provides access to underlying data that is writable. This is not meant in the trivial already-possible sense wherein you pass data to an API method that stores them into a database you can’t otherwise access. We mean it in a more fundamental sense: that the underlying data is writable. That anyone or any application can directly access the data storage layer and add new information to it – without the knowledge of the people who stored the original data. That sounds pretty radical. But if you have a model of control in which objects are not owned but their pieces are, it’s not scary at all. In fact it’s liberating.
And, you guessed it, Fluidinfo has exactly that model of control. Any information stored into Fluidinfo instantly has a writable API in the sense just described. Let’s see a concrete example from the recent Boing Boing data imported into Fluidinfo.
Below is an illustration of an object in Fluidinfo, showing a subset of the tags that are on every Fluidinfo object representing a Boing Boing article. (The image was generated using Nick Radcliffe‘s fun About Tag image generator for Fluidinfo objects. Click the image to see the all its tags.)
Simply by virtue of being stored in Fluidinfo, Boing Boing instantly got an API for all their articles. The API lets you find Boing Boing articles, as represented by objects in Fluidinfo, via querying on tags such as those shown on the object above. For example, you can use the API to find Boing Boing articles published in December 2008 that were written by Cory Doctorow. Or you can get a list of all the Boing Boing articles that contain a reference to the domain www.whitehouse.gov. (You can see details of these sorts of queries in our article on Mining the Boing Boing API.)
Those kinds of searches on Boing Boing data were not previously possible. We put the whole thing together in a single evening, which illustrates how simple it can be to make a Fluidinfo-fueled API for your own information. As cool as these examples are, though, they’re just reading & searching Boing Boing controlled data, as with a traditional API. What about writing?
The tags on the object above were put there by the Fluidinfo user named boingboing.net. That user controls those tags, and has given the rest of us read permission. But no-one owns the Fluidinfo object that the tags are on. As a result, anyone with a Fluidinfo account (sign up here) can add any information to the exact same object.
To give a very simple example, suppose someone wrote a simple browser extension (or extensions) that let Boing Boing readers mark stories as being funny or not suitable for work. Two users, Alice and Bob, might then put alice/funny and bob/nsfw tags onto the above object. Assuming I had read permission on those tags, I could then find Boing Boing articles by Cory Doctorow that Alice enjoyed and Bob found too risqué for work. Someone else could write a browser extension that popped up a warning about NSFW content based on Bob’s tag. In fact, take a proper look at the object above, you’ll see that I have added a terrycojones/nsfw tag to it (terrycojones is my username in Fluidinfo).
That’s customization and personalization – in our hands. It’s adding data to the exact same objects that Boing Boing created, combining their data and ours as we please, and all without stopping to ask permission or requiring that a database administrator or programmer anticipate our idiosyncratic needs. Boing Boing and any applications they create, may not be aware of, care about, or even be able to detect the new data (depending on permissions).
In other words, we can say that Boing Boing has a writable API, because other people and other applications are always free to add information to the same objects that the Boing Boing API is providing access to. The same applies to any application or API that uses Fluidinfo. A writable API opens the door onto a very different world, allowing unlimited possibilities for mash-ups, new applications, extensions, widgets, etc. It allows arbitrary customization and personalization. Fluidinfo acts like a universal metadata engine, providing guaranteed write access to anything, with a permissions system at the level of the tag, not the object.
We’ll give another example of a simple but fun writable API tomorrow. Next week we’ll release a much more substantial one at the LAUNCH conference in San Francisco. We’re really excited about it, and have a series of not-to-be-missed upcoming blog posts on what we’ve been up to.
Stay tuned!
Image: Education Week
I added marshallk.com/top-blogs/data and marshallk.com/top-blogs/geo tags to the Fluidinto objects that correspond to the URLs in his lists (Fluidinfo has an object for everything; in each case I put the tags onto the logical object in Fluidinfo: the one object whose fluiddb/about value is the URL in question.)
You can then do things like this:
$ curl 'http://fluiddb.fluidinfo.com/values?query=marshallk.com/top-blogs/data%3c%3d10&tag=fluiddb/about&tag=marshallk.com/top-blogs/data' |
jsongrep.py results id '.*'
{u'fluiddb/about': {u'value': u'http://www.readwriteweb.com/cloud'},
u'marshallk.com/top-blogs/data': {u'value': 3}}
{u'fluiddb/about': {u'value': u'http://cloud.gigaom.com'},
u'marshallk.com/top-blogs/data': {u'value': 2}}
{u'fluiddb/about': {u'value': u'http://flowingdata.com'},
u'marshallk.com/top-blogs/data': {u'value': 8}}
{u'fluiddb/about': {u'value': u'http://highscalability.com'},
u'marshallk.com/top-blogs/data': {u'value': 7}}
{u'fluiddb/about': {u'value': u'http://www.calculatedriskblog.com'},
u'marshallk.com/top-blogs/data': {u'value': 6}}
{u'fluiddb/about': {u'value': u'http://www.fivethirtyeight.com'},
u'marshallk.com/top-blogs/data': {u'value': 5}}
{u'fluiddb/about': {u'value': u'http://www.guardian.co.uk/news/datablog'},
u'marshallk.com/top-blogs/data': {u'value': 9}}
{u'fluiddb/about': {u'value': u'http://www.informationisbeautiful.net'},
u'marshallk.com/top-blogs/data': {u'value': 10}}
{u'fluiddb/about': {u'value': u'http://www.zerohedge.com'},
u'marshallk.com/top-blogs/data': {u'value': 1}}
{u'fluiddb/about': {u'value': u'http://freakonomics.com/blog'},
u'marshallk.com/top-blogs/data': {u'value': 4}}
Those are the top 10 on Marshall’s data list (unsorted, obviously). I’ve cleaned up the output using my jsongrep.py program described and available here.
More interestingly, you can see if any sites are on both of Marshall’s lists:
$ curl 'http://fluiddb.fluidinfo.com/values?query=has%20marshallk.com/top-blogs/data%20and%20has%20marshallk.com/top-blogs/geo&tag=fluiddb/about'
{"results": {"id": {"a2e56723-453a-44e5-bd91-5576d0615c8e": {"fluiddb/about": {"value": "http://blog.simplegeo.com"}}}}}
Just a single blog is in both lists: http://blog.simplegeo.com.
So far, so good.
About half an hour ago, I saw a tweet from Daniel Tunkelang (the mind behind TunkRank) saying that eCairn have just released some work based on Marshall’s data, producing a list of 500 top data blogs! Cool.
So I’ve just imported that data to Fluidinfo too, adding a ecairn.com/top-data-blogs tag to the object for each URL on their list. The value of each tag, as with Marshall’s data, is the ranking on the eCairn list.
Let’s see how many blogs are on both lists:
curl 'http://fluiddb.fluidinfo.com/values?query=has%20marshallk.com/top-blogs/data%20and%20has%20ecairn.com/top-data-blogs&tag=fluiddb/about' | jsongrep.py results id '.*' | wc -l
117
Not as many as I expected. But there are some small differences in the URLs used, for example Marshall’s list had http://kaushik.net/avinash whereas the eCairn list has http://www.kaushik.net/avinash. This would be easy to clean up, and of course it’s also possible just to tag the object for both URLs in Fluidinfo.
You can do the Fluidinfo query has marshallk.com/top-blogs/data except has ecairn.com/top-data-blogs to see the sites that Marshall has in his list but which do not appear in the eCairn list, such as Marshall’s #12, http://blog.sqlauthority.com. eCairn’s calculation might have put them in the lower 500 of their list of 1000 (the eCairn article only gives their top 500). There are plenty of other interesting queries too, but this post is long enough already.
So there you go, a fun bit of playing with more data blog data with Fluidinfo. One of these days we’ll even make it into one of these lists 🙂
Here’s the tiny bit of Python code I just wrote to add the data. It uses the Python FOM library for Fluidinfo written by Ali Afshar:
import sys
from fom.session import Fluid
fdb = Fluid()
fdb.login('ecairn.com', 'password')
urls = [i[:-1] for i in sys.stdin.readlines()]
for rank, url in enumerate(urls):
fdb.about[url]['ecairn.com/top-data-blogs'].put(rank + 1)
On Feb 9th I gave a talk the the NYC Ignite event, titled “What the Post-It Note Can Teach Us About Apps and Data.” Below are my 20 slides (21 if you could image credits). These were advanced automatically every 15 seconds during the 5-minute talk. I’ll post a link to the video once it’s up.
While the topic may not seem to have anything to do with Fluidinfo, there is a very close connection. I’ll write about that another time.
We’re really happy to announce that Marc Hedlund has joined the Fluidinfo board!
I’ve gotten to know Marc slowly over the last 10 years. We first met very briefly when he was CEO of the Popular Power, a San Francisco start-up. Nelson Minar (Marc’s co-founder) and Derek Smith, two of my close friends who are very close to Marc, were both working there. Nelson and Derek, as well as several others including Fluidinfo investor and advisor Tim O’Reilly have sky-high opinions of Marc. Hearing regular off-the-charts superlatives about Marc over the years always kept me interested to someday know him better.
Marc was present at my first ever (abysmal!) solo VC pitch for Fluidinfo, to the ill-fated Bryce Roberts and Mark Jacobson of OATV in early 2007. During the presentation, Marc interrupted to ask if he could take a photo of my slide titled “Revenue”. I think he wanted it as an example of how not to pitch a VC. I’ve never forgotten. He snapped the pic, resumed his seat, and told me to carry on 🙂
Marc has a ton of experience. He founded and led Lucas Online, the internet subsidiary of Lucasfilm, was director of engineering at Organic Online, and was also CTO at Webstorm. After Popular Power he was VP of Engineering at Sana Security, and then Entrepreneur in Residence at OATV, gaining intimate knowledge of the world of venture capital and interacting with hundreds of start-up companies. Marc then co-founded Wesabe where he was Chief Product Officer before becoming CEO. These days he’s Chief Product Office at Daylife in New York.
As you can probably imagine, we’re honored and excited to have Marc involved at Fluidinfo.
Image: brandon shigeta
Our core and driving passions are based on thinking about how humans work with information: how we find, consume, create, remember, organize, and share. We’re looking for a UI/UX wizard who also thinks and cares deeply about these things, and who has experience and exquisite taste in building elegant and super low-friction interfaces to information. Someone who can tell us what Tufte got wrong, and why.
Most of our computational experiences take place in read-only environments, or ones in which we can only add information in ways that have been anticipated and approved. This read-only world makes working with information using a computer very unlike the way we work with information in the natural world. A read-only world inhibits ad hoc, creative, unanticipated uses of information. It inhibits personalization and customization and therefore effective filtering. It is wrong because it puts information ontology ahead of evolution, inhibiting the natural emergence of information communication conventions like @addressing and hashtags.
With Fluidinfo we’re trying to imagine and build a computational world in which we always have write permission. In which people are free to add information to anything – to personalize and customize, to filter and search on their information and combinations of information. A computational world in which we can collectively make information more valuable by storing it in context.
There’s a huge and challenging UX/UI component to this. We believe humans are actually very good at working with raw information. It’s applications that are confusing. Part of our challenge is to create an interface to anything (i.e., in a web of things sense) and to do so in as transparent a manner as possible.
We’ve been thinking about these ideas for years and have built Fluidinfo as a platform to support this kind of simple and always-writable information storage. Now we want to put a face on it and we’re looking for someone truly great to join us.
About Fluidinfo Inc.
Fluidinfo is an angel and VC backed start-up based in New York, which is where we want you. (Our development team is currently distributed.) We have a world-class set of investors, including: Betaworks, IA Ventures, RRE Ventures, Lerer Ventures, Chris Dixon and the Founder Collective, Esther Dyson, Tim O’Reilly, Joshua Schacter, Michael Parekh, and Andrew Rasiej.
We’ve been quietly working on architecture since our seed funding in 2010, but are now starting to show the world the kinds of things we want to enable. Just for a small taste, read about the writable API we recently built for BoingBoing in a single evening, or how to use Fluidinfo to put metadata onto tweets. To get a flavor of what Fluidinfo is aiming at, see Truly Social Data, Information. Naturally, Fluidinfo as a universal metadata engine, and Kaleidoscope: 10 takes on Fluidinfo.
We also have some exciting news coming up on Feb 14 at the O’Reilly Tools of Change conference in NY, so keep an eye out for that too.
About you
We’re not going to tell you in detail what skills you need, because you’ll hopefully know that better than we do.
As a guideline though, in terms of technology for UI, we’re mainly interested in things built using ubiquitous web tools and standards like HTML5 and Javascript. We’re less keen on Flash, and cannot stomach heavyweight proprietary UI platforms. Anything that requires a clunky download is a non-starter. We do everything with Linux and will be very happy if you know your way around that world too. Experience with mobile or desktop UI/UX will also be valuable.
The three most important things we’re looking for are, in order of importance: 1) brilliance in UX/UI thought and execution; 2) proficiency in building dynamic web content with Javascript / modern HTML etc; 3) graphic design skill or experience working with graphic design teams to convert working prototypes into beautiful products. 1 and 2 are much more important than 3.
Above all though, you need to be able to show us interactive interfaces you’ve built or designed, be passionate about UI/UX, and be able to talk convincingly and in depth about what makes things work and not work.
Hiring process
To apply, send email to jobs at fluidinfo dotcom and include:
Hiring will involve initial telephone/skype interviews, to be followed by in-person interview(s).
As of today, we’re dropping FluidDB as a product name and will just use Fluidinfo (which is of course also the name of our company). We’ve obviously put a lot of energy into the FluidDB name so it feels bad to know that some of that will be squandered, and that we might create short-term confusion with the change. But the downsides of a bad name are both real and important, and it’s time to fix the mistake. I’ve run the name change idea past dozens of people over the last two months, with almost universal approval—some of it very enthusiastic. So from now on it’s Fluidinfo, and only Fluidinfo.
Here are the main reasons for the change:
We very reluctantly began using “db” with the only purpose being to try to help VCs understand what we were building. That was before ideas of “cloud computing” and data as a service became well understood. It was helpful back then, but is not helpful now. Fittingly, after living with the baggage for a long time, the straw that broke the camel’s back was a VC who told me he almost didn’t take our meeting because he assumed from the name that we were just another of those NoSQL databases. When he said: One way or another I’m going to get you to drop that name, I knew we’d gone full circle—an inappropriate name had finally been recognized as being detrimental by the kind of person the name was supposed to be helping.
The name change can already be seen on our funky website (desperately in need of a facelift) and in the Fluidinfo documentation. We’ve also switched our Twitter account to @fluidinfo and are now using the #fluidinfo channel on IRC.
We’ll soon be moving some web pages around behind the scenes to improve their URLs, but will make sure the old links still work. If you’re interested in technical details of the API, please feel free to ask questions below, to join us in the Fluidinfo users mailing list, or to drop by the #fluidinfo channel on irc.freenode.net.
And….. stay tuned, we have some exciting Fluidinfo news right around the corner.
I’ve just arrived back at Betaworks, who led the investment in Fluidinfo, after a 3-month absence. It’s an amazing environment, somehow buzzing with both intensity and diversity. A lot of people are trying to get an answer to the question “What is Betaworks?” The most obvious (and wrong) answer is that Betaworks is an incubator. I’ve spent time on and off thinking about the question, and I don’t think there’s a single good / conventional answer. Betaworks is something else. So here’s my informal answer, which I think works pretty well:
Betaworks is a Strange Attractor.
From that Wikipedia page: An attractor is a set towards which a dynamical system evolves over time. Of course I don’t mean Betaworks is literally a strange attractor, though their logo looks like an outward-facing version of the attractor on the right (more images). I’ll illustrate what I mean by describing who I ran into at Betaworks the last couple of times I arrived in New York and went into the office.
On June 17, 2010, I flew to NY from Vegas on a very early flight and got to Betaworks mid-morning. Just inside the door of the office I ran into Brady Forrest and Mike Loukides, both of whom work for O’Reilly. I went for coffee with Brady and spent at least an hour talking to Mike. Later in the afternoon Tim O’Reilly showed up. That’s a pretty interesting batch of folks to have run into at Betaworks. But it’s much more interesting than that. Get this: none of the three O’Reilly people knew that the others were going to Betaworks. At least one of the three didn’t know the others were even in town. Think about it. I find it quite extraordinary. NYC is a big place, and though there’s a ton of stuff going on here, there’s apparently only one place to be. Here was the first really tangible evidence for me that Betaworks have become a strange attractor of some kind, and a powerful one.
Second example: Yesterday when I arrived in the office, after several months away, I immediately ran into a bunch of people I know – none of whom works out of the Betaworks offices: Caterina Fake (Flickr, now Hunch), Jyri Engeström (Jaiku, now Google), Reshma Sohoni (Seedcamp), Iain Dodsworth (Tweetdeck), and Mika Salmi (many things). Like me, Mika also lives in Barcelona. But where do I unexpectedly run into him…? At Betaworks. How is that possible? Why were all these great people in the office today? Was there some event going on? No. So what’s going on here?
These are just a couple of days, chosen because they were both first days back in town. There are many similar days. Betaworks is an attractor in other senses too – my examples omit the companies and people who are in the office every day, creating the gravitational pull that’s attracting so many extraordinary visitors.
That’s it for now. I don’t think anyone can pin down exactly what Betaworks is, and it doesn’t really matter. But we can all be strangely attracted.
We’re happy to announce a second closing of Fluidinfo‘s Series A investment round. We’ve raised another $170K, taking the round to just under $1M in total. Some of the people investing in the second closing are:
Michael Parekh: A Wall Streeter for over 20 years and former partner at Goldman Sachs, Michael founded and helped to build the Internet Research effort at the firm (twitter, more info).
Esther Dyson: who seed funded Fluidinfo in late 2007, and who’s been a huge source of help ever since (and before). We’re thrilled to have her following on in this round (twitter, more info).
David Snow: the Editor in Chief of PEI Media. David also participated in the seed funding of Fluidinfo (twitter, more info).
Ted Carroll and Earl Macomber: who were both also seed-stage backers of Fluidinfo. Ted and Earl are the managing principals of traditional information and media focused private equity firm Noson Lawen Partners, and have again made personal investments (twitter, more info).
Ed Carroll: who was also a seed stage Fluidinfo investor. Ed is now entering his senior year in high school and hopes to attend USC next year as a freshman at The Marshall School. Ed spent a month at Marshall this summer and walked away with Top Five honors in their Entrepreneurialism program. Good luck Ed!
There are three other new investors who also came into the round, but who prefer not to be mentioned publicly at this stage (so you’ll have to ask us about them privately :-)). The above all join Betaworks, IA Ventures, RRE Ventures, Lerer Ventures, Chris Dixon & Founder Collective, Joshua Schacter, Andrew Rasiej, Ross Williams, and Esther Speight as Fluidinfo Series A investors.
Our thanks to everyone!
Tim O'Reilly - out standing in his field
I’m an unabashed fan of Tim, his company, and everyone I’ve ever met who works at or has worked at O’Reilly (especially Sara Winge). I’ve been talking to Tim on and off about FluidDB since March of 2007 after being introduced by Esther Dyson. Tim would blog about something, and I’d email him and say “FluidDB will be able to do that” or (more recently) “FluidDB can do that.” When we first met, we spent 90 minutes together and I showed him a demo of a few things. He drilled down hard in the first 2 minutes: “Tell me what’s different about it.” So I went for it. When the meeting was over, Tim left the room and went to the elevator to leave. I was packing up my stuff when suddenly he was back, wanting to ask and suggest more. He came back to the room four times, lastly to get my phone number 🙂
Since then, Tim has been extremely generous, introducing us to many great people. You can see who very easily in the graph of introductions I put together over the years as I talked to people about FluidDB. He made the introduction that led through Gerry Campbell to John Borthwick and Andy Weissman at Betaworks who led the Fluidinfo investment. Tim invited me to the Social Graph Foo camp in 2008, to a Science Foo camp, and to two general Foo camps, and he’s been helping me in the (ongoing) attempt to get a US visa.
It’s a personal thrill to have Tim formally involved with Fluidinfo. During years working on what at times seemed like the dark side of the tech moon in Barcelona, to have had Tim and Esther (and others!) behind us along the way has been wonderful. I often feel I want us to succeed as much for them as for anyone else.
Powered by WordPress
