I think the appearance of Google App Engine and the “massively scalable Google Data Store” buzz took a lot of web app developers in the GAE direction.
GAE has a free level and I think Amazon has been forced to offer a free level due to market pressures.

The major reason for using SDB or GDS for that matter is the data model. Both of these allow dynamic attributes which are very important in modern semi-structured data.

I am working on a project to modernize technology for managing academic bibliographic items – this is an NSF funded project and the core need here is dynamic attributes because on of the core formats ‘bibtex’ is an extensible format. Also the need for a data base which is a key value store where values can be lists == keys can have multiple values.

While you can model this in a relational database using a key-value table, unifying the dynamic and static attributes in queries gets ugly fast.

Both GAE (through its Expando model) and SDB just natively, allow attributes to be added at run time and for keys to have multiple values. If your model needs this and you are not at the bleeding edge of data size then these datastores buy you a lot.

Currently GAE has some obscurely expressed limit on resource cost of a query and web request and it times out when you exceed it but you really don’t have a good idea how to tune it expect to try to do “less bad things”

SDB on the other hand tells you that if your query exceeds a certain well defined CPU usage it will time out – this suggests that if your queries are the equivalent of table scans or are above some level of computational cost you SDB will not work.

One significant disadvantage that GAE has is that the database is not exposed via REST’ish URL’s or via web services of any kind. It is up to the app you write to do that.

I have noticed that AWS continues to incrementally improve each of their web services in feature set and reduced cost as they scale up. I think the SDB data model takes getting used to and also early users of EC2 and S3 often experienced outages so SDB may be in “growing pain stage”

I am currently using GAE for my bibliographic item store and will be experimenting with SDB as well on this real world app and will keep you informed.

Nitin

I think the appearance of Google App Engine and the “massively scalable Google Data Store” buzz took a lot of web app developers in the GAE direction.
GAE has a free level and I think Amazon has been forced to offer a free level due to market pressures.

The major reason for using SDB or GDS for that matter is the data model. Both of these allow dynamic attributes which are very important in modern semi-structured data.

I am working on a project to modernize technology for managing academic bibliographic items – this is an NSF funded project and the core need here is dynamic attributes because on of the core formats ‘bibtex’ is an extensible format. Also the need for a data base which is a key value store where values can be lists == keys can have multiple values.

While you can model this in a relational database using a key-value table, unifying the dynamic and static attributes in queries gets ugly fast.

Both GAE (through its Expando model) and SDB just natively, allow attributes to be added at run time and for keys to have multiple values. If your model needs this and you are not at the bleeding edge of data size then these datastores buy you a lot.

Currently GAE has some obscurely expressed limit on resource cost of a query and web request and it times out when you exceed it but you really don’t have a good idea how to tune it expect to try to do “less bad things”

SDB on the other hand tells you that if your query exceeds a certain well defined CPU usage it will time out – this suggests that if your queries are the equivalent of table scans or are above some level of computational cost you SDB will not work.

One significant disadvantage that GAE has is that the database is not exposed via REST’ish URL’s or via web services of any kind. It is up to the app you write to do that.

I have noticed that AWS continues to incrementally improve each of their web services in feature set and reduced cost as they scale up. I think the SDB data model takes getting used to and also early users of EC2 and S3 often experienced outages so SDB may be in “growing pain stage”

I am currently using GAE for my bibliographic item store and will be experimenting with SDB as well on this real world app and will keep you informed.

Nitin

The costs and benefits list is quite long and complex so I won’t get into it here, but overall I think that only a tiny fraction of applications will find it preferable to MySQL.

The costs and benefits list is quite long and complex so I won’t get into it here, but overall I think that only a tiny fraction of applications will find it preferable to MySQL.

That’s very interesting. I’ve spent many years thinking about and building the product that Fluidinfo (my company) is now working to release. For a long time I resisted the “database” label. But I had enormous trouble getting people (mainly potential investors) to understand what we were trying to do. Finally I relented and started calling it FluidDB. I find the “database” part of the name slow and boring, and now you’ve pointed out that it might even be seriously misleading. We’ve cycled through a few names (one of the advantages of having something that takes a while to build is that you can play with different names), none of them particularly satisfying. It’s a product (initially) for programmers, so giving it a name that they can recognize seems sensible. But if they recognize it and think “that’s not a database”, it’s not good.

Hmmmm…..

Thanks,
Terry

That’s very interesting. I’ve spent many years thinking about and building the product that Fluidinfo (my company) is now working to release. For a long time I resisted the “database” label. But I had enormous trouble getting people (mainly potential investors) to understand what we were trying to do. Finally I relented and started calling it FluidDB. I find the “database” part of the name slow and boring, and now you’ve pointed out that it might even be seriously misleading. We’ve cycled through a few names (one of the advantages of having something that takes a while to build is that you can play with different names), none of them particularly satisfying. It’s a product (initially) for programmers, so giving it a name that they can recognize seems sensible. But if they recognize it and think “that’s not a database”, it’s not good.

Hmmmm…..

Thanks,
Terry

yes and no.

Yes in terms of marketing. Amazon have completely blundered this area, and really should drop the “DB” tag as quickly as possible. It is not a DB in the sense that most developers recognize and here in lies the confusion.

We are sold the wrong product to begin with.

Instead it is a global registry; nothing more, nothing less. A registry is not a database by any stretch.

We use SimpleDB to manage all our instances running in the cloud, in a variety of different of different providers, while we power some of the largest sites in Europe. Without it, we would have to be doing a lot of extra management, and without a doubt it is a very slick and useable service.

The problem is, people don’t know what to use it for; they think database, and are left with effectively a hashmap in the cloud.

yes and no.

Yes in terms of marketing. Amazon have completely blundered this area, and really should drop the “DB” tag as quickly as possible. It is not a DB in the sense that most developers recognize and here in lies the confusion.

We are sold the wrong product to begin with.

Instead it is a global registry; nothing more, nothing less. A registry is not a database by any stretch.

We use SimpleDB to manage all our instances running in the cloud, in a variety of different of different providers, while we power some of the largest sites in Europe. Without it, we would have to be doing a lot of extra management, and without a doubt it is a very slick and useable service.

The problem is, people don’t know what to use it for; they think database, and are left with effectively a hashmap in the cloud.

We do the same: EC2 + S3 + Postgresql

And thanks!

Terry

We do the same: EC2 + S3 + Postgresql

And thanks!

Terry

@Terry Jones – Good luck with FluidInfo.

@Terry Jones – Good luck with FluidInfo.

It’s interesting that nobody has mentioned here ThruDB, which uses S3, and they say is faster and cheaper than SDB.

It’s interesting that nobody has mentioned here ThruDB, which uses S3, and they say is faster and cheaper than SDB.