I like this phrasing of the debate about App Engine’s role, from Doug Cutting: Cloud: commodity or proprietary? (via). (Well, I like it more than the sharecropping phrasing referenced in my last post.) I guess I’m excited about this because like Doug I do want a "cloud" of sorts, and this is a move towards that in a way that makes sense to me. Maybe to state my motivations more clearly: I hate computers. I really hate them a lot. I dream of some world of Platonic ideals where software just exists, and existence that state it works. App Engine feels like a strong move in the direction of computers-not-mattering. What does App Engine run on? I don’t care! Where is the server located? I don’t care! What is BigTable? I am comfortable thinking of it only in its abstract sense, an API that works, and I don’t know how, nor do I need to know how. I don’t need to know these things if they just work. Always. Totally reliably. I’m not shy about digging into code. I tend to be light on my reading of documentation, because I’d much rather open up source code and poke around. But when something can work so reliably that I can treat it as completely opaque then it’s a blessing, because I can start to forget about it and think about bigger goals.
There was a time when people were concerned about Big Endian vs. Little Endian in computers. You had to think about this sort of little detail when programming. People formed actual opinions on which way was best. To think! Similarly XML has removed a large number of fairly pointless format decisions people might make. There is progress. Commodity hosting (reliable, consistent hosting, better than what we have now) feels like similar progress.
Unlike Doug I’m optimistic that App Engine is a move in the direction of a commodity cloud. The APIs seem to lack the stench of proprietary APIs. They are based on Google services, but they reasonably abstract and reasonably minimal. This does not seem like some kind of "play" (and the developers’ seem to be reassuring about their intentions). There’s a tendency to be cynical about any company’s work, that it has underlying intentions that are at odds with any competitor (present or future), that anything good is just a loss leader meant to hook you in so they can squeeze you later. Some companies deserve such cynicism. I don’t know that this company, or this team, deserves that.
Mind you, I don’t say this from a Best-Tool-For-The-Job perspective. I believe in the moral foundations of Free Software, not just the technical advantages of its development process. But I’m a Utilitarian, and it doesn’t make me uncomfortable that not everything is Free if I think it’s a step forward for overall freedom. I think App Engine has the potential to be a very powerful tool for enabling people to create and use web applications. If it was great, but still a dead end, then maybe that wouldn’t be good enough. But I don’t think this is a dead end.
Update: Indeed people are reimplementing the interface: see the appdrop.com announcement
Automatically generated list of related posts:
- App Engine and Pylons So I promised some more technical discussion of App Engine...
- App Engine and Open Source This is about Google App Engine which probably everyone has...
- What Does A WebOb App Look Like? Lately I’ve been writing code using WebOb and just a...
Every publicly traded company has only one purpose in life: Maximize shareholder value. There is no other purpose for their existence anymore.
I don’t fault them for this. I would be happy if I could run a publicly traded company (that does well). There’s nothing wrong with it, it’s just something they have to do.
But I don’t believe for a second that Google does this out of the pure goodness of their hearts. Every last one of their management team is, after all, accountable to their share holders.
Sometimes they will do something that benefits many people. Maybe App Engine is one such thing. Maybe they won’t increase their hosting prices. Maybe they won’t make you buy an expensive SLA when your site has finally become successful and you need decent service levels the most. Maybe they won’t look into your traffic stats and know more about your visitors and your site traffic than you yourself do, before they approach you with an offer.
Maybe they won’t do any of this, in which case it would be wonderful.
But even then they would do this at least as a sort of PR thing, something that helps them with mind-share, brand awareness, brand ubiquity, etc. They are a publicly traded company. They can’t do anything without shareholder value in mind.
Publicly traded companies are supposed to maximize shareholder value. Publicly traded companies don’t do anything. Individuals do things. Management is accountable to shareholders. Management is not subservient to shareholders. Employees are not subservient to management. Companies can provide incentives to get their employees to maximize shareholder value. If the company feels an employee isn’t doing enough, they can fire that employee. But they can’t magically turn employees into moles, embedded in other communities just so they can fuck them over later. Employees can volunteer for that role, but it’s not exactly a popular choice.
Some corporations actually seem to have a fuck-people-over culture. It seems to be cultivated in the executive class. It isn’t cultivated in the programmer class. In corporations where programmers are subservient to executives, I think bad things can happen. This usually requires the underlings to feel desperate or ignorant or disinterested.
Or, sometimes, the underlings buy into this maximize-shareholder-value bullshit. But it’s just bullshit, and it demeans the individual’s ability to make choices.
For some reason your post made me remember the SETI project, which probably was the oldest example of “cloud” on a global scale. When the Internet was young and everyone using it were a nerd (for better or worse), lots of people did devote computing time to SETI. For some reasons it never got beyond a mere curiosity. Perhaps the application itself was at fault – had we discovered alien life as a result of SETI, who knows what effect would it have.
We now have Amazon and Google offering cloud services. Microsoft is seemingly absent, it’s clear that don’t have any idea about what to do (besides buying Yahoo hoping that they will know what to do). But, I’m tempted to ask: is there anything that requires the cloud to run in a proprietary environment? Of course, there’s a huge investment, and management, but it should be possible to implement something like this to run in an open environment.
Maybe someone will come up with a open source implementation, or some company will launch a product that allows people to implement their own “mini-clouds”, in such a way that these mini-clouds can discover and connect to each other, forming a bigger cloud where computing resources can be allocated and shared between applications. I don’t know. I think that’s a matter of time before someone does something like that.
I believe that the idea is important not because this “open cloud” may kill Google or Amazon. I don’t believe it will get to this point. But the existence of an open implementation creates more competition, lower the entry cost, and raise awareness in the market. It also allows people to experiment and create new business model. It’s good for everyone, and it’s kind of a proof that the market is mature.
@Carlos: in some way I see the Apache Hadoop project http://hadoop.apache.org/ going in the direction of an open-source alternative. You may compare their HBase http://hadoop.apache.org/hbase/ to BigTable and HDFS http://hadoop.apache.org/core/docs/current/hdfs_design.html to GFS if you want.
Thanks for your insightful post on these issues. I myself hadn’t thought about the potential implications of being locked into a particular cloud.
The AppEngine solves a general problem of running Web applications. I often see Subversion or Trac mentioned in blogs and articles as applications that can now be run more conveniently using virtual machines or computing clouds. My company, ProjectLocker (http://www.projectlocker.com), and some of our competitors, have been running these specific applications for years in a straightforward ASP model. Clouds or virtual machines still leave you with installation and configuration headaches and costs, while ProjectLocker and companies like us take all the pain out of configuring, administering, and backing up your source code, bug, and Wiki data. We already support continuous integration and are working on more value-added tools as well. And since we’re based on open source tools with standard data formats, it’s easy to pick up and move to us or from us if you don’t like where you are.
You do acknowledge that reliable, consistent hosting is a step in the right direction. The good old ASP model, when implemented properly, is less sexy than clouds and virtual machines but still provides a lot of value.
Except with ProjectLocker, you are locked in. Unless you have a paid account, there’s no way to take your trac instance somewhere else, apparently.
In response to comment 3: Microsoft is not completely absent in this concept. They are just more focused on providing cloud [services](http://www.microsoft.com/sql/dataservices/default.mspx) to their corporate customers.
[The Semantic Web](http://www.w3.org/2001/sw/) is an idea whose time has come. AppEngine, EC2, et. al. will help to enable this framework of data sharing.
> What does App Engine run on? I don’t care!
Until something about how it runs randomly makes your code take longer than 3 seconds to run, at which point Google kills the request.
> What is BigTable? I am comfortable thinking of it only in its abstract sense, an API that works, and I don’t know how, nor do I need to know how.
Until the way you’re using it causes it to take more than a few seconds to run, at which point Google kills the request.
There’s always reasons to care about how your app is run. Since Google will kill requests that take too long, and so far have zero tools for profiling why it was killed, what part of it was taking too long, why did it take too long, etc. this could become rather important. I’d think a tool for gauging how long DataStore requests take would be important since many people will be unfamiliar with changes to how they should think about their data when its not in a RDBMS.
I’m sure they’ll provide some tools at some point, but the fact is, for awhile yet, yes, we will probably still need to care about those things.
I’m still waiting for a App Engine like thing that gives me the freedom to move to another provider when I either outgrow one (maybe Google’s way of scaling doesn’t meet my needs), or when the price of staying there gets too high (whether monetary, or privacy policy, etc.). Lock-in is no fun, no matter how awesome the service might be. At least Google’s lock-in is probably somewhat minimal as their API’s are rather abstract, and its just as trivial to run most of an App Engine app elsewhere.
In the end, would one be better off just running a VM with Apache/lighttpd/etc, a real Django and whatever else you need on Amazon’s EC2? There you get cloud computing with all its advantages, but you are determining your own software/language/server settings. And if you want to move to another provider, or start running your stuff on its own dedicated box… well, all you need is to find a RedHat host somewhere (I think Amazon EC2 uses RedHat for its server images, right?) and off you go. Moving to a different provider therefore should be quite straight forward in that case. Certainly more so than in the case where you develop your app specifically towards a Google API.
@Raphael: thanks for the pointers. I assume that these projects are focused on building “private clouds”. That’s great, but I wonder if it will be possible to do it in a much larger scale, a “peer to peer” global cloud. But that’s just a dream, I don’t know if it’s practical or feasible – only that it seems interesting.