Ian Bicking: the old part of his blog

What Really Makes Rails Work

Often, when pointing out the problem with Python web programming, people have been responding that if we just had something as good as Ruby on Rails we'd be okay. I've tried to respond to that both indirectly and directly, but I think I need to be more direct, and I've also been changing my mind on the topic over time.

My standard response has been that people have been creating and recreating Rails-like systems in Python for some time -- like for years and years. It hasn't fixed anything yet. But they weren't as good as Rails people say... well, maybe not, but if it was such a great idea then three years ago when people were trying this, don't you think we would have gotten there by now? Rails is not old, it doesn't have a huge code investment, and it doesn't have any novel ideas. I'm not trying to talk it down, that's just the reality of what Rails is -- it's not Seaside or Wee (and if it was it wouldn't be so popular anyway).

So, I was reading Dave Hansson's response to Why Web Programming Matters Most, and I came upon this comment that got me thinking:

How can you write ten paragraphs about how Python has no universal web framework and not even mention Zope? I haven't been embedded at all within the Python community, but to me this seems like one of its greatest achievements.

At PyCon I talked with a number of Zope people. And it's funny... Python programmers really don't like Zope. Python programmers that have been employed by Zope Corp and contributed to its design don't even like Zope that much. Jim Fulton (Zope's architect, and now Zope 3's architect) was quite aware of this, and I'm sure a bit disappointed. But I don't really mean to talk Zope down -- rather Zope is an important lesson and example. Because despite all this, Zope is quite popular. People do lots of cool things with it. Having worked with Zope 2 a fair amount in the last year and a half, I still don't like it, and I'm quite sure that won't change -- but clearly that's not true for everyone.

Zope is also unique among web frameworks, and maybe even among Python projects, in that it's something where people choose Zope first, then Python comes along. With most other projects a developer chooses Python then finds a project in Python. As a result, Zope users are selected not for an aspect of the implementation -- Python -- but because they are specifically attracted to Zope and all its metaphors and design decisions.

This is what Ruby on Rails has. People are selecting Rails, and it just happens to be written in Ruby. This seems weird, because it's a developer framework and intimately tied to the language, but it's clearly what's happening. So as a result, the Rails community isn't going to fork as developers get frustrated with the specific metaphors it chooses. They are choosing the framework, not the implementation. In contrast Python web programmers have been choosing implementations, then seeing if they can accept the framework (and forking when they can't).

And actually there are several competing web frameworks in the Ruby world. Rails didn't first take over the Ruby world, then expand to the wider world -- instead it immediately went to the wider world, and has been consolidating the Ruby world more as an afterthought.

This is why Rails-for-Python won't work, or at least what people have been focusing on with Rails isn't correct. It's not about the implementation or the quality of the framework, it's about the pattern of adoption. It's about putting the framework up front, and the implementation in the back -- even if this wasn't the Rails developers intentions (though maybe they are clever and this was their intention).

So, what do we need to do? Well... maybe we should stop trying to convince each other, and start looking more to the wider world. We don't need to compete with the exact niche that Rails speaks to -- though honestly it has a lot of overlap with the niche I'm interested in -- but instead we need to create something for the wider world.

On a positive note: Python has incredible infrastructure and a lot of good code (not just good design patterns) to offer the world. I'm really excited about FormEncode, and I think it will soon become the form processing library for Python. I'm excited about generic functions, and think they could radically change the way we plug together models and views. I'm totally unexcited by things like py2app, py2exe, and Eggs, which is to say I'm really excited that someone else did all the annoying deployment work that I don't want to do ;) I'm really pleased that SQLObject seems to be taking on a life of its own, with both word-of-mouth evangelism and peer support. And that's just a little of what's going on... Yarn, Schevo, LivePage... there's a ton of good stuff, stuff that isn't just cool from a Python perspective, but can serve to actually pull people into Python regardless of the language. A language is only as good as the things people create using it... given that standard I don't feel any need to apologize for Python or come up with any excuses. But maybe we should start selling it to the rest of the world a little harder.

Created 29 Mar '05

Comments:

And after all of that, you can't even mention Zope 3?

While everyone seems to have gone off and tried to make a "more pythonic framework" than Zope, Zope continues to enjoy high use and popularity. There are a lot of lessons learned from it - good and bad. And Zope 3 is an effort to learn from those lessons. I wish that more people would get involved in contributing to and working with the Zope 3 community than making yet-another-clone of the latest hot-web-technology.

(Although, I do give the Subway group serious credit for using existing solutions instead of rolling out new ones).

Twisted 2.0 uses Zope 3's interfaces system, and I've heard from a couple of sources that Zope X3 3.2 will be able to work with Twisted (and, hopefully, cool Nevow technologies like stan and LivePage can be used with Zope 3 with little effort).

Zope 3 offers better than strict model-view-controller separation. Without hard restrictions on "this is the controller, this is the view" (most of the time, a controller and view are so intertwined, why not just connect them as a single View object and not really worry about the Controller?). The model is often just a handful of persistent attributes, but there are more objects that make up an intelligent web application than can be deliniated by such restrictions (bug dependency trackers, etc). I don't think dumping things into directories/packages named 'model/view/controller' is the most "Pythonic" thing in the world. The python package and module model is better than that.

I've been using Python and Zope or some predeccesor of Zope professionally for nine years now. It's been serving Python objects on the web since PHP was just a small collection of Perl scripts trying to be a better Server-Side-Include. Why wouldn't you want to work with the community that's trying to take that success and combining it with the hard lessons learned over the past 8-9 years into the next generation "highly pythonic" platform?

# Jeff Shell

Sorry to forget Zope 3... I think it was in my original writing but I edited it out that paragraph. Zope 3 is both better than Zope 2 for Python programmers, and maybe worse in terms of bringing in non-Python programmers. I don't really see it as accessible to an off-the-street programmer who isn't committed to Python already. But there's also a good possibility that something built on Zope 3 will provide that experience -- something that looks like Plone, perhaps. Or another application that is itself a framework, but a more directed framework than the open space of Zope 3 as a whole.

It's getting off the original topic, but I do have some problems with Zope 3 (I haven't used it a whole lot, so I might be looking at it wrong). But it seems to put modeling up front, with the web UI coming along for the ride. But I think modeling is really hard, but making a web UI is pretty easy. And I think it's best to do the easy stuff first. So I'd prefer a system where development of the application UI came first (in a crude procedural manner if necessary), and the model grew out of that. I don't think this describes Rails, or Subway, and certainly not Zope 3. I guess it describes PHP, except PHP is good for that first step and bad for all following steps. Rails I suspect is appealing to a more mature developer that has gone through the PHP (or PHP-like) process. Zope 3 is appealing to a developer with a level of maturity that, I think, is very uncommon.

That said, I'd certainly like to take ideas from Zope 3 as well in my future work. Once I figure out what they are ;) I've become wary of Adaptation, though I've only tried to use it once (but it wasn't very successful). I like interfaces, especially as documentation. I should learn more to figure out what else there is. Hopefully Zope 3 people will also start distributing more stand-alone packages, which I think will increase its visibility, and let people become incrementally more comfortable with its concepts before they jump in entirely.

# Ian Bicking

You've misunderstood Rails if you think it doesn't support building the UI first and extracting a model from that. That's exactly the way DHH likes to approach development. You can write a Rails app that doesn't use a database (e.g. Instiki, a great Wiki based on a pre-release Rails, that uses a RAM/disk backend).

That means you can cruft up a UI, store values in memory or whatever, iterate until you like your UI, and extract a proper model.

If that's what you want, I suggest you give Rails a try, Ian :) (Apologies if you already have.)

# Gavin Sinclair

Maybe I misread the point of Ian's post, but the biggest reason I don't lean more towards Zope is that the model isn't for me. While if I were writing my own ideal web system I would probably do so in Python (because I like the language), when I'm looking for something pre-existing the language is much less important. As Ian said, people choose Rails because they like how the model helps them work and Ruby comes from that choice, and people choose Zope for similar reasons and Python comes as part of that choice. If the python community wants to have a dynamite web development platform that draws in excited new people, then make one or refine an existing one, but perhaps the key to drawing developers is that it is a "dynamite web development platform", not whether it is in Python or not.

That said, I do need to look at Zope3 and reconsider whether it provides a model I'd be comfortable working in, and whether it would make my life as a developer that much better. It's been a few years now, I should probably re-codify the things I'm looking for and figure out what system will get me closest and let me easily engineer the additional things I want.

# Luke Opperman

Hi Ian. I just checked out FormEncode. The API looks really nice! Great work. I'll keep it in mind the next time i'm doing a web programming project.

Would you consider renaming ForEach, though? It doesn't make sense to me -- i think List or ListOf would be a much clearer name.

# Ping

This is what Ruby on Rails has. People are selecting Rails, and it just happens to be written in Ruby. This seems weird, because it's a developer framework and intimately tied to the language, but it's clearly what's happening.

It is weird, especially with all the comments along the lines of "I use Rails even though I don't really like Ruby". I can't understand not liking Ruby, but I do enjoy reading these comments.

But it's easily explained: Rails lets you create applications using absurdly little code, so the barrier to entry is low. Low enough that it's worth trying, even if you're unsure about the language it forces you to use.

And actually there are several competing web frameworks in the Ruby world. Rails didn't first take over the Ruby world, then expand to the wider world -- instead it immediately went to the wider world, and has been consolidating the Ruby world more as an afterthought.

A few people have doubted that it will consolidate the Ruby world, because some/many Rails newbies seem to totally understand that Ruby is a very powerful general-purpose 12+ year old programming language, not merely the Rails scripting language. That may be a PHP background, where the language is the web development environment (or something).

I disagree with that opinion, because you can't do much in Rails without programming some Ruby, and if your app does anything remotely interesting, you're going to have to learn how to manipulate data, etc. So education will have to take place somewhere. However, if you weren't attracted to Ruby for the langauge itself, it's not certain you'll strengthen its community. It's already clear that the Ruby and Rails communities are distinct, though very much overlapping. That's good, because the two communities serve different purposes.

# Gavin Sinclair

QMTest is another example of a system that brings people to python (it's what brought me - starting from "there's got to be something that kills DejaGnu out there, oh look, QMTest might be it" and learning enough python to hack tests ("wow, this sucks so much less than TCL! it actually has quoting!") and then coming upon a reason to use Python instead of perl, since it was at hand, and bootstrapping from there... but again, the batteries mattered as an enticement more than the language did. It took something like a year for me to really decide the language was a winner.)

As for ZOPE - it has traditionally been positioned as solving a problem I don't want to have :-) This means I've been able to recommend it to IT types and nod positively about it when it comes up in content-oriented rather than engineering-oriented contexts, but it's very much got a "them not us" feel to it. I think that has a lot to do with why it isn't talked about as a "unversal" framework - it doesn't scale down to the individual python hacker trying something clever, if they're not already swimming in it.

# Mark Eichin

Or maybe what we should do is provide one clear answer, rather than half a dozen that are three-quarters done. I spend half or more of my time trying to persuade people who _aren't? already Python users (primarily educators and scientists) to adopt Python; please believe me when I say that the lack of credible, definitive entry-level web programming framework is a real problem. Talking ourselves into believing that it isn't will just do more damage.
# Greg Wilson

Agreed. The community doesn't necessarily want one framework to exist to the exclusion of all others - I think a lot of people got the wrong end of the stick and thought that people were advocating one true solution backed by GvR himself, but that's not what most of the proposals are about - but newcomers really do need clear answers about how to write Web applications that don't suddenly run out when you hit a tricky part of the development process.

Some work has already been done on stuff like deployment, although I'd argue that too much time has been spent in philosophy over less pertinent details than the big picture, but it should be possible both for a newcomer to fire up a simple Python Web server running their application and for an expert to drop that application into a more scalable or sophisticated environment. After that, deployment should be an issue that we don't have to think about so much.

The biggest challenge is offering "good enough" tools that fit nicely into the scheme of things. I'd argue that stuff like templates and data access libraries don't need to be glued on top of frameworks in exciting ways - just having such stuff as easy-to-call APIs behaving sensibly is far better than some of the bizarre autoconfiguring magic that often goes on in the average overengineered Web solution. If you make it clear that "here is the part where you get the user input in format XYZ" and "here is where you send it out as a Web page", people are going to be more likely to (a) get it, and (b) test that stuff separately when they need to.

Anyway, I suppose that more discussion of this will happen on the Web-SIG mailing list - something that all interested parties should be following, I think.

# Paul Boddie

Ian, I was looking for FormEncode all over on your site, but I couldn't find it and I had to dive into your blog archives to find it. You could use a site map or something.

-Ken

# Ken Kinder

Wow. I really don't agree with the core point here, that we don't need a Rails competitor.

I recently re-implemented in Rails a Web app I wrote originally with Webware, Cheetah, and SQLObject, and it took me a fourth of the time. Unfair comparison? Okay, yes, I'm aware of the problem domain, which may have drastically reduced the amount of time necessary for the re-implementation, I'll grant that. But there was more to it.

Rails was easy to develop with. ActiveRecord did basic things automatically for me that I didn't have to hand-code or futz with at all. It was intuitive. It was quick. It was very, very clean.

That said, I don't care for Ruby especially. Yeah, I'll say it. In a choir of people singing hymns to Ruby, I'm hitting a sour note, and for some admittedly arbitrary reasons: All those frickin' "end" statements; A huge Unix bias; A shrill community containing "religious" sorts who, in some cases, go so far as to impugn your worth as a developer based on the kind of hardware you're using. God. I can just see all those Ruby Doobies out there goose-stepping a love march in honor of their big, pink language.

But I do believe that saying Python doesn't need a Rails competitor feels to me like smart people over-thinking the problem and believing that underneath it all, naaah, we're in a different domain and we don't need no stinking Rails, when, actually, we kinda do, as much as anyone. You're correct, Ian, in saying that Rails is no Seaside, I agree with that, but okay, so what? Rails does make certain things really, almost trivially easy in a way that Webware/Cheetah/SQLObject or CherryPy/Cheetah/SQLObject make only kind've easy, and that makes me want to consign the argument about Web frameworks to the bloggers and just use Ruby and Rails until the Pythonistas come up with something that's definitively equivalent or better. After all, I've got a living to make and I can't spend all day (this entry notwithstanding) arguing about how typing up class definitions with SQLObject is, arguably, only a little less convenient then editing an ActiveRecord YAML configuration file.

So why is it again that Python can't have a solid, non-hacked, definitive Rails competitor, that's more than just existing components and some Python tape? Because I think it may be ego, a little. I think there are certain very smart people out there that will take some umbrage at the thought of having Dave Hansson and his pets point a finger and say, "See! See! God does love us best! We're so smart! Yay!" And I also think that some of the most talented programmers in the Python world -- the kinds who would be writing such a framework -- find some things so second nature and quick for themselves right now that they forget that those less familiar with the language or the toolsets will find comfort in the copious, organized documentation or the very active IRC community, or the rapid responses to questions which the Rails community offers, shrill evangelists aside.

Please don't perceive this as criticism, because I use SQLObject every freakin' day and, for that matter, my first Webware version of the app I mention above is faster than the Rails version, even though I had to code a lot of bookkeeping and error-checking by hand. And I love Python. Python syntax pistol-whips Ruby's, cruelly. But I think the Python community could stand to take a page out of Ruby's book where marketing, inviting newbies, and providing great docs and community are concerned, in addition to a complete, shiny framework that did, for a change, try to do everything and well.

# Greg McClure

Wow. I really don't agree with the core point here, that we don't need a Rails competitor.

# used rolex daytona

A small comment if I may...

It seems that the fundamental difference between Zope and all the other web frameworks I know of is simply that all the others are not frameworks.

A framework expects you to embed and extend inside the framework, its metaphors and its design decisions. A library expects you to access it externally, and if you do not like its design decisions, it does not matter that much because you are insulated from the library by a fairly well defined interface.

For a Windows analogy, most Pythonistas seem to prefer libraries, probably for the same reason Word is seen as a COM "server" more often than a COM "client" - its a lot neater to control Word from python and shove any data it needs in than it is to use Word as control point and create Python objects to serve it.

And so we tend to avoid frameworks in a similar way most of us avoid developing for Word.

cheers

# paul

A small comment if I may...

It seems that the fundamental difference between Zope and all the other web frameworks I know of is simply that all the others are not frameworks.

A framework expects you to embed and extend inside the framework, its metaphors and its design decisions. A library expects you to access it externally, and if you do not like its design decisions, it does not matter that much because you are insulated from the library by a fairly well defined interface.

For a Windows analogy, most Pythonistas seem to prefer libraries, probably for the same reason Word is seen as a COM "server" more often than a COM "client" - its a lot neater to control Word from python and shove any data it needs in than it is to use Word as control point and create Python objects to serve it.

And so we tend to avoid frameworks in a similar way most of us avoid developing for Word.

cheers

# paul

I think your comment illustrates quite nicely the suggestion that Zope was the original Web success story for Python, but it evolved into a parallel community and left a vacant spot for another compelling solution. And if you're making the point that I think you're making, you might be onto something about how people shy away from sophisticated frameworks with lots of opportunity to embed and extend inside the frameworks themselves.

Although badly designed libraries can be limiting, libraries are almost indisputably the most successful units of reuse around. Moreover, conventional libraries (where you call in and get results straight back) are very easily understood by most people; in contrast, many developers become uneasy when confronted with things like callbacks, even though such things are fairly straightforward in Python.

Like GUI programming, Web programming already employs an event-style model which people find disconcerting. Adding more magic on top, and thereby not giving straight answers to people about how their framework objects come into being and how they get called, just makes it more of a burden to use a "conceptually heavy" framework. If people are serious about offering an easy way into Web development in Python, they will need to throw out as many of the caveats and the magical framework incantations as possible.

# Paul Boddie

If people are serious about offering an easy way into Web development in Python, they will need to throw out as many of the caveats and the magical framework incantations as possible.

From my cursory look over Rails I see a lot of Magic and from the docs I see the phrase "Magic is not inherently a bad word". Egad! These people believe in Magic, savages!

But perhaps your point is to remove framework-specific incantations and boilerplate and if so, I agree with that.

Personally I think a Python version of Rails isn't possible because the Rails developers are willing to do something alien to Pythonista's, that is, break our "Explicit is better than Implicit" creed and create a domain-specific language for Web Applications.

Admittedly, the history of this kind of magick in Python isn't encouraging. Zope and it's Aquisition turned off a whole section of the Python community.

So I think the problem is philosophical. It hardly matters that Python isn't quite as good as Ruby for creating these domain-specific languages (and it isn't, we must admit it) because doing so runs contrary to the principles we as pythonistas hold dear.

Making something with as much utility as Rails would mean standing up and saying (gulp!) :

"Implicit is better than Explicit (in specific domains)"

Can we do it?

# Ian Sparks

"But perhaps your point is to remove framework-specific incantations and boilerplate and if so, I agree with that."

That was exactly my point. I've looked at a lot of frameworks over the years, but now if I see anything which imposes arbitrary behaviour, then I just hit the back button, and I think people get put off by seemingly arbitrary restrictions encoded in special ways. I'm trying not to be too controversial here, but if you look at various object publishing mechanisms, you see two things: that the developers assume that a rigid hierarchy is how everyone wants to design their applications; the various ways of telling the framework about exposed objects. Clearly, due to Python's introspection capabilities and the potential for security breaches (ask anyone who has written an object publisher about that) you need the special notation and it needn't be too intrusive (although Quixote's notation seems a bit contrived), but I can't help feeling that many people are thinking, "Hold on! I'm not sure if this even suits me."

It's a similar story with templating. Whilst some of the PSP solutions can be quite productive, if you don't believe it suits your application then the whole thing is going to be a turn off. It's interesting that the most actively developed PSP solution of them all, CherryPy, backed away from special template languages in release 2.0.

"Personally I think a Python version of Rails isn't possible because the Rails developers are willing to do something alien to Pythonista's, that is, break our "Explicit is better than Implicit" creed and create a domain-specific language for Web Applications."

Yes, and Ruby probably supports that quite well. But as you note...

"Admittedly, the history of this kind of magick in Python isn't encouraging. Zope and it's Aquisition turned off a whole section of the Python community."

...it may be the case that Rails eventually is to Ruby what Zope is to Python: something that lives alongside the creation which spawned it.

# Paul Boddie

Why arguing about which of the two of the greatest products should come first?

To me, zope2.x seems harder to modeling/manage enterprise-sized application while 3 seems making routine-task a lot harder for a new-comer than how it can be done with 2.x's ZMI. --Just like in Linux world, some prefer GUI while others value CLI.

Can't they co-exist and collaborate?

Wouldn't it be nice if the future zope3 combines the simplicity of configuring objects with zope2(through its ZMI) and having better capability of handling M/C/V with its current underlying framework?

P.S. I am by no means an expert in either zope or python, so hope you guys don't mind, just trying to make a point here :)

# Michael Lee

To me, zope2.x seems harder to modeling/manage enterprise-sized application while 3 seems making routine-task a lot harder for a new-comer than how it can be done with 2.x's ZMI. --Just like in Linux world, some prefer GUI while others value CLI.


That was exactly my point. I've looked at a lot of frameworks over the years, but now if I see anything which imposes arbitrary behaviour, then I just hit the back button, and I think people get put off by seemingly arbitrary restrictions encoded in special ways. I'm trying not to be too controversial here, but if you look at various object publishing mechanisms, you see two things: that the developers assume that a rigid hierarchy is how everyone wants to design their applications; the various ways of telling the framework about exposed objects. Clearly, due to Python's introspection capabilities and the potential for security breaches (ask anyone who has written an object publisher about that) you need the special notation and it needn't be too intrusive (although Quixote's notation seems a bit contrived), but I can't help feeling that many people are thinking, "Hold on! I'm not sure if this even suits me."


Personally I think a Python version of Rails isn't possible because the Rails developers are willing to do something alien to Pythonista's, that is, break our "Explicit is better than Implicit" creed and create a domain-specific language for Web Applications.

# rhodes furniture stores

A framework expects you to embed and extend inside the framework, its metaphors and its design decisions. A library expects you to access it externally, and if you do not like its design decisions, it does not matter that much because you are insulated from the library by a fairly well defined interface.

# louis vuitton replica luggage

Wow. I could not agree less with this whole article.

After being a programmer through many years and languages, I chose Ruby because it is just simply the best language out there today in many ways. And Rails, because it is without any doubt the most capable of all the web frameworks for Ruby.

I did not -- even a little -- choose Ruby because of Rails. I chose Rails because of Ruby. Together they are simply awesome... and I say that despite the over-use of that word in recent years.

My strong opinion is that this article represents "rationalization" of why people choose RoR over Python and its frameworks. But rationalization only works for a little while. Eventually, the truth comes out. RoR is simply superior. That is why people like it.

# Lonny Eachus