When I was writing the summary of differences between WebOb and other request objects, to remind myself of web frameworks I might have forgotten I went to the WebFrameworks page on the Python wiki.
Looking through that page I’m reminded how many framework options there have been. And I was further reminded of how few relevant options there are now. From all this, there have emerged just a few options: Django, Pylons, TurboGears, Zope. No offense to anyone left out of that list — I know there’s some other actively developed frameworks out there. But frankly they aren’t serious choices; they might be fine internal tools, or interesting experiments, but they are clearly on a different tier (and they all have questionable futures).
And now that TurboGears 2 will be based on Pylons the list looks smaller still.
For a long, long time (longer than most of those frameworks have existed) people have complained about the proliferation of web frameworks in Python. Those of us involved in developing web frameworks in Python haven’t been able to respond all that well. Complaining doesn’t magically lead to solutions, and you can’t just will there to be a single Python web framework. You can work towards that, but that’s what we’ve been doing… mostly people don’t seem to notice. It’s just not an easy thing to work towards; the problem space for a web framework isn’t well defined, its end goal is far more vague than most people immediately realize, and it involves consensus, which makes everything much harder. We said the market would decide, which is kind of a cop out (the market decides through the decisions of developers) but that’s the best answer we had.
But after all this time, it seems clear that we are getting much closer to that goal. If you squint really hard, you can almost imagine we are there. The total list of frameworks only gets longer over time — that’s how open source works — but the list of choices has become quite compact.
How we get to the next level is a little less clear. We’ve gotten this way largely through attrition, but that’s not going to get us any further. I’ll at least assure people that we are discussing this stuff — it’s slow going, but everyone is interested. And if anyone actually wants to do some leg work to move this forward, a lot of the work is actually technical, not political, so don’t be afraid to jump in.
Automatically generated list of related posts:
- WebOb Do-It-Yourself Framework My old do-it-yourself framework tutorial was getting a bit long...
- A Python Web Application Package and Format (we should make one) At PyCon there was an open space about deployment, and...
- Opening Python Classes So, I was reading through comments to despam my old...
- Python Application Package I’ve been thinking some more about deployment of Python web...
- 2 Python Environment Experiments two experiments in the Python environment. The first is virtualenv,...
This is exactly why I’m interested in making Django’s Request/Response compatible with the one used by Pylons. If we can come up with a documented standard (ala WSGI) we can collapse the stack even further.
Although you apologized in advance, I still think Quixote and web.py are fine choices, and quite different from others in their approaches.
LWN uses Quixote. Reddit uses web.py.
The next step will hopefully be that these frameworks stabilize. It’s frustrating to develop against an ever-changing framework. At least it was for me when I developed against TurboGears 0.x.
I’m not so sure that the concept of mega-frameworks like TurboGears or Pylons helps here. Perhaps a “certified” set of libraries (like Pylons version x, Routes version y, Paste version z, etc.) could build more confidence here.
Yes, thanks for going in and saying, “This page has never been very actively maintained.” I’m sure that the page history contradicts that assertion satisfactorily. Pointing people to “blogs, reoccurring threads on comp.lang.python” is great if they’re already well aware of all the different blogs out there, which ones from Planet Python are worth reading, how to most effectively search the newsgroups for advice, and so on. However, I don’t think this applies for a large number of people in the audience in question, and the whole point of having resources such as those on the Wiki is to provide a well-known reference for everyone, not to wave people off to a lower density medium where opinions are formed by people who have the most to say.
Sure, everyone wants their favourite framework listed – I know that from experience with maintaining these guides – and yet people worry about newcomers choosing “minority software” (or insert another euphemism for “not the software I endorse” like saying that they are not “serious choices”, perhaps). However, you’ll notice that there aren’t hundreds of frameworks on that page, which would be the case if the now-tedious jokes about Web frameworks actually had a substantial basis in fact, and while it’s possible that some listed frameworks should be dropped due to lack of heavy project activity, some either do what they’re supposed to and don’t need continuous redevelopment (which doesn’t then prevent them from being valid choices), or they re-appear at a later point in time (eg. Aquarius) and presumably remain valid choices.
Meanwhile, I dispute that “a lot of the work is actually technical, not political”. It can be argued that the reason why no-one has had any stamina for standardising Python’s Web programming scene is because the work to do so isn’t particularly glamourous (eg. look at what the Java people have done, take notes), and because people still get a kick out of differentiation at the cost of standardisation, four years or so after this particular discussion began.
Ian,
I agree that the Python web application framework world is shrinking, but there is also a rather large elephant sitting in the room. Some Python programmers who would much rather program in Python are using ROR and I think this trend is increasing not decreasing. I have no answer or solution. Just pointing at the elephant….
Noah
I assume you haven’t heard of web.py. See, reddit is a large-traffic site and it is built on web.py and infogami on top of it. openlibrary.org is a web.py+infogami combinaiton as well. Those two and many more are much more than “fine internal tools, or interesting experiments”.
@Seo and Tzury: You do realize that both those sites were written by the same person – and that Reddit has since moved away from web.py, right? The new Reddit beta is just Python, no web.py. One developer, no matter how popular he is, using his own framework, does not make a trend.
Reddit’s new beta doesn’t use web.py. AFAIK its not publicly released what framework, if any, replaced it, however its still Python (and probably uses Mako templates).
Paul: The WebFrameworks page lists frameworks like Crusader, Cymbeline, PyLucid (which is now written in Django), JOTWeb2, maki, PyWebLib, and Spark (and to clarify, many of these are abandoned or hibernating). It lists a bunch of really old books. The popularity section contains only a very rough estimate of Django and TurboGears. It’s a horrible landing place for people looking to make a decision about a Python web framework. I was just trying to be helpful to readers by warning them about that.
Also, it’s my belief that any framework (or any open source project) without a champion is dead. A lot of those frameworks don’t have champions anymore. They’ll still have users, maybe they still will for a long time, but without a champion the framework is not a good choice.
The technical work I refer to isn’t just work, it’s the right work. I believe there’s some projects which, right now, are the right work. Identifying and improving the best server deployment options, for example.
As for differentiation vs. standardization, you can’t make that choice when there isn’t something — even just a strong convention — to standardize against. And as for Java, frankly I think we’re beyond the point at which there’s much to learn there. Maybe there’s still some nuggets there, but when it comes to Python ideas taken from Java I am usually left very disappointed by the result.
Tzury, Seo: I was going to note the same thing John just did about web.py. Regarding Quixote, it’s a fine framework but its forward momentum is just about nil, and I don’t think that bothers the people using it, and it’s perfectly fine that they feel that way but that doesn’t make Quixote a good choice.
We get to the next level by pushing WSGI even more :)
I don’t think we’ll ever reach the one web framework that rules them all. But it’s freaking cool that you can build your own web-framework in a number of hours! That’s the real power of Python web-development.
I fell in love with [CherryPy](http://www.cherrypy.org) despite my strong feelings towards Pylons (which I struggle to understand due to my own limitations).
Following your post I questioned the CP community. I think the [answers](http://groups.google.com/group/cherrypy-users/browse_thread/thread/49b1610e524839b0) are relevant to this post. For people interested in doing web stuff in Python CherryPy is a fun, and I believe viable, option!
I’m agree with amix
Why focus on framework ? They will be all out of date or will ask users to change theirs codes one day… It’s why i use my own framework since years like many others. It’s was not difficult to do and now it’s become realy easy !
What we need is killer web libraries around wsgi, and i’m confident that WebOb will be one of them. Thanks to your work ian.
While not a full blown Framework – more an extensible HTTP Server, [CherryPy](http://www.cherrypy.org/ “CherryPy”) is definatly worth a consideration. Especially if you don’t need/use integrated templating mechanisms or ORMs like us. We are using XSLT (pretty understimated in the Python world) and Firebird (sharing SQL statement logic with Win32 Delphi clients). CherryPy makes it as easy as possible to plugin your stuff, leading you fast to your own specialized framework. It has an extreme friendly learning curve but all the enhanced capabilities you will need later. And finally it’s build for speed as well. Therefore, we love it ;).
Pylons could be a contender but Zope, Django, and Turbogears are to bloated for our taste.
I also use CherryPy’s WSGI server in all of my projects, which from my benchmarking is still the fastest WSGI server (and it’s HTTP 1.1 compatible).
It would be cool if the community developed on one real good WSGI server. Currently there are quite some implementations that are lacking.
Or why not release a fully implemented WSGI server with Python? This would spread WSGI love even more :)
amix: Python 2.5 has now wsgiref. I’m geek enough to develop my own framework, and wsgi is doing my day.
Nuno: It’s only a reference implementation.
Yes it is, but works. What is the advantages of using the CherryPy’s WSGI Server, or other?
I do agree with amix. Why can’t we have one real good WSGI server and also a library ? CherryPY server, wsgi library such as amix’s werkzeug, ian’s webob, luke arno’s stuff (just to name a few) can be a good start. Let’s shape the future of Python web development around the library and not the framework.
Logically, the blog entry is argumentum ad populum. Politically, if it is necessary to resist inroads by rival programming languages, it is expedient to rally around major frameworks. However, I submit that any such major framework needs to be critically evaluated and re-engineered as necessary to by pythonic, then locked down tight.
Having reviewed the field of frameworks recently to fulfill my requirements, I did not like any of the major frameworks. I felt they were too cludgy for me. I did not feel like I was asked to write Python, but some other language. Granted, this is more obviously apparent in the GUI world than WUI, but if it does not feel like Python I have problems using it (when writing Python). Looking at some tutorials, I observed significant shift in the implementation…
As one other responder mentioned, a fear of adopting one framework is the likelihood of its having radical changes. That’s why I stopped using Drupal many years ago. I wrote a set of modules for one minor revision that were totally broken for the next. Having seen similar behavior in at least one framework, I am wary of adopting any.
Yes, it is possible to fork a project, but popular projects do not lend themselves to a fork without enough momentum within its community (e.g. X11 v. XORG).
While suggesting we all jump on a bandwagon of conformity, it is imperative that there also be a call for stability and pythonic engineering of the major frameworks.
I specifically avoided mentioning any framework because my purpose is not to defame any framework. My purpose is stability and language conformity. If you can promise both, then I might be willing to drink the Kool-Aid.
“And as for Java, frankly I think we’re beyond the point at which there’s much to learn there.”
I think a lot of frameworks could start out by learning how the Java technologies manage to work with Unicode, given the widespread problems many Python-based frameworks have had with Unicode until very recently. I note that someone in the Django community eventually stepped up and dealt with the issue, but it probably still remains in other frameworks, given the occurrence of discussion threads on such matters in various places.
It’s not all about momentum, you know – sometimes you want pieces of the stack to be stable, even mature. The problem has been that everyone has wanted to show how much they’re innovating (which has indeed happened), but that can lead to egregious and ubiquitous breakage as the developers refine their work. You wouldn’t want your C library API to mutate every few weeks or months. Likewise, it’s a relief to be able to use technologies which improve but don’t break your code needlessly as they evolve, get updated or get fixed. (I don’t update WebStack very often at all, or at least not the central abstractions, but that doesn’t mean it doesn’t work any more, or that it isn’t useful for other components or applications.)
As for the Wiki page…
“It’s a horrible landing place for people looking to make a decision about a Python web framework. I was just trying to be helpful to readers by warning them about that.”
The dates next to the frameworks give the discerning reader a good indication of framework vitality, and I don’t even think that this was even my idea. Nevertheless, the page, which was previously a mess of links and opportunistic promotion, got an overhaul a while back after I got fed up with how incoherent it had become. Whether it plays an important role in informing people or not (or whether it represents the “new consensus” around Python Web frameworks as you see it), giving it the brush off is rather unfair to the people who update it (usually not me, unless I’m despamming it or making superficial edits) as well as the people who find it useful.
“given the widespread problems many Python-based frameworks have had with Unicode…[they] probably still [remain] in other frameworks”
Not really. Pylons and Turbogears recommend Genshi or Mako templates which are both built for Unicode from the ground up. SQLAlchemy, used by both, can represent tables in Chinese (not just the data; the schema).
There are issues remaining with unicode, but at this point its primarily a Python problem (and in some cases, a database problem, like MySQL which can present some encoding challenges); strings are still represented as bytestrings by default, and converting into unicode objects can be tricky. That’s pretty much where the confusion comes from.
“It’s not all about momentum, you know – sometimes you want pieces of the stack to be stable, even mature. “
I dont find this to be true in the Java world either. A year ago, we were using Hibernate and Struts. Now we’re using JPA and Struts2 (which is an entirely different product); Hibernate’s API has been totally replaced, and Struts has literally been chucked wholesale. While we might want to look at Java for insight on how to handle things, my observation is that they have been looking at us (us being, Ruby and Python) a lot as of late, working at making their APIs more friendly, adding decorator-like “annotations” and iterator syntaxes, copying our ideas for web controllers, and cutting down on the XML bloat. I think the “stability”/”maturity” perception on the java side comes down to the sheer number of developers working on things over there (and being paid full time to develop them by big corporations). Whereas Python projects are often volunteer/part-time/not-really-funded ventures with much smaller userbases.
Zope 3 has been unicode safe from the start. Since 2002 or thereabouts. I was somewhat bemused at seeing Django announce unicode support proudly a few months ago. They have backwards compatibility hacks and all, right? We’ve have had this, without any hacks, for half a decade and probably nobody even knows it. Shows how badly we market Zope 3 (but this is changing, with Grok).
“Not really. Pylons and Turbogears recommend Genshi or Mako templates which are both built for Unicode from the ground up.”
Nice to see that there’s a consensus. It wasn’t always this way, I believe.
“There are issues remaining with unicode, but at this point its primarily a Python problem (and in some cases, a database problem, like MySQL which can present some encoding challenges); strings are still represented as bytestrings by default, and converting into unicode objects can be tricky.”
Ignoring the database issues, which can be awkward, the “tricky” part of converting byte sequences (plain strings) into characters (Unicode) is hardly a “Python problem” unless that’s a label for the work that framework developers don’t want to do: it’s a fundamental part of interpreting the data. Python 3000 doesn’t magically solve this for you because the issues don’t go away with Python 2.x. Instead, they reside in the nature of the data the framework is busy pumping backward and forward on your behalf – precisely at the point where the framework should be lending a hand.
I didn’t bother exposing data as Unicode in WebStack for some time, mostly because I used the excuse that it was but a thin wrapper over other frameworks, but having full Unicode support for Java Servlets and “pot luck” for all the CPython stuff was highly embarrassing – it would be like an XML toolkit giving you plain strings and making you do encoding conversions manually.
If I wanted to point the finger at a guilty party on the subject of Unicode, encodings, and so on, it’d be the people who wrote the HTTP specifications, along with the people writing the next generation of them who are probably more inclined to dream up fancy new form widgets than define unambiguous behaviour. Nevertheless, other people have considered such matters before, and it isn’t particularly sensible to just ignore their work, even if they aren’t the “cool” people any more.
It’s interesting that some people now consider CherryPy “not a framework”. When the angst over the burgeoning number of frameworks climaxed in 2005 at the PyWebOff talk in PyCon 2005, CherryPy was one of the foremost frameworks. So what happened?
TurboGears introduced the concept of “megaframework”, which Django and Pylons are also competing in. It looks like this has now become people’s concept of a “framework”, and the simpler frameworks are now… framework components?
The original definition of a framework was “it calls you” — i.e., a container for your application — as opposed to a library where is “you call it”. So all these server/dispatcher thingys are frameworks under this definition, including small ones like Paste and web.py.
Quixote is still maintained and in production sites. Its lack of activity is due to its stability as well as its minimalistic philosophy: fewer features means fewer bugs. The Quixote developers have chosen not to compete in the megaframework arena but instead to let people come to it. It does what its paying sponsors need, in any case. :) Future development has been targeted to Qpy, which has a more extensible codebase.
I just entered into Python world. I speak Perl. Once I grab up enough syntax, I’ll look at the framework. Thanks
Regarding RoR, I’ve arrived back at Python from Ruby, finding that they have a much bigger elephant in their room, probably several elephants.
First of all, the only deployment game in town is currently Mongrel, and the proposal is that you deploy 5 or so 30 meg Rails processes PER APPLICATION. So you have a bunch of little clusters popping up everywhere hogging all the RAM, and in lieu of a single server, they propose that this all be managed by a thing called Capistrano. I simply can’t do this.
I went back to Django after seeing the simple and familiar Apache prefork deployment on mod_python. Should be good enough. RoR/Mongrel is some kind of bandaid. Who thought of this per-app clustered architecture, at 30 megs a pop? Am I seriously willing to deploy that? Guess I’ll have to take a look…
Who knows though…maybe I’ll end up going back. Get to know Capistrano. Seems to me they’ve become as complex as Java. Hey, maybe I’ll just go back to Java.
Django is the best framework not only in the python world. it is the best web framework ever built. Unlike other framework it is scalable and is ideal framework for the agile web development.
you can download the django book(pdf) at bir2su.blogspot.com
Here is a video respose to this discussion. It makes the list longer but I am trying to make a point about usability and software engineering: 1) you cannot require your average user to be as knowledgeble as the developers of the framework; 2) choose a top-down design, not a bottom-up one, otherwise your users will be disappointed by APIs that change on a weekly basis. With Gluon I have tried to address these two issues. Remember that our competitors are not other Python frameworks. Our competitors are JSP, PHP and perhaps Ruby on Rails.
I started to use Django a few months ago and I am very impressed by its ease of use and performance. I was new to Python (coming from C#) and was a bit surprised by then quickly amazed by the simplicity and power of this language !
Concerning Python frameworks, Django is definitely my favorite ! I just found the frequency of the release a bit slow. A good start to evaluate other Python frameworks : http://www.therightsoft.com/softwaretechnologies/webframeworks/?languageid=2
(though as said before in the discussion, all the items are not comparable…)
With regard to RoR’s elephants, yeah. It still lacks speed-wise and, ridiculously, requires multiple concurrent mongrels if you want any type of throughput. The hope, however, is that using fibers, event driven DB access and Ruby 1.9′s faster VM will help it avoid some of those problems, but…well…it ain’t yet :) Hopefully sometime. Now to see if we can get Rails thread-safe… Django is indeed nice. I’d put it right up there with RoR, if not above. -R
What about a robust and mature (started in 2000) Python framework that recently went public under the LGPL and has had for years the features that will be very hard to add to django, like multi-database support ? CubicWeb is a semantic web application framework worth reading about.
I tried both Django (1.1 Final) and Web2py (1.74.11) I think both these are good frameworks. WEb2py is a tag easier than Django. Great job Missio!
There is no doubt Web2Py is worth being learned and used. It’s definitively a good framework.