Ian Bicking: the old part of his blog

Full Stack vs. Glue

Comments:

Summarizing a comment I made in reply to you on my blog:

The full-stack/glue distinction is very real, and can be revealed by a couple of practical tests; the best one is to ask where a bug report for a major component goes -- if it gets directed to a third-party project, then you're glue. If you handle it yourself, then you're full-stack. Again, it's the difference between supporting code that helps other things work together, and supporting a full set of components.

And Django doesn't claim to be decoupled, it claims that its components are loosely coupled. That may seem like semantic hair-splitting, but it is a significant difference -- while we obviously don't go out of our way to make it hard to swap components, having "plug and play" core components is not one of our design goals; instead we focus on the components we have, on making them better and making them work better together. That is, again, the difference between full-stack and glue in a nutshell: is your goal to primarily provide and support one set of components, or is your goal to primarily provide and support ways to hook arbitrary sets of components together? For Django it's the former and so we're full-stack. For Pylons it seems to be largely the latter, and so Pylons is glue.

The coupled/decoupled alternative also really feels like the same thing in different words: the primary difference is, again, whether you're aiming to support a single set of components together, or to support swapping and interchanging of many components and sets of components. I'll admit "glue" can be something of a loaded word in the Python community (I guess it reminds us too much of Perl -- which is a shame, because good glue code is some of the most insanely useful stuff out there), but "coupled" is, as you've said, going to suffer from the same problem. We need to come up with some better terminology, I think...

# James Bennett

---

Could the difference between a 'glue' and 'full stack' framework be summarised as ... If, as a framework representative I find a third party module that does something significantly better;

• If I were a "glue" framework I would use it
• if I were a "full stack" framework I wouldn't

Or would it be OK to use the third party module as long as it becomes a component part of the "full stack" framework..

Or! If MochiKit, Kid, CherryPy and SQLObject all shared a single svn repository and all prefixed their project names with 'TG' would it then become a "Full Stack" framework?

If so then the definition is virtually meaningless..

# Tim Parkin

---

Tim, personally I think it comes down to development focus: does the framework primarily focus on a single set of components, and provide support and maintenance for those components and for the code it uses to hook them together? If so, it's full-stack ("coupled" in Ian's terminology), and you don't have to develop all your components in-house to do that -- you could, for example, use someone else's open-source components and distribute patched versions which work with the rest of your framework (and to be a good citizen you'd submit those patches back to the original project, of course). On the other hand, you can primarily focus on developing interfaces and adapters which aren't meant for any specific component or set of components, but rather to be generic enough to plug any component into any slot in the framework; that's what I mean when I say "glue" and what I think Ian means when he says "decoupled". And you don't have to use only third-party stuff as your default set -- you could develop an ORM, template system, etc. and offer them as the default set.

Either way it's a lot of work -- supporting a specific component stack is non-trivial, and neither is developing a full WSGI system with the necessary support and adapters to truly support "plug and play" components. I'm incredibly glad there are dedicated people on both sides who are willing to put in the time and effort to give the Python community the embarrassment of riches we have in the web-dev sector now, because I remember all too well the days when I desperately wanted to use Python on the web but couldn't because Zope was the only option and wouldn't fit in my head :)

# James Bennett

---

This is where things get confusing.

TurboGears does, as a simple matter of fact, pull together a single set of components, and encourage their use. We are trying to tell a unified story about how everything works together, and we've had help from CherryPy, Kid, and other components in bringing everything together in the way we want.

Let's look at the way we work with templating systems. We clearly aren't distributing a patched version of Kid, but the Kid maintainer is doing it because he uses it in TurboGears. When you use a kid template you just specify the name of the template in the @expose decorator, and everything "just works."

So far we are just like Django and the Django templating system. Simple defaults, and easy configuration make life good. But we do have a differnt philsophy. The maintainers of Kid understand that some people will want to use Kid in other ways -- outside of the framework -- and they work to support that.

At the same time we also understand that some people have hundreds of thousands of lines of template code written with Cheetah and we support them if they want to use TurboGears. So, we've allowed them to define the engine they want used as well as the template name in the @expose decorator.

And that's the difference between the TG/Pylons way and the Django way. It's not Glue VS Full Stack (TurboGears at least tries very hard to be Full Stack). It's about code reuse. We are open to pulling in code from outside and working with those developers to get what we need from their components, and we are open to the idea that others will want to use some of our components outside of our world.

Of course this is a trade-off, we're dependent on the good judgement of the people who work on Kid, unless of course we want to fork the project.

Now, I think TurboGears is doing the right thing here, but I also think there is a benefit to the centralized control that the Django guys have.

The documentation story in TurboGears is harder, because we have to pull together a clean story that includes each of the components we use. And because those projects all have a lot of internal innovation going on, it's a lot harder to keep up with all the cool new stuff that's coming in. We're definitely struggling with how to do that in the best way.

So, I recognize that we're traiding away some control of our own destiny, and inviting extra documentation and bug-tracking work, in order to get better code-reuse, cleaner decoupling, and more end-user freedom.

As Jacob once said, "it's trade-offs all the way down." I'm happy with the trade-offs we make, even though there is a cost.

I'm sure you're happy with the trade-offs you make. But you should admit that there is a real cost there too.

# Mark Ramm

---

you know what reminds me of Perl ? this:

>>> from django.template import Context
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  [...snip...]
EnvironmentError: Environment variable DJANGO_SETTINGS_MODULE is undefined.

theres nothing "loose" about that, id characterize it more as "arc-welded to a hardcoded idea of runtime configuration".

# mike bayer

---

NOOO! not the P-word, NOOOOO!

at the same this this is a straw-man argument,

seriously do you think it would be difficult to load the config from another file? As of now very few people would want to use the Django templating system outside of Django. And since there are already lots of settings needed to run a webapp, why not keep them in the same python file as the rest. Is your problem that the setting are in a python file? That is a different issue altogether and many would prefer Python to some kind of configuration mini-language.

In the end this whole discussion comes down the argument of the whole being more valuable than the sum of parts. For any subpart of Django there is a python module/package that is "better". Yet no one has put together a whole package like Django did.

Here is another tidbit, the stuff works, I used cherrypy, paste whatever and you know what, over the long term eventually each had shown some very serious problems, that come from them being more about the concept rather than a program that you need to make a living off. It will take years to iron out those kinds of problems.

Django works, it will hold up in traffic, it will not blow up in your face. That quality it is far superior to any other. Interoperability, reuse, decoupling are fine in theory, in practice adhering to them too strictly just makes life more difficult.

PixieDust

# PixieDust

---

if i read you correctly, you agree with my sentiment that django's components are not very loosely coupled, your argument being essentially, "so what ? it works." that is completely fine, and im glad django is there, running very well, and attracting lots of new people to Python. its just that, there are definite advantages to componentized and environment-agnostic architectures, and im glad the community has now accepted that there will always be multiple web framework approaches (in constrast to the calls for "one true framework" a couple of years ago) so that those of us who want them can choose to use our own frameworks like Pylons which make this a priority.

# mike bayer

---

The full-stack/glue distinction is very real, and can be revealed by a couple of practical tests; the best one is to ask where a bug report for a major component goes -- if it gets directed to a third-party project, then you're glue. If you handle it yourself, then you're full-stack.

Like I said, I think Django's choices are easier to justify based on development methodologies and community building, for reasons like this. It's clear who responds to bugs. Django is never stuck when they really want to make a change in something, but the upstream maintainer doesn't agree or doesn't have the same priorities. Django can provide a very concise and clear set of documentation, and activity is all centralized around one basic set of metaphors, mailing lists, bug trackers, etc. And I actually like centralized development, so I'm quite sympathetic to these arguments. (Though as a counterexample: Mozilla is a single project, and it's hardly easy to navigate as a developer.)

As an engineering principle I don't think this distinction is meaningful. That is, from an engineering perspective you can get to the same solution but with slightly different names, and based on names one solution is "full-stack" while the other is "glue". And I don't think that's a reasonable distinction to make. But of course from a mathematical/symbolic point of view names don't mean much, but for humans names do mean a lot.

Again, it's the difference between supporting code that helps other things work together, and supporting a full set of components.

But see, there you start drifting. This is not the difference. You can achieve this regardless of how the packages are laid out.

# Ian Bicking

---

I agree that the package layout doesn't necessarily impact it (see my other reply in this thread -- it's certainly possible, for example, to be "coupled" while using third-party components, and "decoupled" while using in-house components as the default). And I still agree wholeheartedly that we don't have a good way to talk about this, because the terms that come to mind are all loaded with connotations from other things. I do think there's a clear difference in focus between Django and some of the other frameworks, though -- we are focused pretty much exclusively (partly for historical reasons and partly because of available developer time) on our single stack of components, while other frameworks focus much more on adaptability and interchangeability. And I still think that's a choice which will end up affecting every part of a framework, and that it's not possible for most projects to pull off an equal focus on both, because both routes require major investments of developer time and effort.

# James Bennett

---

Yea, the Django people have a lot easier time with documentation than we do. On many mornings I find myself envying them.

Which is not to say that I'd give up the code-reuse paradigm in order to make my life writing documentation easier.

# Mark Ramm

---

Hmm, no capitalists in the debate? I guess "let the market decide" isn't the favored approach? Or are we trying to save the folks wandering in from making a terrible wrong decision?

Concept-wise I tend to fall towards the ["Thou that shall not be named"] WSGI-ish style of Pylons, but in a practical sense (there's some place for that, right?), the WSGI-ish approach really needs a project as approachable and documented and "polished" (at least for basic stuff and beginners) as Django to be able to argue on a level playing field. At least if the point of all these debates (that look like developer pissing contests) is to educate people new to python webdev. It's a bit detached from reality to argue you're 10x better technically but then not have the average newcomer able to figure out how to use your kit. Though again, this is rampant in technology.

Also rather than the Windows/Mac, fullstack/glue analogies mentioned, it looks a lot to me like Windows webdev vs Java webdev (wherein the java flavor is say Spring + Hibernate + 23 other libs from diff vendors or groups compared to the all-MS tools on the other side). "Single-source" vs. "multi-vendor"? I guess thousands of people still fight over those two also, but it doesn't seem like an ideal use of energies.

# ToddG

---

Hmm, no capitalists in the debate? I guess "let the market decide" isn't the favored approach?

The market is made up of individuals making informed and conscious decisions with many factors that are ultimately weighed intuitively. Discussions like this inform those decisions, and inform the market. As a result I think deferring to the market is a little silly: we are the market, this is the market working, there's nothing external to be deferred to. And programmers are uniquely positioned to be both consumers and producers, and to frequently make decisions about what role they want to play in a certain circumstance.

For instance, it would be an entirely valid choice for someone to use a piece of Django that they take from the framework; to choose decoupling whether or not the framework encourages it. It would be valid to wrap Django against its will, though you have to remember that it will also be fragile (and that package is almost certain to be broken right now as a result). The choices we have are infinite, and not just in theory but in practice. With so many choices, how is someone supposed to make a good choice without substantial amounts of discussion?

It's a bit detached from reality to argue you're 10x better technically but then not have the average newcomer able to figure out how to use your kit.

I didn't actually say the decoupled approach is "better", though of course it's generally accepted that decoupled is better than coupled. But that's only true when everything else is equal, and I am not claiming everything else is equal, or even that coupling is a free variable when considering other important attributes. But decoupled and coupled are fairly objective terms, so I feel justified using them clearly in this context; better and worse are highly subjective and contextual.

# Ian Bicking

---

Discussions like this inform those decisions, and inform the market.

...

we are the market, this is the market working, there's nothing external to be deferred to. And programmers are uniquely positioned to be both consumers and producers, and to frequently make decisions about what role they want to play in a certain circumstance.

Well sorta -- I think the number of people involved at low levels with the projects in question (variable) is a tiny part of the overall "market" in this case -- the handful of people doing (at least publicly) WSGI stuff and/or core Django devs is (I hope) a very small part of the overall userbase. But certainly it's still an example of the market working, even if it seems weighted towards the producer end of things. I was casually dropping the market comment in relation to "the masses" (we can hope!).

It seems to me the fact that programmers are often "consumers and producers" is often a reason many projects become opaque and unapproachable to people who aren't as deeply involved -- that is the "occasional" consumers or new consumers who are at a totally different level of "consumer" than the producers. This of course doesn't negate any sound reasoning in design or whatever, but many projects fail to successfully address these consumers. Often on this point a more "packaged" or "designed to all seamlessly work together" solution has an inherent headstart, as it's a much simpler story to tell.

In other words, until the solution you (and myself to my more limited abilities) support can demonstrate the ability to retain (build on) its technical strengths but still appeal to non-"producers" and more casual users and make _those_ things equal, the endless technical ping-pong might be unecessary(?)

# ToddG

---

"Hmm, no capitalists in the debate? I guess "let the market decide" isn't the favored approach?"

I don't necessarily regard myself as a capitalist, but I decided to ignore the "debate" and do my own thing, although this isn't news in any sense.

Most of what I've been reading is just advocacy dressed up as methodology: framework A is doing "the right thing" because of the perceived attitudes of their developers, the architectural choices, the ease of use (delete as appropriate), whereas framework B is doing "the wrong thing" because it does something different for all those things, all according to someone's sense of taste (or lack thereof).

Really, the TurboGears and Pylons tendencies to seemingly mix and match with varying levels of readiness obviously provides benefits, although you're still buying into someone's vision of Web programming, and although there's a book out for the former (has the code even reached 1.0 yet?) the documentation may not convey that vision effectively. Meanwhile, Django evolved in relative isolation, although one of its creators seemingly attempted early on to further the Python Web standards effort (with few results), but you've got to buy into the vision a little bit more, although there's a possibility of slightly better documentation about that vision.

Frequently, people who don't know how to start writing their Web application want to buy into a vision. Once sold, they'll produce any amount of hot air to justify their choices. That there's a technological disconnect between various communities says enough about people's willingness to produce hot air instead of working on the things they should all have in common.

# Paul Boddie

---

Most of what I've been reading is just advocacy dressed up as methodology: framework A is doing "the right thing" because of the perceived attitudes of their developers, the architectural choices, the ease of use (delete as appropriate), whereas framework B is doing "the wrong thing" because it does something different for all those things, all according to someone's sense of taste (or lack thereof).

As someone who writes software and discusses software, obviously advocacy and methodology mix. I program with the methodology I think is right. I advocate the methodology I think is right. I advocate the framework that implements the methodology I think is right. It's not dressing one thing up as the other thing -- it's the two faces of thoughtfully pursuing the craft.

# Ian Bicking

---

Projects can become very difficult to maintain when you throw in a lot of third party libraries/software. On the other hand, the effect of this is no different than having your web framework releasing backwards incompatible APIs. I don't want to have to worry about the future. I want the python web framework that provides me with the most confidence -- the one that won't break my code when I need to update for a bug fix (or enhancement need) in the web framework.

# Bryan

---