Zope and an RDBMS are a horrible combination. It's awkward and annoying for a while, until it becomes unmanageable and horrid. Like, did you know if you get a conflict error from your RDBMS, Zope will retry the entire transaction (i.e., rerun the request)? It would be handy, unless it's not what you want, and if it's not what you want nobody knows what to do next. That's just one of many gotchas when using an RDBMS with Zope.
I would just dive in and change stuff around, but I know I'd just recreate another set of problems, because everything is so damn subtle (read: tricky and likely to break). Zope learning starts out hard, then gets easy after an hour, then gets hard again after another hour, then gets impossible after a week. Or it stays just hard, but you've unwittingly created buggy, unscalable, insecure applications.
All my Zope unhappiness might be okay -- not great, but maybe okay -- if it was just pleasant once in a while. But it's never pleasant, just bearable.
Damn, I'd almost rather work in PHP. Actually I would. That's how much I dislike Zope. At least you get all their crap up front, and you can just work around it.
I'd be happy to get you a copy of Programming Perl, if that might help. :)
Ian,
Very similar experiences to what I had. It was deathly hard to get my head around zope, then when i tried to do anything beyond simple it got very hard again. I found that when just using plone to have a basic intranet portal (read link dump to internal documents) it was straight forward. But once I tried to use a RDBMS I kept on feeling like I was wrestling with pretty simple stuff to make it zope like. The quixote and webware approach seem much more friendly in hindsight :)
Wha? It's just Python, guys.
I agree with Chris M. I've been doing RDBMS with Zope for years, and it was very natural to me from day one.
In fact, I have yet to find a RDBMS interaction system in Python that I would prefer. And believe me, I've looked. (In house frameworks should eventually satisfy me).
Zope's never felt like Python to me. Obviously it is Python... maybe I mean "Pythonic", for some internal concept of what Pythonic means to me. Maybe it's just that I've only used Zope professionally, and that colors my perspective, but it's never been emotionally satisfying, which isn't true of programming or Python in general.# Ian Bicking
I don't think you've explained the actual problem very well. Maybe you'd rather not, which I can understand. ;-)
In part it really is an emotional response, which is hard to pinpoint in one place. It's not any one detail -- there's always details that are problematic with any system. I could create a long list of details, and that would account for some of it, but even those cumulative details don't really account for it.
I think it's really a feeling that I am not in control. I don't feel confident about my applications, and the way they will work in production. They depend intimately on a system that I don't understand, that I feel I can't understand. Learning about Zope doesn't lead me to feel more in control -- often it's exactly the opposite, I feel less in control the more I learn, which is incredibly discouraging. And in the end, I simply don't buy the It's Complicated But It's Powerful argument -- not just for Zope, but in an abstract sense I find that argument flawed. Even its "power" feels like a flaw to me, so I'm left with very little motivation.# Ian Bicking
I'm sympathetic because it's not as if I haven't cursed Zope up and down for one reason or another on a daily basis. But it does its job more or less most of the time. Then again, I've never asked Zope to satisfy me emotionally, so maybe I just have low standards. ;-)
When I say "emotional" of course I mean the difference between enjoying what I'm doing, and simply doing it because I have to. The word "emotional" might carry more weight than that, though...# Ian Bicking
Zope, asyncore, medusa,...
This is the downside of Python: it allows hard-working people to produce something working. The result is of course far from elegant, difficult to grasp, difficult to use, ...
I would recommend Python only to talented people. Otherwise, Python acts as a bullshit magnifier ...
My 2 cents
Although it's often a problem with Web applications, Zope never really seemed to lend itself to incremental development and testing. I know that they've improved things recently, and the emphasis on components is very welcome, but there's nothing worse than having to stop and start servers, redeploy stuff, and so on (especially if the software runs like treacle).
I've come round to thinking that architectures should be as shallow as possible because they then lend themselves to being comprehensively tested and understood. Zope and various J2EE frameworks seem to be like towers in contrast to such ideals.# Paul Boddie
For some projects I'm working on, the lack of explicit control over transactions is a deal breaker. (There may very well be a way to get explicit transaction control, but I sure can't find it.) Not to mention the fact that in several respects the bog-standard python DB-API is superior to what you have to wrastle with inside ZSQL templates. Chris's "it's just python, guys" comment is misleading, while true; python is both a way of doing things and a specific implementation. Zope strips from you almost everything you'd expect to be doing if you had experience working with relational DBs with python outside of zope.
I think Ethan hit on what I was trying to say. ZSQL templates just seemed unnatural to me. Also, once I had everything worked out in ZSQL scripts it wasn't usable outside of Zope so it seemed limiting.
I found working with the DB-API much more straightforward.
I've worked with Perl, PHP, and Python (in that order). I've settled into Python, I like it very much. It's "satisfying".
I tried Python CGI, and ended up getting really frustrated. I tried Zope, and have had mostly good experiences. I have a few applications working, and am introducing my co-workers to it now, because I think it's relatively easy to use. The web interface is comforting if you haven't done a lot of web development. I like the fact that I don't have to rewrite common code, like user authentication, it's just built in.
Now though, I'm really not sure what direction to go in. I'd like to be able to commit to a direction for a while. The first step was focusing on a language, I chose Python, now I need to focus on a web framework. It's a really muddy field, and it's very frustrating.
Ethan: yes, the DB-API is a good example. In fact, the DB-API is pretty lame to work with -- it's long-winded and not that pleasant to use. But, all its problems are up-front and obvious, and it lends itself well to serving as a base for a more pleasant system. Z-SQL methods are, of course, quite the opposite. Besides being one language (Z SQL) piled ontop of another language (DTML), neither of which are good languages (and they definitely are not Python!), they also resist safely abstracting the SQL generation process, and they resist sensible composition.# Ian Bicking
My thoughts are that Zope is terribly unsatisfying when you try and coerce DTML or ZPT do too much of the work that SHOULD be done by an external python script. Go here http://mail.zope.org/pipermail/zope-db/2003-July/thread.html
and read the thread "Hello!" - I found it enlightening.# Erik
Note: no condesention towards the author of the original bit - this link would only be helpful to others at my, lower, level of zope enlightenment.
It really is a multi-leveled app server. It makes somewhat hard things easy, and then they get awkward as you want them to do more, and then, do they get easier again? Hopefully one day I will find out...# Erik
I've been _loving_ using ZPT and python scripts to display and edit databases using ZSQL. After researching lots of different ways of doing it, (Tapestry, jsp, webware, php, jhtml, etc...) I've found my favorite way is using zope.
I've put together about five different sites that work with databases, and the first one took me about five days of part time, and subsequent ones haven't taken more than about two hours each. (Each one deals with 3 SQL tables max.)
I have forms that act as both an add and an edit, and I've never had any databese requests fail. The ZSQL Connection has lots of parameters that let me control everything that I'd want to, and it does some nice database pooling so I don't have to.
I think my advice to anyone else would be to avoid all the shiny things in zope that are supposed to make things easier, and keep it to Page Templates, Python Scripts, and ZSQL methods.
-Jim
Well, as for speed of implementation, zope can be a RAD tool for web development. It's just that it seems to lock everything up inside the mysterious ZODB.
Ian, saying that you'd prefer php, cmon we all know you'd be using webware long before you had to resort to php ;P
My "it's just Python" comment was meant to mean you can use the DB-API if you want... just write the code to do it and plug it in an external method or product on the filesystem. Those developing in Zope through the web will always be terribly disappointed at some point, sorry, nothing can be done about that.
I sort of call bullshit. If you want a raw rdbms connection
object in Zope. If you have a DA Adapter in ZODB you can grab a hold of the raw db-api connection object. As far as catching ConflictError - I believe you can catch this.
from ZODB.POSException import ConflictError
try:
your_code()
except ConflictError:
log('eek, conflicterror')
maybe this works?
~runyaga
They way I enjoy using ZSQL with a MySQLDA and ZPT + Scripts is described here:
http://www.maplesong.com/zptzsql.html
Let me know if this isn't very zopish...
-Jim
Chris, Alan: I generally worry that I'm going to get burned when I try to treat Zope as "just Python". This is mostly due to Zope's security, which at once stymies me and scares me -- I know there's big holes in my applications, but none of the holes that I want. Anyway, passing objects into and out of Zope is less than trivial -- I only feel confident when I'm passing builtin structures (strings, lists, dicts, etc).
But yes, I could figure this out and use the DB-API and ignore Zope entirely. That's frustrating, because that's not how I've developed the application so far. And I've been following the documentation's suggestions -- the problem is that most Zope documentation suggests a style of programming that is not appropriate for complex applications. And now that the applications exist in their current form, moving into the plain Python environment is not that easy. And if I do, why do I want to use Zope at all?
Okay, okay, there's ways to deal with it all, and there's ways ways to use Zope better, and there are things that Zope provides -- but none of that addresses the real driving factor behind the post: Zope makes me unhappy. You can't argue with that! My programmer-happiness expectations are high -- as well they should be! This is Python after all, we should expect more. I don't like the feeling that I don't understand the system, that I can't easily understand the system; that's there's always a technique I'm missing; that there's always a system interaction bug I'm not expecting; I don't like that I feel ignorant, and yet I feel like I know more than many Zope programmers -- I can deal with ignorance, but there's something wrong when there's so many people starving for too little knowledge. And I hate, hate, hate the idea that Zope Is Complex But Powerful. The whole idea of Complex But Powerful feels fucked up to me; a great way for consultants to sell their services. I'm just trying to build stupid websites, not rocket science, these aren't advancing the state of the art... shouldn't that be easy? I'd like to get to the rocket science part, but if I can't build the stupid website easily I'm not going to buy into the rocket science.
Now I'm just getting ranty. But it's frustrating.# Ian Bicking
Ian, if you're interested in using something more pythonic, you might want to try twisted (www.twistedmatrix.com). I started using Zope in 1.x days and have since left it for the same reasons you've highlighted here. I've found twisted difficult to get started but once you get your head around some of the basic concepts, it's actually pretty fun to work with.
You could use the 'high level' web application development package Nevow (http://nevow.com/) which is the next generation woven. Nevow is usable but still not 1.0, so perhaps you might want to use twisted.web.server and twisted.web.resource directly. It even supports ssl/tls. You could plugin processors for other types of 'documents' (twisted supports php and quixote templates, for example).
One negative point is that being new, twisted docs are...well, have plenty of room to improve.# Mojo
I wrote a long response to this here about TTW development vs. filesystem-based development but just erased it. It feels pointless, as this topic has been discussed ad-nauseam on Zope lists and elsewhere.
Maybe you should consider recusing yourself from doing Zope development for a while? I'm not trying to be snarky at all by saying that, it's a serious suggestion. There are plenty of other Python frameworks for building web apps out there (but you know that...) Zope really just isn't for everyone, and it's *ok* that you prefer using something else. I'd miss the outlook of your posts about Zope, actually, but they seem to be at the expense of your "programming happiness" which no one should deny you of! ;-)
Chris: the Zope development I do is purely as an employee, so I don't entirely get to choose to recuse myself. But I am actually pushing pretty hard at work for us to try new things. Like Webware (obviously, considering my background with that project). I've held back from making those suggestions before now, because I haven't felt as confident about my own assessment of Zope, or my assessments of the company's needs. Given a little more time with both of those, I'm being more active in pushing specific technologies and strategies.
It's funny though... am I pushing these changes because they are the right choice, or just the right choice for me? Of course, with only three and a half developers, even if it's only the right choice for me, that doesn't mean it's not the right choice for the company. It's hard to be objective, but at the same time I'm not a disinterested consultant. But I'm proud of that, you can't be disinterested and constructive at the same time. And sometimes I feel partisan, but maybe I'm just opinionated and invested.
(BTW, Chris, I found the document you wrote about collaborative development in Zope to be quite helpful in thinking about our processes -- if I felt committed to Zope development, that would be the direction I'd be pushing; at the same time, it seems like nearly as radical a change to our development process as using Webware)# Ian Bicking
Ian can't agree with you more!! Bascially my feeling is that writing Python should be fun and almost always is!! Python is usually a refreshing break from writing Java, everyday. But Zope isn't a refreshing break from J2EE if you will. Specifically Servlets part of the spec. Sure love plone and the content managment management stuff is great! If Zope was to J2EE as Python is to Java. From what little I have read about Zope 3 it is supposed to be better (which is quite a loose term). Here's hopping.
LHJ# Lateef Jackson
The "Zope is Python's J2EE" view is quite compelling, especially if you look at some of the Zope 3 documentation. However, after reading the recent comments of James Gosling (and having done a fair amount of J2EE development), J2EE doesn't really seem to be a great role model, at least where simple and understandable Web development is concerned.# Paul Boddie
I don't get the impression that J2EE is a model for Zope 3 at all, at least on any technical level. In terms of strategy and target domain, sure, but that's probably not where J2EE's flaws come from.# Ian Bicking
I think Erik's comment about using external scripts is right on target. I have had more than my share of frustration with Zope as well, Ian. And I really do get your emotional reaction. I find coding python one of the most enjoyable things I can do with a computer and Zope has often been very frustrating in contrast. And, like you, my major use of zope has been for a work project.
But lately I have traced my dislike of zope to two main factors. Really, really muddled documentation (which has gotten somewhat better since 2.6) and the fact that I was using the wrong parts (muddled documentation, anyone?) of it in the wrong way. Switching from DTML to ZPT, understanding and extensively using external scripts and developing as products instead of internally has gone a long way toward easing my pain. I won't say that I find developing in zope a dream, but honestly I find developing with any extensive framework always to have a good share of annoyances and gotchas as I have to wrap my head around someone else's idea of the best way to do something. That might just be me being a little slow to pick things up unless I coded them, tho.# John P. Withers
I faced the same problem with one of my last Zope projects (a Zope product with tons of ZSQL statements in a seperate .py file which were rather awkward to integrate with an OO program structure).
For the successor of this project i used SQLObject :) Worked well (i hooked up the transactions to the Zope transactions), but i wasn't able to directly publish SQLObjects on the web (with bobo_traverse) & therefore had to use wrapper code.
For my next project I will try out Twisted + Nevow, although I am not sure if async programming is a good idea...
Guenther: it depends on what your project is up to, but TooFPy (http://pyds.muensterland.org/wiki/toolserver.html) might be an alternative for you. Just plug in a good template engine and for example SQLObject for persistency. But be warned that TooFPy is more targeted towards web services than web applications with browser based GUIs, so it's rather low-level.
Is Zope3 hoping to address any of the problems discussed in this thread?
Not directly, but I wrote a small wrapper around SQLObject (Hi Ian! :) that solves a great part of this problem, if not all. It integrates very nicely with the transaction machinery of ZODB, because the transaction api in Zope 3 is much better defined. Unfortunately, Zope 3 may not be an option for everyone :)
I've resorted to building my own application framework in PHP that persists objects in individual files vs. a "black box" database and implements limited Zope-like acquisition of methods and properties. It's not a complete Zope knock-off by any stretch, but it will do 90% of what I'd use Zope for, sans the hair-pulling.# James
I started to look into zope as I was looking for a decent templating system for CherryPy. I stumbled on SimpleTAL (ZPT without Zope) which seems to work somewhat well with CherryPy.
Still undecided what to do, I may know better in a few days.
# Eric Baker