Note
Looking for Concurrency: looking for positive models?Prothon looks quite interesting. Reading through the description, here are a few of my thoughts.
Prothon uses whitespace for continuations, so if a line is indented from the previous line it is considered a continuation of that line. I have a really hard time keeping lines under 80 characters in width, and \ and () always break up my coding -- the first is ugly, and the second requires me to back up to the beginning of the line once I realize I'm going over 80 characters.
This is purely syntactic, Python could also do this.
This is really at the heart of Prothon. This is actually my plan for OO in PyLogo (except it's called TELL there). When you use with var:, var becomes the "self" object (though Prothon doesn't use the word "self").
This kind of scares me, though. My experience with DTML shows that this is a horrible feature. Though, that said, I'm not sure if with is quite as dangerous as dtml-with -- so long as it only applies locally it's not too bad. The issue would be something like:
with File:
def .writeAll(*objs):
for ob in objs:
.write(CustString(ob))
def CustString(ob):
if .filename.endswith('.txt'):
ob = Str(obj)
else:
ob = Str(obj).encode('UTF-8')
I don't know, I'm kind of contriving that example. But the question being, do you "inherit" the . (self) according to call order? Or only statically? Eh, they probably do the right (static only) thing here.
Or, more specifically, something like:
def Point.move(xofs, yofs): ...
In this case, adding a method move to Point. I like this, it's very natural feeling. It is used to more effect in Prothon than it would be in normal Python.
In Prothon, locals start with lower-case letters or the underscore, and globals are capitalized. This is okay. I still like lower-case functions, but I can live with upper-case functions.
I do not like all the prefixes they come up with. I think this is clearly a case of starting with one or two prefixes, then it spiraling out of control. Here's the prefixes:
| Prefix | Meaning |
|---|---|
| . | .x is like self.x, kind of. Problem: . can't be pronounced, and it's really small. I still like self, though I can understand taking it out of method signatures. I think they should make self a keyword. |
| ^ | Like "super", though the exact equivalent in Python is terribly confusing (the super() function). I'm used to it from Smalltalk, and like the brevity compared to Python's standard mechanism. A super keyword would still be better (to go with a self keyword). |
| & | This is a closure variable -- a variable that is defined in the parent scope. Anyone who has tried Scheme-style closures in Python has probably known the suffering of trying to figure out how scopes work. This makes it very explicit, which actually makes it usable. I also imagine it makes it easier to implement, since you actually know which variables need to be kept with the closure. Horrible punctuation, but a good idea. |
| @ | Dynamic local variables. There's no equivalent in Python -- you'd have to open up the current frame object and look in parent scopes. While it's expedient, I think it's also a very bad idea to provide this. That kind of mucking about with parent scopes is just asking for problems. I guess it could be used to macro-like effect in some ways. |
__init__ allows for more playing around than in Python -- you can return an object besides self and effect the result of the constructor. Of course, you've already constructed the object anyway, so maybe __new__ really is called for, if you actually want to avoid creating an object. (If you had to call ^__init__ to construct the object, then it would be more general)
In the list of little tweaks, they also allow identifiers to contain !, with the usual meaning (indicating the method mutates the object). I always like that punctuation, so I like that it's in there. It's much more useful than Python's convention of returning None -- I can tell statically that a method name contains !, but I can't do the same for returning None, and returning None is annoying. Prothon also returns self by default, instead of returning None (there's some conflicting information on this, so they might not be doing this). I like ? in identifiers as well, but Prothon doesn't add that.
There also seems to be a tendency to global methods, like obj.chr() instead of str(obj). I'm not entirely clear about this -- I think it falls out of the way builtins are defined. I generally prefer Python's style with magic methods, purely from an aesthetic point of view, though I can understand why some people would prefer using methods for everything.
There's also some changes in generators, but I haven't used generators enough to have an opinion on the subject. I think the change is fairly significant, from a stylistic/usage point of view.
It's an interesting effort. There are some small aesthetic/syntactic changes, some of which I like and some of which I don't. Then there's the prototypes... which are something else entirely.
I actually wonder to what degree prototypes could be implemented in Python with metaclasses. While some of the syntax would currently be crufy with Python (e.g., you'd have to create a function, then assign the function to an object), that's kind of a separate issue. The basic object model in Python feels like it should be replaceable, especially since we already have both new and old style classes. I think it's still a bit hard to think about these with the syntax getting in the way or otherwise being distracting, but feel like the potential is in there somewhere.
If it could be hacked onto Python instead of developed entirely separately from Python, it could make Prothon much more interesting from the perspective of practical programming. Ultimately it would probably have to be like Stackless -- a patch against Python implementing some necessary changes to syntax. But that's a more viable development effort, and one that's more likely to percolate ideas into the main Python interpreter.
I checked out your comments thinking you'd certainly have a reaction to the elimination of the GIL. *That* is what caught my attention regarding Prothon.
I seem to remember you saying you don't think the GIL is much of a design flaw. I do. I'm very interested to see if this really might be the end of the GIL. Preferably in Python. I see the GIL as the #1 weakness in Python. For me, at least, with access to machines with 16 processors.# John Mudd
The GIL needs to stay as-is if Python is going to be reasonably programmable at the C API. It would also make Python a lot slower, and it would break just about every extension... neither of which it needs. A GIL-free Python is going to have to wait until something smarter comes along, like PyPy, which could just automatically parallelize your code if it makes sense to do so.
If you have 16 processors, use 16 processes. You'll do fine. With shared memory on a decent kernel, the only real latency you'll see is going to be your fault ;)
You're probably thinking of this post:
http://blog.colorstudy.com/ianb/weblog/2003/10/31.html#P24
Wherein I agree with Bob, and think the GIL isn't so bad.# Ian Bicking
PyPy already sounded cool. If it can eliminate the GIL or find anothre route to SMP performance then it's just that much better. Cool.
I do read often that the GIL must stay. But what about Java? No GIL there. I'v even converted Python to Jython, switched to Java threads and, wow, it works great. Of course, only on a JVM. But doesn't that show the way for CPython developers? (If Sun can do it...) And I guess Prothon might also show the way if it suceeds.
Until the GIL is gone from CPython I feel like Java is still ahead, which is depressing. It's hard for me to honestly persuade Java programers to try CPython so long as there's a GIL.# John Mudd
Have you ever compared Java's performance for Python-style data structures like lists (Vectors) and dictionaries (Hashtables)? It's just a guess, but I think you'll find that Python's structures beat the snot out of Java's, because Python's don't require synchronized methods. They're implicitly synchronized by the GIL. Whereas Java objects need to have locks around each method call.
For Java, this is acceptable *only* because vectors and hashtables aren't on the critical path for performance. Java object fields are accessed using fixed offsets. In Python, those structures are used for *everything* (including all attribute lookups and use of globals), so adding item-specific synchronization kills Python performance dead dead dead. There was a guy who actually made a version of Python without the GIL, and it was "only" 50% SLOWER than Python with the GIL.
# Phillip J. Eby
>I actually wonder to what degree prototypes couldcould you describe this more?
that java needs to synchronize every method call?
i've done some java programming (2years), and when i wanted to synchronize, i used synchronize, and when i did not want it, i did not use it.
that's all.
there is absolutely NO need to put synchronize around every method call imho.
# Gabor Farkas
>be implemented in Python with metaclasses.
Prototypes could certainly be implemented in current Python (I think that Lulu Lotus Eater IBM guy posted something on the Slashdot story about it), but it's a bit like asking to implement OOP in C.
See the also comp.lang.python for more debate.# Michael
On the subject of the GIL again: I think it would be far, far more interesting to make multiple-process Python programs easier to write -- to make interprocess communication seemless, and just generally to establish robust (and easy to use!) patterns for that kind of programming. The benefits would far excede the benefits of mere SMP threading performance. But even right now, I imagine it's way easier to write a multi-process Python program than a threaded Java program.
On prototypes, Hans Nowak has played around with systems implemented in Python, in posts like:
http://zephyrfalcon.org/weblog/arch_d7_2003_09_13.html#e343
I think Prothon takes it a bit further... but maybe not that much further once you look past the syntax.# Ian Bicking
Personally, I think that pre-emptive threads with the same object space is THE WRONG PARADIGM ANYWAY. It makes for impossible to predict and debug code, thread locks suck, doesn't cooperate with event systems like GUIs, etc. I'm willing to bet that much higher performance and reliability can be attained with something else that is actually easier to code (something else that doesn't exist for a Python-like-language yet). Processes (pre-emptive switching between separate execution contexts that can be explicitly parallelized to multiple CPUs) and stackless-like tasklets (task switching in a single execution context) are the best we can currently do in Python, live with it, it's not going to change.
Everything I keep reading and hearing about repeats the theme that threads are an accident that came from people trying to "perfect" processes and thus making them too slow. Threading is way too hard and there are no indications that it will continue to be anything but a dead end. Other forms of concurrency, especially ones that are more OO and that remove problems like deadlock and race conditions, are much more preferable.
Re: "Threading is way too hard"
For me threads are much simpler to use than forking processes and, for example, using xmlrpclib for inter-thread communication. I've done both, and God bless xmlrpclib, but processes are still tedious compared to simple, almost magical, threads.
Did you mean hard to use or hard to build into the language? Even CPython already gives access to real native threads. I have to keep reminding myself of that. It's just the GIL design that kills (not just 2X, more like 10X) potential performance. I have a Jython thread demo that run circles around CPython. Of course, only with multiple CPUs.
Re: "Other forms of concurrency... are much more preferable."
I admit I'm kinda stuck on threads. What else should I try?# John Mudd
As far as I know, concurrency is far from being a solved problem in computer science.
Stackless offers some different -- or at least, unfamiliar to me -- models. Which reminds me: must beat up Christian until he agrees to talk about them at europython :-)
On another note ... I haven't seen anyone upset at the potential that alternative Python implementations like Prothon have to fragment the Python user base. People are already talking about the profileration of web frameworks in Python ... what about when the language itself has 3-4 different implementations? Jython has a good reason for existence, but much of Prothon's design seems to be related to personal preference, not actual improvements.# Brandon Corfman
Prothon IS NOT PYTHON. Did you read the post?
> I think it would be far, far more interesting to make
> multiple-process Python programs easier to write
The problem is that under Windows, a process-switch takes a lot more times than a thread-switch.
So replacing a multi threaded prog by several processes communicating with some protocols is not as much a solution as it can be with, say, Linux (I don't know it, but I have been told that under Linux process ~ thread under Windows).
That's not to say it can't be done, but there are possibly more cases under windows where you have no other choice than using threads whereas you have the option under Linux to use a multi processes solution.
I wish processes were not so heavy with Windows: I made a lot of multi-threading programming until now, I know the problems it involves and how it can be frustrating to debug such programs !# popov
Prothon is close enough to Python (notice the spelling here) to consider it an alternative implementation in my mind. Except for the 64-bit/threading issues which are a minor issue to most (wasn't this supposed to be a _simpler_ language??), the "big changes" of tabs/spaces, removal of self, and removal of classes/metaclasses is just noise. Why introduce another implementation that solves no apparent issues except syntax?# Brandon Corfman
Yes, of course, it's exactly the same except it has a totally different idea of scope, a completely different object system, and a fresh implementation intended for tomorrow's computers and very large problem sets. Come on :)
]] Dynamic local variables. There's no equivalent in Python -- you'd have to open up the current frame object and look in parent scopes. While it's expedient, I think it's also a very bad idea to provide this. That kind of mucking about with parent scopes is just asking for problems. I guess it could be used to macro-like effect in some ways. [[
Actually aren't all local variables in Python dynamic?
def func():
print x, y
x = 1
y = 2
func()
I agree that this is a really bad idea...
No. Those are globals!
>>> def func():
... print x,y
...
>>> import dis
>>> dis.dis(func.func_code)
2 0 LOAD_GLOBAL 0 (x)
3 PRINT_ITEM
4 LOAD_GLOBAL 1 (y)
7 PRINT_ITEM
8 PRINT_NEWLINE
9 LOAD_CONST 0 (None)
12 RETURN_VALUE
I can almost guarantee that the process switching overhead in Windows is far less than the overhead that you get with the GIL... because guess what, two chunks of python code can NEVER EVER RUN AT THE SAME TIME with Python threads! :) It's only carefully written C code that can explicitly allow ONE python thread to run concurrently while it's doing something way outside of the Python domain.
That said, I don't use Windows anymore, so I can't do a benchmark, but I think you are blowing the win32 process switching overhead out of proportion.
Ah, thanks for clearing that up, Bob.
To the person who was complaining about process-switch... are you sure you're not talking about the slowness of creating processes vs. threads on Windows? (By "switch" I assume everyone is talking about context switching?)