I think the most interesting work in programming languages right now is about the runtime, not syntax or even the languages themselves. Which places PyPy in an interesting position, as they have put a great deal of effort into abstracting out the concept of runtime from the language they are implementing (Python).
There are of course other runtime environments available to Python. The main environment has and continues to be CPython — the runtime developed in parallel with the language, and with continuous incremental feedback and improvement by the Python developer community. It is the runtime that informs and is informed by the language. It’s also the runtime that is most easy-going about integrating with C libraries, and by extension it is part of the vague but important runtime environment of "Unix". There’s also Jython and IronPython. I frankly find these completely uninteresting. They are runtimes controlled by companies, not communities, and the Python implementations are neither natural parts of their runtime environments, nor do the runtimes include many concessions to make themselves natural for Python.
PyPy is somewhere different. It still has a tremendous challenge because Python was not developed for PyPy. Even small changes to the language seem impossible — something as seemingly innocuous as making builtins static seems to be stuck in a conservative reluctance to change. But unlike Jython and IronPython they aren’t stuck between a rock and a hard place; they just have to deal with the rock, not the hard place.
So here is my unsolicited advice on what PyPy-the-runtime should consider. Simple improvements to performance and the runtime are fine, but being incrementally better than CPython only goes so far, and I personally doubt it will ever make a big impact on Python that way.
PyPy should push hard on concurrency and reliability. If it is fast enough then that’s fine; that’s done as far as I’m concerned. I say this because I’m a web programmer, and speed is uninteresting to me. Certainly opinions will differ. But to me speed (as it’s normally defined) is really really uninteresting. When or if I care about speed I’m probably more drawn to Cython. I do care about latency, memory efficiency, scalability/concurrency, resource efficiency, and most of all worst cases. I don’t think a JIT addresses any of these (and can even make things worse). I don’t know of benchmarks that measure these parameters either.
I want a runtime with new and novel features; something that isn’t just incrementally better than CPython. This itself might seem controversial, as the only point to such novel features would be for people to implement at least some code intended for only PyPy. But if the features are good enough then I’m okay with this — and if I’m not drawn to write something that will only work on PyPy, I probably won’t be drawn to use PyPy at all; natural conservatism and inertia will keep me (and most people) on CPython indefinitely.
What do I want?
- Microprocesses. Stackless and greenlets have given us micro-threads, but it’s just not the same. Which is not entirely a criticism — it shows that unportable features are interesting when they are good features. But I want the next step, which is processes that don’t share state. (And implicitly I don’t just want standard async techniques, which use explicit concurrency and shared state.)
- Shared objects across processes with copy-on-write; then you can efficiently share objects (like modules!) across concurrent processes without the danger of shared state, but without the overhead of copying everything you want to share. Lack of this is hurting PHP, as you can’t have a rich set of libraries and share-nothing without killing your performance.
- I’d rather see a break in compatibility for C extensions to support this new model, than to abandon what could be PyPy’s best feature to support CPython’s C extension ecosystem. Being a web programmer I honestly don’t need many C modules, so maybe I’m biased. But if the rest of the system is good enough then the C extensions will come.
- Make sure resource sharing that happens outside of the Python environment is really solid. C libraries are often going to be unfriendly towards microprocesses; make sure what is exposed to the Python environment is solid. That might even mean a dangerous process mode that can handle ctypes and FFI and where you carefully write Python code that has extra powers, so long as there’s a strong wall between that code and "general" code that makes use of those services.
- Cython — it’s doing a lot of good stuff, and has a much more conservative but also more predictable path to performance (through things like type annotation). I think it’s worth leaning on. I also have something of a hunch that it could be a good way to do FFI in a safe manner, as Cython already supports multiple targets (Python 2 and 3) from the same codebase. Could PyPy be another target?
- Runtime introspection of the runtime. We have great language introspection (probably much to the annoyance of PyPy developers who have to copy this) but currently runtime introspection is poor-to-nonexistant. What processes are running? How much memory is each using? Where? Are they holding on to resources? Are they blocking on some non-Python library? How much CPU have they been using? Then I want to be able to kill processes, send them signals, adjust priorities, etc.
And I guess it doesn’t have to be "PyPy", but a new backend for PyPy to target; it doesn’t have to be the only path PyPy pursues.
With a runtime like this PyPy could be an absolutely rocking platform for web development. Python could be as reliable as, oh… PHP? Sorry, I probably won’t win arguments that way ;) As good as Erlang! Maybe we could get the benefits of async without the pain of callbacks or Deferreds. And these are features people would use. Right now I’m perceiving a problem where there’s lots of people standing on the sidelines cheering you on but not actually using PyPy.
So: I wouldn’t tell anyone what to do, and if someone tries this out I’ll probably only be on the sidelines cheering you on… but I really think this could be awesome.
Update: there’s some interesting comments on Hacker News as well.
No related posts.
There’s a lot to digest here, so this is by no means a comprehensive response, but I know we have a student who’s preparing a GSOC proposal on adding a PyPy backend to Cython.
Interesting. I [hear elsewhere](http://www.reddit.com/r/Python/comments/gidcj/myunsolicitedadviceforpypy/c1nskvp) that it’s intended to compile Cython code to plain Python and ctypes. I guess I can kind of understand this, but it also kind of feels like PyPy is unwilling to commit to any runtime at all — PyPy after all runs on something but it’s an unnamed and (I presume) shifting target. Therefore the only way into PyPy is through the front door — Python itself, and some things like ctypes that PyPy has committed to supporting. Cython developers seem to have a lot of opinions about ctypes, I’d be curious about the tradeoffs they would see in this approach.
I think its an interesting thought experiment – but I think you grossly underestimate the draw of a significantly faster Python interpreter by itself. Something that’s blazingly faster than CPython, and has C library compatibility would simply become the de-facto “future of the python interpreter” because (and I’d put money on this) it can support the massive c-extension ecosystem we have, and is generally usable outside of a myopic web application.
That said: Breaking compatibility with CPython means you abandon a massive contingent of the community, though as you say – if the features are compelling enough, maybe that’s worth it – but then again look at the history and adoption of Stackless. Stackless has a compelling series of features, but it’s never seen widespread adoption.
I think there’s a middle ground for what your asking for – and one that nets us both of our unicorns. First, PyPy finishes C extension compatibility. Second, the features your talking about – ones that can be done on the interpreter level, without changing the language (Runtime introspection of the runtime) should and could be done.
Then, handle shared objects and microprocesses as from pypy import (blah) – you add low level support as needed for these, and you offer them as pypy specific addons. We’re in the position now of moving towards a common, shared standard library, with CPython-specific modules marked as such, and there’s no reason why it couldn’t be the same for PyPy.
Or you flip it on with a runtime flag :)
There’s a lot of people cheering PyPy on; but not using it because it’s not as fast as it could be – yet, and most of us are locked down to c-extensions we must use. Not because we don’t want to use PyPy – it’s already faster in a lot of respects, but it eats more memory, or doesn’t support the C extensions.
I personally am more interested in worst cases than best cases, and here I think PyPy will have an incredibly hard time competing with CPython. Benchmarks tend to be focused on best cases, things that can be reasoned about, not too many interacting pieces. Performance worst cases are tucked away, hard to test, hard to even know what to test, or what expectations might be. These are the O(2^n) cases where the programmer never really noticed a particular n accumulating in their code.
CPython has in many ways addressed these issues through the brute force of time and iteration and education. Bad cases have been solved. Cases that are too hard to solve have been avoided. A library has some bad performance, and someone patches it to make it better for CPython. But all those patches, all that knowledge, is unlikely to translate to PyPy.
Some people’s work actually looks like a microbenchmark — scientific work, perhaps, or other data processing. But for people who are trying to solve more general problems — to make something that works across data sets not yet seen — those worst cases are a tremendous draw towards runtime conservatism.
True. But Stackless was also fairly overtly rejected by Guido and in turn by the community — I’m not sure it failed on the merit of its features alone. I also think it was a poor compromise to reject Stackless in no small part to allow Jython and IronPython. But… there are political concerns that PyPy developers have to keep in mind as well, and a narrow focus like performance will solve those political concerns. But it solves that problem at the expense of creativity in my opinion.
I’m not actually sure what changes to “the language” means exactly. Stackless is syntactically just Python, after all. Greenlets are no more portable, though they are less intrusive to the actual implementation of the runtime. The C Extension API is not really considered to be “the language”. While we have no primitive to freeze an object, it is not unreasonable that such a thing could be implemented on existing platforms; and even if it can’t, is it still a change to the language if you implement such a routine? A pragma or external annotation to a module to declare that it can be frozen doesn’t seem really like a change, though of course it is likely to be awkward in the way transitional features are.
I think maybe a more accurate way of phrasing what I think you want is: you want existing code to work well on PyPy without change. That in part means supporting the language as specified, or more accurately as implemented by CPython. But it’s expanded to also mean supporting the ecosystem of C extensions. On this path I’m not sure there’s a clear point at which PyPy is free to stop following CPython.
http://speed.pypy.org/, and http://speed.twistedmatrix.com/ would like to talk to you. Speed.pypy.org is at least based on mostly real-world, non synthetic benchmarks (something the Unladen Swallow guys lead). Also, we are in progress for a speed/performance.python.org – based on the speed.pypy.org project, we will have a single system showing performance of real-world benchmarks for CPython, Jython, PyPy, etc.
So, I disagree with benchmarks being focused solely on the best cases :)
You’re right; but the PyPy folks have taken the right approach to that. They have a standing offer – bring them your slow python library, and they will help you optimize it/make it fast on PyPy. Sure, there’s plenty of institutional knowledge about CPython trickery to speed things up, but if PyPy’s goal is “write python – we’ll make it fast” then things should “just work”.
Oh, not disagreeing that there’s runtime conservatism: I know there is, I’m insanely conservative when it comes to the runtime. But PyPy is rapidly becoming a contender in this space, and as we work (we being the PSF and the community) on showing PyPy on equal footing as CPython, I think you’ll see interest translate to usage as they both get faster, and hit C extension compatibility.
I’m going to call foul on this. Attributing the lack of stackless adoption on Guido in any part is folly. The problem is that its features were not compelling enough for people to adopt it. Just like we’re seeing with the slow adoption of Python 3; compelling features rule the roost (and marketing helps a lot).
And there’s aren’t any political concerns. If you think there are; I’d be happy to address them offline via email since I doubt we want to hijack things, but I’m pretty comfortable saying that this is not the same community of bygone years. The PSF is here to financially and otherwise support all of Python, and as those of us who were at the language summit know – there’s a large amount of encouragement and support from python-core itself for PyPy.
For example – we’re working on breaking out the stdlib to work with PyPy. We’re working on making sure there’s a pure Python implementation as well as a C fallback for speed for module, so that PyPy isn’t punished due to lack of a python implementation, etc.
These recent, and significant changes in the thinking of everyone are changing the nature of the game, for the better.
My only point is to implement them in a way that makes the code run on CPython, PyPy and IronPython/Jython. If they’re compartmentalized into extension modules within a slightly expanded Stdlib (marked as @pypy_only) great!
As for the C extension API – it’s more of the language than you think (see the stable ABI and other things that have come up). People rely on that heavily, daily – changing it is almost as hard as changing the core language syntax itself.
Yes, I want existing code to work well on PyPy without change. And yes, that’s my fundamental point – breaking compatibility with CPython is a mistake, even with a compelling feature set. Every day that clicks by sees more and more Python code being written against CPython, which is the reference implementation.
Now, adding features in a way that’s cross compatible – that I can get behind. Even if the modules are PyPy specific, it still means that PyPy is compatible with my code today, which is insanely important. If PyPy is faster, supports my existing code, and it has some built-in add on modules that use some dark magic within PyPy to support Actor concurrency, then awesome – I’ll switch tomorrow.
Once I have more ram :)
Just to narrow in on the “political” thing…
Probably political isn’t the right word; but I mean choices-that-aren’t-just-technical. Thinking back to Stackless again, I remember a lot of enthusiasm, and then after some iterations with Python Core and Guido there was a sense of despair and giving up. If Stackless-as-a-project could have confidently said “whatever, we’ll just keep doing our thing” then I think it could have seen considerably more uptake. That it exists and seems fairly viable now is a testament to it from a technical perspective, considering that it was widely seen as abandoned and a red herring for a long time. Somehow it just refused to die.
Python 3, in contrast, is an alternate Python implementation that persists without being held up to any Darwinian implementation standard. People don’t say “well, they can write Python 3, and if anyone starts using it we’ll see, but let’s let the market decide”. This isn’t a criticism, but it’s a question of process, consensus, and something I maybe incorrectly labelled as “politics”. I don’t believe build it and they will come is a great strategy (though its opposite make sure people will come then build it is also not a great strategy). For PyPy to be “accepted” by the community doesn’t mean it needs market dominance. It needs to feel acceptable, even to the people who aren’t using it.
And if it isn’t clear, I respect giving these concerns some weight in deciding where to go with PyPy — when I called a concern political it wasn’t disparaging.
It’s worth mentioning that in the brave new world where PyPy reigns supreme, a lot of existing C extensions don’t need to be C extensions any more. You can write them in native Python with, in theory, almost no performance hit. I do not have a lot of experience writing such extensions, but my sense from perusing the source of, say, Numpy is that much of the code (e.g., ref counting) is just “dealing” with the Python interpreter, and a fair portion of what remains is inevitable clumsiness that results from writing C in a Pythonic manner — a list comprehension becomes 10 lines of loops and variable assignments, etc.
In other words, the complexity and bulk of the code belie the “core functionality” of the module itself. That largely disappears with PyPy.
I am by no means minimizing the gargantuan task of “porting” the huge C module ecosystem back to Python, but it’s the possibility of doing should be taken into consideration.
Oh, totally. I’d be perfectly happy never to see another c extension module ever again – pure python made insanely fast by pypy is what I want!
But we have a real ecosystem, right now.
Maybe I’m being pessimistic, but unless the JIT can also optimize data structures as well as code, it will always be possible to write significantly faster code in C for certain tasks. Using the same algorithm, I can currently write C code that is commonly 3 orders of magnitude or better than the equivalent python code, for math heavy tasks.
There are many things that I, and the C compiler, do to make the code more efficient that the JIT can and does do as well. Things like inlining, branch minimization, less dynamic memory usage, efficient use of registers, etc. But one thing I don’t know how it can do is organize the data in memory efficiently, in a general manner. Utilizing memory in a linear fashion is highly optimized in today’s CPUs, so much so that there are entire units in processors dedicated to this (SIMD). Random access of data, much like branching of code, is far less efficient when that data is too large to remain cached.
Unless it is possible at runtime to optimize data structures as well as code, I doubt that things like numpy, and special-purpose number crunching C extensions, will be going away anytime soon, if ever.
That said, I do enjoy watching the success that pypy has been having, and I can only hope that all those smart folks prove me wrong.
http://pypy.readthedocs.org/en/latest/architecture.html
By those goals it would appear that PyPy is doing quite well.
Like Parrot in many respects, they’ve shifted from simply trying to implement one language to being a “framework” for building dynamic languages. Python just happens to be their benchmark where Perl 6 happens to be Parrot’s.
Quibble: Parrot didn’t shift – it was always envisioned as multilingual. So many other mistakes, but not that one.
Main point (somewhat off topic to OP): Perl 6, as very distinct from Parrot, has been one of very few attempts to date to create a “framework” for language implementation in general, static as well as dynamic. This, at the level of language, not (only) VM.
The story being: pay the painful cost of creating a non-cripplingly powerful and flexible language, compilation, and runtime infrastructure (mostly language – it’s startling how much language choices dial the rest to anywhere from straightforward to intractable) once, thus making it rather easy to implement (most all) other languages using it.
Thus radically changing the pace of language evolution. Instead of “Please, tiny[1] developer community of one of the few variants (PyPy) of one of the few active languages (Python), I’d like you to build me feature X (above wish list), even though you can only share out the cost at a very low level (VM and C/Java/Haskell libraries) and very high level (CS papers), so that I might consider spending a high migration cost to switch to your language and/or variant and join your small community”, instead you could have “hey, I just saw a fun paper on microthreading implementation on modern x86, so I wrote up a new module on the EcumenicalPackageIndex – it should be a seamless drop-under for all languages which have a threading library adaptor to the Threading2 monad api – download and play – feedback encouraged – new rev next week”. Languages and their implementation could be evolving soooo much more rapidly than at present. ([1] Actually, PyPy is perhaps the best-case non-tiny of this class.)
Now, without Perl 6, we seem stuck waiting for Haskell-derived dynamic languages, or a decade plus of sheer human-wave brute-force succession of new languages, and major revs of old ones, until some other language crosses the “I’ll not cripple you before you even get started” critical power threshold.
(Anyone who thinks tech advances rapidly… hasn’t been paying attention.)
It sounds like what you really want is erlang and not python.
I think improving the performance of Python is one of the most important tasks the Python community faces. We’re a general purpose lang popular on the server side, claiming that performance doesn’t matter or won’t make a big impact is just a cop-out.
The only way for PyPy to attract the conservative users you mention is to continue the speed improvements while making it reliable and as compatible with CPython as possible. When it’s a simple, speedier drop-in replacement those users will eventually come. Experimental new features aren’t going to attract conservative users.
So kudos to the PyPy crew for narrowing their focus over the past couple years to work on these goals (particularly CPyExt for drawing those stuck with extensions). Before that they seemed to be a more academicically driven project that produced many experimental features (like sandboxing or the lazily computed thunk objspace), more like you’re encouraging. I don’t want to tell anyone what to do either, but for now I prefer the narrower focus for PyPy of mostly just working at kicking CPython’s butt in perf benchmarks.
I think the recent resurgence in server-side Javascript the past few years is mostly due to their runtime performance gains. You can thank the performance competitions between their major runtimes for that (all runtimes controlled by companies, btw)
All the more serious Javascript runtimes are runtimes dedicated to Javascript (except Rhino, but I don’t think it’s really part of that resurgence). Also the diversity and company control has led Javascript to get better performance (easy to compete on), but painfully slow improvements to the language itself. Some good things there… but not a great model.
I agree with Tom. Personally, I’ve made a few projects at work using a combination of Python and Erlang.
A few points:
Python and Erlang made opposite trade-offs. Erlang had to make some tough concessions to be fault-tolerant and highly concurrent, where ease of binding to C/C++ was one of the things it had to ditch (lest you want to let C/C++ crash the VM, which goes against fault-tolerance). Don’t try to make Python be Erlang. You can’t get there from here. Instead, utilize projects like py-interface and erlport to let each language do what they do best.
It’s suicide to suggest Python (in any form) forsake its library support. CPython’s libraries, both built-in and 3rd party, is the main reason why I use Python at work and not Lisp. :-) If it were just web stuff, I’d use Erlang + Lisp.
Humbly confessing that you are more interested in web programming and a performance profile that matches the problems you solve doesn’t make other people’s requirements go away. I would be extremely disappointed if the PyPy devs took your advice because that would nullify any chance that I could use PyPy. Most of my projects are in industrial automation and such, and I am almost never interested in web apps. If you make a programming language implementation, you can’t be concerned with just one particular developer’s interests.
All that said, there’s no reason why concurrency, etc. shouldn’t be improved. It just shouldn’t be the first priority.
I’ve gotten great results using Python code to implement external processes used by Erlang “port processes”, where the combination of Erlang and Python are better than one language alone. For example, I’m using ctypes and comtypes to talk to COM/DCOM where the reads/writes are from Python behind-the-scenes but Erlang is a distributed front-end that can restart the Python process if it crashes (which can happen sometimes due to ctypes/comtypes and COM/DCOM issues crashing the Python VM).
As someone who uses Python for physics, I can say the only thing stopping me from making PyPy my default interpreter is the lack of compatibility with C extension modules. Numpy alone would cover 20% of my usage, and the rest is an extension module that wraps a large body of C++.
Are you by chance talking about PyROOT? If yes, PyPy has a slow-moving plan to support that and make it really fast: http://morepypy.blogspot.com/2010/07/cern-sprint-report-wrapping-c-libraries.html
Thanks for the advice, but I think it is severely misplaced. We are on the verge of being able to serve the Python community with a faster and in all essentials compatible implementation of Python. This is not the time to go off into research land and solve the problems of concurrency in a multi-cpu environment. This is the time to consolidate and make sure we actually solve people’s problems and to catch up with the development of CPython.
I’m sure the problems of concurrency will still be there when PyPy is mature enough as a platform to be suitable for further wild experiments. It is the Python implementation that is best suited for such work, and probably one of a very few to be considered when all programming languages are taken into account.
As a web developer-administrator, I’d also like to see the features that Ian discussed. FWIW, the C extension modules that I use are MySQLdb, PIL, and PyCrypto. And I could probably do without the latter two in the web application processes themselves by delegating some work to separate Unix processes.
On further consideration, it might be useful to ask why microprocesses are desirable. Why isn’t a pool of Unix processes, each having one thread, good enough?
To be sure, one consideration is memory usage. And a JIT compiler exacerbates this problem. But I wonder if PyPy can adopt any techniques from the Dalvik VM (from Android) to increase the proportion of memory pages that can be shared between processes. Or PyPy could implement the memory dumping technique used by the Emacs compilation procedure to reduce the memory usage of multiple Emacs processes.
Are there any other reasons to favor microprocesses over good old Unix processes?
It’s a question of prioritisation and goal-setting. The pypy+cpyext combination currently has an excellent set of measurable goals to work towards:
For building enthusiasm and motivation, it’s hard to go past a set of concrete goals like that. CPython has nothing that is anywhere near that clear, which is why we have adopted “improve test coverage” as a general “get involved” type goal (it gives people a chance to create patches with a high chance of being accepted, helps flush out lingering bugs in dark corners of the standard library, and generally holds the implementation to a higher standard in the future).
Now, if people specifically wanted to work on the kinds of things that you suggest, then we’d be amongst the first to recommend PyPy as a better platform for experimentation than CPython. However, when we suggest that to people, we currently get pushback on the idea. You know why? Because PyPy isn’t yet a viable drop-in replacement for CPython, so they don’t want to work on an implementation that is PyPy specific.
So even though PyPy is already a much better platform for these ideas than CPython will ever be (e.g. getting rid of refcounting is a prerequisite to basically any interesting copy-on-write based memory savings), the PyPy devs have the right idea in targeting drop-in replacement levels of compatibility. The day that a Linux distro decides to point /usr/bin/python at a PyPy installation rather than a CPython one to provide an immediate performance boost without any additional effort on the part of application developers will be an impressive achievement for all concerned (and my personal assumption regarding that day is that it really is a matter of “when” rather than “if”).
Matt: I fully agree with you. It would be incredibly nice if cPython or PyPy addressed this, and made it possible to share as much as possible (at least in CoW fashion) between processes. Currently, if you have 20 Python processes, each of them imports all the modules on its own heap. This can result in a large amount of memory wasted. On small devices and cheap hosting accounts, this is a killer. I know that dmalcolm did some research in this regard, but he decided to focus on other issues. If anyone is going to work on this, please contact me, I’d like to help. I am not deep enough into Python internals to start on my own, at this moment, but I’m willing to learn :)