Ian Bicking: the old part of his blog


I thought I was all clever because I figured out a good technique of making configuration files executable with #!. But apparently I was not so clever.

On FreeBSD I got this working nicely:

#!/usr/bin/env paster exe serve

The exe told paster that it should expect the shebang setup (config file follows, etc). And while putting paster itself as the command didn't work, /usr/bin/env parsed everything nicely for me (and since paster itself uses a shebang line to be executable, only /usr/bin/env was willing to recursively interpret that).

paster would then end up seeing an argv like ['paster', 'exe', 'serve', config_file, extra_args...] where extra_args were any arguments you passed on the command line. The environmental variable _ would be set to config_file.

Usually something that seems sensible on FreeBSD will work on Linux; in comparison to BSD tools, the GNU tools (standard on Linux) generally favor sensibility over backward compatibility, and FreeBSD seems very sensible in this case. I could only expect more sensibility on Linux, right?

Sadly, no. /usr/bin/env parses that as a request to run "paster exe serve" -- i.e., a file with embedded spaces. Sigh. It does still set _, and passes extra arguments. But I still want to know when I'm in a shebang environment, so paster can respond accordingly, and I'm not sure how to detect that. When environ['_'] != sys.argv[0]? Is that reliable? Is there another way to detect this? Any hints for MacOS (like FreeBSD?) and/or Solaris (like stupid Linux?)

Created 07 Nov '05


See http://svn.eby-sarna.com/PEAK/INSTALL.txt?view=markup under the heading "SCRIPTS, BATCH FILES, AND '#!'":

invoke allows an arbitrary number of space-separated arguments to be passed to the command it invokes, thus working around various Unixes' #! parsing problems, as well as the "can't use a script as an interpreter" problem. It also searches the system PATH for the specified command. You may find this useful for non-PEAK script interpreters as well.

In short, you can use PEAK's "invoke" command (a simple C program) to work around this and various other cross-platform #! issues. PEAK has been doing executable config files for rather a long time, so we've run into this before.

# Phillip J. Eby

It sounds like invoke works like /usr/bin/env does on FreeBSD (but not everywhere). That's nice; though the one issue I see is that /usr/bin/env is pre-installed into that exact location on most modern systems (and if it isn't, then the sysadmin should fix that, because it's a common idiom). If I can live with env's lowest-common denominator (which maybe I can), then it seems preferable. I already support moving the server argument into the main configuration file, so if exe isn't necessary then I should be set.

Incidentally, I probably thought about adding this feature of executable configuration files to paster because I'd seen it PEAK.

# Ian Bicking

It's not env that's the problem. It's the operating system. Linux is passing only one argument to env, so even if you used the exact same env program, it would make no difference.

# Phillip J. Eby

Then env should have fixed that by splitting the arguments itself! Especially since the env installed by the system should already be customized for the system (and whatever flaws it might have).

# Ian Bicking

Um, no. That's not what env is even for. Env is a command to "run a program in a modified environment". The fact that env is often used in #! lines as a convenient hack to make #! pay attention to PATH has nothing to do with env's specified and intended behavior. If env were to do what you're suggesting, it would be violating its own documentation and the normal argument processing conventions for Unix commands. See also "man env".

Also, see http://homepages.cwi.nl/~aeb/std/shebang/ for information on various platforms' #! handling, including some test code you can use to find out what particular way(s) a given platform is borked.

# Phillip J. Eby

In my recollection, older Solaris systems had /bin/env rather than /usr/bin/env. This was a simple matter to fix with a symlink, but it limited script portability.

# Jeff Bauer