Processing Form Data in Shell CGI Scripts

This page presents a little /bin/sh shell script that will help you processing form data in a CGI shell script, without needing C or perl. You receive the form values straight into your shell environment, where you can then access them just like other shell variables. About no special requirements are needed, it just uses the standard Unix utilities dd, test, grep, cut, dc and echo. The latter must be able to print arbitrary octal codes.

Note: Writing this script was interesting, but the program is also kind of a joke. I wanted to prove that it is indeed possible to pick apart CGI data with a shell script, and I succeeded. Yet the script has some deficiencies.
Don't get me wrong. Programming CGI shell scripts isn't a bad idea in general, it indeed works quite well for small jobs. But evaluating the form data shouldn't be done in a script.
One problem is that this script is slow. For biting its way through the data string, it must continuously invoke the afore-mentioned utilities. Hundreds of times. Second, there are a few tiny differences in the tools' implementations on various platforms, so in rare cases, obscure problems may occur.
I never expected the overwhelming feedback on my script. Unfortunately, it didn't always work out perfectly. It is much better having an integrated piece of C code that does the job than a shell script. I have decided to write such a tool, proccgi, to help all folks that were having problems getting the script to work. Unless you know what you're doing, or are looking for fascinating shell scripts, please use that program to evaluate form data in your shell CGI scripts instead, it's much easier to use.
If you are currently using the script on your site, you can simply switch over to proccgi. It works the same, and you only need to change a single line of code in your CGI scripts.
A big thank you to all who have used this shell script in the past, present, and future, and to those who commented on it.
Yet another option is to switch to the Tcl language for CGI scripting. It's powerful and easy to use; I use it for most of my scripting these days. Of course I also have a Tcl script for CGI data evaluation: proccgi.tcl.

Now back to the shell script ...

Features

Handles both GET and POST methods transparently.
Also accepts assignments on the command line (great for testing).
If the extra pathname features assignments (like /A=trash), it is processed as well.
Handles '+' (replaced by space) and '%XX' sequences.
Exports all found variables to the shell environment.
Can also handle multiline "Textarea" input fields.

Bugs

Most of them are shell limitations, and there isn't much I can do to fix them. Should you find a solution, tell me!

All exported variables are case sensitive.
Cannot properly handle newlines (%0A); they are replaced by spaces.
Hex escapes must be uppercase. This should be guaranteed by the remote browser.
Slow.

Usage

After downloading the script (with a name of 'proccgi.sh' or whatever) and installing it (don't forget to grant exec permission), all left to do in your own scripts is to call

eval `proccgi.sh $*`

In some cases, you might need to give the full pathname if it's not found automatically. After this call, you have everything in your shell.

If something goes wrong, you can also enable some debugging output by setting the variable DEBUG to 1, causing a log to be printed to standard error. To get a debugging logfile, you would call

DEBUG=1
export DEBUG
eval `proccgi.sh $* 2> logfile`

Example

This is a very simple example of an automatic software-by-email program. You can fill in your email address and a file name which is then automatically mailed to you. Do not, I repeat, do not install this piece of code. It would be a major security leak.

The Form

<form action="http://our-server/cgi-stuff/mailer" method="post">
<dl>
  <dt> Your Email <dd> <input name="email" size="50">
  <dt> Filename   <dd> <input name="file"  size="50">
</dl>
<input type="submit" value="Submit">
</form>

The Script

#!/bin/sh
eval `proccgi.sh $*`
mail $FORM_email < $FORM_file
cat - << \END
echo Content-type: text/plain
echo
echo done.
END

As you can see, after the call to proccgi.sh, the email address from the form is stored in the shell variable $FORM_email, and the file name is in $FORM_file.

Note

This was just an experiment for me to experience myself what's possible with shell scripts. I know it's not perfect, but I'm missing the time to put much more efforts into it. If you desperately need a feature, read man sh and fix it yourself.

The script may be freely used and distributed at no charge provided that the copyright notice remains at its top.

Script & Code

Don't forget to download the code, proccgi.sh.

Frank Pilhofer <fp -AT- fpx.de> Back to the Homepage

Last modified: Fri Aug 29 15:35:12 1997