[Mail merge icon] Mailmerge CGI Script, v1.1

Abstract

Mailmerge is a Perl CGI script for handling simple HTML forms without doing any actual scripting. You can use it to create user comment forms, surveys, software registration pages, and bug reports.

It's basically an e-mail gateway: you write the form in HTML, users fill it out and submit it, and the contents of the form are then e-mailed to you or to the recipient of your choice. However, unlike other scripts of this sort, Mailmerge gives you control over the way in which the outgoing mail is formatted by letting you supply printmerge-style templates that specify the appearance of both the input form and the outgoing message.

As of version 1.1, mailmerge can also be used to write messages out to a file, so that you can create voting scripts, guestbooks, and so on.

Contents

Installation

Preliminaries

You must have the following installed on your system for this gateway to work:
  1. An HTTP (Web) server that supports CGI scripts.
  2. A sendmail-compatible e-mail agent (this requirement limits this script to servers running on Unix systems by and large).
  3. Perl version 5.001 or higher.

Download the mailmerge package

If you are a Unix user and have access to the tar and compress programs, you can obtain it in one neat package in the mailmerge tar archive.

Otherwise you can download the various components individually. After the text of the scripts appear on your screen, save them to disk.

Install the Files in their Correct Locations

After you untar the packages, you must place the files in their correct locations for mailmerge to work:
mailmerge.cgi
Place this in your scripts directory, usually /cgi-bin. Alternatively, if your server administrator has enabled the recognition of ".cgi" extensions, you can place this script anywhere you like.
CGI.pm
Place this in your perl5 library directory, most commonly /usr/local/lib/perl5. Check with your system administrator if you're not sure.
Send.pm
Place this in your perl5 library directory as well.

Check the Configuration of mailmerge.cgi

There are a few configuration options in mailmerge.cgi that may need to be adjusted for your system. The most important of these is the very top line that reads:
      #!/usr/local/bin/perl
This defines the path to the perl interpreter. Adjust this as necessary and make sure that it points to perl version 5.001 or higher.

The other configuration parameters are at the bottom of mailmerge.cgi following a line that reads __END__. They define default values for the input and output forms, the e-mail headers to use, and such things as the title of the input page, the contact address for the form's author, and formatting preferences. Any of these values can be changed on the fly in the template file (see Using Mailmerge) but you will probably want to set some reasonable defaults here.

Specific values you will probably want to change include:

AUTHOR=Your Name Here
A contact name for your users to use when they encounter problems with the mailmerge gateway (please don't put my name here!)
ADDRESS=<A HREF="/">Your affiliation here</A>
Your affiliation, usually chosen in such a way that it is a pointer back to your site's welcome page.
TO=webmaster
The default recipient of mail sent through this gateway. (Remember that you can always override this on a case-by-base basis in the template files.)
Table of contents

Using Mailmerge

Introduction

The mailmerge program operates on template files that tell it what fill-out form to produce and how to reformat the contents of the form to create an e-mail message or an entry in a file. By specifying different template files, you can produce various types of mail gateways, user surveys, comment forms, guestbooks, etc.

You usually invoke mailmerge by creating hypertext links that look like this:

  <A HREF="/cgi-bin/mailmerge.cgi/path/to/template">
     Fill out the user comment form
  </A>
To understand how this link works, you need to understand a bizarre but very useful feature of links that involve CGI scripts. The first part of the HREF, /cgi-bin/mailmerge.cgi is the path to the mailmerge script on your server. The second part, /path/to/template, is actually additional "path information" that is passed to mailmerge to tell it where it can find the template file to use for formatting an e-mail message.

The additional path information can be interpreted in several ways, allowing you a bit of flexibility when referring to the template. Mailmerge tries all the various interpretations till it finds the template file. For example, let's say you call mailmerge this way:

<A HREF="/cgi-bin/mailmerge.cgi/templates/templ1">
Mailmerge will try to find the template in the following ways:
  1. As a path relative to the document root. If your document root begins at /local/web, then mailmerge will look for the template at /local/web/templates/templ1.
  2. As a physical path on the local file system, i.e. /templates/templ1.
  3. As a path relative to the mailmerge.cgi program, i.e. ./templates/templ1 (which would be a subdirectory of the scripts directory).
Note that you cannot specify a template file relative to the current document (the one that the link to mailmerge is in). This is because the document containing the link could just as easily be on a remote host as on the host that mailmerge runs on, in which case mailmerge would have no way of obtaining its template.

The Contents of a Template

The template contains instructions to mailmerge that tell it what input form to present to the user and how to format the e-mail message or file entry that's generated. Like the configuration files used for HTTP servers, blank lines and lines starting with the hash sign (#) are ignored. A simple template might contain these lines:
   ACTION=Mail

   OUTPUT_FORM=
                      * MEMO *
   Subject:  @SUBJECT@
   To:       @TO@
   From:     @FROM@
   Date:     @DATE@
   Priority: @PRIORITY@

   @BODY@
   .
There are two instructions in this template file. The first instruction, ACTION= tells mailmerge that the message is to be sent out by e-mail. The second instruction defines the output form. The form definition starts with the line OUTPUT_FORM= and ends with a period (.) alone on a line. The text of the output form consists of normal text interspersed with special variables that have the form @VARIABLE@. A variable name is surrounded by "@" signs and can contain any combination of letters, numbers and the underscore (_) character. Case is significant, so @SUBJECT@ and @Subject@ are two different variables.

When mailmerge formats an e-mail message it substitutes appropriate values for the variables:

                      * MEMO *
   Subject:  Release date is close!
   To:       jwilmer@capricorn.org
   From:     abs@zoo.com
   Date:     Sat Aug  5 18:48:14 EDT 1995
   Priority: High

   John,

   The release date is perilously close and marketing is getting
   antsy.  Isn't it time to get the product out of beta test and
   start planning the party?

   -Agatha
Where do the values for the variables come from? They can come from several sources, but the most important source is from named fields in the fill-out form presented to the remote user. One of the ways to specify this form is to define it with a INPUT_FORM= instruction in the template file. Here's an input form that will go well with the memo example:
   INPUT_FORM=
   <H2>Memo</H2>
   To: <INPUT TYPE="text" NAME="TO"><BR>
   From: <INPUT TYPE="text" NAME="FROM"><BR>
   Subject: <INPUT TYPE="text" NAME="SUBJECT">
   Priority: <INPUT TYPE="radio" NAME="PRIORITY" VALUE="High">High
             <INPUT TYPE="radio" NAME="PRIORITY" VALUE="Normal" CHECKED>Normal
   <P>
   <TEXTAREA ROWS=20 COLS=60 NAME="BODY">
   </TEXTAREA>

   <INPUT TYPE="submit" VALUE="Send">
   .
This form defines three one-line text input fields named "TO", "FROM", and "SUBJECT", plus a large 20-line input field named "BODY" and a pair of radio buttons named "PRIORITY". When this form is submitted, mailmerge uses the various named fields to fill in the variables in the output form. Thus the contents of the "TO" field is substituted for the variable @TO@ the contents of the "SUBJECT" field is substituted for the variable @SUBJECT@, and so on.

The INPUT_FORM= definition is written in HTML, using the fill-out form tags. For information on using these tags, see NCSA's tutorial, or my book, How to Set Up and Maintain a World Wide Web Site. Note that should not surround the definition with the <FORM> and </FORM> tags. Mailmerge does this for you. You can also embed a form directly in a larger HTML document and use mailmerge to process it, bypassing the INPUT_FORM= definition. See Embedding mailmerge forms in your own pages for details.

What about the @DATE@ variable? It isn't defined in the input form, so where does its value come from? Many variables are predefined by mailmerge, either as defaults that you can adjust, or calculated by mailmerge on the fly. @DATE@ is one of the predefined variables, and always contains the date and time at which the message was submitted. A full list of the predefined variables is given in Predefined variables.

The ACTION Definition

The ACTION definition tells mailmerge what to do with the formatted output form. Three different values are possible:
ACTION=Mail
Send the formatted form out via e-mail to whatever address is indicated in the definition. Things are arranged so by default this address will be found in the TO variable.
ACTION=Mailto: somebody@somewhere.org
Send the formatted form out via e-mail, overriding the contents of the TO variable and using the indicated address for the recipient. This is useful if you don't want the remote user to be able to change the recipient.
ACTION=File: /path/to/file
Append the output form to the indicated file. You can use a full physical path name, a path name relative to the mailmerge script, or a partial URL. If you refer to a home directory, such as ~bob, it will be interpreted as a URL and look for a file located in ~bob/public_html.

It's important to know that in the interests of security, the file must already be created in order for the script to append new entries to it. It must also be writable by the user ID that the Web server runs under (usually an unprivileged user named "nobody"). You can create an empty starting file and make it writable by the Web server in this way:

       $ touch log_file
       $ chmod o+w log_file
       
The default, if you do not specify an ACTION in your template, is to assume ACTION=Mail. This can be changed by modifying the like-named variable in mailmerge.cgi.

Other Definitions Allowed in the Template File

The MAIL_HEADER definition
In addition to the INPUT_FORM= and OUTPUT_FORM= definitions, you can make a MAIL_HEADER= definition. This is the e-mail header that is used on all outgoing messages. By default the definition looks like this:
   MAIL_HEADER=
   To: @TO@
   From: @FROM@
   Reply-to: @REPLY_TO@
   Cc: @CC@
   Bcc: @BCC@
   Subject: @SUBJECT@
   X-mail-agent: mailmerge v1.0
   .
This header definition assigns special meaning to the following variables:
@TO@
The e-mail address to send the letter to. This can be a comma-separated list of e-mail addresses if you wish to send a letter out to multiple parties.
@FROM@
The return address.
@REPLY_TO@
The reply-to address (usually, but not always the same as From:).
@CC@
Address(es) to send a carbon copy of the letter to.
@BCC@
Address(es) to send a blind carbon copy of the letter to. Unlike the Cc: field, the recipient of the letter will not be notified of the existence of a carbon copy.
@SUBJECT@
The subject of the message.
If you don't like the default e-mail header, you are free to make your own MAIL_HEADER= definition. In fact, the working example at the top of this documentation uses this trick in order to set both the sender and recipient of the message to the same address (see example 3).
Default Definitions
In addition to the three form definitions (INPUT_FORM, OUTPUT_FORM and MAIL_HEADER), you can define default values for variables. These values will be used unless the variables are overridden by the contents of the submitted form. Variable default definitions are single lines in the form
   VARIABLE=VALUE
You can define defaults for variables that have special meaning in the mail header, or create variables of your own choosing to use in the input, output forms and mail headers. For example:
   TO=webmaster@capricorn.org
   FROM=nobody@capricorn.org
   SUBJECT=Feedback on the Web Site
   BCC=fred@zoo.com
   PRIORITY=Normal
These lines define defaults for the @TO@, @FROM@, @SUBJECT@, @BCC@ and @PRIORITY@ variables, and ensure reasonable behavior even when the input form isn't filled out completely. For example, if the "TO" field isn't filled out (or if a "TO" field isn't even provided in the input form), then the default value of webmaster@capricorn.org will be used for the recipient of the e-mail. This is one way to force the e-mail to go to the recipient of your choice.

Mailmerge doesn't yet support multiline variable definitions.

Formatting Directives
Several variables have special meaning to mailmerge and control the way the outgoing message is formatted. These variables are defined in the same way as other variables, with VARIABLE=VALUE lines in the template file.
WORDWRAP=72
When variable substitution causes a line to exceed the specified width, mailmerge will wordwrap its paragraph. Leave the width blank in order to disable word wrapping.
MULTIVALUE_FORMAT=BRACES
When an input form provides the same variable name multiple times, as can happen with scrolling multiple selection lists, checkboxes, and any place where the same name is used for more than one fields, mailmerge has to indicate the multiple values somehow. MULTIVALUE_FORMAT dictates how this is to be done. There are two choices:
BRACES
Multiple values will be surrounded by curly braces like this: {value1,value2,value3}. This is the default.
COMMAS
Multiple values will be inserted into the next in a reasonable approximation of English: value1, value2 and value3.
ESCAPE_HTML=1
If this variable is set to a nonzero value, all special characters in user-provided input, such as angle brackets (>, <), will be escaped using the HTML escape mechanism. This is useful if you are appending the input to an HTML file and you don't want incorrect HTML code provided by the user to break the file. The example guestbook template uses this technique.
TITLE=User Feedback Form
The value of TITLE is used to create the title of the input form page. If not specified it defaults to "Mail Merge Gateway".
AUTHOR=Fred Flintstone
The value of AUTHOR is used to create the <ADDRESS>Author</ADDRESS> line at the bottom of the input form page. You should place your name here, or a symbolic name, such as "webmaster" for users to send comments and complaints to.
ADDRESS=<A HREF="/">Bedrock Home Page</A>
ADDRESS contains a bit of HTML code that gives the author's affiliation. It is usually a link back to the site's welcome page.
How the Defaults are Processed
Ordinarily mailmerge is called twice for every e-mail message that gets sent out. The first time it is called from a link to create the user-visible fill-out form. The second time it is called from the fill-out form to format and send out the e-mail message.

There is a particular order in which mailmerge tries to assign values to the variables. You can take advantage of this order in order to create global defaults, and then override the defaults in individual template files.

Mailmerge first reads its own defaults from the bottom of the mailmerge.cgi file itself. Everything following the line __DATA__ is actually a template that follows all the rules for template files described above. This default template defines reasonably generic values for INPUT_FORM, OUTPUT_FORM and MAIL_HEADER. (If you make a link to mailmerge without specifying any template file, these defaults will be used to give you a "plain vanilla" e-mail gateway.) Mailmerge also calculates many of its built-in variables, such as @DATE@, at this point.

After reading its built-in defaults, mailmerge reads the template file you provide it. Any variables and forms that you redefine in this file replace the defaults.

Finally, if mailmerge is being called to process a fill-out form, the contents of each named field is substituted for the like-named variables.

Built-In Variables
Here is a complete list of built-in variables. They are calculated on the fly every time mailmerge is invoked:
@DATE@
The current local date and time.
@REMOTE_USER@
The name of the remote user. If user authentication is in effect (i.e. mailmerge is protected by user ID/password) then this will be the same as the user ID. Otherwise mailmerge makes a good-faith effort to determine the name by a number of means. However it can only get the user's name if the remote brower is kind enough to provide the information. Many browsers do not do so.
@REMOTE_HOST@
The name of the computer that the remote user is running his/her browser software on. This will be the computer's domain name or its IP address if the name is not available.
@USER_AGENT@
The name of the browser software that the remote user is running.
@REFERER@
The URL of the page the remote user was browsing prior to invoking mailmerge. You can use this to create a back button in your input form:
<A HREF="@REFERER@">Back</A>
This is only supported by a subset of browsers.
@SERVER@
The user name under which the HTTP server is running, often "nobody". You can use this as a default FROM return address.
@SERVER_HOST@
The name of the host the HTTP server is running on.
@ADMINISTRATOR@
The e-mail address of the HTTP administrator, usually something like "webmaster@zoo.org". This variable is only supported under certain servers (such as Apache) that provide this information.
Table of Contents

Embedding Mailmerge Forms in your own Pages

Instead of specifying the fill-out form with an INPUT_FORM definition in the template file, you can place the form directly into your own HTML page and arrange to have mailmerge process it when the submit button is pressed. You should point the <FORM> tag's ACTION attribute to the mailmerge script, and set METHOD to POST (this is very important). Follow this example:
   <FORM ACTION="/cgi-bin/mailmerge.cgi/path/to/template" METHOD=POST>
      (your form here)
   </FORM>
Remember to specify a template file to use. Although the INPUT_FORM definition, if any, will be ignored, mailmerge still needs to read the OUTPUT_FORM definition to format the contents of the fields you declare in the fill-out form.

Table of Contents

Tips

  1. You can create default values for text fields in the INPUT_FORM by incorporating a variable directly into the field definition. For example, if you define COUNTRY=USA in the template, then the string "USA" will appear by default if this line appears in the INPUT_FORM definition:
    <INPUT TYPE="text" NAME="COUNTRY" VALUE="@COUNTRY@">
  2. If you want to limit the possible recipients of e-mail from the mailmerge gateway to a few alternatives, it's useful to create a popup menu for the TO field. You can do so with lines like these in the INPUT_FORM definition:
             <SELECT NAME="TO">
             <OPTION>fred@bedrock.com
             <OPTION>wilma@bedrock.com
             <OPTION>webmaster@capricorn.org
             </SELECT>
           
  3. If you want to limit the recipient of e-mail to a single or fixed list of individuals, define a TO= variable in your template file and avoid creating any TO fields in the INPUT_FORM. To be absolutely certain that the e-mail can only be sent to the recipient(s) of your choice, you can redefine the MAIL_HEADER in this way:
             MAIL_HEADER=
             To: fred@bedrock.com (Fred Flintstone)
             From: @FROM@
             Subject: @SUBJECT@
             Date: @DATE@
             .
  4. A useful default return address to use for the FROM variable is @REMOTE_USER@@@REMOTE_HOST@. This allows you to create forms that incorporate the user's name and host in the return address field (e.g. fred@bedrock.com). See example 4 (try it).
  5. You can incorporate inline graphics into the INPUT_FORM just by placing a <SRC> there. In fact, you are free to incorporate any HTML tags you choose, including hypertext links and Netscape/HTML3 tables.
Table of Contents

Common Problems

When I select a link to mailmerge, I get a "Server Misconfiguration" error. The mysterious words "Malformed line from script" appear in the server error log.
This happens when the script is not installed correctly and is producing an error message. The error message usually appears in the server error log. Check again to see if there isn't a message there. Things to check include:
  1. Does the topmost line of mailmerge.cgi read "#!/usr/local/bin/perl", and does this path point to version 5 of the perl interpreter?
  2. Is CGI.pm installed in the perl library directory, usually /usr/local/lib/perl5?
  3. Does your system use sendmail, and is this program located in the usual location of /usr/lib/sendmail?
  4. Is the mailmerge.cgi script readable and executable by the user that the HTTP server runs under (usually "nobody").

The input form only displays partially and then terminates.
This usually means that you've made a mistake in the HTML code in the template's INPUT_FORM definition. Remember that you shoudn't put the <FORM> or </FORM> tags in the definition. Mailmerge does this for you.

I don't see my custom fill-out form. I always get mailmerge's default generic form!
This can happen if the path to the template isn't correct. Remember that the path to the template begins after immediately after the name of the script and can refer to a physical path, a virtual path relative to the HTTP server's document root, or a physical path relative to the mailmerge script's location. Also remember that the template file must be readable by the user that the HTTP server runs under.

Everything appears to work, but the mail never gets sent out!
Your system probably doesn't have sendmail installed. Mailmerge can be modified easily to handle other mail agents, but I didn't have any handy to test out. See your system or network administrator for help with this one. E-mail problems can be hard to sort out, but here's one hint: it usually pays to examine the e-mail log files.
I'm using ACTION=File:, but nothing's getting written!
Make sure that the file is writable by the Web server programmer. This usually means that the file must be "world" writable. Check the server's error log file, there may be an error message there giving you more details about the problem.
Table of Contents

Usage Terms

This script is provided free to the Web community. Although it is not in the public domain, you may use and redistribute it freely so long as the copyright statement in the source code remains intact. You may modify it and borrow code and ideas from it, and but I ask that I be credited in any derivatives.

©1995, Lincoln D. Stein, All Rights Reserved

You're encouraged to send me questions and bug reports. My address is given at the bottom of this documentat.

Latest Version of the Software

The latest version of this software can be found at http://www-genome.wi.mit.edu/ftp/pub/software/WWW/mailmerge/

You'll find pointers to other CGI scripts on my home page and at How to Setup and Maintain a World Wide Web Site.

Table of Contents


Lincoln D. Stein, lstein@genome.wi.mit.edu
Whitehead Institute/MIT Center for Genome Research