Issue 4413 - Merge formmail.py and formmail2.py

sitescripts/formmail/README.md

 # formmail
 
-This script generates a form and sends an email.? It provides the following URLs:

[Vasily Kuznetsov, 2019/01/18 16:52:13]

The script doesn't generate the form, it only accepts the POST request that
comes when it's submitted, plugs the values from the request into an email
template and sends it to the configured address. I guess this introduction could
read something like:

    The web handler that extracts form data from a POST request, uses it to
    populate an email template and then sends the produced email to a
    configured list of addresses.

I think we probably shouldn't talk about provided URL(s) in this section because
it all depends on the configuration.

[rhowell, 2019/01/18 20:08:07]

Done.

-
-* */?
+The web handler that extracts form data from a POST request, uses it to
+populate an email template and then sends the produced email to a configured
+list of addresses.
 
 ## Dependencies
 
 * [Python 2.7](https://www.python.org/download/releases/2.7/)
-* [mock](https://pypi.python.org/pypi/mock) (Only for the tests)
-* _Anthing else?_

[Vasily Kuznetsov, 2019/01/18 16:52:13]

The handler also uses Jinja2 for rendering the template. I'm not sure if it
makes much sense to mention testing dependencies here -- there are many and they
are listed in tox.ini, it's probably better to just refer to it instead.

[rhowell, 2019/01/18 20:08:07]

Done.

+* [Jinja2](http://jinja.pocoo.org/docs/2.10/intro/)
+* Other packages are required for testing, please see the list of 'deps' in
+  [`tox.ini`](../../tox.ini)
 
-## Running the script
+## Running the web handler
 
-The script can be run as a module, using the following command:

[Vasily Kuznetsov, 2019/01/18 16:52:13]

formmail.py is not really a standalong script, it's a web handler that is
normally run via the multiplexer (multiplexer.py or multiplexer.fcgi at the
root, they are described in the main README file). I'd say we mention this here,
refer to the multiplexer section in the main README and explain how to activate
the handler (can use the text from the section about mutliplexer config below
from lines 28-32).

[rhowell, 2019/01/18 20:08:07]

Done.

+Normally, the formmail web handler is run by the multiplexer, and configured
+via the sitescripts config file. Please refer to the main
+[README](../../README.md) for more information about the multiplexer and
+configuring `sitescripts.ini`.
 
-```commandline
-    sitescripts.formmail.web.formmail.sendMail
-```
-
-The script can also be run directly from the file, using:
-```commandline
-    python sitescripts/formmail/web/formmail.py
-```
-
-## Configuration
-
-The handler is configured via sitescripts config. In order to activate this
-handler, add the following line to multiplexer config:
+In order to activate this handler, add the following line to the multiplexer
+config file:
 
     [multiplexer]

[Vasily Kuznetsov, 2019/01/18 16:52:13]

This part should probably be in # Running the web handler, because this is about
configuring the multiplexer.

[rhowell, 2019/01/18 20:08:07]

Done.

     sitescripts.formmail.web.formmail =
 
-Its own configuration is in the section `formmail`:
+## Configuring the web handler
+
+`formmail.py` can handle multiple URLs and forms. Each URL will correspond to a
+group of config variables that all start with the same prefix, for example:
+handler1. These variables are configured in the [formmail] section of the
+config file.
+
+The URL of the form, where the POST request comes from:
 
     [formmail]

[Vasily Kuznetsov, 2019/01/18 16:52:13]

This part will need more explanation. I think we should start by saying that
formmail.py can handle multiple URLs and  forms and that each URL will
correspond to a group of config variables that all start with the same prefix,
for example: handler1.

Then we can go over each variable, starting from `handler1.url` (the URL where
the form should be submitted), `handler1.csv_log` (CSV file into which all
submissions will be saved, optional), `handler1.template` (Jinja2 template for
the email including the headers, such as From: and To:), and
`handler1.fields.xxx` (a subgroup that includes the descriptions of the form
fields).

I think you can draft the text even better then I would. Feel free to ping be by
IRC if you have questions about the config (but don't forget there are examples
and the source ;).

[rhowell, 2019/01/18 20:08:07]

I'm still not exactly clear what the `.fields` section refers to, but maybe
whoever is configuring the server would understand it more clearly. Does it
change the behavior of the actual form? Or send an error response to the POST
request if something mandatory is missing?

[Vasily Kuznetsov, 2019/01/21 13:54:17]

The form itself is usually another page, so its behavior is unchanged by this
config but the two have to match. Also the handler can return error messages
that depend on the configuration here and can be further changed by it. I think
we should document all of it here, otherwise the people configuring the server
would have to ask.

The ".fields" group describes the fields that are expected in the POST request
and then become available to the template. Each variable in the group defines a
field and its value can contain "mandatory" (which makes the field mandatory)
and/or "email" (which makes the field an email) or it can be empty (just a
normal optional field).

For mandatory fields we can also define "group-name.fields.field-name.mandatory"
to override the error message that will be returned by the handler if the field
was empty. Likewise for email fields we can define
"group-name.fields.field-name.email" to set the error message that's returned if
the content of the field doesn't look like an email.

This is not the most elegant way to configure forms, but it's simple and
flexible enough. See validate_fields() at
https://hg.adblockplus.org/sitescripts/file/tip/sitescripts/formmail/web/form...
 for more insight on how this works.

[rhowell, 2019/01/22 22:53:54]

Ahh, gotcha. Makes sense. Thank you for the very clear explanation, I used most
of it.

-    test.csv_log = /var/log/something.csv_log
-    test.url=test/apply/submit
-    test.template=formmail/test/template/test.mail
-    test.fields.email=mandatory, email
-    test.fields.email.mandatory=You failed the email test
-    test.fields.email.email=You failed the email validation
-    test.fields.non_mandatory_email=email
-    test.fields.non_mandatory_message=
-    test.fields.mandatory=mandatory
+    handler1.url = formmail/test/apply/submit
+
+The CSV file into which all submissions will be saved (optional):
+
+    handler1.csv_log = /var/log/handler1-log.csv
+
+The Jinja2 template for the email. This is where the recipient email addresses
+are entered.
+[(See an example email template here.)](formmail/test/template/test.mail)
+
+    handler1.template = formmail/handler1/mail-template.tmpl
+
+The `handler1.fields.xxx` subgroup includes the descriptions of the form
+fields, and these must match the fields on the form. These are the fields
+expected in the POST request and then made available to the template. Each
+variable in the group defines a field and its value can be:
+* "mandatory" (which makes the field mandatory)
+* and/or "email" (which makes the field an email)
+* or it can be empty (just a normal optional field).
+
+For mandatory fields we can also set "group-name.fields.field-name.mandatory"
+to override the error message that will be returned by the handler if the field
+was empty. Likewise for email fields we can define
+"group-name.fields.field-name.email" to set the error message that's returned
+if the content of the field doesn't look like an email. See an example:
+
+    handler1.fields.email = mandatory, email
+    handler1.fields.email.mandatory = You failed the email test
+    handler1.fields.email.email = You failed the email validation
+    handler1.fields.non_mandatory_email = email
+    handler1.fields.non_mandatory_message =
+    handler1.fields.mandatory = mandatory