This is mailfromd.info, produced by makeinfo version 6.7 from mailfromd.texi. Published by the Free Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Copyright (C) 2005-2020 Sergey Poznyakoff Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". INFO-DIR-SECTION Email START-INFO-DIR-ENTRY * Mailfromd: (mailfromd). General-purpose mail-filtering software. * mailfromd: (mailfromd) Invocation. Mail Filtering and Real-time Modification daemon. * calloutd: (mailfromd) calloutd. A Stand-Alone Callout Daemon. * mfdbtool: (mailfromd) mfdbtool. Database Management Tool. * mtasim: (mailfromd) mtasim. MTA simulator. * pmult: (mailfromd) pmult. Pmilter multiplexer program. END-INFO-DIR-ENTRY Dedico aquest treball a Lluis Llach, per obrir els nous horitzons.  File: mailfromd.info, Node: EOM Functions, Next: Current Message Functions, Prev: Mail body functions, Up: Library 5.12 EOM Functions ================== The following function is available only in the 'eom' handler: -- Built-in Function: void progress () Notify the MTA that the filter is still processing the message. This causes MTA to restart its timeouts and allows additional amount of time for execution of 'eom'. Use this function if your 'eom' handler needs additional time for processing the message (e.g. for scanning a very big MIME message). You may call it several times, if the need be, although such usage is not recommended.  File: mailfromd.info, Node: Current Message Functions, Next: Mailbox functions, Prev: EOM Functions, Up: Library 5.13 Current Message Functions ============================== -- Built-in Function: number current_message () This function can be used in 'eom' handlers only. It returns a message descriptor referring to the current message. *Note Message functions::, for a description of functions for accessing messages. The functions below access the headers from the current message. They are available in the following handlers: 'eoh', 'body', 'eom'. -- Built-in Function: number current_header_count ([string NAME]) Return number of headers in the current message. If NAME is specified, return number of headers that have this name. current_header_count() => 6 current_header_count("Subject") => 1 -- Built-in Function: string current_header_nth_name (number N) Return the name of the Nth header. The index N is 1-based. -- Built-in Function: string current_header_nth_value (number N) Return the value of the Nth header. The index N is 1-based. -- Built-in Function: string current_header (string name [, number N]) Return the value of the named header, e.g.: set s current_header("Subject") Optional second argument specifies the header instance, if there are more than 1 header of the same name, e.g.: set s current_header("Received", 2) Header indices are 1-based. All current_header function raise the 'e_not_found' exception if the requested header was not found.  File: mailfromd.info, Node: Mailbox functions, Next: Message functions, Prev: Current Message Functions, Up: Library 5.14 Mailbox Functions ====================== A set of functions is provided for accessing mailboxes and messages within them. In this subsection we describe the functions for accessing mailboxes. A mailbox is opened using 'mailbox_open' function: -- Built-in Function: number mailbox_open (string URL [, string MODE, string PERMS]) Open a mailbox identified by URL. Return a "mailbox descriptor": a unique numeric identifier that can subsequently be used to access this mailbox. The optional MODE argument specifies the access mode for the mailbox. Its valid values are: Value Meaning ------------------------------------------------------------------ r Open mailbox for reading. This is the default. w Open mailbox for writing. If the mailbox does not exist, it is created. rw Open mailbox for reading and writing. If the mailbox does not exist, it is created. wr Same as 'rw'. w+ Open mailbox for reading and writing. If the mailbox does not exist, it is created. a Open mailbox for appending messages to it. If the mailbox does not exist, an exception is signalled. a+ Open mailbox for appending messages to it. If the mailbox does not exist, it is created. The optional PERMS argument specifies the permissions to use in case a new file (or files) is created. It is a comma-separated list of: [go](+|=)[wr]+ The initial letter controls which users' access is to be set: users in the file's group ('g') or other users not in the file's group ('o'). The following character controls whether the permissions are added to the default ones ('+') or applied instead of them ('='). The remaining letters specify the permissions: 'r' for read access and 'w' for write access. For example: g=rw,o+r The number of mailbox descriptors available for simultaneous opening is 64. This value can be changed using the 'max-open-mailboxes' runtime configuration statement (*note max-open-mailboxes: conf-runtime.). -- Built-in Function: number mailbox_messages_count (number NMBX) Return the number of messages in mailbox. The argument NMBX is a valid mailbox descriptor as returned by a previous call to 'mailbox_open'. -- Built-in Function: number mailbox_get_message (number MBX, number N) Retrieve Nth message from the mailbox identified by descriptor MBX. On success, the function returns a "message descriptor", an integer number that can subsequently be used to access that message (*note Message functions::). On error, an exception is raised. Messages in a mailbox are numbered starting from 1. -- Built-in Function: void mailbox_close (number NMBX) Close a mailbox previously opened by 'mailbox_open'. -- Built-in Function: void mailbox_append_message (number NMBX, number NMSG) Append message NMSG to mailbox NMBX. The message descriptor NSMG must be obtained from a previous call to 'mailbox_get_message' or 'current_message' (*note current_message::).  File: mailfromd.info, Node: Message functions, Next: Quarantine functions, Prev: Mailbox functions, Up: Library 5.15 Message Functions ====================== The functions described below retrieve information from RFC822 messages. The message to operate upon is identified by its "descriptor", an integer number returned by the previous call to 'mailbox_get_message' (*note mailbox_get_message: Mailbox functions.) or 'current_message' (*note current_message::) function. The maximum number of message descriptors is limited by 1024. You can change this limit using the 'max-open-messages' runtime configuration statement (*note max-open-messages: conf-runtime.). -- Built-in Function: number message_size (number NMSG) Return the size of the message NMSG, in bytes. _Notice_, that if NMSG refers to current message (*note current_message::), the returned value is less than the size seen by the MTA, because 'mailfromd' recodes CR-LF sequences to LF, i.e. removes carriage returns (ASCII 13) occurring before line feeds (ASCII 10. To obtain actual message length as seen by the MTA, add the number of lines in the message: set actual_length message_size(nmsg) + message_lines(nmsg) -- Built-in Function: boolean message_body_is_empty (number NMSG) Returns 'true' if the body of message NMSG has zero size or contains only whitespace characters. If the 'Content-Transfer-Encoding' header is present, it is used to decode body before processing. -- Built-in Function: void message_close (number NMSG) Close the message identified by descriptor NMSG. -- Built-in Function: number message_lines (number NMSG) Return total number of lines in message NMSG. The following relation holds true: message_lines(X) = message_body_lines(X) + message_header_lines(X) + 1 -- Built-in Function: string message_read_line (number NMSG) Read and return next line from the message NMSG. If there are no more lines to read, raise the 'eof' exception. Use 'message_rewind' to rewind the message stream and read its contents again. -- Built-in Function: void message_rewind (number NMSG) Rewind the stream associated with message referred to by descriptor NMSG. -- Built-in Function: number message_from_stream (number FD; string FILTER_CHAIN) Converts contents of the stream identified by FD to a mail message. Returns identifier of the created message. Optional FILTER_CHAIN supplies the name of a "Mailutils filter chain", through which the data will be passed before converting. See , for a description of filter chains. -- Built-in Function: void message_to_stream (number FD, number NMSG; string FILTER_CHAIN) Copies message NSMG to stream descriptor FD. The descriptor must be obtained by a previous call to 'open'. Optional FILTER_CHAIN supplies the name of a "Mailutils filter chain", through which the data will be passed before writing them to FD. See , for a description of filter chains. * Menu: * Header functions:: * Message body functions:: * MIME functions:: * Message digest functions::  File: mailfromd.info, Node: Header functions, Next: Message body functions, Up: Message functions 5.15.1 Header functions ----------------------- -- Built-in Function: number message_header_size (number NMSG) Return the size, in bytes of the headers of message NMSG. See the note to the 'message_size', above. -- Built-in Function: number message_header_lines (number NMSG) Return number of lines occupied by headers in message NMSG. -- Built-in Function: number message_header_count (number NMSG, [string NAME]) Return number of headers in message NMSG. If NAME is supplied, count only headers with that name. -- Built-in Function: string message_find_header (number NMSG, string NAME [, number IDX]) Return value of header NAME from the message NMSG. If the message contains several headers with the same name, optional parameter IDX may be used to select one of them. Headers are numbered from '1'. If no matching header is not found, the 'not_found' exception is raised. If another error occurs, the 'failure' exception is raised. The returned string is a verbatim copy of the message contents (except for eventual CR-LF -> LF translation, see above). You might need to apply the 'unfold' function to it (*note unfold: Mail header functions.). -- Built-in Function: string message_nth_header_name (number NMSG, number N) Returns the name of the Nth header in message NMSG. If there is no such header, 'e_range' exception is raised. -- Built-in Function: string message_nth_header_value (number MSG, number N) Returns the value of the Nth header in message NMSG. If there is no such header, 'e_range' exception is raised. -- Built-in Function: boolean message_has_header (number NMSG, string NAME [, number IDX]) Return 'true' if message NMSG contains header with the given NAME. If there are several headers with the same name, optional parameter IDX may be used to select one of them.  File: mailfromd.info, Node: Message body functions, Next: MIME functions, Prev: Header functions, Up: Message functions 5.15.2 Message body functions ----------------------------- -- Built-in Function: number message_body_size (number NMSG) Return the size, in bytes, of the body of message NMSG. See the note to the 'message_size', above. -- Built-in Function: number message_body_lines (number NMSG) Return number of lines in the body of message referred to by descriptor NMSG. -- Built-in Function: void message_body_rewind (number NMSG) Rewind the stream associated with the body of message referred to by descriptor NMSG. A call to 'message_body_read_line' (see below) after calling this function will return the first line from the message body. -- Built-in Function: string message_read_body_line (number NMSG) Read and return next line from the body of the message NMSG. If there are no more lines to read, raise the 'eof' exception. Use 'message_body_rewind' (see above) to rewind the body stream and read its contents again. -- Built-in Function: void message_body_to_stream (number FD, number NMSG; string FILTER_CHAIN) Copies the body of the message NSMG to stream descriptor FD. The descriptor must be obtained by a previous call to 'open'. Optional FILTER_CHAIN supplies the name of a "Mailutils filter chain", through which the data will be passed before writing them to FD. oSee , for a description of filter chains.  File: mailfromd.info, Node: MIME functions, Next: Message digest functions, Prev: Message body functions, Up: Message functions 5.15.3 MIME functions --------------------- -- Built-in Function: boolean message_is_multipart (number NMSG) Return 'true' if message NMSG is a multipart (MIME) message. -- Built-in Function: number message_count_parts (number NMSG) Return number of parts in message NMSG, if it is a multipart (MIME) message. If it is not, return '1'. Use 'message_is_multipart' to check whether the message is a multipart one. -- Built-in Function: number message_get_part (number nmsg, number N) Extract Nth part from the multipart message NMSG. Numeration of parts begins from '1'. Return message descriptor referring to the extracted part. Message parts are regarded as messages, so any message functions can be applied to them.  File: mailfromd.info, Node: Message digest functions, Prev: MIME functions, Up: Message functions 5.15.4 Message digest functions ------------------------------- "Message digests" are specially formatted messages that contain certain number of mail messages, encapsulated using the method described in RFC 934. Such digests are often used in mailing lists to reduce the frequency of sending mails. Messages of this format are also produced by the "forward" function in most MUA's. The usual way to handle a message digest in MFL is to convert it first to a MIME message, and then to use functions for accessing its parts (*note MIME functions::). -- Built-in Function: number message_burst (number NMSG ; number FLAGS) Converts the message identified by the descriptor NMSG to a multi-part message. Returns a descriptor of the created message. Optional argument FLAGS controls the behavior of the bursting agent. It is a bitwise OR of error action and bursting flags. "Error action" defines what to do if a part of the digest is not in RFC822 message format. If it is 'BURST_ERR_FAIL' (the default), the function will raise the 'e_format' exception. If ONERR is 'BURST_ERR_IGNORE', the improperly formatted part will be ignored. Finally, the value 'BURST_ERR_BODY' instructs 'message_burst' to create a replacement part with empty headers and the text of the offending part as its body. "Bursting flags" control various aspects of the agent behavior. Currently only one flag is defined, 'BURST_DECODE', which instructs the agent to decode any MIME parts (according to the 'Content-Transfer-Encoding' header) it encounters while bursting the message. Parts of a message digest are separated by so-called "encapsulation boundaries", which are in essence lines beginning with at least one dash followed by a non-whitespace character. A dash followed by a whitespace serves as a "byte-stuffing" character, a sort of escape for lines which begin with a dash themselves. Unfortunately, there are mail agents which do not follow byte-stuffing rules and pass lines beginning with dashes unmodified into resulting digests. To help handle such cases a global variable is provided which controls how much dashes should the line begin with for it to be recognized as an encapsulation boundary. -- Built-in variable: number burst_eb_min_length Minimal number of consecutive dashes an encapsulation boundary must begin with. The default is 2. The following example shows a function which saves all parts of a digest message to separate disk files. The argument ORIG is a message descriptor. The resulting files are named by concatenating the string supplied by the STEM argument and the ordinal number (1-based) of the message part. func burst_digest(number orig, string stem) do number msg message_burst(orig) number nparts message_count_parts(msg) loop for number i 1, while i <= nparts, set i i + 1 do number part message_get_part(msg, i) number out open(sprintf('>%s%02d', stem, i)) message_to_stream(out, part) done message_close(msg) done  File: mailfromd.info, Node: Quarantine functions, Next: SMTP Callout functions, Prev: Message functions, Up: Library 5.16 Quarantine Functions ========================= -- Built-in Function: void quarantine (string TEXT) Place the message to the quarantine queue, using TEXT as explanatory reason.  File: mailfromd.info, Node: SMTP Callout functions, Next: Compatibility Callout functions, Prev: Quarantine functions, Up: Library 5.17 SMTP Callout Functions =========================== -- Library Function: number callout_open (string URL) Opens connection to the callout server listening at URL. Returns the descriptor of the connection. -- Library Function: void callout_close (number FD) Closes the connection. FD is the file descriptor returned by the previous call to 'callout_open'. -- Library Function: number callout_do (number FD, string EMAIL [, string REST]) Instructs the callout server identified by FD (a file descriptor returned by a previous call to 'callout_open') to verify the validity of the EMAIL. Optional REST argument supplies additional parameters for the server. Possible return values: '0' Success. The EMAIL is found to be valid. 'e_not_found' EMAIL does not exist. 'e_temp_failure' The email validity cannot be determined right now, e.g. because remote SMTP server returned temporary failure. The caller should retry verification later. 'e_failure' Some error occurred. The function will throw the 'e_callout_proto' exception if the remote host doesn't speak the correct callout protocol. Upon return, 'callout_do' modifies the following variables: 'last_poll_host' Host name or IP address of the last polled SMTP server. 'last_poll_greeting' Initial SMTP reply from the last polled host. 'last_poll_helo' The reply to the 'HELO' ('EHLO') command, received from the last polled host. 'last_poll_sent' Last SMTP command sent to the polled host. If nothing was sent, 'last_poll_sent' contains the string 'nothing'. 'last_poll_recv' Last SMTP reply received from the remote host. In case of multi-line replies, only the first line is stored. If nothing was received the variable contains the string 'nothing'. The "default callout server" is defined by the 'callout-url' statement in the configuration file, or by the 'callout' statement in the 'server milter' section (*note configuring default callout server::. The following functions operate on that server. -- Built-in Function: string default_callout_server_url () Returns URL of the default callout server. -- Library Function: number callout (string EMAIL) Verifies the validity of the EMAIL using the default callout server.  File: mailfromd.info, Node: Compatibility Callout functions, Next: Internet address manipulation functions, Prev: SMTP Callout functions, Up: Library 5.18 Compatibility Callout Functions ==================================== The following functions are wrappers over the callout functions described in the previous section. They are provided for backward compativbility. These functions are defined in the module 'poll.mf', which you must require prior to using any of them. -- Library Function: boolean _pollhost (string IP, string EMAIL, string DOMAIN, string MAILFROM) Poll SMTP host IP for email address EMAIL, using DOMAIN as 'EHLO' domain and MAILFROM as 'MAIL FROM'. Returns 0 or 1 depending on the result of the test. In contrast to the 'strictpoll' function, this function does not use cache database and does not fall back to polling MX servers if the main poll tempfails. The function can throw one of the following exceptions: 'e_failure', 'e_temp_failure'. -- Library Function: boolean _pollmx (string IP, string EMAIL, string DOMAIN, string MAILFROM) Poll MXs of the DOMAIN for email address EMAIL, using DOMAIN as 'EHLO' domain and MAILFROM as 'MAIL FROM' address. Returns 0 or 1 depending on the result of the test. In contrast to the 'stdpoll' function, '_pollmx' does not use cache database and does not fall back to polling the IP if the poll fails. The function can throw one of the following exceptions: 'e_failure', 'e_temp_failure'. -- Library Function: boolean stdpoll (string EMAIL, string DOMAIN, string MAILFROM) Performs standard poll for EMAIL, using DOMAIN as 'EHLO' domain and MAILFROM as 'MAIL FROM' address. Returns 0 or 1 depending on the result of the test. Can raise one of the following exceptions: 'e_failure', 'e_temp_failure'. In 'on' statement context, it is synonymous to 'poll' without explicit HOST. -- Library Function: boolean strictpoll (string HOST, string EMAIL, string DOMAIN, string MAILFROM) Performs strict poll for EMAIL on host HOST. See the description of 'stdpoll' for the detailed information. In 'on' context, it is synonymous to 'poll host HOST'. The MAILFROM argument can be a comma-separated list of email addresses, which can be useful for servers that are unusually picky about sender addresses. It is advised, however, that this list always contain the '<>' address. For example: _pollhost($client_addr, $f, "domain", "postmaster@my.net,<>") See also *note mail-from-address: conf-callout. Before returning, all described functions set the following built-in variables: Variable Contains -------------------------------------------------------------------------- last_poll_host Host name or IP address of the last polled host. last_poll_sent Last SMTP command, sent to this host. If nothing was sent, it contains literal string 'nothing'. last_poll_recv Last SMTP reply received from this host. In case of multi-line replies, only the first line is stored. If nothing was received the variable contains the string 'nothing'. cache_used '1' if cached data were used instead of polling, '0' otherwise. This variable is set by 'stdpoll' and 'strictpoll'. If it equals '1', none of the above variables are modified. *Note cache_used example::, for an example. Table 5.1: Variables set by polling functions  File: mailfromd.info, Node: Internet address manipulation functions, Next: DNS functions, Prev: Compatibility Callout functions, Up: Library 5.19 Internet address manipulation functions ============================================ Following functions operate on IPv4 addresses and CIDRs. -- Built-in Function: number ntohl (number N) Converts the number N, from host to network byte order. The argument N is treated as an unsigned 32-bit number. -- Built-in Function: number htonl (number N) Converts the number N, from network to host byte order. The argument N is treated as an unsigned 32-bit number. -- Built-in Function: number ntohs (number N) The argument N is treated as an unsigned 16-bit number. The function converts this number from network to host order. -- Built-in Function: number htons (number N) The argument N is treated as an unsigned 16-bit number. The function converts this number from host to network order. -- Built-in Function: number inet_aton (string S) Converts the Internet host address S from the standard numbers-and-dots notation into the equivalent integer in host byte order. inet_aton("127.0.0.1") => 2130706433 _The numeric data type in MFL is signed, therefore on machines with 32 bit integers, this conversion can result in a negative number:_ inet_aton("255.255.255.255") => -1 _However, this does not affect arithmetical operations on IP addresses._ -- Built-in Function: string inet_ntoa (number N) Converts the Internet host address N, given in host byte order to string in standard numbers-and-dots notation: inet_ntoa(2130706433) => "127.0.0.1" -- Built-in Function: number len_to_netmask (number N) Convert number of masked bits N to IPv4 netmask: inet_ntoa(len_to_netmask(24)) => 255.255.255.0 inet_ntoa(len_to_netmask(7)) => 254.0.0.0 If N is greater than 32 the function raises 'e_range' exception. -- Built-in Function: number netmask_to_len (number MASK) Convert IPv4 netmask MASK into netmask length (number of bits preserved by the mask): netmask_to_len(inet_aton("255.255.255.0")) => 24 netmask_to_len(inet_aton("254.0.0.0")) => 7 -- Library Function: boolean match_cidr (string IP, string CIDR) This function is defined in the module 'match_cidr.mf' (*note Modules::). It returns 'true' if the IP address IP pertains to the IP range CIDR. The first argument, IP, is a string representation of an IP address. The second argument, CIDR, is a string representation of a IP range in CIDR notation, i.e. '"A.B.C.D/N"', where A.B.C.D is an IPv4 address and N specifies the "prefix length" - the number of shared initial bits, counting from the left side of the address. The following example will reject the mail if the IP address of the sending machine does not belong to the block '10.10.1.0/19': if not match_cidr(${client_addr}, "10.10.1.0/19") reject fi  File: mailfromd.info, Node: DNS functions, Next: Geolocation functions, Prev: Internet address manipulation functions, Up: Library 5.20 DNS Functions ================== The functions are implemented in two layers: "primitive" built-in functions which raise exceptions if the lookup fails, and library calls that are warranted to always return meaningful value without throwing exceptions. The built-in layer is always available. The library calls become available after requesting the 'dns' module (*note Modules::): require dns -- Built-in Function: string dns_getaddr (string DOMAIN) Returns a whitespace-separated list of IP addresses ('A' records) for DOMAIN. This function does not use the DNS cache. -- Built-in Function: string dns_getname (string IPSTR) Returns a whitespace-separated list of domain names ('PTR' records) for the IPv4 address IPSTR. This function does not use the DNS cache. -- Built-in Function: string getmx (string DOMAIN [, boolean IP]) Returns a whitespace-separated list of 'MX' names (if IP is not given or if it is '0') or 'MX' IP addresses (if 'IP!=0')) for DOMAIN. Within the returned string, items are sorted in order of increasing 'MX' priority. If DOMAIN has no 'MX' records, an empty string is returned. If the DNS query fails, 'getmx' raises an appropriate exception. Examples: getmx("mafra.cz") => "smtp1.mafra.cz smtp2.mafra.cz relay.iol.cz" getmx("idnes.cz") => "smtp1.mafra.cz smtp2.mafra.cz relay.iol.cz" getmx("gnu.org") => "mx10.gnu.org mx20.gnu.org" getmx("org.pl") => "" _Note_: 1. The number of items returned by 'getmx(DOMAIN)' can differ from that obtained from 'getmx(DOMAIN, 1)', e.g.: getmx("aol.com") => mailin-01.mx.aol.com mailin-02.mx.aol.com mailin-03.mx.aol.com mailin-04.mx.aol.com getmx("aol.com", 1) => 64.12.137.89 64.12.137.168 64.12.137.184 64.12.137.249 64.12.138.57 64.12.138.88 64.12.138.120 64.12.138.185 205.188.155.89 205.188.156.185 205.188.156.249 205.188.157.25 205.188.157.217 205.188.158.121 205.188.159.57 205.188.159.217 2. This interface will change in future releases, when array data types are implemented. -- Built-in Function: boolean primitive_hasmx (string DOMAIN) Returns 'true' if the domain name given by its argument has any 'MX' records. If the DNS query fails, this function throws 'failure' or 'temp_failure'. -- Library Function: boolean hasmx (string DOMAIN) Returns 'true' if the domain name given by its argument has any 'MX' records. Otherwise, if DOMAIN has no 'MX's or if the DNS query fails, 'hasmx' returns 'false'. -- Built-in Function: string primitive_hostname (string IP) The IP argument should be a string representing an IP address in "dotted-quad" notation. The function returns the canonical name of the host with this IP address obtained from DNS lookup. For example primitive_hostname (${client_addr}) returns the fully qualified domain name of the host represented by Sendmail variable 'client_addr'. If there is no 'PTR' record for IP, 'primitive_hostname' raises the exception 'e_not_found'. If DNS query fails, the function raises 'failure' or 'temp_failure', depending on the character of the failure. -- Library Function: string hostname (string IP) The IP argument should be a string representing an IP address in "dotted-quad" notation. The function returns the canonical name of the host with this IP address obtained from DNS lookup. If there is no 'PTR' record for IP, or if the lookup fails, the function returns IP unchanged. The previous 'mailfromd' versions used the following paradigm to check if an IP address resolves: if hostname(ip) != ip ... -- Built-in Function: boolean primitive_ismx (string DOMAIN, string HOST) The DOMAIN argument is any valid domain name, the HOST is a host name or IP address. The function returns 'true' if HOST is one of the 'MX' records for the DOMAIN. If DOMAIN has no 'MX' records, 'primitive_ismx' raises exception 'e_not_found'. If DNS query fails, the function raises 'failure' or 'temp_failure', depending on the character of the failure. -- Library Function: boolean ismx (string DOMAIN, string HOST) The DOMAIN argument is any valid domain name, the HOST is a host name or IP address. The function returns 'true' if HOST is one of the 'MX' records for the DOMAIN. Otherwise it returns 'false'. If DOMAIN has no 'MX' records, or if the DNS query fails, the function returns 'false'. -- Built-in Function: string primitive_resolve (string HOST, [string DOMAIN]) Reverse of 'primitive_hostname'. The 'primitive_resolve' function returns the IP address for the host name specified by HOST argument. If HOST has no A records, the function raises the exception 'e_not_found'. If DNS lookup fails, the function raises 'failure' or 'temp_failure', depending on the character of the failure. If the optional DOMAIN argument is given, it will be appended to HOST (with an intermediate dot), before querying the DNS. For example, the following two expressions will return the same value: primitive_resolve("puszcza.gnu.org.ua") primitive_resolve("puszcza", "gnu.org.ua") There is a considerable internal difference between one-argument and two-argument forms of 'primitive_resolve': the former queries DNS for an 'A' record, whereas the latter queries it for any record matching HOST in the domain DOMAIN and then selects the most appropriate one. For example, the following two calls are equivalent: primitive_hostname("213.130.0.22") primitive_resolve("22.0.130.213", "in-addr.arpa") This makes it possible to use 'primitive_resolve' for querying DNS black listing domains. *Note match_dnsbl::, for a working example of this approach. See also *note match_rhsbl::, for another practical example of the use of the two-argument form. -- Library Function: string resolve (string HOST, [string DOMAIN]) Reverse of 'hostname'. The 'resolve' function returns IP address for the host name specified by HOST argument. If the host name cannot be resolved, or a DNS failure occurs, the function returns '"0"'. This function is entirely equivalent to 'primitive_resolve' (see above), except that it never raises exceptions. -- Built-in Function: string ptr_validate (string IP) Tests whether the DNS reverse-mapping for IP exists and correctly points to a domain name within a particular domain. First, it obtains all PTR records for IP. Then, for each record returned, a look up for A records is performed and IP addresses of each record are compared against IP. The function returns true if a matching A record is found. -- Built-in Function: boolean primitive_hasns (string DOMAIN) Returns 'True' if the domain DOMAIN has at least one 'NS' record. Throws exception if DNS lookup fails. -- Library Function: boolean hasns (string DOMAIN) Returns 'True' if the domain DOMAIN has at least one 'NS' record. Returns 'False' if there are no 'NS' records or if the DNS lookup fails. -- Built-in Function: string getns (string DOMAIN ; boolean RESOLVE, boolean SORT) Returns a whitespace-separated list of all the 'NS' records for the domain DOMAIN. Optional parameters RESOLVE and SORT control the formatting. If RESOLVE is 0 (the default), the resulting string will contain IP addresses of the NS servers. If RESOLVE is not 0, hostnames will be returned instead. If SORT is 1, the returned items will be sorted. If the DNS query fails, 'getns' raises an appropriate exception.  File: mailfromd.info, Node: Geolocation functions, Next: Database functions, Prev: DNS functions, Up: Library 5.21 Geolocation functions ========================== The "geolocation functions" allow you to identify the country where the given IP address or host name is located. These functions are available only if the 'GeoIP' library is installed and 'mailfromd' is compiled with the 'GeoIP' support. The 'm4' macro 'WITH_GEOIP' is defined if it is so. The 'GeoIP' is a geolocational package distributed by 'MaxMind' under the terms of the GNU Lesser General Public License. The library is available from . -- Built-in Function: string geoip_country_code_by_addr ( string IP [, bool TLC]) Look up the 'ISO 3166-1' country code corresponding to the IP address IP. If TLC is given and is not zero, return the 3 letter code, otherwise return the 2 letter code. -- Built-in Function: string geoip_country_code_by_name ( string NAME [, bool TLC]) Look up the 'ISO 3166-1' country code corresponding to the host name NAME. If TLC is given and is not zero, return the 3 letter code, otherwise return the 2 letter code. If it is impossible to locate the country, both functions raise the 'e_not_found' exception. If an error internal to the 'GeoIP' library occurs, they raise the 'e_failure' exception. Applications may test whether the GeoIP support is present and enable corresponding code blocks conditionally by testing if the 'WITH_GEOIP' m4 macro is defined. For example, the following code adds to the message the 'X-Originator-Country' header, containing the 2 letter code of the country where the client machine is located. If 'mailfromd' is compiled without 'GeoIP' support, it does nothing: m4_ifdef(`WITH_GEOIP',` header_add("X-Originator-Country", geoip_country_code_by_addr($client_addr)) ')  File: mailfromd.info, Node: Database functions, Next: I/O functions, Prev: Geolocation functions, Up: Library 5.22 Database Functions ======================= The functions described below provide a user interface to DBM databases. Each DBM database is a separate disk file that keeps "key/value pairs". The interface allows to retrieve the value corresponding to a given key. Both 'key' and 'value' are null-terminated character strings. To lookup a key, it is important to know whether its length includes the terminating null byte. By default, it is assumed that it does not. Another important database property is the "file mode" of the database file. The default file mode is '640' (i.e. 'rw-r----', in symbolic notation). Both properties can be configured using the 'dbprop' pragma: #pragma dbprop PATTERN PROP [PROP] The PATTERN is the database name or shell-style globbing pattern. Properties defined by that pragma apply to each database whose name matches this pattern. If several 'dbprop' pragmas match the database name, the one that matches exactly is preferred. The rest of arguments define properties for that database. The valid values for PROP are: 1. The word 'null', meaning that the terminating null byte is included in the key length. Setting 'null' property is necessary, for databases created with 'makemap -N hash' command. 2. File mode for the disk file. It can be either an octal number, or a symbolic mode specification in ls-like format. E.g., the following two formats are equivalent: 640 rw-r---- For example, consider the following pragmas: #pragma dbprop /etc/mail/whitelist.db 640 It tells that the database file 'whitelist.db' has privileges '640' and do not include null in the key length. Similarly, the following pragma: #pragma dbprop `/etc/mail/*.db' null 600 declares that all database files in directory '/etc/mail' have privileges '640' and include null terminator in the key length. _Notice_, the use of 'm4' quoting characters in the example below. Without them, the sequence '/*' would have been taken as the beginning of a comment. Additionally, for compatibility with previous versions (up to 5.0), the terminating null property can be requested via an optional argument to the database functions (in description below, marked as NULL). -- Built-in Function: boolean dbmap (string DB, string KEY, [boolean NULL]) Looks up KEY in the DBM file DB and returns 'true' if it is found. See above for the meaning of NULL. *Note whitelisting::, for an example of using this function. -- Built-in Function: string dbget (string DB, string KEY [, string DEFAULT, boolean NULL]) Looks up KEY in the database DB and returns the value associated with it. If the key is not found returns DEFAULT, if specified, or empty string otherwise. See above for the meaning of NULL. -- Built-in Function: void dbput (string DB, string KEY, string VALUE [, boolean NULL, number MODE ]) Inserts in the database a record with the given KEY and VALUE. If a record with the given KEY already exists, its value is replaced with the supplied one. See above for the meaning of NULL. Optional MODE allows to explicitly specify the file mode for this database. See also '#pragma dbprop', described above. -- Built-in Function: void dbinsert (string DB, string KEY, string VALUE [, boolean REPLACE, boolean NULL, number MODE ]) This is an improved variant of 'dbput', which provides a better control on the actions to take if the KEY already exists in the database. Namely, if REPLACE is 'True', the old value is replaced with the new one. Otherwise, the 'e_exists' exception is thrown. -- Built-in Function: void dbdel (string DB, string KEY [, boolean NULL, number MODE]) Delete from the database the record with the given KEY. If there are no such record, return without signalling error. If the optional NULL argument is given and is not zero, the terminating null character will be included in KEY length. Optional MODE allows to explicitly specify the file mode for this database. See also '#pragma dbprop', described above. The functions above have also the corresponding exception-safe interfaces, which return cleanly if the 'e_dbfailure' exception occurs. To use these interfaces, request the 'safedb' module: require safedb The exception-safe interfaces are: -- Library Function: string safedbmap (string DB, string KEY [, string DEFAULT, boolean NULL]) This is an exception-safe interface to 'dbmap'. If a database error occurs while attempting to retrieve the record, 'safedbmap' returns DEFAULT or '0', if it is not defined. -- Library Function: string safedbget (string DB, string KEY [, string DEFAULT, boolean NULL]) This is an exception-safe interface to 'dbget'. If a database error occurs while attempting to retrieve the record, 'safedbget' returns DEFAULT or empty string, if it is not defined. -- Library Function: void safedbput (string DB, string KEY, string VALUE [, boolean NULL]) This is an exception-safe interface to 'dbput'. If a database error occurs while attempting to retrieve the record, the function returns without raising exception. -- Library Function: void safedbdel (string DB, string KEY [, boolean NULL]) This is an exception-safe interface to 'dbdel'. If a database error occurs while attempting to delete the record, the function returns without raising exception. The verbosity of 'safedb' interfaces in case of database error is controlled by the value of 'safedb_verbose' variable. If it is '0', these functions return silently. This is the default behavior. Otherwise, if 'safedb_verbose' is not '0', these functions log the detailed diagnostics about the database error and return. The following functions provide a sequential access to the contents of a DBM database: -- Built-in Function: number dbfirst (string NAME) Start sequential access to the database NAME. The return value is an opaque identifier, which is used by the remaining sequential access functions. This number is '0' if the database is empty. -- Built-in Function: number dbnext (number DN) Select next record form the database. The argument DN is the access identifier, returned by a previous call to 'dbfirst' or 'dbnext'. Returns new access identifier. This number is '0' if all records in the database have been visited. The usual approach for iterating over all records in a database DBNAME is: loop for number dbn dbfirst(DBNAME) do ... done while dbnext(dbn) The following two functions can be used to access values of the currently selected database record. Their argument, DN, is the access identifier, returned by a previous call to 'dbfirst' or 'dbnext'. -- Built-in Function: string dbkey (number DN) Return the key from the selected database record. -- Built-in Function: string dbvalue (number DN) Return the value from the selected database record. -- Built-in Function: number db_expire_interval (string FMT) The FMT argument is a database format identifier (*note Database Formats::). If it is valid, the function returns the expiration interval for that format. Otherwise, 'db_expire_interval' raises the 'e_not_found' exception. -- Built-in Function: string db_name (string FMTID) The FMT argument is a database format identifier (*note Database Formats::). The function returns the file name for that format. If FMTID does not match any known format, 'db_name' raises the 'e_not_found' exception. -- Built-in Function: number db_get_active (string FMTID) Returns the flag indicating whether the cache database FMTID is currently enabled. If FMTID does not match any known format, 'db_name' raises the 'e_not_found' exception. -- Built-in Function: void db_set_active (string FMTID, boolean ENABLE) Enables the cache database FMTID if ENABLE is 'True', or disables it otherwise. For example, to disable DNS caching, do: db_set_active("dns", 0) -- Built-in Function: boolean relayed (string DOMAIN) Returns 'true' if the string DOMAIN is found in one of relayed domain files (*note relayed-domain-file: conf-base.). The usual construct is: if relayed(hostname(${client_addr})) ... which yields 'true' if the IP address from 'Sendmail' variable 'client_addr' is relayed by the local machine.  File: mailfromd.info, Node: I/O functions, Next: System functions, Prev: Database functions, Up: Library 5.23 I/O functions ================== MFL provides a set of functions for writing to disk files, pipes or sockets and reading from them. The idea behind them is the same as in most other programming languages: first you open the resource with a call to 'open' which returns a "descriptor" i.e. an integer number uniquely identifying the resource. Then you can write or read from it using this descriptor. Finally, when the resource is no longer needed, you can close it with a call to 'close'. The number of available resource descriptors is limited. The default limit is 1024. You can tailor it to your needs using the 'max-streams' runtime configuration statement. *Note max-streams: conf-runtime, for a detailed description. -- Built-in Function: number open (string NAME) The NAME argument specifies the name of a resource to open and the access rights you need to have on it. The function returns a descriptor of the opened stream, which can subsequently be used as an argument to other I/O operations. First symbols of NAME determine the type of the resource to be opened and the access mode: '>' The rest of NAME is a name of a file. Open the file for read-write access. If the file exists, truncate it to zero length, otherwise create the file. '>>' The rest of NAME is a name of a file. Open the file for appending (writing at end of file). The file is created if it does not exist. '|' Treat the rest of NAME as the command name and its arguments. Run this command and open its standard input for writing. The standard error is closed before launching the program. This can be altered by using the following versions of this construct: |2>null: COMMAND Standard error is redirected to '/dev/null'. |2>file:NAME COMMAND Execute COMMAND with its standard error redirected to the file NAME. If the file exists, it will be truncated. |2>>file:NAME COMMAND Standard error of the COMMAND is appended to the file NAME. If file does not exist, it will be created. The '|2>null:' construct described above is a shortcut for |2>>file:/dev/null COMMAND |2>syslog:FACILITY[.PRIORITY] COMMAND Standard error is redirected to the given syslog FACILITY and, optionally, PRIORITY. If the latter is omitted, 'LOG_ERR' is assumed. Valid values for FACILITY are: 'user', 'daemon', 'auth', 'authpriv', 'mail', and 'local0' through 'local7'. Valid values for PRIORITY are: 'emerg', 'alert', 'crit', 'err', 'warning', 'notice', 'info', 'debug'. Both FACILITY and PRIORITY may be given in upper, lower or mixed cases. Notice, that no whitespace characters are allowed between '|' and '2>'. '|<' Treat the rest of NAME as the command name and its arguments. Run this command with its stdin closed and stdout open for reading. The standard error is treated as described above (see '|'). '|&' Treat the rest of NAME as the command name and its arguments. Run this command and set up for two-way communication with it, i.e writes to the descriptor returned by 'open' will send data to the program's standard input, reads from the descriptor will get data from the program's standard output. The standard error is treated as described above (see '|'). For example, the following redirects it to syslog 'mail.debug': |&2>syslog:mail.debug COMMAND '@' Treat the rest of NAME as the URL of a socket to connect to. Valid URL forms are described in *note milter port specification::. If none of these prefixes is used, NAME is treated as a name of an existing file and 'open' will attempt to open this file for reading. The 'open' function will signal exception 'e_failure' if it is unable to open the resource or get the required access to it. -- Built-in Function: number spawn (string CMD [, number IN, number OUT, number ERR]) Runs the supplied command CMD. The syntax of the CMD is the same as for the NAME argument to 'open' (see above), which begins with '|', excepting that the '|' sign is optional. That is: spawn("/bin/cat") has exactly the same effect as open("|/bin/cat") Optional arguments specify file stream descriptors to be used for the program standard input, output and error streams, correspondingly. If supplied, these should be the values returned by a previous call to 'open' or 'tempfile'. The value '-1' means no redirection. The example below starts the 'awk' program with a simple expression as its argument and redirects the content of the file '/etc/passwd' to its standard input. The returned stream descriptor is bound to the command's standard output (see the description of '|<' prefix above). The standard error is closed: number fd spawn(" 2006-11-25 19:59:24 EET strftime('%Y-%m-%d %H:%M:%S %Z', 1164477564, 1) => 2006-11-25 17:59:24 GMT -- Built-in Function: string uname (string FORMAT) This function returns system information formatted according to the format specification FORMAT. Ordinary characters placed in the format string are copied to the output without conversion. Conversion specifiers are introduced by a '%' character. The following conversions are defined: %s Name of this system. %n Name of this node within the communications network to which this node is attached. Note, that it does not necessarily coincide with the actual machine name in DNS. %r Kernel release. %v Kernel version. %m Name of the hardware type on which the system is running. For example: uname('%n runs %s, release %r on %m') => "Trurl runs Linux, release 2.6.26 on i686" Notice the use of single quotes. -- Built-in Function: void unlink (string NAME) Unlinks (deletes) the file NAME. On error, throws the 'e_failure' exception. -- Built-in Function: number system (string STR) The function 'system' executes a command specified in STR by calling '/bin/sh -c string', and returns -1 on error or the return status of the command otherwise. -- Built-in Function: void sleep (number SECS[, USEC]) Sleep for SECS seconds. If optional USEC argument is given, it specifies additional number of microseconds to wait for. For example, to suspend execution of the filter for 1.5 seconds: sleep(1,500000) This function is intended mostly for debugging and experimental purposes. -- Built-in Function: number umask (number MASK) Set the umask to MASK & 0777. Return the previous value of the mask. ---------- Footnotes ---------- (1) _Note_, that the return code is inverted in respect to the system function 'access(2)'.  File: mailfromd.info, Node: Passwd functions, Next: Sieve Interface, Prev: System functions, Up: Library 5.25 System User Database ========================= -- Built-in Function: string getpwnam (string NAME) -- Built-in Function: string getpwuid (number UID) Look for the user NAME ('getpwnam') or user ID UID ('getpwuid') in the system password database and return the corresponding record, if found. If not found, raise the 'e_not_found' exception. The returned record consists of six fields, separated by colon sign: uname:passwd:uid:gid:gecos:dir:shell Field Meaning ------------------------------------------------------------------- uname user name passwd user password uid user ID gid group ID gecos real name dir home directory shell shell program For example: getpwnam("gray") => "gray:x:1000:1000:Sergey Poznyakoff:/home/gray:/bin/bash" Following two functions can be used to test for existence of a key in the user database: -- Built-in Function: boolean mappwnam (string NAME) -- Built-in Function: boolean mappwuid (number UID) Return 'true' if NAME (or UID) is found in the system user database.  File: mailfromd.info, Node: Sieve Interface, Next: Interfaces to Third-Party Programs, Prev: Passwd functions, Up: Library 5.26 Sieve Interface ==================== 'Sieve' is a powerful mail filtering language, defined in RFC 3028. 'Mailfromd' supports an extended form of this language. For a description of the language and available extensions, see *note Sieve Language: (mailutils)Sieve Language. -- Built-in Function: boolean sieve (number MSG, string SCRIPT [, number FLAGS, string FILE, number LINE]) Compile the Sieve program SCRIPT and execute it over the message identified by the descriptor NMSG. Optional FLAGS modify the behavior of the function. It is a bit-mask field, consisting of a bitwise 'or' of one or more of the following flags, defined in 'sieve.mf': 'MF_SIEVE_FILE' The SCRIPT argument specifies the name of a Sieve program file. This is the default. 'MF_SIEVE_TEXT' The SCRIPT argument is a string containing entire Sieve program. Optional arguments FILE and LINE can be used to fix source locations in Sieve diagnostic messages (see below). 'MF_SIEVE_LOG' Log every executed 'Sieve' action. 'MF_SIEVE_DEBUG_TRACE' Trace execution of 'Sieve' tests. 'MF_SIEVE_DEBUG_INSTR' Log every instruction, executed in the compiled 'Sieve' code. This produces huge amounts of output and is rarely useful, unless you suspect some bug in 'Sieve' implementation and wish to trace it. For example, 'MF_SIEVE_LOG|MF_SIEVE_DEBUG_TRACE' enables logging 'Sieve' actions and tests. The 'sieve' function returns 'true' if the message was accepted by the SCRIPT program, and 'false' otherwise. Here, the word "accepted" means that some form of 'KEEP' action (*note keep: (mailutils)Actions.) was executed over the message. While executing the Sieve script, Sieve environment ('RFC 5183') is initialized as follows: domain The domain name of the server Sieve is running on. host Host name of the server Sieve is running on. location The string 'MTA'. name The string 'GNU Mailutils'. phase The string 'pre'. remote-host Defined to the value of 'client_ptr' macro, if it was required. remote-ip Defined to the value of 'client_addr' macro, if it was required. version The version of GNU Mailutils. The following example discards each message not accepted by the 'Sieve' program '/etc/mail/filter.siv': require 'sieve' group eom do if not sieve(current_message(), "/etc/mail/filter.siv", MF_SIEVE_LOG) discard fi done The Sieve program can be embedded in the MFL filter, as shown in the example below: require 'sieve' prog eom do if not sieve(current_message(), "require \"fileinto\";\n" "fileinto \"/tmp/sieved.mbox\";", MF_SIEVE_TEXT | MF_SIEVE_LOG) discard fi done In such cases, any Sieve diagnostics (error messages, traces, etc.) will be marked with the locations relative to the line where the call to 'sieve' appears. For example, the above program produces the following in the log: prog.mf:7: FILEINTO; delivering into /tmp/sieved.mbox Notice, that the line number correctly refers to the line where the 'fileinto' action appears in the source. However, there are cases where the reported line number is incorrect. This happens, for instance, if SCRIPT is a string variable defined elsewhere. To handle such cases, 'sieve' accepts two optional parameters which are used to compute the location in the Sieve program. The FILE parameter specifies the file name where the definition of the program appears, and the LINE parameter gives the number of line in that file where the program begins. For example: require 'sieve' const sieve_prog_line __line__ + 2 string sieve_prog <. MFL provides an interface to DSPAM functionality if the 'libdspam' library is installed and 'mailfromd' is linked with it. The 'm4' macro 'WITH_DSPAM' is defined if it is so. The DSPAM functions and definitions become available after requiring the 'dspam' module: require 'dspam' -- Built-in Function: number dspam (number MSG, number MODE_FLAGS; number CLASS_SOURCE) Analyze a message using DSPAM. The message is identified by its descriptor, passed in the MSG argument. The MODE_FLAGS argument controls the function behavior. Its value is a bitwise OR of operation mode, flag, tokenizer and training mode. "Operation mode" defines what 'dspam' is supposed to do with the message. Its value is either 'DSM_PROCESS' if full processing of the message is intended (the default), or 'DSM_CLASSIFY', if the message must only be classified. Optional "flag" bits turn on additional functionality. The 'DSF_SIGNATURE' bit instructs 'dspam' to create a signature for the message - a unique string which can subsequently be used to identify that particular message. Upon return from the function, the signature is stored in the 'dspam_signature' variable. The 'DSF_NOISE' bit enables Bayesian noise reduction, and 'DSF_WHITELIST' enables automatic whitelisting. Additional flags are available for defining the algorithm to split the message into tokens ("tokenizer") and "training mode". *Note flags-dspam::, for a complete list of these. All these are optional, any missing values will be read from the DSPAM "configuration file". The configuration file must always be present. Its full file name must be stored in the global variable 'dspam_config'. There is no default value, so make sure this variable is initialized. If a specific profile section should be read, store the name of that profile in the variable 'dspam_profile'. When called to process or classify the message, 'dspam' returns an integer code of the class of the message. The value 'DSR_ISSPAM' means that this message was classified as spam. The value 'DSR_ISINNOCENT' means it is a clean ("ham") message. The probability and confidence values are returned in global variables 'dspam_probability' and 'dspam_confidence'. Since MFL lacks floating-point data type, both variables keep integers, obtained from the corresponding floating point values by shifting the decimal point 'dspam_prec' digits to the right and rounding the resulting value to the nearest integer. The same method is used in 'spamc' function (*note sa-floating-point-conversion::). The default value for 'dspam_prec' variable is 3. You can use the 'sa_format_score' function to convert these values to strings representing floating point numbers, e.g.: require 'dspam' require 'sa' prog eom do if dspam(current_message(), DSM_PROCESS | DSM_SIGNATURE) == DSR_ISSPAM header_add("X-DSPAM-Result", "Spam") else header_add("X-DSPAM-Result", "Innocent") fi header_add("X-DSPAM-Probability", sa_format_score(dspam_probability, dspam_prec)) header_add("X-DSPAM-Confidence", sa_format_score(dspam_confidence, dspam_prec)) header_add("X-DSPAM-Signature", dspam_signature) done Optional CLASS_SOURCE argument is used when training the DSPAM classifier. It is a bitwise OR of the message class and message source values. "Message class" specifies the class this message belongs to. Possible values are 'DSR_ISSPAM', for spam messages, and 'DSR_ISINNOCENT', for clean messages. "Message source" informs DSPAM where this message comes from. The value 'DSS_ERROR' means the message was previously misclassified by DSPAM. The value 'DSS_CORPUS' indicates the message comes from a corpus feed. Finally, the value 'DSS_INOCULATION' means that the message is in pristine form, and should be trained as an inoculation. "Inoculation" is a more intense mode of training, usually used on honeypots. The following example calls 'dspam' to train the classifier on the current message if it was sent to a honeypot address, and uses 'dspam' to analyze the message class otherwise. The 'honeypot' variable is supposed to be set elsewhere in the code (e.g. in the 'envrcpt' handler): prog eom do number res if honeypot set res dspam(current_message(), DSM_PROCESS, DSR_ISSPAM | DSS_INOCULATION) discard else if dspam(current_message(), DSM_PROCESS | DSM_SIGNATURE) == DSR_ISSPAM header_add("X-DSPAM-Result", "Spam") else header_add("X-DSPAM-Result", "Innocent") fi header_add("X-DSPAM-Probability", sa_format_score(dspam_probability, dspam_prec)) header_add("X-DSPAM-Confidence" sa_format_score(dspam_confidence, dspam_prec)) header_add("X-DSPAM-Signature", dspam_signature) fi done * Menu: * flags-dspam:: DSPAM Operation Modes and Flags. * class-dspam:: DSPAM Class and Source Bits. * vars-dspam:: DSPAM Global Variables.  File: mailfromd.info, Node: flags-dspam, Next: class-dspam, Up: DSPAM 5.27.2.1 DSPAM Operation Modes and Flags. ......................................... The tables below summarize flags which can be used in the MODE_FLAGS argument to 'dspam' function. The argument is a bitwise OR of "operation mode", "flags", "tokenizer" and "training mode" bits. Only one operation mode bit can be used. Flags, tokenizer and training mode are optional. Any number of flags, but no more than one tokenizer type and one training mode bit are allowed. Missing values will be supplied from the configuration file. Mode Meaning -------------------------------------------------------------------------- DSM_PROCESS Process message DSM_CLASSIFY Classify message only (do not write changes) Table 5.2: DSPAM Operation modes Flag Meaning -------------------------------------------------------------------------- DSF_SIGNATURE Create a signature DSF_NOISE Use Bayesian Noise Reduction DSF_WHITELIST Use Automatic Whitelisting Table 5.3: DSPAM flags Constant Meaning -------------------------------------------------------------------------- DSZ_WORD Use "WORD" tokenizer DSZ_CHAIN Use "CHAIN" tokenizer DSZ_SBPH Use "SBPH" tokenizer DSZ_OSB Use "OSB" tokenizer Table 5.4: DSPAM Tokenizer bits Mode Meaning -------------------------------------------------------------------------- DST_TEFT Train Everything DST_TOE Train-on-Error DST_TUM Train-until-Mature Table 5.5: DSPAM Training Modes  File: mailfromd.info, Node: class-dspam, Next: vars-dspam, Prev: flags-dspam, Up: DSPAM 5.27.2.2 DSPAM Class and Source Bits .................................... The tables below summarize flags which can be used in the CLASS_SOURCE argument to 'dspam' function. The argument is a bitwise OR of "classification" and "source" bits. At most one classification and one source bit can be given. If not supplied, 'DSR_NONE|DSS_NONE' is used. The classification flags are also used as the return code, as shown in the following table. Mode As return value As argument --------------------------------------------------------------------------- DSR_NONE N/A Classify message DSR_ISSPAM Message is spam Learn as spam DSR_ISINNOCENT Message is innocent Learn as innocent Table 5.6: DSPAM Classification Source Meaning -------------------------------------------------------------------------- DSS_NONE No classification source (use only with DSR_NONE) DSS_ERROR Misclassification by libdspam DSS_CORPUS Message came from a corpus feed DSS_INOCULATION Message inoculation Table 5.7: DSPAM Source  File: mailfromd.info, Node: vars-dspam, Prev: class-dspam, Up: DSPAM 5.27.2.3 DSPAM Global Variables ............................... Following global variables affect the behavior of the 'dspam' function: -- Built-in variable: string dspam_config Name of the DSPAM configuration file. You must set this variable prior to calling 'dspam'. There is no default value. -- Built-in variable: string dspam_profile Name of the configuration profile to be used. If empty (the default), use global configuration settings. -- Built-in variable: string dspam_user Name of the user on behalf of which 'dspam' is called. Default is empty (no user). -- Built-in variable: string dspam_group Name of the user group on behalf of which 'dspam' is called. Default is empty (no group). -- Built-in variable: number dspam_prec Number of decimal digits to retain in the 'dspam_probability' and 'dspam_confidence' values. *Note dspam probability and confidence::, for more information and examples. Before returning, 'dspam' stores additional information in the following variables: -- Built-in variable: string dspam_signature Signature of the classified message. This variable is initialized if 'DSF_SIGNATURE' bit is set in the MODE_FLAGS argument (*note dspam classify example::), -- Built-in variable: number dspam_probability Spam probability value converted to integer by shifting decimal point 'dspam_prec' positions to the right and rounding the resulting number. *Note dspam probability and confidence::, for more information and examples. -- Built-in variable: number dspam_confidence Spam confidence converted to integer using the same algorithm as for 'dspam_probability'. *Note dspam probability and confidence::, for more information and examples.  File: mailfromd.info, Node: ClamAV, Prev: DSPAM, Up: Interfaces to Third-Party Programs 5.27.3 ClamAV ------------- -- Built-in Function: boolean clamav (number MSG, string URL) Pass the message MSG to the ClamAV daemon at URL. Return 'true' if it detects a virus in it. Return virus name in 'clamav_virus_name' global variable. The 'clamav' function can signal the following exceptions: 'e_failure' if failed to connect to the server, 'e_url' if the supplied URL is invalid and 'e_range' if the supplied port number is out of the range 1-65535. An example usage: prog eom do if clamav(current_message(), "tcp://192.168.10.1:6300") reject 550 5.7.0 "Infected with %clamav_virus_name" fi done  File: mailfromd.info, Node: Rate limiting functions, Next: Greylisting functions, Prev: Interfaces to Third-Party Programs, Up: Library 5.28 Rate limiting functions ============================ -- Built-in Function: number rate (string KEY, number SAMPLE-INTERVAL, [number MIN-SAMPLES, number THRESHOLD]) Returns the mail sending rate for KEY per SAMPLE-INTERVAL. Optional MIN-SAMPLES, if supplied, specifies the minimal number of mails needed to obtain the statistics. The default is 2. Optional THRESHOLD controls rate database updates. If the observed rate (per SAMPLE-INTERVAL seconds) is higher than the THRESHOLD, the hit counters for that key are not incremented and the database is not updated. Although the THRESHOLD argument is optional(1), its use is strongly encouraged. Normally, the value of THRESHOLD equals the value compared with the return from RATE, as in: if rate("$f-$client_addr", rate_interval, 4, maxrate) > maxrate tempfail 450 4.7.0 "Mail sending rate exceeded. Try again later" fi This function is a low-level interface. Instead of using it directly, we advise to use the 'rateok' function, described below. -- Library Function: boolean rateok (string KEY, number SAMPLE-INTERVAL, number THRESHOLD, [number MIN-SAMPLES]) To use this function, require the 'rateok' module (*note Modules::), e.g.: 'require rateok'. The 'rateok' function returns 'True' if the mail sending rate for KEY, computed for the interval of SAMPLE-INTERVAL seconds is less than the THRESHOLD. Optional MIN-SAMPLES parameter supplies the minimal number of mails needed to obtain the statistics. It defaults to 4. *Note Sending Rate::, for a detailed description of the 'rateok' and its use. The 'interval' function (*note interval::) is often used in the second argument to 'rateok' or 'rate'. -- Built-in Function: boolean tbf_rate (string KEY, number COST, number SAMPLE-INTERVAL, number BURST-SIZE) This function implements a classical token bucket filter algorithm. Tokens are added to the bucket identified by the KEY at constant rate of 1 token per SAMPLE-INTERVAL microseconds, to a maximum of BURST-SIZE tokens. If no bucket is found for the specified key, a new bucket is created and initialized to contain BURST-SIZE tokens. If the bucket contains COST or more tokens, COST tokens are removed from it and 'tbf_rate' returns 'True'. Otherwise, the function returns 'False'. For a detailed description of the Token Bucket Algorithm and its use to limit mail rates, see *note TBF::. ---------- Footnotes ---------- (1) It is made optional in order to provide backward compatibility with the releases of mailfromd prior to 5.0.93.  File: mailfromd.info, Node: Greylisting functions, Next: Special test functions, Prev: Rate limiting functions, Up: Library 5.29 Greylisting functions ========================== -- Built-in Function: boolean greylist (string KEY, number INTERVAL) Returns 'True' if the KEY is found in the greylist database (controlled by 'database greylist' configuration file statement, *note conf-database::). The argument INTERVAL gives the greylisting interval in seconds. The function stores the number of seconds left to the end of greylisting period in the global variable 'greylist_seconds_left'. *Note Greylisting::, for a detailed explanation. The function 'greylist' can signal 'e_dbfailure' exception. -- Built-in Function: boolean is_greylisted (string KEY Returns 'True' if the KEY is still greylisted. If 'true' is returned, the function also stores the number of seconds left to the end of greylisting period in the global variable 'greylist_seconds_left'. This function is available only if Con Tassios implementation of greylisting is used. *Note greylisting types::, for a discussion of available greylisting implementations. *Note greylist::, for a way to switch to Con Tassios implementation.  File: mailfromd.info, Node: Special test functions, Next: Mail Sending Functions, Prev: Greylisting functions, Up: Library 5.30 Special Test Functions =========================== -- Library Function: boolean portprobe (string HOST, [number PORT]) -- Library Function: boolean listens (string HOST, [number PORT]) Returns 'true' if the IP address or host name given by HOST argument listens on the port number PORT (default 25). This function is defined in the module 'portprobe'. -- Built-in Function: boolean validuser (string NAME) Returns 'true' if authenticity of the user NAME is confirmed using mailutils authentication system. *Note Local Account Verification::, for more details. -- Library Function: boolean valid_domain (string DOMAIN) Returns 'true' if the domain name DOMAIN has a corresponding A record or if it has any 'MX' records, i.e. if it is possible to send mail to it. To use this function, require the 'valid_domain' module (*note Modules::): require valid_domain -- Library Function: number heloarg_test (string ARG, string REMOTE_IP, string LOCAL_IP) Verify if an argument of 'HELO' ('EHLO') command is valid. To use this function, require the 'heloarg_test' module (*note Modules::). Arguments: ARG 'HELO' ('EHLO') argument. Typically, the value of '$s' Sendmail macro; REMOTE_IP IP address of the remote client. Typically, the value of '$client_addr' Sendmail macro; LOCAL_IP IP address of this SMTP server; The function returns a number describing the result of the test, as described in the following table. Code Meaning -------------------------------------------------------------------------- HELO_SUCCESS ARG successfully passes all tests. HELO_MYIP ARG is our IP address. HELO_IPNOMATCH ARG is an IP, but it does not match the remote party IP address. HELO_ARGNORESOLVE ARG is an IP, but it does not resolve. HELO_ARGNOIP ARG is in square brackets, but it is not an IP address. HELO_ARGINVALID ARG is not an IP address and does not resolve to one. HELO_MYSERVERIP ARG resolves to our server IP. HELO_IPMISMATCH ARG does not resolve to the remote client IP address.  File: mailfromd.info, Node: Mail Sending Functions, Next: Blacklisting Functions, Prev: Special test functions, Up: Library 5.31 Mail Sending Functions =========================== The mail sending functions are new interfaces, introduced in version 3.1. The underlying mechanism for sending mail, called "mailer", is specified by '--mailer' command line option. This global setting can be overridden using the last optional argument to a particular function. In any case, the mailer is specified in the form of a URL. "Mailer URL" begins with a protocol specification. Two protocol specifications are currently supported: 'sendmail' and 'smtp'. The former means to use a 'sendmail'-compatible program to send mails. Such a program must be able to read mail from its standard input and must support the following options: '-oi' Do not treat '.' as message terminator. '-f ADDR' Use ADDR as the address of the sender. '-t' Get recipient addresses from the message. These conditions are met by most existing MTA programs, such as 'exim' or 'postfix' (to say nothing of 'sendmail' itself). Following the protocol specification is the "mailer location", which is separated from it with a colon. For the 'sendmail' protocol, the mailer location sets the full file name of the Sendmail-compatible MTA binary, for example: sendmail:/usr/sbin/sendmail A special form of a sendmail URL, consisting of protocol specification only ('sendmail:') is also allowed. It means "use the sendmail binary from the '_PATH_SENDMAIL' macro in your '/usr/include/paths.h' file". This is the default mailer. The 'smtp' protocol means to use an SMTP server directly. In this case the mailer location consists of two slashes, followed by the IP address or host name of the SMTP server, and, optionally, the port number. If the port number is present, it is separated from the rest of URL by a colon. For example: smtp://remote.server.net smtp://remote.server.net:24 -- Built-in Function: void send_mail (string MSG [, string TO, string FROM, string MAILER]) Sends message MSG to the email address TO. The value of MSG must be a valid RFC 2822 message, consisting of headers and body. Optional argument TO can contain several email addresses. In this case the message will be sent to each recipient specified in TO. If it is not specified, recipient addresses will be obtained from the message headers. Other optional arguments are: FROM Sets the sender address. By default '<>' is used. MAILER The URL of the mailer to use Sample usage: set message <<- EOT Subject: Test message To: Postmaster From: Mailfromd X-Agent: %__package__ (%__version__) Dear postmaster, This is to notify you that our /etc/mailfromd.mf needs a revision. -- Mailfromd filter administrator EOT send_mail(message, "postmaster@gnu.org.ua") -- Built-in Function: void send_text (string TEXT, string HEADERS [, string TO, string FROM, string MAILER]) A more complex interface to mail sending functions. Mandatory arguments: TEXT Text of the message to be sent. HEADERS Headers for the message. Optional arguments: TO Recipient email addresses. FROM Sender email address. MAILER URL of the mailer to use. The above example can be rewritten using 'send_text' as follows: set headers << -EOT Subject: Test message To: Postmaster From: Mailfromd X-Agent: %__package__ (%__version__) EOT set text <<- EOT Dear postmaster, This is to notify you that our /etc/mailfromd.mf needs a revision. -- Mailfromd filter administrator EOT send_text(text, headers, "postmaster@gnu.org.ua") -- Built-in Function: void send_message (number MSG [string TO, string FROM, string MAILER]) Send the message identified by descriptor MSG (*note Message functions::). Optional arguments are: TO Recipient email addresses. FROM Sender email address. MAILER URL of the mailer to use. -- Built-in Function: void send_dsn (string TO, string SENDER, string RCPT, string TEXT [, string HEADERS, string FROM, string MAILER]) This is an experimental interface which will change in the future versions. It sends a message disposition notification (RFC 2298, RFC 1894), of type 'deleted' to the email address TO. Arguments are: TO Recipient email address. SENDER Original sender email address. RCPT Original recipient email address. TEXT Notification text. Optional arguments: HEADERS Message headers FROM Sender address. MAILER URL of the mailer to use. -- Built-in Function: void create_dsn (string SENDER, string RCPT, string TEXT [, string HEADERS, string FROM]) Creates DSN message and returns its descriptor. Arguments are: SENDER Original sender email address. RCPT Original recipient email address. TEXT Notification text. HEADERS Message headers FROM Sender address.  File: mailfromd.info, Node: Blacklisting Functions, Next: SPF Functions, Prev: Mail Sending Functions, Up: Library 5.32 Blacklisting Functions =========================== The functions described in this subsection allow to check whether the given IP address is listed in certain "black list" DNS zone. -- Library Function: boolean match_dnsbl (string ADDRESS, string ZONE, string RANGE) This function looks up ADDRESS in the DNS blacklist zone ZONE and checks if the return falls into the given RANGE of IP addresses. It is intended as a replacement for the Sendmail macros 'dnsbl' and 'enhdnsbl'. To use 'match_dnsbl', require the 'match_dnsbl' module (*note Modules::). Arguments: ADDRESS IP address of the SMTP server to be tested. ZONE FQDN of the DNSbl zone to test against. RANGE The range of IP addresses in CIDR notation or the word 'ANY', which stands for '127.0.0.0/8'. The function returns 'true' if dns lookup for ADDRESS in the zone DNSBL yields an IP that falls within the range, specified by CIDR. Otherwise, it returns 'false'. This function raises the following exceptions: 'e_invip' if ADDRESS is invalid and 'e_invcidr' if CIDR is invalid. -- Library Function: boolean match_rhsbl (string EMAIL, string ZONE, string RANGE) This function checks if the IP address, corresponding to the domain part of EMAIL is listed in the RHS DNS blacklist zone ZONE, and if so, whether its record falls into the given range of IP addresses RANGE. It is intended as a replacement for the Sendmail macro 'rhsbl' by Derek J. Balling. To use this function, require the 'match_rhsbl' module (*note Modules::). Arguments: EMAIL E-mail address, whose domain name should be tested (usually, it is '$f') ZONE Domain name of the RHS DNS blacklist zone. RANGE The range of IP addresses in CIDR notation.  File: mailfromd.info, Node: SPF Functions, Next: DKIM, Prev: Blacklisting Functions, Up: Library 5.33 SPF Functions ================== "Sender Policy Framework", or SPF for short, is an extension to SMTP protocol that allows to identify forged identities supplied with the 'MAIL FROM' and 'HELO' commands. The framework is explained in detail in RFC 4408 () and on the SPF Project Site (http://www.openspf.org/). The following description is a short introduction only, and the users are encouraged to refer to the original specification for the detailed description of the framework. The domain holder publishes an "SPF record" - a special DNS resource record that contains a set of rules declaring which hosts are, and are not, authorized to use a domain name for 'HELO' and 'MAIL FROM' identities. This resource record is usually of type 'TXT'.(1) The MFL script can verify if the identity matches the published SPF record by calling 'check_host' function and analyzing its return code. The function can be called either in 'helo' or in 'envfrom' handler. Its arguments are: IP The IP address of the SMTP client that is emitting the mail. Usually it is '$client_addr'. DOMAIN The domain that provides the sought-after authorization information; Normally it is the domain portion of the 'MAIL FROM' or 'HELO' identity. SENDER The 'MAIL FROM' identity. HELO_DOMAIN The 'HELO' identity. MY_DOMAIN The SMTP domain served by the local server. The function returns a numeric result code. For convenience, all possible return values are defined as macros in module 'spf.mf'. The table below describes each value along with the recommended actions for it: 'None' A result of 'None' means that no records were published by the domain or that no checkable sender domain could be determined from the given identity. The checking software cannot ascertain whether or not the client host is authorized. Such a message can be subject to further checks that will decide about its fate. 'Neutral' The domain owner has explicitly stated that he cannot or does not want to assert whether or not the IP address is authorized. This result must be treated exactly like 'None'; the distinction between them exists only for informational purposes 'Pass' The client is authorized to send mail with the given identity. The message can be subject to further policy checks with confidence in the legitimate use of the identity or it can be accepted in the absence of such checks. 'Fail' The client is not authorized to use the domain in the given identity. The proper action in this case can be to mark the message with a header explicitly stating it is spam, or to reject it outright. If you choose to reject such mails, we suggest to use 'reject 550 5.7.1', as recommended by RFC 4408. The reject can return either a default explanation string, or the one supplied by the domain that published the SPF records, as in the example below: reject 550 5.7.1 "SPF check failed:\n%spf_explanation" (for the description of 'spf_explanation', *note spf_explanation::) 'SoftFail' The domain believes the host is not authorized but is not willing to make that strong of a statement. This result code should be treated as somewhere in between a 'Fail' and a 'Neutral'. It is not recommended to reject the message based solely on this result. 'TempError' A transient error occurred while performing SPF check. The proper action in this case is to accept or temporarily reject the message. If you choose the latter, we suggest to use SMTP reply code of '451' and DSN code '4.4.3', for example: tempfail 451 4.4.3 "Transient error while performing SPF verification" 'PermError' This result means that the domain's published records could not be correctly interpreted. This signals an error condition that requires manual intervention to be resolved, as opposed to the 'TempError' result. The following example illustrates the use of SPF verification in 'envfrom' handler: #include_once require spf prog envfrom do switch check_host($client_addr, domainpart($f), $f, $s) do case Fail: string text "" if spf_explanation != "" set text "%text\n%spf_explanation" fi reject 550 5.7.1 "SPF MAIL FROM check failed: %text" case Pass: accept case TempError: tempfail 451 4.4.3 "Transient error while performing SPF verification" default: on poll $f do when success: accept when not_found or failure: reject 550 5.1.0 "Sender validity not confirmed" when temp_failure: tempfail 450 4.7.0 "Temporary failure during sender verification" done done done The SPF support is implemented in MFL in two layers: a built-in layer that provides basic support, and a library layer that provides a convenience wrapper over the library function. The library layer is implemented in the module 'spf.mf' (*note Modules::). The rest of this node describes available SPF functions and variables. -- Built-in Function: number spf_check_host (string IP, string DOMAIN, string SENDER, string HELO_DOMAIN, string MY_DOMAIN) This function is the basic implementation of the 'check_host' function, defined in RFC 4408, chapter 4. It fetches SPF records, parses them, and evaluates them to determine whether a particular host (IP) is or is not permitted to send mail from a given email address (SENDER). The function returns an "SPF result code". Arguments are: IP The IP address of the SMTP client that is emitting the mail. Usually it is '$client_addr'. DOMAIN The domain that provides the sought-after authorization information; Normally it is the domain portion of the 'MAIL FROM' or 'HELO' identity. SENDER The 'MAIL FROM' identity. HELO_DOMAIN The 'HELO' identity. MY_DOMAIN The SMTP domain served by the local server. Before returning the 'spf_check_host' function stores additional information in global variables: 'spf_explanation' If the result code is 'Fail', this variable contains the explanation string as returned by the publishing domain, prefixed with the value of the global variable 'spf_explanation_prefix'. For example, if 'spf_explanation_prefix' contains 'The domain %{o} explains: ', and the publishing domain 'example.com' returns the explanation string 'Please see http://www.example.com/mailpolicy.html', than the value of 'spf_explanation' will be: The domain example.com explains: Please see http://www.example.com/mailpolicy.html (see RFC 4408 (http://tools.ietf.org/html/rfc4408), chapter 8, for the description of SPF macro facility). 'spf_mechanism' Name of the SPF mechanism that decided about the result code of the SPF record. If one or more 'include' or 'redirect' mechanisms were traversed before arriving at that mechanism, their values are appended in the reverse order. -- Built-in Function: number spf_test_record (string RECORD, string IP, string DOMAIN, string SENDER, string HELO_DOMAIN, string MY_DOMAIN) Evaluate SPF record RECORD as if it were published by DOMAIN. The rest of arguments are the same as for 'spf_check_host' above. This function is designed primarily for testing and debugging purposes. You would hardly need to use it. The 'spf_test_record' function sets the same global variables as 'spf_check_host'. -- Library Function: number check_host (string IP, string DOMAIN, string SENDER, string HELO) This function implements the 'check_host' function, defined in RFC 4408, chapter 4. It fetches SPF records, parses them, and evaluates them to determine whether a particular host (IP) is or is not permitted to send mail from a given email address (SENDER). The function returns an "SPF result code". This function is a wrapper over the built-in 'spf_check_host'. The arguments are: IP The IP address of the SMTP client that is emitting the mail. Usually it is the same as the value of '$client_addr'. DOMAIN The domain that provides the sought-after authorization information; Normally it is the domain portion of the 'MAIL FROM' or 'HELO' identity. SENDER The 'MAIL FROM' identity. HELO The 'HELO' identity. -- Library Function: string spf_status_string (number CODE) Converts numeric SPF result CODE to its string representation. -- Built-in variable: string spf_explanation If 'check_host' (or 'spf_check_host' or 'spf_test_record') returned 'Fail', this variable contains the explanation string as returned by the publishing domain, prefixed with the value of the global variable 'spf_explanation_prefix'. For example, if 'spf_explanation_prefix' contains 'The domain %{o} explains: ', and the publishing domain 'example.com' returns the explanation string 'Please see http://www.example.com/mailpolicy.html', than the value of 'spf_explanation' will be: The domain example.com explains: Please see http://www.example.com/mailpolicy.html -- Built-in variable: string spf_mechanism Set to the name of a SPF mechanism that decided about the result code of the SPF record. -- Built-in variable: string spf_explanation_prefix The prefix to be appended to the explanation string before storing it in the 'spf_explanation' variable. This string can contain valid SPF macros (see RFC 4408 (http://tools.ietf.org/html/rfc4408), chapter 8), for example: set spf_explanation_prefix "%{o} explains: " The default value is '""' (an empty string). ---------- Footnotes ---------- (1) Although RFC 4408 introduces a special 'SPF' record type for this purpose, it is not yet widely used. As of version 8.8, MFL does not support 'SPF' DNS records.  File: mailfromd.info, Node: DKIM, Next: Sockmaps, Prev: SPF Functions, Up: Library 5.34 DKIM ========= DKIM or "DomainKeys Identified Mail" is an email authentication method that allows recipients to verify if an email was authorized by the owner of the domain that email claims to originate from. It does so by adding a digital signature which is verified using a public key published as a DNS TXT record. For technical details about DKIM, please refer to RFC 6376 (). MFL provides functions for DKIM signing and verification. -- Built-in Function: number dkim_verify (number MSG) Verifies the message MSG (a message descriptor, obtained from a call to 'current_message', 'mailbox_get_message', 'message_from_stream' or a similar function). Return value (constants defined in the 'status' module): DKIM_VERIFY_OK The message contains one or more 'DKIM-Signature' headers and one of them verified successfully. DKIM_VERIFY_PERMFAIL The message contains one or more 'DKIM-Signature' headers, all of which failed to verify. DKIM_VERIFY_TEMPFAIL The message was not signed using DKIM, or the DNS query to obtain the public key failed, or an internal software error occured during verification. The following two global variables are always set upon return from this function: 'dkim_explanation' and 'dkim_explanation_code'. These can be used to clarify the verification result to the end user. Upon successful return, the variable 'dkim_verified_signature' is set to the value of the successfully verified DKIM signature. -- Built-in variable: string dkim_explanation An explanatory message clarifying the verification result. -- Built-in variable: number dkim_explanation_code A numeric code corresponding to the 'dkim_explanation' string. Its possible values are defined in 'status.mf': 'DKIM_EXPL_OK' DKIM verification passed. 'DKIM_EXPL_NO_SIG' No DKIM signature. 'DKIM_EXPL_INTERNAL_ERROR' Internal error 'DKIM_EXPL_SIG_SYNTAX' Signature syntax error 'DKIM_EXPL_SIG_MISS' Signature is missing required tag 'DKIM_EXPL_DOMAIN_MISMATCH' Domain part of the 'i=' tag does not match and is not a subdomain of the domain listed in the 'd=' tag. 'DKIM_EXPL_BAD_VERSION' Incompatible DKIM version listed in the 'v=' tag. 'DKIM_EXPL_BAD_ALGORITHM' Unsupported algorithm. 'DKIM_EXPL_BAD_QUERY' Unsupported query method. 'DKIM_EXPL_FROM' From field not signed. 'DKIM_EXPL_EXPIRED' Ssignature expired. 'DKIM_EXPL_DNS_UNAVAIL' Public key unavailable. 'DKIM_EXPL_DNS_NOTFOUND' Public key not found. 'DKIM_EXPL_KEY_SYNTAX' Key syntax error. 'DKIM_EXPL_KEY_REVOKED' Key revoked. 'DKIM_EXPL_BAD_BODY' Body hash did not verify. 'DKIM_EXPL_BAD_BASE64' Can't decode the content of the 'b=' tag. 'DKIM_EXPL_BAD_SIG' Signature did not verify. -- Built-in variable: string dkim_verified_signature Upon successful return from the 'dkim_verify' function, this variable holds the value of the successfully verified DKIM header. This value is unfolded and all whitespace is removed from it. An example of using the 'dkim_verify' function: require status require dkim prog eom do strint result switch dkim_verify(current_message()) do case DKIM_VERIFY_OK: set result "pass; verified for " . dkim_verified_signature_tag('i') case DKIM_VERIFY_PERMFAIL: set result "fail (%dkim_explanation)" case DKIM_VERIFY_TEMPFAIL: set result "neutral" done header_add("X-Verification-Result", "dkim=%result") done The 'dkim' module defines convenience functions for manipulating with DKIM signatures: -- Library Function: dkim_signature_tag (string SIG, string TAG) Extracts the value of the tag TAG from the DKIM signature SIG. Signature must be normalized by performing the header unwrapping and removing whitespace characters. If the tag was not found, returns empty string, unless TAG is one of the tags listed in the table below. If any of these tags are absent, the following values are returned instead: Tag Default value ------------------------------------------------------------------- c 'simple/simple' q 'dns/txt' i '@' + the value of the 'd' tag. -- Library Function: dkim_verified_signature_tag (string TAG) Returns the value of tag TAG from the 'dkim_verified_signature' variable. -- Built-in Function: void dkim_sign (string D, string S, string KEYFILE, [ string CH, string CB, string HEADERS ]) This function is available only in the 'eom' handler. Signs the current message. _Notice_, that no other modification should be attempted on the message after calling this function. Doing so wold make the signature invalid. Mandatory arguments: D Name of the domain claiming responsibility for an introduction of a message into the mail stream. It is also known as the signing domain identifier (SDID). S The selector name. This value, along with D identifies the location of the DKIM public key necessary for verifying the message. The public key is stored in the DNS TXT record for S._domainkey.D KEYFILE Name of the disk file that keeps the private key for signing the message. The file must be in PEM format. Optional arguments: CH Canonicalization algorithm for message headers. Valid values are: 'simple' and 'relaxed'. 'simple' is the default. CB Canonicalization algorithm for message body. Valid and default values are the same as for CH. HEADERS A colon-separated list of header field names that identify the header fields that must be signed. Optional whitespace is allowed at either side of each colon separator. Header names are case-insensitive. This list must contain at least the 'From' header. It may contain names of headers that are not present in the message being signed. This provides a way to explicitly assert the absence of a header field. For example, if HEADERS contained 'X-Mailer' and that header is not present in the message being signed, but is added by a third party later, the signature verification will fail. Similarly, listing a header field name once more than the actual number of its occurrences in a message allows you to prevent any further additions. For example, if there is a single 'Comments' header field at the time of signing, putting 'Comments:Comments:' in the HEADERS parameter is sufficient to prevent any surplus 'Comments' headers from being added later on. Multiple instances of the same header name are allowed. They mean that multiple occurrences of the corresponding header field will be included in the header hash. When such multiple header occurrences are referenced, they will be presented to the hashing algorithm in the reverse order. E.g. if the HEADER list contained 'Received:Received') and the current message contained three 'Received' headers: Received: A Received: B Received: C then these headers will be signed in the following order: Received: C Received: B The default value for this parameter, split over multiple lines for readability, is as follows: * "From:From:" * "Reply-To:Reply-To:" * "Subject:Subject:" * "Date:Date:" * "To:" * "Cc:" * "Resent-Date:" * "Resent-From:" * "Resent-To:" * "Resent-Cc:" * "In-Reply-To:" * "References:" * "List-Id:" * "List-Help:" * "List-Unsubscribe:" * "List-Subscribe:" * "List-Post:" * "List-Owner:" * "List-Archive" An example of using this function: precious domain "example.org" precious selector "s2048" prog eom do dkim_sign("example.org", "s2048", "/etc/pem/my-private.pem", "relaxed", "relaxed", "from:to:subject") done Note on interation of dkim_sign with MMQ ---------------------------------------- The functions 'header_add' and 'header_insert' (*note Header modification functions::) as well as the 'add' action (*note header manipulation::) cannot interact properly with 'dkim_sign' due to the shortcomings of the Milter API. If any of these was called, 'dkim_sign' will throw the 'e_badmmq' exception with the diagnostics following diagnostics: MMQ incompatible with dkim_sign: OP on H, value V where OP is the operation code ('ADD HEADER' or 'INSERT HEADER'), H is the header name and V is its value. The following example shows one graceful way of handling such exception: prog eom do try do dkim_sign("example.org", "s2048", "/etc/pem/my-private.pem") done catch e_badmmq do # Purge the message modification queue mmq_purge() # and retry dkim_sign("example.org", "s2048", "/etc/pem/my-private.pem") done done *Note Message modification queue::, for a discussion of the message modification queue. * Menu: * Setting up a DKIM record::  File: mailfromd.info, Node: Setting up a DKIM record, Up: DKIM 5.34.1 Setting up a DKIM record ------------------------------- Follow these steps to set up your own DKIM record: 1. Generate the key pair Use the 'openssl genrsa' command. Run: openssl genrsa -out private.pem 2048 The last argument is the size of the private key to generate in bits. 2. Extract the public key openssl rsa -in private.pem -pubout -outform PEM -out public.pem 3. Set up a DKIM record in your domain A DKIM record is a TXT type DNS record that holds the public key part for verifying messages. Its format is defined in RFC 4871(1). The label for this record is composed as follows: S._domainkey.D where D is your domain name, and S is the selector you chose to use. You will use these two values as parameters to the 'dkim_sign' function in your 'eom' handler. E.g. if your domain in 'example.com' and selector is 's2048', then the DKIM TXT record label is 's2048._domainkey.example.com'. The public key file generated in step 2 will have the following contents: -----BEGIN PUBLIC KEY----- BASE64 -----END PUBLIC KEY----- where BASE64 is the key itself in base64 encoding. The minimal DKIM TXT record will be: "v=DKIM1; p=BASE64" The only mandatory "tag" is in fact 'p='. The use of 'v=' is recommended. More tags can be added as needed. In particular, while testing the DKIM support, it is advisable to add the 't=y' tag. ---------- Footnotes ---------- (1)  File: mailfromd.info, Node: Sockmaps, Next: NLS Functions, Prev: DKIM, Up: Library 5.35 Sockmap Functions ====================== Socket map ("sockmap" for short) is a special type of database used in Sendmail and MeTA1. It uses a simple server/client protocol over INET or UNIX stream sockets. The server listens on a socket for queries. The client connects to the server and sends it a query, consisting of a "map name" and a "key" separated by a single space. Both map name and key are sequences of non-whitespace characters. The map name serves to identify the type of the query. The server replies with a response consisting of a "status indicator" and "result", separated by a single space. The result part is optional. For example, following is the query for key 'smith' in map 'aliases': 11:aliases news, A possible reply is: 18:OK root@domain.net, This reply means that the key 'news' was found in the map, and the value corresponding to that key is 'root@domain.net'. The following reply means the key was not found: 8:NOTFOUND, For a detailed description of the sockmap protocol, see *note (smap)Protocol::. The MFL library provides two primitives for dealing with sockmaps. Both primitives become available after requiring the 'sockmap' module. -- Library Function: string sockmap_lookup (number FD, string MAP, string KEY) This function look ups the KEY in the MAP. The FD refers to the sockmap to use. It must be obtained as a result of a previous call to 'open' with the URL of the sockmap as its first argument (*note open: I/O functions.). For example: number fd open("@ unix:///var/spool/meta1/smap/socket") string ret sockmap_query(fd, "aliases", $rcpt_to) if ret matches "OK (.+)" set alias \1 fi close(fd) -- Library Function: string sockmap_single_lookup (string URL, string MAP, string KEY) This function connects to the sockmap identified by the URL, queries for KEY in MAP and closes the connection. It is useful when you need to perform only a single lookup on the sockmap.  File: mailfromd.info, Node: NLS Functions, Next: Syslog Interface, Prev: Sockmaps, Up: Library 5.36 National Language Support Functions ======================================== The "National Language Support" functions allow you to write your scripts in such a way, that any textual messages they display are automatically translated to your native language, or, more precisely, to the language required by your current locale. This section assumes the reader is familiar with the concepts of program "internationalization" and "localization". If not, please refer to *note The Purpose of GNU 'gettext': (gettext)Why, before reading further. In general, internationalization of any MFL script follows the same rules as described in the 'GNU gettext manual'. First of all, you select the program "message domain", i.e. the identifier of a set of translatable messages your script contain. This identifier is then used to select appropriate translation. The message domain is set using 'textdomain' function. For the purposes of this section, let's suppose the domain name is 'myfilter'. All NLS functions are provided in the 'nls' module, which you need to require prior to using any of them. To find translations of textual message to the current locale, the underlying 'gettext' mechanism will look for file 'DIRNAME/LOCALE/LC_MESSAGES/DOMAINNAME.mo', where DIRNAME is the message catalog hierarchy name, LOCALE is the locale name, and DOMAINNAME is the name of the message domain. By default DIRNAME is '/usr/local/share/locale', but you may change it using 'bindtextdomain' function. The right place for this initial NLS setup is in the 'begin' block (*note begin/end::). To summarize all the above, the usual NLS setup will look like: require nls begin do textdomain("myfilter") bindtextdomain("myfilter", "/usr/share/locale"); done For example, given the settings above, and supposing the environment variable 'LC_ALL' is set to 'pl', translations will be looked in file '/usr/share/locale/pl/LC_MESSAGES/myfilter.mo'. Once this preparatory work is done, you can request each message to be translated by using 'gettext' function, or '_' (underscore) macro. For example, the following statement will produce translated textual description for '450' response: tempfail 450 4.1.0 _("Try again later") Of course it assumes that the appropriate 'myfile.mo' file already exists. If it does not, nothing bad happens: in this case the macro '_' (as well as 'gettext' function) will simply return its argument unchanged, so that the remote party will get the textual message in English. The 'mo' files are binary files created from 'po' source files using 'msgfmt' utility, as described in *note Producing Binary MO Files: (gettext)Binaries. In turn, the format of 'po' files is described in *note The Format of PO Files: (gettext)PO Files. -- Built-in Function: string bindtextdomain (string DOMAIN, string DIRNAME) This function sets the base directory of the hierarchy containing message catalogs for a given message domain. DOMAIN is a string identifying the textual domain. If it is not empty, the base directory for message catalogs belonging to domain DOMAIN is set to DIRNAME. It is important that DIRNAME be an absolute pathname; otherwise it cannot be guaranteed that the message catalogs will be found. If DOMAIN is '""', 'bindtextdomain' returns the previously set base directory for domain DOMAIN. The rest of this section describes the NLS functions supplied in the 'nls' module. -- Built-in Function: string dgettext (string DOMAIN, string MSGID) 'dgettext' attempts to translate the string MSGID into the currently active locale, according to the settings of the textual domain DOMAIN. If there is no translation available, 'dgettext' returns MSGID unchanged. -- Built-in Function: string dngettext (string DOMAIN, string MSGID, string MSGID_PLURAL, number N) The 'dngettext' functions attempts to translate a text string into the language specified by the current locale, by looking up the appropriate singular or plural form of the translation in a message catalog, set for the textual domain DOMAIN. *Note Additional functions for plural forms: (gettext)Plural forms, for a discussion of the plural form handling in different languages. -- Library Function: string textdomain (string DOMAIN) The 'textdomain' function sets the current message domain to DOMAIN, if it is not empty. In any case the function returns the current message domain. The current domain is 'mailfromd' initially. For example, the following sequence of 'textdomain' invocations will yield: textdomain("") => "mailfromd" textdomain("myfilter") => "myfilter" textdomain("") => "myfilter" -- Library Function: string gettext (string MSGID) 'gettext' attempts to translate the string MSGID into the currently active locale, according to the settings of the current textual domain (set using 'textdomain' function). If there is no translation available, 'gettext' returns MSGID unchanged. -- Library Function: string ngettext (string MSGID, string MSGID_PLURAL, number N) The 'ngettext' functions attempts to translate a text string into the language specified by the current locale, by looking up the appropriate singular or plural form of the translation in a message catalog, set for the current textual domain. *Note Additional functions for plural forms: (gettext)Plural forms, for a discussion of the plural form handling in different languages.  File: mailfromd.info, Node: Syslog Interface, Next: Debugging Functions, Prev: NLS Functions, Up: Library 5.37 Syslog Interface ===================== The basic means for outputting diagnostic messages is the 'echo' instruction (*note Echo::), which sends its arguments to the currently established logging channel. In daemon mode, the latter is normally connected to syslog, so any echoed messages are sent there with the facility selected in mailfromd configuration and priority 'info'. If you want to send a message to another facility and/or priority, use the 'syslog' function: -- Built-in Function: void syslog (number PRIORITY, string TEXT) Sends TEXT to syslog. The priority argument is formed by ORing the facility and the level values (explained below). The facility level is optional. If not supplied, the currently selected logging facility is used. The facility specifies what type of program is logging the message, and the level indicates its relative severity. The following symbolic facility values are declared in the 'syslog' module: 'LOG_KERN', 'LOG_USER', 'LOG_MAIL', 'LOG_DAEMON', 'LOG_AUTH', 'LOG_SYSLOG', 'LOG_LPR', 'LOG_NEWS', 'LOG_UUCP', 'LOG_CRON', 'LOG_AUTHPRIV', 'LOG_FTP' and 'LOG_LOCAL0' through 'LOG_LOCAL7' The declared severity levels are: 'LOG_EMERG', 'LOG_ALERT', 'LOG_CRIT', 'LOG_ERR', 'LOG_WARNING', 'LOG_NOTICE', 'LOG_INFO' and 'LOG_DEBUG'.  File: mailfromd.info, Node: Debugging Functions, Prev: Syslog Interface, Up: Library 5.38 Debugging Functions ======================== These functions are designed for debugging the MFL programs. -- Built-in Function: void debug (string SPEC) Enable debugging. The value of SPEC sets the debugging level. *Note debugging level specification::, for a description of its format. For compatibility with previous versions, this function is also available under the name 'mailutils_set_debug_level'. -- Built-in Function: number debug_level ([string SRCNAME]) This function returns the debugging level currently in effect for the source module SRCNAME, or the global debugging level, if called without arguments. For example, if the program was started with '--debug='all.trace5;engine.trace8'' option, then: debug_level() => 127 debug_level("engine") => 1023 debug_level("db") => 0 -- Built-in Function: boolean callout_transcript ([boolean VALUE]) Returns the current state of the callout SMTP transcript. The result is 1 if the transcript is enabled and 0 otherwise. The transcript is normally enabled either by the use of the '--transcript' command line option (*note SMTP transcript::) or via the 'transcript' configuration statement (*note transcript: conf-server.). The optional VALUE, supplies the new state for SMTP transcript. Thus, calling 'callout_transcript(0)' disables the transcript. This function can be used in bracket-like fashion to enable transcript for a certain part of MFL program, e.g.: number xstate callout_transcript(1) on poll $f do ... done set xstate callout_transcript(0) Note, that the use of this function (as well as the use of the '--transcript' option) makes sense only if callouts are performed by the 'mailfromd' daemon itself. It will not work if a dedicated callout server is used for that purpose (*note calloutd::). -- Built-in Function: string debug_spec ([string CATNAMES, bool SHOWUNSET]) Returns the current debugging level specification, as given by '--debug' command line option or by the 'debug' configuration statement (*note conf-debug::). If the argument SRCNAMES is specified, it is treated as a semicolon-separated list of categories for which the debugging specification is to be returned. For example, if 'mailfromd' was started with '--debug=all.trace5;spf.trace1;engine.trace8;db.trace0', then: debug_spec() => "all.trace5,engine.trace8" debug_spec("all;engine") => "all.trace5,engine.trace8" debug_spec("engine;db") => "db.trace0;engine.trace8" debug_spec("prog") => "" When called without arguments, 'debug_spec' returns only those categories which have been set, as shown in the first example above. Optional SHOWUNSET parameters controls whether to return unset module specifications. To print all debugging specifications, whether set or not, use debug_spec("", 1) These three functions are intended to complement each other. The calls to 'debug' can be placed around some piece of code you wish to debug, to enable specific debugging information for this code fragment only. For example: /* Save debugging level for 'dns.c' source */ set dlev debug_spec("dns", 1) /* Set new debugging level */ debug("dns.trace8") . . . /* Restore previous level */ debug(dlev) -- Built-in Function: void program_trace (string MODULE) Enable tracing for a set of modules given in MODULE argument. *Note --trace-program::, for a description of its format. -- Built-in Function: void cancel_program_trace (string MODULE) Disable tracing for given modules. This pair of functions is also designed to be used together in a bracket-like fashion. They are useful for debugging 'mailfromd', but are not advised to use otherwise, since tracing slows down the execution considerably. -- Built-in Function: void stack_trace () Generate a stack trace in this point. *Note tracing runtime errors::, for the detailed description of stack traces. The functions below are intended mainly for debugging MFL run-time engine and for use in 'mailfromd' testsuite. You will hardly need to use them in your programs. -- Built-in Function: void _expand_dataseg (number N) Expands the run-time data segment by at least N words. -- Built-in Function: number _reg (number R) Returns the value of the register R at the moment of the call. Symbolic names for run-time registers are provided in the module '_register': Name Register -------------------------------------------------------------------------- REG_PC Program counter REG_TOS Top of stack REG_TOH Top of heap REG_BASE Frame base REG_REG General-purpose accumulator REG_MATCHSTR Last matched string pointer -- Built-in Function: void _wd ([number N]) Enters a time-consuming loop and waits there for N seconds (by default - indefinitely). The intention is to facilitate attaching to 'mailfromd' with the debugger. Before entering the loop, a diagnostic message is printed on the 'crit' facility, informing about the PID of the process and suggesting the command to be used to attach to it, e.g.: mailfromd: process 21831 is waiting for debug mailfromd: to attach: gdb -ex 'set variable mu_wd::_count_down=0' /usr/sbib/mailfromd 21831  File: mailfromd.info, Node: Using MFL Mode, Next: Mailfromd Configuration, Prev: Library, Up: Top 6 Using the GNU Emacs MFL Mode ****************************** MFL sources are usual ASCII files and you may edit them with any editor you like. However, the best choice for this job (as well as for many others) is, without doubt, GNU Emacs. To ease the work of editing script files, the 'mailfromd' package provides a special Emacs mode, called "MFL mode". The elisp source file providing this mode, 'mfl-mode.el', is installed automatically, provided that GNU Emacs is present on your machine. To enable the mode, add the following lines to your Emacs setup file (either system-wide 'site-start.el', or your personal one, '~/.emacs'): (autoload 'mfl-mode "mfl-mode") (setq auto-mode-alist (append auto-mode-alist '(("/etc/mailfromd.mf" . mfl-mode) ("\\.mf$" . mfl-mode)))) The first directive loads the MFL mode, and the second one tells Emacs to apply it to any file whose name ends in '/etc/mailfromd.mf'(1) or in a '.mf' suffix. MFL mode provides automatic indentation and syntax highlighting for MFL sources. The default indentation setup is the same as the one used throughout this book: * Handler and function definitions start at column 1; * A block statement, i.e. 'do', 'done', 'if', 'else', 'elif' and 'fi', occupies a line by itself, with the only exception that 'do' after an 'on' statement is located on the same line with it; * A 'do' statement that follows function or handler definition is placed in column 1. * Each subsequent level of nesting is indented two columns to the right (*note mfl-basic-offset::). * A closing statement ('done', 'else', 'elif', 'fi') is placed at the same column as the corresponding opening statement; * Branch statements ('case' and 'when') are placed in the same column as their controlling keyword ('switch' and 'on', correspondingly (*note mfl-case-line-offset::). * Loop substatements (*note Loops::) are offset 5 columns to the right from the controlling 'loop' keyword. (*note mfl-loop-statement-offset::). Continuation statements within loop header are offset 5 columns from the indentation of their controlling keyword, either 'for' or 'while' (*note mfl-loop-continuation-offset::). The mode provides two special commands that help navigate through the complex filter scripts: 'C-M-a' Move to the beginning of current function or handler definition. 'C-M-e' Move to the end of current function or handler definition. Here, "current function or handler" means the one within which your cursor currently stays. You can use 'C-M-e' repeatedly to walk through all function and handler definitions in your script files. Similarly, repeatedly pressing 'C-M-a' will visit all the definitions in the opposite direction (from the last up to the very first one). Another special command, 'C-c C-c', allows to verify the syntax of your script file. This command runs 'mailfromd' in syntax check mode (*note Testing Filter Scripts::) and displays its output in a secondary window, which allows to navigate through eventual diagnostic messages and to jump to source locations described by them. All MFL mode settings are customizable. To change any of them, press 'M-x customize' and visit 'Environment/Unix/Mfl' customization group. This group offers two subgroups: 'Mfl Lint group' and 'Mfl Indentation group'. 'Mfl Lint group' controls invocation of mailfromd by 'C-c C-c'. This group contains two variables: -- MFL-mode setting: mfl-mailfromd-command The 'mailfromd' to be invoked. By default, it is 'mailfromd'. You will have to change it, if 'mailfromd' cannot be found using 'PATH' environment variable, or if you wish to pass it some special options. However, do not include '--lint' or '-I' options in this variable. The '--lint' option is given automatically, and include paths are controlled by 'mfl-include-path' variable (see below). -- MFL-mode setting: mfl-include-path A list of directories to be appended to 'mailfromd' include search path (*note include search path::). By default it is empty. 'Mfl Indentation group' controls automatic indentation of MFL scripts. This group contains the following settings: -- MFL-mode setting: mfl-basic-offset This variable sets the basic indentation increment. It is set to 2, by default, which corresponds to the following indentation style: prog envfrom do if $f = "" accept else ... fi done -- MFL-mode setting: mfl-case-line-offset Indentation offset for 'case' and 'when' lines, relative to the column of their controlling keyword. The default is 0, i.e.: switch x do case 0: ... default: ... done -- MFL-mode setting: mfl-returns-offset Indentation offset of 'returns' and 'alias' statements, relative to the controlling 'func' keyword. The default value is 2, which corresponds to: func foo() alias bar returns string -- MFL-mode setting: mfl-comment-offset Indentation increment for multi-line comments. The default value is 1, which makes: /* first comment line second comment line */ -- MFL-mode setting: mfl-loop-statement-offset Indentation increment for parts of a 'loop' statement. The default value is 5, which corresponds to the following style: loop for STMT, while COND, INCR do -- MFL-mode setting: mfl-loop-continuation-offset If any of the 'loop' parts occupies several lines, the indentation of continuation lines relative to the first line is controlled by 'mfl-loop-continuation-offset', which defaults to 5: loop for set n 0 set z 1, while n != 10 or z != 2, set n n + 1 ---------- Footnotes ---------- (1) This will match most existing installations. In the unlikely case that your '$sysconfdir' does not end in '/etc', you will have to edit the directive accordingly.  File: mailfromd.info, Node: Mailfromd Configuration, Next: Invocation, Prev: Using MFL Mode, Up: Top 7 Configuring 'mailfromd' ************************* Upon startup, 'mailfromd' checks if the file '/etc/mailfromd.conf' exists.(1) If it does, the program attempts to retrieve its configuration settings from that file. The 'mailfromd.conf' file must be written in the "GNU mailutils configuration format", as described in *note Configuration File Syntax: (mailutils)conf-syntax. This format can be summarized as follows: Comments Inline comments begin with '//' or '#' and end at the end of the line. Multiline comments are delimited by '/*' and '*/'. Multiline comments cannot be nested, but can contain inline comment markers. Empty lines and whitespace Empty lines are ignored. Whitespace characters (i.e. horizontal, vertical space, and newline) are ignored, except as they serve to separate tokens. Statements A statement consists of a keyword and a value, separated by whitespace. Statements terminate with a semicolon. E.g. pidfile /var/run/mailfromd.pid; Block statements A block statement consists of a keyword and a list of statements enclosed in '{' and '}' characters. Optional "label" can appear between the keyword and opening curly brace. E.g.: logging { syslog on; facility mail; } Block statement is not required to terminate with a semicolon, although it is allowed to. File Inclusion The 'include' statement causes inclusion of the file listed as its value: include /usr/share/mailfromd/config.inc; The 'mailfromd.conf' file is used by all programs that form the 'mailfromd' package, i.e. 'mailfromd', 'calloutd', 'mfdbtool', and 'pmult'. Since the sets of statements understood by each of them differ, special syntactic means are provided to separate program-specific configurations from each other. First of all, if the argument to 'include' is a directory, then the program will search that directory for a file with the same name as the base name of the program itself. If found, this file will be loaded after finishing parsing the 'mailfromd.conf' file. Otherwise, this statement is ignored. Secondly, the special block statement 'program TAG' is processed only if TAG matches the base name of the program being run. Again, it is processed after the main 'mailfromd.conf' file. Thus, if you need to provide configuration for the 'calloutd' component, there are two ways of doing so. First, you can place it to a file named 'calloutd' placed in a separate directory (say, '/etc/mailfromd.d'), and use the name of that directory in a 'include' statement in the main configuration file: include /etc/mailfromd.d; Secondly, you can use the 'program' statement as follows: program calloutd { ... } * Menu: * conf-types:: Special Configuration Data Types * conf-base:: Base Mailfromd Configuration * conf-server:: Server Configuration * conf-milter:: Milter Connection Configuration * conf-debug:: Logging and Debugging configuration * conf-timeout:: Timeout Configuration * conf-callout:: Call-out Configuration * conf-priv:: Privilege Configuration * conf-database:: Database Configuration * conf-runtime:: Runtime Constants * conf-mailutils:: Standard Mailutils Statements ---------- Footnotes ---------- (1) The exact location is determined at compile time: the '/etc' directory is the system configuration directory set when building 'mailfromd' (*note Building::).  File: mailfromd.info, Node: conf-types, Next: conf-base, Up: Mailfromd Configuration 7.1 Special Configuration Data Types ==================================== In addition to the usual data types (*note (mailutils)Statements::), 'mailfromd' configuration introduces the following two special ones: time interval specification The "time interval specification" is a string that defines an interval, much the same way we do this in English: it consists of one or more pairs 'number'-'time unit'. For example, the following are valid interval specifications: 1 hour 2 hours 35 seconds 1 year 7 months 2 weeks 2 days 11 hours 12 seconds The pairs can occur in any order, however unusual it may sound to a human ear, e.g. '2 days 1 year'. If the 'time unit' is omitted, seconds are supposed. Connection URL unix://FILE unix:FILE local://FILE local:FILE A named pipe (socket). inet://ADDRESS:PORT inet:PORT@ADDRESS An IPv4 connection to host ADDRESS at port PORT. Port must be specified either as a decimal number or as a string representing the port name in '/etc/services'. inet6:PORT@ADDRESS An IPv6 connection to host ADDRESS at port PORT. This port type is not yet supported.  File: mailfromd.info, Node: conf-base, Next: conf-server, Prev: conf-types, Up: Mailfromd Configuration 7.2 Base Mailfromd Configuration ================================ -- Mailfromd Conf: script-file file Read filter script from FILE. By default it is read from 'SYSCONFDIR/mailfromd.mf'. -- Mailfromd Conf: setvar name value Initialize MFL variable NAME to VALUE. -- Mailfromd Conf: include-path path Add directories to the list of directories to be searched for header files. *Note include search path::. Argument is a list of directory names separated by colons. -- Mailfromd Conf: state-directory dir Set program state directory. *Note statedir::. -- Mailfromd Conf: relayed-domain-file file Append domain names from the named FILE to the list of relayed domains. This list can be inspected from MFL script using the 'relayed' function (*note relayed::). The FILE argument is either a single file name or a list of file names, e.g.: relayed-domain-file /etc/mail/sendmail.cw; relayed-domain-file (/etc/mail/sendmail.cw, /etc/mail/relay-domains); -- Mailfromd Conf: source-ip ipaddr Set source IP address for outgoing TCP connections. -- Mailfromd Conf: pidfile file Set the name of the file to store PID value in. The file must be writable for the user or group 'mailfromd' runs as (*note conf-priv::).  File: mailfromd.info, Node: conf-server, Next: conf-milter, Prev: conf-base, Up: Mailfromd Configuration 7.3 Server Configuration ======================== A single 'mailfromd' daemon can run several "servers". These are configured in the following statement: server TYPE { id NAME; listen URL; backlog NUM; max-instances NUM; single-process BOOL; reuseaddr BOOL; option LIST; default BOOL; callout URL; acl { ... } } -- Mailfromd Conf: server TYPE Define a server. The TYPE is either 'milter' or 'callout'. *Note SMTP Timeouts::, for a description of various types of servers. The substatements in the 'server' block provide parameters for configuring this server. -- server: id name Assign an identifier to this server. This identifier is used as a suffix to syslog tag (*note syslog tag::) in messages related to this server. For example, if a server block had the following statement in it: id main; then all messages related to this server will be marked with tag 'mailfromd#main'. The part before the '#' is set using the 'tag' statement in 'logging' block (*note Mailutils Configuration File: (mailutils)Logging Statement.). -- server: listen url Listen for connections on the given URL. *Note milter port specification::, for a description of allowed URL formats. Example: listen inet://10.10.10.1:3331; -- server: backlog NUM Configures the size of the queue of pending connections. Default value is 8. -- server: max-instances number Sets the maximum number of instances allowed for this server. -- server: single-process bool When set to 'yes', this server will run in "single-process" mode, i.e. it will not fork sub-processes to serve requests. This option is meant exclusively to assist in debugging 'mailfromd'. Don't use it for anything else but for debugging! -- server: reuseaddr bool When set to 'yes', 'mailfromd' will attempt to reuse existing socket addresses. This is the default behavior. If the server TYPE is 'callout', the following statement is also allowed: -- server: option LIST Configures server options. As of version 8.8 only one option is defined: default Mark this server as the default one. This means it will be used by every milter server that doesn't define the 'callout-url' statement. -- server: default bool When set to 'yes', this server is marked as a default callout server for all milter servers declared in the configuration. This is equivalent to 'option default'. if the server TYPE is 'milter', you can use the following statement to query a remote callout server: -- server: callout url Use a callout server at URL (*note milter port specification::). You can also set a "global callout server", which will be used by all milter servers that do not set the 'callout' statement: -- Mailfromd Conf: callout-url url Set global callout server. *Note milter port specification::, for allowed URL formats.  File: mailfromd.info, Node: conf-milter, Next: conf-debug, Prev: conf-server, Up: Mailfromd Configuration 7.4 Milter Connection Configuration =================================== -- Mailfromd Conf: milter-timeout time Sets the timeout value for connection between the filter and the MTA. Default value is 7210 seconds. You normally do not need to change this value. -- Mailfromd Conf: acl This block statement configures "access control list" for incoming Milter connections. *Note Mailutils Configuration File: (mailutils)ACL Statement, for a description of its syntax. E.g.: acl { allow from 10.10.10.0/24; deny from any; }  File: mailfromd.info, Node: conf-debug, Next: conf-timeout, Prev: conf-milter, Up: Mailfromd Configuration 7.5 Logging and Debugging configuration ======================================= -- Mailfromd Conf: logger mech Set default logger mechanism. Allowed values for MECH are: stderr Log everything to the standard error. syslog Log to syslog. syslog:async Log to syslog using the asynchronous syslog implementation. *Note Logging and Debugging::, for a detailed discussion. See also *note Using non-blocking syslog: syslog-async, for information on how to set default syslog implementation at compile time. -- Mailfromd Conf: debug spec Set 'mailfromd' debug verbosity level. The SPEC must be a valid debugging level specification (*note debugging level specification::). -- Mailfromd Conf: stack-trace bool Enables stack trace dumps on runtime errors. This feature is useful for locating the source of an error, especially in complex scripts. *Note tracing runtime errors::, for a detailed description. -- Mailfromd Conf: trace-actions bool Enable action tracing. If BOOL is 'true', 'mailfromd' will log all executed actions. *Note Logging and Debugging::, for a detailed description of action tracing. -- Mailfromd Conf: trace-program modlist Enable program instruction tracing for modules in MODLIST, a comma-separated list of source code modules, e.g.: trace-program (bi_io,bi_db); This statement enables tracing for functions from modules 'bi_io.c' and 'bi_db.c' (notice, that you need not give file suffixes). This tracing is useful for debugging 'mailfromd', but is not advised to use otherwise, since it is very time-costly. -- Mailfromd Conf: transcript bool Enable transcripts of call-out SMTP sessions. *Note SMTP transcript::, for a detailed description of SMTP transcripts.  File: mailfromd.info, Node: conf-timeout, Next: conf-callout, Prev: conf-debug, Up: Mailfromd Configuration 7.6 Timeout Configuration ========================= The SMTP timeouts used in callout sessions are configured via 'smtp-timeout' statement: Syntax ------ smtp-timeout TYPE { connection INTERVAL; initial-response INTERVAL; helo INTERVAL; mail INTERVAL; rcpt INTERVAL; rset INTERVAL; quit INTERVAL; } -- Mailfromd Conf: smtp-timeout TYPE Declare SMTP timeouts of the given TYPE, which may be 'soft' or 'hard'. Callout SMTP sessions initiated by polling functions, are controlled by two sets of timeouts: 'soft' and 'hard'. "Soft timeouts" are used by the mailfromd milter servers. "Hard timeouts" are used by callout servers (*note callout server::). When a soft timeout is exceeded, the calling procedure is delivered an 'e_temp_failure' exception and the session is scheduled for processing by a callout server. The latter re-runs the session using hard timeouts. If a hard timeout is exceeded, the address is marked as 'not_found' and is stored in the cache database with that status. Normally, soft timeouts are set to shorter values, suitable for use in MFL scripts without causing excessive delays. Hard timeouts are set to large values, as requested by RFC 2822 and guarantee obtaining a definite answer (see below for the default values). Statements ---------- The TIME argument for all 'smtp-timeout' sub-statements is expressed in time interval units, as described in *note time interval specification::. -- smtp-timeout: connection time Sets initial connection timeout for callout tests. If the connection is not established within this time, the corresponding callout function returns temporary failure. -- smtp-timeout: initial-response time Sets the time to wait for the initial SMTP response. -- smtp-timeout: helo time Timeout for a response to 'HELO' (or 'EHLO') command. -- smtp-timeout: mail time Timeout for a response to 'MAIL' command. -- smtp-timeout: rcpt time Timeout for a response to 'RCPT' command. -- smtp-timeout: rset time Timeout for a response to 'RSET' command. -- smtp-timeout: quit time Timeout for a response to 'QUIT' command. Default Values -------------- The default timeout settings are: Timeout Soft Hard --------------------------------------------------------------------------- connection 10s 5m initial-response 30s 5m helo I/O 5m mail I/O 10m rcpt I/O 5m rset I/O 5m quit I/O 2m Table 7.1: Default SMTP timeouts -- Mailfromd Conf: io-timeout time Sets a general SMTP I/O operation timeout. This timeout is used as the default for entries marked with 'I/O' in the above table. The default is 3 seconds.  File: mailfromd.info, Node: conf-callout, Next: conf-priv, Prev: conf-timeout, Up: Mailfromd Configuration 7.7 Call-out Configuration ========================== -- Mailfromd Conf: ehlo-domain string Sets default domain used in 'EHLO' (or 'HELO') SMTP command when probing the remote host. This value can be overridden by 'from' parameter to 'poll' command (*note poll::). This statement assigns the value STRING to the 'ehlo_domain' variable (*note ehlo_domain::), and is therefore equivalent to setvar ehlo_domain STRING; -- Mailfromd Conf: mail-from-address string Sets default email addresses used in 'MAIL FROM:' SMTP command when probing the remote host. This value can be overridden by 'as' parameter to 'poll' command (*note poll::). This statement assigns the value STRING to the 'mailfrom_address' variable (*note mailfrom_address::), and is therefore equivalent to setvar mailfrom_address STRING; -- Mailfromd Conf: enable-vrfy bool Enables the use of SMTP VRFY statement prior to normal callout sequence. If VRFY is supported by the remote server, 'mailfromd' relies on its reply and does not perform normal callout. The use of this statement is not recommended, because many existing VRFY implementations always return affirmative result, no matter is the requested email handled by the server or not. The default is 'enable-vrfy no', i.e. VRFY is disabled.  File: mailfromd.info, Node: conf-priv, Next: conf-database, Prev: conf-callout, Up: Mailfromd Configuration 7.8 Privilege Configuration =========================== -- Mailfromd Conf: user name Switch to this user's privileges after startup. *Note Starting and Stopping::, for a discussion of the privileges 'mailfromd' runs under and the options that affect them. See also 'group' below. -- Mailfromd Conf: group name Retain the supplementary group NAME when switching to user privileges. By default 'mailfromd' clears the list of supplementary groups when switching to user privileges, but this statement allows to retain the given group. It can be specified multiple times to retain several groups. This option may be necessary to maintain proper access rights for various files. *Note Starting and Stopping::.  File: mailfromd.info, Node: conf-database, Next: conf-runtime, Prev: conf-priv, Up: Mailfromd Configuration 7.9 Database Configuration ========================== Syntax ------ database DBNAME { file NAME; enable BOOL; expire-interval INTERVAL; positive-expire-interval INTERVAL; negative-expire-interval INTERVAL; } -- Mailfromd Conf: database dbname The 'database' statement controls run-time parameters of a DBM database identified by DBNAME. Allowed values for the latter are: 'cache', 'rate' and 'greylist' for main cache, DNS lookup, sending rate and greylisting databases, correspondingly. Statements ---------- -- database: file name Set the database file name. -- database: enable bool Enable or disable this database. -- database: expire-interval time Set the expiration interval for this database DBNAME. *Note time interval specification::, for a description of TIME format. -- database: positive-expire-interval time This statement is valid only for 'cache' database. It sets the expiration interval for positive ('success') cache entries. -- database: negative-expire-interval time This statement is valid only for 'cache' database, where it sets expiration interval for negative ('not_found') cache entries. Additional Statements --------------------- -- Mailfromd Conf: database-type TYPE Set default database type. TYPE is one of the database types supported by mailutils (i.e., for Mailutils 3.0: 'gdbm', 'ndbm', 'bdb', 'kc', and 'tc'). Run mailfromd --show-defaults | grep 'supported databases:' to get a list of type names supported by your build of 'mailfromd'. -- Mailfromd Conf: database-mode MODE Defines file mode for newly created database files. MODE must be a valid file mode in octal. -- Mailfromd Conf: lock-retry-count number Set maximum number of attempts to acquire the lock. The time between each two successive attempts is given by 'lock-retry-timeout' statement (see below). After the NUMBER of failed attempts, 'mailfromd' gives up. -- Mailfromd Conf: lock-retry-timeout time Set the time span between the two locking attempts. Any valid time interval specification (*note time interval specification::) is allowed as argument.  File: mailfromd.info, Node: conf-runtime, Next: conf-mailutils, Prev: conf-database, Up: Mailfromd Configuration 7.10 Runtime Constants Configuration ==================================== -- Mailfromd Conf: runtime { statements } The statements in the 'runtime' section configure various values used by MFL builtin functions. -- runtime: max-streams number Sets the maximum number of stream descriptors that can be opened simultaneously. Default is 1024. *Note I/O functions::. -- runtime: max-open-mailboxes number Sets the maximum number of available mailbox descriptors. This value is used by MFL mailbox functions (*note Mailbox functions::). -- runtime: max-open-messages number Sets the maximum number of messages that can be opened simultaneously using the 'mailbox_get_message' function. *Note Message functions::, for details.  File: mailfromd.info, Node: conf-mailutils, Prev: conf-runtime, Up: Mailfromd Configuration 7.11 Standard Mailutils Statements ================================== The following standard Mailutils statements are understood: Statement Reference ------------------------------------------------------------------- auth *Note Mailutils Configuration File: (mailutils)auth statement. debug *Note Mailutils Configuration File: (mailutils)debug statement. include *Note Mailutils Configuration File: (mailutils)include. logging *Note Mailutils Configuration File: (mailutils)logging statement. mailer *Note Mailutils Configuration File: (mailutils)mailer statement. locking *Note Mailutils Configuration File: (mailutils)locking statement.  File: mailfromd.info, Node: Invocation, Next: MTA Configuration, Prev: Mailfromd Configuration, Up: Top 8 'Mailfromd' Command Line Syntax ********************************* The 'mailfromd' binary is started from the command line using the following syntax (brackets indicate optional parts): $ mailfromd [OPTIONS] [ASGN] [SCRIPT] The meaning of each invocation part is described in the table below: OPTIONS The command line options (*note options::). ASGN Sendmail macro assignments. These are currently meaningful only with the '--test' option (*note test mode::), but this may change in the future. Each assignment has the form: VAR=VALUE where VAR is the name of a Sendmail macro and VALUE is the value to be assigned to it. SCRIPT The file name of the filter script, if other than the default one. * Menu: * options:: Command Line Options. * Starting and Stopping:: How to Start and Shut Down the Daemon.  File: mailfromd.info, Node: options, Next: Starting and Stopping, Up: Invocation 8.1 Command Line Options. ========================= * Menu: * Operation Modifiers:: * General Settings:: * Preprocessor Options:: * Timeout Control:: * Logging and Debugging Options:: * Informational Options::  File: mailfromd.info, Node: Operation Modifiers, Next: General Settings, Up: options 8.1.1 Operation Modifiers ------------------------- '--daemon' Run in daemon mode (default). '--run[=START]' Load the script named in the command line and execute the function named START, or 'main', if START is not given. *Note Run Mode::, for a detailed description of this feature. '--show-defaults' Show compilation defaults. *Note Databases::. '-t[STATE]' '--test[=STATE]' Run in test mode. *Note Testing Filter Scripts::. Default STATE is 'envfrom'. This option implies '--stderr' (*note --stderr::).  File: mailfromd.info, Node: General Settings, Next: Preprocessor Options, Prev: Operation Modifiers, Up: options 8.1.2 General Settings ---------------------- '--callout-socket=STRING' Set socket for the default callout server. This is mainly useful together with the '--mtasim' option. '--foreground' Stay in foreground. When given this option, 'mailfromd' will not disconnect itself from the controlling terminal and will run in the foreground. '-g NAME' '--group=NAME' Retain the group NAME when switching to user privileges. *Note Starting and Stopping::. This option complements the 'group' configuration statement (*note group: conf-priv.). '--include=DIR' '-I DIR' Add the directory DIR to the list of directories to be searched for header files. This will affect the functioning of '#include' statement. *Note include::, for a discussion of file inclusion. '--mailer=URL' '-M URL' Set the URL of the mailer to use. *Note Mail Sending Functions::. '--mtasim' This option is reserved for use by 'mtasim' (*note mtasim::). '-O[LEVEL]' '--optimize[=LEVEL]' Set optimization level for code generator. Two levels are implemented: '0', meaning no optimization, and '1', meaning full optimization. '-p STRING' '--port=STRING' '--milter-socket=STRING' Set communication socket. Overrides the 'listen' configuration statement, which you are advised to use instead (*note listen: conf-milter.). '--pidfile=FILE' Set pidfile name. Overrides the 'pidfile' configuration statement, which you are advised to use instead (*note pidfile: conf-base.). '--relayed-domain-file=FILE' Read relayed domains from FILE. Overrides the 'relayed-domain-file' configuration statement (*note relayed-domain-file: conf-base.), which you are advised to use instead. *Note Avoiding Verification Loops::, and the description of 'relayed' function (*note relayed::) for more information. '--resolv-conf-file=FILE' Read resolver settings from FILE, instead of the default '/etc/resolv.conf'. '--state-directory=DIR' Set new program state directory. *Note Local state directory: statedir, for the description of this directory and its purposes. This option overrides the settings of 'state-directory' configuration statement, described in *note state-directory: conf-base. '-S IP' '--source-ip=IP' Set source address for TCP connections. Overrides the 'source-ip' configuration statement, which you are advised to use instead (*note source-ip: conf-base.). '-u NAME' '--user NAME' Switch to this user privileges after startup. Overrides the 'user' configuration file statement, which you are advised to use instead (*note user: conf-priv.). Default user is 'mail'. '-v VAR=VALUE' '--variable VAR=VALUE' Assign VALUE to the global variable VAR. The variable must be declared in your startup script. *Note overriding initial values::, for a detailed discussion of this option.  File: mailfromd.info, Node: Preprocessor Options, Next: Timeout Control, Prev: General Settings, Up: options 8.1.3 Preprocessor Options -------------------------- Following command line options control the preprocessor feature. *Note Preprocessor::, for a detailed discussion of these. '--no-preprocessor' Do not run the preprocessor. '--preprocessor=COMMAND' Use COMMAND as the external preprocessor instead of the default 'm4'. '-D NAME[=VALUE]' '--define=NAME[=VALUE]' Define a preprocessor symbol NAME to have a value VALUE. '-U NAME' '--undefine=NAME' Undefine the preprocessor symbol NAME. '-E' Stop after the preprocessing stage; do not run the compiler proper. The output is in the form of preprocessed source code, which is sent to the standard output.  File: mailfromd.info, Node: Timeout Control, Next: Logging and Debugging Options, Prev: Preprocessor Options, Up: options 8.1.4 Timeout Control --------------------- *Note time interval specification::, for information on INTERVAL format. '--milter-timeout=INTERVAL' Set MTA connection timeout. Overrides 'milter-timeout' statement in the 'mailfromd' configuration file, which you are advised to use instead (*note milter-timeout: conf-milter.).  File: mailfromd.info, Node: Logging and Debugging Options, Next: Informational Options, Prev: Timeout Control, Up: options 8.1.5 Logging and Debugging Options ----------------------------------- '--location-column' '--no-location-column' Mention column number in error messages. *Note location-column: Testing Filter Scripts. Use '--no-location-column' to disable '-d STRING' '--debug=STRING' Set debugging level. *Note Logging and Debugging::. '--dump-code' Parse and compile the script file and dump the disassembled listing of the produced code to the terminal. *Note Logging and Debugging::. '--dump-grammar-trace' Enable debugging the script file parser. While parsing the file, the detailed dump of the parser states and tokens seen will be output. '--dump-lex-trace' Enable debugging the lexical analyzer. While parsing the script file, the detailed dump of the lexer states and matched rules will be output. '--dump-macros' Show Sendmail macros used in the script file. The macro names are displayed as comma-separated lists, grouped by handler names. *Note Sendmail::, for a detailed description of this option and its usage. '--dump-tree' Parse and compile the script file and dump the parse tree in a printable form to the terminal. '--dump-xref' Print a cross-reference of variables used in the filter script. *Note Testing Filter Scripts::. '-E' Stop after the preprocessing stage; do not run the compiler proper. The output is in the form of preprocessed source code, which is sent to the standard output. *Note Preprocessor::. '--lint' Check script file syntax and exit. If the file is OK, return 0 to the shell, otherwise print appropriate messages to stderr and exit with code 78 ('configuration error'). '--single-process' Do not fork sub-processes to serve requests. This option is meant to assist in debugging 'mailfromd'. Don't use it for anything else but for debugging, as it terribly degrades performance! '--stack-trace' '--no-stack-trace' Add MFL stack trace information to runtime error output. Overrides 'stack-trace' configuration statement. Use the '--no-stack-trace' to disable trace information. *Note tracing runtime errors::, for more information on this feature. '--gacopyz-log=LEVEL' Set desired logging level for 'gacopyz' library (*note Gacopyz::). There are five logging levels. The following table lists them in order of decreasing priority: fatal Log fatal errors. err Log error messages. warn Log warning messages. info Log informational messages. In particular, this enables printing messages on each subprocess startup and termination, which look like that: Apr 28 09:00:11 host mailfromd[9411]: connect from 192.168.10.1:50398 Apr 28 09:00:11 host mailfromd[9411]: finishing connection This level can be useful for debugging your scripts. debug Log debugging information. proto Log Milter protocol interactions. This level prints huge amounts of information, in particular it displays dumps of each Milter packet sent and received. Although it is possible to set these levels independently of each other, it is seldom practical. Therefore, the option '--gacopyz-log=LEVEL' enables all logging levels from LEVEL up. For example, '--gacopyz-log=warn' enables log levels 'warn', 'err' and 'fatal'. It is the default. If you need to trace each subprocess startup and shutdown, set '--gacopyz-log=info'. Setting the logging level to 'proto' can be needed only for 'Gacopyz' developers, to debug the protocol. *Note Testing Filter Scripts::. '--logger=MECH' Set logger mechanism (MECH is one of 'stderr', 'syslog', 'syslog:async'). *Note Logging and Debugging::. '--log-facility=FACILITY' Output logs to syslog FACILITY. '--log-tag=STRING' Tag syslog entries with the given STRING, instead of the program name. '--source-info' '--no-source-info' Include C source information in debugging messages. This is similar to setting 'line-info yes' in the 'debug' configuration block (*note line-info: (mailutils)debug statement.). The '--no-source-info' can be used to cancel the effect of the 'line-info yes' configuration statement. You do not need this option, unless you are developing or debugging 'mailfromd'. '--syntax-check' Synonym for '--lint'. '--trace' '--no-trace' Enable or disable action tracing. If '--trace' is given, 'mailfromd' will log all executed actions. *Note Logging and Debugging::. '--trace-program[=STRING]' Enable program instruction tracing. With this option 'mailfromd' will log execution of every instruction in the compiled filter program. The optional arguments allows to specify a comma-separated list of source code modules for which the tracing is to be enabled, for example '--trace-program=bi_io,bi_db' enables tracing for functions from modules 'bi_io.c' and 'bi_db.c' (notice, that you need not give file suffixes in STRING). This option is useful for debugging 'mailfromd', but is not advised to use otherwise, since it is very time-costly. '-X' '--transcript' '--no-transcript' Enable or disable transcript of the SMTP sessions to the log channel. *Note Logging and Debugging::. '--syslog' Selects default syslog mechanism for diagnostic output. '--stderr' Directs all logging to standard output. Similar to '--logger=stderr'. '--xref' Same as '--dump-xref'. *Note Logging and Debugging::.  File: mailfromd.info, Node: Informational Options, Prev: Logging and Debugging Options, Up: options 8.1.6 Informational Options --------------------------- '-?' '--help' Give a short help summary. '--usage' Give a short usage message. '-V' '--version' Print program version.  File: mailfromd.info, Node: Starting and Stopping, Prev: options, Up: Invocation 8.2 Starting and Stopping ========================= Right after startup, when 'mailfromd' has done the operations that require root privileges, it switches to the privileges of the user it is configured to run as (*note default user privileges::) or the one given in its configuration file (*note user: conf-priv.). During this process it will drop all supplementary groups and switch to the principal group of that user. Such limited privileges of the daemon can cause difficulties if your filter script needs to access some files (e.g. 'Sendmail' databases) that are not accessible to that user and group. For example, the following fragment using 'dbmap' function: if dbmap("/etc/mail/aliases.db", $f, 1) ... fi will normally fail, because '/etc/mail/aliases.db' is readable only to the root and members of the group 'smmsp'. In such situations you need to instruct 'mailfromd' to retain the privileges of one or several supplementary groups when switching to the user privileges. This is done using 'group' statement in the 'mailfromd' configuration file (*note group: conf-priv.). In example above, you need to use the following settings: group smmsp; (The same effect can be achieved with '--group' command line option: 'mailfromd --group=smmsp'). To stop a running instance of 'mailfromd' use one of the following signals: 'SIGQUIT', 'SIGTERM', 'SIGINT'. All three signals have the same effect: the program cancels handling any pending requests, uninitializes the communication socket (if it is a UNIX socket, the program unlinks it) and exits. To restart the running 'mailfromd' instance, send it 'SIGHUP'. For restart to be possible, two conditions must be met: 'mailfromd' must be invoked with the full file name, and the configuration file name must be full as well. If either of them is not met, 'mailfromd' displays a similar warning message: warning: script file is given without full file name warning: restart (SIGHUP) will not work or: warning: mailfromd started without full file name warning: restart (SIGHUP) will not work The reaction of 'mailfromd' on 'SIGHUP' in this case is the same as on the three signals described previously, i.e. cleanup and exit immediately. The PID of the master instance of 'mailfromd' is kept on the "pidfile", which is named 'mailfromd.pid' and is located in the program "state directory". Assuming the default location of the latter, the following command will stop the running instance of the daemon: kill -TERM `head -n1 /usr/local/var/mailfromd/mailfromd.pid` The default pidfile location is shown in the output of 'mailfromd --show-defaults' (*note Databases::), and can be changed at run time using 'pidfile' statement (*note pidfile: conf-base.). To facilitate the use of 'mailfromd', it is shipped with a shell script that can be used to launch it on system startup and shut it down when the system goes down. The script, called 'rc.mailfromd', is located in the directory '/etc' of the distribution. It takes a single argument, specifying the action that should be taken: start Start the program. stop Shut down the program reload Reload the program, by sending it 'SIGHUP' signal. restart Shut down the program and start it again. status Display program status. It displays the PID of the master process and its command line, for example: $ /etc/rc.d/rc.mailfromd status mailformd appears to be running at 26030 26030 /usr/local/sbin/mailfromd --group smmsp If the second line is not displayed, this most probably mean that there is a 'stale' pidfile, i.e. the one left though the program is not running. An empty 'rc.mailfromd status' output means that 'mailfromd' is not running. configtest [FILE] Check the script file syntax, report any errors found and exit. If FILE is given it is checked instead of the default one. macros [-c] [FILE] Parse the script file (or FILE, if it is given, extract the names of Sendmail macros it uses and generate corresponding export statements usable in the Sendmail configuration file. By default, 'mc' statements are generated. If '-c' ('--cf') is given, the statements for 'sendmail.cf' are output. See the next chapter for the detailed description of this mode. You can pass any additional arguments to 'mailfromd' by editing 'ARGS' variable near line 22. The script is not installed by default. You will have to copy it to the directory where your system start-up scripts reside and ensure it is called during the system startup and shut down. The exact instructions on how to do so depend on the operating system you use and are beyond the scope of this manual.  File: mailfromd.info, Node: MTA Configuration, Next: calloutd, Prev: Invocation, Up: Top 9 Using 'mailfromd' with Various MTAs ************************************* The following sections describe how to configure various Milter-capable MTAs to work with 'mailfromd'. * Menu: * Sendmail:: * MeTA1:: * Postfix::  File: mailfromd.info, Node: Sendmail, Next: MeTA1, Up: MTA Configuration 9.1 Using 'mailfromd' with Sendmail. ==================================== This chapter assumes you are familiar with Sendmail configuration in general and with Milter configuration directives in particular. It concentrates only on issues, specific for 'mailfromd'. To prepare 'Sendmail' to communicate with 'mailfromd' you need first to set up the "milter port". This is done with 'INPUT_MAIL_FILTER' statement in your 'Sendmail' file: INPUT_MAIL_FILTER(`mailfrom', `S=unix:/usr/local/var/mailfromd/mailfrom') Make sure that the value of 'S' matches the value of 'listen' statement in your 'mailfromd.conf' file (*note listen: conf-milter.). Notice, however, that they may not be literally the same, because 'listen' allows to specify socket address in various formats, whereas Sendmail's 'S' accepts only milter format. If you prefer to fiddle directly with 'sendmail.cf' file, use this statement instead: Xmailfrom, S=unix:/usr/local/var/mailfromd/mailfrom If you are using Sendmail version 8.14.0 or newer, you may skip to the end of this section. These versions implement newer Milter protocol that enables 'mailfromd' to negotiate with the MTA the macros it needs for each state. Older versions of Sendmail do not offer this feature. For Sendmail versions prior to 8.14.0, you need to manually configure 'Sendmail' to export macros you need in your 'mailfromd.mf' file. The simplest way to do so is using 'rc.mailfromd' script, introduced in the previous chapter. Run it with 'macros' command line argument and copy its output to your 'sendmail.mc' configuration file: $ rc.mailfromd macros If you prefer to work with 'sendmail.cf' directly, use '-c' ('--cf') command line option: $ rc.mailfromd macros -c Finally, if you use other mailfromd script file than that already installed (for example, you are preparing a new configuration while the old one is still being used in production environment), give its name in the command line: $ rc.mailfromd macros newscript.mf # or: $ rc.mailfromd macros -c newscript.mf If you use this method, you can skip the rest of this chapter. However, if you are a daring sort of person and prefer to do everything manually, follow the instructions below. First of all you need to build a list of macros used by handlers in your 'mailfromd.mf' file. You can obtain it running 'mailfromd --dump-macros'. This will display all macros used in your handlers, grouped by handler name, for example: envfrom i, f, {client_addr} envrcpt f, {client_addr}, {rcpt_addr} Now, modify 'confMILTER_MACROS_HANDLER' macros in your 'mc' file. Here, HANDLER means the uppercase name of the 'mailfromd' handler you want to export macros to, i.e. the first word on each line of the above 'mailfromd --dump-macros' output. _Notice,_ that in addition to these macros, you should also export the macro 'i' for the very first handler ('rc.mailfromd macros' takes care of it automatically, but you preferred to do everything yourself...) It is necessary in order for 'mailfromd' to include 'Message-ID' in its log messages (*note Message-ID::). For example, given the above macros listing, which corresponds to our sample configuration (*note Filter Script Example::), the 'sendmail.mc' snippet will contain: define(`confMILTER_MACROS_ENVFROM',dnl confMILTER_MACROS_ENVFROM `, i, f, {client_addr}') define(`confMILTER_MACROS_ENVRCPT',dnl confMILTER_MACROS_ENVRCPT `, f, {client_addr}, {rcpt_addr}') Special attention should be paid to 's' macro ('HELO' domain name). In 'Sendmail' versions up to 8.13.7 (at least) it is available only to 'helo' handler. If you wish to make it available elsewhere you will need to use the method described in *note HELO Domain:: Now, if you are a _really_ daring person and prefer to do everything manually _and_ to hack your 'sendmail.cf' file directly, you certainly don't need any advices. Nonetheless, here's how the two statements above _could_ look in this case: O Milter.macros.envfrom=i, {auth_type}, {auth_authen}, \ {auth_ssf}, {auth_author}, {mail_mailer}, {mail_host}, \ {mail_addr} ,{mail_addr}, {client_addr}, f O Milter.macros.envrcpt={rcpt_mailer}, {rcpt_host}, \ {rcpt_addr} ,i, f, {client_addr}  File: mailfromd.info, Node: MeTA1, Next: Postfix, Prev: Sendmail, Up: MTA Configuration 9.2 Using 'mailfromd' with MeTA1. ================================= MeTA1 () is an MTA of next generation which is designed to provide the following main features: * Security * Reliability * Efficiency * Configurability * Extendibility Instead of using Sendmail-compatible Milter protocol, it implements a new protocol, called "policy milter", therefore an additional program is required to communicate with 'mailfromd'. This program is a "Pmilter-Milter multiplexer" 'pmult', which is part of the 'Mailfromd' distribution. *Note pmult::, for a detailed description of its configuration. The configuration of 'Meta1--Mailfromf' interaction can be subdivided into three tasks. 1. Configure 'mailfromd' This was already covered in previous chapters. No special 'MeTA1'-dependent configuration is needed. 2. Configure 'pmult' to communicate with 'mailfromd' This is described in detail in *note pmult::. 3. Set up MeTA1 to communicate with 'pmult' The MeTA1 configuration file is located in '/etc/meta1/meta1.conf'. Configure the 'smtps' component, by adding the following section: policy_milter { socket { type = TYPE; address = ADDR; [path = PATH;] [port = PORT-NO;] }; [timeout = INTERVAL;] [flags = { FLAG };] }; Statements in square brackets are optional. The meaning of each instruction is as follows: 'type = TYPE' Set the type of the socket to communicate with 'pmult'. Allowed values for TYPE are: inet Use INET socket. The socket address and port number are set using the 'address' and 'port' statements (see below). unix Use UNIX socket. The socket path is given by the 'path' statement (see below). Notice, that depending on the 'type' setting you have to set up either 'address'/'port' or 'path', but not both. 'address = ADDR' Configure the socket address for 'type = inet'. ADDR is the IP address on which 'pmult' is listening (*note listen statement: pmult-conf.). 'port = PORT-NO' Port number 'pmult' is listening on (*note listen statement: pmult-conf.). 'path = SOCKET-FILE' Full pathname of the socket file, if 'type = unix'. 'timeout = INTERVAL' Sets the maximum amount of time to wait for a reply from 'pmult'. The behavior of 'smtps' in case of time out depends on the 'flags' settings: 'flags = { FLAG }' FLAG is one of the following: abort If 'pmult' does not respond, abort the current SMTP session with a '421' error. accept_but_reconnect If 'pmult' does not respond, continue the current session but try to reconnect for the next session. For example, if the 'pmult' configuration has: listen inet://127.0.0.1:3333; then the corresponding part in '/etc/meta1/meta1.conf' will be smtps { policy_milter { socket { type = inet; address = 127.0.0.1; port = 3333; }; ... }; ... }; Similarly, if the 'pmult' configuration has: listen unix:///var/spool/meta1/pmult/socket; then the '/etc/meta1/meta1.conf' should have: smtps { policy_milter { socket { type = unix; path = /var/spool/meta1/pmult/socket; }; ... }; ... };  File: mailfromd.info, Node: Postfix, Prev: MeTA1, Up: MTA Configuration 9.3 Using 'mailfromd' with Postfix ================================== To configure 'postfix' to work with your filter, you need to inform it about the socket your filter is listening on. The 'smtpd_milters' (or 'non_smtpd_milters') statement in '/etc/postfix/main.cf' serves this purpose. If the filter is to handle mail that arrives via SMTP, use 'smtpd_milters'. If it is to handle mail submitted locally to the queue, use 'non_smtpd_milters'. In both cases, the value is a whitespace-separated list of socket addresses. Note, that Postfix syntax for socket addresses differs from that used by Sendmail and mailfromd. The differences are summarized in the following table: Sendmail Mailfromd Postfix --------------------------------------------------------------------------- inet:PORT@HOST inet://HOST:PORT inet:HOST:PORT unix:FILE unix://FILE unix:FILE Table 9.1: Socket addresses in various formats For example, if your 'mailfromd' listens on 'inet://127.0.0.1:4111', add the following to '/etc/postfix/main.cf': smtpd_milters = inet:127.0.0.1:4111 'Mailfromd' uses Milter protocol version 6. Postfix, starting from version 2.6 uses the same version. Older versions of Postfix use Milter protocol 2 by default. Normally, it should not be a problem, as 'mailfromd' tries to detect what version the server is speaking. If, however, it fails to select the proper version, you will have to instruct Postfix what version to use. To do so, add the following statement to '/etc/postfix/main.cf': milter_protocol = 6 The way Postfix handles macros differs from that of Sendmail. Postfix emulates a limited subset of Sendmail macros, and not all of them are are available when you would expect them to. In particular, the 'i' macro is not available before the 'DATA' stage, which brings two consequences. First, 'mailfromd' log messages will not include message ID until the 'DATA' stage is reached. Secondly, you cannot use 'i' in handlers 'connect', 'helo', 'envfrom' and 'envrcpt', If you wish to tailor Postfix defaults to export the actual macros used by your filter, run 'mailfromd --dump-macros' and filter its output through the 'postfix-macros.sed' filter, which is installed to the 'PREFIX/share/mailfromd' directory, e.g.: $ mailfromd --dump-macros | \ sed -f /usr/share/mailfromd/postfix-macros.sed milter_helo_macros = {s} milter_mail_macros = {client_addr} {s} {f} milter_rcpt_macros = {rcpt_addr} {f} {client_addr} milter_end_of_data_macros = {i} Cut and paste its output to your '/etc/postfix/main.cf'. For more details regarding Postfix interaction with Milter and available Postfix configuration options, see Postfix before-queue Milter support (http://www.postfix.org/MILTER_README.html).  File: mailfromd.info, Node: calloutd, Next: mfdbtool, Prev: MTA Configuration, Up: Top 10 'calloutd' ************* The callout verification is usually performed by a special instance of 'mailfromd' (*note callout server::). However, it is also possible to set up a dedicated callout server on a separate machine. You can choose to do so, for instance, in order to reduce the load on the server running 'mailfromd'. This stand-alone callout facility is provided by the 'calloutd' daemon. * Menu: * config-calloutd:: Calloutd Configuration. * invocation-calloutd:: Calloutd Command-Line Options. * protocol-calloutd:: The Callout Protocol.  File: mailfromd.info, Node: config-calloutd, Next: invocation-calloutd, Up: calloutd 10.1 Calloutd Configuration =========================== Main configuration file '/etc/mailfromd.conf' is used (*note Mailfromd Configuration::). The configuration statements are basically the same as for 'mailfromd'. The address to listen on is defined in the 'server' statement. Basically, it is the only statement the configuration file is required to have. The minimal configuration can look like: program calloutd { server { listen inet://198.51.100.1:3535; } } To instruct the 'mailfromd' daemon to use this server, the following statement should be added to the '/etc/mailfromd.conf' file: program mailfromd { callout-url inet://198.51.100.1:3535; } The 'server' statement differs a little from the similar statement for 'mailfromd'. This and another 'calloutd'-specific statements are described in detail in the subsections that follow. The rest of statements is shared with 'mailfromd'. The following table lists all supported configuration statements along with cross-references to the correspondent descriptions: Statement Reference ------------------------------------------------------------------- 'acl' *Note Mailutils Configuration File: (mailutils)acl statement. 'auth' *Note Mailutils Configuration File: (mailutils)auth statement. 'database' *Note conf-database::. 'database-mode' *Note database-mode: conf-database. 'database-type' *Note database-type: conf-database. 'debug' (section) *Note Mailutils Configuration File: (mailutils)debug statement. 'debug' *Note conf-calloutd-log::. 'ehlo-domain' *Note ehlo-domain: conf-callout. 'enable-vrfy' *Note enable-vrfy: conf-callout. 'group' *Note group: conf-priv. 'include' *Note Mailutils Configuration File: (mailutils)include. 'io-timeout' *Note io-timeout: conf-timeout. 'locking' *Note Mailutils Configuration File: (mailutils)locking statement. 'lock-retry-count' *Note lock-retry-count: conf-database. 'lock-retry-timeout' *Note lock-retry-timeout: conf-database. 'logger' *Note conf-calloutd-log::. 'logging' *Note Mailutils Configuration File: (mailutils)logging statement. 'mailer' *Note Mailutils Configuration File: (mailutils)mailer statement. 'mail-from-address' *Note mail-from-address: conf-callout. 'pidfile' *Note pidfile: conf-calloutd-setup. 'server' *Note conf-calloutd-server::. 'source-ip' *Note source-ip: conf-calloutd-setup. 'smtp-timeout' *Note conf-timeout::. 'state-directory' *Note state-directory: conf-calloutd-setup. 'transcript' *Note conf-calloutd-log::, 'user' *Note user: conf-priv. * Menu: * conf-calloutd-setup:: 'calloutd' General Setup. * conf-calloutd-server:: The 'server' Statement. * conf-calloutd-log:: 'calloutd' Logging.  File: mailfromd.info, Node: conf-calloutd-setup, Next: conf-calloutd-server, Up: config-calloutd 10.1.1 'calloutd' General Setup ------------------------------- -- Calloutd Conf: source-ip IP Sets source IP address for TCP connections. -- Calloutd Conf: pidfile filename Defines the name of the file to store PID value in. -- Calloutd Conf: state-directory dir Sets the name of the program state directory. *Note statedir::.  File: mailfromd.info, Node: conf-calloutd-server, Next: conf-calloutd-log, Prev: conf-calloutd-setup, Up: config-calloutd 10.1.2 The 'server' statement ----------------------------- The 'server' statement configures how 'calloutd' will communicate with the client 'mailfromd' server. server { id NAME; listen URL; backlog NUM; max-instances NUM; single-process BOOL; reuseaddr BOOL; default BOOL; callout URL; acl { ... } } -- Calloutd Conf: server Define a server. Optional label may follow the 'server' keyword. The label is ignored. The substatements in the 'server' block provide parameters for configuring this server. -- server: id name Assign an identifier to this server. This identifier is used as a suffix to syslog tag (*note syslog tag::) in messages related to this server. For example, if a server block had the following statement in it: id main; then all messages related to this server will be marked with tag 'calloutd#main'. The part before the '#' is set using the 'tag' statement in 'logging' block (*note Mailutils Configuration File: (mailutils)Logging Statement.). -- server: listen url Listen for connections on the given URL. *Note milter port specification::, for a description of allowed URL formats. Example: listen inet://10.10.10.1:3331; -- server: backlog NUM Configures the size of the queue of pending connections. Default value is 8. -- server: max-instances number Sets the maximum number of instances allowed for this server. -- server: single-process bool When set to 'yes', this server will run in "single-process" mode, i.e. it will not fork sub-processes to serve requests. This option is meant exclusively to assist in debugging 'calloutd'. Don't use it for anything else but for debugging! -- server: reuseaddr bool When set to 'yes', 'calloutd' will attempt to reuse existing socket addresses. This is the default behavior. -- server: acl STATEMENTS Defines access control list for this server. *Note Mailutils Configuration File: (mailutils)ACL Statement, for a detailed discussion. If the global ACL is defined as well, an incoming connection is checked against both lists: first the per-server ACL, then the global one. The connection will be permitted only if it passes both checks.  File: mailfromd.info, Node: conf-calloutd-log, Prev: conf-calloutd-server, Up: config-calloutd 10.1.3 'calloutd' logging ------------------------- -- Calloutd Conf: logger mech Set default logger mechanism. Allowed values for MECH are: stderr Log everything to the standard error. syslog Log to syslog. syslog:async Log to syslog using the asynchronous syslog implementation. *Note Logging and Debugging::, for a detailed discussion. See also *note Using non-blocking syslog: syslog-async, for information on how to set default syslog implementation at compile time. -- Calloutd Conf: debug spec Set 'mailfromd' debug verbosity level. The SPEC must be a valid debugging level specification (*note debugging level specification::). -- Calloutd Conf: transcript BOOL If the boolean value BOOL is 'true', enables the transcript of call-out SMTP sessions.  File: mailfromd.info, Node: invocation-calloutd, Next: protocol-calloutd, Prev: config-calloutd, Up: calloutd 10.2 Calloutd Command-Line Options ================================== The 'calloutd' invocation syntax is: calloutd [OPTION...] The following options are available: Server configuration modifiers ------------------------------ '--foreground' Stay in foreground. When given this option, 'calloutd' will not disconnect itself from the controlling terminal and will run in the foreground. '-g NAME' '--group=NAME' Retain the group NAME when switching to user privileges. *Note Starting and Stopping::. '--pidfile=FILE' Set pidfile name. Overrides the 'pidfile' configuration statement, which you are advised to use instead (*note pidfile: conf-base.). '--resolv-conf-file=FILE' Read resolver settings from FILE, instead of the default '/etc/resolv.conf'. '-S IP' '--source-ip=IP' Set source address for TCP connections. Overrides the 'source-ip' configuration statement, which you are advised to use instead (*note source-ip: conf-base.). '--single-process' Do not fork sub-processes to serve requests. This option is meant to assist in debugging 'calloutd'. Don't use it for anything else but for debugging, as it terribly degrades performance! '--state-directory=DIR' Set new program state directory. *Note Local state directory: statedir, for the description of this directory and its purposes. '-u NAME' '--user NAME' Switch to this user's privileges after startup. Overrides the 'user' configuration file statement, which you are advised to use instead (*note user: conf-priv.). Default user is 'mail'. Logging and debugging options ----------------------------- '-d STRING' '--debug=STRING' Set debugging level. *Note Logging and Debugging::. '--log-facility=FACILITY' Output logs to syslog FACILITY. '--log-tag=STRING' Tag syslog entries with the given STRING, instead of the program name. '--logger=MECH' Set logger mechanism (MECH is one of 'stderr', 'syslog', 'syslog:async'). *Note Logging and Debugging::. '--syslog' Selects default syslog mechanism for diagnostic output. '--stderr' Directs all logging to standard output. Similar to '--logger=stderr'. '-S IP' '--source-ip=IP' Set source address for TCP connections. Overrides the 'source-ip' configuration statement, which you are advised to use instead (*note source-ip: conf-base.). '--debug-level=LEVEL' Set Mailutils debugging level. See , for a detailed discussion of LEVEL argument. '--source-info' '--no-source-info' Include C source information in debugging messages. This is similar to setting 'line-info yes' in the 'debug' configuration block (*note line-info: (mailutils)debug statement.). The '--no-source-info' can be used to cancel the effect of the 'line-info yes' configuration statement. You do not need this option, unless you are developing or debugging 'calloutd'. '-X' '--transcript' '--no-transcript' Enable or disable transcript of the SMTP sessions to the log channel. *Note Logging and Debugging::. Configuration file control -------------------------- '--config-file=FILE' Load this configuration file. '--config-lint' Check syntax of configuration files and exit. Exit code is 0 if the file or files are OK, and 78 otherwise. '--config-verbose' Verbosely log parsing of the configuration files. '--no-site-config' '--no-config' Don't load site-wide configuration file. '--set=PARAM=VALUE' Set configuration parameter Informational options --------------------- '--config-help' Show configuration file summary. '--show-config-options' Show compilation options. '-?' '--help' Give a short help list. '--usage' Give a short usage message. '-V' '--version' Print program version  File: mailfromd.info, Node: protocol-calloutd, Prev: invocation-calloutd, Up: calloutd 10.3 The Callout Protocol ========================= This section describes the protocol used to communicate with the 'calloutd' server. The protocol works over stream-oriented TCP/IP transport. Either UNIX or IPv4 socket can be used. Commands and responses are terminated by a single CR LF pair. Each command occupies exactly one line. If the server succeeded in executing the command, it replies with a line starting with the word 'OK'. Depending on the command, this keyword may be followed by a single space character and additional information. More information can be returned in "unsolicited replies" before the 'OK' line. Each unsolicited reply line starts with and asterisk followed by a single horizontal space character. On error, the server replies with 'NO' followed by a horizontal space character and human-readable description of the problem. The valid commands are discussed below. In examples illustrating the commands, the lines sent by the client are prefixed with 'C:', and lines sent by the server are prefixed with 'S:'. -- Command: vrfy EMAIL [OPTION ARG] Adds EMAIL to the queue of email addresses to be verified. Available OPTIONs are: 'mode KW' Sets verification mode for this email address. Available modes are: mxfirst default The default mode. If the 'host' option is also given, its argument is taken as the domain name. Otherwise, domain part of EMAIL is used. The verification goes as follows. First, determine MX servers for that domain. Query each of them in order of increasing priority. First of them that replies determines the result of the test. If no MX servers are defined for that domain, look for its 'A' record. If available, run SMTP probe on that IP. mxonly Query MX servers for the domain specified with the 'host' option. hostonly Query the server whose name or IP address is supplied with the 'host' option. hostfirst The reverse of 'mxfirst': first query the host, then the MX servers. The domain must be specified using the 'host' option. 'host NAME' Supplies the domain name for 'mxonly' and mode, and host name or IP address for 'hostfirst' and 'hostonly' modes. The use of this keyword with any of these modes is mandatory. 'ehlo STRING' Use STRING as the argument to the SMTP 'EHLO' command. 'mailfrom EMAIL' Use EMAIL in the SMTP 'MAIL FROM' command. On success, the server replies with 'OK', followed by a non-negative session ID for that email: C: VRFY gray@example.org S: OK 0000000001 -- Command: get ARG [ARG] Query value of internal callout parameters. Valid values for ARG are: ehlo Return the string used as argument to the SMTP 'EHLO' command. mailfromd Return the email address that is used in the SMTP 'MAIL FROM' command. On success, the server returns the requested value (if found) in an unsolicited reply: C: GET ehlo timeout S: * ehlo=example.net S: OK -- Command: sid STRING Sets STRING as session identifier for that session. Example: C: SID deadbeef S: OK -- Command: timeout CONNECT INITIAL HELO MAIL RCPT RSET QUIT Sets timeouts for various stages of SMTP session. On success, 'OK' is returned. C: timeout 300 300 300 600 300 300 120 S: OK timeouts set -- Command: run Runs callout session for emails registered with the 'vrfy' command. On success, results of the check are returned after the 'OK' keyword in a whitespace-separated list of 'ID=RESULT' pairs. In each pair, ID is its identifier as returned in the reply to the 'VRFY' command and RESULT is one of the following result strings: 'success', 'not_found', 'failure', 'temp_failure', 'timeout'. Additional information about each callout session is returned in unsolicited replies. Each such reply is prefixed with the email identifier and callout stage name. Stage names are: 'INIT REMOTE_NAME' The 'calloutd' server is establishing communication with the remote SMTP server REMOTE_NAME. 'GRTNG LINE' 'calloutd' received initial response from the remote server. LINE is the first line of the reply. 'HELO LINE' 'calloutd' received response to the 'EHLO' (or 'HELO') command. In case of multiline response, LINE is the first line. 'SENT COMMAND' The SMTP command COMMAND has been sent to the remote server. 'RECV LINE' The remote server returned LINE in response. In case of multiline response, LINE is the first line. Example of verification session: C: RUN S: * 0000000000 INIT mx.example.org S: * 0000000000 GRTNG 220 mx.example.org ESMTP Ready S: * 0000000000 HELO 250-mx.example.org Hello tester S: * 0000000000 SENT RCPT TO: S: * 0000000000 RECV 250 Accepted S: * 0000000001 INIT foo.example.net S: * 0000000001 GRTNG 220 foo ESMTP server ready S: * 0000000001 HELO 250-foo.example.net Hello S: * 0000000001 SENT RCPT TO: S: * 0000000001 RECV 450 4.7.0 You are greylisted for 3600 seconds OK 0000000000=success 0000000001=temp_failure -- Command: drop SERIAL Drop the email with the given SERIAL number from the verification queue. Example: C: DROP 0000000002 S: OK -- Command: quit Finishes the current session and disconnects from the callout server. C: QUIT S: OK bye  File: mailfromd.info, Node: mfdbtool, Next: mtasim, Prev: calloutd, Up: Top 11 'mfdbtool' ************* The 'mfdbtool' utility manages 'mailfromd' databases. * Menu: * Invoking mfdbtool:: * Configuring mfdbtool::  File: mailfromd.info, Node: Invoking mfdbtool, Next: Configuring mfdbtool, Up: mfdbtool 11.1 Invoking 'mfdbtool' ======================== The following options request the operation to be performed on the database. Exactly one of them must be specified. Each of them implied the '--stderr' option (*note --stderr::). '--list' List the database. By default, 'cache' database is assumed. To list another database, use '--format' option (*note --format::). *Note Basic Database Operations::. '--delete' Delete given entries from the database (*note deleting from databases::). By default, 'cache' database is assumed. To specify another database, use '--format' option (*note --format::). *Note Basic Database Operations::. '--expire' Delete all expired entries from the database (*note Database Maintenance::). By default, 'cache' database is assumed. To specify another database, use '--format' option (*note --format::). Full database name can be given in the command line (see '--file' option below), if it differs from the one specified in the script file. Use with the option '--all' (*note --all::) to expire all databases. *Note Database Maintenance::. '--compact' Compact database (*note compaction::). By default, 'cache' database is compacted. To specify another database, use '--format' option (*note --format::). Full database name can be given in the command line (see '--file' option below), if it differs from the one specified in the script file. Use with the option '--all' (*note --all::) to compact all databases. *Note compaction::. The following options modify the behavior of 'mfdbtool': '--all' When used with '--compact' or '--expire' option, applies the action to all available databases. *Note compact cronjob::. '-d STRING' '--debug=STRING' Sets debugging level. The STRING argument must be a valid mailfromd debug level specification, as described in *note debugging level specification::. '-e INTERVAL' '--expire-interval=INTERVAL' Set expiration intervals for all databases to the specified interval. *Note time interval specification::, for a description of INTERVAL format. The option overrides the 'expire-interval' configuration statement (*note expire-interval-conf::), which you are advised to use instead. '-f FILENAME' '--file=FILENAME' Set the name of the database to operate upon (for '--compact', '--delete', '--expire', and '--list' options). Useful if, for some reason, you need to operate on a database whose file name does not match the one 'mfdbtool' is configured to use. '-H DBFORMAT' '--format=DBFORMAT' Use database of the given format, instead of the default 'cache'. *Note Basic Database Operations::. '--ignore-failed-reads' Ignore records that cannot be retrieved while compacting the database. Without this option, 'mfdbtool' will abort the compaction if any such error is encountered. '--predict=RATE-LIMIT' Used with '--list' enables printing of the estimated times of sending along with the 'rate' database dump. Implies '--list --format=rate'. *Note estimated time of sending::. '--state-directory=DIR' Sets program state directory. *Note Local state directory: statedir, for the description of this directory and its purposes. This option overrides the 'state-directory' configuration statement, described in *note state-directory: conf-base. '--time-format=FORMAT' Set format to be used for timestamps in listings, produced by '--list'. The FORMAT is any valid 'strftime' format string, see *note Time and Date Formats::, for a detailed description. The default FORMAT is '%c' (*note %c time format::). To analyze 'mfdbtool --list' output using text tools, such as 'awk' or 'grep', the following format might be useful: '%s' (*note %s time format::). Another format I find useful is '%Y-%m-%d_%H:%M:%S'.  File: mailfromd.info, Node: Configuring mfdbtool, Prev: Invoking mfdbtool, Up: mfdbtool 11.2 Configuring 'mfdbtool' =========================== Configuration settings are read from the '/etc/mailfromd.conf' file (*note Mailfromd Configuration::). The following statements are understood: Statement Reference ------------------------------------------------------------------- 'database' *Note conf-database::. 'database-mode' *Note database-mode: conf-database. 'database-type' *Note database-type: conf-database. 'debug' *Note conf-calloutd-log::. 'lock-retry-count' *Note lock-retry-count: conf-database. 'lock-retry-timeout' *Note lock-retry-timeout: conf-database. 'state-directory' *Note state-directory: conf-calloutd-setup.  File: mailfromd.info, Node: mtasim, Next: pmult, Prev: mfdbtool, Up: Top 12 'mtasim' -- a testing tool ***************************** The 'mtasim' utility is a MTA simulator for testing 'mailfromd' filter scripts. By default it operates in "stdio" mode, similar to that of 'sendmail -bs'. In this mode it reads SMTP commands from standard input and sends its responses to the standard output. There is also another mode, called "daemon", where 'mtasim' opens a TCP socket and listens on it much like any MTA does. In both modes no actual delivery is performed, the tool only simulates the actions an MTA would do and responses it would give. This tool is derived from the program 'mta', which I wrote for GNU Anubis test suite. * Menu: * interactive mode:: * expect commands:: * traces:: * daemon mode:: * command summary:: * option summary::  File: mailfromd.info, Node: interactive mode, Next: expect commands, Up: mtasim 12.1 'mtasim' interactive mode mode =================================== If you start 'mtasim' without options, you will see the following: 220 mtasim (mailfromd 8.8) ready (mtasim) _ The first line is an usual RFC 2821 reply. The second one is a prompt, indicating that 'mtasim' is in interactive mode and ready for input. The prompt appears only if the package is compiled with GNU Readline and 'mtasim' determines that its standard input is connected to the terminal. This is called "interactive mode" and is intended to save the human user some typing by offering line editing and history facilities (*note Command Line Editing: (readline)Command Line Editing.). If the package is compiled without GNU Readline, you will see: 220 mtasim (mailfromd 8.8) ready _ where '_' represents the cursor. Whatever the mode, 'mtasim' will wait for further input. The input is expected to consist of valid SMTP commands and special 'mtasim' statements. The utility will act exactly like a RFC 2821-compliant MTA, except that it will not do actual message delivery or relaying. Try typing 'HELP' to get the list of supported commands. You will see something similar to: 250-mtasim (mailfromd 8.8); supported SMTP commands: 250- EHLO 250- HELO 250- MAIL 250- RCPT 250- DATA 250- HELP 250- QUIT 250- HELP 250 RSET You can try a simple SMTP session now: 220 mtasim (mailfromd 8.8) ready (mtasim) ehlo localhost 250-pleased to meet you 250 HELP (mtasim) mail from: 250 Sender OK (mtasim) rcpt to: 250 Recipient OK (mtasim) data 354 Enter mail, end with `.' on a line by itself (mtasim) . 250 Mail accepted for delivery (mtasim) quit 221 Done Notice, that 'mtasim' does no domain checking, so such thing as 'rcpt to: ' was eaten without complaints. So far so good, but what all this has to do with 'mailfromd'? Well, that's what we are going to explain. To make 'mtasim' consult any milter, use '--port' ('-X') command line option. This option takes a single argument that specifies the milter port to use. The port can be given either in the usual Milter format (*Note milter port specification::, for a short description), or as a full 'sendmail.cf' style 'X' command, in which case it allows to set timeouts as well: $ mtasim --port=inet:999@localhost # This is also valid: $ mtasim --port='mailfrom, S=inet:999@localhost, F=T, T=C:100m;R:180s' If the milter is actually listening on this port, 'mtasim' will connect to it and you will get the following initial prompt: 220-mtasim (mailfromd 8.8) ready 220 Connected to milter inet://localhost:999 (mtasim) Notice, that it makes no difference what implementation is listening on that port, it may well be some other filter, not necessarily 'mailfromd'. However, let's return to 'mailfromd'. If you do not want to connect to an existing 'mailfromd' instance, but prefer instead to create a new one and run your tests with it (a preferred way, if you already have a stable filter running but wish to test a new script without disturbing it), use '--port=auto'. This option instructs 'mtasim' to do the following: 1. Create a unique temporary directory in '/tmp' and create a communication socket within it. 2. Spawn a new instance of 'mailfromd'. The arguments and options for that instance may be given in the invocation of 'mtasim' after a double-dash marker ('--') 3. Connect to that filter. When 'mtasim' exits, it terminates the subsidiary 'mailfromd' process and removes the temporary directory it has created. For example, the following command will start 'mailfromd -I. -I../mflib test.rc': $ mtasim -Xauto -- -I. -I../mflib test.rc 220-mtasim (mailfromd 8.8) ready 220 Connected to milter unix:/tmp/mtasim-j6tRLC/socket (mtasim) The '/tmp/mtasim-j6tRLC' directory and any files within it will exist as long as 'mtasim' is running and will be removed when you exit from it.(1) You can also instruct the subsidiary 'mailfromd' to use this directory as its state directory (*note statedir::). This is done by '--statedir' command line option: $ mtasim -Xauto --statedir -- -I. -I../mflib test.rc (notice that '--statedir' is the 'mtasim' option, therefore it must appear _before_ '--') Special care should be taken when using 'mtasim' from root account, especially if used with '-Xauto' and '--statedir'. The 'mailfromd' utility executed by it will switch to privileges of the user given in its configuration (*note Starting and Stopping::) and will not be able to create data in its state directory, because the latter was created using 'root' as owner. To help in this case, 'mtasim' understands '--user' and '--group' command line options, that have the same meaning as for 'mailfromd'. Now, let's try 'HELP' command again: 250-mtasim (mailfromd 8.8); supported SMTP commands: 250- EHLO 250- HELO 250- MAIL 250- RCPT 250- DATA 250- HELP 250- QUIT 250- HELP 250- RSET 250-Supported administrative commands: 250- \Dname=value [name=value...] Define Sendmail macros 250- \Ecode Expect given SMTP reply code 250- \L[name] [name...] List macros 250- \Uname [name...] Undefine Sendmail macros 250 \Sfamily hostname address [port] Define sender socket address While the SMTP commands do not need any clarification, some words about the "administrative commands" are surely in place. These commands allow to define, undefine and list arbitrary Sendmail macros. Each administrative command consists of a backslash followed by a command letter. Just like SMTP ones, administrative commands are case-insensitive. If a command takes arguments, the first argument must follow the command letter without intervening whitespace. Subsequent arguments can be delimited by arbitrary amount of whitespace. For example, the '\D' command defines Sendmail macros: (mtasim) \Dclient_addr=192.168.10.1 f=sergiusz@localhost i=testmsg (mtasim) Notice that 'mailfromd' does not send any response to the command, except if there was some syntactic error, in which case it will return a '502' response. Now, you can list all available macros: (mtasim) \L 220-client_addr=192.168.10.1 220-f=sergiusz@localhost 220 i=testmsg (mtasim) or just some of them: (mtasim) \Lclient_addr 220 client_addr=192.168.10.1 (mtasim) To undefine a macro, use '\U' command: (mtasim) \Ui (mtasim) \l 220-client_addr=192.168.10.1 220 f=sergiusz@localhost (mtasim) The '\S' command declare sender socket and host name. These parameters are passed to the 'connect' handler, if one is declared (*note connect handler::). To give you a chance to use this command, 'mtasim' does not invoke 'connect' handler right after connecting to the milter. Instead it waits until either the '\S' command or any SMTP command (except 'HELP') is given. After calling 'connect' handler the '\S' is disabled (to reflect it, it also disappears from the 'HELP' output). The '\S' command takes 1 to 4 arguments. The first argument supplies the socket family (*note Table 4.3: socket-families.). Allowed values are: 'stdio', 'unix', 'inet', 'inet6' or numbers from '0' to '3'. The '\S stdio' (or '\S 0') command needs no additional arguments. It indicates that the SMTP connection is obtained from the standard input. It is the default if sender socket is not declared explicitly. The command '\S unix' indicates that the connection is accepted from a UNIX socket. It requires two more argument. The first one supplies sender host name and the second one supplies full path name of the socket file. For example: \S unix localhost /var/run/smtp.sock The commands '\S inet' and '\S inet6' indicate that the connection came from an 'INET' IPv4 or IPv6 socket, correspondingly(2). They require all four arguments to be specified. The additional arguments are: host name, IP address, and port number, in that order. For example: \S inet relay.gnu.org.ua 213.130.31.41 34567 or \S inet6 relay.gnu.org.ua 2001:470:1f0a:1be1::2 34567 Sender socket address can also be configured from the command line (*note sender-socket: option summary.). Now, let's try a real-life example. Suppose you wish to test the greylisting functionality of the filter script described in *note Filter Script Example::. To do this, you start 'mtasim': $ mtasim -Xauto -- -I. -I../mflib test.rc 220-mtasim (mailfromd 8.8) ready 220 Connected to milter unix:/tmp/mtasim-ak3DEc/socket (mtasim) The script in 'test.rc' needs to know 'client_addr' macro, so you supply it to 'mtasim': (mtasim) \Dclient_addr=10.10.1.13 Now, you try an SMTP session: (mtasim) ehlo yahoo.com 250-pleased to meet you 250 HELP (mtasim) mail from: 250 Sender OK (mtasim) rcpt to: 450 4.7.0 You are greylisted for 300 seconds OK, this shows that the greylisting works. Now quit the session: (mtasim) quit 221 Done ---------- Footnotes ---------- (1) However, this is true only if the program is exited the usual way (via 'QUIT' or end-of-file). If it is aborted with a signal like 'SIGINTR', the temporary directory is not removed. (2) Depending on how 'mailfromd' is configured, 'inet6' may be not available.  File: mailfromd.info, Node: expect commands, Next: traces, Prev: interactive mode, Up: mtasim 12.2 'mtasim' expect commands ============================= Until now we were using 'mtasim' interactively. However, it is often useful in shell scripts, for example the 'mailfromd' test suite is written in shell and 'mtasim'. To avoid the necessity to use auxiliary programs like 'expect' or 'DejaGNU', 'mtasim' contains a built-in expect feature. The administrative command '\E' introduces the SMTP code that the next command is expected to yield. For example, \E250 rcpt to: tells 'mtasim' that the response to 'RCPT TO' command must begin with '250' code. If it does, 'mtasim' continues execution. Otherwise, it prints an error message and terminates with exit code 1. The error message it prints looks like: Expected 250 but got 470 The expect code given with the '\E' command may have less than 3 digits. In this case it specifies the first digits of expected reply. For example, the command '\E2' matches replies '200', '220', etc. This feature can be used to automate your tests. For example, the following script tests the greylisting functionality (see the previous section): # Test the greylisting functionality # \E220 \Dclient_addr=10.10.1.13 \E250 ehlo yahoo.com \E250 mail from: \E450 rcpt to: \E221 quit This example also illustrates the fact that you can use '#'-style comments in the 'mtasim' input. Such a script can be used in shell programs, for example: mtasim -Xauto --statedir -- -I. -I../mflib test.rc < scriptfile if $? -ne 0; then echo "Greylisting test failed" fi  File: mailfromd.info, Node: traces, Next: daemon mode, Prev: expect commands, Up: mtasim 12.3 Trace Files ================ It is possible to log an entire SMTP session to a file. This is called "session tracing". Two options are provided for this purpose: '--trace-file=FILE' Sets the name of the trace file, i.e. a file to which the session transcript will be written. Both the input commands, and the 'mtasim' responses are logged. If the file FILE exists, it will be truncated before logging. This, however, can be changed using the following option: '-a' '--append' If the trace file exists, append new trace data to it.  File: mailfromd.info, Node: daemon mode, Next: command summary, Prev: traces, Up: mtasim 12.4 Daemon Mode ================ To start 'mtasim' in "daemon" mode, use the '--daemon' (or '-bd') command line option. This mode is not quite the same as Sendmail '-bd' mode. When started in "daemon" mode, 'mtasim' selects the first available TCP port to use from the range '1024 -- 65535'. It prints the selected port number on the standard output and starts listening on it. When a connection comes, it serves a _single_ SMTP session and exits immediately when it is ended. This mode is designed for use in shell scripts and automated test cases.  File: mailfromd.info, Node: command summary, Next: option summary, Prev: daemon mode, Up: mtasim 12.5 Summary of the 'mtasim' Administrative Commands ==================================================== This section provides a summary of administrative commands available in 'mtasim'. -- mtasim command: \D name=value [name=value...] Defines Sendmail macro NAME to the given VALUE. Any number of NAME=VALUE pairs can be given as arguments. *Note D command::. -- mtasim command: \E code Instructs 'mtasim' to expect next SMTP command to return given CODE (a three-digit decimal number). *Note expect commands::. -- mtasim command: \L [name...] Lists defined macros. *Note L command::. -- mtasim command: \U name [name...] Undefines macros given as its arguments. -- mtasim command: \S family [hostname address [port]] Declares the sender socket parameters. *Note S command::, for a detailed description and examples. This command is available only at the initial stage of a 'mtasim' session, before the first SMTP command was given. It is disabled if the '--sender-socket' option was given in the command line (*note sender-socket: option summary.). The 'help' output reflects whether or not this command is available. If neither this command nor the '--sender-socket' option were given, 'mtasim' behaves as if given the '\S stdio' command. The FAMILY argument supplies the socket family, i.e. the first argument to the 'connect' handler (*note connect handler::). It can have either literal or numeric value, as described in the table below: Literal Numeric Meaning ------------------------------------------------------------ stdio 0 Standard input/output (the MTA is run with '-bs' option) unix 1 UNIX socket inet 2 IPv4 protocol inet6 3 IPv6 protocol Table 12.1: Socket families See also *note Table 4.3: socket-families. Depending on the FAMILY, the rest of arguments supply additional parameters: stdio The HOSTNAME argument can be specified. It defines the first argument of the 'connect' handler (*note hostname in connect handler::). inet inet6 All arguments must be specified. argument 'connect' meaning argument --------------------------------------------------------------------------- hostname 1 Sender host name address 4 Sender IP address port 3 Sender port number unix HOSTNAME and ADDRESS must be supplied. The ADDRESS argument must be a full pathname of the UNIX socket.  File: mailfromd.info, Node: option summary, Prev: command summary, Up: mtasim 12.6 'mtasim' command line options ================================== This section summarizes all available 'mtasim' command line options. '--append' '-a' Append to the trace file. *Note traces::. '--body-chunk=NUMBER' Set the body chunk length (bytes) for 'xxfi_body' calls. '--daemon' '-bd' Run as daemon. *Note daemon mode::. '--define=MACRO=VALUE' '-D MACRO=VALUE' Define Sendmail macro MACRO to the given VALUE. It is similar to the '\D' administrative command (*note D command::) '--gacopyz-log=LEVEL' Set desired logging level for 'gacopyz' library (*note Gacopyz::). *Note gacopyz-log option::, for a detailed description of LEVEL. Notice, that unless this option is used, the '--verbose' ('-v') command line option implies '--gacopyz-log=debug'. '--group=NAME' '-g NAME' When switching to user's privileges as requested by the '--user' command line option, retain the additional group NAME. Any number of '--group' options may be given to supply a list of additional groups. '--user=NAME' '-u NAME' Run with this user privileges. This option and the '--group' option have effect only if 'mtasim' was started with root privileges. '--help' '-?' Display a short help summary '--milter-version=VERSION' Force using the given Milter protocol version number. The VERSION argument is either a numeric version (e.g. '2'), or a version string in form 'MAJOR.MINOR[.PATCH]', where square brackets indicate optional part. The default is '1.0.0'. If VERSION is any of '2', '3' or '1.0.0', the default protocol capabilities and actions for that version are set automatically. This option is intended for development and testing of the Gacopyz library (*note Gacopyz::). '--milter-proto=BITMASK' Set Milter protocol capabilities. See 'gacopyz/gacopyz.h' for the meaning of various bits in the BITMASK. Look for the C macros with the prefix 'SMFIP_'. '--milter-timeout=VALUES' Set timeouts for various Milter operations. VALUES is a comma-separated list of assignments 'T=V', where T is a "timeout code", indicating which timeout to set, and V is its new value. Valid timeout codes are: C Timeout for connecting to a filter. W S Timeout for sending information from the simulator to a filter. R Timeout for reading reply from the filter. E Overall timeout between sending end-of-message to filter and receiving final acknowledgment. Indirectly, it configures the upper limit on the execution time of the 'eom' handler (*note eom handler::). '--milter-actions=BITMASK' Set Milter actions. See 'gacopyz/gacopyz.h' for the meaning of various bits in the BITMASK. Look for the C macros with the prefix 'SMFIF_'. '--no-interactive' Not-interactive mode (disable readline). *Note Command Line Editing: (readline)Command Line Editing. '--port=PORT' '-X PORT' Communicate with given Milter PORT. *Note mtasim milter port::. '--prompt=STRING' Set readline prompt. The default prompt string is '(mtasim) '. '--sender-socket=FAMILY[,HOSTNAME,ADDRESS[,PORT]]' Declare sender socket address. This option has the same effect as the 'S command'. *Note S command::, for a detailed discussion and a description of its arguments. '--statedir' When using '-Xauto', use the temporary directory name as 'mailfromd' state directory (*note statedir mtasim option::). '--stdio' '-bs' Use the SMTP protocol on standard input and output. This is the default mode for 'mtasim'. *Note interactive mode::. '--trace-file=FILE' Set name of the trace file. *Note traces::. '--usage' Display option summary '--verbose' '-v' Increase verbosity level. Implies '--gacopyz-log=debug', unless that option is used explicitly. '--version' '-V' Print program version  File: mailfromd.info, Node: pmult, Next: Reporting Bugs, Prev: mtasim, Up: Top 13 Pmilter multiplexer program. ******************************* 'Pmult' is a "Pmilter-Milter multiplexer", i.e. a program that acts as a mediator between the Pmilter server and one or several Milter clients. Usually, the former is an instance of 'smtps' from MeTA1, and the latter are running 'mailfromd' instances. 'Pmult' receives Pmilter commands from the server, translates them into equivalent Milter commands and passes the translated requests to a preconfigured set of Milter filters. When the filters reply, the reverse operation is performed: Milter responses are translated into their Pmilter equivalents and are sent back to the server. +-----------------+ +----->| Milter Client 1 | | +-----------------+ | +-----------+ +---------+ | +-----------------+ | MeTA1 |<=====>| Pmult |<--+----->| Milter Client 1 | +-----------+ +---------+ | +-----------------+ | +---------> // | | +-----------------+ +----->| Milter Client N | +-----------------+ Due to the specifics nature of the threaded MeTA1 libraries, 'pmult' does not detach from the controlling terminal (i.e. does not become a daemon). To run it as a background process, we recommend to use 'pies' daemon. 'Pies' is a powerful utility that allows you to launch several foreground-designed programs in the background and control their execution. *Note Pies Manual: (pies)Top, for a detailed description of the program. For a practical advice on how to use it with 'pmult', see *note Using Pies to Run Pmult: (pies)Simple Pies. For a description on how to start both 'pmult' and MeTA1 from the same 'pies' configuration file, see *note Using Pies to Run Pmult and MeTA1: (pies)Hairy Pies. * Menu: * pmult configuration:: * pmult example:: * pmult invocation::  File: mailfromd.info, Node: pmult configuration, Next: pmult example, Up: pmult 13.1 Pmult Configuration ======================== 'Pmult' reads its configuration from the main configuration file '/etc/mailfromd.conf'. Unless it is the only component of the 'Mailfromd' package that is being run, its configuration should be explicitly marked as such by using either 'program' or 'include' statement, as described in *note Mailfromd Configuration::. The following standard Mailutils statements are understood: Statement Reference ------------------------------------------------------------------- debug *Note Mailutils Configuration File: (mailutils)debug statement. logging *Note Mailutils Configuration File: (mailutils)logging statement. include *Note Mailutils Configuration File: (mailutils)include. * Menu: * pmult-conf:: Multiplexer Configuration. * pmult-macros:: Translating MeTA1 macros. * pmult-client:: Pmult Client Configuration. * pmult-debug:: Debugging Pmult.  File: mailfromd.info, Node: pmult-conf, Next: pmult-macros, Up: pmult configuration 13.1.1 Multiplexer Configuration. --------------------------------- 'Pmult' listens for Pmilter requests on a socket, configured using 'listen' statement: -- Pmult Conf: listen URL Listen on the given URL. Argument is a valid Mailutils URL. *Note milter port specification::, for a description of URL. Since 'pmult' runs as a foreground program, it does not write its PID number to a file by default. If this behavior is required, it can be enabled using the following statement: -- Pmult Conf: pidfile FILE Store PID of the 'pmult' process in FILE. The following three limits require MeTA1 version 'PreAlpha30.0' or later. -- Pmult Conf: max-threads-soft N "Soft" limit on the number of 'pmilter' threads. Default is 2. -- Pmult Conf: max-threads-hard N "Hard" limit on the number of 'pmilter' threads. This is roughly equivalent to the number of emails 'pmult' is able to handle simultaneously. The default value is 6. Raise this limit if you experience long delays when connecting to the SMTP port. -- Pmult Conf: max-pmilter-fd N Maximum number of file descriptors 'pmilter' library is allowed to open simultaneously. Default is 10.  File: mailfromd.info, Node: pmult-macros, Next: pmult-client, Prev: pmult-conf, Up: pmult configuration 13.1.2 Translating MeTA1 macros. -------------------------------- MeTA1's notion of macros differs considerably from that of Sendmail. Macros in MeTA1 are identified by integer numbers and only a limited number of macros can be provided for each "Pmilter stage". Pmilter stages mostly correspond to Milter states (*note handler names::), except that there are no distinct header and body stages, instead these two are combined into a single 'data' stage. This comes unnoticed to mailfromd scripts, because 'pmult' takes care to invoke right Milter handlers within the single 'data' Pmilter state. Therefore in the discussion that follows we will refer to Mailfromd handlers, rather than to Pmilter stages. The most important standard Milter macros are always provided by 'pmult' itself. These are: client_addr The IP address of the SMTP client. As of version 8.8, only IPv4 addresses are supported. Defined in all handlers. client_port The port number of the SMTP client. Defined in all handlers. i MeTA1 session ID. Defined in all handlers. f The envelope sender (from) address. Defined in 'envfrom' and subsequent handlers. nbadrcpts The number of bad recipients for a single message. Defined in 'envfrom' and 'envrcpt' handlers. ntries The number of delivery attempts. As of version 8.8 it is always '1'. Defined in 'envfrom' and subsequent handlers. nrcpts The number of validated recipients for a single message. Defined in 'envfrom' and 'envrcpt' handlers. r Protocol used to receive the message. The value of this macro is always 'SMTP'. Defined in all handlers. rcpt_host The host from the resolved triple of the address given for the SMTP RCPT command. Defined in 'envrcpt' handler. rcpt_addr The address part of the resolved triple of the address given for the SMTP RCPT command. Defined in 'envrcpt' handler. s Sender's helo domain (parameter to 'EHLO' or 'HELO' command). Two additional macros are provided for all handlers that allow to identify whether the message is processed via 'pmult': multiplexer Canonical name of the multiplexer program, i.e. 'pmult'. mult_version Version of 'pmult'. These macros can be used in mailfromd filters to provide alternative processing for messages coming from a MeTA1 server. Macros defined in MeTA1 can be made available in Mailfromd handlers using the 'define-macros' statement. -- Pmult Conf: define-macros HANDLER MACROS Define a set of Sendmail macros for the given Mailfromd handler. Allowed values for HANDLER are: 'connect', 'helo', 'mail' (or 'envfrom'), 'rcpt' (or 'envrcpt'), 'data' (or 'header' or 'body'), 'dot' ('eom'). A list of these values is also accepted, in which case MACROS are defined for each handler from the list. The second argument specifies a list of names of the macros that should be defined in this handler. Allowed macro names are: hostname Hostname of SMTP server. client_resolve Result of client lookup. tls_version TLS/SSL version used. tls_cipher_suite TLS cipher suite used. tls_cipher_bits Effective key length of the symmetric encryption algorithm. tls_cert_subject The DN (distinguished name) of the presented certificate. tls_cert_issuer The DN (distinguished name) of the CA (certificate authority) that signed the presented certificate (the cert issuer). tls_alg_bits Maximum key length of the symmetric encryption algorithm. This may be less than the effective key length for export controlled algorithms. tls_vrfy The result of the verification of the presented cert. tls_cn_subject cn_subject The CN (common name) of the presented certificate. tls_cn_issuer cn_issuer The CN (common name) of the CA that signed the presented certificate. auth_type The mechanism used for SMTP authentication (only set if successful). auth_authen The client's authentication credentials as determined by authentication (only set if successful). The actual format depends on the mechanism used, it might be just 'user', or 'user@realm', or something similar. auth_author The authorization identity, i.e. the 'AUTH=' parameter of the SMTP MAIL command if supplied. taid MeTA1 transaction id. msgid Message-Id of the message. c The hop count. Basically, this is the number of 'Received:' headers. Notice the following limitations: 1. 'taid' cannot be requested before 'mail' stage. 2. 'msgid' can be requested only in 'dot' stage. 3. All 'tls_*' macros are valid only after a 'STARTTLS' command. 4. The number of MeTA1 macros per stage is limited by 'PM_MAX_MACROS' define in 'include/sm/pmfdef.h'. In MeTA1 versions up to and including 1.0.PreAlpha28.0, this number is 8. If you need more macros, increase this number and recompile MeTA1. -- Pmult Conf: auth-macros BOOL If BOOL is 'true' (*note boolean: (mailutils)Statements.), pass auth macros to mailfromd 'mail' handler. It is equivalent to: define-macros mail (auth_type, auth_authen, auth_author);  File: mailfromd.info, Node: pmult-client, Next: pmult-debug, Prev: pmult-macros, Up: pmult configuration 13.1.3 Pmult Client Configuration. ---------------------------------- In 'pmult' terminology, remote Milters are "clients". The number of clients 'pmult' is able to handle is not limited. Each client is declared using 'client' statement: client [IDENT] { # Set remote protocol type. type PROTOCOL-TYPE; # Set remote client URL. url ARG; # Set write timeout. write-timeout DURATION; # Set read timeout. read-timeout DURATION; # Set timeout for EOM. eom-timeout DURATION; # Set connect timeout. connect-timeout DURATION; # Set log verbosity level. log-level LEVEL; }; -- Pmult Conf: client [IDENT] { STATEMENTS } Declare a Milter client. Optional IDENT gives the identifier of this client, which will be used in diagnostics messages. STATEMENTS are described below. -- Pmult Conf: type TYPESTR This statement is reserved for future use. In version 8.8 it is a no-op. If given, the value of TYPESTR must be 'milter'. In future versions this statement will declare the protocol to be used to interact with this client. The syntax for TYPESTR is TYPE [VERSION] where TYPE is either 'milter' or 'pmilter', and optional VERSION is minimal protocol version. -- Pmult Conf: url ARG Set remote client URL. *Note milter port specification::, for a description of URL. -- Pmult Conf: connect-timeout INTERVAL Configure Milter initial connection timeout. Default is 300. *Note time interval specification::, for information on INTERVAL format. -- Pmult Conf: write-timeout INTERVAL Configure Milter write timeout. Default is 10. *Note time interval specification::, for information on INTERVAL format. -- Pmult Conf: read-timeout INTERVAL Configure Milter read timeout. Default is 10. *Note time interval specification::, for information on INTERVAL format. -- Pmult Conf: eom-timeout INTERVAL Configure Milter end of message timeout. Default is 300. *Note time interval specification::, for information on INTERVAL format. -- Pmult Conf: log-level ARG Set Milter log verbosity level for this client. Argument is a list of items separated by commas or whitespace. Each item is a log level optionally prefixed with '!' to indicate "any level except this", '<', meaning "all levels up to and including this", or with '>', meaning "all levels starting from this". Log levels in order of increasing priority are: 'proto', 'debug', 'info', 'warn', 'err', 'fatal'. The first two levels are needed for debugging 'libgacopyz' and Milter protocol. *Note Gacopyz::, for the description of the 'libgacopyz' library. See also the following subsection.  File: mailfromd.info, Node: pmult-debug, Prev: pmult-client, Up: pmult configuration 13.1.4 Debugging Pmult ---------------------- If needed, 'pmult' can be instructed to provide additional debugging information. The amount of this information is configured by three configuration statements. First of all, the standard 'debug' block statement controls debugging of the underlying GNU Mailutils libraries (*note Mailutils Configuration File: (mailutils)Debug Statement.). Secondly, the 'debug' statement controls debugging output of the 'pmult' utility itself. The 'pmilter-debug' statement controls debugging output of the underlying MeTA1 libraries, and, finally, the 'log-level' statement, described in the previous subsection, defines debugging level for the Milter library ('libgacopyz'). -- Pmult Conf: debug SPEC Set debugging level for the 'pmult' code. *Note Mailutils Configuration File: (mailutils)Debug Statement, for a description of SPEC syntax. Multiplexor-specific debugging is enabled by the 'pmult' category. The following levels are used: pmult.trace1 Prints the following information: * opening and closing incoming connections; * entering particular Pmilter stage handlers; * received requests with unknown command code; * header modification requests that does not match any headers. pmult.trace2 Information about milter to Pmilter request translation. pmult.trace7 Detailed dump of message body chunks received during Pmilter 'DATA' stage. pmult.error Logs bad recipient addresses. This information is printed using the output channel defined in the 'logging' statement (*note Mailutils Configuration File: (mailutils)Logging Statement.). -- Pmult Conf: pmilter-debug LEVEL Set debug verbosity level for the Pmilter library. Argument is a positive integer between zero (no debugging, the default), and 100 (maximum debugging). Pmilter debugging information is printed on the standard error. Use 'pies' 'stderr' statement to capture this stream and redirect it to the syslog or file (*note (pies)Output Redirectors::).  File: mailfromd.info, Node: pmult example, Next: pmult invocation, Prev: pmult configuration, Up: pmult 13.2 Pmult Example ================== The following is an example of a working 'pmult' configuration. The multiplexer listens on localhost, port '3333'. It prints its diagnostics using syslog facility 'local2'. A single Mailfromd client is declared, which listens on UNIX socket '/usr/local/var/mailfromd/mailfrom'. The log verbosity level for this client is set to 'info' and higher, i.e.: 'info', 'warn', 'err' and 'fatal'. listen inet://127.0.0.1:3333; logging { facility local2; }; debug { level "pmult.trace7"; } define-macros envmail (auth_type, auth_authen, auth_author, tls_vrfy); define-macros envrcpt (auth_type, auth_authen, auth_author); client { type milter; url /usr/local/var/mailfromd/mailfrom; log-level ">info"; # Set write timeout. write-timeout 30 seconds; # Set read timeout. read-timeout 5 minutes; # Set timeout for EOM. eom-timeout 5 minutes; }  File: mailfromd.info, Node: pmult invocation, Prev: pmult example, Up: pmult 13.3 Pmult Invocation ===================== Normally, 'pmult' is invoked without command line arguments. However, it does support several command line options. First of all, the common GNU Mailutils options are understood, which are useful for checking 'pmult' configuration file for syntax errors. *Note Common Options: (mailutils)Common Options, for a detailed description of these. The rest of command line options supported by 'pmult' is useful mostly for debugging. These options are summarized in the table below: '--log-tag=STRING' Set the identifier used in syslog messages to STRING. This option mostly is for debugging purposes. We advise to use 'logging' configuration statement for this purpose (*note Logging Statement: (mailutils)Logging Statement.). '--no-signal-handler' Disable signal handling in the main thread. This is for debugging purposes. '--syslog' Log to the syslog. This is the default. *Note Logging Statement: (mailutils)Logging Statement, for information on how to configure syslog logging. '-s' '--stderr' Log to the standard error stream. '--url=URL' Listen on the given URL. This overrides the 'url' configuration statement (*note url: pmult-client.). '-x' '--debug=LEVEL' Set debug verbosity level. This overrides the 'debug' configuration statement. *Note pmult-debug::, for more information.  File: mailfromd.info, Node: Reporting Bugs, Next: Gacopyz, Prev: pmult, Up: Top 14 How to Report a Bug ********************** Documentation is like sex: when it is good, it is very, very good; and when it is bad, it is better than nothing. Dick Brandon Although the author has tried to make this documentation as detailed as is possible and practical, he is well aware that the result is rather "better than nothing", than "very good". So, if you find that some piece of explanation is lousy or if you find anything that should have been mentioned here, but is not, please report it to . Similarly, if the program itself fails to meet your expectations, or does not do what is described in this document; if you have found a bug or happen to have any suggestion... or have written a useful function you wish to share with the rest of 'mailfromd' users, or wish to express your thanks, email it to the same address, . If you think you've found a bug, please be sure to include maximum information needed to reliably reproduce it, or at least to analyze it. The information needed is: * Version of the package you are using. * Compilation options used when configuring the package. * Run-time configuration ('mailfromd.mf' file and the command line options used). * Conditions under which the bug appears.  File: mailfromd.info, Node: Gacopyz, Next: Time and Date Formats, Prev: Reporting Bugs, Up: Top Appendix A Gacopyz ****************** Gacopyz, panie, to mówia̧ ze to mysa... Ze to tako mysa co świeckȩ w kościele zjadła i wniebowsta̧pienia dosta̧piła. A to nie je mysa, ino gacopyz! To nadprzyrodzłune, to głowa̧ na dół śpi! Kazimierz Grześkowiak 'Gacopyz' is the client library implementing Milter protocol. It differs considerably from the Sendmail implementation and offers a new and more flexible API. The old API is supported for compatibility with 'libmilter'. The library name comes from the song 'Rozprawa o robokach' by Kazimierz Grzeskowiak (http://grzeskowiak.art.pl). The phrase 'A to nie je mysa, ino gacopyz' exactly describes what the library is: 'That is no libmilter, but gacopyz'. Future versions of this documentation will include a detailed description of the library.  File: mailfromd.info, Node: Time and Date Formats, Next: s-expression, Prev: Gacopyz, Up: Top Appendix B Time and Date Formats ******************************** This appendix documents the time format specifications understood by the command line option '--time-format' (*note --time-format::). Essentially, it is a reproduction of the man page for GNU 'strftime' function. Ordinary characters placed in the format string are reproduced without conversion. Conversion specifiers are introduced by a '%' character, and are replaced as follows: %a The abbreviated weekday name according to the current locale. %A The full weekday name according to the current locale. %b The abbreviated month name according to the current locale. %B The full month name according to the current locale. %c The preferred date and time representation for the current locale. %C The century number (year/100) as a 2-digit integer. %d The day of the month as a decimal number (range 01 to 31). %D Equivalent to '%m/%d/%y'. %e Like '%d', the day of the month as a decimal number, but a leading zero is replaced by a space. %E Modifier: use alternative format, see below (*note conversion specs::). %F Equivalent to '%Y-%m-%d' (the ISO 8601 date format). %G The ISO 8601 year with century as a decimal number. The 4-digit year corresponding to the ISO week number (see '%V'). This has the same format and value as '%y', except that if the ISO week number belongs to the previous or next year, that year is used instead. %g Like '%G', but without century, i.e., with a 2-digit year (00-99). %h Equivalent to '%b'. %H The hour as a decimal number using a 24-hour clock (range 00 to 23). %I The hour as a decimal number using a 12-hour clock (range 01 to 12). %j The day of the year as a decimal number (range 001 to 366). %k The hour (24-hour clock) as a decimal number (range 0 to 23); single digits are preceded by a blank. (See also '%H'.) %l The hour (12-hour clock) as a decimal number (range 1 to 12); single digits are preceded by a blank. (See also '%I'.) %m The month as a decimal number (range 01 to 12). %M The minute as a decimal number (range 00 to 59). %n A newline character. %O Modifier: use alternative format, see below (*note conversion specs::). %p Either 'AM' or 'PM' according to the given time value, or the corresponding strings for the current locale. Noon is treated as 'pm' and midnight as 'am'. %P Like '%p' but in lowercase: 'am' or 'pm' or a corresponding string for the current locale. %r The time in 'a.m.' or 'p.m.' notation. In the POSIX locale this is equivalent to '%I:%M:%S %p'. %R The time in 24-hour notation ('%H:%M'). For a version including the seconds, see '%T' below. %s The number of seconds since the Epoch, i.e., since 1970-01-01 00:00:00 UTC. %S The second as a decimal number (range 00 to 61). %t A tab character. %T The time in 24-hour notation ('%H:%M:%S'). %u The day of the week as a decimal, range 1 to 7, Monday being 1. See also '%w'. %U The week number of the current year as a decimal number, range 00 to 53, starting with the first Sunday as the first day of week 01. See also '%V' and '%W'. %V The ISO 8601:1988 week number of the current year as a decimal number, range 01 to 53, where week 1 is the first week that has at least 4 days in the current year, and with Monday as the first day of the week. See also '%U' and '%W'. %w The day of the week as a decimal, range 0 to 6, Sunday being 0. See also '%u'. %W The week number of the current year as a decimal number, range 00 to 53, starting with the first Monday as the first day of week 01. %x The preferred date representation for the current locale without the time. %X The preferred time representation for the current locale without the date. %y The year as a decimal number without a century (range 00 to 99). %Y The year as a decimal number including the century. %z The time-zone as hour offset from GMT. Required to emit RFC822-conformant dates (using '%a, %d %b %Y %H:%M:%S %z') %Z The time zone or name or abbreviation. %+ The date and time in 'date(1)' format. %% A literal '%' character. Some conversion specifiers can be modified by preceding them by the 'E' or 'O' modifier to indicate that an alternative format should be used. If the alternative format or specification does not exist for the current locale, the behaviour will be as if the unmodified conversion specification were used. The Single Unix Specification mentions '%Ec', '%EC', '%Ex', '%EX', '%Ry', '%EY', '%Od', '%Oe', '%OH', '%OI', '%Om', '%OM', '%OS', '%Ou', '%OU', '%OV', '%Ow', '%OW', '%Oy', where the effect of the 'O' modifier is to use alternative numeric symbols (say, roman numerals), and that of the 'E' modifier is to use a locale-dependent alternative representation.  File: mailfromd.info, Node: s-expression, Next: Upgrading, Prev: Time and Date Formats, Up: Top Appendix C S-Expression *********************** An "s-expression" is a 'sed'-like transformation expression: s/REGEXP/REPLACE/[FLAGS] where REGEXP is a "regular expression", REPLACE is a replacement for each part of the input that matches REGEXP. Both REGEXP and REPLACE are described in detail in *note The "s" Command: (sed)The "s" Command. As in 'sed', you can give several replace expressions, separated by a semicolon. Supported FLAGS are: 'g' Apply the replacement to _all_ matches to the REGEXP, not just the first. 'i' Use case-insensitive matching 'x' REGEXP is an "extended regular expression" (*note Extended regular expressions: (sed)Extended regexps.). 'NUMBER' Only replace the NUMBERth match of the REGEXP. Note: the POSIX standard does not specify what should happen when you mix the 'g' and NUMBER modifiers. 'Mailfromd' follows the GNU 'sed' implementation in this regard, so the interaction is defined to be: ignore matches before the NUMBERth, and then match and replace all matches from the NUMBERth on. Any delimiter can be used in lieue of '/', the only requirement being that it be used consistently throughout the expression. For example, the following two expressions are equivalent: s/one/two/ s,one,two, Changing delimiters is often useful when the REGEX contains slashes. For instance, it is more convenient to write 's,/,-,' than 's/\//-/'.  File: mailfromd.info, Node: Upgrading, Next: Copying This Manual, Prev: s-expression, Up: Top Appendix D Upgrading ******************** The following sections describe procedures for upgrading between the consecutive Mailfromd releases. The absense of a section for a pair of versions X-Y numbers means that no specific actions are required for upgrading from X to Y. * Menu: * 870-880:: Upgrading from 8.7 to 8.8 * 850-860:: Upgrading from 8.5 to 8.6 * 820-830:: Upgrading from 8.2 to 8.3 (or 8.4) * 700-800:: Upgrading from 7.0 to 8.0 * 600-700:: Upgrading from 6.0 to 7.0 * 5x0-600:: Upgrading from 5.x to 6.0 * 500-510:: Upgrading from 5.0 to 5.1 * 440-500:: Upgrading from 4.4 to 5.0 * 43x-440:: Upgrading from 4.3.x to 4.4 * 420-43x:: Upgrading from 4.2 to 4.3.x * 410-420:: Upgrading from 4.1 to 4.2 * 400-410:: Upgrading from 4.0 to 4.1 * 31x-400:: Upgrading from 3.1.x to 4.0 * 30x-31x:: Upgrading from 3.0.x to 3.1 * 2x-30x:: Upgrading from 2.x to 3.0.x * 1x-2x:: Upgrading from 1.x to 2.x  File: mailfromd.info, Node: 870-880, Next: 850-860, Up: Upgrading D.1 Upgrading from 8.7 to 8.8 ============================= DKIM support (*note DKIM::) introduced in this version requires the Nettle cryptographic library(1). It you need DKIM, make sure Nettle is installed prior to compiling 'mailfromd'. Otherwise, no special actions are required. ---------- Footnotes ---------- (1)  File: mailfromd.info, Node: 850-860, Next: 820-830, Prev: 870-880, Up: Upgrading D.2 Upgrading from 8.5 to 8.6 ============================= New configure option '--with-dbm' allows you to select any DBM flavor supported by GNU mailutils as the default DBM implementation for 'mailfromd'.  File: mailfromd.info, Node: 820-830, Next: 700-800, Prev: 850-860, Up: Upgrading D.3 Upgrading from 8.2 to 8.3 (or 8.4) ====================================== Versions 8.3 and 8.4 differ only in required minimal version of mailutils (3.3 and 3.4, correspondingly). Apart from that, the following instructions apply to both versions. In version 8.3 I abandoned the legacy DNS resolver and switched to "GNU adns". GNU ands is a standalone resolver library, which provides a number of advanced features. It is included in most distributions. The source code of the recent release is available from . This change brought a number of user-visible changes. In particular, arbitrary limits on the sizes of the RRs are removed. Consequently, the following configurations statements are withdrawn: 'runtime.max-dns-reply-a' 'runtime.max-dns-reply-ptr' 'runtime.max-dns-reply-mx' 'max-match-mx' 'max-callout-mx' Secondly, the new resolver is less tolerant to deviations from the standard. This means that remote DNS misconfigurations that would have slipped unnoticed in previous versions, will be noticed by mailfromd 8.3. For example, a CNAME record pointing to another CNAME is treated as an error. A new command line option was added to 'mailfromd' and 'calloutd': '--resolv-conf-file'. This option instructs the programs to read resolver settings from the supplied file, instead of the default '/etc/resolv.conf'. Another user-visible change is in handling of SPF checks. Previously, results if SPF checks were cached in a database. This proved to cause more problems than solutions and was removed in this version. As a result, the following MFL global variables have been withdrawn: 'spf_ttl' 'spf_cached' 'spf_database' 'spf_negative_ttl'  File: mailfromd.info, Node: 700-800, Next: 600-700, Prev: 820-830, Up: Upgrading D.4 Upgrading from 7.0 to 8.0 ============================= Version 8.0 is a major rewrite, that introduces a lot of new concepts and features. Nevertheless, it is still able to run the MFL scripts from version 7.0 without modifications. Note the following important points: * The 'listen' configuration statement withdrawn Use the 'server milter' statement instead. *Note conf-server::. * The '--remove' option withdrawn This option was a noop since version 7.0.91. * The use of '%' before variable names is no longer supported The '%' characters is used as modulo operator. *Note Arithmetic operations::. * The 'debug_spec' built-in function changed signature. *Note debug_spec::. * 'listens' and 'portprobe' The 'listens' function was moved to the 'portprobe' module. It is actually an alias to the 'portprobe' function. If your filter uses 'listens', make sure to 'require' the 'portprobe' module. *Note portprobe: Special test functions. * '_pollhost', '_pollmx', 'stdpoll', 'strictpoll' These functions have been moved to the 'poll' module, which must be required prior to using any of them. * The 'message_header_count' function. This function takes an optional string argument, supplying the header name. *Note message_header_count::.  File: mailfromd.info, Node: 600-700, Next: 5x0-600, Prev: 700-800, Up: Upgrading D.5 Upgrading from 6.0 to 7.0 ============================= The release 7.0 removes the features which were declared as obsolete in 6.0 and introduces important new features, both syntactical, at the MFL level, and operational. Unless your filter used any deprecated features, it should work correctly after upgrade to this version. It will, however, issue warning messages regarding the deprecated features (e.g. the use of '%' in front of identifiers, as described below). To fix these, follow the upgrade procedure described in *note upgrade procedure::. The removed features are: * Old-style functional notation * The use of functional operators * Implicit concatenations * #pragma option * #pragma database The MFL syntax has changed: it is no longer necessary to use '%' in front of a variable to get its value. To reference a variable, simply use its name, e.g.: set x var + z The old syntax is still supported, so the following statement will also work: set x %var + %z It will, however, generate a warning message. Of course, the use of '%' to reference variables within a string literal remains mandatory. Another important changes to MFL are user-defined exceptions (*note User-defined Exceptions::) and the "try-catch" construct (*note try-catch: Catch and Throw.). Several existing MFL functions have been improved. In particular, it is worth noticing that the 'open' function, when opening a pipe to or from a command, provides a way to control where the command's standard error would go (*note standard error: open.). The 'accept' function (or action) issues a warning if its use would cancel any modifications to the message applied by, e.g., 'header_add' and similar functions. *Note Message modification queue::, for a detailed discussion of this feature. The most important change in 'mailfromd' operation is that the version 7.0 is able to run several servers (*note conf-server::). Dedicated "callout servers" make it possible to run sender verifications in background, using a set of long timeouts, as prescribed by RFC 2822 (*note SMTP Timeouts::). This diminishes the number of false positives, allows for coping with servers showing large delays and also reduces the number of callouts performed for such servers. This release no longer includes the 'smap' utility. It was moved into a self-standing project, which by now provides much more functionality and is way more flexible than this utility was. If you are interested in using 'smap', visit , for a detailed information, including pointers to file downloads.  File: mailfromd.info, Node: 5x0-600, Next: 500-510, Prev: 600-700, Up: Upgrading D.6 Upgrading from 5.x to 6.0 ============================= The 6.0 release is aimed to fix several logical inconsistencies that affected the previous versions. The most important one is that until version 5.2, the filter script file contained both the actual filter script, and the run-time configuration for 'mailfromd' (in form of '#pragma option' and '#pragma database' statements). The new version separates run-time configuration from the filter script by introducing a special configuration file 'mailfromd.conf' (*note Mailfromd Configuration::). Consequently, the '#pragma option' and '#pragma database' statements become deprecated. Furthermore, the following deprecated pragmas are removed: '#pragma option ehlo', '#pragma option mailfrom'. These pragmas became deprecated in version 4.0 (*note 31x-400::). The second problem was that the default filter script file had '.rc' suffix, which usually marks a configuration file, not the source. In version 6.0 the script file is renamed to 'mailfromd.mf'. In the absence of this file, the legacy file 'mailfromd.rc' is recognized and parsed. This ensures backward compatibility. This release also fixes various inconsistencies and dubious features in the MFL language. The support for unquoted literals is discontinued. This feature was marked as deprecated in version 3.0. The following features are deprecated: '#pragma option' (pragma-option (http://mailfromd.man.gnu.org.ua/historic/6/html_node/pragma_002doption.html) and '#pragma database' (pragma-database (http://mailfromd.man.gnu.org.ua/historic/6/html_node/pragma_002ddatabase.html)) directives, the legacy style of function declarations (old-style function declarations (http://mailfromd.man.gnu.org.ua/historic/6/html_node/old_002dstyle-function-declarations.html)), calls to functions of one argument without parentheses (operational notation (http://mailfromd.man.gnu.org.ua/historic/6/html_node/implicit-concatenation.html)), the '#require' statement (*Note import::, for the new syntax) and implicit concatenation (implicit concatenation (http://mailfromd.man.gnu.org.ua/historic/6/html_node/implicit-concatenation.html)). See Deprecated Features (http://mailfromd.man.gnu.org.ua/historic/6/html_node/Deprecated-Features.html), for more information about these. This release also introduces important new features, which are summarized in the table below: Feature Reference -------------------------------------------------------------------------- Configuration *Note Mailfromd Configuration::. Module system *Note Modules::. Explicit type casts *Note explicit type casts::. Concatenation operator *Note Concatenation::. Scope of visibility *Note scope of visibility::. Precious variables *Note rset::. 'Mailfromd' version '6.0' will work with unchanged scripts from '5.x'. When started, it will verbosely warn you about any deprecated constructs that are used in your filter sources and will create a script for upgrading them. To upgrade your filter scripts, follow the steps below: 1. Run 'mailfromd --lint'. You will see a list of warnings similar to this: mailfromd: Warning: using legacy script file /usr/local/etc/mailfromd.rc mailfromd: Warning: rename it to /usr/local/etc/mailfromd.mf or use script-file statement in /usr/local/etc/mailfromd.conf to disable this warning mailfromd: /usr/local/etc/mailfromd.rc:19: warning: this pragma is deprecated: use relayed-domain-file configuration statement instead mailfromd: /usr/local/etc/mailfromd.rc:23: warning: this pragma is deprecated: use io-timeout configuration statement instead mailfromd: Info: run script /tmp/mailfromd-newconf.sh to fix the above warnings ... 2. At the end of the run 'mailfromd' will create a shell script '/tmp/mailfromd-newconf.sh' for fixing these warnings. Run it: $ sh /tmp/mailfromd-newconf.sh 3. When the script finishes, run 'mailfromd --lint' again. If it shows no more deprecation warnings, the conversion went correctly. Now you can remove the upgrade script: $ rm /tmp/mailfromd-newconf.sh Notice, that the conversion script attempts to fix only deprecation warnings. It will not try to correct any other type of warnings or errors. For example, you may get warning messages similar to: mailfromd: /etc/mailfromd.mf:7: warning: including a module file is unreliable and may cause subtle errors mailfromd: /etc/mailfromd.mf:7: warning: use `require dns' instead This means that you use '#include' where you should have used 'require'. You will have to fix such warnings manually, as suggested in the warning message. If, for some reason, you cannot upgrade your scripts right now, you may suppress deprecation warnings by setting the environment variable 'MAILFROMD_DEPRECATION' to 'no' before starting 'mailfromd'. Nonetheless, I recommend to upgrade as soon as possible, because the deprecated features will be removed in version '6.1'.  File: mailfromd.info, Node: 500-510, Next: 440-500, Prev: 5x0-600, Up: Upgrading D.7 Upgrading from 5.0 to 5.1 ============================= Upgrading from 5.0 to 5.1 does not require any changes in your filter scripts. Notice, however, the following important points: * Starting from this release 'mailfromd' supports Milter protocol version 6, which is compatible with Sendmail 8.14.0 and newer. While being backward compatible with earlier Sendmail releases, it allows you to use the new 'prog data' handler (*note data: Handlers.). It also supports macro negotiation, a feature that enables Mailfromd to ask MTA to export the macros it needs for each particular handler. This means that if you are using Sendmail 8.14.0 or higher (or Postfix 2.5 or higher), you no longer need to worry about exporting macro names in 'sendmail.cf' file. The same feature is also implemented on the server side, in 'mtasim' and 'pmult'. Consequently, using 'define-macros' in 'pmult' configuration file is not strictly necessary. However, keep in mind that due to the specifics of MeTA1, the number of symbols that may be exported for each stage is limited (*note pmult-macros::). * The semantics of '__preproc__' and '__statedir__' built-in constant is slightly different from what it used to be in 5.0. These constants now refer to the _current_ values of the preprocessor command line and program state directory, correspondingly. This should not affect your script, unless you redefine the default values on run time. If your script needs to access default values, use '__defpreproc__' and '__defstatedir__', correspondingly (*note Built-in constants::). The following example explains the difference between these: $ cat pval.mf prog envfrom do echo "Default value: " __defstatedir__ echo "Current value: " __statedir__ done $ mailfromd --state-directory=/var/mfd --test pval.mf Default value: /usr/local/var/mailfromd Current value: /var/mfd * If your filter uses the 'rate' function, you might consider using the new function 'rateok' or 'tbf_rate' instead. For a detailed discussion of these functions, see *note Sending Rate::. * If your script extensively uses database access functions, you might be interested in the new '#pragma dbprop' (*note dbprop::).  File: mailfromd.info, Node: 440-500, Next: 43x-440, Prev: 500-510, Up: Upgrading D.8 Upgrading from 4.4 to 5.0 ============================= This version of Mailfromd requires GNU mailutils (http://www.gnu.org/software/mailutils) version 2.0 or later. Upgrading from version 4.4 to 5.0 requires no additional changes. The major differences between these two versions are summarized below: 1. Support for 'MeTA1'. 2. New 'Mailutils' configuration file. 3. New MFL functions. a. Message functions. *Note Message functions::. b. Mailbox functions. *Note Mailbox functions::. c. Mail body functions. *Note Mail body functions::. d. Header modification functions. *Note Header modification functions::. e. Envelope modification functions. *Note Envelope modification functions::. f. Quarantine functions. *Note Quarantine functions::. g. 'getopt' and 'varptr'. *Note getopt::. h. Macro access functions. *Note Macro access::. i. Character type functions. *Note Character Type::. j. New string functions (*note String manipulation::): 'verp_extract_user', 'sa_format_report_header', 'sa_format_score'. k. Sequential access to DBM files. *Note dbm-seq::. 4. Changes in MFL 1. *Note variadic functions::. 2. *Note function alias::. 5. New operation mode: *Note Run Mode::. 6. Improved stack growth technique. The stack can be grown either by fixed size blocks, or exponentially. Upper limit can be specified. *Note stacksize::. 7. Milter ports can be specified using URL notation. 8. Removed deprecated features. Support for some deprecated features has been withdrawn. These are: a. Command line options '--ehlo', '--postmaster-email', and '--mailfrom'. These became deprecated in version 4.0. *Note 31x-400::.  File: mailfromd.info, Node: 43x-440, Next: 420-43x, Prev: 440-500, Up: Upgrading D.9 Upgrading from 4.3.x to 4.4 =============================== The deprecated '--domain' command line option has been withdrawn. The short option '-D' now defines a preprocessor symbol (*note Preprocessor Options::). This version correctly handles name clashes between constants and variables, which remained unnoticed in previous releases. *Note variable--constant shadowing::, for a detailed description of it. To minimize chances of name clashes, all symbolic exception codes has been renamed by prefixing them with the 'e_', thus, e.g. 'divzero' became 'e_divzero', etc. The 'ioerr' exception code is renamed to 'e_io'. *Note status.mf::, for a full list of the new exception codes. For consistency, the following most often used codes are available without the 'e_' prefix: success, not_found, failure, temp_failure. This makes most existing user scripts suitable for use with version 4.4 without any modification. If your script refers to any exception codes other than these four, you can still use it by defining a preprocessor symbol 'OLD_EXCEPTION_CODES', for example: $ mailfromd -DOLD_EXCEPTION_CODES  File: mailfromd.info, Node: 420-43x, Next: 410-420, Prev: 43x-440, Up: Upgrading D.10 Upgrading from 4.2 to 4.3.x ================================ Upgrading from 4.2 to 4.3 or 4.3.1 does not require any changes to your configuration and scripts. The only notable change in these versions is the following: The asynchronous syslog implementation was reported to malfunction on some systems (notably on Solaris), so this release does not enable it by default. The previous meaning of the '--enable-syslog-async' configuration option is also restored. Use this option in order to enable asynchronous syslog feature. To set default syslog implementation, use 'DEFAULT_SYSLOG_ASYNC' configuration variable (*note syslog-async::). The following deprecated features are removed: 1. '#pragma option ehlo' statement. It became deprecated in version 4.0. *Note pragma-option-ehlo::. 2. '#pragma option mailfrom' statement. It became deprecated in version 4.0. *Note pragma-option-ehlo::. 3. The '--config-file' command line option. It became deprecated in version 3.1. *Note 30x-31x::. 4. Built-in exception codes in catch statements. They are deprecated since version 4.0. *Note 31x-400::.  File: mailfromd.info, Node: 410-420, Next: 400-410, Prev: 420-43x, Up: Upgrading D.11 Upgrading from 4.1 to 4.2 ============================== Upgrading to this version does not require any special efforts. You can use your configuration files and filter scripts without any changes. The only difference worth noticing is that starting from this version 'mailfromd' is always compiled with asynchronous syslog implementation. The '--enable-syslog-async' configuration file option is still available, but its meaning has changed: it sets the _default_ syslog implementation to use (*note syslog-async::). Thus, it can be used the same way it was in previous versions. You can also select the syslog implementation at run time, see *note -syslog-async option: Logging and Debugging, for more detailed information.  File: mailfromd.info, Node: 400-410, Next: 31x-400, Prev: 410-420, Up: Upgrading D.12 Upgrading from 4.0 to 4.1 ============================== Upgrading to this version does not require any special efforts. You can use your configuration files and filter scripts without any changes. Notice only the following major differences between 4.1 and 4.0: * Input files are preprocessed before compilation. *Note Preprocessor::, for more information. * There is a way to discern between a not-supplied optional parameter, and a supplied one, having null value (*note defined::). * The version 4.1 implements 'sprintf' function (*note String formatting::) and 'printf' macro (*note printf: Preprocessor.). * Support for some obsolete features is withdrawn. These include: 1. Using '&CODE' to specify exception codes 2. Pragma options: 'retry', 'io-retry', and 'connect-retry'.  File: mailfromd.info, Node: 31x-400, Next: 30x-31x, Prev: 400-410, Up: Upgrading D.13 Upgrading from 3.1.x to 4.0 ================================ Before building this version, please re-read the chapter *Note Building::, especially the section *note Using non-blocking syslog: syslog-async. Starting from the version 4.0, MFL no longer uses the predefined symbolic names for exception codes (previous versions used the '&' prefix to dereference them). Instead, it relies on constants defined in the include file 'status.mfh' (*note status.mf::). However, the script files from 3.1 series will still work, but the following warning messages will be displayed: Warning: obsolete constant form used: &failure Warning: remove leading '&' and include Warning: Using built-in exception codes is deprecated Warning: Please include Another important difference is that pragmatic options 'ehlo' and 'mailfromd' are now deprecated, as well as their command line equivalents '--ehlo' and '--domain'. These options became superfluous after the introduction of 'mailfrom_address' and 'ehlo_domain' built-in variables. For compatibility with the previous versions, they are still supported by 'mailfromd' 4.0, but a warning message is issued if they are used: warning: `#pragma option ehlo' is deprecated, consider using `set ehlo_domain "domain.name"' instead To update your startup scripts for the new version follow these steps: 1. Change '#pragma option mailfrom VALUE' to 'set mailfrom_address VALUE'. Refer to *note mailfrom_address::, for a detailed discussion of this variable. 2. Change '#pragma option ehlo VALUE' to 'set ehlo_domain VALUE'. Refer to *note ehlo_domain::, for a detailed discussion of this variable. 3. Include 'status.mfh'. Add the following line to the top of your startup file: #include_once 4. Remove all instances of '&' in front of the constants. You can use the following 'sed' expression: 's/&\([a-z]\)/\1/g'. 5. If your code uses any of the following functions: 'hostname', 'resolve', 'hasmx' or 'ismx', add the following line to the top of your script: #require dns *Note Modules::, for a detailed description of the module system. 6. Replace all occurrences of 'next' with 'pass'. 7. If your code uses function 'match_cidr', add the following line to the top of your script: #require match_cidr *Note Modules::, for a description of MFL module system.  File: mailfromd.info, Node: 30x-31x, Next: 2x-30x, Prev: 31x-400, Up: Upgrading D.14 Upgrading from 3.0.x to 3.1 ================================ 1. The 'mailfromd' binary no longer supports '--config-file' ('-c') option. To use an alternative script file, give it as an argument, i.e. instead of: $ mailfromd --config-file FILE.RC write: $ mailfromd FILE.RC For backward compatibility, the old style invocation still works but produces a warning message. However, if 'mailfromd' encounters the '-c' option it will print a diagnostic message and exit immediately. This is because the semantics of this option will change in the future releases. 2. If a variable is declared implicitly within a function, it is created as automatic. This differs from the previous versions, where all variables were global. It is a common practice to use global variables to pass additional information between handlers (*Note HELO Domain::, for an example of this approach). If your filter uses it, make sure the variable is declared as global. For example, this code: prog helo do # Save the host name for further use set helohost $s done Figure D.1: Implicit declaration, old style has to be rewritten as follows: set helohost "" prog helo do # Save the host name for further use set helohost $s done Figure D.2: Implicit declaration, new style 3. Starting from version 3.1 the function 'dbmap' takes an optional third argument indicating whether or not to count the terminating null character in key (*note dbmap::). If your startup script contained any calls to 'dbmap', change them as follows: in 3.0.x in 3.1 -------------------------------------------------------------------------- dbmap(DB, KEY) dbmap(DB, KEY, 1)  File: mailfromd.info, Node: 2x-30x, Next: 1x-2x, Prev: 30x-31x, Up: Upgrading D.15 Upgrading from 2.x to 3.0.x ================================ Update your startup scripts and/or crontab entries. The 'mailfromd' binary is now installed in '${prefix}/sbin'. We also encourage you to update the startup script (run 'cp etc/rc.mailfromd /wherever-your-startup-lives'), since the new version contains lots of enhancements.  File: mailfromd.info, Node: 1x-2x, Prev: 2x-30x, Up: Upgrading D.16 Upgrading from 1.x to 2.x ============================== If you are upgrading from version 1.x to 2.0, you will have to do the following: 1. Edit your script file and enclose the entire code section into: prog envfrom do ... done *Note Handlers::, for more information about the 'prog' statement. 2. If your code contained any 'rate' statements, convert them to function calls (*note rate: Rate limiting functions.), using the following scheme: Old statement: if rate KEY LIMIT / EXPR New statement: if rate(KEY, interval("EXPR")) > LIMIT For example, rate $f 180 / 1 hour 25 minutes should become rate($f, interval("1 hour 25 minutes")) > 180 3. Rebuild your databases using the following command: mailfromd --compact --all This is necessary since the format of 'mailfromd' databases has changed in version 2.0: the key field now includes the trailing 'NUL' character, which is also reflected in its length. This allows for empty (zero-length) keys. *Note Database Maintenance::, for more information about the database compaction.  File: mailfromd.info, Node: Copying This Manual, Next: Concept Index, Prev: Upgrading, Up: Top Appendix E GNU Free Documentation License ***************************************** Version 1.2, November 2002 Copyright (C) 2000,2001,2002 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. 0. PREAMBLE The purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others. This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software. We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference. 1. APPLICABILITY AND DEFINITIONS This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law. A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language. A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them. The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none. The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words. A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque". Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only. The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text. A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements", "Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" according to this definition. The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License. 2. VERBATIM COPYING You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3. You may also lend copies, under the same conditions stated above, and you may publicly display copies. 3. COPYING IN QUANTITY If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects. If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages. If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public. It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document. 4. MODIFICATIONS You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version: A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission. B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement. C. State on the Title page the name of the publisher of the Modified Version, as the publisher. D. Preserve all the copyright notices of the Document. E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices. F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below. G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice. H. Include an unaltered copy of this License. I. Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence. J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission. K. For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein. L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles. M. Delete any section Entitled "Endorsements". Such a section may not be included in the Modified Version. N. Do not retitle any existing section to be Entitled "Endorsements" or to conflict in title with any Invariant Section. O. Preserve any Warranty Disclaimers. If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles. You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard. You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one. The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version. 5. COMBINING DOCUMENTS You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers. The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work. In the combination, you must combine any sections Entitled "History" in the various original documents, forming one section Entitled "History"; likewise combine any sections Entitled "Acknowledgements", and any sections Entitled "Dedications". You must delete all sections Entitled "Endorsements." 6. COLLECTIONS OF DOCUMENTS You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects. You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document. 7. AGGREGATION WITH INDEPENDENT WORKS A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an "aggregate" if the copyright resulting from the compilation is not used to limit the legal rights of the compilation's users beyond what the individual works permit. When the Document is included an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document. If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document's Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate. 8. TRANSLATION Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail. If a section in the Document is Entitled "Acknowledgements", "Dedications", or "History", the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title. 9. TERMINATION You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 10. FUTURE REVISIONS OF THIS LICENSE The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See . Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. E.1 ADDENDUM: How to use this License for your documents ======================================================== To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page: Copyright (C) YEAR YOUR NAME. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License''. If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the "with...Texts." line with this: with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation. If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.