The following sections describe procedures for upgrading between the consecutive Mailfromd releases. The absense of a section for a pair of versions X-Y numbers means that no specific actions are required for upgrading from X to Y.

Upgrading from 8.7 to 8.8
=========================

DKIM support (see DKIM) introduced in this version requires the Nettle cryptographic library (http://www.gnu.org/software/nettle).  It you need DKIM, make sure Nettle is installed prior to compiling mailfromd.  Otherwise, no special actions are required.

Upgrading from 8.5 to 8.6
=========================

New configure option --with-dbm allows you to select any DBM flavor supported by GNU mailutils as the default DBM implementation for mailfromd.

Upgrading from 8.2 to 8.3 (or 8.4)
==================================

Versions 8.3 and 8.4 differ only in required minimal version of mailutils (3.3 and 3.4, correspondingly).  Apart from that, the following instructions apply to both versions.

In version 8.3 I abandoned the legacy DNS resolver and switched to GNU adns.  GNU ands is a standalone resolver library, which provides a number of advanced features.  It is included in most distributions.  The source code of the recent release is available from http://www.chiark.greenend.org.uk/~ian/adns/adns.tar.gz.

This change brought a number of user-visible changes.  In particular, arbitrary limits on the sizes of the RRs are removed.  Consequently, the following configurations statements are withdrawn:

runtime.max-dns-reply-a

runtime.max-dns-reply-ptr

runtime.max-dns-reply-mx

max-match-mx

max-callout-mx

Secondly, the new resolver is less tolerant to deviations from the standard.  This means that remote DNS misconfigurations that would have slipped unnoticed in previous versions, will be noticed by mailfromd 8.3.  For example, a CNAME record pointing to another CNAME is treated as an error.

A new command line option was added to mailfromd and calloutd: --resolv-conf-file.  This option instructs the programs to read resolver settings from the supplied file, instead of the default /etc/resolv.conf.

Another user-visible change is in handling of SPF checks.  Previously, results if SPF checks were cached in a database.  This proved to cause more problems than solutions and was removed in this version.  As a result, the following MFL global variables have been withdrawn:

spf_ttl

spf_cached

spf_database

spf_negative_ttl

Upgrading from 7.0 to 8.0
=========================

Version 8.0 is a major rewrite, that introduces a lot of new concepts and features.  Nevertheless, it is still able to run the MFL scripts from version 7.0 without modifications.

Note the following important points:

  * The listen configuration statement withdrawn

     Use the server milter statement instead.  See conf-server.

  * The --remove option withdrawn

     This option was a noop since version 7.0.91.

  * The use of % before variable names is no longer supported

     The % characters is used as modulo operator.  See Arithmetic operations.

  * The debug_spec built-in function changed signature.

     See debug_spec.

  * listens and portprobe

     The listens function was moved to the portprobe module.  It is actually an alias to the portprobe function.  If your filter uses listens, make sure to require the portprobe module.

     See portprobe.

  * _pollhost, _pollmx, stdpoll, strictpoll

     These functions have been moved to the poll module, which must be required prior to using any of them.

  * The message_header_count function.

     This function takes an optional string argument, supplying the header name.  See message_header_count.

Upgrading from 6.0 to 7.0
=========================

The release 7.0 removes the features which were declared as obsolete in 6.0 and introduces important new features, both syntactical, at the MFL level, and operational.

Unless your filter used any deprecated features, it should work correctly after upgrade to this version.  It will, however, issue warning messages regarding the deprecated features (e.g. the use of % in front of identifiers, as described below).  To fix these, follow the upgrade procedure described in upgrade procedure.

The removed features are:

  * Old-style functional notation

  * The use of functional operators

  * Implicit concatenations

  * #pragma option

  * #pragma database

The MFL syntax has changed: it is no longer necessary to use % in front of a variable to get its value.  To reference a variable, simply use its name, e.g.:

     set x var + z

The old syntax is still supported, so the following statement will also work:

     set x %var + %z

It will, however, generate a warning message.

Of course, the use of % to reference variables within a string literal remains mandatory.

Another important changes to MFL are user-defined exceptions (see User-defined Exceptions) and the try--catch construct (see try--catch).

Several existing MFL functions have been improved.  In particular, it is worth noticing that the open function, when opening a pipe to or from a command, provides a way to control where the command's standard error would go (see standard error).

The accept function (or action) issues a warning if its use would cancel any modifications to the message applied by, e.g., header_add and similar functions.  See Message modification queue, for a detailed discussion of this feature.

The most important change in mailfromd operation is that the version 7.0 is able to run several servers (see conf-server). Dedicated callout servers make it possible to run sender verifications in background, using a set of long timeouts, as prescribed by RFC 2822 (see SMTP Timeouts).  This diminishes the number of false positives, allows for coping with servers showing large delays and also reduces the number of callouts performed for such servers.

This release no longer includes the smap utility.  It was moved into a self-standing project, which by now provides much more functionality and is way more flexible than this utility was.  If you are interested in using smap, visit http://www.gnu.org.ua/software/smap, for a detailed information, including pointers to file downloads.

Upgrading from 5.x to 6.0
=========================

The 6.0 release is aimed to fix several logical inconsistencies that affected the previous versions.  The most important one is that until version 5.2, the filter script file contained both the actual filter script, and the run-time configuration for mailfromd (in form of #pragma option and #pragma database statements). The new version separates run-time configuration from the filter script by introducing a special configuration file mailfromd.conf (see Mailfromd Configuration).

Consequently, the #pragma option and #pragma database statements become deprecated.  Furthermore, the following deprecated pragmas are removed: #pragma option ehlo, #pragma option mailfrom. These pragmas became deprecated in version 4.0 (see 31x-400).

The second problem was that the default filter script file had .rc suffix, which usually marks a configuration file, not the source.  In version 6.0 the script file is renamed to mailfromd.mf.  In the absence of this file, the legacy file mailfromd.rc is recognized and parsed.  This ensures backward compatibility.

This release also fixes various inconsistencies and dubious features in the MFL language.

The support for unquoted literals is discontinued.  This feature was marked as deprecated in version 3.0.

The following features are deprecated: #pragma option (6,pragma_002doption,pragma-option and #pragma database (6,pragma_002ddatabase,pragma-database) directives, the legacy style of function declarations (6,old_002dstyle-function-declarations, old-style function declarations), calls to functions of one argument without parentheses (6,implicit-concatenation, operational notation), the #require statement (see import, for the new syntax) and implicit concatenation (6,implicit-concatenation, implicit concatenation).  See 6,Deprecated-Features,Deprecated Features, for more information about these.

This release also introduces important new features, which are summarized in the table below:

Feature              Reference
Configuration            see Mailfromd Configuration.
Module system            see Modules.
Explicit type casts      see explicit type casts.
Concatenation operator   see Concatenation.
Scope of visibility      see scope of visibility.
Precious variables       see rset.

Mailfromd version 6.0 will work with unchanged scripts from 5.x.  When started, it will verbosely warn you about any deprecated constructs that are used in your filter sources and will create a script for upgrading them.

To upgrade your filter scripts, follow the steps below:

 1. Run mailfromd --lint.  You will see a list of warnings similar to this:

          mailfromd: Warning: using legacy script file
           /usr/local/etc/mailfromd.rc
          mailfromd: Warning: rename it to /usr/local/etc/mailfromd.mf
           or use script-file statement in /usr/local/etc/mailfromd.conf
           to disable this warning
          mailfromd: /usr/local/etc/mailfromd.rc:19: warning: this pragma is
           deprecated: use relayed-domain-file configuration statement instead
          mailfromd: /usr/local/etc/mailfromd.rc:23: warning: this pragma is
           deprecated: use io-timeout configuration statement instead
          mailfromd: Info: run script /tmp/mailfromd-newconf.sh
           to fix the above warnings
          ...

 2. At the end of the run mailfromd will create a shell script /tmp/mailfromd-newconf.sh for fixing these warnings.  Run it:

          $ sh /tmp/mailfromd-newconf.sh

 3. When the script finishes, run mailfromd --lint again.  If it shows no more deprecation warnings, the conversion went correctly. Now you can remove the upgrade script:

          $ rm /tmp/mailfromd-newconf.sh

Notice, that the conversion script attempts to fix only deprecation warnings.  It will not try to correct any other type of warnings or errors.  For example, you may get warning messages similar to:

     mailfromd: /etc/mailfromd.mf:7: warning: including a module file is unreliable and may cause subtle errors
     mailfromd: /etc/mailfromd.mf:7: warning: use `require dns' instead

This means that you use #include where you should have used require.  You will have to fix such warnings manually, as suggested in the warning message.

If, for some reason, you cannot upgrade your scripts right now, you may suppress deprecation warnings by setting the environment variable MAILFROMD_DEPRECATION to no before starting mailfromd.  Nonetheless, I recommend to upgrade as soon as possible, because the deprecated features will be removed in version 6.1.

Upgrading from 5.0 to 5.1
=========================

Upgrading from 5.0 to 5.1 does not require any changes in your filter scripts.  Notice, however, the following important points:

  * Starting from this release mailfromd supports Milter protocol version 6, which is compatible with Sendmail 8.14.0 and newer.  While being backward compatible with earlier Sendmail releases, it allows you to use the new prog data handler (see data).  It also supports macro negotiation, a feature that enables Mailfromd to ask MTA to export the macros it needs for each particular handler.  This means that if you are using Sendmail 8.14.0 or higher (or Postfix 2.5 or higher), you no longer need to worry about exporting macro names in sendmail.cf file.

     The same feature is also implemented on the server side, in mtasim and pmult.  Consequently, using define-macros in pmult configuration file is not strictly necessary.  However, keep in mind that due to the specifics of MeTA1, the number of symbols that may be exported for each stage is limited (see pmult-macros).

  * The semantics of __preproc__ and __statedir__ built-in constant is slightly different from what it used to be in 5.0.  These constants now refer to the current values of the preprocessor command line and program state directory, correspondingly.  This should not affect your script, unless you redefine the default values on run time.  If your script needs to access default values, use __defpreproc__ and __defstatedir__, correspondingly (see Built-in constants). The following example explains the difference between these:

          $ cat pval.mf
          prog envfrom
          do
            echo "Default value: " __defstatedir__
            echo "Current value: " __statedir__
          done
          $ mailfromd --state-directory=/var/mfd --test pval.mf
          Default value: /usr/local/var/mailfromd
          Current value: /var/mfd

  * If your filter uses the rate function, you might consider using the new function rateok or tbf_rate instead.  For a detailed discussion of these functions, see Sending Rate.

  * If your script extensively uses database access functions, you might be interested in the new #pragma dbprop (see dbprop).

Upgrading from 4.4 to 5.0
=========================

This version of Mailfromd requires GNU mailutils (http://www.gnu.org/software/mailutils) version 2.0 or later.

Upgrading from version 4.4 to 5.0 requires no additional changes.  The major differences between these two versions are summarized below:

 1. Support for MeTA1.

 2. New Mailutils configuration file.

 3. New MFL functions.

      a. Message functions.  See Message functions.

      b. Mailbox functions.  See Mailbox functions.

      c. Mail body functions.  See Mail body functions.

      d. Header modification functions.  See Header modification functions.

      e. Envelope modification functions.  See Envelope modification functions.

      f. Quarantine functions.  See Quarantine functions.

      g. getopt and varptr.  See getopt.

      h. Macro access functions.  See Macro access.

      i. Character type functions.  See Character Type.

      j. New string functions (see String manipulation): verp_extract_user, sa_format_report_header, sa_format_score.

      k. Sequential access to DBM files.  See dbm-seq.

 4. Changes in MFL

      1. See variadic functions.

      2. See function alias.

 5. New operation mode: see Run Mode.

 6. Improved stack growth technique.

     The stack can be grown either by fixed size blocks, or exponentially.  Upper limit can be specified.  See stacksize.

 7. Milter ports can be specified using URL notation.

 8. Removed deprecated features.

     Support for some deprecated features has been withdrawn.  These are:

      a. Command line options --ehlo, --postmaster-email, and --mailfrom. These became deprecated in version 4.0.  See 31x-400.

Upgrading from 4.3.x to 4.4
===========================

The deprecated --domain command line option has been withdrawn.  The short option -D now defines a preprocessor symbol (see Preprocessor Options).

This version correctly handles name clashes between constants and variables, which remained unnoticed in previous releases. See variable--constant shadowing, for a detailed description of it.

To minimize chances of name clashes, all symbolic exception codes has been renamed by prefixing them with the e_, thus, e.g. divzero became e_divzero, etc.  The ioerr exception code is renamed to e_io.  See status.mf, for a full list of the new exception codes.

For consistency, the following most often used codes are available without the e_ prefix: success, not_found, failure, temp_failure.  This makes most existing user scripts suitable for use with version 4.4 without any modification.  If your script refers to any exception codes other than these four, you can still use it by defining a preprocessor symbol OLD_EXCEPTION_CODES, for example:

     $ mailfromd -DOLD_EXCEPTION_CODES

Upgrading from 4.2 to 4.3.x
===========================

Upgrading from 4.2 to 4.3 or 4.3.1 does not require any changes to your configuration and scripts.  The only notable change in these versions is the following:

The asynchronous syslog implementation was reported to malfunction on some systems (notably on Solaris), so this release does not enable it by default.  The previous meaning of the --enable-syslog-async configuration option is also restored.  Use this option in order to enable asynchronous syslog feature.  To set default syslog implementation, use DEFAULT_SYSLOG_ASYNC configuration variable (see syslog-async).

The following deprecated features are removed:

 1. #pragma option ehlo statement.

     It became deprecated in version 4.0.  See pragma-option-ehlo.

 2. #pragma option mailfrom statement.

     It became deprecated in version 4.0.  See pragma-option-ehlo.

 3. The --config-file command line option.

     It became deprecated in version 3.1.  See 30x-31x.

 4. Built-in exception codes in catch statements.

     They are deprecated since version 4.0.  See 31x-400.

Upgrading from 4.1 to 4.2
=========================

Upgrading to this version does not require any special efforts.  You can use your configuration files and filter scripts without any changes.  The only difference worth noticing is that starting from this version mailfromd is always compiled with asynchronous syslog implementation.  The --enable-syslog-async configuration file option is still available, but its meaning has changed: it sets the default syslog implementation to use (see syslog-async).  Thus, it can be used the same way it was in previous versions.  You can also select the syslog implementation at run time, see --syslog-async option, for more detailed information.

Upgrading from 4.0 to 4.1
=========================

Upgrading to this version does not require any special efforts.  You can use your configuration files and filter scripts without any changes. Notice only the following major differences between 4.1 and 4.0:

  * Input files are preprocessed before compilation. See Preprocessor, for more information.

  * There is a way to discern between a not-supplied optional parameter, and a supplied one, having null value (see defined).

  * The version 4.1 implements sprintf function (see String formatting) and printf macro (see printf).

  * Support for some obsolete features is withdrawn.  These include:

      1. Using &CODE to specify exception codes

      2. Pragma options: retry, io-retry, and connect-retry.

Upgrading from 3.1.x to 4.0
===========================

Before building this version, please re-read the chapter see Building, especially the section Using non-blocking syslog.

Starting from the version 4.0, MFL no longer uses the predefined symbolic names for exception codes (previous versions used the & prefix to dereference them).  Instead, it relies on constants defined in the include file status.mfh (see status.mf).

However, the script files from 3.1 series will still work, but the following warning messages will be displayed:

     Warning: obsolete constant form used: &failure
     Warning: remove leading '&' and include <status.mfh>

     Warning: Using built-in exception codes is deprecated
     Warning: Please include <status.mfh>

Another important difference is that pragmatic options ehlo and mailfromd are now deprecated, as well as their command line equivalents --ehlo and --domain.  These options became superfluous after the introduction of mailfrom_address and ehlo_domain built-in variables.  For compatibility with the previous versions, they are still supported by mailfromd 4.0, but a warning message is issued if they are used:

     warning: `#pragma option ehlo' is deprecated,
      consider using `set ehlo_domain "domain.name"' instead

To update your startup scripts for the new version follow these steps:

 1. Change #pragma option mailfrom VALUE to set mailfrom_address VALUE.  Refer to mailfrom_address, for a detailed discussion of this variable.

 2. Change #pragma option ehlo VALUE to set ehlo_domain VALUE.  Refer to ehlo_domain, for a detailed discussion of this variable.

 3. Include status.mfh.  Add the following line to the top of your startup file:

          #include_once <status.mfh>

 4. Remove all instances of & in front of the constants.  You can use the following sed expression: s/&\([a-z]\)/\1/g.

 5. If your code uses any of the following functions: hostname, resolve, hasmx or ismx, add the following line to the top of your script:

          #require dns

     See Modules, for a detailed description of the module system.

 6. Replace all occurrences of next with pass.

 7. If your code uses function match_cidr, add the following line to the top of your script:

          #require match_cidr

     See Modules, for a description of MFL module system.

Upgrading from 3.0.x to 3.1
===========================

 1. The mailfromd binary no longer supports --config-file (-c) option.  To use an alternative script file, give it as an argument, i.e. instead of:

          $ mailfromd --config-file FILE.RC

     write:

          $ mailfromd FILE.RC

     For backward compatibility, the old style invocation still works but produces a warning message.  However, if mailfromd encounters the -c option it will print a diagnostic message and exit immediately.  This is because the semantics of this option will change in the future releases.

 2. If a variable is declared implicitly within a function, it is created as automatic.  This differs from the previous versions, where all variables were global.  It is a common practice to use global variables to pass additional information between handlers (see HELO Domain, for an example of this approach).  If your filter uses it, make sure the variable is declared as global.  For example, this code:

     Figure: Implicit declaration, old style

          prog helo
          do
            # Save the host name for further use
            set helohost $s
          done

     has to be rewritten as follows:

     Figure: Implicit declaration, new style

          set helohost ""

          prog helo
          do
            # Save the host name for further use
            set helohost $s
          done

 3. Starting from version 3.1 the function dbmap takes an optional third argument indicating whether or not to count the terminating null character in key (see dbmap).  If your startup script contained any calls to dbmap, change them as follows:

     in 3.0.x   in 3.1
     dbmap(DB, KEY)   dbmap(DB, KEY, 1)

Upgrading from 2.x to 3.0.x
===========================

Update your startup scripts and/or crontab entries.  The mailfromd binary is now installed in ${prefix}/sbin.

We also encourage you to update the startup script (run cp etc/rc.mailfromd /wherever-your-startup-lives), since the new version contains lots of enhancements.

Upgrading from 1.x to 2.x
=========================

If you are upgrading from version 1.x to 2.0, you will have to do the following:

 1. Edit your script file and enclose the entire code section into:

          prog envfrom
          do
            ...
          done

     See Handlers, for more information about the prog statement.

 2. If your code contained any rate statements, convert them to function calls (see rate), using the following scheme:

          Old statement: if rate KEY LIMIT / EXPR
          New statement: if rate(KEY, interval("EXPR")) > LIMIT

     For example,

             rate $f 180 / 1 hour 25 minutes

     should become

             rate($f, interval("1 hour 25 minutes")) > 180

 3. Rebuild your databases using the following command:

          mailfromd --compact --all

     This is necessary since the format of mailfromd databases has changed in version 2.0: the key field now includes the trailing NUL character, which is also reflected in its length.  This allows for empty (zero-length) keys.  See Database Maintenance, for more information about the database compaction.