v0.7.0

CHANGELOG.md

@@ -2,6 +2,28 @@
 
 All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/).
 
+## [0.7.0] - 2024-04-09
+
+This version uses a different database layout and introduces multiple breaking changes in the configuration files. Please read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.
+
+## Added
+- Web-based administration interface.
+- REST API for management and configuration.
+- Automatic RSA and ED25519 DKIM key generation.
+- Support for compressing binaries in the blob store (#227).
+- Improved performance accessing IMAP mailboxes with a large number of messages.
+- Support for custom DNS resolvers.
+- Support for multiple loggers with different levels and outputs.
+
+### Changed
+
+### Fixed
+- Store quotas as `u64` rather than `u32`.
+- Second IDLE connections disconnects the first one (#280).
+- Use relaxed DNS parsing, allowing underscores in DNS labels (#172).
+- Escape regexes within `matches()` expressions (#155).
+- ManageSieve LOGOUT should reply with `OK` instead of `BYE`.
+
 ## [0.6.0] - 2024-02-14
 
 This version introduces breaking changes in the configuration file. Please read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.

CONTRIBUTING.md

@@ -1,26 +1,35 @@
 # Contributing
 
-Thank you for your interest in contributing to the Stalwart Mail Server project! We appreciate your support and enthusiasm. This document provides guidelines for contributing to the project.
+When contributing to this repository, please first discuss the change you wish to make via issue,
+email, or any other method with the owners of this repository before making a change. 
 
-## Not accepting Pull Requests at this time
-As of now, the Stalwart Mail Server project is in a phase of rapid development and evolution. Until we reach version 1.0, **we are not open to direct code contributions** in the form of pull requests. This is due to the following reasons:
-- The project is undergoing significant changes which might make integrating external contributions challenging.
-- Our current resources do not allow for the efficient evaluation and testing of submitted pull requests.
+Please note we have a code of conduct, please follow it in all your interactions with the project.
 
-We understand that this might be disappointing, but rest assured, this measure is temporary and is aimed at maintaining the integrity and consistency of the project during its critical development phase.
+## Any contributions you make will be under AGPL
 
-## Other Ways to Contribute
-While we're not accepting pull requests at this moment, there are several other valuable ways you can contribute to the Stalwart Mail Server project:
+This software is licensed under the Affero General Public License (AGPL). Any contributions made to 
+this project will be under this license. Before any contributions can be made, contributors are 
+required to sign a Contributor License Agreement (CLA). The purpose of the agreement is to clarify 
+and document the rights granted by contributors to us.
 
-- **Enhancement Requests**: If you have an idea for a new feature or an improvement, we encourage you to file an enhancement request on GitHub. This way, your idea can be evaluated and potentially included in future versions of the project. To do this, create a new issue on our GitHub repository and label it as an 'enhancement'.
-- **Extensive Testing**: One of the most helpful contributions is to rigorously test the software in various environments and use-cases. Your testing can uncover important issues and help improve the stability of the project.
-- **Writing Unit Tests**: Contributing new unit tests or improving existing ones is a great way to ensure the reliability and robustness of our codebase.
-- **Reviewing Documentation**: As the project evolves, so does the need for accurate and up-to-date documentation. Reviewing, updating, or writing new documentation is a critical contribution that helps new and existing users understand and effectively use Stalwart Mail Server.
+## Pull Request Process
 
-## Stay Updated
-We anticipate opening up for code contributions once version 1.0 is launched. We recommend keeping an eye on our project announcements for any updates regarding contribution opportunities.
+1. Ensure any install or build dependencies are removed before the end of the layer when doing a 
+   build.
+2. Update the README.md with details of changes to the interface, this includes new environment 
+   variables, exposed ports, useful file locations and container parameters.
+3. Increase the version numbers in any examples files and the README.md to the new version that this
+   Pull Request would represent. The versioning scheme we use is [SemVer](http://semver.org/).
+4. You may merge the Pull Request in once you have the sign-off of two other developers, or if you 
+   do not have permission to do that, you may request the second reviewer to merge it for you.
 
-## Questions or Suggestions?
-If you have any questions or suggestions regarding the contribution process, feel free to open a discussion on our GitHub repository.
+## Code of Conduct
 
-Thank you for your support and understanding. Together, we are building a robust and reliable Stalwart Mail Server!
+We as members, contributors, and leaders pledge to make participation in our community a harassment-free 
+experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex 
+characteristics, gender identity and expression, level of experience, education, socio-economic status, 
+nationality, personal appearance, race, religion, or sexual identity and orientation.
+We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, 
+and healthy community.
+
+You can read the full Code of Conduct [here](https://github.com/stalwartlabs/.github/blob/main/CODE_OF_CONDUCT.md).

README.md

@@ -75,6 +75,13 @@ Key features:
   - Sieve scripting language with support for all [registered extensions](https://www.iana.org/assignments/sieve-extensions/sieve-extensions.xhtml).
   - Email aliases, mailing lists, subaddressing and catch-all addresses support.
   - Integration with **OpenTelemetry** to enable monitoring, tracing, and performance analysis.
+- **Web-based administration**:
+  - Account, domain, group and mailing list management.
+  - SMTP queue management for messages and outbound DMARC and TLS reports.
+  - Report visualization interface for received DMARC, TLS-RPT and Failure (ARF) reports.
+  - Configuration of every aspect of the mail server.
+  - Log viewer with search and filtering capabilities.
+  - Self-service portal for password reset and encryption-at-rest key management.
 - **Secure and robust**:
   - Encryption at rest with **S/MIME** or **OpenPGP**.
   - Automatic TLS certificate provisioning with [ACME](https://datatracker.ietf.org/doc/html/rfc8555).
@@ -106,7 +113,7 @@ Additionally you may become a sponsor to obtain priority support from Stalwart L
 
 - [x] Performance enhancements
 - [x] Distributed SMTP queues
-- [ ] Web-based admin panel
+- [x] Web-based admin panel
 - [ ] JMAP Calendar, Contacts and Tasks support
 - [ ] CalDAV and CardDAV support
 

SECURITY.md

@@ -6,9 +6,9 @@ We provide security updates for the following versions of Stalwart Mail Server:
 
 | Version | Supported          |
 | ------- | ------------------ |
-| 0.5.x   | :white_check_mark: |
-| 0.4.x   | :white_check_mark: |
-| < 0.3   | :x:                |
+| 0.7.x   | :white_check_mark: |
+| 0.6.x   | :white_check_mark: |
+| < 0.5   | :x:                |
 
 ## Reporting a Vulnerability
 

UPGRADING.md

@@ -1,3 +1,38 @@
+Upgrading from `v0.6.0` to `v0.7.0`
+-----------------------------------
+
+Version `0.7.0` of Stalwart Mail Server introduces significant improvements and features that enhance performance and functionality. However, it also comes with multiple breaking changes in the configuration files and a revamped database layout optimized for accessing large mailboxes. Additionally, Stalwart now supports compression for binaries stored in the blob store, further increasing efficiency.
+Due to these extensive changes, the recommended approach for upgrading is to perform a clean reinstallation of Stalwart and manually migrate your accounts to the new version.
+
+## Pre-Upgrade Steps
+- Download the `v0.7.0` mail-server and CLI binaries for your platform from the [releases page](https://github.com/stalwartlabs/mail-server/releases/latest/).
+- Initialize the setup on a distinct directory using the command `sudo ./stalwart-mail -- init /path/to/new-install`. This command will print the administrator password required to access the web-admin.
+- Create the `bin` directory using `mkdir /path/to/new-install/bin`.
+- Move the downloaded binaries to the `bin` directory using the command `mv stalwart-mail stalwart-cli /path/to/new-install/bin`.
+- Open `/path/to/new-install/etc/config.toml` in a text editor and comment out all listeners except the HTTP listener for port `8080`.
+- Start the new installation from the terminal using the command `sudo /path/to/new-install/bin/stalwart-mail --config /path/to/new-install/etc/config.toml`.
+- Point your browser to the web-admin at `http://yourserver.org:8080` and login using the auto-generated administrator password. 
+- Configure the new installation with your domain, hostname, certificates, and other settings following the instructions at [stalw.art/docs/get-started](https://stalw.art/docs/get-started). Ignore the part about using the installation script, we are performing a manual installation.
+- Add your user accounts.
+- Configure Stalwart to run as the `stalwart-mail` user and `stalwart-mail` group from `Settings` > `Server` > `System`. This is not necessary if you are using Docker.
+- Stop the new installation by pressing `Ctrl+C` in the terminal.
+
+## Upgrade Steps
+- On your `v0.6.0` installation, open in a text editor the `smtp/listener.toml`, `imap/listener.toml` files and comment out all listeners except the JMAP/HTTP listener (we are going to need it to export the user accounts) and then restart the service.
+- If you are using an external store, backup the database using the appropriate method for your database system.
+- Create the `~/exports` directory, here we will store the exported accounts.
+- Using the existing CLI tool (not the one you just downloaded as it is not compatible), export each user account using the command `./stalwart-cli -u https://your-old-server.org -c <ADMIN_PASSWORD> export account <ACCOUNT_NAME> ~/exports`.
+- Stop the `v0.6.0` installation using the command `sudo systemctl stop stalwart-mail`.
+- Move the old `v0.6.0` installation to a backup directory, for example `mv /opt/stalwart-mail /opt/stalwart-mail-backup`.
+- Move the new `v0.7.0` installation to the old installation directory, for example `mv /path/to/new-install /opt/stalwart-mail`.
+- Set the right permissions for the new installation using the command `sudo chown -R stalwart:stalwart /opt/stalwart-mail`.
+- Start the new installation using the command `sudo systemctl start stalwart-mail`.
+- Import the accounts using the new CLI tool with the command `./stalwart-cli -u http://yourserver.org:8080-c <ADMIN_PASSWORD> import <ACCOUNT> ~/exports/<ACCOUNT>`.
+- Using the admin tool, reactivate all the necessary listener (SMTP, IMAP, etc.)
+- Restart the service using the command `sudo systemctl restart stalwart-mail`.
+
+We apologize for the complexity of the upgrade process associated with this version of Stalwart. We understand the challenges and inconveniences that the requirement for a clean reinstallation and manual account migration poses. Moving forward, an automated migration tool will be included in any future releases that necessitate changes to the database layout, aiming to streamline the upgrade process for you. Furthermore, as we approach the milestone of version 1.0.0, we anticipate that such foundational changes will become increasingly infrequent, leading to more straightforward updates. We appreciate your patience and commitment to Stalwart during this upgrade.
+
 Upgrading from `v0.5.3` to `v0.6.0`
 -----------------------------------
 

install.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env sh
 # shellcheck shell=dash
 
-# Stalwart Mail install script -- based on the rustup installation script.
+# Stalwart Mail Server install script -- based on the rustup installation script.
 
 set -e
 set -u
@@ -34,26 +34,43 @@ main() {
         _account="_stalwart-mail"
     fi
 
-    # Start configuration mode
-    if [ "$#" -eq 1 ] && [ "$1" = "--init" ]  ; then
-        init
-        configure
-        return 0
-    fi
+    # Read arguments
+    local _dir="/opt/stalwart-mail"
+
+    # Default component setting
+    local _component="stalwart-mail"
+
+    # Loop through the arguments
+    for arg in "$@"; do
+        case "$arg" in
+            --fdb)
+                _component="stalwart-mail-foundationdb"
+                ;;
+            *)
+                if [ -n "$arg" ]; then
+                    _dir=$arg
+                fi
+                ;;
+        esac
+    done
 
     # Detect platform architecture
     get_architecture || return 1
     local _arch="$RETVAL"
     assert_nz "$_arch" "arch"
 
+    # Create directories
+    ensure mkdir -p "$_dir" "$_dir/bin" "$_dir/etc" "$_dir/logs"
+
     # Download latest binary
-    say "⏳ Downloading installer for ${_arch}..."
-    local _dir
-    _dir="$(ensure mktemp -d)"
-    local _file="${_dir}/stalwart-install.tar.gz"
-    local _url="${BASE_URL}/stalwart-install-${_arch}.tar.gz"
+    say "⏳ Downloading ${_component} for ${_arch}..."
+    local _file="${_dir}/bin/stalwart-mail.tar.gz"
+    local _url="${BASE_URL}/${_component}-${_arch}.tar.gz"
     ensure mkdir -p "$_dir"
     ensure downloader "$_url" "$_file" "$_arch"
+    ensure tar zxvf "$_file" -C "$_dir/bin"
+    ignore chmod +x "$_dir/bin/stalwart-mail"
+    ignore rm "$_file"
 
     # Create system account
     if ! id -u ${_account} > /dev/null 2>&1; then
@@ -86,16 +103,92 @@ main() {
         fi
     fi
 
-    # Copy binary
-    say "⬇️  Running installer..."
-    ensure tar zxvf "$_file" -C "$_dir"
-    ignore $_dir/stalwart-install
-    ignore rm "$_file"
-    ignore rm "$_dir/stalwart-install"
+    # Run init
+    ignore $_dir/bin/stalwart-mail --init "$_dir"
+
+    # Set permissions
+    say "🔐 Setting permissions..."
+    ensure chown -R ${_account}:${_account} "$_dir"
+    ensure chmod -R 755 "$_dir"
+    ensure chmod 700 "$_dir/etc/config.toml"
+
+    # Create service file
+    say "🚀 Starting service..."
+    if [ "${_os}" = "linux" ]; then
+        printf "\n[server.run-as]\nuser = \"stalwart-mail\"\ngroup = \"stalwart-mail\"\n" >> "$_dir/etc/config.toml"
+        create_service_linux "$_dir"
+    elif [ "${_os}" = "macos" ]; then
+        create_service_macos "$_dir"
+    fi
+
+    # Installation complete
+    local _host=$(hostname)
+    say "🎉 Installation complete! Continue the setup at http://$_host:8080/login"
 
     return 0
 }
 
+# Functions to create service files
+create_service_linux() {
+    local _dir="$1"
+    cat <<EOF | sed "s|__PATH__|$_dir|g" > /etc/systemd/system/stalwart-mail.service
+[Unit]
+Description=Stalwart Mail Server Server
+Conflicts=postfix.service sendmail.service exim4.service
+ConditionPathExists=__PATH__/etc/config.toml
+After=network-online.target
+ 
+[Service]
+Type=simple
+LimitNOFILE=65536
+KillMode=process
+KillSignal=SIGINT
+Restart=on-failure
+RestartSec=5
+ExecStart=__PATH__/bin/stalwart-mail --config=__PATH__/etc/config.toml
+PermissionsStartOnly=true
+StandardOutput=syslog
+StandardError=syslog
+SyslogIdentifier=stalwart-mail
+ 
+[Install]
+WantedBy=multi-user.target
+EOF
+    systemctl daemon-reload
+    systemctl enable stalwart-mail.service
+    systemctl restart stalwart-mail.service
+}
+
+create_service_macos() {
+    local _dir="$1"
+    cat <<EOF | sed "s|__PATH__|$_dir|g" > /Library/LaunchAgents/stalwart.mail.plist
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
+    "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+    <dict>
+        <key>Label</key>
+        <string>stalwart.mail</string>
+        <key>ServiceDescription</key>
+        <string>Stalwart Mail Server</string>
+        <key>ProgramArguments</key>
+        <array>
+            <string>__PATH__/bin/stalwart-mail</string>
+            <string>--config=__PATH__/etc/config.toml</string>
+        </array>
+        <key>RunAtLoad</key>
+        <true/>
+        <key>KeepAlive</key>
+        <true/>
+    </dict>
+</plist>
+EOF
+    launchctl load /Library/LaunchAgents/stalwart.mail.plist
+    launchctl enable system/stalwart.mail
+    launchctl start system/stalwart.mail
+}
+
+
 get_architecture() {
     local _ostype _cputype _bitness _arch _clibtype
     _ostype="$(uname -s)"
@@ -369,7 +462,7 @@ get_endianness() {
 }
 
 say() {
-    printf 'stalwart-mail: %s\n' "$1"
+    printf '%s\n' "$1"
 }
 
 err() {