For some time now, we have run the prosody service on a small Raspberry Pi as a chat service for our domestic requirements. This turns out to be really useful. Helen spends much of the day in her dyeshed, while I spend much of the day in the house, and while we could use the DECT phone system, the signal doesn't quite reach her shed. In addition, if she is occupied with her dyepots, a ringing phone is a distraction. But a jabber (xmpp) client on her desktop is perfect. We can message each other without the possibility of $MEGACORP listening in, tracking metadata, moving private information to their other systems, and maybe ultimately selling our data for nefarious purposes. (Have we already forgotten the source of Cambridge Analytica data?)

The Extensible Messaging and Presence Protocol (XMPP) protocol is really rather interesting. It's of less interest to global winner-takes-all data companies, because it is designed to be interoperable with other such services. So for example, Google's chat client was originally xmpp, but moved away from that protocol, perhaps in order to control their users and their data better.  The xmpp protocol is also designed to be extendable, so as new capabilities develop, these can be added to the base system without impacting on the core protocol.  For example, voice and video chat is possible with such "XEPs", as these extensions are called.

This has worked well for some years, but recently I decided to put a little more effort into our little service.  For example, we want to use the service on our phones as well. In the past, we set up separate users on the systems for the phones, but with the "carbons" extension, it is possible to get multiple devices to sync with each other. So we can start a conversation on the desktop, move to the phone to reply, continue on the desktop etc.

Also when we started, we used the OTR "off-the-record" encryption option. This was a good encryption system, but which had difficulties when it came to anything other than one-to-one communication. So the Omemo project was developed to bring a double-ratchet-based multi-end to multi-end encryption scheme to xmpp. Most clients now support this, one way or another, and a leading Android-based client, Conversations, has dropped OTR for Omemo.  So now we use Omemo on all our clients.

We run a TURN server for using Nextcloud Talk, as discussed elsewhere on this blog. It turns (no pun intended) out that prosody can use this to help in placing voice and video calls between clients.  We have used this from time to time, and it works quite well.

But we then ran into some issues when we wondered whether our system could be extended to family use.  In particular, when trying to set up group chats on the Conversations client, we got the dreaded message "No group chat server could be found". We've now resolved this problem.

As I researched this, quite a few articles mentioned hitting issues with prosody, and recommended ejabberd instead.  I knew prosody is relatively simple to set up, while ejabberd needed more initial work, but quite a few posts I found suggested after it was configured it worked well. So I set aside a day to look at ejabberd as an alternative.  It did indeed help, but only by helping to point the way to resolving prosody's issues.  ejabberd would be very good on a dedicated or high capacity server, I am sure.  It has a built-in ACME client which allows it to request and deploy Letsencrypt certificates, and a built-in STUN and TURN server.  However, try as I might, I was unable to get ejabberd to co-exist in an acceptable way within my existing server, which already had its certificates, and already had a TURN and STUN server.

Eventually I concluded that, for a small system, prosody held advantages over ejabberd, and could be configured more elegantly too.  When running on a very small system such as a Raspberry Pi, prosody uses few resources, while ejabberd is, as befits a much more scalable system, rather more resource hungry.

Regarding Letsencrypt, we use the certbot utility to deploy certificates. It turns out that there is a hook for prosody to allow it to use new certificates. So now, all we need to do is add

--deploy-hook "prosodyctl --root cert import /etc/letsencrypt/live"

to the certbot command for the certificates to be usable by prosody, an option easily added to the renewal scripts.

But I did learn some things from ejabberd, as some things are sort of hard-coded into ejabberd, while they are choices in prosody, so following those standards made life easier.  For example, for multi-user chat (muc), prosody asks for a server/service name. I had set the name of the overall service to the machine name and domain, so I chose chatroom.<domain_name> for the muc service.  The recommendation is to use "conference" as the name for this service, but what I ended up with, based on what ejabberd defaulted to, was a working system with "conference.<machine_name>.<domain_name>".  In addition, ejabberd warned that if one was using a muc service, you would probably also was a "mam_muc" service.  I thought I was using such a module in prosody, but I had the module in the wrong place in the configuration file.

There are many sources of documentation for the basic setting up of a prosody server on the web. I'll not repeat them here.  Our set up is a standard Raspberry Pi OS Buster system, so is Debian, basically.  I did download, as the documentation suggests, the prosody community modules, which I copied, as I needed them, rather than all of them, to a specified location which I included in the config file.  The three I ended up using were mod_group_bookmarks, mod_muc_cloud_notify, and importantly mod_turncredentials. The muc_cloud_inotify option is untested, but seems to be necessary if anyone is running an iOS client, where it seems any notifications have to be approved by Apple.

I did have frequent failures when stepping through additions to the configuration. This was caused by the fact the lua, in which prosody is written, uses two dashes as a comment indicator, and I was merrily using a # instead.

Here's my sanitised, working configuration file, prosody.cfg.lua

--------------------------
--  For this redacted version:
--  I have replaced my domain name and system name with "machine.domain.org"
-------------------------


-- Prosody Example Configuration File
--
-- Information on configuring Prosody can be found on our
-- website at https://prosody.im/doc/configure
--
-- Tip: You can check that the syntax of this file is correct
-- when you have finished by running this command:
--     prosodyctl check config
-- If there are any errors, it will let you know what and where
-- they are, otherwise it will keep quiet.
--
-- The only thing left to do is rename this file to remove the .dist ending, and fill in the
-- blanks. Good luck, and happy Jabbering!


---------- Server-wide settings ----------
-- Settings in this section apply to the whole server and are the default settings
-- for any virtual hosts

-- This is a (by default, empty) list of accounts that are admins
-- for the server. Note that you must create the accounts separately
-- (see https://prosody.im/doc/creating_accounts for info)
-- Example: admins = { "user1@example.com", "user2@example.net" }
admins = { "user1@machine.domain.org", "user2@machine.domain.org" }

-- Enable use of libevent for better performance under high load
-- For more information see: https://prosody.im/doc/libevent
use_libevent = true
--use_libevent = true

-- Prosody will always look in its source directory for modules, but
-- this option allows you to specify additional locations where Prosody
-- will look for modules first. For community modules, see https://modules.prosody.im/
-- For a local administrator it's common to place local modifications
-- under /usr/local/ hierarchy:
plugin_paths = { "/usr/local/lib/prosody/modules" }

-- This is the list of modules Prosody will load on startup.
-- It looks for mod_modulename.lua in the plugins folder, so make sure that exists too.
-- Documentation for bundled modules can be found at: https://prosody.im/doc/modules
modules_enabled = {

    -- Generally required
        "roster"; -- Allow users to have a roster. Recommended ;)
        "saslauth"; -- Authentication for clients and servers. Recommended if you want to log in.
        "tls"; -- Add support for secure TLS on c2s/s2s connections
        "dialback"; -- s2s dialback support
        "disco"; -- Service discovery


    -- Not essential, but recommended
        "carbons"; -- Keep multiple clients in sync
        "pep"; -- Enables users to publish their avatar, mood, activity, playing music and more
        "private"; -- Private XML storage (for room bookmarks, etc.)
        "blocklist"; -- Allow users to block communications with other users
        "vcard4"; -- User profiles (stored in PEP)
        "vcard_legacy"; -- Conversion between legacy vCard and PEP Avatar, vcard

    -- Nice to have
        "version"; -- Replies to server version requests
        "uptime"; -- Report how long server has been running
        "time"; -- Let others know the time here on this server
        "ping"; -- Replies to XMPP pings with pongs
        "register"; -- Allow users to register on this server using a client and change passwords
        "mam"; -- Store messages in an archive and allow users to access it
        "csi_simple"; -- Simple Mobile optimizations

    -- Added by Stevan - see https://jacksonjs.github.io/2016/09/09/prosodyonpi/
        "smacks"; -- stream management - useful for occasional network outage, eg mobile
        "cloud_notify"; --required for chatsecure on ios
        "group_bookmarks"; -- should allow creating groups into clients
        -- Oops - this shouldn't be here - see further down "mam_muc";

    -- Admin interfaces
        "admin_adhoc"; -- Allows administration via an XMPP client that supports ad-hoc commands
        --"admin_telnet"; -- Opens telnet console interface on localhost port 5582

    -- HTTP modules
        --"bosh"; -- Enable BOSH clients, aka "Jabber over HTTP"
        --"websocket"; -- XMPP over WebSockets
        "http_files"; -- Serve static files from a directory over HTTP
        "http_upload"; -- Serve static files from a directory over HTTP

    -- Other specific functionality
        "posix"; -- POSIX functionality, sends server to background, enables syslog, etc.
        --"limits"; -- Enable bandwidth limiting for XMPP connections
        "groups"; -- Shared roster support
        --"server_contact_info"; -- Publish contact information for this service
        --"announce"; -- Send announcement to all online users
        --"welcome"; -- Welcome users who register accounts
        --"watchregistrations"; -- Alert admins of registrations
        --"motd"; -- Send a message to users when they log in
        --"legacyauth"; -- Legacy authentication. Only used by some old clients and bots.
        --"proxy65"; -- Enables a file transfer proxy service which clients behind NAT can use
        "turncredentials";
}

-- These modules are auto-loaded, but should you want
-- to disable them then uncomment them here:
modules_disabled = {
    -- "offline"; -- Store offline messages
    -- "c2s"; -- Handle client connections
    -- "s2s"; -- Handle server-to-server connections
}

--- for ios push notify
    -- For iOS push notification...
-- mod_cloud_notify settings
push_notification_with_body = false;
push_notification_with_sender = true;

-- group bookmarks
group_bookmarks_file = "/etc/prosody/group_bookmarks.txt";


--HTTP STUFF-----
--    http_ports = { 5280 }
--    http_interfaces = { "*" }
 
    https_ports = { 5281 }
    https_interfaces = { "*" }
    http_upload_file_size_limit = 1024 * 1024 * 20 --20MB
     https_ssl = {
            certificate = "/etc/prosody/certs/machine.domain.org.crt";
            key = "/etc/prosody/certs/machine.domain.org.key";
        }
--END HTTP STUFF------


-- Disable account creation by default, for security
-- For more information see https://prosody.im/doc/creating_accounts
allow_registration = false

-- Debian:
--   Do not send the server to background, either systemd or start-stop-daemon take care of that.
--
daemonize = false;

-- Debian:
--   Please, don't change this option since /run/prosody/
--   is one of the few directories Prosody is allowed to write to
--
pidfile = "/run/prosody/prosody.pid";

-- Force clients to use encrypted connections? This option will
-- prevent clients from authenticating unless they are using encryption.

c2s_require_encryption = true

-- Force servers to use encrypted connections? This option will
-- prevent servers from authenticating unless they are using encryption.

s2s_require_encryption = true

-- Force certificate authentication for server-to-server connections?

s2s_secure_auth = true

-- Some servers have invalid or self-signed certificates. You can list
-- remote domains here that will not be required to authenticate using
-- certificates. They will be authenticated using DNS instead, even
-- when s2s_secure_auth is enabled.

--s2s_insecure_domains = { "insecure.example" }

-- Even if you disable s2s_secure_auth, you can still require valid
-- certificates for some domains by specifying a list here.

--s2s_secure_domains = { "jabber.org" }

-- Select the authentication backend to use. The 'internal' providers
-- use Prosody's configured data storage to store the authentication data.

authentication = "internal_hashed"

-- Select the storage backend to use. By default Prosody uses flat files
-- in its configured data directory, but it also supports more backends
-- through modules. An "sql" backend is included by default, but requires
-- additional dependencies. See https://prosody.im/doc/storage for more info.

--storage = "sql" -- Default is "internal" (Debian: "sql" requires one of the
-- lua-dbi-sqlite3, lua-dbi-mysql or lua-dbi-postgresql packages to work)

-- For the "sql" backend, you can uncomment *one* of the below to configure:
sql = { driver = "SQLite3", database = "prosody.sqlite" } -- Default. 'database' is the filename.
--sql = { driver = "MySQL", database = "prosody", username = "prosody", password = "secret", host = "localhost" }
--sql = { driver = "PostgreSQL", database = "prosody", username = "prosody", password = "secret", host = "localhost" }


-- Archiving configuration
-- If mod_mam is enabled, Prosody will store a copy of every message. This
-- is used to synchronize conversations between multiple clients, even if
-- they are offline. This setting controls how long Prosody will keep
-- messages in the archive before removing them.

archive_expires_after = "1w" -- Remove archived messages after 1 week

-- You can also configure messages to be stored in-memory only. For more
-- archiving options, see https://prosody.im/doc/modules/mod_mam

-- Logging configuration
-- For advanced logging see https://prosody.im/doc/logging
--
-- Debian:
--  Logs info and higher to /var/log
--  Logs errors to syslog also
log = {
    -- Log files (change 'info' to 'debug' for debug logs):
    info = "/var/log/prosody/prosody.log";
    error = "/var/log/prosody/prosody.err";
    -- Syslog:
    { levels = { "error" }; to = "syslog";  };
}

-- Uncomment to enable statistics
-- For more info see https://prosody.im/doc/statistics
-- statistics = "internal"

-- Certificates
-- Every virtual host and component needs a certificate so that clients and
-- servers can securely verify its identity. Prosody will automatically load
-- certificates/keys from the directory specified here.
-- For more information, including how to use 'prosodyctl' to auto-import certificates
-- (from e.g. Let's Encrypt) see https://prosody.im/doc/certificates

-- Location of directory to find certificates in (relative to main config file):
certificates = "certs"

-- HTTPS currently only supports a single certificate, specify it here:
--https_certificate = "/etc/prosody/certs/localhost.crt"

----------- Virtual hosts -----------
-- You need to add a VirtualHost entry for each domain you wish Prosody to serve.
-- Settings under each VirtualHost entry apply *only* to that host.
-- It's customary to maintain VirtualHost entries in separate config files
-- under /etc/prosody/conf.d/ directory. Examples of such config files can
-- be found in /etc/prosody/conf.avail/ directory.

------ Additional config files ------
-- For organizational purposes you may prefer to add VirtualHost and
-- Component definitions in their own config files. This line includes
-- all config files in /etc/prosody/conf.d/

-- Additional downloaded module
turncredentials_host = "ADD TURN SERVER HERE"
turncredentials_secret="SUPER SECRET TURN SERVER SECRET HERE"


--VirtualHost "example.com"
--    certificate = "/path/to/example.crt"
VirtualHost "machine.domain.org"
--ssl = {
--      key = "/etc/prosody/certs/machine.domain.org.key";
--     certificate = "/etc/prosody/certs/machine.domain.org.crt";
--}

------ Components ------
-- You can specify components to add hosts that provide special services,
-- like multi-user conferences, and transports.
-- For more information on components, see https://prosody.im/doc/components

---Set up a MUC (multi-user chat) room server on conference.example.com:
--Component "conference.example.com" "muc"

Component "conference.machine.domain.org" "muc"
    modules_enabled = { "muc_mam"; "muc_cloud_notify" }
--    restrict_room_creation = "local"
    restrict_room_creation = "true"
    name = "Private Family Chatrooms"

--- Store MUC messages in an archive and allow users to access it
-- modules_enabled = { "muc_mam"; "muc_cloud_notify" }
-- modules_enabled = { "muc_cloud_notify" }

---Set up an external component (default component port is 5347)
--
-- External components allow adding various services, such as gateways/
-- transports to other networks like ICQ, MSN and Yahoo. For more info
-- see: https://prosody.im/doc/components#adding_an_external_component
--
--Component "gateway.example.com"
--    component_secret = "password"


Include "conf.d/*.cfg.lua"