MongooseIM 2.2 - The Global Server

By Piotr Nosek

While every MongooseIM release comes with important improvements, sometimes they are more relevant for certain groups of users. In MongooseIM 2.2 we present 3 major features: SASL EXTERNAL authentication mechanism, stable Global Distribution, and Jingle/SIP proxy.

All of them are especially beneficial to larger deployments or big companies.

They often already provide certificate-based authentication to their users, so with SASL EXTERNAL it is no longer required to introduce new passwords or tokens to your existing system.

Also, when they hit a certain cluster scaling limit, they need to distribute a service to several data centers. In order to retain user experience, Global Distribution comes into play.

The third element, Jingle/SIP proxy, will be appreciated by VoIP app owners who would like to incorporate MongooseIM and integrate it with existing clients and components.

If you’d like to learn more, please keep reading.

One more thing: I think we owe a word of explanation for the community members who were tracking our repository and our responses to issues on GitHub. We were on track to release MongooseIM 3.0, but it turned out that certain important Pull Requests required further work. We didn’t want to keep you waiting, so the new version is here anyway. Only with different version number increase.

Welcome to MongooseIM 2.2!

Authentication with certificates

Typical XMPP authentication involves a password or a token that is sent to the server. There are several SASL (Simple Authentication and Security Layer) mechanisms with SCRAM-SHA-1 being the recommended one.

In 2006, a set of recommendations was published as XEP-0178. They describe best practices for SASL EXTERNAL mechanism usage. It is actually a pretty mature extension, but it gained extra attention recently when more and more companies wished to use certificate-based authentication.

SASL EXTERNAL implementation in MongooseIM 2.2 follows the aforementioned XEP and tightly integrates with TLS encryption used for connection. It means that client certificate used for securing the stream may be reused to log the client in. It is not only a very secure method, but also avoids introducing concepts of passwords and tokens in systems where they are not actually needed.

Stable Global Distribution

All of you know what mobile network and landline are. The former allows you to call everyone in the world, as long as they carry their device with them, always with the same number. The latter requires the recipient to be present at a specific location in order to receive a call. What is more, you actually have to know where they are, as landline phone numbers (at least usually) include prefixes assigned to certain districts or cities.

I’ve included a reminder of all these attributes to explain a difference between Global Distribution and classic XMPP Federation. The former is an XMPP cellular network and the latter is an XMPP landline. GD is a custom extension developed by the MongooseIM team, exclusive to our server. With GD enabled, your clients no longer need to be aware of the recipient’s exact location. They may be connected to any cluster.

As you may guess, message routing in this case is more expensive than the old model. Actually, it requires a distributed routing table. The current solution may be used with a Netflix Dynomite layer on top of Redis.

The teams behind smaller apps may rest assured — when your solution grows, it will be possible to enable Global Distribution later, for existing cluster(s). You don’t need to worry about the Dynomite and Redis overhead already. :)

Global Distribution module was already present and mentioned in the 2.1.1 release. It was considered highly experimental back then but we’ve received many reports about bugs and possible improvements. We’ve resolved all of them, so in MongooseIM 2.2 we consider it stable. It is still not fully compatible with some extensions, as e.g. users migrating between clusters could lead to fragmented message archive. Nevertheless, it’s core purpose — routing stanzas between users and components in different clusters — works very well and features several mechanisms to prevent message loss and ensure efficiency.

Jingle/SIP proxy

Many VoIP applications rely on SIP — a protocol with good support and wide range of software implementing it. It has its shortcomings though. First of all, it is difficult to extend and is focused on P2P session management. Secondly, it does not use long-lived sessions.

XMPP suffers from neither of them, but usually you can’t simply turn off old parts of your system and enable the new one in order to migrate. A transition period is not uncommon. In some cases, SIP-exclusive apps might require really long maintenance, especially in environments where hardware and software are not updated that often.

Thanks to MongooseIM 2.2, when you decide to switch to XMPP in your application, you’ll be able to seamlessly connect SIP and XMPP worlds. Our server may translate essential SIP packets to Jingle stanzas and back. Please check documentation for mod_jingle_sip module for further details.

Jingle/SIP switch feature is sponsored by Sonetel.

A tutorial for this feature will be available soon!

Important security fix for error routing

This patch deserves a separate section as it is especially important for deployments based on previous MongooseIM versions that use the mod_event_pusher_push extension. A malicious user may send a prepared stanza that eventually could lead to an endless routing loop that crashes the MongooseIM node due to the out of memory exception.

It is highly recommended to update every deployment. Those using heavily customised server versions, that are challenging to merge, should at least apply PR #1800.

Misc. technical improvements & changes

It’s always cool to have a strong highlights in a release but much more always happens behind the scenes. MongooseIM 2.2 includes many smaller changes or the ones relevant to developers rather than users and devops.

Header files refactoring

MongooseIM header files underwent significant refactoring, including split into smaller files and moving definitions to more appropriate places. Here are some examples: ejabberd.hrl has been renamed to mongoose.hrl All error macros from jlib.hrl have been transformed into mongoose_xmpp_errors.erl module. All namespace definitions from jlib.hrl have been moved to mongoose_ns.hrl JID-related type definitions have been moved to jid.erl

For a full diff, please consult PR #1570

Mongoose Services

Classic MongooseIM extension modules are started for every served XMPP domain, as configured in a hosts option in a config file. In most cases it is a perfectly valid approach, e.g. in case of roster or privacy lists support. It does not apply though to features that should be started only once, e.g. Global Distribution (because it strictly operates only with two domains: global and local one) or mod_admin_extra.

Until 2.2, various hacks were necessary to ensure single init for these extensions, such as: Custom ETS table to count “reference/init count” Initialisation only for specific domain Completely ignoring the problem :)

In order to resolve this problem, Services are introduced in MongooseIM 2.2. These are very similar to modules but are always started before them and only once per node. When an extension depends on a service, it is impossible to provide a default configuration to use (as opposed to module dependencies) so required services must always be properly configured. They are prefixed with service_, instead of mod_.

For now, only mod_admin_extra has been converted into service_admin_extra. More extensions will follow, most probably in 3.0’s successor.

Hidden components

Sometimes XMPP components connected to the server do not interact directly with ordinary clients but they are listed in Service Discovery result anyway. It is inconvenient if the deployment uses many of them and the Disco response to a client is not only big but simply cluttered.

It is now possible to open a separate endpoint for such components. With proper option enabled, they will act as usual but won’t show up in Disco results sent to the clients. They are “hidden components”.

Please note that other services may still discover them. Also, the server won’t prevent clients from contacting hidden components, so it does not increase security but rather efficiency and makes frontend apps easier to maintain (if they rely on Service Discovery much).

Changelog

Please feel free to read the detailed changelog, where you can find a full list of source code changes and useful links.

Contributors

Special thanks to our contributors: @igors @jacksgt @sstrigler @GalaxyGorilla @varnerac!!

What’s next?

MongooseIM 3.0 is going to be released on May 22nd. It will bring exciting improvements that all MongooseIM users will appreciate for sure.

Inbox has been present in many applications for a couple of years already and our experience with various implementations resulted in an extension that covers common use cases for conversations view. It is undergoing last reviews and improvements and will be included in 3.0.

Erlang terms, while very practical for MongooseIM developers, were never really intuitive for administrators and devops. In order to make their lives easier, we’re going to change the config file format to a completely new one. TOML is the most promising candidate.

We’re experimenting with a new XML parser. The current one is built on top of expat and the new one uses RapidXML. Initial benchmarks indicate significant improvements in raw performance and the new implementation is being polished.

Obviously, the 3.0 release will be one of the most important milestones in MongooseIM history!

Test our work on MongooseIM 2.2 and share your feedback

Help us improve the MongooseIM platform:

1. Star our repo: esl/MongooseIM
2. Report issues: esl/MongooseIM/issues
3. Share your thoughts via Twitter: twitter.com/MongooseIM
4. Download Docker image with new release
5. Sign up to our dedicated mailing list to stay up to date about MongooseIM, messaging innovations and industry news.
6. Check out our MongooseIM product page for more information on the MongooseIM platform.

Originally published at www.erlang-solutions.com.