# XMPP: basics, security, constrained networks



## sidetone (Oct 5, 2020)

Core features of XMPP are defined by RFC's, which can be found at IETF.org
Extended features of XMPP are defined by XEP's, which can be found at XMPP.org/extensions
XMPP is an open standard, rather than open software (it includes plenty of open software as well)
The XMPP standard is approved by the IETF (Internet Engineering Task Force)
Aside from chatting, XMPP can be used for automation (via instructions), feeds and micro-blogging
50 million XMPP users in 2009
Zoom, Kontalk and Jitsi messaging services are based on XMPP; combined, these consist of over 200 million users
There are over 1 billion users on XMPP based proprietary networks
Google and DuckDuckGo used to offer XMPP based services
To find XMPP/Jabber related ports: `psearch -c net-im -so xmpp jabber`. The _s_ option allows a deep search into the long description; the _o_ option is for or. As of now, there are a minimal amount of XMPP/Jabber related ports is in the _net_ category.

XMPP user accounts
A *Jabber Identifier* (*JID*) consists of the username, and domain name of an XMPP user account. This can also include the extended *resource part*, which is an id for each device or connection.

XMPP *server directory* and test results by security grade:





						IM Observatory
					






					xmpp.net
				



Servers not listed yet or that haven't been checked recently can be tested on this website. Server to server (s2s), and client to server (c2s) connections can be checked. This security grade is by technical measures, so also, go by server's reputation.

To test XMPP servers for limited XEP's that are used with Conversations Messenger:
https://compliance.conversations.im/tests/

Stream & Connection
An XMPP communication session is a stream over a TCP connection that lasts the length of a conversation. There are 2 streams established, 1 for each direction. The server and client each create their own XML XMPP document that is added to as a conversation progresses. A server doesn't *poll* the client for new messages or updates. Rather than constantly waiting for messages, the server sends or holds a message for sending as soon as it's received.

The route an XMPP communication takes is a direct route from client to server, to server, to client: this is called *direct federation*. (email uses indirect federation, which many indirect routes are taken between servers.)

In the traditional mode of XMPP communication, messages can get lost and aren't resent.

The resource part of a full JID can be used for further routing communication. Typically, only one device receives a message, which can be set by priority. It can be configured for many devices to receive a message.

The clients on their end close their XML/XMPP documents when a disruption in the stream is sensed.


XMPP components
When connection is started, XML begins as normal, with: `<?xml version="1.0"?>`

Under this is the `<stream:stream>` element, which doesn't close until the end of the connection. Then, after the opening `<stream:stream>` element, there's the `<stream:features>` self-closing element, which lists the features available for use in XMPP.

Three primary XML elements exchanged between server and client are called *stanzas*: *MESSAGE*, *IQ* and *PRESENCE*.

message stanza
Attribute `type` for message: error, normal, chat, groupchat, headline. Other attributes for message: to, from, id. Directs where message is sent in groupchat.

iq stanza
Is used for queries. In groupchat, can be used to change user permissions, or kick someone out of room. Can be used to reserve chatroom name.

presence stanza
Indicates users' availability. Extensions can indicate when user is typing, and another presence stanza is sent when there's a pause in typing. Join notification is indicated by this stanza. Tells when client leaves, unless the server to server connection goes down unexpectedly. Can indicate client's `role` (temporary status) or `affiliation` (persists across logins) in chatroom: visitor, participant, member, moderator, admin, owner.

File transfer
Bits of Binary (XEP-0231)/In-Band Bytestreams (IBB: XEP-0047) can be used to transfer small files over XMPP. For larger files, it is inefficient, and can cause congestion. It encodes the file into base 64, then breaks it down, to the a maximum size allowed for each message to be transferred. It is sent by message, with an indication of sequence. The IQ stanza then returns a receipt, that the messages were delivered.

Out-of-Band (OOB), Socks5 Bytestream (XEP-0065) is used to transfer messages bypassing the server, or directly from client to client. XMPP negotiates the transfer over the SOCKS5 protocol. The bytestream proxy can be on a separate machine than the XMPP server.

Jingle (XEP-0234) can be used for file transfer, as a more modern alternative to XEP-0065.

XEP-0087, XEP-0095 and XEP-0096 for transferring files are obsolete or retracted.

Server configuration (Prosody) and telephony messaging (Asterisk) are topics outside the scope of this thread.

XMPP extensions





						XMPP | Specifications
					






					xmpp.org
				



For more features, see post #11 and other posts below. For a previous discussion about extensions, see: Thread xmpp-extensions.77183.

About Opensource communication frameworks:

Thread opensource-communication-frameworks-xmpp-sip-amqp-mqtt-cap-iax.79474

References (by order of use)

XMPP: Definitive Guide - Peter Saint-Andre, Kevin Smith, Remko Troncon. O'Reilly (2009)
xmpp.org
ietf.org
other XMPP related threads on this forum
Practical XMPP - Lloyd Watkins, David Koelle. Packt (2016)


----------



## sidetone (Oct 5, 2020)

The direct federation model has advantages over client to client negotiations, and the indirect federation model, for both security and for improved network reliability.


*Security*
The XMPP server coordinates encryption, spoof prevention and other security features. Security features can't be ignored from the client end. XMPP servers generally have encryption features available.

Because servers connect directly to each other, spoofing is prevented from originating between these two servers. Address spoofing originating from the client is also prevented, because the servers add this relevant information to the _from _attribute_._

The ability for encryption comes with the core of XMPP (RFC 5246 and RFC 4422).

There is a *certificate authority* (CA) that is available for free, for XMPP servers: xmpp.org/ca. This is important to avoid the use of self-signed certificates, which can be a vulnerability for network spoofing. According to https://blog.prosody.im/mandatory-encryption-on-xmpp-starts-today/, there is intent within the XMPP community to replace CA's (Certificate Authorities) with models in the spirit of XMPP. DNSSEC, http://web.monkeysphere.info/, and fingerprints were mentioned.

Use TLS encryption plus SASL authentication.

TLS (there's no TLS subset of XMPP) is used to secure connections. The connection starts off unencrypted until TLS negotiates and encryption is set.

Don't use *PLAIN* text for sending authentication/password data, unless the connection is already encrypted with TLS. See about other *SASL* authentication methods. Password is sent over base64. Once authenticated, a new XML session is started, with `<stream:features>` set with the enabled security features. Anonymous is an option for allowing clients to use an XMPP server without a password, for instance, when a service will be used very few times.

As for connecting, avoid *server dialback*, as this can be spoofed. DNS validation needs DNSSEC for better security.

While messages to the servers are usually encrypted, the servers can still have access to this as unencrypted data for viewing and logging. As of 2009, there wasn't end to end encryption between clients.

The more recent Omemo (XEP-0384) offers end to end encryption of text. It is in experimental state. Jingle Encrypted Transports (JET, XEP-0391) and the additional Jingle Encrypted Transports - OMEMO (XEP-0396) offer file encryption.

Small filesize images can probably be transferred securely by in-band bytestream methods through Omemo encrypted chat.

If a XMPP connection manager (CM) is used (this relates to BOSH), that is another travel point that must be secured, unless it is on the same computer as the XMPP server.

Watch out for Unicode text that substitutes letters of recognized JID's, as an attempt to deceive XMPP users.

See XEP-0205 for best practices to discourage DDOS attacks. Also of concern is the reputation of the XMPP servers.

Serverless connections negotiated through XEP-0174 aren't secure. This document refers to using TLS and SASL through RFC 7622 to secure connections. XEP-0174 makes use of Multicast DNS (mDNS) & DNS Service Discovery (DNS-SD). For clients, this extension uses the .local tld (top level domain).


Firewall
For basic connection port numbers for XMPP, and firewall examples in PF, see: Thread firewall-settings-for-xmpp.61219.


*Constrained/Intermittent Networks*
Because XMPP uses a direct federation model, chose a suitable server relatively close to your region. Server to server connections, are often much more reliable. A shorter distance from a client on a constrained network to a server, would be much better, than a client having to send messages a longer distance to another client on its own.

The ability for data compression comes with the core of XMPP (rfc 3749). Streaming compression comes with the extensions XEP-0138 for zlib, or XEP-0229 for lzw. These go under the `<stream:features>` element. With these compressions enabled, XML data surrounding messages and presence is expected to lower bandwidth, for each transport of a stanza at a time.

It is better that the server handles most of the features, so that the client won't have to do as much heavy lifting.

Because XMPP uses streams that require a constant connection, it isn't suitable, as is, for mobile phones, intermittent Internet and other constrained networks. An intermittent Internet service causes the network to have to be reestablished, using up plenty of bandwidth. *BOSH* (Bidirectional-streams Over Synchronous HTTP) allows a *connection manager* (*CM*) to sit between the XMPP server and client, to take care of the XML end, and send (primarily stanza) data to the device, without it requiring a constant connection. BOSH is handled by XEP-0124, XEP-0156 and XEP-0206.

In many ways, *WebSockets* (RFC 7395) is more advanced than BOSH, but it may possibly be less efficient for clients with intermittent connection or for clients on constrained networks.

XEP-0352: Client State Indication is another extension that saves bandwidth for mobile phones. It reduces the need for establishing XMPP connections and resending data required for interrupted XMPP streams. This extension is available on some mobile clients like Conversations. Client State Indication is in the proposed state.

Presence settings can typically use around 90% of bandwidth data. For constrained networks, use minimal presence settings, or turn this off depending on needs.

For transferring files, In-Band Bytestreams (XEP-0047), a kind of Bits of Binary, can be a source of congestion on a constrained network.


----------



## Mjölnir (Oct 6, 2020)

FTR, most will already know this: XMPP is also known as _Jabber_.  These are two names for the same protocol.
The (non-) availability/support of extensions of a particular client and/or server can be a source of failure.
You need to have a reliable password manager, since many XMPP/Jabber servers allow anonymous registration; i.e. when you forget your password and/or account name, you're lost...  Depending on the configuration of the server, your account will be blocked after repeatedly failed connection attempts (for an hour or so).  I.e. you can not connect during that period, even if you remembered the correct password.
Naturally, some extensions (e.g. STUN) or environments (firewall) require additional configuration that can be non-trivial & hairy for newbies.
Setting up a server can be non-trivial.
Many server software allow to set up bridges/gateways to other instant messaging systems.
XMPP/Jabber can be considered a well-established standard, it is used e.g. in military and governmental environments, and some proprietary IM systems use XMPP under the hood; but there seems to be a shift towards the newer Matrix protocol (due to the mentionend issues?).


----------



## a6h (Oct 6, 2020)

mjollnir said:


> there seems to be a shift towards the newer Matrix protocol


For whatever reason, everybody want to avoid XMPP. They present different rationale, but I think it's all about XML. I don't know why and certainly it's not my position, but most people don't like XML.


----------



## ekvz (Oct 6, 2020)

mjollnir said:


> FTR, most will already know this: XMPP is also known as _Jabber_.  These are two names for the same protocol.



Yes, Jabber is pretty much the _oldschool_ name for XMPP.



mjollnir said:


> there seems to be a shift towards the newer Matrix protocol (due to the mentionend issues?).



While i am not very deep into this i think Matrix might profit a lot from the being _new_ and the general hype around it. In any case i don't get the impression that it's technically less complicated than XMPP (rather the opposite). Sure it might be better at hiding this from users and given it's new it does not have to rely on a ton of extensions but i really don't feel comfortable with it's current state.

There seem to even be closed source components to it which wouldn't be much of a problem as the standard is actually open but from what i read those closed parts don't actually fully follow the published standard thereby complicating the development of open alternatives and resulting in what pretty much seems to be a semi centralized system right now. It's overall kinda messy and in my opinion privacy isn't taken very serious by the Matrix developers at all.

There is a bit of discussion here https://github.com/privacytools/services/issues/17 and there is more if one pokes around.


----------



## Mjölnir (Oct 6, 2020)

IMHO it's not about XML, but the interoperability between different clients.  E.g. I have 3 different authentication methods to verify an OTR encrypted channel, and the other side does not neccessarily support them.  It's non-trivial, for newbies it's too complicated.


----------



## ekvz (Oct 6, 2020)

mjollnir said:


> IMHO it's not about XML, but the interoperability between different clients.  E.g. I have 3 different authentication methods to verify an OTR encrypted channel, and the other side does not neccessarily support them.  It's non-trivial, for newbies it's too complicated.



I agree. XML is not _that_ bad but XMPP (the protocol) is simply braindamagingly complicated and convoluted with all it's extensions leading in part to said incompatibilities. As far as the choice of XML is concerned it's more about cleartext being a bad fit to transfer binary data than XML itself. XMPP is sadly pretty much stuck in a couple of ways and i think that's sad as i am watching recent developments like Matrix with a lot of scepticism.


----------



## a6h (Oct 6, 2020)

Signaling is a problem: session, metadata, NAT, etc. WebRTC doesn't implement Signaling either. That's why there's ICE, STUN, etc.
OT: Signalling or Signaling? which one is correct!


----------



## Mjölnir (Oct 6, 2020)

vigole said:


> Signaling is a problem: session, metadata, NAT, etc. WebRTC doesn't implement Signaling either. That's why there's ICE, STUN, etc.
> OT: Signalling or Signaling? which one is correct!


Now this should really go into a group chat...   I'd say signaling is correct.


----------



## a6h (Oct 11, 2020)

I've recently read a lot of posts, on the topic of Jabber/XMPP, for the most part posted by sidetone and mjollnir
I've never had a Jabber ID. Now I have one! and I'm going to use it. I think it's a interesting protocol. I hope it gains some popularity.
Thank you: sidetone and mjollnir


----------



## sidetone (Oct 12, 2020)

*Additional uses of XMPP*
These generally require extensions.

*Remote commands*
Can be used to forward and manage XMPP messages, and to control client devices. An additional use is for automated devices (automation data). IQ stanza is to query capabilities. Data can also be sent to SOAP and RPC protocols. See: XEP-0050, XEP-0133, XEP-0146,  and XEP-0244.

For IOT, see: XEP-0323, XEP-0324, XEP-0325, XEP-0326 and XEP-0347.

*Pubsub*
Publish/subscribe can be used for micro-blogging, and subscribing to other updates. Each subscription is a node on a Jabber ID (JID).

*Data broadcasting*
This is for use in Multi-user chat (MUC). MUC isn't only for chatrooms. MUC simply multiplies a message for many receivers. Data broadcasting (XEP-0045) is for relaying information to multiple devices.

XEP-0166 and XEP-0167, can be used for signaling for group sessions.

Mediated Information eXchange (MIX), XEP-0369, is intended to be a replacement for MUC, for its perceived shortcomings. This extension is in the experimental state.

*Jingle*
For Jingle, use ICE signaling, which is designed for optimal use with NAT. XMPP negotiates a client to client Jingle session. Previously mentioned is using Jingle for transferring files, such as images. Additional uses for Jingle is a topic outside the scope of this thread.

*Other*
Some of these features can be used with above capabilities.

vcard extension - For showing JID profiles. Can be queried from server, unless the information is wanted from a specific device on the base JID.
chat state notification - Turning this off, saves bandwidth.
privacy lists and blocking - Shows as if user hasn't logged on. Messages can get dropped by server.
receipts - Can be sent to sender of message, to know if important message was received.
message archiving
advanced message processing
discovery protocol - Find services on servers. disco#items discovers entities, and disco#info discovers features on these entities.
data forms
captcha forms
geolocation (XEP-0080)
atom feeds (RFC 4287) - Since Atom is XML, it can be used directly by XMPP.
jukebox/radio (XEP-0118)
application and gaming data
XEP-0333: Chat Markers - sends an indication to the sender when a message has been read.


----------



## ekvz (Oct 12, 2020)

vigole said:


> I think it's a interesting protocol.



In any case it's pretty much the most trustworthy and functional protocol there is right now in my opinion. Sadly that only means so much. There is quite some ugly stuff about it also (often more or less by design so it's unlikely to be fixed).



vigole said:


> I hope it gains some popularity.



I hate to be pessimistic but i seriously doubt it. It's simply to much of a mess technically to really move forward.


----------



## sidetone (Oct 13, 2020)

*BOSH*
BOSH (Bidirectional-streams Over Synchronous HTTP, XEP-0124) uses tcp port 5280. https may be a fall-back port for BOSH.

*XMPP clients with BOSH support*

JWChat - jwchat.org uses JWChat webclient
Tigase messenger
Pidgin can be configured to use BOSH.
SparkWeb (uses Flash)
Webclients either generally or mostly use BOSH or WebSocket.

*Other mentions of BOSH support*

According to wiki.xmpp.org, BOSH support was being added to Gajim in 2008. However, it seems that it's not currently in Gajim. Gajim currently uses WebSocket.
Kiawa is a webchat application that likely uses BOSH. This can also be used at jwchat.org
Gloox is a C++ XMPP library that has BOSH support.
XIFF is a BOSH library for connection managers
net-im/py-punjab  is an XMPP connection manager that uses BOSH. This would still require TLS encryption that is not based on a self-signed certificate, and would still require SASL authentication.

*Further reading on XMPP over BOSH*

XMPP: Tech pages/BOSH - xmpp.org
Understand Finesse BOSH Implementation and Troubleshoot Finesse Agent Desktop Presence Disconnect Issues - cisco.com
https://xmpp.org/about/technology-overview.html#bosh (contains links to XEP-0124 & XEP-0206)
https://xmpp.org/extensions/xep-0156.html - Discovering Alternative XMPP Connection Methods
Setting up a BOSH server - prosody.im
https://wiki.xmpp.org/web/Bidirectional-streams_Over_Synchronous_HTTP_(BOSH)_(XEP-0124)

*Possible Alternatives or Complements to BOSH*
As mentioned in an above post, WebSocket is more advanced than BOSH, but it may possibly be less efficient for intermittent and constrained networks.

Jabber HTTP Polling (XEP-0025) has been obsoleted by BOSH.

A possible addition for web management is Stream Management (XEP-0198), which synchronizes messages when XMPP reconnects.

XEP-0273: Stanza Interception and Filtering Technology (SIFT) blocks downloading unwanted stanzas (for instance, excessive presence updates) from the client end, to minimize bandwidth on constrained networks and mobile phones.

XEP-0352: Client State Indication (CSI) reduces bandwidth to mobile phones, including by reducing the need for reconnecting on intermittent networks. This has similarities to SIFT, but this is done on the server end. It can hold messages until user is active on client. CSI is possibly an alternative to BOSH. I haven't researched how well this would work in place of BOSH, or if this can be used in conjunction with BOSH.


----------



## Mjölnir (Oct 13, 2020)

sidetone: your last two posts perfectly show the shortcomings of _Jabber/XMPP_: it's kind of _nerdy_ in that it offers solutions for many issues via it's extensions, OTOH that makes it hairy to configure for the average, innocent user.  AFAICT, _TOX_ & maybe _Matrix_ seem to be much easier to set up on the client-side.


----------



## a6h (Oct 13, 2020)

mjollnir said:


> it's kind of _nerdy_ in that it offers solutions for many issues via it's extensions,


I had to spend a lot of time to choose and setup a perfect XMPP client. Also, one of the FreeBSD users gave me help with correct usage of XMPP client.
But If you only want to chat, i.e. not paying attention to details e.g. privacy and security, then XMPP web interface (register/usage) is very straightforward.


----------



## getopt (Oct 13, 2020)

mjollnir said:


> it's kind of _nerdy_ in that it offers solutions for many issues via it's extensions,


Yes, XMPP has a learning curve, it includes learning about it's many XEPs. Or at least the ones you have in your client-app.

Lamenting about "nerdyness" is lamenting about a certain unwillingness to learn. XMPP may not be for _Greti & Pleti_, as FreeBSD is not for everyone. Both requires to RTFM. If that's too much effort there are apps for everything you can go shopping for.



mjollnir said:


> for the average, *innocent* user.


I do not want to contribute to further off-topic in the *Howtos & FAQs Forum*. But coming here along with the attribute "innocent" is a funny entertainment.


----------



## sidetone (Oct 13, 2020)

mjollnir said:


> it's kind of _nerdy_ in that it offers solutions for many issues via it's extensions, OTOH that makes it hairy to configure for the average, innocent user.



Let the servers, connection managers and clients make a standardized offering or set (grouped/recommended extensions) on XMPP. For mobile phones, there's a need for BOSH and/or Client State Indication to become common. The XMPP platform is there as an open standard, that is ready for what is needed by any particular group, commonality or other entity.

Some features are only needed on one end, such as BOSH or other implements for constrained and other intermittent networks, until the server is reached.

End to end encryption is often needed on both ends. Encryption is in base, and ready to use, but it allows that traffic to be unencrypted at the server for viewing or logging. Omemo is becoming a new standard, replacing OTR but a bit difficult to use.

Let the server, connection manager and client be reputable, and let it inform the user how to ensure the needed features are enabled.

A lot of extensions are for nifty uses, and not necessary for typical person to person chatting.


----------



## sidetone (Oct 14, 2020)

*Low-power Lossy Networks (LLN) update

XEP-0322: Efficient XML Interchange (EXI)*
This compresses data efficiently, because it is made for XML. Additional XML schemas (including XMPP extension data) can be on server to further improve efficiently.

https://xmpp.org/uses/iot/bindings.html
https://xmpp.org/extensions/xep-0322.html


----------



## ekvz (Oct 14, 2020)

mjollnir said:


> sidetone: your last two posts perfectly show the shortcomings of _Jabber/XMPP_: it's kind of _nerdy_ in that it offers solutions for many issues via it's extensions, OTOH that makes it hairy to configure for the average, innocent user.  AFAICT, _TOX_ & maybe _Matrix_ seem to be much easier to set up on the client-side.



Exactly and it's almost worse when having to deal with the actual implementation.



getopt said:


> Yes, XMPP has a learning curve, it includes learning about it's many XEPs. Or at least the ones you have in your client-app.



While i agree that it's obviously possible to deal with it i think _nerdyness_ is actually quite fitting. XMPP as a whole is quite nerdy. In my opinion the way it's protocol is designed also shows clear signs of _nerdyness_. I think it's obvious how having a protocol consist of abstract building blocks with potential reusability rather than simple feature definitions would be appealing to this group. Actually i can very much understand myself how this might look very elegant, clean and superior in theory. Sadly the practical side of things is a bit different.



sidetone said:


> This compresses data efficiently, because it is made for XML. Additional XML schemas (including XMPP extension data) can be on server to further improve efficiently.



While that's nice i can't help but notice how this wouldn't have been a problem in the first place if the designers just had accepted that binary protocols obviously have clear advantages over XML. My impression is that XML mostly won due to _nerdyness_ while real world problems (transfering binary data...) were simply ignored.


----------



## sidetone (Oct 14, 2020)

Obviously, it's an XML format, and they should have known to use compression based on that in the beginning. Maybe hindsight is 20/20, or they used what was already available and didn't bother to make a more fitting adaption for compressing XML type data. That's a extension that should go into base, to a large extent.

I read somewhere, that there were disadvantages to using a format that wasn't XML. That XML is structured for keeping data organized, when transferring data/communicating. If bandwidth is the issue, XML can already be compressed, and an ideal means for it should be made standard.


----------



## ekvz (Oct 15, 2020)

sidetone said:


> Obviously, it's an XML format, and they should have known to use compression based on that in the beginning. Maybe hindsight is 20/20, or they used what was already available and didn't bother to make a more fitting adaption for compressing XML type data. That's a extension that should go into base, to a large extent.



Size isn't really the major problem. As you stated that's even somewhat fixable. It's more about how XML is quite bad when dealing with binary data. I think that's at least part of the reason why for example XMPP's concepts of transferring images rely on external solutions. External solutions either being direct connections between clients (which are problematic for multiple reasons) or an external HTTP server which also isn't exactly nice (even if it's by far the better option out of the two). The fact that going for a text based protocol will force you to rely on an encoding like base64 if you ever have to deal with binary data is something that's actually visible from a mile away and it's hard to imagine that this being a requirement would simply be overlooked. XML itself wasting bandwidth is not that bad in most cases. I think in a general sense it becomes mostly noticeable where low latency would be desirable but TCP isn't great in that respect anyways so it might be somewhat excusable.



sidetone said:


> I read somewhere, that there were disadvantages to using a format that wasn't XML. That XML is structured for keeping data organized, when transferring data/communicating. If bandwidth is the issue, XML can already be compressed, and an ideal means for it should be made standard.



Sure, binary protocols aren't without drawbacks either but when you put things like having to deal with byte order in a couple of places in contrast to having to base64 encode every bit of binary data it think it's quite obvious which is the lesser evil. Structure is not much of an argument in my opinion as you could easily define a binary layout that's pretty much equivalent to XML if you really wanted to.


----------



## sidetone (Oct 16, 2020)

*Further Reading*

_A Public Statement Regarding Ubiquitous Encryption on the XMPP Network_, by Peter Saint-Andre of xmpp.org (XSF) and jabber.org, https://raw.githubusercontent.com/stpeter/manifesto/master/manifesto.txt (2013-2014)
XMPP overlay (XO) - for disconnected, intermittent, limited (DIL) environments: https://www.nrl.navy.mil/itd/ncs/products/xo
XO: XMPP Overlay Service for Distributed Chat - https://apps.dtic.mil/dtic/tr/fulltext/u2/a534103.pdf
https://developer.ibm.com/technologies/messaging/tutorials/x-xmppintro/

*More XEP's*

https://xmpp.org/extensions/
XEP-0127: Common Alerting Protocol (CAP) over XMPP - CAP is used by NOAA and EAS for notifications; robotic voice for tv, radio, telephone; text alerts for tv
XEP-0286: Mobile Considerations on LTE Networks
XEP-0289: Federated MUC for Constrained Environments
XEP-0361: Zero Handshake Server to Server Protocol - s2s connection for constrained networks; servers must be configured to work together beforehand
XEP-0365: Server to Server communication over STANAG 5066 ARQ - this is for military use, but is an example of how XMPP can be used over HF radio; this extension relies on XEP-0361
XEP-0423: XMPP Compliance Suites 2020 - yearly update on status changes on XMPP extensions and their categories
XEP-0443: XMPP Compliance Suites 2021

*Additional Links*

https://www.cryptoparty.in/connect/contact/jabber - list of Jabber servers with Tor ability, and configuration directions
https://compliance.conversations.im/tests/ - tests XMPP servers for limited XEP's that are used with Conversations Messenger
http://jabber.markmail.org/ - search XMPP mailing lists


----------



## sidetone (Nov 11, 2020)

*Setting up BOSH for client use*

Check if your XMPP server has BOSH capability. See if that website that hosts your XMPP server has a BOSH URL. For instance, https://dismail.de/info.html:

```
XMPP Server: ejabberd
s2s encryption: is required
Client-Port: 5222 (STARTTLS only)
MUC: rooms.dismail.de (multi user chat)
Proxy: proxy65.dismail.de Port 5555 (dismail.de only)
BOSH URL: https://dismail.de/http-bind
Websocket: wss://dismail.de/ws
```

If your server doesn't have BOSH capability, find another XMPP server, or find a BOSH connection manager (CM) that is an intermediary between the XMPP server and your XMPP client. It may be better to use the same XMPP server that has BOSH capability.

*For Gajim and Pidgin*

From Gajim: Gajim --> Preferences --> Advanced --> Global Proxy --> Manage
Add a profile, and enter the correct settings, including for BOSH URL.

In Pidgin, when adding or modifying accounts, go to Advanced, then locate the BOSH URL box. Unfortunately, Pidgin doesn't have OMEMO encryption support. It rather has OTR support.

*Remarks*

You'll have to do your homework to make sure these settings are secure and functional. Similar instructions could be used for Websocket or other features.


----------



## sidetone (Jan 16, 2021)

*XMPP Clients*
* net-im/pidgin doesn't have OMEMO capability. It has OTR which is older. It's described as outdated for XMPP in general. Lurch libpurple plugin for OMEMO?
* The available version of net-im/gajim on FreeBSD is outdated for OMEMO to work (OMEMO has worked a few months ago). I attempted to upgrade it on my computer through an altered Makefile. The available version for FreeBSD is 1.1.3. The current version of Gajim is 1.2.2.
* net-im/dino doesn't list BOSH in its XEP's.
* net-im/psi: The website says it has OMEMO, but I could only find PGP encryption in the settings. Doesn't have BOSH or Websocket settings. With PSI, one must check all options before using it.
* conversations.im has a lot of features stuitable for mobile. Unsure if it uses Websockets by default.

WebSockets comes with the core of XMPP. Clients still need to have ability to configure and use it.

*Comparison of encryption used by clients*
PGP is supposedly more secure than OMEMO. However, OMEMO is easier to use, and good enough all around. OTR is obsolete, and wasn't good for poor internet connections.

(*) edited afterwards


----------



## a6h (Jan 16, 2021)

My XMPP/OMEMO report (*)
1. net-im/profanity on FreeBSD works, but settings is a little tricky.
2. If you have to use Windows, custom-build "Psi+" is the best solution (supports OMEMO). No OMEMO support in pre 2.0, but it scheduled for 2.0 release.

(*) XMPP without OMEMO is useless IMO.


----------



## sidetone (Jan 19, 2021)

*WebSocket*
WebSockets comes with XMPP through RFC 7395. RFC 6455 is the overall WebSocket protocol. This was meant to replace previous technologies of Bidirectional data transfer (including BOSH) over HTTP. It works over TCP, allowing connections in both directions.

wss:// is the URI prefix for secure WebSocket. Data is secured under this transport, and not via frameworks under it. wss shouldn't be redirected to un-secure protocols like ws:// and http://. WebSocket was meant to work under HTTP ports 80 and 443. It is adaptable to have its own ports in the future, and to be improved to have a better handshake.

There were (or are) two ways to use XMPP with WebSocket (XMPP/WS): through a proxy server (like BOSH through a connection manager proxy), or have XMPP integrated with WebSocket at the server. Enabling WebSocket on an XMPP server requires configuration of a module.

WebSocket communications consist of: a handshake, and data transfer. Each XMPP stanza is en-capsuled or carried by 1 WebSocket message. WebSockets also uses pings to maintain or coordinate XMPP connections.

WSAPI is the WebSocket API for joining HTTP web applications to this protocol. For more details on WSAPI: https://www.w3.org/TR/websockets/


*Comparisons to BOSH*
WebSocket was meant for HTML5, for use with many applications. BOSH was made for XMPP. (BOSH previously replaced Jabber HTTP Polling [XEP-0025])
WebSocket can be used with Javascript. Bosh relies on Javascript with complicated libraries.

WebSocket uses less bytes for a message transfer than BOSH. Bosh uses long polling, and has "high transport overhead compared to XMPP's native binding to TCP." WebSocket overcomes webpolling problems over HTTP. Supposedly, WebSocket uses under 10 bytes of overhead for small messages. BOSH may use over 150 bytes of each pair of request and response headers.

From earlier information I read, it seemed that BOSH was better than WebSockets for intermittent network branches. WebSockets may be better, at least in many ways. WebSocket is “a more elegant, modern and faster replacement to Bosh,” according to ejabberd-websocket's module readme.

Bayeux and Comet were other methods for HTTP  that haven't gained as much traction.


*Improving Reliability*
WebSocket can use Stream Management Extension [XEP-0198] to better manage stream connections, and to help overcome shortcomings compared to BOSH. Stream Management Extension does a good job of managing interrupted (temporary disconnected) connections.

Client State Indication [XEP-0352] also helps on intermittent mobile networks. It allows the server to hold data until mobile client has a better connection.

XMPP Ping Extension [XEP-0199] can also be used, but many browsers aren't compatible with it.

For mobile connections, see: Mobile Considerations on LTE Networks [XEP-0286].


*Additional Refs and See also:*
* https://stackoverflow.com/questions...all-for-bosh-over-websockets-and-long-polling
* Known Issues and Best Practices for the Use of Long Polling and Streaming in Bidirectional HTTP - https://tools.ietf.org/html/rfc6202
* The Definitive Guide to HTML5 WebSocket: [Chapter 4] Building Instant Messaging and Chat over WebSocket with XMPP
* https://www.websocket.org/aboutwebsocket.html
* https://www.websocket.org/quantum.html


----------



## sidetone (Jan 24, 2021)

Is OMEMO broken only for programs on FreeBSD? Or is it broken across the XMPP network? I've seen a comment on the Internet, saying this happened a time before. Maybe OMEMO is supposed to be compatible across different clients, but hasn't been recently?

I tried OMEMO on Gajim and Dino recently, and they didn't work. It worked on Gajim late last year.

Lurch is an OMEMO plugin available for libpurple.


----------



## a6h (Jan 25, 2021)

AFAIK It works fine on custom build (OMEMO) of Psi+ for Windows; at least for past few days.


----------



## Mjölnir (Jan 25, 2021)

Maybe that relates to recent political events?  "They" don't like others to use strong encryption...


----------



## sidetone (Feb 23, 2021)

OMEMO recently worked between Gaijim and Dino.

There weren't updates available for Gajim for a long time. It may have been one of the cryptographic dependencies that needed upgrading.


----------



## sidetone (May 29, 2021)

I had trouble with OMEMO not working between two phones.

OMEMO has worked between Dino and Gajim from FreeBSD before. I also found it useful then, to upgrade these programs, plus their dependencies, to make OMEMO start working again.

So far, I found out that OMEMO on Conversations on my Android works with both Dino and Gajim on FreeBSD. The problem may be on the other user's phone. Or the account needs to be of the same domain.

I'll ask on their venue for support about it.


----------



## sidetone (Nov 17, 2021)

XMPP, A Comeback Story: A 20 Year Old Messaging Protocol For Robust, Private and Decentralized Communications
					

Everyone loves a comeback story.  Can a 20 year old overlooked technology make a comeback as a completely decentralized, free, and scalable technology for the growing masses of people who value freedom and privacy?




					takebackourtech.org
				












						Introducing BOSH and WebSocket Transport protocols - Wildix blog
					

In this blog article we are going to talk about transport protocols over the web, in particular, about BOSH and WebSocket.




					blog.wildix.com
				












						Why is jitsi using bosh instead of websocket for xmpp
					

@Pawel_Domas 🙏




					community.jitsi.org
				




*Updates:*

OMEMO for XMPP was adopted from Signal.
With appropriate XEP's, Websockets may be enough to overcome problems for intermittent/constrained networks that weren't as problematic to BOSH. Otherwise, without those XEP's, Websockets has less overhead and is faster than BOSH. This is written about in a previous post above, and also discussed in a link above. BOSH is also obsolete, but still in wide use. Websockets may be the way to go, as it is also HTML5 compatible.


----------

