mserver_web_logo

MServer C++ library

v2.1.4

Table of contents

Overview

MServer is a C++ library that implements the VStreamer interface. It serves H.264 (RFC 6184), H.265 (RFC 7798) and MJPEG (RFC 2435) video over RTSP, RTSPS, RTSP over HTTP/HTTPS, UDP multicast, direct RTP push, WebRTC and HLS.

This README calls each delivery path a leg: the served leg (clients that connect over RTSP, RTSPS or RTSP over HTTP/HTTPS, unicast or multicast), the push leg (direct RTP push), WebRTC and HLS.

MServer starts no external process. On Linux it links one external library: OpenSSL 3.0 or newer; the other libraries it links are vendored submodules. The FIPS 140-3 statements in this README depend on OpenSSL. On Windows MServer links no external library (cryptography goes through the system’s CNG API), and the legs that need TLS or DTLS are not available; see Windows build with MSVC.

The caller supplies the frames: RAW pictures, or pictures already encoded as H.264, H.265 or JPEG. MServer does the rest itself: pixel-format conversion, letterbox/crop scaling, overlay, encoding (through the VCodec object the caller passes in), Annex-B parsing, parameter-set extraction, RTP packetization, MPEG-TS multiplexing, KLV carriage, RTCP, TLS, digest authentication, ICE, DTLS-SRTP, WHEP signalling, session management and pacing.

Note: B-frames are not supported. This is deliberate. Without B-frames the presentation time (PTS) always equals the decoding time (DTS). RTP timestamps therefore never go backwards, and no part of the chain needs a reordering buffer.

Implemented protocols and standards

Area Standard Status
Control RTSP 1.0 – RFC 2326 full server: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET/SET_PARAMETER (request bodies framed by Content-Length), state machine with 404/405/454/455/457/461/501/551
  RTSPS (RTSP over TLS) yes, own listener, TLS 1.2 minimum
  RTSP over HTTP / HTTPS tunnel (Apple/QuickTime) yes, GET+POST paired by x-sessioncookie; the POST body is decoded as one base64 stream
  SDP – RFC 4566 a=control, a=rtpmap, a=fmtp, a=range, multicast c= with TTL
  Digest auth – RFC 7616 (SHA-256), RFC 2617 (MD5) both, selectable by compliance mode; rotating nonces, stale=TRUE re-challenge, per-cnonce nonce-count replay rejection (a nonce whose replay table is full is retired with stale=TRUE, so the replay window never reopens)
WebRTC WHEP signalling over HTTP/1.1 and HTTPS one POST carrying the offer, one DELETE ending the session; same digest authentication as RTSP
  ICE-lite – RFC 8445 sec. 2.4 host candidate only, never probes back; STUN RFC 5389 with MESSAGE-INTEGRITY and FINGERPRINT
  DTLS-SRTP – RFC 5764 DTLS 1.2 server role, self-signed certificate bound by a=fingerprint, keys from SSL_export_keying_material
  Demultiplexing – RFC 7983 STUN, DTLS, SRTP and SRTCP on one port; a=rtcp-mux
  Feedback – RFC 4585 / RFC 5104 PLI and FIR are honoured as key-frame requests
HLS HTTP Live Streaming – RFC 8216 protocol version 3, MPEG-TS segments, live sliding-window playlist, EXT-X-DISCONTINUITY; over HTTP or HTTPS on the same listener
  Segment encryption – RFC 8216 sec. 4.3.2.4 AES-128-CBC, per-segment DRBG IV, key rotation, key served from the same listener
Transport RTP / RTCP – RFC 3550 RTP, and Sender Reports that map NTP time to RTP time. The NTP time is the wall clock when the report is sent; a step of the wall clock sends the next report at once. SDES. A Receiver Report is a keep-alive: on both tracks’ RTCP ports, over UDP only from the client’s address, and on SAVP only when it verifies as SRTCP. BYE when a session (RTSP or WebRTC), a multicast group or the push leg leaves the RTP session: after its last packet, and as SRTCP on SAVP and WebRTC
  Feedback on RTSP / RTSPS sessions – RFC 4585, RFC 5104, RFC 4588 PLI and FIR ask for a key frame (at most one request per 200 ms per session is honoured, and at most one per 200 ms per stream path, whatever the number of sessions). Generic NACK is answered with RTX: payload type 99 for H.264 and JPEG, 100 for H.265, announced with apt and rtx-time=3000. The RTX payload type is derived from each packet’s own payload type, so it follows a live codec switch. Feedback is read as it arrives, over UDP and over TCP. It is accepted only when it names the video SSRC and, over UDP, only from the client’s address. A packet stays repairable for 3 s (at most 4 MiB per session). It is retransmitted at most once per feedback packet and three times in all. At most 32 NACK entries of one feedback packet are read. Unicast only: a multicast presentation announces no feedback (nothing reads a group’s RTCP), and an MPEG-TS presentation announces none and its NACKs are not answered
  UDP unicast, UDP multicast, TCP interleaved all three
  Direct RTP push (no RTSP) yes, list of destinations, unicast or multicast
  SRTP – RFC 3711, RFC 7714 AES_CM_128_HMAC_SHA1_80, AEAD_AES_128_GCM
  SRTCP – RFC 3711 sec. 3.4, RFC 7714 sec. 9.1 both profiles, own KDF labels, 31-bit index, replay window; applied on WebRTC and on SAVP RTSP subscribers, where RTCP that does not verify is discarded whole
  Key management – SDES RFC 4568 a=crypto in the SDP of the secure presentation, TLS listener only; proved end to end by a test that decrypts the media with the key it advertised
  Key management – MIKEY RFC 3830 pre-shared-key form is built (HDR/T/RAND/KEMAC+MAC) but not advertised: it has never been checked against a real ONVIF client, and offering an unverified key exchange fails worse than not offering one
Video H.264 over RTP – RFC 6184 single NAL + FU-A, packetization-mode=1
  H.265 over RTP – RFC 7798 single NAL + FU, no DONL
  MJPEG over RTP – RFC 2435 in-line quantization tables, restart markers; <= 2040x2040
  No B-frames enforced and proven by an automated slice_type census
Container MPEG-TS over RTP – RFC 2250 + MISB ST 0804 PT 33, 7x188 bytes per packet, PAT/PMT/PCR/PES. PAT and PMT every 100 ms and before every key frame. The PMT version_number follows its content. PTS and PCR are continuous 33-bit values. The PCR runs one frame interval + 10 ms behind the PTS (T-STD). On a live leg two PCRs are never more than 100 ms apart, down to one access unit a second
  MPEG-TS over UDP – MISB ST 1402 on the direct-push leg (7x188 per datagram, no RTP header)
Metadata MISB ST 0601 (UAS Datalink Local Set) byte-exact pass-through, optional validation, optional restamp; BER-OID keys (tags above 127 included)
  KLV over RTP – RFC 6597 smpte336m/90000 on its own payload type
  KLV in MPEG-TS – MISB ST 1402.1 asynchronous (0x06 + KLVA) and synchronous (0x15 + metadata descriptors)
  MISP precision timestamp – MISB ST 0604.6 user_data_unregistered SEI before the first slice of every H.264 / H.265 access unit whose KLV is a valid ST 0601 packet with Tag 2, carrying the Tag 2 value; Time Status 0x9F (no lock claimed, MISB ST 0603)

On Windows, everything in this table that needs TLS or DTLS is unavailable: RTSPS, the RTSP-over-HTTPS tunnel, HTTPS on the WHEP and HLS listeners, SRTP and SRTCP, and WebRTC. See Windows build with MSVC.

Regulatory compatibility

Standard Status How it is achieved
ONVIF Profile S / T Streaming Specification implemented; conformance belongs to the device all mandated transports, digest auth, keep-alive by any request and by RTCP RR, SetSynchronizationPoint via GENERATE_KEYFRAME, multicast with real group address. MServer has no SOAP layer – it exposes the control points a host’s SOAP service drives
FIPS 140-3 consumes CMVP #4985 (OpenSSL FIPS Provider 3.1.2) on Linux; MServer itself is not a validated module; open points under Verdict a private OpenSSL library context shared with CryptoProvider, RAND_bytes_ex for every secret, provider enforcement proved at load (an approved fetch must succeed and MD5 must fail), and a mode that runs without the module says so instead of claiming validation. Does not apply to a Windows build (there fipsActive only reflects the Windows FIPS policy flag)
EU CRA (Reg. 2024/2847) engineering requirements partly implemented (see Verdict) secure-by-default mode (CRA_STRICT, not the default), bounded parsers, no UDP amplification, fuzzing (harness/fuzz/, eight libFuzzer targets). Annex I Part II documents since 2.1.3: SECURITY.md (coordinated vulnerability disclosure policy, support period, Article 14 reporting process), sbom.cdx.json (SBOM) and SECURITY-ADVISORIES.md. Note: a written threat model and the conformity assessment are not in this repository; whoever places MServer, or a product with it, on the market supplies them. This README documents the engineering. Its trust boundaries are summarised under Cryptographic module boundary and Limitations.

Why compliance is switchable

These three standards contradict each other on the wire, so no single configuration satisfies all of them:

  • ONVIF Core sec. 5.9.3 makes MD5 the default RTSP digest, and nearly every deployed client implements only MD5. The FIPS provider does not implement MD5 at all.
  • ONVIF sec. 5.1.1.4 requires SRTP. Both SRTP profiles derive their keys with the SRTP KDF, which is absent from the approved-algorithm table of CMVP #4985. So any SRTP puts key derivation outside the validated module.
  • AES-GCM SRTP builds its IV outside the module, which the module’s security policy names as a non-conformance.
  • The CRA wants encryption and authentication on by default. ONVIF interoperability testing expects a device to stream with no prior configuration.

MServer therefore makes the choice a runtime switch: VStreamerParams::securityProfile. One product can then be certified against several standards. Each evaluation runs in the mode it requires, and the mode is a documented configuration item, not a separate firmware image.

A FIPS mode (STRICT_FIPS, FIPS_ONVIF_BRIDGE) does not start when the validated module is absent (2.1.3): a FIPS profile is a claim about the module, so initVStreamer() fails closed rather than running a server believed to be validated. The same hardened configuration without a module is CRA_STRICT, which makes no FIPS claim. (Up to 2.1.2 a FIPS mode started on the default provider and reported itself as (unvalidated provider).) See Compliance summary.

See Security and compliance modes for the full matrix.

Versions

Version Release date What’s new
1.0.0 23.08.2026 - First version.
1.1.0 09.09.2026 - Added HLS video streaming protocol.
- Added HTML page return for WebRTC and HLS.
- Fixed multiple bugs.
1.1.1 10.09.2026 - Fixed RTSP session expiry for a client that keeps its session alive with RTCP alone over TCP or the HTTP tunnel.
2.0.0 22.09.2026 - Parameters redefined.
- Fixed ONVIF compatibility.
- Fixed minor errors for stream stability.
2.1.0 30.09.2026 - Implemented PLI and RTX functions.
- Fixed minor errors for stream stability.
2.1.1 30.09.2026 - Fixed NACK, PLI and FIR over UDP being read only on the next 50 ms reactor poll.
- Fixed the JPEG SDP not announcing the RTX payload type it retransmits with.
- Fixed the first client of a just-started stream waiting a whole GOP for a key frame.
- Fixed a data race when the push leg’s container is changed at runtime.
- Fixed a possible SIGPIPE in closeVStreamer() with RTSPS clients connected.
- Fixed a disabled push leg sending RTCP Sender Reports.
- Fixed MServerTest describing an hevc or jpeg push with the H.264 payload type.
- Fixed RTSP request bodies not being framed by Content-Length.
- Fixed tunnelled RTSP requests split across TCP segments being lost.
- Fixed ;srtp and ;multicast leaking into the control URLs.
- Fixed the MPEG-TS PMT version_number not changing with the PMT content.
- Fixed MPEG-TS PTS and PCR wrapping at 2^32 instead of 2^33.
- Fixed KLV units too large for one MPEG-TS PES being dropped without a count.
- Fixed the MISB ST 0604 time stamp SEI never being emitted.
- Fixed klvUnits and klvRejected not being reset by initVStreamer().
- Fixed an RTSPS request over 4 KiB in one TLS record waiting for the client’s next bytes.
- Fixed one NACK drawing unbounded retransmissions; RTCP over UDP is taken only from the client’s address.
- Fixed RTCP that fails SRTCP authentication being acted on in SRTP sessions.
- Fixed NACK, PLI and FIR being taken for sources other than the video SSRC.
- Fixed the RTX payload type not following a live codec switch.
- Fixed MPEG-TS sessions answering NACKs with RTX.
- Fixed the RTX history holding less than the announced rtx-time.
- Fixed multicast presentations announcing NACK and RTX.
- Fixed the metadata track’s RTCP socket never being read.
- Fixed the HTTP tunnel’s POST leg being closed after 30 s.
- Fixed a refused request’s connection being reset instead of closed.
- Fixed TEARDOWN closing sockets the sender thread was still using.
- Fixed a video SETUP after a metadata-only one dropping the metadata track.
- Fixed a live MULTICAST_TTL not reaching the push leg.
- Fixed MServerTest’s KLV declaring 30 bytes for 29, so none of it was valid ST 0601 and its video carried no ST 0604 SEI.
- Fixed Sender Reports ignoring wall-clock steps after start-up, which put SR-mapped capture times seconds off the receiver’s clock.
- Fixed a wall-clock step reaching receivers only with the next Sender Report, up to 1 s later.
- Fixed WebRTC Sender Reports of every stream but the first being stamped from another stream’s clock.
- Fixed the push-leg pacer sending below its configured rate, which let the push latency grow to 2 s.
- Fixed MServerTest pacing its push leg at exactly the encoder bitrate.
- Fixed packets queued before a push-leg container change going out in the new framing.
- Fixed a re-typed or re-enabled push leg waiting for the next natural key frame.
- Fixed the key frame that reopens an MPEG-TS leg arriving without PAT and PMT.
- Fixed a live change of the served leg’s KLV signalling taking effect only at the next re-init.
- Fixed the MPEG-TS PCR being equal to the PTS (T-STD violation on every TS leg).
- Fixed ST 0601 tags above 127 breaking KLV validation and the ST 0604 SEI.
- Fixed the ST 0604 SEI claiming a locked clock (Time Status 0x1F, now 0x9F).
- Fixed two data races (metadata track flag, push-leg payload type at runtime enable).
- Fixed MServerTest’s JPEG + MPEG-TS refusal message and its direct mode printing no statistics.
- Fixed data races between the RTSP sender thread and a live codec switch (a playing session’s payload type), a repeated metadata SETUP (its destination) and PLAY (the RTP-Info sequence number).
- Fixed a NACK arriving while a packet was being sent retransmitting it ahead of the original: the retransmission history holds only packets already handed to the transport, so a NACK for a number not sent yet draws nothing.
- Added the RTCP BYE (RFC 3550 sec. 6.6): an RTSP session sends it on TEARDOWN, when a SETUP replaces it, when it expires or its connection closes and when the stream stops; a WebRTC session when it ends; a multicast group and the push leg when the stream stops; SRTCP on SRTP and WebRTC sessions.
- Fixed SETUP ignoring the order of the Transport alternatives (RFC 2326 sec. 12.39): the first one the server can serve is taken, and only it is named in the reply.
- Fixed MPEG-TS PCRs, and the PAT and PMT, more than 100 ms apart below 10 fps on the served and the push leg (ISO/IEC 13818-1 sec. 2.7.2).
- Fixed a push-queue overflow cutting frames in two: whole access units are dropped, the leg resumes at a key frame, and the drops are counted in directSendFailures.
- Fixed the SRTCP session keys not being cleansed when an SRTP context is destroyed.
- Documentation corrected.
2.1.2 03.10.2026 - Linux build fixed (Winsock types on POSIX, the CryptoProvider DRBG call); builds on Windows with MSVC and MinGW: listeners with SO_EXCLUSIVEADDRUSE, UDP sockets without SIO_UDP_CONNRESET, 1 ms system timer, WebRTC switched off with one warning.
- RTSP: replies queued per connection bounded (256 KiB), at most 160 connections, the listener pauses when the process runs out of descriptors, a slow interleaved viewer loses whole frames up to the next key frame instead of building seconds of latency, metadata-only multicast sessions, digest nonces retired when their replay table is full, PLAY key frames rate-limited (200 ms per session, 1 s per path).
- HTTP: half-closed connections no longer spin, 10 s write-progress deadline.
- HLS: segment names never repeat across restarts, private cache control, fixed target duration, EXT-X-DISCONTINUITY-SEQUENCE, the keys of every listed segment kept.
- WebRTC: DTLS cookie exchange (no amplification) and peer address binding, OpenSSL’s retransmission backoff, the first SHA-256 fingerprint of an offer used.
- Crypto: a FIPS-first process without the FIPS module falls back to the standard mode (no claim), one library context shared with CryptoProvider, keys and PEM copies wiped.
- Core: DIRECT_STREAM_ENABLE safe outside the lifecycle, data races on the published paths fixed, push queue refusals only for dependent pictures and at most one forced IDR per second, re-initialisation resets every per-session state, the SDP republished on a parameter set change, the RTP marker on the last packet of the access unit, MISB ST 1402 cell service id and sequence number, the last frame no longer lost on close(), parameter validation (TYPE, LOG_LEVEL, METADATA_MODE, NaN), refused values not stored, H.265 slice headers parsed fully, MJPEG sizes that RFC 2435 cannot describe refused, hls.js pinned with SRI.
- Security: the modes that require authentication (STRICT_FIPS, FIPS_ONVIF_BRIDGE, CRA_STRICT) refuse an empty password at initialisation too (setParam() always did); the linker hardening of libMServer (-z relro, -z now, -z noexecstack) is PUBLIC and reaches the executables (it was PRIVATE and reached none), MSVC builds with /GS /guard:cf; the compliance summary corrected (CRA Annex I numbering, open points of the FIPS and CRA verdicts, one shared OpenSSL context).
- Code style (Allman braces everywhere) and comments reviewed; harness: tests for every fix, ports and SIGPIPE robustness, optional GStreamer elements, the GStreamer interop also over H.265 and JPEG and decoding every picture (gst-libav) when installed, the fuzz targets compile again.
2.1.3 03.10.2026 Security hardening (EU CRA / FIPS 140-3 review follow-up).
- TLS and DTLS: the server contexts offer ECDHE + AEAD only and NIST groups only (no RSA key transport, no CBC), and a FIPS deployment drops a TLS/DTLS 1.2 handshake without the Extended Master Secret (IG D.Q).
- FIPS profiles (STRICT_FIPS, FIPS_ONVIF_BRIDGE) now REQUIRE the validated module and fail to initialise without it, instead of running with the claim withdrawn; CRA_STRICT is the hardened profile for a deployment without a module. Windows CNG does the same against the system FIPS policy.
- RTSP digest requires qop=auth in every mode (an RFC 2069 response without a nonce-count, replayable for the nonce lifetime, is refused); MD5 stays offered only in the ONVIF interop profile.
- CORS secure by default: no Access-Control-Allow-Origin unless corsAllowedOrigin is set (the former wildcard let any page reach an unauthenticated server); the built-in same-origin player pages are unaffected.
- Brute-force and flood resistance: a connection of the RTSP listeners is dropped after 5 wrong-credential responses (the HTTP listeners follow in 2.1.4), a remote source may hold at most three quarters of the RTSP connection slots (loopback exempt), and getStats() reports authFailures and connectionsRefused (EU CRA 2(l)).
- A failed DRBG draw now fails the request instead of leaving a nonce or session id empty.
- Build: ImageResizer and FormatConverter compile with -fstack-protector-strong / _FORTIFY_SOURCE (MSVC /GS /guard:cf) and no longer use -march=native (reproducible builds).
- The reference client library was renamed VSourceRtp -> VSourceStream in comments and docs.
- Annex I Part II material added: SECURITY.md, SECURITY-ADVISORIES.md and an SBOM (sbom.cdx.json).
2.1.4 09.10.2026 Security review (EU CRA / FIPS 140-3 follow-up). The public interface is unchanged. The behaviour changes – all of them for invalid or hostile input, none for a valid call – are marked “behaviour change”.
- Memory: a frame’s claimed geometry no longer sizes the pipeline’s buffers (a 16-byte unit labelled 8192 x 8192 made every stage allocate a quarter of a gigabyte); a compressed unit is copied by its payload size (behaviour change: sendFrame() answers FALSE for a RAW frame that carries less than one byte per pixel and for a frame whose geometry is not positive or does not fit its buffer, also while the stream is switched off or has no viewer, where it answered TRUE).
- Exceptions: std::bad_alloc is answered with FALSE by sendFrame(), setParam(), executeCommand() and initVStreamer(), which rolls back every thread and claim it started; the pipeline and push threads (whatever the caller’s codec or overlay throws included) and the WebRTC, DTLS and HTTP reactors drop the frame, datagram or connection that failed instead of ending the process. The RTSP reactor and sender threads have no such barrier: an exception there still ends the process.
- Behaviour change: initVStreamer() refuses an empty suffix (setParam(SUFFIX) already did); setParam(securityProfile) is refused while the instance is not running; closeVStreamer() lets go of the codec and the overlay, so a RESTART of the closed instance runs without them.
- Compliance profiles fail closed (behaviour change): setParam(DIRECT_STREAM_ENABLE) is refused under a profile that requires TLS, and a live switch into a stricter profile is refused while the push leg is on, while the stream is published on a weaker HLS listener, and while a session of the TLS listener plays media the new profile forbids (plain RTP on UDP, a multicast group, SRTP suites); the policy reaches the TLS listener before it accepts; a PLAY re-checks the transport against the policy in force and is answered 461 if it no longer complies; a metadata SETUP honours the SRTP and multicast rules of the video SETUP, so CRA_STRICT never sends KLV in the clear to a group.
- WebRTC: a session that has not completed DTLS 30 s after its POST is dropped (behaviour change); ICE consent is refreshed only by checks from the address the media goes to (behaviour change); at most one key frame request per stream per second reaches the encoder (behaviour change: a second viewer joining within a second of another one waits for its own request or the next GOP); DTLS issues no session tickets and caps a handshake message at 16 KiB (behaviour change); a refused duplicate suffix leaves the owner’s credentials, where it replaced them; a live suffix change moves the credentials with the name (before, the renamed stream answered offers without authentication); the last stream to leave stops the endpoint, also one that never came up, without unpublishing a stream claimed meanwhile.
- HTTP and HLS: the request head is validated (obs-fold, non-token field names, CR / LF / NUL, control bytes: no request smuggling; behaviour change: such a request is refused), nothing is answered after Connection: close (behaviour change), a connection is dropped after 5 wrong digest answers (behaviour change), one remote address holds at most 24 of the 32 connections (behaviour change; loopback exempt), an exception on the reactor ends one connection and fails closed; HLS keys are wiped, a failed start leaves no TLS context behind, a renamed stream is never served without its credentials, the exit no longer races the reactor, and the listener is released with its last stream (behaviour change: its port then stops accepting).
- RTSP: multicast group sockets are non-blocking (a slow link stalled the encoder and every client), a control stream is dispatched in linear time, receivers’ PLI / FIR are limited per path (behaviour change: at most one per 200 ms reaches the encoder), long parameter sets no longer cut the SDP fmtp line, the metadata SETUP parses client_port and interleaved strictly (behaviour change: what the old parser let through – a sign, text after the number, a second port above 65535 – is answered 461), the multicast pool prefix of rtspMulticastIp is parsed strictly too (behaviour change: /24x or an overflowing number is refused), a second tunnel GET leg for an open cookie is refused with 409 (behaviour change), SRTP keys and passwords are wiped.
- Parsers (behaviour change for the input described): an access unit of more than 262144 NAL units is dropped whole, a JPEG scan above 2^24 bytes (the RFC 2435 fragment offset) is refused, NAL types 24 to 31 (H.264) and 48 to 63 (H.265) are not sent, and VPS / SPS / PPS above 64 KiB are refused all or nothing; an Exp-Golomb shift is fixed.
- Lifetime: an MServer with static storage duration is closed before the process-wide singletons it uses are destroyed (they were destroyed first: use after destruction at exit); a listener whose thread cannot be started, or an allocation that fails while a listener or a path is set up, leaves no bound port, no descriptor and no claimed path behind, and initVStreamer() answers FALSE as documented (the shared RTSP listener, the HLS listener, the WebRTC media socket and the path tables roll back; a path whose description could not be assigned no longer refuses every later claim of its suffix); a BYE that cannot be built for want of memory is not sent, and the teardown of the stream, which runs in the destructor, completes.
- Cryptography hygiene: the generated TLS identity is exported only to regular files and never replaces a backup (.unloadable, .unloadable.1 to .99; behaviour change: a path naming a directory, a link, a device or a FIFO is left untouched and nothing is generated); SDES key copies and the MD5 state are wiped; every socket and pipe is close-on-exec.
- Tests: 223 new checks (run against the code before the fixes, the tests fail 71 checks and two programs abort with an uncaught exception), the fuzz targets of the HTTP, HLS, RTSP and SDP parsers now initialise the crypto context they need, the GStreamer script tolerates the rtspsrc shutdown race of GStreamer 1.20, strict compiler warnings fixed in the library, examples and harness (GCC 9 to 12, Clang 18).
- VStreamer 3.3.1: deserialize() is all or nothing and its size arithmetic is overflow-safe.

Library files 

CMakeLists.txt                top-level build; options and the FFmpeg probe
README.md                     this file
src/
  CMakeLists.txt              builds libMServer; OpenSSL is required on Linux
  MServer.h                   the ONLY public header (pimpl)
  MServer.cpp                 VStreamer implementation and frame pipeline
  MServerVersion.h.in         version template, filled in by CMake
  MServerVersion.h            generated from it; MSERVER_VERSION etc.
  impl/
    MServerBitstream.{h,cpp}  Annex-B parsing, parameter sets, slice_type
    MServerRtp.{h,cpp}        RTP packetizers, 90 kHz clock, RTCP
    MServerRtsp.{h,cpp}       RTSP server, SDP, tunnel, endpoint registry
    MServerCrypto.{h,cpp}     providers, digests, DRBG, TLS contexts and the
                              generated TLS identity (OpenSSL on Linux,
                              CryptoProvider's CNG backend on Windows)
    MServerSrtp.{h,cpp}       SRTP, SDES, MIKEY
    MServerTs.{h,cpp}         MPEG-TS muxer, MISB ST 0601 KLV
    MServerIce.{h,cpp}        STUN, ICE-lite agent
    MServerDtls.{h,cpp}       DTLS-SRTP handshake and key export
    MServerHttp.{h,cpp}       HTTP/1.1 and HTTPS signalling listener
    MServerWebRtc.{h,cpp}     WHEP endpoint, sessions, SRTP send loop
    MServerHls.{h,cpp}        HLS segmenter, playlists, AES-128, listener
    MServerPlayer.h           the built-in WebRTC and HLS player pages.
                              Header-only: it is string builders and no state
    MServerCompliance.{h,cpp} compliance-mode policy
    MServerPlatform.h         sockets on POSIX and on Winsock (Windows)
    MServerVStreamerCompat.h  MSVC fix for VStreamer 3.2.0's VStreamer.h,
                              force-included only when that header needs it
3rdparty/                     submodules: VStreamer (the interface MServer
                              implements), CryptoProvider (cryptographic primitives
                              provider), FormatConverter, ImageResizer
example/                      smallest working program
test/                         interactive test application for users
harness/                      full automated functional and performance suite
  README.md                   what each test proves and how to run it
  CMakeLists.txt              registers every test target with CTest
  *_test.cpp                  the suite itself, one binary per area
  live_server.cpp             the server the interop scripts drive
  ffmpeg_tests.sh             interop against ffmpeg / ffprobe
  gstreamer_tests.sh          interop against gst-launch-1.0
  fps_test.sh                 delivered frame rate across a sweep of rates
  check_no_ffmpeg.cmake       asserts libMServer references no FFmpeg symbol
  tsan.supp, lsan.supp        sanitiser suppressions (helper libraries,
                              OpenMP runtime, test codec)
  field/                      two-machine rig: hardware encoder, remote clients
  fuzz/                       libFuzzer targets for the untrusted parsers
static/                       images used by this README
test_streaming_windows.py     end-to-end check with ffmpeg on Windows

Consumers include MServer.h and nothing else. The impl/ headers are private: anything under impl/ may change between versions.

MServerVersion.h is generated from MServerVersion.h.in by configure_file() when CMake configures the project. Both files are tracked in git, so you can read the version without configuring first.

Internal architecture

Frame pipeline

The pipeline has two stages, each on its own thread: Stage A (the preprocess thread) and Stage B (the encoder thread). One-frame slots separate them from the caller and from each other, so sendFrame() never waits for the encoder or the network:

                                                        +-> packetizer -> RTSP / RTSPS / multicast / WebRTC subscribers
                                                        |
sendFrame() -> [slot] -> Stage A -> [slot] -> Stage B ---+-> MPEG-TS muxer -> direct push leg
                       convert           encode         |
                       resize            parse NAL      +-> HLS segmenter -> playlist and segments
                       overlay           cache SPS/PPS/VPS
                       -> NV12

MServer caches the parameter sets (SPS, PPS and, for H.265, VPS). The SDP’s sprop-* parameters are built from this cache, and so is the key-frame gate, which holds a new receiver back until a decodable key frame arrives. The cached sets are not inserted into the bitstream. A receiver without an SDP – an MPEG-TS or HLS player, or a push receiver – therefore starts decoding at the first key frame that carries its own parameter sets. An encoder that repeats them with every key frame gives it that; VCodecLibav’s H.264 and H.265 output does. Pre-encoded input is forwarded as it arrives. If such a source sent its parameter sets only once, these receivers have nothing to decode until the stream restarts.

One encode feeds every leg. A leg adds its own packetization, never a second pass through the codec. That is why turning RTSP and HLS on together does not double the CPU load. The RTSP, RTSPS and WebRTC clients share one packetization. The push leg reuses it when its container, KLV signalling and payload size match; otherwise it packetizes for itself, because it may carry bare MPEG-TS. HLS always muxes for itself, because it needs its PSI at the head of every segment rather than on a time cadence.

When scaling or the overlay is needed, the helper libraries fix the route. ImageResizer accepts only 3-byte interleaved data, and VOverlay draws on that data. So the chain is any RAW -> YUV24 -> resize -> overlay -> NV12 -> encoder. It runs on the preprocess thread; by default, conversion and scaling may use two threads (see Compute threads for preprocessing).

When neither scaling nor the overlay is needed, the pipeline skips that chain. If the stream size equals the input size and no overlay is drawn, only the encoder’s input format is left to satisfy. FormatConverter reaches NV12 from every RAW format in a single pass:

Input Stream size = input size, overlay off Otherwise
NV12 nothing at all – the frame goes to the encoder untouched RAW -> YUV24 -> resize -> overlay -> NV12
YUV24, RGB24, BGR24, YUYV, UYVY, GRAY, … one conversion, straight to NV12 RAW -> YUV24 -> resize -> overlay -> NV12
H.264 / H.265 / JPEG pass-through: no conversion, no scaling – re-encoding compressed input is out of scope same

A caller that already produces NV12 at the stream resolution therefore pays for no conversion, no scaling and no copy in preprocessing. Both helpers keep state and are not thread-safe, so the preprocess thread owns its own instance of each.

Holding the configured frame rate

The caller calls sendFrame() at the source’s rate, which may differ from the configured fps. Holding fps is the server’s job. The preprocess stage does it, between the input slot and the encoder. This part of the pipeline is called the rate governor:

Source What happens Counter
faster than fps (a 60 fps camera on a 30 fps stream) the frame waits in the input slot until its deadline; sendFrame() overwrites the slot with anything newer, so the deadline is met with the newest picture framesDropped counts the superseded ones
on rate nothing: a frame up to 1/8 of a period late is still used for its period, so two unsynchronised clocks do not cost a fifth of the frames –
slower than fps (25 fps on a 30 fps stream) the previous picture is repeated to fill the period framesDuplicated

The deadlines advance by whole periods, so over time the rate is exactly the configured one, whatever the source does. This matters: the encoder allocates bits per frame for fps. Without the governor, a 60 fps source on a 30 fps stream would produce twice the configured bitrate. That would overflow the subscriber queues and freeze every RTSP client for a second at a time.

In four cases the governor deliberately does not repeat a picture:

  • Nothing has arrived yet. There is nothing to repeat.
  • The source has been silent for two seconds. A frozen picture sent for as long as the service runs would hide the failure. So the stream stops instead, and resumes when the source does.
  • Nobody is watching. Repeating for no one would undo Idle streams cost nothing for every stream that ever had a client. A stream switched off with VStreamerCommand::OFF stops repeating at once, not after the two-second grace window.
  • The input is already compressed (pass-through). Re-sending an encoded P-frame makes the decoder apply the same residual twice. The result is a corrupt picture, not a still one, so a pass-through source keeps its own rate.

framesDuplicated is the counter worth watching. A stream whose source matches its fps never duplicates. A rising count means the source is slower than the rate the SDP advertises.

Shared-stream model

Many clients may be connected at once, and all of them receive the same stream. Each access unit (one encoded picture) is packetized exactly once. Every subscriber shares the resulting payload bytes; only the 12-byte RTP header differs. UDP unicast sends a two-element iovec (the subscriber’s own header and the shared payload), so the media bytes are never copied.

Sequence numbering and drop policy

The RTP sequence number is assigned per subscriber, when the packet is sent, not when it is packetized. This matters because each subscriber has its own bounded queue and may drop packets when it falls behind.

If the number were stored in the shared packet, a drop would leave a hole in that subscriber’s sequence. A conformant receiver reads a hole as packet loss, and its jitter buffer then waits out the reorder timeout on every drop. Here a packet that is never sent never uses a number. A subscriber that drops therefore sees a contiguous sequence with a jump in the RTP timestamp. To the receiver this looks exactly like a source that lowered its frame rate.

A subscriber always drops a whole access unit, and then waits for the next key frame. Cutting a unit half-way would let the receiver rebuild a corrupt NAL from the surviving fragments. If that case ever occurs, RtpSequencer detects it and deliberately leaves a gap instead. truncatedUnits() must always read zero.

Shared listening port

All instances of MServer in a process share one RTSP listener, owned by a process-wide registry. Requirements this satisfies:

  • a second instance joins the existing port rather than opening its own, even if it asked for another port (getParams() then reports the port that is really bound);
  • changing the port through any instance re-binds the shared listener, so every instance follows;
  • re-binding opens the new socket before it closes the old one, so a failed change leaves the previous listener serving;
  • instances are told apart by the URL path (the stream suffix); a duplicate suffix is refused;
  • the last instance to detach releases the port.

Of the two RTSP listeners, only the plaintext one is shared. Each instance’s RTSPS listener is a socket of its own. So rtspsPort is a per-instance value, and two instances must not name the same one: the second bind() fails, and so does that instance’s initVStreamer(). Give every TLS stream a port of its own. This is also why the TLS listener takes its credentials as a listener-wide default rather than per path: it serves exactly one stream.

Three listeners are process-wide, not one: the plaintext RTSP listener above, the WHEP signalling endpoint on webRtcPort, and the HLS listener on hlsPort. They follow the same rules:

  • streams are told apart by suffix;
  • a second instance that names a different bind address is refused, not merged;
  • the last instance to detach releases the socket;
  • credentials stay per path. Protecting one stream never challenges another stream’s viewers, and a strict instance that attaches beside a permissive one does not withdraw the permissive one’s published stream.

They differ on the port. A second instance that names a different webRtcPort or hlsPort is refused. A second instance that names a different rtspPort joins the RTSP port that is already bound.

What the listener owns, and what the stream owns

A shared listener does not make every parameter shared. The rule is simple: a value that describes the socket belongs to the listener, and a value that describes the stream belongs to the stream. The table tells an operator what moves when they change one stream:

Owned by the listener – one value for the whole process Owned by the stream – its own value, independent of its neighbours
rtspPort (re-binds for every instance), the WHEP signalling and media ports, hlsPort user / password, on every leg – the plaintext listener, RTSPS, WHEP and HLS alike
bindAddress, rtpPortMin / rtpPortMax – refused when they disagree with a running listener, rather than merged bitrateKbps and the send budget derived from it
the HTTP tunnel, and the compliance policy, which is the most restrictive combination of every attached stream and is lifted when a stream detaches rtspMulticastIp / rtspMulticastPort / multicastTtl
corsAllowedOrigin, which the shared signalling endpoint holds one of rtspsPort – the TLS listener is the stream’s own socket, so two streams cannot share a value
  hlsEncryption and the segment window and key generations behind it
  everything about the picture, the encoder, the metadata and the push leg

Two rules behind this table:

  • Credentials are per path, not per listener, and so is the requirement for them. The listener answers a challenge for the path the request names. So two streams on one port may demand different passwords, and a stream that wants none is served without a challenge next to a protected one. A credential change closes the sessions of that path only. The close is needed because a client that is already PLAYING sends no further request; without it, the client would keep receiving a stream whose password was just revoked. The requirement follows the same rule: a compliance profile that demands authentication demands it only for the stream that chose the profile. A strict instance that attaches to a shared listener therefore cannot refuse (403) the viewers of a stream that was deliberately published without credentials.
  • The send budget is per path. Each subscriber is paced against the ceiling of its own stream, so one stream’s pacing never slows another stream’s clients.

Threading model

Thread Count Responsibility
caller external sendFrame() copies into the input slot and returns
preprocess 1 per instance rate governor, convert, resize, overlay, NV12. Conversion and scaling may use extra OpenMP threads, two in all by default – see Compute threads for preprocessing
encoder 1 per instance encode, parse NAL, publish access unit
reactor 1 per listener poll() (WSAPoll() on Windows) over the listener, the control connections and the RTCP sockets of both tracks of UDP sessions. NACK, PLI and FIR are answered as they arrive, not on a timer. At most 16 datagrams are read per socket per round, so a flood on one port cannot starve the others. Also TLS handshakes and RTSP parsing. Closes every connection itself when the listener stops
sender 1 per listener drains subscriber queues, paces, sends RTP and RTCP
signalling 1 per process, only with WebRTC poll() over the WHEP listener, TLS handshakes, HTTP parsing
WebRTC media 1 per process, only with WebRTC one UDP socket: STUN checks, DTLS handshakes, SRTP out, RTCP in
HLS 1 per process, only with HLS poll() over the HLS listener: playlist and key requests answered inline, segments written straight out of the segmenter’s buffer. It never encodes – the segments are built on the encoder thread, which is why a slow HLS client cannot hold the pipeline back
direct push 1 per instance, whenever the push leg has a destination drains the push queue, applies directStreamBitrateKbps pacing, sends RTP/UDP and its Sender Reports. Separate on purpose: pacing means waiting, and waiting on the encoder thread would hold every RTSP, RTSPS and WebRTC client back to the push leg’s rate. Started even when directStreamEnable is FALSE: it waits on an empty queue, which costs nothing and lets the leg be switched on at runtime. While the leg is off it sends nothing at all, Sender Reports included

On Linux the reactor and the sender block SIGPIPE for their own threads. A TLS write to a viewer that has gone then returns EPIPE instead of killing the embedding application. MServer never changes the process-wide signal disposition. This is also why the reactor, not the thread that calls closeVStreamer(), closes the connections when a listener stops: the TLS close_notify sent to a peer that has vanished is such a write. Windows has no SIGPIPE, so nothing needs blocking there.

Media is never written from the encoder thread: a blocking write to one slow client would otherwise delay every other client and the control channel.

The HLS listener writes segments by reference. A segment goes to the reactor as a shared buffer, not as a copy in the output queue. So it is not duplicated per client, and it does not count against the queue cap, which is smaller than one segment. A client that is still downloading keeps its segment alive after the sliding window has dropped it. A second request on the same connection waits until the body has gone out: responses share one byte stream, and a half-written body must not have another answer’s headers queued behind it.

The sender thread never waits for a socket either. Every media socket gets a 4 MB send buffer. A datagram that the kernel still refuses is counted (rtpSendFailures, directSendFailures) and skipped. It is deliberately not retried: one sender thread serves every subscriber of a listener, so pausing for a congested socket would stall every healthy client behind it. The buffer size matters: a default-sized socket refuses the tail of a 1080p key frame with ENOBUFS. Those packets already carry sequence numbers, so the receiver reports them as lost.

Idle streams cost nothing

Conversion, scaling, overlay and encoding are the expensive part of this library. With nobody watching, all of that work is wasted. So MServer ignores input while a stream has no consumer, and takes it up again the moment it gains one. A consumer is anything that receives the stream; see What counts as a consumer.

The frame is dropped in sendFrame(), before it is even copied, so the whole chain behind it is skipped. sendFrame() still returns TRUE: the frame was accepted and handled, and the caller has nothing to decide.

Measured on a 640×480 H.264 stream at 25 fps: 8.5 % of a core with one viewer, 0.6 % with none. What remains is the caller handing frames over and a consumer check four times a second. On a real device the capture path costs extra, and MServer does not control that.

Nothing is asked of the calling code. Keep sending frames normally. getStats() reports what happened: framesIgnored counts the skipped frames, and idle says whether the stream is ignoring input right now. A watchdog that reads framesEncoded to decide whether the pipeline is alive must check idle too: a healthy stream that nobody watches encodes nothing.

framesIgnored is deliberately separate from framesDropped. framesDropped counts frames that a newer frame replaced in a hand-off slot. Usually the cause is a source faster than the configured fps, which is holding the rate at work; only sometimes is it an encoder that fell behind. Reading one counter for the other would make a healthy idle server look overloaded, or hide a rate mismatch behind an idle count.

What counts as a consumer

A consumer is wider than a client: two kinds of consumer have no client to count.

  Consumer while
RTSP / RTSPS a session is in PLAYING
WebRTC a session has completed DTLS and media is flowing — a page that fetched an SDP answer but never finished ICE is not watching
UDP multicast the group is transmitting. Members join without telling the server and there is no back channel, so once a group has been set up the stream stays awake for as long as it is assigned
Direct RTP push the leg is enabled. It has a destination rather than a client, and nothing on the far end reports back
HLS a playlist, a segment or a key was fetched within two target durations. HTTP is stateless, so this is “somebody is watching” rather than a session tally — and it is what keeps the encoder from parking between two playlist reloads

MServer counts consumers per path, not per listener. Instances on the same port share one listener. Counting the listener’s sessions would keep every stream in the process encoding because one of them has a viewer.

Two states keep a stream awake with no consumer at all. Each ends as soon as the stream has produced something, not after a timer:

  • until the stream is describable, because its SDP is built from encoder output;
  • until the HLS leg has one segment, when an HLS path is published and the codec is not JPEG.

Both are explained under What stays awake regardless.

Waking up

Every change of state – a PLAY, a WebRTC session becoming ready, an HLS request, a connection closing – increments a process-wide counter. sendFrame() compares that counter with the value it last acted on, and recounts consumers only when it has moved. In the steady state this costs one atomic load and one comparison. A 250 ms poll covers the states that nothing announces, such as a session removed by the idle timeout.

The first client gets a key frame at once. Waking asks the encoder for a key frame and keeps the publish gate shut until it arrives. The client therefore receives SPS, PPS and an IDR, not a GOP of inter frames it cannot decode. Measured on a 25-frame GOP at 25 fps, where waiting for the next natural key frame can take up to a second: PLAY to a complete IDR in 9–30 ms.

The same happens when a stream has never idled but is still in a start-up hold (no description yet, or no first HLS segment yet). The first real consumer after the hold asks the encoder for a key frame, just as a wake-up does. A client that retries DESCRIBE until the stream appears is exactly this case.

A client that joins a stream that is already running does not wait either. An RTSP or RTSPS PLAY that starts a session with a video track asks the encoder for a key frame, and so does every WebRTC session when its DTLS handshake completes. These requests are rate-limited: at most one per session every 200 ms and one per stream path every second, so a client that loops PLAY cannot turn every frame into an IDR. Pre-encoded input has no encoder to ask, so there a new client waits for the source’s next key frame.

Waking closes the HLS segment in flight. The gap lasts as long as nobody was watching. Appending across it would produce one segment with a PTS jump of any size, which a demuxer reports as a stream error. So the segment is closed, and the break is marked EXT-X-DISCONTINUITY in the playlist; the transport stream’s discontinuity_indicator says the same to the demuxer.

What stays awake regardless

A stream keeps encoding until its description exists, because the description is made of encoder output: the SDP’s sprop-parameter-sets come from the first encoded frames. Idling before that would answer every DESCRIBE with 404, and nothing could ever clear it. In practice this costs a handful of frames at start-up. A runtime codec or resolution change puts the stream back in this state until the new parameter sets arrive.

MPEG-TS is the exception the other way round. The codec is signalled in the PMT, not in the SDP, so there is nothing to wait for: such a stream idles from the very first frame, having encoded nothing.

A published HLS path adds one more bounded hold, for the same reason. The playlist has nothing to serve until a segment exists, and an empty playlist is worse than a 404: a conformant player reloads and eventually starts, but ffmpeg’s demuxer reports Empty segment and gives up at once. So the stream encodes until it has closed its first segment, whether or not anybody is watching, and then idles like any other stream. The window keeps that segment. A stream that idles and is fetched again therefore hands the player media at once, while the pipeline restarts underneath. JPEG is excluded: it produces no segments by design, so it would otherwise never idle.

DESCRIBE, WHEP and the HLS playlist are answered normally while a stream is idle. The description and the segment window are both cached, so signalling never waits for the pipeline.

Compute threads for preprocessing

Format conversion and scaling are the only work MServer can spread over several threads. custom1 sets how many threads they may use. MServer calls the overlay on the preprocess thread and gives it no thread limit.

params.custom1 = 4;   // the converter and the scaler may take up to 4 threads
Value Meaning
0 the default (the interface’s own default for the field): up to two threads, min(2, hardware threads). A caller who never touched custom1 gets two threads, never the whole machine
1 one thread. Conversion and scaling stay on the preprocess thread
n up to n threads, capped at what the machine has
above the hardware thread count refused by setParam(), because a caller asking for more cores than exist has misconfigured something, and clamping would hide it. Clamped by initVStreamer(), which does not range-check this field. A large value passed at initialisation therefore does not fail: it quietly gets a worker team as wide as the machine

The bound is std::thread::hardware_concurrency(). setParam(CUSTOM1, …) accepts [0 : bound]. initVStreamer() applies whatever the structure holds, clamped to the bound. A refused value leaves the stored one unchanged.

The threads come from OpenMP inside FormatConverter and ImageResizer. If those libraries were built without OpenMP, they run on one thread whatever custom1 says.

A warning for callers that reuse custom1 for something else. Before VStreamer 3.2, custom1 was a deprecated alias for other fields. An application that still copies a value into it – a payload size, for instance – is not passing an opaque number. It is asking for that many threads, and initVStreamer() clamps rather than complains. If custom1 carries application data in your code, set the thread count last, just before initVStreamer().

It changes live. The preprocess thread picks up the new limit on its next frame, so a deployment can trade latency against cores without restarting the stream. A lower limit also takes effect on the next frame. The worker threads themselves stay parked until the stream closes; that is OpenMP’s pool, not MServer’s, and a parked thread costs only its stack.

Why the preprocess thread applies it. OpenMP keeps a worker team per thread, and its thread-count setting applies to the calling thread only. Setting the limit from whichever thread called setParam() would start a team that the conversions never use, leave a second team to be built on first use, and race with the preprocess thread reading the value. So setParam() only records the request, and the thread that owns the team applies it.

What to set it to. Keep the default unless measurement says otherwise; set 1 if you want strictly one thread. The pipeline is already a chain of threads (source, preprocessing, encoding, sending), so a stream that meets its frame rate gains nothing from more. More threads pay off on large frames that are scaled: 4K input scaled to 1080p, several streams on one machine, or a CPU too slow to convert a frame within its frame period. To see the effect, read cycleTimeUs with getParams(): it is the measured interval between encoded frames.

Transports

Transport SETUP form Notes
RTP/UDP unicast RTP/AVP;unicast;client_port=n-n+1 server port pair is even/odd per RFC 3550 sec. 11
RTP/RTSP/TCP interleaved RTP/AVP/TCP;interleaved=0-1 $-framed; replies and media share one output queue per connection, so a reply waits behind media already queued. An access unit is queued only when it fits whole into about 1 s of media (256 KiB - 2 MiB); otherwise the queue is dropped up to the next key frame and a key frame is requested, so a slow viewer loses whole frames, never parts of one, and its latency stays bounded. On Linux the kernel keeps at most 64 KiB unsent per connection (TCP_NOTSENT_LOWAT). Replies are never dropped
RTP/UDP multicast RTP/AVP;multicast one sequence space for the group, TTL bounded; no feedback and no RTX – nothing reads a group’s RTCP, so the multicast presentation announces none
RTSPS TLS on its own port the control channel is TLS. Media is protected when it is interleaved inside TLS, or sent as SRTP (see the secure presentation). Not available on Windows
RTSP over HTTP/HTTPS GET+POST tied by x-sessioncookie Apple/QuickTime de-facto tunnel; the POST body is one base64 stream – a request may be split anywhere, and each may be padded on its own. The POST leg owns no session and lives as long as its GET leg has one; without one it is closed after the 30 s sessionless timeout like any other connection
Direct RTP push not RTSP at all UDP only, fixed destination list, unicast or multicast; RTP or bare MPEG-TS, see stream type values. Container, payload size and bitrate are independent of the RTSP/RTSPS/WebRTC legs, and the leg transmits on its own thread so its pacing cannot hold the other transports back
WebRTC not RTSP at all WHEP over HTTP or HTTPS, then DTLS-SRTP on one UDP port; see WebRTC
HLS not RTSP at all a player fetches a playlist and then the segments it names, over HTTP or HTTPS; see HLS

A SETUP may list several transports, separated by commas, in the client’s order of preference (RFC 2326 sec. 12.39, RFC 7826 sec. 18.54). For example: Transport: RTP/AVP;unicast;client_port=5000-5001,RTP/AVP/TCP;unicast;interleaved=0-1. MServer takes the first one it can serve, names only that one in its reply, and answers 461 when it can serve none. An alternative can be served when all of these hold:

  • its profile is RTP/AVP or RTP/SAVP, over UDP or TCP;
  • SRTP (RTP/SAVP) is asked for only on a connection that was described SRTP over TLS, and never with multicast;
  • the compliance mode’s transport policy allows it;
  • UDP unicast names a client_port;
  • multicast is asked for only on a stream that has a multicast configuration;
  • on an existing session, it uses the transport family of the track that the session already holds.

Up to 2.1.0 MServer searched the header for words instead. It served the example above over TCP. It also refused an RTP/SAVP alternative listed before an RTP/AVP one on a connection not described SRTP, instead of falling back to RTP/AVP.

What each transport carries, and to which standard

Every delivery path and the specification it follows, so a receiver can be matched to a mode without reading the source:

Path Container / framing Standard Metadata
RTSP / RTSPS, rtp codec-specific RTP H.264 RFC 6184, H.265 RFC 7798, MJPEG RFC 2435 none
RTSP / RTSPS, rtp-klv* codec-specific RTP + a muxed metadata track as above, plus KLV over RTP RFC 6597 m=application PT 98, smpte336m/90000 or vnd.onvif.metadata/90000
RTSP / RTSPS, mpegts-rtp-klv* MPEG-TS inside RTP RFC 2250 PT 33, MISB ST 0804 KLV on its own PID
Direct push, rtp codec-specific RTP over UDP RFC 6184 / 7798 / 2435 none
Direct push, rtp-klv* codec-specific RTP over UDP + its own metadata track RFC 6184 / 7798 / 2435, plus KLV over RTP RFC 6597 PT 98 with its own SSRC and sequence space, muxed onto the same destination port and told apart by payload type. Selected by directStreamType independently of the served leg; there is no SDP on this leg, so the receiver has to be told out of band
Direct push, mpegts-klv-* MPEG-TS straight into UDP, 7x188 per datagram STANAG 4609, MISB ST 1402 KLV PID, async or sync
Direct push, mpegts-rtp-klv-* MPEG-TS inside RTP RFC 2250 PT 33, MISB ST 0804 KLV PID, async or sync
WebRTC codec-specific RTP inside SRTP RFC 5764 DTLS-SRTP, RFC 8445 ICE-lite, WHEP none
HLS MPEG-TS segments over HTTP or HTTPS, optionally AES-128-CBC RFC 8216 v3 KLV on its own PID inside the segments, but not signalled in the playlist – a player reads the video and ignores it

Asynchronous vs synchronous KLV (the -klv-async / -klv-sync suffixes). Asynchronous carriage is PES private_data with stream type 0x06 and a KLVA registration descriptor: raw KLV, no wrapper. Synchronous carriage is stream type 0x15 with a metadata_descriptor: the KLV is wrapped in a metadata_AU_cell and aligned to the PTS. MISB ST 0604 requires the synchronous form, and a STANAG 4609 receiver expects it when the metadata must match a frame. mpegts-rtp-klv selects the synchronous form.

How a client learns the metadata format. On the MPEG-TS paths it is in the PMT. On the codec-RTP paths it is in the SDP, which is the only thing an ONVIF client can read:

m=video 0 RTP/AVP 96
a=rtpmap:96 H264/90000
a=control:rtsp://host:8554/live/trackID=0
m=application 0 RTP/AVP 98
a=rtpmap:98 SMPTE336M/90000
a=control:rtsp://host:8554/live/trackID=1

The second media section appears whenever the stream carries metadata on a codec-RTP container. It is a real second track. It has its own control URL, and the client SETUPs it separately. The KLV then arrives on its own transport, with its own SSRC and sequence space. A client that wants only video sets up trackID=0 and never receives metadata packets it did not negotiate. metadataSuffix chooses the encoding name: SMPTE336M for MISB ST 0601 KLV, VND.ONVIF.METADATA for an ONVIF metadata stream.

The a=control line matters. A media section without one is aggregate-controlled, and clients such as FFmpeg and GStreamer would then not set up the metadata track on its own. With the line, they do.

SETUP of trackID=1 opens a session of its own when none exists, and joins the video session when one does. RFC 2326 s10.4 lets a client set up any single media stream of a presentation, and an ONVIF metadata client does exactly that – it never touches the video track.

A metadata-only session carries no video transport, so:

  • the video fan-out skips it – it is never sent packets it did not negotiate;
  • the key-frame gate does not apply to it: that gate exists so a client is not shown pictures it cannot decode, and there are no pictures here;
  • RTP-Info names trackID=1, the track the session actually plays, and the sequence number comes from the metadata space;
  • interleaved channels 0-1 are available to it, because no video track has claimed them;
  • its metadata sequencer is configured here, with its own SSRC, its own initial sequence number and payload type 98;
  • the client’s address (clientAddr) is recorded here too, so UDP-unicast metadata packets go to the client;
  • SRTP gets its own context. The metadata track is a second RTP source and so a second cryptographic context (RFC 3711 s3.2.1), keyed from the same master material. The Transport reply names RTP/SAVP when it did select SRTP, as RFC 2326 s12.39 requires;
  • its RTCP is read. The metadata pair’s RTCP socket is polled like the video one, and a Receiver Report arriving there from the client’s address keeps the session alive. Feedback on it is a keep-alive and nothing more: the metadata track has no key frames and no retransmission history;
  • multicast works for it too. A metadata-only multicast SETUP activates the group itself. The group then sends video only once a member has set the video track up, and its metadata does not wait for a video key frame;
  • a video SETUP that follows adds the video track to this session rather than replacing it, so the metadata transport the client has just set up keeps working. It has to use the same transport family (UDP or interleaved) and the same profile (AVP or SAVP) as the metadata track, and interleaved channels the metadata track does not use; otherwise it is refused 461.

The transport policy is the same as for a video SETUP. The STRICT_FIPS, FIPS_ONVIF_BRIDGE and CRA_STRICT profiles require interleaved or encrypted media, so a metadata-only SETUP on plain UDP is refused 461 there, exactly as a video one is. The metadata track is not a way around the profile.

A SETUP of trackID=1 on a stream whose SDP advertises no metadata track answers 404 Not Found: it names a stream that does not exist.

When the session already has a video track, the metadata track follows its transport family:

Video transport Metadata transport Where it arrives
interleaved over TCP/TLS interleaved its own channel pair, default 2-3
UDP unicast UDP unicast its own server/client port pair
UDP multicast UDP multicast the same group, on port + 2

Multicast is the one case where the transport is not per client. A group is one datagram shared by every member, so the metadata stream belongs to the group. It goes out once, with its own SSRC, on the port pair above the video one. Each path holds a group address of its own, so nothing else is listening there. The multicast SDP names that real port, exactly as it names the real group.

Units larger than one packet are fragmented (RFC 6597 section 4.2), with the marker bit on the last packet of a unit. MServer does not control the size of the caller’s metadata. Sending an oversized unit as one RTP packet would build a datagram larger than the path MTU: IP fragmentation on a good link, silent loss on a real one.

Over SRTP the metadata track has its own cryptographic context. RFC 3711 section 3.2.1 scopes a context to one SSRC: the IV is built from the SSRC, and the rollover counter tracks that SSRC’s sequence space. Both contexts are keyed from the same master key; the SRTP KDF does not take the SSRC, so they share the same session keys. Sharing the video context would encrypt the metadata under the wrong SSRC, and the failure would be silent: the authentication tag still verifies, so the packets look authentic but decrypt to noise.

Which presentation DESCRIBE returns

The SDP describes what the client asked for, decided per request:

Request URL c= line For
rtsp://host/live c=IN IP4 0.0.0.0, m=video 0 any client; SETUP then negotiates unicast, interleaved or multicast
rtsp://host/live?multicast c=IN IP4 <group>/<ttl>, real group port ONVIF multicast streaming
rtsps://host/live?srtp m=video 0 RTP/SAVP, plus the RFC 4568 a=crypto key SRTP – see below

A multicast presentation names the metadata port too, because that one is real as well: m=application <group port + 2>.

Why decide per request? Advertising the group always would make the presentation multicast-only, and GStreamer’s rtspsrc correctly rejects that for a unicast request (“no protocols left”). Never advertising it breaks ONVIF, which requires the real group address in a multicast DESCRIBE. Deciding per request satisfies both. It also mirrors how a host’s ONVIF media service hands out a separate stream URI for multicast. ;multicast and ?mcast are accepted too, because clients differ in which form they keep.

The marker only selects the presentation. It is not part of the control URLs: a=control, Content-Base and RTP-Info name the stream without it (rtsp://host/live/trackID=1, not rtsp://host/live;multicast/trackID=1). A client that sets the tracks up with the URLs from the SDP therefore lands on the track it named.

The secure presentation, and why SRTP needs one

SRTP keys travel in the SDP. a=crypto (RFC 4568) and a=key-mgmt (RFC 4567) are SDP attributes, not RTSP header fields, so the description is the only place a client finds them. MServer therefore creates the master key and salt while it builds the SDP, remembers them for that connection, and keys the session that follows with exactly the bytes the client was given:

DESCRIBE rtsps://host/live?srtp   ->  m=video 0 RTP/SAVP 96
                                      a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:<key+salt>
SETUP    ... Transport: RTP/SAVP  ->  Transport: RTP/SAVP;unicast;...

It is a per-request presentation for the same reason multicast is. A client that cannot do SRTP cannot play an SDP whose media line says RTP/SAVP. So a plain DESCRIBE keeps describing RTP/AVP and carries no keys. This matches the separate stream URI that an ONVIF media service hands out for SecureRTSPStreaming; MServer has no SOAP layer, so the host’s service publishes this URL form instead. ;srtp and ?savp are accepted too, and, as with multicast, the marker stays out of the control URLs.

Three consequences worth knowing:

  • SETUP with RTP/SAVP is refused 461 unless that connection was described SRTP. Without the secure DESCRIBE the client holds no key. A session that encrypts media the client cannot read is worse than a refusal, because it looks established for as long as it runs.
  • The secure presentation requires TLS. ONVIF forbids returning RTP/SAVP or MIKEY on an unprotected control connection – keys readable by anyone on the path are worse than no encryption, because they look like some. Over plain rtsp:// the request is answered 461.
  • MIKEY accompanies the SDES attribute only when the mode advertises it, which no mode does by default: the message is structurally complete but has never been checked against a reference peer.

Multicast addressing

Multicast is the one transport where the server, not the client, decides the address – so the rules are worth stating in full.

Configuring the group

rtspMulticastIp accepts two forms, and they behave differently on purpose: some clients take whatever group the server names, others are provisioned with one fixed address and accept nothing else.

The configuration is per stream, not per listener: the pool, the base port and the TTL belong to the stream that set them, so one stream’s reconfiguration can neither move another’s group nor change the scope its datagrams leave with.

rtspMulticastIp Behaviour
239.1.0.0/16 (a CIDR range) a pool: each stream path is allocated its own group from the range, all on that stream’s rtspMulticastPort, so several streams never collide on one address
239.1.1.1 (a bare address) a fixed group: every stream uses that exact address, and successive streams take successive port ranges on it instead

Details that decide whether a configuration does what you meant:

  • The base address must be multicast (224.0.0.0/4). A unicast address is refused: setParam() returns FALSE and the previous value stays. (At initVStreamer(), such a value leaves the stream without multicast.) A server that sends RTP to whatever address it was given would be an amplification reflector built from a typo.
  • The prefix must be between /4 and /31; /32 is a single address and therefore a fixed group. Anything else (/1, /33, 239.1.0.0/abc) is refused rather than silently degraded.
  • rtspMulticastPort must be even: RTCP for the group uses port+1 (RFC 3550 sec. 11), and ONVIF restates this for multicast configuration. An odd value is refused.
  • A group reserves a range of ports, not one: port and port+1 for video RTP and RTCP, plus port+2 and port+3 when the path advertises a metadata track. Reservation tests for overlap over that range. Two streams whose base ports differ by less than their span therefore still collide, and the second is offered a different address (or, for a fixed group, the next free range).
  • The TTL comes from multicastTtl, where 0 means 1. It is clamped to 32, not reset: asking for 64 gives 32. The push leg applies the same cap, both when the leg is built (at initVStreamer(), or when it is first enabled at runtime) and on every live MULTICAST_TTL change. Each group has its own socket. IP_MULTICAST_TTL, IP_MULTICAST_LOOP and IP_MULTICAST_IF are socket-wide options, so with one shared socket every group would leave with the TTL of whichever stream set it last.
  • A transmitting group follows a live codec change. Re-publishing the description re-stamps an active group’s payload type and closes its key-frame gate, so the first thing it sends under the new codec can be decoded on its own. Only the payload type changes. The SSRC and the sequence space continue: to the receiver this is one uninterrupted RTP source, and a restart would read as mass packet loss. Sessions that are already playing are re-stamped the same way. A metadata-only session is skipped, because its payload type is 98 whatever the video codec is. Retransmissions follow too: the RTX payload type is derived from the payload type each packet was sent with. After H.264 -> H.265, a NACKed packet therefore comes back as payload type 100 (apt=97), the mapping a new DESCRIBE announces.
  • rtspMulticastIp, rtspMulticastPort and multicastTtl apply immediately through setParam(). A path that is already transmitting keeps its address; only its TTL follows the change. An assignment that nobody has set up yet is dropped, so the next DESCRIBE derives it again. A live group is never moved, because a group has no back channel to tell its receivers. multicastTtl reaches the push leg’s socket at once too.
  • No feedback for a group. Nothing reads a group’s RTCP port, so the multicast presentation announces no a=rtcp-fb and no RTX payload type. (RFC 4588 sec. 5.3 forbids the SSRC-multiplexed RTX of that media line in a multicast session anyway.) So a member never sends a NACK and waits for a repair that cannot come. A client that DESCRIBEs the plain presentation and then sets up a multicast transport has been told about feedback it will not get. Ask for the multicast presentation instead (?multicast / ;multicast).

How an address is chosen, and when

With a pool, addresses are allocated in order of first use, not by path name. The first path to be described or set up for multicast takes the base address. The next path moves to base+1 only if its port range would overlap one already handed out on the base address. Two streams that share a pool but use different rtspMulticastPort values therefore both land on the base address: a stream reserves an address and a port range, not an address alone.

With a fixed group, the address never changes. Instead, the base port moves on by the range each path needs: two ports, or four when the path advertises a metadata track.

The allocation happens on the first multicast DESCRIBE or SETUP for a path. It then stays the same for the life of the stream: the address in the SDP is the address SETUP confirms and the address the media arrives on. A client that reconnects lands on the same group. The group is released when the stream goes away (closeVStreamer(), or a suffix change, which makes a new stream on a new path), and the address returns to the pool.

When the pool has no free address left (or, for a fixed group, no free range), SETUP is refused with 461 Unsupported Transport. Handing out an address that already carries another stream would put two RTP streams in one sequence space and break both.

A client asking for a particular group

A client may name the group it wants in SETUP with Transport: ...;destination=. It is honoured only when all three hold:

  • it falls inside the configured pool (or equals the configured fixed group) – another network’s group, or a unicast address, is refused, and refusing is an EU CRA Annex I 2(i) requirement rather than caution;
  • no other stream already holds it, since two streams on one address and port share a sequence space and break each other;
  • the stream is not already transmitting, unless the request names the address it is already on. A live group cannot be moved: its members have joined an address and there is no back channel with which to tell them otherwise.

Anything else is refused with 461 Unsupported Transport and no session is created. A recorder provisioned with one fixed address therefore asks for it on its first SETUP, which is what such clients do.

What is on the wire

  • One stream per group. Each path owns its own group, its own SSRC, its own sequence space and its own payload type. Two streams multicasting at once are two independent RTP streams; sharing a counter between them would make each of them look like the other’s packet loss.
  • A group nobody asked for is silent. A path is transmitted only after a client has set it up for multicast – configuring a pool does not put every stream on the wire.
  • Transmission is a property of the stream, not of a session. Once a group is live, it carries media for as long as the stream runs, however many members are joined. There is no per-member state to start or stop it, and a TEARDOWN by one client cannot silence a group that others may be watching.
  • No per-member queueing or dropping. One datagram serves every member, so a slow receiver cannot be served a reduced stream – it simply loses packets, which is what multicast is.
  • The group waits for a key frame before its first packet, so a receiver that joins at the start can decode. A member that joins later through RTSP PLAY makes the encoder send a key frame at once (see Waking up). A receiver that joins the group without RTSP waits for the next key frame.
  • RTCP Sender Reports go to port+1 of the group once a second, and at once after a step of the wall clock (see Sender Reports and the wall clock). Members have no back channel, so the SR is their only source of the RTP-to-wall-clock mapping (RFC 3550 sec. 6.4.1).

Which SDP a client is given

DESCRIBE returns a multicast presentation only when the request asks for one (?multicast); see Which presentation DESCRIBE returns. A multicast DESCRIBE allocates the group, so the c= line names the real address with its TTL suffix and m=video carries the real even port.

Scope and limits

  • Multicast is configured on both listeners, so an RTSPS-only deployment has it too. The two listeners keep separate allocations, so a path can sit on different addresses depending on which listener a client came through.
  • The multicast configuration is per stream even though the listener is shared, so two streams may use different pools with different scopes. The registry keeps its own copy of every stream’s policy and re-applies it when it creates the listener. A stream configured before any listener existed, or one that outlives a port re-bind, therefore keeps its policy.
  • A suffix change moves the stream to a new path. The old path’s group is released: its socket closes and the address returns to the pool. Members that already joined cannot be told it moved. The policy is registered again under the new name, so the next multicast DESCRIBE derives a fresh group from the same pool, with the same port and TTL.
  • A multicast session counts against the 64-session limit and is subject to the same 60-second inactivity timeout as any other, though its expiry does not stop the group.
  • No SRTP over multicast. SRTP keys are per session; a group is one datagram shared by every member, so there is no key that could protect it. A SETUP for RTP/SAVP;multicast is refused with 461 Unsupported Transport rather than handing out keys and then sending the group in the clear.
  • When bindAddress is set, multicast leaves through that interface only (IP_MULTICAST_IF), so a group cannot leave through an interface the operator excluded. With bindAddress left at 0.0.0.0, the kernel’s multicast route chooses the interface. On a multi-homed host that is not necessarily the interface the receivers are on: the group leaves through one interface, a client on another sees nothing, and SETUP and PLAY still both answer 200. A host that serves multicast should name its address rather than rely on the default route.
  • IPv4 only.

The direct push leg

RTSP, RTSPS and WebRTC all serve clients that connect. The push leg does the opposite: it sends to an address that was configured, with no session, no negotiation and no back channel. It is a stream in its own right, parallel to the served ones, and everything about it is its own.

Parameter Effect When
directStreamEnable transmits or does not immediately, both ways; switching it on is refused under a profile that requires TLS
directStreamIp one address, or a comma-separated list to push the same stream to several receivers immediately
directStreamPort destination port; RTCP for the leg uses port+1 immediately
directStreamType container and KLV signalling – see Which leg carries which container immediately
directStreamPacingMode 0 spends a token bucket at directStreamBitrateKbps; 1 sends on the frame boundary and lets the kernel’s send buffer provide the back-pressure immediately
directStreamBitrateKbps the rate mode 0 paces toward: the channel rate, which has to be above the stream’s own – see below immediately
directStreamMaxPayloadSize fragmentation size for this leg, [576:9000] next initVStreamer(), a live directStreamType change, or a runtime enable that builds the leg

Five things are worth knowing before you configure it:

  • Its container is independent of the served leg. A STANAG 4609 receiver can get raw MPEG-TS while RTSP clients get codec RTP. When the two legs differ, the push leg packetizes and muxes for itself, with its own payload type, sequence space and SSRC.
  • A container change applies at once, with a clean start. The PSI tables restart. Packets still queued for the old framing are dropped: sent after the change, they would go out behind the new framing’s header, or bare, and no receiver could parse them. The encoder is asked for a key frame, and the leg waits for it, because a TS demuxer cannot be switched mid-stream. On a transport stream that key frame always comes with the PAT and PMT (they go out before every key frame as well as every 100 ms), so a receiver that needs the PMT first does not drop it. With MServer encoding, the leg is back within a frame or two, not at the next natural key frame. Pre-encoded input still waits for its own key frame. Switching the leg off drops its queue too, and switching it on again gives the same clean start. The served leg cannot do this; see Re-initialising instead of refusing.
  • The pacer holds the channel rate, so the channel has to be faster than the stream. Mode 0 sends at most directStreamBitrateKbps on average, with a quarter-second burst allowed. Every packet the stream produces above that rate waits in the leg’s queue. If the channel rate equals the encoder’s rate, the queue grows with every burst and never drains. The latency then climbs until the queue is full (about 2048 packets, roughly 2 s at 10 Mbit/s), and the oldest access units are dropped. They are dropped whole: never the rest of the unit being sent, never part of one. After a dropped picture, every unit up to the next key frame goes too, because those pictures reference it. The leg then asks the encoder for a key frame and resumes there, at most once per second (never for pre-encoded input, which has no encoder to ask), so a congested channel does not turn every frame into an IDR. A key frame larger than the whole queue still goes out, once only the rest of the unit being sent is queued. Dropped units are counted in directSendFailures. Give the leg headroom (MServerTest uses twice the encoder bitrate), or use mode 1 on a link that needs no pacing.
  • It is built even when it is off. The socket, the sequencers and the sender thread exist whenever the leg has a usable address and port. So directStreamEnable is a real on/off switch that needs no restart. A leg that never had an address can still be enabled at runtime; the socket is then created on demand. While the leg is off it sends nothing, Sender Reports included. Under STRICT_FIPS, FIPS_ONVIF_BRIDGE and CRA_STRICT it cannot be switched on at all, whether it was built or not: it carries RTP in the clear (see What each mode changes).
  • It follows a live codec change, packetizer included. When the leg packetizes for itself, a CODEC change reconfigures its packetizer as well as the served leg’s. The leg takes its new payload type from it and waits for the next key frame. The payload type is the only signal the receiver gets: there is no SDP on this leg. Getting it wrong would be silent and total: the served leg would announce RTP/AVP 26 for JPEG while this leg kept sending H.264 FU-A fragments under payload type 96, and the receiver could not recover. ONVIF’s START AND STOP MULTICAST STREAMING – JPEG runs exactly that sequence: SetVideoEncoderConfiguration switches the encoder, then the group is joined with no RTSP in between. Only this leg’s container decides whether the re-stamp applies. A served leg in MPEG-TS with an RTP push leg is exactly the case where the push leg packetizes for itself, so it must not be skipped.

The leg has two limits. First, a push destination counts as a consumer, so a leg pointed at an address nobody reads keeps the stream awake and encoding (What counts as a consumer). Second, there is no retransmission: a datagram the kernel refuses is loss the receiver cannot repair. directSendFailures counts these datagrams together with the packets the queue dropped.

The leg also ends cleanly. When the stream stops (closeVStreamer(), a restart), a leg that is on and sends RTP sends a BYE to port+1 (RFC 3550 sec. 6.6) for its video SSRC, and for its metadata track’s SSRC if it carried one. Switching the leg off is a pause, not a departure: the same SSRC continues when it is switched on again.

Binding and port ranges

Two configurations for a firewalled or multi-homed deployment:

bindAddress confines every listener MServer opens – RTSP, RTSPS, the WHEP signalling port, the WebRTC media socket and the HLS listener – to one local address. A value that cannot be honoured (a hostname, an unparsable literal, an address that is not local) fails initialisation. Falling back to every interface would put the stream exactly where the operator excluded it.

It also moves the ICE candidate. With a bind address set and no explicit publicAddress, the candidate names the bound address, not whatever the routing table would offer. Otherwise the browser would get a candidate for an interface the media socket does not listen on, and ICE would never complete.

rtpPortMin / rtpPortMax put the server’s UDP-unicast RTP/RTCP pairs in a known range, so a firewall rule can cover them. The RTP port stays even and RTCP takes port+1 (RFC 3550 sec. 11). Allocation starts from a rotating cursor, so consecutive clients do not compete for the same numbers. When the range is exhausted, the SETUP fails. Falling back to an ephemeral port would give the client a port the firewall does not admit, and to the operator that looks like a stream that sometimes works.

The range covers only ports MServer allocates: multicast and the WebRTC media port are configured explicitly and are not drawn from it.

Both are properties of a shared listener, so an instance that sets one constrains every instance on that port. A conflicting value is refused, not merged. Widening a bind address would expose a stream that asked to be confined. Narrowing it would withdraw a stream that is already published. And a port range that covers only part of a port’s traffic is not something a firewall rule can be written for.

WebRTC

WebRTC puts the stream in a browser with no plugin and no player. A viewer opens a page, the page makes one HTTP request, and video appears. Below the handshake, WebRTC uses the machinery MServer already has: the same packetizers, the same packetize-once fan-out, the same SRTP. A WebRTC viewer therefore costs one encryption per packet and no extra packetization. WebRTC needs DTLS, so it is not available on Windows.

What a viewer connects to

Two ports, both fixed by configuration:

Port Protocol Carries
webRtcPort TCP, HTTP/1.1 or TLS WHEP signalling: the SDP offer in, the answer back
webRtcMediaPort, or webRtcPort + 1 when it is 0 UDP everything else – STUN, DTLS, SRTP and SRTCP on the one port

The media port defaults to signalling + 1. Set webRtcMediaPort when a firewall was opened for a specific port. The two ports must differ, or initialisation fails. Both are shared by every MServer instance in the process, as the RTSP listener is. Instances are told apart by the URL path, which is suffix.

The signalling endpoints are:

Request Meaning
POST /<suffix>/whep offer in the body, 201 with the answer and a Location header
PATCH /<suffix>/whep/<id> trickle-ICE candidates as an SDP fragment (RFC 9725 sec. 4.6). Acknowledged 204 and discarded – an ice-lite agent never checks a remote candidate, but answering 405 makes the reference WHEP reader abandon a healthy session. An unknown session id gets 404, as DELETE does
DELETE /<suffix>/whep/<id> end that session
OPTIONS /<suffix>/whep CORS pre-flight and method discovery

The handshake, step by step

  1. The browser POSTs an SDP offer. MServer parses it, picks a payload type that the offer named and the encoder can actually produce, and answers.
  2. The answer carries this server’s ICE credentials, its DTLS fingerprint and one host candidate that names the media port. The candidate address is detected from the routing table at startup. Behind a NAT, override it with publicAddress.
  3. The browser sends a STUN Binding Request to that candidate. MServer answers only if MESSAGE-INTEGRITY verifies against the password in the answer. The response is never larger than the request: an unauthenticated port that replies with more than it receives is an amplifier.
  4. The browser starts the DTLS handshake. MServer is always the DTLS server (a=setup:passive). The handshake fails unless the browser’s certificate hashes to exactly the fingerprint in its offer. That fingerprint is the only thing that authenticates the media peer. The first ClientHello gets only a HelloVerifyRequest, which is smaller than the ClientHello and never retransmitted. Its cookie is an HMAC-SHA256 of the browser’s address and port under a per-process key, so the large server flight goes only to an address that has proven it receives (no amplification). Once the cookie is verified, the session is bound to that address: records from any other address are ignored. Lost flights are retransmitted after 1 s, then 2 s, doubling up to 60 s.
  5. SRTP keys are exported from the completed handshake with the label EXTRACTOR-dtls_srtp. Media starts at the next key frame, because a viewer that joined mid-GOP cannot decode anything before it. MServer asks the encoder for that key frame as soon as the handshake completes.
  6. A PLI or FIR from the browser becomes a key-frame request for that stream. A viewer that joins or loses a frame then gets an IDR in milliseconds instead of waiting a whole GOP. The packet is verified as SRTCP first. The whole compound packet is read, so feedback behind a leading report is found whether or not a=rtcp-rsize was negotiated. One request per second per session is honoured, and one per second per stream whatever the number of sessions; the rest are counted and refused, so even an authenticated peer cannot turn every frame into an IDR. A pass-through deployment has no encoder to ask, so no request is honoured there.

Configuring it

Parameter Effect
webRtcEnable starts the endpoint at initVStreamer()
webRtcPort signalling port; the media port is this + 1 unless webRtcMediaPort names one
webRtcMediaPort UDP media port; 0 means webRtcPort + 1. Must differ from the signalling port, or initialisation fails. Read when the shared endpoint starts, so it never moves under a live stream
webRtcEncryption TLS policy for the signalling leg: no (default), optional or strict. Any other value fails initialisation. optional and strict behave the same: both ask for HTTPS, and initialisation fails if TLS cannot be set up
webRtcCert, webRtcKey HTTPS signalling. A certificate and key turn HTTPS on, whatever webRtcEncryption says. Without them, the RTSP pair (rtspCert / rtspKey) is used if there is one. With neither, signalling is plaintext only when webRtcEncryption is no and the compliance mode does not require TLS. With optional or strict and no material anywhere, the endpoint comes up on the generated identity, never in the clear, because the SDP carries the ICE credentials and the DTLS fingerprint an active attacker needs. In a mode that requires TLS that case fails initialisation instead, because an unauthenticated identity would not support the mode’s claim. Material given for this leg that fails to build a TLS context is always fatal. Borrowed RTSP material that fails is fatal only when webRtcEncryption asked for TLS
user, password digest authentication on the signalling endpoint, per published path: a WHEP request names its stream in the URL, so each stream answers for its own credentials and clearing one does not unprotect the others
securityProfile compliance mode – STRICT_FIPS refuses to start the endpoint at all

Requesting WebRTC in a mode that forbids it fails initialisation. Starting quietly without WebRTC would mislead: an operator who asked for WebRTC and got a running server would believe a viewer can connect.

Signalling works over HTTP or HTTPS, and the media is encrypted either way, because DTLS-SRTP is mandatory in WebRTC. But the SDP carries the ICE credentials and the DTLS fingerprint, which is everything an active attacker needs to take over the session setup. So the modes that require TLS refuse to open a plaintext signalling listener.

user/password follow the library-wide convention: "" or "no" means authentication is off, exactly as on the RTSP listener. As there, they are keyed by path. Setting a password drops that path’s live WHEP sessions. A viewer authenticates once, on the POST, and the media then flows over DTLS-SRTP with no further HTTP request; dropping the sessions makes a revocation take effect at once.

CORS. A page that reaches WHEP from another origin is cross-origin, and application/sdp is not a safelisted content type, so every real browser sends a pre-flight request first. (MServer’s own built-in player is served from the stream path itself, so it is same-origin. That is why it works with digest authentication and a file:// page does not; see The built-in player pages.) Cross-origin access is off by default (since 2.1.3), and what the endpoint answers depends on corsAllowedOrigin alone:

corsAllowedOrigin Answer to a cross-origin request and to its pre-flight
not set (no, the default; a wildcard * is refused and counts as not set) no Access-Control-Allow-* and no Access-Control-Expose-Headers header (a pre-flight still carries Access-Control-Max-Age): a browser refuses every cross-origin page
an origin Access-Control-Allow-Origin with exactly that value, Vary: Origin, the methods and request headers WHEP needs, and Access-Control-Expose-Headers: Location, WWW-Authenticate

The request’s own Origin is never reflected, and Access-Control-Allow-Credentials is never sent. A page can therefore use only a credential it supplies itself, never the one the browser holds for the realm. The pre-flight itself is never challenged: a browser never puts credentials on a pre-flight, so challenging it would make every cross-origin client fail before it could authenticate.

The endpoint is shared, and it belongs to the streams, not to the process. Like the RTSP listener, the first stream that needs it creates it, and the last one that stops releases it. So its ports do not stay bound after streaming ends. A second instance gets this same endpoint, and its policy is merged restrictively: a permission survives only if every attached stream grants it. If the running endpoint cannot meet a requirement of the new instance (TLS on a listener that already accepts plaintext, or authentication where none is configured), that instance fails to initialise. It is never served by a laxer endpoint while it believes otherwise.

What the answer offers

Attribute Value Why
a=sendonly always MServer is a source; it never receives media
a=setup:passive always the browser is the DTLS client
a=ice-lite always the server has a routable address and never probes back
a=rtcp-mux always one port for everything, which is what browsers require
a=fingerprint SHA-256 binds the DTLS certificate to this signalling exchange. In the offer, the first SHA-256 fingerprint counts (a SHA-256 line also replaces an earlier one in another hash); an offer without one is answered 400
a=fmtp packetization-mode=1, real profile-level-id taken from the actual SPS, so the description matches the bitstream
a=msid videoserver videoserver-video without it ontrack fires with an empty streams array, so the usual e.streams[0] is undefined and nothing renders (RFC 8830, RFC 8829 sec. 5.2.1)
a=ssrc cname and msid the per-SSRC form as well, for stacks that read RFC 5576 rather than a=msid

H.264 and H.265 are answered. MJPEG and MPEG-TS are refused: neither has a WebRTC payload format, and a browser has no decoder for either.

The codec parameters are checked against the offer, not just stated. For H.264, MServer reads the profile and constraint flags from the real SPS and compares them with every payload type the browser named. An offer that names only profiles the encoder cannot produce is refused with a reason. It is not answered with a profile-level-id the browser never asked for. Constraint-flag variants within one profile_idc count as one decoder, so a High-profile stream is still answered on Chrome’s constrained-High payload type. level-asymmetry-allowed is echoed only when the offer set it, and a stream above the offered level is refused. For H.265, the answer’s profile-space/profile-id/tier-flag/level-id come from this stream’s own SPS. Echoing the offer’s sprop-* back would describe the browser, not the stream.

Scope and limits of the WebRTC leg

  • Send-only video. No audio, no data channels, no simulcast.
  • No TURN and no STUN server. A host candidate only, so the media port must be reachable from the viewer. A deployment behind NAT forwards the UDP port and sets the public address.
  • No retransmission (NACK/RTX) and no congestion control. The stream is paced by the encoder, as every other MServer transport is. nack alone is deliberately never offered – only nack pli – because advertising it would promise retransmissions that never come.
  • Enabling WebRTC caps the shared packetizer at 1200 bytes, so the RTSP and RTSPS legs are capped with it. webRtcEnable is on by default, so this cap applies unless you switch WebRTC off. The reason: SRTP appends an authentication tag after packetization. A packet sized to fill the Ethernet MTU exactly would become a 1510-byte IP datagram and be fragmented. Every full-size packet belongs to a key frame, so fragment loss would cost key frames while small packets still arrive. The served and WebRTC legs share one packetization (that is what makes the fan-out cheap), so the smallest requirement wins. The push leg is not affected: when its container, KLV signalling or payload size differ from the served leg’s, it packetizes for itself at directStreamMaxPayloadSize. The cost is about 2 % more header overhead on the other legs.
  • A send failure is not silent. sendto() results are checked and counted (WebRtcStats::sendFailures); on failure the rest of the access unit is dropped and the session waits for a fresh key frame rather than delivering a half-written one. There is still no retransmission and no pacing within an access unit – a retry loop would block the encoder thread on one congested peer, which is the property every MServer transport is built to avoid.
  • RTCP is protected, in both directions. Sender Reports go out under SRTCP and incoming feedback is verified before it is acted on, so the NTP<->RTP anchor of RFC 3550 sec. 6.4.1 is available and a forged PLI is no longer possible. See SRTCP.
  • IPv4 only, like the rest of the library.
  • packetization-mode=0 and profile-level-id values the encoder does not produce are refused rather than answered wrongly.
  • Not available on Windows, which has no DTLS. There webRtcEnable gives one warning on stderr, the endpoint is not started, and the other legs run normally (see Windows build with MSVC).

HLS

RFC 8216 over the HTTP listener, on hlsPort. MPEG-TS segments, protocol version 3, a live sliding-window playlist, and optional AES-128 on the segments.

Request Serves
GET /<suffix>/index.m3u8 the media playlist
GET /<suffix>/master.m3u8 a one-variant master, for players that insist on one. Its RESOLUTION and BANDWIDTH describe this stream and follow a live geometry or bitrate change
GET /<suffix>/seg<N>.ts one segment
GET /<suffix>/key<N>.bin the AES-128 key of one generation, when encrypting

One listener serves either HTTP or HTTPS, as the WHEP endpoint does; never both at once. hlsEncryption carries two independent decisions, so read this table carefully:

hlsEncryption Transport Segments
no (default) HTTP plaintext
yes HTTP AES-128
optional HTTPS plaintext
strict HTTPS AES-128

yes protects the media itself and needs no certificate at all. That is the point: AES-128 segments are worth having on a plain HTTP listener. optional and strict ask for TLS. They take hlsCert/hlsKey, or the RTSP pair as a fallback. When neither exists and the compliance profile permits it, they take the generated identity. A compliance profile that requires TLS asks for HTTPS too, whatever hlsEncryption says. TLS is never picked up silently: a certificate that exists for RTSP does not move this listener to https://, because the scheme is part of the address a viewer was given. On Windows, optional and strict fail initialisation, because there is no TLS.

Segments are cut at key frames, and only at key frames. A segment closes on the first key frame at or after the target duration. So every segment opens with a PAT, a PMT and an IDR, and can be played on its own – provided the key frames carry their parameter sets. MServer forwards them as the bitstream has them and does not insert them (see Frame pipeline). Media before the first key frame produces no segment at all, rather than one a player cannot decode.

The leg muxes for itself, from the access units, not from the served leg’s transport stream; see Which leg carries which container.

A codec or geometry change ends the segment in flight and marks the next one EXT-X-DISCONTINUITY. Otherwise one segment’s PMT would declare one codec while part of its media used another. Once a discontinuity has been used, the playlist also carries EXT-X-DISCONTINUITY-SEQUENCE.

The target duration is fixed by the first playlist that is served, as RFC 8216 requires. A segment is cut at it even without a key frame, at three target durations before the first playlist is published, and at 16 MiB, so a long GOP cannot stretch a segment beyond what the playlist announced.

Segment names never repeat across restarts. The media sequence starts at the wall clock (Unix seconds), so a restarted server does not serve a new seg0.ts under a name a cache or player still holds. Segments are served with Cache-Control: private, max-age=<window x target duration>: a shared cache never keeps them, and a player’s own cache drops them once they have left the window.

JPEG produces no segments. MPEG-TS has no stream type for it (neither STANAG 4609 nor MISB ST 1402 defines one). The playlist stays valid and empty until the codec changes back, rather than listing segments no player can decode.

Segments are served by reference. The reactor writes straight from the segment buffer. A segment is therefore not copied per client and does not count against the output-queue cap, which is smaller than one segment. A client in the middle of a download keeps its segment alive after the sliding window has dropped it.

HLS and the compliance profiles

This is the one media leg permitted in every profile, including STRICT_FIPS. Nothing on it runs a key derivation outside the validated module. The segment key and the IV come from the module’s DRBG, and the segment is encrypted with the module’s AES-128-CBC; both are approved. There is no equivalent of the SRTP KDF to exclude, and therefore no limited-conformance statement.

The profiles constrain the transport and the key, not the leg:

  ONVIF FIPS_ONVIF_BRIDGE STRICT_FIPS CRA_STRICT
leg permitted yes yes yes yes
plaintext listener allowed refused refused refused
authentication optional required required required
segment key over plaintext served refused refused refused

The key endpoint is the security boundary of an encrypted stream: whoever fetches the key can decrypt every segment of that key generation. The profiles that set requireEncryptedMedia refuse the key on a plaintext listener, because they exist to guarantee that media never travels unprotected. In ONVIF mode the media itself may be plaintext, so withholding the key would protect nothing and only break playback.

A profile that requires TLS refuses to open a plaintext HLS listener at all. It does not serve the playlist and the key URI in the clear. requireTls treats the plaintext RTSP listener the same way.

The listener takes its policy when it starts and keeps it. A stream published on HLS therefore cannot be switched at runtime into a profile that is stricter on any line of this table, or about MD5 digest: setParam(SECURITY_PROFILE) is refused until the leg is withdrawn (hlsEnable off) or the stream is re-initialised in the new profile. A switch to a less strict profile is accepted and leaves the leg as strict as it was.

Limits worth knowing

  • One listener per process, keyed by suffix, like the RTSP listener and the WHEP endpoint. A second instance that names a different port or bind address is refused, never merged. A running listener’s policy can only get stricter: a later instance can add mandatory authentication or the key restriction, but it cannot add TLS to a plaintext listener (it is refused instead), and nothing is ever loosened. A caller with no certificate is not asking for plaintext. It has no opinion, and gets whatever the listener already has.
  • Credentials stay the stream’s own. A profile that demands authentication demands it only for the stream that chose the profile. A strict instance that attaches beside a permissive one does not withdraw the permissive one’s published stream. Credentials follow the same rule on the RTSP listener.
  • Memory is the window times the bitrate. Six 2-second segments at 5 Mbit/s take about 7.5 MB per stream; twelve streams take about 90 MB. The window is capped at 24 segments, so a misconfiguration cannot turn the device into a segment cache.
  • Latency is about three target durations, so roughly 6 s at the 2 s default. Low-latency HLS (partial segments, blocking playlist reload) is not implemented.
  • H.265 is served as TS. ffmpeg, VLC and GStreamer play it; Safari does not. Apple devices require HEVC in fragmented MP4, and there is no fMP4 muxer here.
  • JPEG produces no segments at all (see above). The RTSP leg carries JPEG well; the HLS leg has no stream type to declare.
  • A key rotates every 12 segments, and only the current and the previous key generation can be fetched. A generation number is never reused with a different key, not even across an encryption toggle. A player that fetches keyN.bin always gets the key that encrypted the segments that name generation N.
  • The listener accepts 32 connections at a time, shared by every stream in the process. The WHEP endpoint has the same cap, for the same reason: a peer that connects and says nothing holds a slot until the 30 s idle timeout. One remote address may hold at most 24 of the 32 (loopback is exempt), so a single host cannot lock the other viewers out. A player normally holds one keep-alive connection, so a dozen streams fit easily. A deployment with many more viewers should put a reverse proxy in front, which is how HLS is designed to scale anyway.
  • Turning the leg on at runtime needs a restart. setParam(HLS_MODE, 0) withdraws the stream at once. setParam(HLS_MODE, 1) is refused while the leg is off, because opening a listener needs the compliance decision and the TLS material that only initialisation has. Enabling RTSPS works the same way.

SRTCP

RTCP on a protected session is SRTCP, not plain RTCP. RFC 3711 sec. 3.4 makes authentication mandatory for SRTCP; for SRTP it is only recommended. RFC 5764 sec. 4.1 puts every RTCP packet of a UDP/TLS/RTP/SAVPF session under SRTCP. MServer applies it on both protected legs: WebRTC, and any RTSP subscriber that negotiated SAVP.

The 8-byte RTCP header stays in the clear, so a receiver can still demultiplex. Everything after it is encrypted. A trailer with an E flag and a 31-bit index is authenticated together with the packet. The two profiles order that trailer differently, and the wrong order produces packets that every receiver silently drops:

Profile Layout
AES_CM_128_HMAC_SHA1_80 (RFC 3711 sec. 3.4) header | encrypted | E+index | tag
AEAD_AES_128_GCM (RFC 7714 sec. 9.1) header | encrypted | tag | E+index

The SRTCP session keys come from the same master key as the SRTP ones, but with the labels of RFC 3711 sec. 4.3.1 (0x03 encryption, 0x04 authentication, 0x05 salt), so RTP and RTCP never share a keystream. Received packets are checked against a 64-entry replay window (sec. 3.3.2). The window advances only after the tag verifies, so a forged index cannot lock out the genuine sender.

What this buys, concretely:

  • PLI works with every browser, including inside compound packets.
  • Feedback is authenticated. RTCP that does not verify under the track’s own context – plaintext, a replayed index, a bad tag – is discarded whole and is not treated as a keep-alive.
  • Sender Reports carry the NTP<->RTP correspondence of RFC 3550 sec. 6.4.1, which is what a receiver needs to place the stream on a wall clock, and what inter-stream synchronisation would need the moment audio is added. Each WebRTC stream’s reports are stamped from that stream’s own clock (see Sender Reports and the wall clock).
  • On the RTSP leg, a SAVP subscriber’s report is protected under SRTP/SRTCP encryption.

Sender Reports and the wall clock

Every leg that sends RTP (RTSP and RTSPS over UDP and TCP, multicast groups, the push leg, WebRTC) reports once a second which RTP timestamp belongs to which wall-clock (NTP) time. A receiver dates each frame from the latest report: capture = NTP + (RTP timestamp - report's RTP timestamp) / 90 kHz. Three rules keep that date right:

  • The RTP timestamps run on the steady clock from a random start (RFC 3550 sec. 5.1), sampled when MServer publishes the access unit: after encoding for raw input, a fraction of a millisecond after sendFrame() for pre-encoded input. A wall-clock step never moves them, so a jitter buffer never sees time run backwards. The instant a frame was captured is the caller’s: it travels in ST 0601 Tag 2 and the ST 0604 SEI, and differs from the SR-mapped time by the pipeline’s latency.
  • The NTP time is the wall clock as it reads when the report is sent, not a wall-clock reading from start-up extrapolated along the steady clock. ST 0601 Tag 2, the ST 0604 SEI and every receiver read that same wall clock.
  • A step is reported at once. Whenever a sender could send a report, it compares the wall clock with the steady clock. If they moved apart by more than 1 ms since the last report, the next report goes out immediately (at most one early report per 20 ms, and before the next packet). Without this, the report could come up to a second later, and until then a receiver would date frames with the old mapping. WebRTC checks at its 250 ms reactor sweep.

The WebRTC endpoint is shared by every instance in the process, and each instance’s clock starts at its own random offset, so each stream’s reports are stamped from that stream’s clock.

Two things SRTCP does not change. First, the SRTP key derivation is still outside the validated module (SP 800-135 sec. 5.3 is not in CMVP #4985), so STRICT_FIPS still refuses SRTP and WebRTC. Second, Receiver Reports still do not extend the ICE consent timer: RFC 7675 sec. 5.1 says explicitly that received traffic is not evidence of consent. An authenticated peer is authenticated, not trusted.

Codec packetization

Codec Payload type Format
H.264 96 (dynamic) single NAL + FU-A, packetization-mode=1
H.265 97 (dynamic) single NAL + FU, no DONL
MJPEG 26 (static) RFC 2435 with in-line quantization tables, 8-bit only
MPEG-TS 33 (static) RFC 2250, 7x188 bytes per RTP packet
KLV 98 (dynamic) RFC 6597: one unit per RTP timestamp, fragmented across packets with the marker on the last. On a codec-RTP container it is a real second track (m=application, smpte336m/90000 or vnd.onvif.metadata/90000, trackID=1) with its own SSRC and sequence space, delivered to a subscriber that set that track up (multicast: the port pair above the video one). On an MPEG-TS container the bytes travel on a TS PID instead.

Aggregation packets (STAP-A / AP) are deliberately not produced: they are an optimisation, and every receiver must accept the single-NAL and fragmentation forms.

a=fmtp fields are derived from the actual bitstream after removing emulation-prevention bytes (0x000003 -> 0x0000) – parsing the escaped bytes yields wrong profile and level values.

MJPEG is limited by RFC 2435 to 2040x2040 (dimensions are one byte each in 8-pixel units). Larger images, and images whose width or height is not a multiple of 8, are refused rather than truncated into a silently wrong header; use H.264 or H.265 instead. A picture carrying 16-bit quantization tables is refused for the same reason: the RFC 2435 quantization header has a precision bit per table, but receivers overwhelmingly assume 8-bit, so shipping one would decode to garbage. Huffman tables. RFC 2435 carries no Huffman tables: every receiver decodes with the standard ones (ITU-T T.81 Annex K.3), whatever the scan was coded with. A picture whose scan uses other tables is packetised as it is and decodes to garbage at every receiver (a few damaged rows and an empty picture). The MJPEG encoders of FFmpeg and OpenCV write such optimised tables by default (FFmpeg: -huffman default writes the standard ones), and so does the VCodecLibav test codec of the harness; an encoder must be set to the standard tables. A picture without a DHT segment is taken as using the standard tables.

MPEG-TS and KLV metadata

KLV is encoded by the caller and forwarded byte-exact by default. Two mutually exclusive carriage forms are supported (MISB ST 1402.1):

  Asynchronous (SMPTE RP 217) Synchronous (ISO/IEC 13818-1 sec. 2.12.4)
stream_type 0x06 private_data 0x15 metadata
stream_id 0xBD 0xFC
PTS absent (PTS_DTS_flags='00') mandatory on every PES
Descriptor registration_descriptor “KLVA” metadata_descriptor + metadata_std_descriptor
KLV wrapper raw metadata_AU_cell: metadata_service_id 0x00, as in the PMT descriptors, and a sequence_number that counts the cells

Three KLV handling modes exist:

  • PassThrough (default): byte-exact.
  • Validate: checks the universal key, the BER length and the ST 0601 sec. 6 checksum.
  • Restamp: rewrites Tag 2 with the current time and recomputes Tag 1. Use it only for sources with no clock of their own, because it destroys the ST 0601.8 “time of birth” meaning of Tag 2.

The codec-RTP stream types select the mode (rtp-klv-validate, rtp-klv-restamp). Every mpegts-* value forwards the caller’s bytes unchanged. The mode is applied to the buffer before any container carries it. A unit that fails is counted in klvRejected and not sent. Only serverStreamType selects the mode: directStreamType chooses the push leg’s container, not its KLV handling.

PSI and timing. PAT and PMT are repeated every 100 ms and before every key frame (on HLS: at the head of every segment). A receiver that joins at a key frame, or rejoins after a container change, therefore finds the PMT first.

The PMT’s version_number changes (modulo 32) whenever the PMT’s content changes. That happens on a codec switch (the video stream_type), when the first KLV unit arrives on a stream whose type did not name KLV, and when the KLV signalling changes (a re-typed push leg, or a live SERVER_STREAM_TYPE change). The new PMT goes out in that same access unit, before the data it describes. A demuxer re-reads a PMT only when its version changes; a PMT that kept version 0 for ever would never tell it about a switch to H.265 or a metadata PID added later.

PTS and PCR are 33-bit values extended from the stream’s 90 kHz RTP timestamp; their low 32 bits equal it. Every MPEG-TS leg shares them. They do not fall back when the 32-bit timestamp wraps, which can happen seconds into a stream because the timestamp starts at a random value. They wrap at 2^33, as MPEG-TS defines.

The PCR runs one frame interval plus 10 ms behind the PTS (100 ms when the rate is unknown, never more than 1 s). With one PCR per access unit, the last byte of an access unit, and the KLV muxed after it, arrive at about the next unit’s PCR. The T-STD of ISO/IEC 13818-1 requires them to be in the decoder by the unit’s decoding time. Up to 2.1.0 the PCR equalled the PTS, so a strict decoder saw every access unit on every MPEG-TS leg arrive almost a frame interval late. MServer sends nothing later because of this delay. A player that slaves its clock to the PCR simply presents that much later than one that does not (VSourceStream reads no PCR).

On a live leg, PCRs are never more than 100 ms apart (ISO/IEC 13818-1 sec. 2.7.2), whatever the frame rate. One PCR per access unit achieves that only above 10 fps. So when no access unit has come for 60 ms, the served leg and the push leg send a clock reference of their own. It is one datagram with the PAT, the PMT and a transport packet on the PCR PID that carries only the PCR, timed at that instant on the same clock as the access units’ PCRs. The packet has an adaptation field and no payload, so the PID’s continuity counter does not advance. Up to 2.1.0 a 5 fps stream went 200 ms between PCRs, and as long between PATs. Only streams below about 16 fps ever send clock references. None is sent once a second has passed without an access unit: a clock does not keep a stopped source alive. HLS is not a live leg (a player fetches whole segments), and its segments carry one PCR per access unit.

KLV size limit. A KLV PES must state its PES_packet_length, which is 16 bits. On MPEG-TS one unit can therefore be at most 65532 bytes asynchronous or 65522 bytes synchronous. A larger unit is left out of the transport stream; the access unit’s video still goes out, and a codec-RTP metadata track still carries the unit, fragmented. The unit is counted once in klvRejected, however many MPEG-TS legs refused it.

Which leg carries which container

The two legs choose independently, and only one of them is restricted:

  Accepted streamType values Where the KLV goes
served leg (serverStreamType, i.e. RTSP, RTSPS, RTSP-over-HTTP) rtp, rtp-klv, mpegts-rtp-klv* rtp-klv: an RFC 6597 metadata track, payload type 98, on the same session. mpegts-rtp-klv*: inside the transport stream. rtp: video only
push leg (directStreamType) all of the above plus the raw forms mpegts-klv-sync / mpegts-klv-async same, and the raw forms put the KLV on a TS PID in datagrams with no RTP header at all
HLS leg not configurable – always MPEG-TS segments on a TS PID inside the segments, and not signalled in the playlist, so a player reads the video and ignores it. Whether it is synchronous or asynchronous follows serverStreamType, which is the only KLV decision this leg takes from the configuration

The HLS leg muxes for itself; it does not tap the served leg’s transport stream. This is not wasted work. In normal configurations serverStreamType is rtp or rtp-klv, so the served leg carries RTP and builds no transport stream to tap. Where it does build one, its PSI comes every 100 ms, not at the segment boundaries HLS needs. The push leg works the same way when its container differs from the served leg’s.

A transport stream reaches the served leg only inside RTP (MISB ST 0804 / RFC 2250, payload type 33). An RTSP client negotiates an RTP transport, so the raw mpegts-klv-* forms have nowhere to go there and are refused. They are exactly what the push leg exists for (MISB ST 1402).

Whether the served leg carries a metadata track follows serverStreamType live. Adding klv to a running stream starts the track and updates the SDP that the next DESCRIBE returns. Removing it stops advertising the track. MServer does not read the metadataEnable field at all: the stream type decides carriage, and the caller decides whether to offer a KLV buffer in the first place.

MServer also emits the MISP precision time stamp SEI (MISB ST 0604.6). It carries the same microsecond value as the frame’s ST 0601 Tag 2 (in Restamp mode, the restamped value). It is added to every H.264 and H.265 access unit whose frame came with an ST 0601 KLV buffer (sendFrame()’s userData) that has a Tag 2 and passes the ST 0601 checks:

  • the universal key and the BER length are valid;
  • the items fill the set exactly (keys are read as BER-OID, so tags above 127 are handled);
  • the Tag 1 checksum, when present, is the last item.

This holds whatever the stream type, including serverStreamType rtp, where the served leg does not carry the KLV. The SEI is the one wall-clock link that reaches every receiver, WebRTC and plain-RTP viewers included.

It is a user_data_unregistered SEI. The UUID is MISPmicrosectime for H.264 and a8687dd4-d759-3758-a5ce-f0338b6545f1 for H.265. The payload is the 8 time bytes, with an 0xFF after every second byte except the last. The SEI sits right before the first slice, after the AUD, the parameter sets and any SEI the encoder wrote. Every leg carries it: RTP, MPEG-TS, the push leg, HLS and WebRTC. An access unit that already carries an ST 0604 time stamp keeps its own, and JPEG has no SEI.

So pre-encoded input is not forwarded byte for byte when sendFrame() is given such KLV, even on a video-only stream type. Each access unit grows by 36 bytes (H.264) or 37 bytes (H.265). If you need byte-exact forwarding of pre-encoded input, do not pass KLV with it.

The SEI’s Time Status byte is 0x9F. MISB ST 0603 / ST 0604.6 define bit 7 as the lock indicator (0 = the time source is locked to an absolute reference such as GPS), bit 6 as the discontinuity indicator, bit 5 as the reverse indicator, and bits 4-0 as reserved. The time is the caller’s Tag 2. ST 0601 carries no lock flag, and MServer cannot know whether the clock behind it was locked, so it claims no lock: unlocked, no discontinuity, forward. Version 2.1.0 wrote 0x1F, which claimed a lock nobody had verified. A receiver reads the time the same way in both cases (VSourceStream reads the time and ignores the status); the bit only says how far to trust it.

Security and compliance modes

Some obligations of FIPS 140-3, ONVIF and the EU CRA are mutually exclusive on the wire, so one configuration cannot satisfy all of them. MServer therefore makes the choice a runtime switch, so that one device can be certified against several standards (see Why compliance is switchable).

Selecting a mode

Set VStreamerParams::securityProfile:

Value Mode
ONVIF maximum interoperability
STRICT_FIPS everything inside the validated module
FIPS_ONVIF_BRIDGE approved primitives, SRTP for ONVIF clients
CRA_STRICT secure by default

The value is matched case-insensitively. "" and "no" select ONVIF. Any other value is refused: initVStreamer() and setParam() both fail rather than pick a default. This is deliberate. The field decides whether the validated provider is loaded, whether TLS and authentication are mandatory, whether MD5 digest is offered and whether WebRTC starts. A typo that silently relaxed all four is exactly the failure this rule prevents.

custom2 is opaque caller data, like custom3. Setting it does not change the compliance mode, and it survives the getParams() round trip. The mode lives only in securityProfile, which rejects a value it does not know.

On Linux the mode also selects the OpenSSL provider. There is no build-time crypto switch. initVStreamer() asks OpenSSL for the validated module when the mode requires one (STRICT_FIPS, FIPS_ONVIF_BRIDGE). Since 2.1.3 these two modes require it: when the module cannot be activated, initVStreamer() fails instead of starting on the default provider. A FIPS profile is a promise to the clients, and a misconfigured host must not run a server that is believed to be validated. The hardened configuration for a host without the module is CRA_STRICT, which makes no FIPS claim. Initialisation also fails when no provider at all can be loaded or the DRBG is unusable. Activation is proved, not assumed: the library checks that an approved digest resolves and that a non-approved one does not. “FIPS enabled” and “FIPS enforced” are not the same thing. On Windows the FIPS state comes from CryptoProvider instead; see Cryptography provider configuration.

Provisioning the module is the deployment’s job. MServer loads the system OpenSSL configuration and never a path of its own, so openssl.cnf decides which provider is available, not this library.

The provider is a process-wide property, because the process has one crypto context. The first instance to initialise establishes it. A later instance that asks for the validated module when it is not active is refused (2.1.3): its initVStreamer() fails, exactly as it would on a host without the module. So a process that needs a FIPS stream brings that stream up first, or provisions the module for the whole process; a later stream that only needs the hardened configuration uses CRA_STRICT, which runs on either provider. A mode that does not require the module runs normally in a process where the module is active. What it may offer is then tightened automatically: ONVIF drops MD5, because the module does not implement it.

What each mode changes

  ONVIF STRICT_FIPS BRIDGE CRA_STRICT
MD5 digest (RFC 2617) yes no no no
SHA-256 digest (RFC 7616) yes yes yes yes
SRTP AES-CM yes no yes yes
SRTP AES-GCM yes no no only without the FIPS provider
SDES key exchange yes no yes yes
MIKEY key exchange no – unverified, see Limitations no no no
TLS required no yes yes yes
Authentication required no yes yes yes
Plaintext RTSP listener yes no no no
Unprotected media on request yes no no no
Multicast yes no no no
WebRTC yes no yes yes
Plaintext WebRTC signalling yes – no no
HLS leg yes yes yes yes
Plaintext HLS listener yes no no no
Segment key over plaintext yes no no no
Generated TLS identity when no certificate is supplied yes no no no
Full module coverage no yes no no

HLS is the one media leg permitted in every mode, STRICT_FIPS included. Nothing on it derives keys outside the validated module: the segment key and IV come from the module’s DRBG, and the cipher is the module’s AES-128-CBC. The strict modes constrain its transport, not the leg. They also refuse the generated TLS identity. A mode that requires TLS claims a protected channel, and an unauthenticated identity does not support that claim. So in those modes, no certificate still means no listener.

STRICT_FIPS and FIPS_ONVIF_BRIDGE require the validated module (2.1.3; up to 2.1.2 they preferred it and ran without it with the claim withdrawn). Every restriction they enforce is MServer’s own policy: no MD5, SHA-256 digest only, mandatory TLS and authentication, no SRTP in STRICT_FIPS, no plaintext listener. The default OpenSSL provider implements every primitive involved, so the same hardened configuration is available on a stock OpenSSL – as CRA_STRICT. The module does not decide whether the configuration works; it decides whether a FIPS claim may be made, and a mode that makes one refuses to run without it.

That distinction is made visible rather than assumed:

  validated module active default provider
initVStreamer() succeeds fails (2.1.3; up to 2.1.2: succeeded with the claim withdrawn)
algorithm restrictions enforced – (the stream does not run; CRA_STRICT enforces the same restrictions without a module)
getStats().complianceMode STRICT_FIPS – (CRA_STRICT for the hardened mode without a module)
full module coverage claimed yes (STRICT_FIPS only) no
compliance statement names the validated module says the module is NOT active and that no FIPS 140-3 claim applies (resolved before the refusal)

The statement is resolved inside the library and is not exposed through getStats(); the four texts are reproduced verbatim in Compliance summary so they can be copied into certification paperwork without reading the source.

So a FIPS profile runs only where the module is provisioned, and a hardened deployment without the module runs CRA_STRICT. A deployment that needs the certified claim still checks complianceMode: a bare STRICT_FIPS is the only string that carries it.

CRA_STRICT is about configuration rather than algorithm choice, so it has never needed the module.

Why STRICT_FIPS refuses SRTP: both profiles derive session keys with the SRTP KDF, which is not in the approved-algorithm table of CMVP #4985. Any SRTP therefore performs key derivation outside the cryptographic boundary. In strict mode media travels as RTP interleaved inside TLS instead, which keeps the entire media path inside module-provided primitives.

Why STRICT_FIPS refuses WebRTC: for the same reason it refuses SRTP. WebRTC media is SRTP by definition, so the session keys are derived by the SRTP KDF outside the validated module, and there is no interleaved-inside-TLS alternative the way there is for RTSP. The endpoint therefore does not start at all in that mode.

Why CRA_STRICT does not claim full module coverage: it permits SRTP, whose key derivation is the same out-of-module KDF that STRICT_FIPS refuses SRTP over. Two modes with the same media path cannot honestly make different coverage claims, so only STRICT_FIPS – the mode with no SRTP at all – makes the unqualified one.

Why a strict mode refuses the direct RTP push leg: that leg carries media with no TLS, no SRTP and no handshake. A mode that will not open a plaintext RTSP listener cannot consistently send the same media unprotected to a fixed destination, so initVStreamer() refuses the combination. So does the running stream: setParam(DIRECT_STREAM_ENABLE, 1) is refused under such a mode, also for a leg initialisation built while it was switched off, and a stream whose leg is on cannot be switched into such a mode with setParam(SECURITY_PROFILE). Note that the interface defaults directStreamEnable to true with a default destination (127.0.0.1:5004). A configuration that never mentioned the push leg therefore still requests one. Set directStreamEnable = false explicitly when you use CRA_STRICT, FIPS_ONVIF_BRIDGE or STRICT_FIPS.

Why ONVIF enables MD5: ONVIF Core sec. 5.9.3 makes MD5 the default digest algorithm for RTSP, and almost every deployed client implements only MD5. The FIPS provider does not implement MD5 at all (EVP_MD_fetch returns NULL). Without a switch, the choice would be between breaking ONVIF and having no FIPS mode.

Certificates and keys

On Linux three legs can be TLS-protected: RTSPS (and the RTSP-over-HTTPS tunnel on the same listener), the WHEP signalling endpoint, and the HLS listener. Windows has no TLS. Each leg takes a PEM certificate and a PEM private key by file path:

Leg Asks for TLS with Material Falls back to
RTSPS rtspEncryption = optional or strict rtspCert / rtspKey –
WHEP webRtcEncryption = optional or strict, or a usable certificate and key (its own or the RTSP pair) webRtcCert / webRtcKey the RTSP pair
HLS hlsEncryption = optional or strict hlsCert / hlsKey the RTSP pair

"no" and the empty string both mean not set. RTSPS and HLS never pick up material on their own. A certificate that happens to exist for RTSP does not silently move the HLS listener from http:// to https://, because the scheme is part of the address a viewer was given. The WHEP endpoint is the exception: it uses HTTPS whenever it has a certificate and key, its own or the RTSP pair.

When no certificate is configured

When a leg asks for TLS and no certificate is supplied, MServer generates an identity for itself. Where that identity lives depends on what the caller configured:

Configuration What happens
No certificate or key named ("no" or empty) An identity is generated and kept in memory only. Nothing is written to disk. It is new on every start, so its fingerprint changes at every restart
A certificate and key are named but cannot be loaded – missing on a first boot, unreadable, corrupt RTSPS: an identity is generated, written to those paths and loaded back. The next start finds it and reuses it, so the fingerprint is stable. Only when each path is missing or a regular file and the two are different files: a path naming a directory, a symbolic link, a device or a FIFO is never touched, and the RTSPS leg then does not come up (initialisation fails when rtspEncryption is strict). WHEP and HLS: initialisation fails, unless they name the same files as the RTSPS leg, which has just written them
A certificate and key are named and load Used as given. Nothing is generated
Any of the above under a profile that requires TLS (STRICT_FIPS, FIPS_ONVIF_BRIDGE, CRA_STRICT) Refused. Initialisation fails rather than come up on an identity nothing certifies

A path is a statement of intent: naming one says a persistent identity is wanted there. Keeping that identity in memory would give every restart a different fingerprint and break every client that pinned the last one. Naming no path says the opposite.

Nothing already on disk is ever destroyed. “Cannot be loaded” covers a wrong passphrase or a passing read error as well as a missing file. So a regular file present at either path is renamed to <path>.unloadable before the new material is written. When that name holds an earlier backup, the first free one of <path>.unloadable.1 to <path>.unloadable.99 is used instead: a backup is never replaced, and with all of them taken nothing is generated. A path that holds anything other than a regular file – a directory, a symbolic link, a device, a FIFO, a socket – is left untouched and nothing is generated, as when the certificate and the key name the same file. Both paths are checked before either is moved. The key file is created with O_EXCL | O_NOFOLLOW | O_CLOEXEC and mode 0600 in one call. It is not chmod-ed afterwards, which would leave a moment in which a world-readable private key exists. Both files are fsync-ed before initialisation continues, because the whole point of writing them is that the next boot finds them.

getStats() tells the three cases apart. tlsSelfSigned is TRUE for the in-memory identity. tlsGeneratedToDisk is TRUE when the identity was generated and written to the named paths. Both are FALSE when the named files loaded.

How the identity is built (on Linux; Windows has no TLS):

  1. Key. An EC P-256 key pair from EVP_PKEY_Q_keygen, bound to MServer’s own OSSL_LIB_CTX, so the entropy comes from the validated module’s DRBG when the module is active. P-256 is used rather than RSA-2048: both are approved and both interoperate, but the EC key generates in microseconds, while RSA can take a noticeable fraction of a second on an embedded CPU.
  2. Certificate. A self-signed X.509 v3 certificate over that key. The serial number comes from the DRBG, with bit 62 forced so that it is positive and non-zero (RFC 5280 sec. 4.1.2.2). The machine’s host name is the subject and, being self-signed, the issuer. The validity is 10 years, backdated by an hour, so a device whose clock has not been set still works. It carries basicConstraints=CA:TRUE, a keyUsage that includes keyCertSign, and extendedKeyUsage=serverAuth. The CA bits look odd on a certificate used as a leaf, and they are deliberate: they let an operator give the certificate to a client as a trust anchor, and get a verified connection instead of switching verification off. subjectAltName carries the host name, localhost and the loopback addresses.
  3. Signature. SHA-256 through an EVP_MD_CTX bound to the same library context. (X509_sign(cert, key, EVP_sha256()) would resolve the digest in the default context, which in a FIPS deployment is not the validated module.)
  4. Storage. The result is kept as PEM bytes, not as OpenSSL objects. Bytes hold no reference to a library context, so the identity neither pins a context nor breaks when the context is replaced. Keeping an X509 and an EVP_PKEY would freeze the context for the life of the process and stop a later instance from upgrading to the validated module. Each leg parses the same bytes into whatever context is current, which keeps one device to one fingerprint.
  5. Reuse. Generated once per process and shared by all three legs, so a fingerprint pinned for RTSPS matches HLS and WHEP.

What the in-memory identity does not do:

  • Nothing is written to disk. There is no file for a backup, a core dump or an operator to carry off the device. The private key exists only in process memory, and no accessor exposes it.
  • VStreamerParams is not modified. getParams() still returns the "no" you configured. The identity is internal, and a caller that round-trips its parameters sees no trace of it.
  • It is reported, not hidden. getStats().tlsSelfSigned is TRUE and getStats().tlsFingerprint carries the SHA-256 fingerprint, so nobody can mistake it for a certificate they installed. The flag is read from the running listener, not assumed from the configuration. On the process-wide WHEP and HLS listeners, this instance may be joining a listener that another instance started with a real certificate. Reporting the generated fingerprint there would make an operator pin something no client is ever shown.

What it gives, and what it does not. Every byte on the wire is protected from a passive observer, and the connection is bound to a key a client can pin. It carries no authenticity: nothing certifies that this identity belongs to this device. A client that does not pin the fingerprint cannot tell the server from an active man in the middle. The identity also changes on every restart; that is the price of never storing a private key.

Treat it as a first-run convenience, not a deployment posture. A device that must be verifiable installs its own certificate.

# The fingerprint the server reports, from the client's side:
openssl s_client -connect <host>:8080 </dev/null 2>/dev/null \
  | openssl x509 -noout -fingerprint -sha256
# It must equal getStats().tlsFingerprint. If it does not, you are not talking
# to this server.

Where the fallback is refused

A profile that requires TLS makes a claim about the channel, and an unauthenticated identity does not support that claim. So STRICT_FIPS, FIPS_ONVIF_BRIDGE and CRA_STRICT refuse it: no material means no listener, and initVStreamer() fails. Only ONVIF, the mode that makes no such claim, accepts the generated identity.

The same rule holds at runtime. A stream running on the in-memory identity cannot be switched into one of those profiles with setParam(SECURITY_PROFILE, ...). The switch already refuses a stream with no TLS listener at all. A listener on the generated identity is not one a client can authenticate either, so the switch is refused; re-initialise with real material instead.

Material that is named but cannot be loaded is the case where the RTSPS leg writes the identity to disk instead of keeping it in memory (see the table above). In a profile that requires TLS that case still fails outright: a typo in a path must not quietly become a self-signed certificate under a mode that promises a verifiable one.

Making a real certificate 

# Self-signed, for a lab. Name the address clients will dial in the SAN, or a
# verifying client rejects it even after you trust the certificate.
openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
  -keyout key.pem -out cert.pem \
  -subj "/CN=camera-1" -addext "subjectAltName=IP:192.168.1.83,DNS:camera-1"

Then params.rtspCert = "/path/cert.pem"; params.rtspKey = "/path/key.pem";. For anything beyond a lab, use a certificate from whatever CA the deployment already trusts.

Cryptographic module boundary

All cryptographic primitives, and all use of OpenSSL, are confined to seven translation units in src/impl/:

  • MServerCrypto.cpp: providers, digests, the DRBG, TLS contexts and the generated self-signed identity;
  • MServerSrtp.cpp: AES-CTR, HMAC-SHA1 and AES-GCM for SRTP, through CryptoProvider;
  • MServerRtsp.cpp: SSL_read/SSL_write on accepted RTSPS connections;
  • MServerIce.cpp: HMAC-SHA1 for STUN MESSAGE-INTEGRITY and constant-time comparison, through CryptoProvider;
  • MServerDtls.cpp: the DTLS-SRTP handshake and the key export;
  • MServerHttp.cpp: TLS on the signalling and HLS listeners;
  • MServerHls.cpp: AES-128-CBC on HLS segments, through CryptoProvider.

No OpenSSL or CryptoProvider type appears in any public header.

On Linux, MServer uses a private OpenSSL library context (OSSL_LIB_CTX), so it never competes with the host application for global OpenSSL state. Since 2.1.2 it is one context, shared with CryptoProvider: it holds the TLS and DTLS contexts, the keys, the certificates and the DRBG, and it runs CryptoProvider’s SHA-256, HMAC and AES as well. In a FIPS mode it holds the validated module. Every random value – SSRC, initial sequence number, initial RTP timestamp, session identifiers, digest nonces, SRTP keys – comes from RAND_bytes_ex() on MServer’s context. On Windows, the random values and the primitives come from CryptoProvider’s CNG backend, and there is no TLS or DTLS.

Exactly one cryptographic primitive is implemented in MServer itself: MD5, for RFC 2617 digest interoperability. (Two constructions are composed in MServer from module primitives as well, the SRTP key derivation and the SRTP IV; see FIPS 140-3.) MD5 is a non-approved legacy function, and no FIPS mode can reach it: ONVIF is the only mode that enables MD5 digest. On Linux, Crypto::init() proves this at load: whenever the validated module is active, an approved fetch must succeed and an MD5 fetch must fail.

Compliance summary: FIPS 140-3, EU CRA, ONVIF

A condensed, decision-oriented view of what MServer does and does not give a certification effort. The mechanics are in Security and compliance modes; this section is what to put in front of an auditor, and what not to promise them.

What MServer is, in certification terms

MServer is not a cryptographic module and does not aspire to be one. It is a consumer of one. The validated boundary is the OpenSSL FIPS provider (CMVP #4985, OpenSSL FIPS Provider 3.1.2), and every line of MServer is outside it. Nothing in this repository can invalidate that module, and nothing in this repository is itself validated.

That distinction decides the shape of every claim below. A FIPS statement is about which module performed the cryptography; a CRA statement is about the product placed on the market; an ONVIF statement is about the device’s interfaces. MServer supplies evidence for all three and is the subject of none.

Two further scoping facts:

  • A claim is about a mode, not about the library. securityProfile selects one of four mutually exclusive configurations, and what is on the wire differs between them. Every test report must name the mode it ran in.
  • On Linux the library has one external dependency: OpenSSL. FFmpeg is not linked (MServerNoFFmpegDependency asserts it), so the SBOM of the streaming layer is one entry, and the CVE surface is one project. A Windows build links no OpenSSL and makes no FIPS claim.

FIPS 140-3

Inside the boundary. In a FIPS mode with the module active, the cryptography comes from private OpenSSL library contexts that hold the FIPS provider. The one context MServer shares with CryptoProvider provides the TLS 1.2+ and DTLS 1.2+ contexts and every random value the server produces (SSRC, initial sequence number and RTP timestamp, session identifiers, digest nonces, SRTP master keys), through EVP_*_fetch(libctx, ...) and RAND_bytes_ex(libctx, ...), and CryptoProvider’s SHA-256, HMAC, AES-CTR, AES-GCM and AES-CBC. A private context also means that MServer never competes with the host application for global OpenSSL state.

Enforcement is proved, not assumed. Loading the provider is not accepted as proof that it is in charge. After EVP_default_properties_enable_fips(), MServer asks its own context two questions that only an enforcing module answers correctly: SHA2-256 must fetch, and MD5 must not. If MD5 still resolves, the provider is unloaded and the module is treated as absent. “FIPS enabled” and “FIPS enforced” are different statements, and only the second is accepted. Since CryptoProvider runs in the same context, the proof covers its primitives too.

What is outside the boundary, and what MServer does about it.

Operation Why it is outside Consequence
SRTP / SRTCP session-key derivation The SRTP KDF is approved by SP 800-135 Rev.1 sec. 5.3 but is not implemented by CMVP #4985 STRICT_FIPS refuses SRTP entirely and carries media as RTP interleaved inside TLS; FIPS_ONVIF_BRIDGE permits AES-CM and declares limited conformance
AES-GCM SRTP (RFC 7714) The IV is constructed externally from SSRC/ROC/SEQ, which the module’s security policy names as a non-conformance refused in both FIPS modes
WebRTC media SRTP by definition, and with no interleaved-inside-TLS alternative STRICT_FIPS refuses the endpoint outright; requesting it fails initialisation
RFC 2617 MD5 digest MD5 is absent from the module the only cryptographic primitive implemented in-tree; unreachable in STRICT_FIPS, FIPS_ONVIF_BRIDGE and CRA_STRICT, and refused on receipt of the Authorization header, not merely left out of the challenge

Running without the validated module. A FIPS mode does not start (2.1.3): initVStreamer() fails, because a FIPS profile is a claim about the module. The restrictions themselves are MServer’s own policy, and the default OpenSSL provider implements every primitive involved, so the hardened configuration is available on stock OpenSSL as CRA_STRICT, which claims nothing about FIPS. The ` (unvalidated provider) suffix of complianceMode remains defined for the case the provider is lost after a successful start. **A bare STRICT_FIPS with fullModuleCoverage == true` is the only runtime evidence that a validated claim applies.** Check it in a self-test; do not infer it from the configuration.

What the laboratory will still need from you. MServer supplies the technical facts; the submission is the integrator’s. Expect to:

  • state the module boundary, and that MServer is a consumer of it;
  • map every cryptographic operation to an approved algorithm;
  • list the non-approved functions and prove they are unreachable in the approved mode. For MServer that list has exactly one entry: MD5, for RFC 2617 interoperability;
  • show that entropy comes from the module’s DRBG.

The one non-approved entry, and the evidence that it is unreachable, are described under Cryptographic module boundary. The written submission dossier is the integrator’s to produce; it is not part of this repository.

EU CRA (Regulation 2024/2847)

The CRA applies to every product with digital elements placed on the market, and a software library sold as such is one: whoever places MServer on the market, and the manufacturer of the product that integrates it, carry the obligations. The code of MServer can supply only the Annex I Part I engineering half, and only partly (see Verdict).

Annex I Part I What MServer provides
2(b) secure by default CRA_STRICT: TLS mandatory, authentication with a non-empty password mandatory, no plaintext RTSP listener, MD5 refused. Not the default: an untouched configuration is ONVIF with RTSP, the RTP push, WebRTC and HLS enabled and no authentication
2(d) protection from unauthorised access RFC 7616 SHA-256 digest, constant-time comparison, per-nonce replay rejection by nonce-count, per-path credentials on a shared listener; 2.1.3: a connection of the RTSP listeners is dropped after 5 wrong-credential responses (2.1.4: the WHEP and HLS listeners too), and getStats().authFailures counts them on the RTSP listener
2(e) confidentiality in transit RTSPS / RTSP-over-HTTPS, DTLS-SRTP for WebRTC, SRTP and SRTCP for RTSP; in CRA_STRICT a SETUP that would put media in the clear is refused rather than served
2(h) availability: no unbounded allocation every parameter that bounds an allocation or names a port is range-checked and rejected, never clamped
2(h) availability: resilience against DoS 16 KiB request-head cap, 64 KiB request-body cap (Content-Length; a larger one is answered 413 and the connection closed, a malformed one 400 – the write side is shut down first and what the peer still sends is drained, at most 1 MiB for 2 s, so the refusal is followed by a FIN and not destroyed by a reset), 64 headers, 512-byte URL, 64 sessions, 160 RTSP connections (accept-and-close beyond; when the process runs out of descriptors the listener pauses for 100 ms instead of spinning), at most 256 KiB of queued RTSP replies per connection (requests are no longer read beyond it; closed after 5 s without write progress), on the HTTP (WHEP / HLS) listener 32 connections with at most 24 per remote source (loopback exempt) and a 10 s write-progress deadline on half-closed connections, 1 MiB per-subscriber queue with whole-access-unit drop, 2048-packet push queue with whole-access-unit drop, 4 MiB output queue, idle and sessionless timeouts, at most 16 RTCP datagrams per socket per reactor round and 32 NACK entries per RTCP packet
2(i) minimise impact on other services no UDP amplification: unicast RTP goes only to the source address of the control connection, and a client-supplied destination= is honoured only for a multicast group inside the configured pool. RTCP feedback over UDP is taken only from the client’s address, and a NACK draws each packet it names once, three times at most over the packet’s life
2(j) limit attack surfaces one external dependency (OpenSSL, on Linux); no external process; compliance modes that remove reachable code paths rather than merely discouraging them
2(k) exploitation mitigation compiled with -fstack-protector-strong and -D_FORTIFY_SOURCE=2 (MSVC: /GS /guard:cf); linker hardening reaches every executable (see Build and connect to your project)
2(l) recording and monitoring of activity getStats() counters: since 2.1.3 authFailures (wrong credentials) and connectionsRefused (admission limits), beside the TLS / SRTP failure counters. The library writes no log file of its own; the deployment polls the counters into its logging
Annex VII documentation this README, SECURITY.md, SECURITY-ADVISORIES.md, the SBOM sbom.cdx.json and the fuzzing corpus in harness/fuzz/ (eight libFuzzer targets). A standalone threat model is not in this repository and has to be written for the product

Annex I Part II – vulnerability handling – is a process obligation on the manufacturer. Since 2.1.3 the repository carries its documents: SECURITY.md (coordinated vulnerability disclosure policy, support period, security-update channel, Article 14 reporting process), sbom.cdx.json (CycloneDX 1.5) and SECURITY-ADVISORIES.md. Executing the process is still the manufacturer’s: the SBOM (Part II (1)), the coordinated vulnerability disclosure policy (Part II (5)), the security-update channel, the support period (Article 13(8)) and the reporting duties of Article 14, which apply since 11 September 2026. None of that lives in this repository, and whoever places MServer, or a product with it, on the market must supply it.

ONVIF

MServer implements the streaming half of ONVIF Profile S / T and nothing else. There is no SOAP layer and no device management. MServer exposes the control points that a host’s ONVIF media service drives: SetSynchronizationPoint should call GENERATE_KEYFRAME, a security configuration should switch securityProfile, and a multicast DESCRIBE returns the real group address. Conformance belongs to the device, not to this library.

What the ONVIF mode changes, and why: ONVIF Core sec. 5.9.3 makes MD5 the default RTSP digest, and nearly every deployed client implements only MD5. So the mode offers MD5, but only while the validated module is absent. If another instance in the same process has put the module in charge, MD5 is withdrawn automatically: a process running on the validated module must not answer challenges with a digest that the module does not contain.

Do not claim ONVIF SecureRTSPStreaming. Streaming sec. 5.1.1.4 requires MIKEY key exchange, and MServer does not advertise it. The pre-shared-key message is structurally complete, but it has never been checked against a reference peer, and an unverified a=key-mgmt makes a client fail the whole SETUP. SRTP itself works and is proved end to end (see the secure presentation). What is missing is the key exchange, not the media protection.

Nuances that decide a certification outcome

  1. A FIPS mode running without the module is a configuration, not a claim. Check the complianceMode string, not the requested profile.
  2. STRICT_FIPS is narrow on purpose. No SRTP, no WebRTC, no UDP unicast and no multicast: every non-interleaved transport is answered 461. On RTSP, media exists only as RTP interleaved inside TLS; the only other media path is HLS over HTTPS (see Verdict). Plan for that before choosing the mode.
  3. Every mode that requires TLS also refuses unprotected media. Withholding the plaintext listener only protects the control channel. Without more, a client authenticated over TLS could still ask for RTP/AVP;unicast or a multicast group and receive unencrypted video. In CRA_STRICT, FIPS_ONVIF_BRIDGE and STRICT_FIPS such a SETUP is answered 461. “Protected” means exactly one of two things: RTP interleaved inside the TLS control connection, or SRTP. Multicast is therefore unavailable in those modes, because a group is one datagram shared by every member and has no per-session key. A deployment that needs multicast must run ONVIF and protect the network segment by other means. The refusal is decided before a group is assigned. A refused SETUP therefore reserves no address and, more importantly, does not mark the group as transmitting; that would put plain RTP on the wire from a mode that promises never to send any.
  4. SRTP over RTSP needs the secure presentation. The keys travel in the SDP, where the standards put them. A client asks for rtsps://host/live?srtp, reads a=crypto, and then SETUPs RTP/SAVP. The reply names RTP/SAVP, and the media decrypts with the advertised key. A SETUP for RTP/SAVP on a connection that was not described SRTP is refused 461, because that client has no key. See the secure presentation. ONVIF SecureRTSPStreaming also requires MIKEY, which is built but not advertised, so that claim must still be withheld.
  5. Every strict mode refuses the direct RTP push leg, because it carries media with no TLS, no SRTP and no handshake. The VStreamer interface defaults directStreamEnable to true with a default destination, so a configuration that never mentioned the push leg still requests one. Set it to false explicitly under CRA_STRICT, FIPS_ONVIF_BRIDGE or STRICT_FIPS, or initialisation fails.
  6. The compliance statement is not exposed at runtime. getStats() reports the mode, the provider state and the coverage flag, but not the sentence. The four texts are reproduced below so they can be quoted directly.
  7. On a shared listener the most restrictive policy wins. Instances that bind the same port share one listener. Its effective policy combines the policies of all attached paths, so one strict stream tightens the digest and SRTP rules for the whole port. Credentials, by contrast, stay per path.
  8. An unrecognised securityProfile is refused, never defaulted. A typo cannot silently re-certify a device as the most permissive mode.
  9. hlsEnable defaults to true. The VStreamer interface sets that default, not MServer. A configuration that never mentioned HLS therefore still publishes an HLS stream, over plain HTTP unless hlsEncryption says otherwise. In a mode that requires TLS, the HLS listener needs a certificate (its own or the RTSP pair); without one, initialisation fails, which is fail-closed. In ONVIF the plain HTTP listener simply appears. Decide about HLS explicitly rather than inheriting the default.
  10. A generated TLS certificate is not a certified one. Where the profile permits it, a leg that asks for TLS with no material comes up on an identity that MServer generates in memory: encrypted, but not authenticated and not stable across a restart. An audit tells the two apart with getStats().tlsSelfSigned. The modes that require TLS refuse this fallback outright. See Certificates and keys.

The four compliance statements, verbatim

Copy these into the security documentation rather than paraphrasing them. They are the exact strings the library resolves (MServerCompliance.cpp).

ONVIF – “ONVIF interoperability mode. MD5 digest and AES-GCM SRTP are enabled; neither is FIPS-approved, so no FIPS claim applies to this configuration.”

STRICT_FIPS, module active – “Strict FIPS 140-3 mode. All cryptography is performed by the validated module; media travels as RTP interleaved inside TLS. MD5 digest and SRTP are refused.” Module absent – “Strict FIPS 140-3 CONFIGURATION on the default OpenSSL provider. The algorithm restrictions are enforced – MD5 digest and SRTP are refused, TLS and authentication are mandatory – but the validated module is NOT active, so NO FIPS 140-3 claim applies to this deployment.”

FIPS_ONVIF_BRIDGE, module active – “FIPS/ONVIF bridge. Only FIPS-approved primitives are used, and SRTP is limited to AES_CM_128_HMAC_SHA1_80. LIMITED CONFORMANCE: the SRTP key derivation (SP 800-135 s5.3) is not implemented by the validated module.” Module absent – “FIPS/ONVIF bridge CONFIGURATION on the default OpenSSL provider. Only FIPS-approved primitives are selected and SRTP is limited to AES_CM_128_HMAC_SHA1_80, but the validated module is NOT active, so NO FIPS 140-3 claim applies to this deployment.”

CRA_STRICT – “EU CRA secure-by-default mode. TLS and authentication are mandatory, MD5 digest is disabled, no plaintext listener is opened, and media is refused unless it is protected – SRTP, or RTP interleaved inside the TLS connection. SRTP remains available, so the media key derivation is outside the validated module and coverage is not claimed to be complete.”

Verdict

FIPS 140-3 – achievable on Linux, as a consumer of a FIPS 140-3 validated module (CMVP #4985, OpenSSL FIPS Provider 3.1.2), in STRICT_FIPS, once the open points below are closed. In this mode MServer performs no approved-mode cryptography outside the module. It proves that the module is enforcing, not merely loaded. It takes all entropy from the module’s DRBG. And it carries exactly one non-approved primitive (MD5), which is provably unreachable in this mode. The conditions are:

  • provision the module on the target, exactly as its Security Policy requires (the stock provider of a distribution is not the validated one);
  • run STRICT_FIPS;
  • check at start-up that complianceMode is the bare string and fullModuleCoverage is true;
  • document MD5 as a non-approved legacy interoperability function;
  • accept that the mode has no SRTP, no WebRTC and no UDP transport.

Open points a laboratory will raise (2.1.3):

  • The provider’s identity (name, version, build) is not compared with the certificate, and the entropy source of the platform needs its own evidence (ESV).

Closed in 2.1.3 (open in 2.1.2): the TLS and DTLS server contexts now offer ECDHE + AEAD suites and NIST groups only and drop a FIPS TLS/DTLS 1.2 handshake without the Extended Master Secret (IG D.Q); a failed DRBG call fails the request instead of leaving a digest nonce or a session identifier empty; and a FIPS mode refuses to start without the validated module instead of running with the claim withdrawn.

FIPS_ONVIF_BRIDGE is not a route to an unqualified claim. Its SRTP key derivation is outside the module by design, which is why the mode itself states limited conformance.

EU CRA – partly. CRA_STRICT gives a secure configuration: TLS and authentication mandatory, no plaintext listener, protected media only, bounded parsers, no UDP amplification and constant-time digest checks, provided the deployment makes the decisions in nuances 3 and 5 explicitly rather than by default. Open points of Annex I Part I: the default profile is ONVIF, with every leg of the interface defaults enabled and no authentication (2(b)), and a Windows build has no TLS or DTLS, so CRA_STRICT cannot start there. Closed in 2.1.3: a connection is dropped after 5 wrong-credential responses and a remote source may hold at most three quarters of the RTSP slots (2(d), 2(h)); getStats() reports authFailures and connectionsRefused as the security-event signals a deployment forwards to its log (2(l)). Annex I Part II in this repository (2.1.3): the SBOM sbom.cdx.json (Part II (1)), the coordinated vulnerability disclosure policy, the support period and the security-update channel in SECURITY.md (Part II (5), Article 13(8)) with the reporting process of Article 14, which applies since 11 September 2026, and the fixed issues in SECURITY-ADVISORIES.md. Executing the Article 14 notifications, the product threat model and the conformity assessment remain with whoever places a product on the market.

One nuance the HLS leg adds. HLS is the only media leg permitted in every mode, and the only one whose cryptography is entirely inside the module. So in STRICT_FIPS, which has no SRTP, no WebRTC and no UDP transport, HLS with AES-128 segments over HTTPS is a streaming path that carries an unqualified claim. The mode therefore offers more than interleaved RTSP. It does not change the identity rule: the generated TLS certificate is refused in every mode that requires TLS, so a strict deployment still installs its own certificate.

The one claim still to withhold is ONVIF SecureRTSPStreaming, and only because of MIKEY. MServerStrictModesTest exercises the SRTP media path end to end. It takes the master key from the SDP, derives the session keys with its own RFC 3711 implementation, verifies every HMAC-SHA1-80 tag and recovers the H.264. So FIPS_ONVIF_BRIDGE’s media path is backed by evidence, not an assertion. What is missing is the MIKEY exchange that ONVIF Streaming sec. 5.1.1.4 mandates. The message is built but has never met a reference peer, so it is not advertised, and the ONVIF secure-streaming claim stays unmade until it has.

MServer class description

MServer class declaration 

class MServer : public VStreamer
{
public:

    /// Get library version.
    static std::string getVersion();

    /// Init video server.
    bool initVStreamer(VStreamerParams& params, VCodec* codec = nullptr,
                       VOverlay* overlay = nullptr) override;

    /// Check initialization status.
    bool isVStreamerInit() override;

    /// Close video server.
    void closeVStreamer() override;

    /// Send frame for streaming.
    bool sendFrame(Frame& frame, uint8_t* userData = nullptr,
                   int userDataSize = 0) override;

    /// Set parameter.
    bool setParam(VStreamerParam id, float value) override;

    /// Set parameter.
    bool setParam(VStreamerParam id, std::string value) override;

    /// Get all parameters.
    void getParams(VStreamerParams& params) override;

    /// Execute action command.
    bool executeCommand(VStreamerCommand id) override;

    /// Decode a command datagram and execute it (the interface's default).
    bool decodeAndExecuteCommand(uint8_t* data, int size) override;

    /// Get statistics.
    struct Stats { /* ... */ };
    void getStats(Stats& stats) const;
};

Exceptions. The methods that return bool answer an allocation that fails (std::bad_alloc) with FALSE instead of letting the exception out; initVStreamer() – and with it RESTART and a container change – also answers a thread that cannot be started, and stops and releases whatever it had already started. getParams() and getStats(), which return nothing, can still pass on std::bad_alloc from copying the parameter strings. No exception leaves a pipeline thread: an allocation that fails there, or an exception from the caller’s codec or overlay, costs the frame in hand and the one waiting behind it, and the thread carries on. The reactor and sender threads of the RTSP listener have no such barrier: an exception there ends the process.

getVersion method

The getVersion() method returns a string of the current version of the MServer class. Method declaration:

static std::string getVersion();

The method can be used without an MServer class instance:

std::cout << "MServer class version: " << cr::video::MServer::getVersion();

Console output:

MServer class version: 2.1.2

Returns: the library version as a string, "Major.Minor.Patch".

initVStreamer method

The initVStreamer(…) method initialises the streamer: it validates the whole parameter structure, binds the listeners the configuration asks for, starts the pipeline threads and publishes the stream path. Method declaration:

bool initVStreamer(VStreamerParams& params,
                   VCodec* codec = nullptr,
                   VOverlay* overlay = nullptr) override;
Parameter Value
params VStreamerParams class object. Every field MServer reads is listed in Parameters. Fields for protocols MServer does not serve are stored unchanged, so a getParams() round trip is faithful. Values are range-checked, not clamped: an out-of-range port, size, rate or enumeration fails the call rather than being silently corrected. The one exception is custom1, which is clamped. metadataSuffix is normalised in the structure you pass.
codec Pointer to a VCodec object that encodes RAW input frames. Required whenever RAW frames will be supplied. A caller that feeds already-compressed H.264, HEVC or JPEG frames may pass nullptr: such frames bypass the encoder entirely. MServer keeps the pointer for the life of the stream, so the object must outlive the MServer instance, or be released only after closeVStreamer(). closeVStreamer() lets go of it: no later call reaches the object – not setParam(), not GENERATE_KEYFRAME – and a RESTART of the closed instance runs without it until it is passed to initVStreamer() again.
overlay Pointer to a VOverlay object that draws over RAW frames when overlayEnable is TRUE. With nullptr nothing is drawn, whatever overlayEnable says. Compressed input is never overlaid, because drawing on it would require a decode. Same lifetime rule as codec.

Returns: TRUE if the streamer was initialised, or FALSE if it was not. It fails when:

  • a parameter is out of range;
  • the codec or securityProfile name is not recognised;
  • a strict compliance mode cannot be honoured by the configuration: TLS or credentials are missing, or WebRTC or the direct push leg is requested where the mode forbids them;
  • a certificate or key was named but cannot be loaded. The exception is the RTSPS leg in a profile that permits the generated identity: it writes a new pair to those paths instead;
  • encryption is asked for with no material at all and the compliance profile forbids the generated identity;
  • metadataSuffix names a format MServer cannot describe. Only SMPTE336M and VND.ONVIF.METADATA are accepted, plus the aliases "", no and metadata;
  • the stream suffix is empty, or already claimed by another instance on the shared RTSP listener, the WebRTC endpoint or the HLS listener. A refused instance leaves the owner’s credentials untouched;
  • bindAddress is not an address of this host, or does not match a shared listener that is already bound, or the RTP port range is unusable;
  • a listening port cannot be bound – including the HLS port and the WebRTC ports, which hlsEnable and webRtcEnable open by default;
  • the codec and container are incompatible: JPEG with any MPEG-TS container, for which STANAG 4609 and MISB ST 1402 define no stream type;
  • on Windows, a leg asks for TLS (see Windows build with MSVC);
  • memory runs out, or a thread cannot be started. Everything the call had started is stopped and everything it had claimed is released before it returns.

MServer writes no log, so FALSE is the only signal. Three causes catch people out most often: the HLS port (see Troubleshooting), metadataSuffix left at a path-like string, and a strict profile with directStreamEnable at its interface default of TRUE.

Threads. sendFrame(), setParam(), getParams() and getStats() may be called at the same time from different threads. initVStreamer() and closeVStreamer() start and stop the threads the other methods rely on, so they are not safe against the other methods or against each other. Call them from one thread, while no other thread uses the instance. executeCommand(RESTART) and a setParam(SERVER_STREAM_TYPE) that changes the container re-initialise the instance in place. Serialise them with each other and with the two calls above; the other methods stay safe meanwhile. sendFrame() returns FALSE until the stream is up again, a value set meanwhile may be overwritten by the stored parameters the re-initialisation applies, and a SECURITY_PROFILE switch is refused.

isVStreamerInit method

The isVStreamerInit() method returns the initialisation status. Method declaration:

bool isVStreamerInit() override;

Returns: TRUE if the streamer is initialised, or FALSE if it is not.

closeVStreamer method

The closeVStreamer() method stops the streamer. It first gives a frame already in the pipeline up to 200 ms to reach the wire. Then it closes every session, stops the pipeline and sender threads, releases the listening ports this instance holds, and removes its path from the shared listeners. It also lets go of the codec and overlay given to initVStreamer(): once it has returned, the caller may release them. It is safe to call on an instance that was never initialised, and safe to call twice. The destructor calls it. Method declaration:

void closeVStreamer() override;

sendFrame method

The sendFrame(…) method hands one frame to the streamer. To produce a video stream the caller must call it for every frame coming from the video source. Method declaration:

bool sendFrame(Frame& frame, uint8_t* userData = nullptr,
               int userDataSize = 0) override;
Parameter Value
frame Frame class object. Accepted formats are listed below. The frame is copied and the method returns at once, so the caller may reuse or destroy the buffer immediately. Nothing here waits for the network or the encoder. Its geometry cannot make the copies larger than the buffer warrants: a RAW frame must carry at least one byte per pixel of width x height in size, which every RAW format needs, and a compressed unit is copied into a buffer sized by its size, whatever width and height say. A compressed frame may claim at most 536 870 911 pixels, and its size may be at most four bytes per pixel.
userData Pointer to user data (telemetry) for this frame: MISB ST 0601 KLV, or an ONVIF metadata document, according to metadataSuffix. The caller encodes the bytes. MServer forwards them byte-exact and never re-encodes them; only the Restamp KLV mode rewrites Tag 2 and Tag 1. nullptr sends no telemetry with this frame. See How to extract KLV from the stream.
userDataSize Size of the user data in bytes. 0 sends no telemetry with this frame. sendFrame() sets no size limit: a unit larger than one RTP packet is fragmented, with the marker bit on the last packet. The MPEG-TS legs have a limit of their own; see MPEG-TS and KLV metadata.

Accepted input formats. Two families, handled quite differently:

Input Handling
RAW: YUV24, NV12, NV21, YU12, YV12, RGB24, BGR24, YUYV, UYVY, GRAY Converted, scaled and overlaid as the parameters require, then encoded. Needs the codec given to initVStreamer().
Compressed: H264, HEVC, JPEG Passed straight through to the packetiser. No encoder, no scaler and no overlay – all three would require a decode first. width, height, fitMode and overlayEnable are therefore ignored for such input, and codec may be nullptr.

Which RAW format to send. Short answer: with an encoder in use, send YUV24, unless you know the frame needs neither scaling nor overlay; then send NV12. Any of the ten formats above is accepted either way. This is about cost per frame, not correctness. There are two answers because the pipeline has two routes:

Situation Send this Why
Scaling or overlay is in use (width/height differ from the source, or overlayEnable is on with an overlay object) YUV24 The scaler and the overlay accept only 3-byte interleaved data, so the pipeline works in YUV24: the route is RAW → YUV24 → resize → overlay → NV12. Supplying YUV24 skips the first conversion.
Neither is in use, and the frame is already at the stream resolution NV12 The encoder takes NV12, and FormatConverter reaches it from any RAW format in one pass. Supplying NV12 at the stream size means no conversion and no copy at all – the frame goes to the encoder untouched.

Neither is a requirement: any listed format is accepted in either situation. Sending RGB24 into a scaling pipeline simply costs one extra conversion per frame.

Returns: TRUE if the frame was accepted, or FALSE if it was not. FALSE means the streamer is not initialised, or frame.data is nullptr, or frame.size is not positive, or width or height is not positive or the geometry does not fit the buffer (see frame above), or memory for the copy ran out. TRUE does not mean the frame was encoded or sent. A frame is legitimately discarded when the stream is disabled (enable is FALSE), when nobody is watching (see Idle streams cost nothing), or when a newer frame replaces it in the input slot, for example because the source runs faster than fps. getStats() counts the idle case in framesIgnored and replaced frames in framesDropped. Frames sent while the stream is disabled are not counted at all.

setParam method

The setParam(…) method changes one parameter on a running or a not-yet-initialised streamer. Two overloads exist because the interface carries both numeric and string parameters. Method declaration:

bool setParam(VStreamerParam id, float value) override;
bool setParam(VStreamerParam id, std::string value) override;
Parameter Value
id Parameter identifier from the VStreamerParam enumeration. Use the numeric overload for numeric parameters and the string overload for string ones; the wrong overload for a given id is rejected.
value New value. Which parameters take effect at once, which are stored until the next initialisation, and which rebuild the stream is set out in When a parameter takes effect, and how far it reaches.

Returns: TRUE if the parameter was accepted, or FALSE if it was not. Everything that bounds an allocation or names a port is range-checked: sizes, frame rate, GOP, bitrates, quality, ports and the mode enumerations. Silently accepting such a value would be an unbounded allocation primitive (EU CRA Annex I 2(h)). The multicast port must also be even, because RTCP uses port+1. A value is rejected, never clamped: a caller that asks for something impossible is told so, not given something else.

FALSE therefore means one of these:

  • the value is out of range;
  • the id is not recognised, or was given through the wrong overload;
  • the parameter is output-only (CYCLE_TIME_USEC) or names a listener that does not exist (METADATA_PORT);
  • the running configuration cannot honour the change, for instance a switch to a compliance mode that demands TLS on a stream that has no TLS listener (all the cases are listed under Applied immediately);
  • memory ran out.

Parameters for protocols MServer does not serve are accepted and stored. A setParam() / getParams() round trip therefore returns exactly what the caller set, even though nothing is started for them.

getParams method

The getParams(…) method returns the streamer’s current parameters. Method declaration:

void getParams(VStreamerParams& params) override;
Parameter Value
params VStreamerParams object to fill. Every field is overwritten.

Some fields need attention:

  • custom1..3 come back unmodified. A getParams() → initVStreamer() round trip must not corrupt them, so MServer writes nothing of its own into them. custom1 holds the preprocessing thread count, but it is stored exactly as given (not normalised, not clamped), so the round trip is faithful there too.
  • cycleTimeUs is measured, not set: it is the observed interval between encoded frames, as the interface defines it. setParam() refuses to write it.
  • rtspPort reports the port that is actually bound, which is not always the one requested. All instances in a process share one plaintext listener, and the first port bound wins. An instance that asked for a different port is told the truth here. See Reach: the stream, or the whole process.
  • metadataSuffix reports the format in force: "", no and metadata come back as SMPTE336M.

getStats method

The getStats(…) method returns the runtime counters. It is an MServer extension. The VStreamer interface has no way to read state back, and putting statistics into the custom fields would corrupt the parameter round trip described above. Method declaration:

void getStats(Stats& stats) const;
Parameter Value
stats MServer::Stats object to fill. Every field is overwritten. Counters are per instance (per path). initVStreamer() resets the pipeline and KLV counters (frames*, accessUnits, rtpPackets, klvUnits, klvRejected). The push-leg counters (directPackets, directSendFailures) count for the life of the instance. The RTSP counters (rtpSendFailures and the feedback counters) are kept per path by the listeners, so initVStreamer() does not necessarily reset them. The compliance fields are read from live state.

The structure:

| Field | Meaning | |——-|———| | framesIn / framesDropped / framesEncoded | pipeline counters. framesDropped includes the frames superseded in the input slot while the source runs faster than fps | | framesDuplicated | pictures repeated to hold fps while the source ran slower than it. A stream whose source matches its fps never duplicates; a rising count means the source is short of the rate the SDP advertises. Reset by initVStreamer(), like the other pipeline counters | | framesIgnored | frames skipped because nobody was watching – see Idle streams cost nothing | | idle | TRUE while input is being ignored for want of a consumer | | accessUnits / rtpPackets | published units and packets produced. A unit the payload format cannot carry – a JPEG picture RFC 2435 cannot describe (above 2040 pixels, or a size that is not a multiple of 8) – is not counted: framesEncoded rising while accessUnits stands still means every frame is being refused | | klvUnits / klvRejected | KLV accepted, and KLV not carried: refused by validation, or larger than one MPEG-TS PES can carry on a leg that muxes MPEG-TS – counted once per unit; such a unit still goes out on a codec-RTP metadata track and is counted in klvUnits too. See MPEG-TS and KLV metadata | | clients | consumers of this stream: its RTSP and RTSPS sessions in PLAYING state, its WebRTC sessions whose DTLS handshake has completed, its multicast group if one is transmitting (a group counts as one consumer with no session behind it), and 1 while HLS has a viewer (a playlist, segment or key fetched within two target durations). Per path, not per process: a client on another stream, or the page watching a different camera, is not counted here | | hlsSegments | HLS segments closed and published since the stream started. A stream serving HLS with this stuck at its previous value has stopped producing key frames – a segment can only close on one – or is running JPEG, for which the leg produces none by design | | hlsBytesServed | bytes of playlist and segment handed to HLS clients. Playlists are counted when rendered, segments when handed to the reactor | | directPackets | video and transport-stream packets queued for the direct push leg (its metadata packets are not counted). Queued, not necessarily sent: a queued packet can still be dropped on overflow, or refused by the kernel (directSendFailures) | | rtpSendFailures / directSendFailures | datagrams the kernel refused, on this stream’s RTSP legs and on its push leg. Each one is a hole in a sequence space the receiver reports as loss – the push leg cannot retransmit it, and an RTSP session repairs it only when its client asks with a NACK – so a non-zero value is the server’s own contribution to a client’s “missed packets”, not the network’s. The RTSP figure includes this path’s departed subscribers, so it never falls. directSendFailures also counts the packets the push queue dropped on overflow (whole access units, see The direct push leg): those leave no hole in the sequence numbers, which are stamped as a packet is sent, and show on the wire only as missing frames | | complianceMode | active mode name, with ` (unvalidated provider) appended when a FIPS mode (STRICT_FIPS, FIPS_ONVIF_BRIDGE) finds itself without the validated module (since 2.1.3 such a mode does not start, so the suffix is not seen on a running stream; it stays defined for a provider lost after the start). A bare STRICT_FIPS together with fullModuleCoverage TRUE is the only pair that carries a validated claim, so never compare the whole field for equality without accounting for the suffix | | fipsActive | FIPS provider in use | | fullModuleCoverage | FALSE means any FIPS claim must be qualified | | tlsSelfSigned | TRUE when at least one TLS leg of this stream came up on the **ephemeral in-memory identity** MServer generated for itself, because encryption was asked for and no certificate was supplied. The channel is encrypted but not authenticated -- see [Certificates and keys](#certificates-and-keys) | | tlsFingerprint | SHA-256 fingerprint of that identity, colon-separated upper-case hex; empty unless tlsSelfSigned is TRUE`. What an operator pins, and what lets them confirm the certificate their client saw is the one this server generated |

The structure has four more fields. Three count feedback on this stream’s RTSP and RTSPS sessions: keyFrameRequests (key-frame requests: PLI and FIR passed to the encoder after the 200 ms per-session limit and the 200 ms per-path one; the key frames asked for at PLAY and after a backlog drop are not counted), nackRequests (NACK entries received) and rtxPacketsSent (retransmissions sent). tlsGeneratedToDisk is TRUE when the RTSPS leg generated an identity and wrote it to the certificate and key paths you named (see When no certificate is configured).

executeCommand method

The executeCommand(…) method executes one command on the streamer. Method declaration:

bool executeCommand(VStreamerCommand id) override;
Parameter Value
id Command identifier from the VStreamerCommand enumeration.
Command Effect
ON Resume streaming. Equivalent to setting enable to TRUE: frames handed to sendFrame() are processed again.
OFF Stop streaming without closing anything. Listeners stay bound and sessions stay open; frames are accepted and discarded.
RESTART Rebuild the stream with the parameters currently held: the teardown closeVStreamer() performs, keeping the encoder and the overlay, followed by the same sequence initVStreamer() performs. On an instance already closed with closeVStreamer(), which let go of both, it rebuilds without them. Clients are dropped and must reconnect. Everything a re-initialisation resets (the pipeline counters, the sequence spaces, the multicast assignments) is reset.
GENERATE_KEYFRAME Ask the encoder for a key frame with fresh parameter sets on the next frame. This is what a host’s ONVIF SetSynchronizationPoint should call. Measured latency from the command to a complete IDR on the wire: 19 ms at 25 fps with a GOP of 3000, i.e. where no key frame was due.

Returns: TRUE if the command was accepted, or FALSE if it was not. RESTART returns what the re-initialisation returns. GENERATE_KEYFRAME returns the encoder’s answer, and FALSE when there is no encoder to ask: either none was given to initVStreamer(), or the pipeline is in pass-through because the caller supplies already-compressed frames. In that case the key frames are whatever the source produces.

Parameters

MServer implements the VStreamer interface, so its whole configuration surface is the interface’s own: the VStreamerParams structure passed to initVStreamer(), the VStreamerParam enumeration used by setParam(), and the VStreamerCommand enumeration used by executeCommand().

The interface covers a family of streamers, so it declares parameters for protocols MServer does not serve. Those are listed below and marked not used: setParam() accepts and stores them so a setParam() / getParams() round trip returns exactly what the caller set, but nothing is started for them. One parameter is rejected outright rather than stored, because accepting it would imply a listener that does not exist.

VStreamerCommand

executeCommand() takes one of four values. All four are implemented.

Command Value Description
RESTART 1 Stop and start the streamer with the current parameters. Client sessions are dropped and the shared listener is re-attached.
ON 2 Enable streaming. Equivalent to MODE = 1: frames are encoded and published again.
OFF 3 Disable streaming. Equivalent to MODE = 0: sendFrame() still accepts frames and discards them, sessions stay open, no media is published.
GENERATE_KEYFRAME 4 Force the encoder to emit a key frame with fresh parameter sets on the next frame. This is what a host’s ONVIF SetSynchronizationPoint should call. Returns FALSE when no encoder is attached, and also when the pipeline is in pass-through (the caller is supplying already-encoded frames, so there is no encoder to ask and the key frames are whatever the source produced).

VStreamerParam

setParam() has a float overload and a string overload; each parameter accepts one or the other. The Range column is what MServer actually enforces: a value outside it makes setParam() return FALSE and changes nothing. The When it takes effect column answers the question most callers have:

  • Live: the running stream follows the change at once; clients keep their sessions, and no RESTART is needed.
  • Next initVStreamer(): the value is stored and read the next time the streamer is initialised.
  • Stored only: MServer never acts on the value.
  • Read-only: setParam() refuses it, and only getParams() reports it.
Parameter Type Range enforced by MServer When it takes effect Description
MODE int 0 or 1 Live Streamer enable / disable. 0 keeps the pipeline and the sessions alive but publishes nothing.
WIDTH int [8:4096] Live Stream width. Applied live: the encoder is reconfigured and clients keep their session.
HEIGHT int [8:4096] Live Stream height. Applied live, as WIDTH.
DIRECT_STREAM_IP string any Live Destination of the direct RTP push, applied live: the destination list is rebuilt and the running leg follows without a restart. A comma-separated list sends the same packets to several receivers; entries that are not IPv4 literals are dropped.
RTSP_PORT int [500:65534] Re-binds the shared listener RTSP port. Shared by every MServer instance in the process, so a change re-binds the listener for all of them. The new value is stored only when the re-bind succeeds; a port that cannot be bound is refused and the old one stays.
RTSPS_PORT int [1024:65535] Next initVStreamer() RTSPS (RTSP over TLS) port. Takes effect at the next initVStreamer().
DIRECT_STREAM_PORT int [1024:65535] Live Destination port of the direct RTP push, applied live (the destination list is rebuilt inside setParam()). Its RTCP Sender Reports go to port+1 only while the leg is enabled and carries RTP; a raw mpegts-klv-* leg has no RTP session and sends none.
WEBRTC_PORT int [1024:65534] Next initVStreamer() WebRTC signalling port; the media port is this + 1 unless WEBRTC_MEDIA_PORT names one, which is why 65535 is out of range. Read at the next initVStreamer(). The endpoint is shared by the whole process: while another instance still holds it, an initialisation asking for a different port fails rather than moving the listener out from under the streams already using it.
HLS_PORT int [1:65535] Next initVStreamer() TCP port of the HLS listener. Shared by every instance in the process. While this stream is published on the HLS listener, a different value is refused, because the socket cannot move under it.
SRT_PORT int [1:65535] Stored only Not used – MServer serves no SRT. The value is range-checked and kept so a setParam() / getParams() round trip returns what you set, but no SRT listener is ever started.
RTMP_PORT int [1:65535] Stored only Not used – MServer serves no RTMP. Range-checked and kept, as SRT_PORT.
RTMPS_PORT int [1:65535] Stored only Not used – MServer serves no RTMPS. Range-checked and kept, as SRT_PORT.
METADATA_PORT int – Rejected Rejected. setParam() returns FALSE. There is no separate metadata listener: KLV travels inside the media stream, so a port for it would be a promise MServer cannot keep.
RTSP_MODE int 0 or 1 Next initVStreamer() RTSP protocol enable / disable. Takes effect at the next initVStreamer().
DIRECT_STREAM_ENABLE int 0 or 1 Live Direct RTP push on/off, applied live. 0 stops the leg with the next packet and drops what is still queued. 1 starts it at once, and builds the socket, the sequencers and the sender thread if initialisation did not. (initVStreamer() builds the leg whenever DIRECT_STREAM_IP and DIRECT_STREAM_PORT name a destination, even when the leg is disabled; that is why switching needs no restart.) setParam() returns FALSE for 1 under a compliance mode that requires TLS, whether or not the leg was built: the leg is plaintext. Otherwise it returns FALSE only when the leg must be built from nothing and cannot be: no destination is configured, or the leg cannot be built (no socket, a stream type or pacer setting it refuses, no sender thread). Once its socket exists, switching it off and on again just rebuilds the destination list and succeeds. Switching it on again closes the key-frame gate and requests a key frame, so the receiver starts on a key frame, not on P-frames it cannot decode.
WEBRTC_MODE int 0 or 1 Next initVStreamer() WebRTC enable / disable. Takes effect at the next initVStreamer().
HLS_MODE int 0 or 1 Off live, on needs a restart Publishes or withdraws this stream on the HLS listener. Switching it off takes effect at once. Switching it on is refused while it is off, and the refused value is not stored: opening a listener needs the compliance decision and the TLS material that only initialisation has, so it takes a restart, the same as enabling RTSPS.
SRT_MODE int 0 or 1 Stored only Not used. Stored only.
RTMP_MODE int 0 or 1 Stored only Not used. Stored only.
METADATA_MODE int 0 or 1 Stored only Not used. Stored only; SERVER_STREAM_TYPE selects KLV carriage.
RTSP_MULTICAST_IP string empty (off), or a multicast base with an optional /prefix in [4:31] (/32 = one address) Live Multicast group or pool of this path, applied live. A /prefix makes it a pool, a bare address a fixed group. The value is validated as it is set: a base outside 224.0.0.0/4, or an unusable prefix, makes setParam() return FALSE and the previous value is kept – nothing is stored and silently ignored. The check is deferred only while no multicast port is configured yet. See Multicast addressing.
RTSP_MULTICAST_PORT int [1024:65534], even Live Multicast RTP port, applied live. Must be even because RTCP uses port+1 (RFC 3550 sec. 11).
USER string any Live User name for digest authentication on every leg of this stream (RTSP, RTSPS, WHEP, HLS), applied live. "" or "no" disables authentication. Refused when the active compliance profile requires authentication and the new value would switch it off ("" or no): a parameter change cannot withdraw the profile’s guarantee.
PASSWORD string any Live Password for the same legs, applied live. Refused when the active compliance profile requires authentication and the new value would switch it off ("" or no), for the same reason.
SUFFIX string non-empty; before initVStreamer() it is only stored Live Stream name, i.e. the URL path. Applied live: the stream moves to the new path with its RTSP/RTSPS credential record, its 90 kHz RTP clock, its send budget and its multicast policy; RTSP sessions on the old path are dropped, and WebRTC viewers follow the stream to the new name. Returns FALSE and leaves the stream where it was when the new name is already claimed on the shared listener or refused by the WebRTC endpoint. The WHEP credential is keyed by the published path and is not re-keyed by the rename.
METADATA_SUFFIX string SMPTE336M, VND.ONVIF.METADATA Live Metadata format, not a URL path. SMPTE336M is MISB ST 0601 KLV (RFC 6597 encoding name smpte336m); VND.ONVIF.METADATA is an ONVIF metadata stream. On the codec-RTP paths the value is what the SDP’s a=rtpmap:98 names, which is how a client learns the format; see what each transport carries. "", no and the interface’s legacy default metadata all select SMPTE336M. Any other value is rejected.
MIN_BITRATE_KBPS int [0:1000000] Live Minimum bitrate for variable-bitrate mode; 0 means no floor. Forwarded to the encoder.
MAX_BITRATE_KBPS int [1:1000000] Live Maximum bitrate for variable-bitrate mode. Forwarded to the encoder.
BITRATE_KBPS int [1:1000000] Live Target bitrate. Forwarded to the encoder.
BITRATE_MODE int 0 or 1 Live 0 – constant bitrate, 1 – variable. Forwarded to the encoder.
FPS float [1:240] Live Frame rate, applied live. Forwarded to the encoder, published in the SDP (a=framerate, so a later DESCRIBE sees the new value) and used as the rate the governor holds: a frame that arrives early waits in the hand-off slot, and a missed deadline repeats the last picture. It is not the pacing budget – that comes from BITRATE_KBPS / MAX_BITRATE_KBPS. See Holding the configured frame rate.
GOP int [1:65535] Live GOP size (the key-frame interval). Forwarded to the encoder. A new RTSP, RTSPS or WebRTC client does not wait for the next scheduled key frame: PLAY and a completed WebRTC handshake each ask the encoder for one. With pre-encoded input there is no encoder to ask, and a new client may wait up to one GOP.
H264_PROFILE int [0:2] Live 0 – baseline, 1 – main, 2 – high. Forwarded to the encoder.
JPEG_QUALITY int [1:100] Live JPEG quality in percent. Forwarded to the encoder.
CODEC string H264/AVC, H265/HEVC, JPEG/MJPEG Live Codec, matched case-insensitively. Applied at the next access-unit boundary: the packetizer, the payload type, the SDP and the encoder’s own output format change together. JPEG cannot be combined with an MPEG-TS container, because the muxer has no stream type for it. The pair is refused on a live change as well as at initialisation, on either leg’s container. (Otherwise JFIF bytes would go out under a PMT that declares H.264, 0x1B.) Move the container off MPEG-TS first.
FIT_MODE int 0 or 1 Live 0 – fit (letterbox, aspect preserved), 1 – fill (stretch). Applied live.
CYCLE_TIME_USEC int – Read-only Read-only. Reported by getParams() as the measured pipeline cycle; setParam() does not accept it.
OVERLAY_MODE int 0 or 1 Live Overlay enable / disable. Applied live.
TYPE int whole number in [0:255] Live Streamer type, forwarded to the encoder as its backend selector (0 – hardware, 1 – software for VCodecLibav).
CUSTOM1 float [0 : hardware threads] Live Compute threads for preprocessing: the thread limit for the format converter and the scaler. 0, the default, means min(2, hardware threads); 1 means one thread; n means up to n. The two entry points differ: setParam() range-checks the request and refuses anything above the machine’s thread count. initVStreamer() does not range-check it and clamps instead, so a large value passed at initialisation silently gets a worker team as wide as the machine. Applied live: the preprocess thread picks it up on its next frame. Stored and returned by getParams() unmodified. Before v3.2 of the interface this field carried the multicast TTL; use MULTICAST_TTL for that. See Compute threads for preprocessing.
CUSTOM2 float any Stored only Not used by MServer. Opaque caller data, returned by getParams() unmodified. Carried the compliance mode before v3.2; that hijack is gone – use SECURITY_PROFILE.
SECURITY_PROFILE string ONVIF, STRICT_FIPS, FIPS_ONVIF_BRIDGE, CRA_STRICT; ""/no – default Live Compliance mode. Matched case-insensitively. An unrecognised value is rejected, never defaulted. A switch is refused while the stream is not running and when the running configuration cannot honour the mode; see Applied immediately and Security and compliance modes.
MULTICAST_TTL int [0:255] (0 is programmed as 1), programmed as at most 32 Live How far a multicast stream is allowed to travel, in router hops. 1 (the default) keeps it on the local link. Values above 32 are accepted but programmed as 32 on every leg – 32 is the administrative-scope boundary, and letting a stream past it is how a camera ends up on somebody else’s network. A live change reaches every leg: the served groups and the push leg’s socket. Interface v3.2 requires an implementation to document its cap; this is MServer’s.
PUBLIC_ADDRESS string IPv4 literal, ""/no – auto Next initVStreamer() Address advertised in SDP and ICE candidates instead of the detected one. Validated when it is set: anything that is not an IPv4 literal (or ""/no) makes setParam() return FALSE and changes nothing, so a host name is refused by the command instead of taking the stream down at the next restart. The accepted value is read at the next initVStreamer(), because it is written into candidate lines that sessions already hold; a bad value handed straight to initVStreamer() still fails initialisation.
BIND_ADDRESS string local IPv4 literal, ""/0.0.0.0 – all Next initVStreamer() Interface every listener binds to. Validated when it is set, exactly as strictly as initVStreamer() validates it: a hostname, an unparsable value, or an address this host does not hold makes setParam() return FALSE rather than being stored and quietly falling back to every interface. The accepted value is read at the next initVStreamer() – a bound socket cannot be moved under its sessions.
RTP_PORT_MIN int [0:65534] Next initVStreamer() Lowest server RTP/RTCP port. 0 – ephemeral.
RTP_PORT_MAX int [0:65535] Next initVStreamer() Highest server RTP/RTCP port. 0 – ephemeral. Must be >= RTP_PORT_MIN + 1 when RTP_PORT_MIN is set. Set both or neither: a range needs max >= min + 1, because RTCP takes the odd port beside each RTP one. A max on its own is accepted by setParam() and then ignored at initialisation, since min of 0 still means “any ephemeral port”.
WEBRTC_MEDIA_PORT int [0:65535] Next initVStreamer() WebRTC media port. 0 keeps the convention of WEBRTC_PORT + 1.
CORS_ALLOWED_ORIGIN string one origin (a wildcard * is refused), ""/no – default policy Live (WHEP) Origin permitted on the WHEP signalling endpoint; applied live there. The process-wide HLS listener takes the value of the instance that starts it, at that instance’s initVStreamer().
SERVER_STREAM_MAX_PAYLOAD int [576:9000] Next initVStreamer() Maximum RTP payload of the served stream. Read at the next initVStreamer(), and again on a live codec change. Clamped to 1200 bytes whenever the WebRTC leg is enabled with a port set (the default), because one packetization serves the served and WebRTC legs; see Scope and limits of the WebRTC leg.
CUSTOM3 float any Stored only Not used by MServer. Free for the application: stored and returned by getParams() unmodified.
RTSP_KEY string any Next initVStreamer() Path to the private key for RTSPS. Read at initVStreamer().
RTSP_CERT string any Next initVStreamer() Path to the certificate for RTSPS. Read at initVStreamer().
WEBRTC_KEY string any Next initVStreamer() Path to the private key for HTTPS signalling. Read at initVStreamer(); falls back to RTSP_KEY when unset.
WEBRTC_CERT string any Next initVStreamer() Path to the certificate for HTTPS signalling. Read at initVStreamer(); falls back to RTSP_CERT when unset.
HLS_KEY string path or no Next initVStreamer() TLS private key for the HLS listener. Read only when HTTPS was asked for; the RTSP pair is the fallback.
HLS_CERT string path or no Next initVStreamer() TLS certificate for the HLS listener.
RTMP_KEY string any Stored only Not used. Stored only.
RTMP_CERT string any Stored only Not used. Stored only.
RTSP_ENCRYPTION string no, optional, strict Next initVStreamer() TLS policy, selecting which listeners open: no – plaintext only; optional – both, so an rtsp:// and an rtsps:// client are served side by side; strict – RTSPS only, and initialisation fails if the TLS context cannot be built. Any other value is rejected. Note that setParam() accepts only the three names: the empty string is refused here even though initVStreamer() reads it as no, so clear the field with no rather than "".
WEBRTC_ENCRYPTION string no, optional, strict Next initVStreamer() TLS policy for signalling (WHEP over HTTP/HTTPS). The media leg is always DTLS-SRTP. optional and strict behave the same: both ask for HTTPS, and initialisation fails if TLS cannot be set up. A WHEP certificate turns HTTPS on even with no; see Configuring it. "yes" is rejected: it is not one of the three values. Note that setParam() accepts only the three names: the empty string is refused here even though initVStreamer() reads it as no, so clear the field with no rather than "".
RTMP_ENCRYPTION string any Stored only Not used. Stored only.
HLS_ENCRYPTION string no, yes, optional, strict Next initVStreamer() What is protected. yes is AES-128 on the segments; optional and strict ask for HTTPS; strict is both. The two are independent – segment encryption needs no certificate. Any other value is refused, by setParam() and by initVStreamer() alike, so a typo cannot be stored and then read as “plaintext, and no segment encryption either”. Read at the next initialisation. Note that setParam() accepts only the four names: the empty string is refused here even though initVStreamer() reads it as no, so clear the field with no rather than "".
LOG_LEVEL int [0:3] Stored only Not used. Stored only: MServer writes no log of its own, it reports through getStats() and the return codes.
DIRECT_STREAM_TYPE string see Stream type values Live Container and framing of the direct push leg, independent of SERVER_STREAM_TYPE and switchable live: unlike the served leg, a container change here needs no re-initialisation. The framing (raw MPEG-TS in datagrams versus TS inside RTP), the KLV signalling and the leg’s payload type (33 for a transport stream, the dynamic type otherwise) are re-derived on the spot, and the receiver is given a clean start: the packets still queued for the old framing are dropped, a key frame is requested and waited for, and on MPEG-TS it comes with the PAT and PMT. The leg packetizes for itself when its container, KLV signalling or payload size differ from the served leg’s, and then its packetizer, TS muxer and PSI tables are rebuilt from scratch. JPEG cannot be combined with an MPEG-TS container, and an unknown value is rejected.
DIRECT_STREAM_BITRATE_KBPS int [1:1000000] Live Channel bandwidth for the push leg’s packet pacer, in kbit/s. Honoured when DIRECT_STREAM_PACING_MODE is 0: a token bucket refilled at this rate, with a quarter-second burst allowance, so a burst from the encoder leaves the interface at the rate the channel was told it can carry. It has to be above the stream’s bitrate: what the stream produces above it waits in the push queue, whose latency then grows to about 2 s before the oldest access units are dropped, whole (see The direct push leg).
DIRECT_STREAM_MAX_PAYLOAD int [576:9000] Next initVStreamer() Maximum RTP/UDP payload of the push leg’s own packetizer; it never shrinks the packets the served leg fans out (initVStreamer() and the live codec-change path size the shared packetizer from SERVER_STREAM_MAX_PAYLOAD alone, capped only by the WebRTC limit). Read at the next initVStreamer(), and re-read at once by a live DIRECT_STREAM_TYPE change or when the leg is enabled at runtime; on its own the field only stores the value.
DIRECT_STREAM_PACING_MODE int 0 or 1 Live 0 – token-bucket pacing toward DIRECT_STREAM_BITRATE_KBPS; 1 – push, sending on the frame boundary and leaving the back-pressure to the kernel’s send buffer.
SERVER_STREAM_TYPE string see Stream type values Live Container and KLV mode of the served (RTSP / RTSPS / WebRTC) leg. Refused: an unknown value; the raw mpegts-klv-* forms without the mpegts-rtp infix, because the served leg carries a transport stream only inside RTP; and any MPEG-TS form while the codec is JPEG. The KLV mode applies live, and the metadata track and the SDP follow at once. So does the KLV signalling of a served transport stream (mpegts-rtp-klv-async <-> -sync, mpegts-rtp <-> mpegts-rtp-klv): the PMT takes the next version_number and declares the new carriage before the next access unit, and the KLV PES that follow use the new form, on the same sessions. A push leg that shared the served leg’s packets and now differs from it starts packetizing for itself, with the clean start of a re-type. A container change cannot be handed to a client that is already playing. So setParam() re-initialises this instance automatically instead of refusing: it runs the RESTART sequence, and clients are dropped and reconnect to an SDP that matches what they are sent.

Stream type values

serverStreamType and directStreamType select the container and whether KLV is carried:

Value Container and framing KLV
rtp codec-specific RTP (RFC 6184 / 7798 / 2435) accepted from sendFrame() but not carried by this leg; the legs that mux MPEG-TS (HLS, a push leg in MPEG-TS) still carry it
rtp-klv same as rtp muxed metadata track, PT 98, declared in the SDP (RFC 6597) – pass-through
rtp-klv-validate same as rtp same track, KLV validated before it is sent
rtp-klv-restamp same as rtp same track, KLV timestamp restamped
mpegts / mpegts-klv-async MPEG-TS straight into UDP, MISB ST 1402 asynchronous
mpegts-rtp / mpegts-rtp-klv-async MPEG-TS inside RTP, RFC 2250 PT 33 asynchronous
mpegts-klv-sync MPEG-TS straight into UDP synchronous
mpegts-rtp-klv-sync MPEG-TS inside RTP synchronous
mpegts-rtp-klv MPEG-TS inside RTP synchronous – the STANAG 4609 value the reference implementation uses; accepted as a synonym of mpegts-rtp-klv-sync

The -rtp- infix is the framing switch: the forms without it put the transport stream directly in the datagram (MISB ST 1402), the forms with it wrap the same stream in RTP (RFC 2250, PT 33).

The two legs choose independently, in container and in framing. serverStreamType describes what the RTSP, RTSPS and WebRTC clients receive. directStreamType describes what the push destination receives. A STANAG receiver can get mpegts-rtp-klv while the RTSP clients keep codec-specific RTP.

When the two differ, the push leg derives its own framing, KLV signalling and payload type, runs its own packetizer and MPEG-TS muxer, and keeps its own continuity counters and PSI tables. It reuses the shared packets (at no extra cost) only when container, KLV signalling and payload size all agree. The served leg carries a transport stream only inside RTP, so the raw mpegts-klv-* forms are refused there, by initVStreamer() and by setParam() alike; they are never silently reframed as RTP. The push leg takes the value literally, raw TS included.

VStreamerParams class

initVStreamer() takes this structure and getParams() fills it. It carries the same information as the VStreamerParam enumeration, so the Not used marks are the same. The defaults below are the interface’s own, from VStreamer 3.3.0 (the version in 3rdparty/VStreamer).

class VStreamerParams
{
public:
    bool        enable{true};
    int         width{1280};
    int         height{720};
    std::string directStreamIp{"127.0.0.1"};
    int         rtspPort{8554};
    /* ... see the table below for the full list ... */
};
Field Type Default Description
enable bool true Streaming enable / disable. false keeps sessions open and publishes nothing.
width int 1280 Stream width, [8:4096]. The input frame may be any size; it is scaled to this.
height int 720 Stream height, [8:4096].
directStreamIp string 127.0.0.1 Destination of the direct RTP push; comma-separated for several receivers.
rtspPort int 8554 RTSP port, shared by every instance in the process.
rtspsPort int 8555 Port of this instance’s own RTSPS listener. Unlike rtspPort it is not shared, so two instances must not name the same one. The listener opens when rtspEnable is set, the port is non-zero and rtspEncryption is optional or strict. Without a certificate and key it never falls back to plaintext: it uses the generated identity, or fails initialisation in a compliance profile that requires TLS. With the default rtspEncryption no, a supplied certificate opens nothing. Not available on Windows.
directStreamPort int 5004 Destination port of the direct push; RTCP on port+1.
webRtcPort int 7000 WebRTC signalling port, [1024:65534]. The media port is this + 1 unless webRtcMediaPort names one. See WebRTC.
hlsPort int 8080 TCP port of the HLS listener. Like rtspPort, it is shared by every instance in the process: a second instance naming a different port is refused rather than merged.
srtPort int 6000 Not used.
rtmpPort int 1935 Not used.
rtmpsPort int 1936 Not used.
metadataPort int 9000 Not used. KLV travels inside the media stream, not on a port of its own.
rtspEnable bool true Open the RTSP listener.
directStreamEnable bool true Enable the direct RTP push leg. With the interface defaults it pushes RTP to 127.0.0.1:5004, so set it to false if you do not want the leg.
webRtcEnable bool true Start the WebRTC endpoint. Refused by STRICT_FIPS, which fails initialisation rather than starting without it.
hlsEnable bool true Publish this stream over HLS. The interface default is true, so a caller that never mentions HLS still opens a listener on hlsPort; set it to false explicitly if that port is spoken for, because a listener that cannot bind fails initialisation.
srtEnable bool true Not used.
rtmpEnable bool true Not used.
metadataEnable bool false Not used.
rtspMulticastIp string 224.1.0.1/16 Multicast group or CIDR pool. Must be inside 224.0.0.0/4, otherwise multicast stays disabled.
rtspMulticastPort int 18000 Multicast RTP port, [1024:65534] and required even so RTCP can use port+1 (RFC 3550 sec. 11). An odd or out-of-range non-zero value fails initialisation and is refused by setParam(); it is not coerced. 0 means multicast is not configured.
user string no RTSP user. "" or "no" disables authentication.
password string no RTSP password.
suffix string live Stream name, used as the URL path. Must not be empty, and must be unique across instances sharing the port.
metadataSuffix string metadata Metadata format: SMPTE336M or VND.ONVIF.METADATA. The interface’s default metadata names no format and is normalised to SMPTE336M at initialisation, so getParams() reports the format actually in force.
minBitrateKbps int 1000 Minimum bitrate for variable-bitrate mode, [0:1000000]; 0 means no floor. Forwarded to the encoder.
maxBitrateKbps int 5000 Maximum bitrate for variable-bitrate mode; forwarded to the encoder.
bitrateKbps int 3000 Target bitrate; forwarded to the encoder.
bitrateMode int 0 0 – constant, 1 – variable; forwarded to the encoder.
fps float 30.0 Frame rate, [1:240]; forwarded to the encoder and held by the rate governor.
gop int 30 GOP size; forwarded to the encoder.
h264Profile int 0 0 – baseline, 1 – main, 2 – high; forwarded to the encoder.
jpegQuality int 80 JPEG quality in percent; forwarded to the encoder.
codec string H264 H264, H265/HEVC or JPEG/MJPEG, matched case-insensitively.
fitMode int 0 0 – fit (letterbox), 1 – fill (stretch).
cycleTimeUs int 0 Output only. getParams() reports the measured pipeline cycle here.
overlayEnable bool true Call the overlay renderer on each frame.
type int 0 Forwarded verbatim to the encoder as its backend selector; the meaning belongs to the encoder, not to MServer. For the bundled VCodecLibav: 0 = hardware (VAAPI, then QSV), 1 = software. The default 0 therefore needs a working GPU – set 1 if you have none.
custom1 float 0.0 Compute threads for preprocessing: the upper limit for the format converter and the scaler, [0 : hardware threads]. The default 0 means up to two threads, min(2, hardware threads). It is never read as a request for the whole machine. 1 keeps conversion and scaling on the preprocess thread. To use more cores, ask for them by number. setParam() refuses a value above the hardware thread count; initVStreamer() clamps it. See Compute threads for preprocessing.
custom2 float 0.0 Not used by MServer. Opaque caller data; the compliance mode moved to securityProfile in v3.2.
multicastTtl int 0 Multicast TTL for every multicast leg; clamped to 32, 0 means the default of 1.
securityProfile string no Compliance mode by name. An unrecognised value is refused.
publicAddress string no IPv4 literal advertised in SDP and ICE candidates; no means auto-detect.
bindAddress string 0.0.0.0 Interface every listener binds to.
rtpPortMin int 0 Lowest server RTP/RTCP port; 0 – ephemeral.
rtpPortMax int 0 Highest server RTP/RTCP port; 0 – ephemeral.
webRtcMediaPort int 0 WebRTC media port; 0 keeps webRtcPort + 1.
corsAllowedOrigin string no Origin permitted on the signalling endpoint (a wildcard * is refused).
serverStreamMaxPayloadSize int 1472 Maximum RTP payload of the server-delivered stream, video and metadata track alike. Clamped to 1200 while the WebRTC leg is enabled.
custom3 float 0.0 Not used by MServer. Free for the application; stored and returned unmodified.
rtspKey string no Path to the RTSPS private key.
rtspCert string no Path to the RTSPS certificate.
webRtcKey string no Private key for HTTPS signalling. no or empty falls back to rtspKey.
webRtcCert string no Certificate for HTTPS signalling. no or empty falls back to rtspCert.
hlsKey string no TLS private key for the HLS listener. Read only when hlsEncryption asked for HTTPS; no or empty falls back to rtspKey, and then to the generated identity.
hlsCert string no TLS certificate for the HLS listener. Same fallback chain as hlsKey.
rtmpKey string no Not used.
rtmpCert string no Not used.
rtspEncryption string no no, optional or strict. Any other value fails initVStreamer().
webRtcEncryption string no TLS policy for WHEP signalling: no, optional or strict. The media leg is always DTLS-SRTP. "yes" is rejected.
rtmpEncryption string no Not used.
hlsEncryption string no no, yes, optional or strict; any other value fails initVStreamer(). Two independent decisions: yes is AES-128 on the segments and needs no certificate; optional and strict ask for HTTPS; strict is both. See HLS.
logLevel int 0 Not used. MServer writes no log of its own.
directStreamType string rtp Container and framing of the push leg; see Stream type values.
directStreamBitrateKbps int 5000 Channel bandwidth for the push leg’s packet pacer, kbit/s. Used when directStreamPacingMode is 0; has to exceed the stream’s own bitrate.
directStreamMaxPayloadSize int 1472 Maximum RTP/UDP payload of the push leg’s own packetizer; it never shrinks the packets served to RTSP or WebRTC clients. At initVStreamer(), 0 means the default size and the [576:9000] range is enforced only when the leg is enabled; setParam() always requires [576:9000].
directStreamPacingMode int 0 0 – pace toward directStreamBitrateKbps; 1 – push on the frame boundary.
serverStreamType string rtp Container and KLV mode of the RTSP leg; see Stream type values.

custom1..3 are returned by getParams() unmodified, so a getParams() -> initVStreamer() round trip never corrupts them. MServer reads custom1, but stores it verbatim rather than normalised, so this holds for it too.

When a parameter takes effect, and how far it reaches

Every parameter answers two separate questions: when does a change apply, and who does it apply to. A value that needs a restart is fine, as long as the caller knows which values those are.

Applied immediately

setParam() returns and the change is already in force.

Parameter Notes
bitrateKbps, minBitrateKbps, maxBitrateKbps, bitrateMode pushed into the codec; the two ceilings also set the RTSP send budget of this stream’s subscribers
fps the rate the stream carries, held by dropping and duplicating – see Holding the configured frame rate
gop, h264Profile, jpegQuality, type codec parameters
width, height, fitMode, overlayEnable see Resolution changes – no client is disturbed
codec new dynamic payload type and fresh parameter sets at the next access unit
user, password per path; closes that path’s sessions so a revoked password stops reaching a client that is already playing. Reaches every authenticated leg – the plaintext listener, RTSPS, WHEP and HLS – so a rotation cannot leave the old secret valid on one of them
suffix moves the stream to the new URL path on every leg at once – RTSP, RTSPS, WHEP and HLS – carrying its credentials, its send budget, its clock, its multicast policy and its HLS segment window with it. A rename refused by any one leg is rolled back on the others rather than leaving the stream published under two names
hlsEnable off withdraws the stream from the HLS listener: the playlist and every segment under it answer 404 – or, when it was the last stream on the listener, the listener closes and releases its port – and the viewers drain away. Turning it back on is refused – opening a listener needs the compliance decision and the TLS material that only initialisation has, so it takes a RESTART, exactly as enabling RTSPS does
rtspPort re-binds the shared listener, so every instance follows
rtspMulticastIp, rtspMulticastPort, multicastTtl a group already transmitting keeps its address and only its TTL follows; an unused assignment is re-derived
directStreamEnable, directStreamIp, directStreamPort, directStreamType, directStreamPacingMode, directStreamBitrateKbps the whole push leg – see The direct push leg. Switching it on is refused under a mode that requires TLS
serverStreamType without a container change, metadataSuffix the KLV mode and the metadata format change freely; the metadata track and the SDP follow at once
custom1 the thread limit the format converter and the scaler work under. Picked up by the preprocess thread on its next frame, so a running stream changes its parallelism without a restart
securityProfile, corsAllowedOrigin, custom2, custom3 securityProfile is refused when the running configuration cannot honour the new mode: a FIPS mode without the validated module; a mode that requires TLS while the stream has no TLS listener, runs one on the generated identity, is served by the plaintext RTSP listener or has the plaintext push leg switched on; a mode that requires authentication without credentials; a mode with other WebRTC rules while the stream is published on WebRTC; a mode stricter about HLS while the stream is published there; a mode stricter about the media while a session on the TLS listener receives media that mode refuses (plain RTP on UDP, a multicast group, SRTP or an SRTP profile it forbids) – the sessions are not dropped to make room. It is also refused while the stream is not running (before initVStreamer(), after closeVStreamer(), during a re-initialisation): the profile of a stream that is not running is the one initVStreamer() is given. corsAllowedOrigin reaches the WHEP endpoint at once; the HLS listener reads it only when it starts. custom2 and custom3 are only stored

Re-initialising instead of refusing

One change cannot be applied to a running stream: the container of the served leg. Clients that are playing already hold the RTP payload type, the SDP and the packetization, and a subscriber cannot be told mid-stream that H.264 has become a transport stream. So setParam(SERVER_STREAM_TYPE) with a container change does not refuse; it rebuilds the stream. The value is stored, and the instance re-initialises itself on the calling thread right after the parameter lock is released. Clients are dropped and reconnect to a stream whose SDP describes what they are actually sent. The push leg needs none of this; it switches container in place.

Two consequences for a caller. First, setParam() may take as long as initVStreamer() for that one parameter, and it returns what the re-initialisation returns. A re-initialisation that fails, for want of memory or of a thread included, leaves the instance closed. Second, everything a re-initialisation resets (the pipeline counters, the sequence spaces, the multicast assignments) is reset. It is the same sequence as VStreamerCommand::RESTART.

Read at the next initialisation

These values are stored when set and take effect at the next initVStreamer() (or VStreamerCommand::RESTART). They are also validated when set, so a value the API refuses cannot reach initialisation through a configuration file:

rtspEnable, rtspsPort, rtspEncryption, rtspKey, rtspCert, webRtcEnable, webRtcPort, webRtcMediaPort, webRtcEncryption, webRtcKey, webRtcCert, hlsEncryption, hlsKey, hlsCert, bindAddress, publicAddress, rtpPortMin, rtpPortMax, directStreamMaxPayloadSize and serverStreamMaxPayloadSize.

Two of them are also read earlier. directStreamMaxPayloadSize is re-read at once by a live directStreamType change, and when a live directStreamEnable = 1 has to build a push leg that initialisation did not build. serverStreamMaxPayloadSize is re-read on a live codec change.

Most of the list is here for one of two reasons: a bound socket cannot be moved, and a listener that already accepts plaintext cannot be given TLS underneath it.

publicAddress is validated when it is set: it must be an IPv4 literal, or ""/no.

Output only

cycleTimeUs is measured by the server and overwritten on every getParams(); setParam() refuses it. metadataPort is refused too, because there is no separate metadata listener. hlsPort is refused while this stream is published on the HLS listener, if the value differs from the bound port. The socket cannot move under the streams already on it, and storing a value that takes effect only at a RESTART would report success for something that did not happen.

Reach: the stream, or the whole process

See What the listener owns, and what the stream owns. In short:

  • The RTSP port, the WebRTC signalling and media ports, the HLS port, bindAddress and the RTP port range are properties of shared listeners. A change of the RTSP port moves it for every instance. A conflicting value of the others is refused.
  • The RTSPS listener is this instance’s own, so rtspsPort reaches only this stream. Two instances cannot share a value: the second one’s bind() fails, and so does its initialisation.
  • Everything else is the stream’s own, including credentials, bitrate and the whole multicast configuration. Two exceptions: corsAllowedOrigin is a property of the shared signalling endpoint, so it applies process-wide; and securityProfile feeds the listener-wide compliance policy, which is the most restrictive combination of every attached stream.

The pair rule for the RTP port range

rtpPortMin and rtpPortMax are validated together at initialisation. 0/0 means “any ephemeral port”. Any other combination must be able to hold a pair (max >= min + 1), because RTCP takes the port after the RTP one. A lower bound with no upper bound describes a range nothing can be allocated from, so it fails initialisation. If you set them one at a time, set the upper bound first.

Resolution changes

Two different things can change. Neither restarts the stream or asks a client to reconnect:

  • The input frame size changes while width/height stay the same. The scaler absorbs it: conversion, letterbox/crop and overlay all adapt to the new source size, the encoder sees no change, and the SDP is untouched. A connected client notices nothing.

  • The stream resolution changes (width/height through setParam). The encoder is reconfigured and emits fresh parameter sets on the next key frame, and the SDP is updated for future DESCRIBEs. Clients that are already playing keep the same RTSP session, the same SSRC and a contiguous sequence space. They are never torn down and never asked to SETUP again. MServerResolutionTest checks exactly this.

    A geometry change also withdraws the description until the new parameter sets exist. The fmtp line carries the encoder’s parameter sets, and those encode the picture size. Between the change and the next key frame, the published SDP would therefore describe the old geometry. So DESCRIBE fails during that window rather than answer with an SDP that a client would decode at the wrong size. Withdrawing the description also keeps the pipeline running: a stream without a description stays awake (see What stays awake regardless). This matters because a DESCRIBE always arrives before any client plays. A stream that stayed “describable” while idle would never produce the new parameter sets, and would serve the old geometry for ever. A codec change follows the same rule, for the same reason.

Build and connect to your project 

git clone --recurse-submodules <repo>
cd MServer
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

Options:

Option Default Meaning
MSERVER_TARGET_ARCH empty value for -march= on libMServer’s own sources (GCC/Clang only); deliberately not native, so builds stay reproducible
<PARENT>_MSERVER_TEST ON standalone the interactive test application
<PARENT>_MSERVER_EXAMPLE ON standalone the minimal example
<PARENT>_MSERVER_HARNESS ON standalone the full automated suite
MSERVER_VCODEC_LIBVA_DIR build/VCodecLibva where the hardware codec’s sources are, for the field test rig. The rig is dropped when the path or libva is missing
MSERVER_FORCE_NO_FFMPEG OFF pretend FFmpeg is absent, to exercise the exclusion path on a machine that has it
MSERVER_HARNESS_WITHOUT_FFMPEG OFF keep harness/ when FFmpeg is missing; see below

Compiler settings of libMServer (all configurations):

  • GCC / Clang: -Wall -Wextra -Wpedantic -Wshadow, -fstack-protector-strong -fPIC, and in Release -O2 -funroll-loops -D_FORTIFY_SOURCE=2. MSVC: /W4 /GS /guard:cf, Release /O2.
  • A static library is never linked itself, so its linker hardening is declared PUBLIC and reaches every executable or shared object that links MServer: -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack on Linux, /guard:cf with MSVC (none with MinGW-w64). Before 2.1.2 the Linux options were PRIVATE and reached no executable.

Dependencies, and what happens when one is missing

Exactly two things decide what gets built.

Cryptographic backends: OpenSSL on Linux, Native Windows CNG on Windows

The cryptographic backend depends on the operating system:

  • Linux: libMServer links OpenSSL 3.0 or newer, directly and through CryptoProvider. TLS, DTLS-SRTP, digest authentication, SRTP, AES on HLS segments and the DRBG all use it, and so does the FIPS provider. There is no switch to build without it. If OpenSSL 3.0 or newer is missing (an older OpenSSL counts as missing), configuration stops with a fatal error:
CMake Error at src/CMakeLists.txt:66 (message):
  MServer requires OpenSSL 3.0 or newer and cannot be built without it.

    Found:
    Debian/Ubuntu : sudo apt install libssl-dev
    RHEL/Fedora   : sudo dnf install openssl-devel
    Alpine        : apk add openssl-dev

  If OpenSSL is installed somewhere CMake does not look, point it there with
  -DOPENSSL_ROOT_DIR=/path/to/openssl.
  • Windows: libMServer links CryptoProvider with its CNG backend (bcrypt.lib, the Windows Cryptography API: Next Generation) and the system libraries ws2_32 (Winsock) and winmm. OpenSSL is not used at all, so no OpenSSL binaries are needed. CNG provides the random numbers, SHA-256, HMAC and AES (HLS segment encryption, and the SRTP primitives). There is no TLS and no DTLS on Windows, so RTSPS, HTTPS, the generated TLS identity, SRTP over RTSP and WebRTC are not available; see Windows build with MSVC.

Without FFmpeg the library builds and the applications do not

test/, example/ and harness/ are all built around VCodecLibav, a wrapper over FFmpeg, because exercising a video server needs a real encoder. FFmpeg is therefore a dependency of those three and of nothing else: libMServer never links it, which MServerNoFFmpegDependency asserts. On Windows, CMake looks for FFmpeg in the directory named by the FFmpeg_DIR environment variable, or else in C:/libs/ffmpeg/ffmpeg-master-latest-win64-gpl-shared.

When FFmpeg is absent, all three are dropped from the build, not failed:

-- MServer: FFmpeg absent -- test, example and harness are excluded from the build

cmake --build then produces libMServer.a (MServer.lib on Windows) and the libraries it links, nothing else. Refusing to configure would deny a working library to anyone who simply has no FFmpeg, which is the normal case on an embedded target.

The FFmpeg check runs before the vendored VCodecLibav is added. That project creates its target unconditionally and reports a missing FFmpeg with message(SEND_ERROR), which fails the whole configuration. Deciding first means that code is never reached, so “the build breaks” becomes “the test applications are not built”.

To check the exclusion really works on a machine that does have FFmpeg:

cmake -S . -B build-noffmpeg -DMSERVER_FORCE_NO_FFMPEG=ON
cmake --build build-noffmpeg -j        # builds libMServer alone

The one escape hatch. MSERVER_HARNESS_WITHOUT_FFMPEG=ON keeps harness/ in the build when FFmpeg is missing. It exists for the field test rig, which encodes through libva, not FFmpeg, on a device that has a GPU and no FFmpeg. Every C++ test in the harness links the FFmpeg-based test codec, so each one is still skipped. What remains is the field rig (when libva and VCodecLibva are found), MServerNoFFmpegDependency, and the fuzz targets if MSERVER_FUZZ is on. The option is off by default, so a plain configure follows the rule above.

# on a GPU-only target, to get the field rig
cmake -S . -B build -DMSERVER_HARNESS_WITHOUT_FFMPEG=ON

Cryptography provider configuration

There is no build option for cryptography. On Linux, which OpenSSL provider backs MServer is a deployment decision. The compliance mode in VStreamerParams::securityProfile asks for the validated module when it wants one. When the module is not there, a FIPS mode does not start (2.1.3); the hardened configuration without a module is CRA_STRICT, so a claim is never made by accident. See Security and compliance modes.

On Windows, CryptoProvider’s CNG backend (bcrypt.lib) is used automatically; OpenSSL is not needed. CNG reports FIPS as active whenever a FIPS profile is requested or the Windows FIPS policy is on, and getStats().fipsActive repeats that answer. The FIPS 140-3 statements in this README are about the OpenSSL FIPS provider on Linux; they do not apply to a Windows build.

Windows build with MSVC

MServer builds on Windows with MSVC (Visual Studio 2022, x64) at warning level /W4, with zero warnings and zero errors.

cmake -S . -B build -G "Visual Studio 17 2022" -A x64
cmake --build build --config Release

What is different on Windows:

  • Sockets. MServer uses Winsock (ws2_32), and its reactors poll with WSAPoll(). Windows has no SIGPIPE, so there is nothing to block. Listeners take SO_EXCLUSIVEADDRUSE (on Windows SO_REUSEADDR would let another socket take over the port), and the UDP sockets turn off SIO_UDP_CONNRESET, so a viewer that went away cannot fail the next receive of a shared socket.
  • Timers. The system timer runs at 1 ms (timeBeginPeriod, winmm) from initVStreamer() to closeVStreamer(), so the frame pacing and the packet pacer keep their schedule; the default 15.6 ms tick would make them late.
  • Cryptography. CryptoProvider’s CNG backend replaces OpenSSL; see Cryptography provider configuration.
  • No TLS and no DTLS. RTSPS and the RTSP-over-HTTPS tunnel, HTTPS on the WHEP and HLS listeners, the generated TLS identity, SRTP over RTSP (its keys travel only over TLS) and WebRTC are not available. In detail:
    • rtspEncryption = optional opens no RTSPS listener; plaintext RTSP still serves. strict fails initVStreamer().
    • webRtcEncryption or hlsEncryption = optional or strict, or a webRtcCert/webRtcKey pair, fails initVStreamer(). hlsEncryption = yes (AES-128 segments over plain HTTP) works.
    • WebRTC is switched off: webRtcEnable (on by default) gives one warning on stderr (“WebRTC (WHEP) is not available on Windows: disabled”), getParams() reports it off, the other legs start normally, and WEBRTC_MODE = 1 is refused.
    • The profiles that require TLS (STRICT_FIPS, FIPS_ONVIF_BRIDGE, CRA_STRICT) fail initVStreamer() when RTSP is enabled.
  • Everything else uses the same code as on Linux, including the preprocessing thread default (custom1 = 0 gives min(2, hardware threads)). test_streaming_windows.py (below) checks part of it.
  • Test applications. They need FFmpeg; see Without FFmpeg the library builds and the applications do not for where CMake looks for it. MServerExample asks for no TLS, so it starts (without WebRTC, as above). MServerTest’s server mode (mode 1) brings up TLS legs, so its initVStreamer() fails on Windows. Its direct mode (mode 2) needs no TLS.
  • Harness. Only these tests are built on Windows: MServerUnitTest, MServerRtpTest, MServerStunTest, MServerTsTest, MServerCodecTest, MServerRuntimeParamsTest and MServerFastPathTest, plus MServerNoFFmpegDependency, which needs nm on the PATH. The other C++ tests need POSIX sockets or OpenSSL, and the shell-script interop tests need bash, so none of them is registered. Run them with ctest -C Release.

Streaming verification with FFmpeg on Windows

test_streaming_windows.py checks the streaming legs end to end against real FFmpeg and ffprobe binaries on Windows:

python test_streaming_windows.py

It expects build/ffmpeg.exe, build/ffprobe.exe and a Release build of build/bin/Release/MServerLiveServer.exe, and it puts C:\libs\ffmpeg\ffmpeg-master-latest-win64-gpl-shared\bin (the FFmpeg DLLs) on the PATH. A watchdog stops the whole run after 48 s. It verifies:

  • RTSP over TCP with H.264, H.265 and MJPEG.
  • RTSP over UDP with H.264, H.265 and MJPEG.
  • RTSP digest authentication: an unauthenticated request and a wrong password are refused, valid credentials are accepted.
  • Four RTSP-over-TCP clients streaming at the same time.
  • RTSP methods: OPTIONS, DESCRIBE with an SDP, and 501 for an unknown method.
  • The direct UDP push leg (MPEG-TS, mpegts-klv-sync), received and decoded by FFmpeg.
  • HLS over plain HTTP: the playlist is served and FFmpeg plays it.
  • HLS with AES-128 segments: the playlist names the key, the 16-byte key downloads, and FFmpeg decrypts and decodes the segments.
  • The delivered frame rate of a 25 fps stream.

Using MServer as a submodule (any platform). Add add_subdirectory(MServer) and link MServer. The library itself pulls in only 3rdparty/ (VStreamer, CryptoProvider, FormatConverter, ImageResizer). The FFmpeg-based test codec is a submodule of each directory that uses it (test/, harness/ and example/) and is never part of the library. In a build that enables several of them, only the first copy is configured. Pin all three copies to the same commit; the build does not check this.

Example

The smallest program that streams video (example/main.cpp). It generates its own picture, so it needs no camera, no file and no decoder. Build the project as above and run it:

./build/bin/MServerExample
# it prints: Streaming on rtsp://127.0.0.1:8554/live
#                    and http://127.0.0.1:8088/live/index.m3u8

Then point any player at that URL (see How to play the stream).

Four things in the code below are worth noticing before you copy it:

  • The input frame and the output stream are different sizes. The frame fed to sendFrame() is 1280x720, while params.width/params.height ask for 640x480. MServer scales for you: the frame you send does not have to match the stream you publish.
  • sendFrame() is checked. It returns FALSE when the frame is refused (the server is not initialised, the frame is invalid, or there is no memory for it), never because nobody watches: a stream without viewers answers TRUE. The loop gives up after 100 refused frames in a row and closes the server.
  • initVStreamer() takes the encoder, and MServer keeps using it until the streamer is closed. So the encoder must outlive the MServer object, which is why it is declared first below.
  • HLS is on by default, so this example also serves HLS. That is the VStreamer interface’s default, not MServer’s choice. The interface’s default port is 8080, which web servers and containers often hold already, and initVStreamer() fails when the port is taken; so the example sets params.hlsPort = 8088 and serves http://127.0.0.1:8088/live/index.m3u8. Set params.hlsEnable = false if you do not want the port opened.

The example leaves the other interface defaults alone too. So it also opens the WebRTC endpoint (signalling on port 7000, media on 7001; see The built-in player pages), and its push leg sends RTP to 127.0.0.1:5004. Because the push leg counts as a consumer, this stream never idles. Set params.webRtcEnable = false and params.directStreamEnable = false if you do not want them.

#include <chrono>
#include <cstring>
#include <iostream>
#include <thread>
#include "MServer.h"
#include "VCodecLibav.h"

namespace
{
/// Frames in a row the server may refuse before the example gives up. A frame
/// is refused (FALSE) when the server is not initialised, the frame is
/// invalid or there is no memory for it; a stream nobody watches answers TRUE.
constexpr int MAX_REFUSED_FRAMES{100};
}



int main()
{
    // Prepare video server params.
    cr::video::VStreamerParams params;
    params.codec    = "H264"; // Codec type.
    params.width    = 640;    // Output video width.
    params.height   = 480;    // Output video height.
    params.fps      = 25.0f;  // Output video FPS.
    params.gop      = 25;     // Output video GOP size (key-frame interval).
    params.rtspPort = 8554;   // Output RTSP port.
    params.suffix   = "live"; // Stream name.
    params.type     = 1;      // Software codec type for VCodecLibav.

    // HLS is enabled by DEFAULT in the VStreamer interface, so this is a
    // deliberate choice either way rather than an omission: leaving it alone
    // opens a listener on the interface default port 8080, which web servers
    // and containers often hold already, and initVStreamer() fails if the port
    // is taken. The example uses 8088 instead. Said out loud here so the
    // second URL below is no surprise.
    params.hlsEnable = true;
    params.hlsPort   = 8088;

    // The encoder is declared BEFORE the server and outlives it: MServer keeps
    // using it until closeVStreamer(), and locals are destroyed in reverse
    // order of construction.
    cr::video::VCodecLibav encoder;

    // Init video server.
    cr::video::MServer server;
    if (!server.initVStreamer(params, &encoder, nullptr))
    {
        return -1;
    }

    std::cout << "Streaming on rtsp://127.0.0.1:8554/live" << std::endl;
    std::cout << "         and http://127.0.0.1:8088/live/index.m3u8"
              << std::endl;

    // Input video frame. It is 1280x720 while the stream above is 640x480:
    // MServer scales, so the frame you send need not match the stream you
    // publish.
    cr::video::Frame frame(1280, 720, cr::video::Fourcc::YUV24);

    // Stream loop, bounded by MAX_REFUSED_FRAMES.
    uint8_t color = 0;
    int refused = 0;
    while (refused < MAX_REFUSED_FRAMES)
    {
        // Reset input frame with color.
        memset(frame.data, color++, static_cast<size_t>(frame.size));
        memset(frame.data + color * frame.width * 3, 255 - color,
               static_cast<size_t>(frame.width) * 3);

        // Send frame to video server.
        if (server.sendFrame(frame))
        {
            refused = 0;
        }
        else
        {
            ++refused;
        }

        // Wait for next frame.
        std::this_thread::sleep_for(std::chrono::milliseconds(35));
    }

    std::cout << "The server refused " << MAX_REFUSED_FRAMES
              << " frames in a row, stopped" << std::endl;
    server.closeVStreamer();
    return -1;
}

How to play the stream

What a client sees the server called

Nothing on the wire carries the implementation’s name. The SDP session name, the Server: header on RTSP and HTTP, and the digest realm say VideoServer. The RTCP CNAME and the WebRTC msid / cname say videoserver. A product that integrates this library need not advertise which library it uses, and a version-specific server banner gives an attacker an inventory hint for free.

One consequence to know before an upgrade: the digest realm is an input to the credential hash (RFC 7616: HA1 = H(user:realm:pass)). A client that cached credentials for an old realm therefore re-authenticates on its next request. Nothing stored on the server changes (MServer keeps the password, not a realm-bound hash), so no reconfiguration is needed.

The built-in player pages

The address of a stream is a page you can open. GET on the stream path of the WebRTC or HLS listener returns a small built-in HTML player:

http://<host>:<webRtcPort>/<suffix>/     -- WebRTC, a WHEP client in ~20 lines
http://<host>:<hlsPort>/<suffix>/        -- HLS

A request without the trailing slash is answered 301 to the slashed form, because the page fetches whep and index.m3u8 relative to itself: at /live a relative whep would resolve to /whep.

Being served by MServer is the point, not a convenience. A page on the same origin as the media inherits the browser’s own credential handling. Opening the page raises the digest dialogue, and every request the page then makes (the WHEP POST, the playlist, each segment) carries those credentials automatically. That is why the built-in pages work with authentication on, and a player loaded from file:// does not. A file:// page is a different origin: it would have to implement digest by hand for fetch(), and it cannot sign the <video> element’s own requests at all.

The pages are deliberately bare: a full-window <video> and nothing else (no heading, no status line, no controls around it). So a page can be embedded in another site in an <iframe> with nothing to hide. Diagnostics go to the browser console, where they are available when wanted and invisible otherwise. The stream name is HTML-escaped before it reaches the page. The name is the caller’s suffix, and a configuration value that reaches served markup unescaped is stored XSS, even when only an integrator can choose it.

The WebRTC page has no external dependency at all: it is about twenty lines of RTCPeerConnection. It therefore works on a device with no route to the internet, and it is the leg to use when you want video in a browser (on Linux; see WebRTC). It logs the ICE state and detects a browser with no H.264 decoder (a Chromium built without proprietary codecs offers only VP8, VP9 and AV1). It also ends its WHEP session with a DELETE on pagehide (RFC 9725 sec. 4.4). Without that, a closed tab would stay a counted client for the whole RFC 7675 consent window: 30 seconds of encoding for a viewer who has left.

HLS in a browser is different, and MServer cannot fix that. No Chromium-based browser has an HLS demuxer (that includes Edge, Chrome, Opera and Brave), and neither does Firefox. Only Safari and iOS do. Media Source Extensions (MSE) do not help directly either: MSE accepts fragmented MP4, not the MPEG-TS segments HLS is made of. The universal answer is hls.js. It fetches the playlist and segments itself and converts TS to fMP4 for MSE; every HLS web player uses it.

So the HLS page uses native HLS first (Safari). Otherwise it loads one pinned hls.js release (1.7.3) from cdn.jsdelivr.net, checked with a Subresource Integrity hash, so the browser refuses any other file. If neither works – the isolated-device case, or a file that fails the integrity check – the page writes to the browser console what is missing, and points at ffplay or VLC and at the WebRTC page, which needs nothing external. MServer serves no other files, so on a device without internet access a browser can play HLS only if you host hls.min.js yourself and change the page to load it from there (the page is in src/impl/MServerPlayer.h).

If you would rather build the page into your own application, the rest of this section is the protocol underneath.

# RTSP over TCP (interleaved)
ffplay -rtsp_transport tcp rtsp://127.0.0.1:8554/live

# RTSP over UDP
ffplay -rtsp_transport udp rtsp://127.0.0.1:8554/live

# UDP multicast
ffplay -rtsp_transport udp_multicast rtsp://127.0.0.1:8554/live

# RTSP tunnelled over HTTP
ffplay -rtsp_transport http rtsp://127.0.0.1:8554/live

# RTSPS. The test application's certificate is self-signed, so verification
# has to be switched off -- that is the certificate, not the server.
ffplay -rtsp_transport tcp -tls_verify 0 rtsps://user:pass@127.0.0.1:8555/live

# HLS -- an ordinary HTTP URL, so anything that plays HLS will do. 8088 is
# the example's port; a server left at the interface default listens on 8080.
ffplay http://127.0.0.1:8088/live/index.m3u8

# HLS over HTTPS. Same note about the certificate as RTSPS above.
ffplay -tls_verify 0 https://127.0.0.1:8088/live/index.m3u8

# GStreamer (h264parse is in gst-plugins-bad, avdec_h264 in gst-libav)
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/live protocols=tcp \
  ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink

A note on HLS from a cold start. The stream keeps encoding until it has closed its first segment, even with nobody watching. So the first playlist a player gets already names media, and ffplay starts on the first try. This is deliberate: an empty playlist is worse than a 404, because ffmpeg’s demuxer reports Empty segment and gives up instead of reloading. See What stays awake regardless.

A note on the HTTPS tunnel. MServer serves it: the GET leg answers 200 OK with Content-Type: application/x-rtsp-tunnelled over TLS. But ffmpeg cannot use it. ffmpeg’s RTSP tunnel works over plain HTTP only, so ffplay -rtsp_transport http rtsps://... fails with an I/O error whatever the server does. Use a client that implements the tunnel over TLS (it was designed for the Apple / QuickTime stack), or connect over plain RTSPS, which every client above supports.

WebRTC needs no player at all: a browser and about fifteen lines are enough. The whole protocol is the one POST below. Save it as whep.html and open it in any browser, straight from disk. A page opened from disk is a cross-origin caller, so set corsAllowedOrigin on the stream to the origin the browser reports for it (null for a page opened from disk; a wildcard * is refused): since 2.1.3 MServer sends no Access-Control-Allow-Origin unless it is configured. The page MServer serves itself is same-origin and needs nothing.

MServer serves this page itself at the stream path (see The built-in player pages). There is nothing to copy from here unless you are wiring WebRTC into your own application.

Put the URL that the server printed into WHEP_URL. The line below uses the library defaults (webRtcPort = 7000, suffix live). MServerTest uses port 8600, names its first stream live0 and brings the TLS leg up beside the others. So for the test application the URL is https://127.0.0.1:8600/live0/whep, exactly the line the program prints. Its certificate is self-signed: open that URL once in the browser and accept the certificate, or the page’s fetch() will be refused. Also start the test application with authentication off (no no). A browser’s fetch() cannot answer a digest challenge, so otherwise this page gets 401 and nothing plays.

<video id="v" autoplay muted playsinline></video>
<script>
const WHEP_URL = 'http://127.0.0.1:7000/live/whep';   // <-- the printed URL

const pc = new RTCPeerConnection();
pc.addTransceiver('video', {direction: 'recvonly'});
pc.ontrack = e => document.getElementById('v').srcObject = e.streams[0];

pc.createOffer()
  .then(o => pc.setLocalDescription(o))
  .then(() => fetch(WHEP_URL, {
      method: 'POST',
      headers: {'Content-Type': 'application/sdp'},
      body: pc.localDescription.sdp
  }))
  .then(r => r.text())
  .then(sdp => pc.setRemoteDescription({type: 'answer', sdp}));
</script>

If you serve the page itself over HTTPS, the signalling endpoint must be HTTPS too: a browser blocks a plaintext fetch from a secure page. The UDP media port (webRtcMediaPort, or webRtcPort + 1) must be reachable from wherever the browser runs. It carries STUN, DTLS and the media itself. A firewall that allows only the signalling port gives you a connection that negotiates and then stays black.

To check the endpoint without a browser, one curl is enough. A 201 Created with an SDP answer means the whole WHEP leg works. The test application’s endpoint is HTTPS with a self-signed certificate, hence -k; add --digest -u admin:admin if you left authentication on:

curl -k -i -X POST -H "Content-Type: application/sdp" \
     --data-binary @offer.sdp https://127.0.0.1:8600/live0/whep

The offer must name a codec MServer can send. For H.264 that means packetization-mode=1 in an a=fmtp: line (the interleaved mode of RFC 6184 is not implemented). Every browser offers it. A hand-written offer that omits it is answered 400, with that exact explanation.

How to extract KLV from the stream

There are two carriages, and which one a client sees depends on serverStreamType.

MPEG-TS in RTP (mpegts-rtp*). KLV has its own PID inside the transport stream, which is what a STANAG 4609 receiver expects. This is the only transport-stream form the served leg accepts. The raw mpegts / mpegts-klv-* forms put TS straight into a datagram; they exist for the direct push leg and are refused on the served one:

# ffprobe reports the KLV PID as a data stream
ffprobe -rtsp_transport tcp -i rtsp://127.0.0.1:8554/live \
  -show_entries stream=codec_name,codec_type

# Dump the KLV elementary stream
ffmpeg -rtsp_transport tcp -i rtsp://127.0.0.1:8554/live \
  -map 0:d -c copy -f data klv.bin

Codec RTP (rtp-klv*). KLV is a second RTP source, payload type 98. The SDP declares it as its own m=application track with its own a=control:.../trackID=1. A client has to SETUP that track as well as the video one, in either order. trackID=1 alone opens a metadata-only session. A trackID=0 SETUP that follows adds the video track to it, on the same transport family and profile (any other is refused 461). The KLV then arrives on its own interleaved channel pair or its own UDP port pair, with its own SSRC and sequence space, never on the video transport. A client without an MPEG-TS parser uses this carriage, an ONVIF client above all: it reads the format straight from the a=rtpmap line.

# The DESCRIBE response names the track and its format
ffprobe -rtsp_transport tcp -i rtsp://127.0.0.1:8554/live 2>&1 | grep -A1 "application"

A KLV unit that fits in one packet travels in one packet; the payload after the 12-byte RTP header is then the whole unit. A larger unit is fragmented across consecutive packets with the same timestamp, and the marker bit is set on the last one (RFC 6597 sec. 4.2). A receiver therefore reassembles up to the marker. The fragment size follows the video’s: serverStreamMaxPayloadSize, capped together with it when the WebRTC leg imposes its 1200-byte limit.

Testing

There are three separate things, and they serve different audiences.

example/ – the smallest program that streams video. Read this first.

test/ – an interactive application for you to point your own software at. It generates its own animated video, so it needs no input file, no camera, no decoder and no OpenCV.

./build/bin/MServerTest 1 [res 1-3] [fps 10-60] [codec] [profile] [user password]   # server
./build/bin/MServerTest 2 [res 1-3] [fps 10-60] [codec] [format 1-6] [ip] [port]    # direct

Run it with no arguments, and it asks the questions that decide what goes on the wire and lists the choices, so there is no order to remember:

Question Choices
What to run 1 server – RTSP, RTSPS, WebRTC and HLS at once; 2 direct – push datagrams to a destination
Stream resolution 1 1920x1080, 2 1280x720, 3 640x512
Stream frame rate any value from 10 to 60

In direct mode it then also asks for the container, the destination address and the port. Everything else is an argument with a default. codec is H264 (or H265, JPEG; avc, hevc, mjpeg and any letter case are accepted too). profile is ONVIF (or CRA_STRICT, FIPS_ONVIF_BRIDGE, STRICT_FIPS). Authentication is on, with admin / admin.

The source is fixed and the stream is not. The program always generates 1920x1080 at exactly 60 Hz and hands that to sendFrame(), whatever you chose above. Reaching the stream you asked for is MServer’s work: it scales 1920x1080 down to 1280x720 or 640x512, and resamples 60 Hz down to as little as 10. Fixing the input is what makes that work visible. A client pointed at the stream reports the size and rate you selected, not the source’s. 640x512 is deliberately 5:4 against a 16:9 source, so it tests the aspect handling (fitMode) and not just a plain rescale.

The generated picture is a diagonal gradient with a rectangle moving on a Lissajous path and a little sensor noise. Each part has a job. The gradient gives the encoder a smooth field. The moving rectangle makes a dropped or duplicated frame visible to the eye. The noise stops the encoder from compressing the synthetic image far better than a real camera’s, which would make every bitrate measurement optimistic. The noise is a fixed 256-entry table, read at an offset that changes each frame. It is not rand(), which at 1920x1080x60 would be 124 million calls a second. The whole generator costs a couple of additions and one table lookup per pixel.

The default bitrate is 10 Mbit/s. The GOP is one second (gop equals the stream rate), and the encoder runs in software (type = 1).

Server mode

Every listener comes up at once, and the program prints a ready-to-paste URL for each:

  rtsp://admin:admin@127.0.0.1:8554/live0
  rtsps://admin:admin@127.0.0.1:8555/live0
  WHEP POST https://127.0.0.1:8600/live0/whep
  http://127.0.0.1:8700/live0/index.m3u8

So RTSP, RTSPS, WebRTC and HLS can be tried against the same running server without restarting it. The program then prints how to connect. Each line is meant to be copied as it stands, with the default admin / admin credentials already inline where the client accepts them (an excerpt):

1) RTSP  -- any player:
     ffplay -rtsp_transport tcp rtsp://admin:admin@127.0.0.1:8554/live0
     vlc rtsp://admin:admin@127.0.0.1:8554/live0
2) RTSPS -- the same, over TLS:
     ffplay -rtsp_transport tcp rtsps://admin:admin@127.0.0.1:8555/live0
3) WebRTC -- open this in a browser:
     https://127.0.0.1:8600/live0/
4) HLS   -- open this in a browser:
     http://127.0.0.1:8700/live0/
     or in any player:
     ffplay http://admin:admin@127.0.0.1:8700/live0/index.m3u8

Two of the four are browser legs, and the program spells out the catch of each.

Both browser legs are single URLs, because MServer serves the player itself (see The built-in player pages). The browser asks for the password once, when it opens the page, and every request the page then makes carries it. That is what makes admin / admin work in a browser at all.

HLS is the real limitation of the four in a browser. Only Safari and iOS have an HLS demuxer. Chrome and Firefox have none and treat the playlist as a file to download, which is why pasting an .m3u8 URL into them gives an empty player. The built-in page covers them with hls.js, fetched from a CDN, so it needs internet access. Without it, the page writes a hint to the browser console instead. The ffplay and vlc lines work everywhere, credentials and all.

Pass no no as the last two arguments to switch authentication off:

./build/bin/MServerTest 1 2 30 H264 ONVIF no no   # server, 1280x720 @ 30, open

WebRTC from your own web page needs no no. A page’s fetch() cannot answer an HTTP digest challenge, so with authentication on, its WHEP request is answered 401 Unauthorized and the video never starts. That is correct behaviour, not a defect, but it is not what you want while trying the demo. (The built-in page is different; see above.) ffplay and VLC do speak digest, so for them leave the credentials on and put them in the URL.

TLS uses the files mserver_test_cert.pem and mserver_test_key.pem in the working directory. When they are missing, MServer generates a self-signed pair and writes it there (see When no certificate is configured); the next run reuses it. The strict profiles refuse a generated identity, so with them the files must already exist: run once with ONVIF first, or put your own certificate there. Because the certificate is self-signed, clients that verify certificates must be told not to: curl -k, “proceed anyway” in a browser, or -tls_verify 0 for an ffmpeg build that verifies by default.

At start-up the program prints the compliance mode it resolved (Active : ...). After that it prints a line whenever the number of connected clients changes: the new count, frames encoded, access units, RTP packets and frames dropped. You can therefore watch your client attach and see whether media really flows. It checks once a second but stays quiet while nothing changes, so the connection instructions do not scroll off the screen. With no client attached the pipeline stands still. That is idle suppression at work, and the pipeline starts the moment something connects.

Direct-stream mode

Mode 2 tests the push leg instead, and it opens no listener at all: RTSP, RTSPS, WebRTC and HLS all stay down, so any packet arriving at the far end can only have come from the leg under test. It asks for a container, a destination address and a port, then transmits whether or not anything is listening – there is no signalling on this leg, no session and nothing to negotiate.

Six containers are offered, and the choice decides both the framing and how the metadata is carried:

# directStreamType On the wire Metadata
1 rtp codec RTP over UDP none
2 rtp-klv codec RTP, PT 96 (H.264) / 97 (H.265) / 26 (JPEG) KLV on its own RTP track, PT 98, same port (RFC 6597)
3 mpegts-klv-async MPEG-TS over UDP, 7x188 per datagram KLV PID, stream_type 0x06 + KLVA registration descriptor (SMPTE RP 217)
4 mpegts-klv-sync MPEG-TS over UDP KLV PID, stream_type 0x15 + metadata descriptors, in metadata_AU cells (MISB ST 0604)
5 mpegts-rtp-klv-async MPEG-TS inside RTP, PT 33 (RFC 2250) as #3
6 mpegts-rtp-klv-sync MPEG-TS inside RTP, PT 33 as #4

The metadata is a valid MISB ST 0601 UAS Datalink Local Set. It holds the 16-byte universal label and the BER length of the set, then Tag 2 first, stamped with the current UTC time in microseconds on every frame, the LS version, the sensor latitude and longitude, and the Tag 1 checksum last. The checksum is recomputed after each stamp, because ST 0601 sec. 6 covers the bytes the timestamp sits in. A receiver therefore gets a real timeline it can correlate with the video, not a constant. And because the set is valid and carries Tag 2, the same time travels in the video as the MISB ST 0604 time stamp SEI (H.264 and H.265).

The push leg is paced at twice the encoder bitrate (directStreamBitrateKbps = 20000 for the default 10 Mbit/s). That headroom keeps bursts above the encoder’s average from building up latency. A receiver that maps capture times from the Sender Reports gets the moment each frame was published, within a millisecond on the same host. A receiver that reads the SEI gets the moment the frame was handed to MServer, which is earlier by the scaling and encoding time.

The program prints exactly what to run at the other end, because nothing on this leg announces itself:

Container Receiver
MPEG-TS over UDP (#3, #4) ffplay udp://@:5600 – a transport stream is self-describing, so the PAT and PMT arrive in band and nothing else is needed (the encoder’s key frames carry the SPS and PPS)
MPEG-TS in RTP (#5, #6) ffplay rtp://@:5600 – PT 33 is a static payload type, so it still needs no SDP
codec RTP (#1, #2) ffplay -protocol_whitelist file,udp,rtp direct_stream.sdp – a dynamic payload type is described by nothing on the wire, so the program writes the SDP for you 
./build/bin/MServerTest 2 2 30 H264 4 192.168.1.60 5600   # TS/UDP, synchronous KLV
./build/bin/MServerTest 2 1 60 H265 6 239.10.0.1  5600    # TS in RTP, multicast group

In this mode the stream keeps encoding even with nothing listening, which is the point. Idle suppression counts a configured push destination as a consumer, because its packets are as real as a client’s and nothing on the far end reports back. Every 10 seconds the program prints a statistics line for the push leg: packets sent, send failures, frames encoded, access units, KLV units accepted and refused, and frames dropped. (A push leg has no clients whose arrival could trigger a line.)

Not every combination is legal. MServer refuses JPEG with an MPEG-TS container (formats 3 to 6), because STANAG 4609 and MISB ST 1402 define no MPEG-TS stream type for motion JPEG. Pick H264 or H265 for those formats, or format 1 or 2 for JPEG; the program says so when it happens. In server mode JPEG is accepted, but the HLS playlist stays empty for the same reason: the RTSP legs carry JPEG, and the HLS leg has no stream type to declare.

harness/ – the full automated suite. In a standalone build (the one above) it is already on, together with the example and the test application, so there is nothing extra to enable:

cd build && ctest --output-on-failure

It is off only when MServer is consumed as a submodule, because a parent project has no reason to build somebody else’s tests. To switch it on there, set the option named in the table above, <PARENT>_MSERVER_HARNESS. The prefix is the value of the PARENT variable in the parent’s build. In a standalone build PARENT is empty, which gives the bare leading underscore below (and there the option is already on):

cmake -S . -B build -D_MSERVER_HARNESS=ON      # standalone spelling

FFmpeg is what the test applications need, not the library. On Linux libMServer links OpenSSL and nothing else. The example, the test application and the harness all need VCodecLibav, a wrapper around FFmpeg. When the FFmpeg development libraries or headers are missing, those three subprojects are excluded from the build, and the library still configures and builds. They are dropped, never failed:

-- VCodecLibav: FFmpeg development libraries/headers not found; test/example/harness disabled
-- MServer: FFmpeg absent -- test, example and harness are excluded from the build

The decision is taken at the top level, before add_subdirectory(). The vendored VCodecLibav creates its target unconditionally and reports a missing FFmpeg with message(SEND_ERROR), which fails the whole configuration. Deciding first means that code is never reached.

To check that the exclusion really works on a machine that does have FFmpeg:

cmake -S . -B build-noffmpeg -DMSERVER_FORCE_NO_FFMPEG=ON
cmake --build build-noffmpeg -j

test/, example/ and harness/ are all dropped, because each is built around VCodecLibav, and libMServer.a (with the libraries it links) is all that is produced. See Dependencies, and what happens when one is missing for the full rule, and for MSERVER_HARNESS_WITHOUT_FFMPEG, the opt-in that keeps the harness for the field rig on a GPU-only target.

The client side of every leg but HLS and WebRTC is verified by the VSourceStream test suite (the sibling repository, VSourceRtp/test, whose reference server is MServer embedded in the test process). Its matrix group takes every delivery mode a client can receive – RTSP over UDP, TCP, multicast and the HTTP tunnel; RTSPS interleaved, with plain UDP or multicast media, with SRTP over UDP or TCP, and through the HTTPS tunnel; the push leg in every container and synonym; H.264, H.265 and JPEG; the KLV and ONVIF metadata tracks and MPEG-TS KLV; payload sizes 576 and 9000; digest authentication on every path; rtspEncryption strict; CRA_STRICT with a transport stream; two streams on a shared port – each byte-exact with its metadata. Its live group takes MServer’s live changes under a connected client (the metadata track added and removed, enable, a password rotation, a rename, codec changes on the MPEG-TS and SRTP legs), seven clients on every transport at once, and a session beyond the 60 s timeout. The suite’s compliance group expects the 2.1.3 behaviour of the FIPS profiles (refusal without the validated module).

Every C++ test below links the test codec, so it is built only when FFmpeg is available. The Needs column lists what a test uses at run time beyond that. On Windows only some of them are registered; see Windows build with MSVC.

Test Covers Needs
MServerUnitTest Annex-B parsing, parameter sets, slice_type, lifecycle –
MServerRtpTest packetizers, sequencing, drop resilience, clock, RTCP –
MServerSrtpTest SRTP profiles, ROC, SDES/MIKEY, compliance modes, and SRTCP – both profiles’ trailer layouts checked against an independent RFC 3711 implementation, replay window, tampering in each region of the packet; every key a context derived (master, RTP and SRTCP) cleansed before its buffer is freed, watched through replaced allocation functions –
MServerTsTest PSI CRC, continuity counters, async/sync KLV, ST 0601 (BER-OID tags above 127, items that must fill the set, Tag 1 last), the PMT version_number on content changes, 33-bit PTS/PCR across the RTP timestamp wrap, the PCR’s delay behind the PTS, the clock reference between access units (PAT, PMT and a PCR-only packet whose continuity counter does not advance), the KLV-per-PES limit, the ST 0604 SEI (layout, Time Status 0x9F, placement, emulation prevention) –
MServerStunTest STUN parsing against the RFC 5769 test vectors, MESSAGE-INTEGRITY, FINGERPRINT, ICE role attributes, the amplification cap, malformed input –
MServerDtlsTest DTLS-SRTP against a real OpenSSL client, key export equality, fingerprint mismatch refusal, GCM refusal, retransmission after a dropped flight –
MServerHttpTest HTTP/1.1 parsing, request framing and smuggling (obs-fold, non-token field names, CR/LF/NUL in field values, control bytes in the target, nothing answered after Connection: close), the request deadline, TLS, digest authentication, nonce replay rejection, five wrong answers per connection, the per-source connection cap, CORS, and an allocation failure on the reactor thread ending one connection only and failing closed in the credential lookup –
MServerWebRtcTest the whole WebRTC leg end to end: WHEP, ICE, DTLS-SRTP and media the test decrypts with keys it derives itself, a Sender Report (decrypted SRTCP) dating a unit of the stream’s own clock at the time it was sent, plus PLI and teardown, which ends with an SRTCP BYE naming the session’s SSRC –
MServerWebRtcIntegrationTest Everything reachable through the public API only: the default configuration answers a browser, CORS, the answer’s a=msid, packet size on the wire, endpoint release on the last stream, compliance modes that must refuse, bindAddress confining every listener (and failing when it cannot be honoured), and RTP port ranges including exhaustion and cross-instance conflicts test codec
MServerCodecTest real encoder, B-frame absence proof test codec
MServerRtspTest RTSP status codes, full session, RTCP reception, malformed input, request bodies framed by Content-Length (pipelining, $-prefixed bodies, split bodies, 413 / 400, the refusal followed by a FIN rather than a reset while the client is still sending), the HTTP tunnel’s POST leg as one base64 stream, RTSPS requests of more than 4 KiB in one TLS record (body, head, pipelined, RTSP over HTTPS), TEARDOWN in the middle of a 700 KB key frame handing no socket to another owner, a video SETUP after a metadata-only one adding the track over UDP and TCP, the first Transport alternative the server can serve taken (UDP or TCP first, an unsupported profile skipped, RTP/SAVP falling back to RTP/AVP, a quoted comma, none servable -> 461), and – over 63 s of real time – the tunnel’s POST leg outliving the sessionless timeout, a metadata-only UDP session kept alive by RTCP on its metadata port alone, and the one that expired saying BYE test codec
MServerRuntimeParamsTest live parameter changes, validation test codec
MServerSharedPortTest shared port across instances test codec
MServerResolutionTest input and stream resolution changes without restarting the stream, and a burst of stream resolution changes while a client plays test codec
MServerMulticastTest pool vs fixed group allocation, client-requested destination=, per-request multicast SDP, including ;multicast and its control URLs, a multicast presentation that announces no feedback and no RTX, and a group saying BYE on its RTCP port when the stream stops test codec
MServerStrictModesTest the modes ffmpeg cannot reach: RTSPS + RFC 7616 SHA-256 digest, MD5 refusal, replayed-Authorization refusal, profile names rejected rather than defaulted, a FIPS mode refusing to start without the validated module (2.1.3) while CRA_STRICT runs the same hardened configuration on the default provider, SRTP over RTSP end to end – including the metadata track, whose packets must decrypt byte-exact under their OWN SSRC – the key taken from the SDP, session keys derived by the test’s own RFC 3711 implementation, every tag verified and the H.264 recovered – a metadata-only SRTP session over TCP and UDP (the SSRC its SETUP reply announced), the ;srtp presentation’s control URLs, and a TLS-mandating mode refusing every transport that would put media in the clear test codec
MServerDirectPushTest the push leg with no RTSP client: RTP framing, MPEG-TS in RTP, bare MPEG-TS in UDP, a push container independent of the server leg proven on the wire, directStreamBitrateKbps pacing measured against its cap, and the codec-RTP metadata track end to end – declared in the SDP with its own control URL for both metadata formats, played by a real client, delivered over interleaved and over UDP on a transport of its own, never leaking onto the video channel, and fragmented and reassembled byte-exact when a unit exceeds one packet; a configured but disabled leg sending neither RTP nor RTCP; the ST 0604 time stamp SEI equal to each frame’s Tag 2 on MPEG-TS and codec RTP, H.264 and H.265; PMT version changes on the wire; the KLV-per-PES limit counted in klvRejected; container changes while frames flow; the direct_stream.sdp the test application writes, for every codec spelling; a live MULTICAST_TTL read off the push leg’s datagrams (IP_RECVTTL), cap included; the test application’s KLV checked as strict ST 0601 and its ST 0604 SEI; Sender Reports dating every frame within 5 ms of its publication on the push leg, RTSP/TCP and RTSP/UDP, also after this process’s wall clock is stepped 3 s (an interposed clock_gettime()), with the new mapping reported at once; the pacer holding its configured rate within 4 %; the queue of the old framing dropped on a re-type; a re-typed leg back within a few frames, PAT and PMT first; a live change of the served leg’s KLV signalling (new PMT version, synchronous KLV, the push leg unaffected); the PCR a frame interval behind the PTS (T-STD); the stream type toggled while KLV flows; an overflowing push queue dropping whole access units, counted, and resuming at a key frame; PCRs, PATs and PMTs at most 100 ms apart at 5 fps on the push leg and the served leg; and the push leg’s BYE when the stream stops test codec
MServerFastPathTest every input format publishes every frame it encodes at the same pacing, so a broken preprocessing fast path shows up as loss test codec
MServerHlsTest the HLS leg end to end: the playlist parsed and every segment it names fetched and checked for the PAT, PMT and stream type a player needs; a codec change marking EXT-X-DISCONTINUITY; AES-128 segments proven to be ciphertext and their key served; the leg’s behaviour in each compliance profile; HTTPS with digest; withdrawing the leg while the encoder is feeding it (a data race and a use-after-free before it was fixed); a password rotation reaching this leg; a suffix rename carrying the segment window across; a key generation number never reused with a different key; the ephemeral TLS identity, verified by comparing the certificate the client saw against the fingerprint getStats() reports; a failed HTTPS start leaving no TLS context or origin behind; a renamed stream never served without its credentials; and the listener released with its last stream, then bound again on another port with HTTPS test codec
MServerIdleTest idle suppression: the encoder stops entirely with no consumer, DESCRIBE is still answered from the cached description, the first client is served SPS/PPS/IDR within a fraction of a GOP, a client that PLAYs before the stream ever idled is served a key frame just as fast, the stream idles again when the last client leaves, the push leg keeps it awake, MPEG-TS idles from the first frame, and one stream’s viewer never wakes another sharing the port test codec
MServerPliRtxTest the 2.1.0 fixes (setParam against the parameter-set cache, the metadata-only SRTP SSRC, H.265 payload type 97), PLI and FIR as key-frame requests, NACK answered with RTX over TCP, and over UDP within milliseconds, the JPEG RTX mapping, RTSPS viewers vanishing mid-stream with SIGPIPE at its default disposition; NACK work bounded (300 FCIs naming 17 packets draw 17 retransmissions, three per packet at most), feedback taken only from the client’s address and only about the video SSRC, a NACK about the metadata track retransmitting no video, SRTCP verified before feedback is acted on (a genuine SRTCP PLI and NACK taken, plaintext, replayed and forged ones not), the RTX payload type after a live H.264 -> H.265 switch, an MPEG-TS session not answered with RTX, stop() after a TLS peer reset while the reactor was held (RtspServer::holdReactor()), a NACK for numbers the session has not sent yet drawing nothing (and the same NACK repairing them once they are sent), the RTCP BYE – after the session’s last packet on TEARDOWN over UDP (naming the RTX SSRC too) and TCP (ahead of the reply), as SRTCP on an SRTP session, for a transport a SETUP replaces, for every session when the stream stops, and none from a session that never sent – and a playing session’s payload type re-stamped and its metadata destination moved hundreds of times while units flow (clean under ThreadSanitizer) test codec
MServerOnvifMcTest ONVIF RTSS-5-1-7 multicast: a live switch to JPEG re-stamps the push leg’s RTP payload type and its SDP, so a receiver told RTP/AVP 26 gets payload type 26 test codec
MServerOnvifDefectsTest media keeps flowing after a live codec and geometry change, and metadata-only RTSP sessions deliver payload type 98 to the client test codec
MServerNoFFmpegDependency asserts FFmpeg is not linked into the library, by reading its symbols with nm nm
MServerInteropFfmpeg ffprobe reads H.264, H.265 and JPEG over TCP interleaved and over UDP unicast; ffmpeg decodes 50 H.264 frames over each; three clients read one stream at once; OPTIONS advertises the methods and an unknown method gets 501. Any error line, or a warning that points at a protocol defect, fails the check ffmpeg, ffprobe
MServerInteropGstreamer rtspsrc + rtph264depay over TCP, UDP and multicast, and H.265 (rtph265depay) and JPEG (rtpjpegdepay) over TCP; the parsers (h264parse, h265parse from gst-plugins-bad, jpegparse) check the bitstream and the decoders (avdec_h264, avdec_h265 from gst-libav, jpegdec) decode every picture when they are installed; any error or warning fails the case gst-launch-1.0
MServerPerfFrameRate measured frame rate within +/-5 % of the configured rate, at 15, 25 and 30 fps ffmpeg

Every harness target excludes itself automatically when its prerequisite is missing. Without GStreamer, or without a fuzzing-capable clang, the suite still builds and runs; it simply covers less, and says so at configure time. Without FFmpeg the harness is not built at all (see above).

Each test binds ports of its own rather than the default 8554, so an unrelated RTSP server already running on the machine does not cause a false failure. Tests that do not test HLS switch the HLS leg off, because its default port 8080 is often taken on a development machine. Within the suite the tests still share the machine’s port space, so every C++ test and every interop script takes the same RESOURCE_LOCK. ctest -j is therefore safe; it simply never runs two of them at once.

The two WebRTC tests are deliberately different, and the difference is the point.

MServerWebRtcTest is not a mock. It builds a browser-like peer from sockets and OpenSSL, runs the real WHEP exchange, the real STUN check and the real DTLS handshake, and then re-implements the RFC 3711 key derivation to decrypt the media. A test that derived the keys with MServer’s own SRTP code would pass even if both sides agreed on the wrong thing. Deriving them independently from the DTLS export proves that the bytes on the wire are what the RFCs require.

It does not call initVStreamer(); it drives the endpoint directly. MServerWebRtcIntegrationTest covers that gap, which is where the defects were: the credential sentinel, CORS, what the answer actually contains, the packet size, the endpoint’s lifetime. None of them can be reached except through the public API, so the suite once stayed green on a configuration no browser could use. That test therefore includes no impl/ header at all, only MServer.h and a socket, which is everything a deployment has.

Sanitizers

The build files set no sanitizer options for the library or the tests. Sanitizers are a developer concern, not a property of the library, so you pass them on the command line, into a separate build directory:

cmake -S . -B build-asan -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -g"
cmake -S . -B build-tsan -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_CXX_FLAGS="-fsanitize=thread -fno-omit-frame-pointer -g"

Run the suite with the leak suppressions, or the H.265 tests fail on leaks that belong to libx265, not to MServer:

cd build-asan
LSAN_OPTIONS=suppressions=$PWD/../harness/lsan.supp ctest -E "Interop|Perf"

The suppression path must be absolute. ctest runs each test binary in a working directory of its own, so a relative path does not resolve, and the sanitizer aborts every test with “failed to read suppressions file”.

harness/lsan.supp is deliberately narrow: it names libx265, libx264, libavcodec and libavutil only, so a leak in MServer itself still fails the run. Those libraries reach the build through the test codec, never through the library, which MServerNoFFmpegDependency asserts.

ThreadSanitizer needs both a suppression file and ASLR off on recent kernels:

cd build-tsan
TSAN_OPTIONS=suppressions=$PWD/../harness/tsan.supp setarch $(uname -m) -R ctest -E "Interop|Perf"

harness/tsan.supp covers FormatConverter, ImageResizer and the OpenMP runtime. Those libraries spread their inner loops over OpenMP threads (two by default; see Compute threads for preprocessing). ThreadSanitizer cannot see the OpenMP runtime’s own synchronisation when that runtime is not instrumented, so it reports the worker threads’ stores as races. Each converter instance is owned by exactly one pipeline thread, which is the contract those libraries document. As with the leak file, the scope is narrow enough that a real race in MServer still fails the run.

One more note: FormatConverter enables its AVX2 code only in Release builds (through -march=native), so build sanitizer trees as Release rather than Debug to test the code that ships.

Run one build directory at a time. ctest’s RESOURCE_LOCK works within one build tree only. Two trees running at the same time still compete for the machine’s ports, and a listener already on 7000 or 8080 makes initVStreamer() fail; every affected test then reports FAIL : server initialised. That looks exactly like a product defect and is not one. A sanitised run and a release run at the same time are enough to cause it.

Every C++ test carries a 1800 s timeout in every build type, set for the slowest configuration: ThreadSanitizer costs roughly an order of magnitude, and the HLS suite waits out real segment durations on top of that. The interop scripts have 900 s (ffmpeg, frame rate) and 600 s (GStreamer).

Fuzzing

harness/fuzz/ holds eight libFuzzer targets that cover everything touching untrusted input:

  • fuzz_rtsp.cpp calls the RTSP URL and base64 helpers directly (fast, narrow).
  • fuzz_rtsp_server.cpp talks to a real listener over a real socket, so request framing, method dispatch, Transport parsing, digest parsing and the session state machine are all on the path (slow, broad).
  • fuzz_klv.cpp walks the MISB ST 0601 parser.
  • fuzz_bitstream.cpp covers Annex-B parsing and the packetizers.

The three parsers that WebRTC added have their own targets. fuzz_stun.cpp drives the STUN parser, the integrity check and the response builder directly. fuzz_http.cpp and fuzz_sdp.cpp talk to a real listener over a real socket: the first explores request framing, the second the SDP offer inside a fixed, valid WHEP request. All three parsers are reached from the network before anything has authenticated, which is why the targets exist.

fuzz_hls.cpp covers what HLS added, for the same reason: a path splitter, a strict decimal parser for segment numbers and key generations, and a resource dispatch, all reachable by an unauthenticated client. It drives a real listener whose stream has already been fed media, so the segment and key branches are reachable. A fuzzer that only ever gets 404 explores nothing.

CMake builds them when MSERVER_FUZZ is on. It is off by default, because libFuzzer needs a fuzzing-capable compiler (clang). MSERVER_FUZZ attaches -fsanitize=fuzzer,address,undefined to the fuzz targets only; libMServer itself is built with the normal flags. The fuzzer then gets no coverage feedback from inside the library, and ASan does not instrument the library’s code. The usual libFuzzer way to instrument the library too is to add -fsanitize=fuzzer-no-link,address,undefined to CMAKE_CXX_FLAGS; the build does not do this by itself. The basic build:

cmake -S . -B build-fuzz -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
      -DMSERVER_FUZZ=ON
cmake --build build-fuzz -j

mkdir -p corpus/rtsp
./build-fuzz/bin/fuzz_rtsp corpus/rtsp -max_total_time=600

Because CMake builds the fuzz targets, a target that no longer compiles against the library breaks that build instead of rotting unnoticed. They are not registered with ctest: a fuzz target runs until it is stopped.

Field test rig

The harness above proves the protocols against a software encoder on one host. A product does neither. The field rig runs the library on the target hardware and judges it from a different machine. The media therefore crosses a real network, and the decoder that judges it is somebody else’s implementation.

Two pieces:

  • MServerFieldServer (harness/field/field_server.cpp): a real MServer instance driven by VCodecLibva, the hardware encoder, and configured by key=value arguments. RTSP and HLS are on by default; RTSPS, WebRTC, the push leg and multicast are switched on by arguments. Its prerequisite is libva, not FFmpeg, so it can build on an embedded target that has a GPU and no FFmpeg. But only when the harness is kept there deliberately, with -DMSERVER_HARNESS_WITHOUT_FFMPEG=ON: a plain configure without FFmpeg drops the whole harness, this rig included.
  • field_test.sh (harness/field/field_test.sh): the driver. It starts the server over ssh on the sender, runs ffmpeg, ffprobe, curl and openssl on the receiver, and compares what arrived with what was asked for.
# Defaults: SENDER=192.168.1.83, RECEIVER=192.168.1.60
SENDER=10.0.0.5 RECEIVER=10.0.0.6 ./harness/field/field_test.sh

# One section at a time, for a failure worth looking at closely:
./harness/field/field_test.sh hls fps idle

Getting it onto the machines 

# Source to the sender, plus the hardware codec beside it
rsync -az --exclude='.git' --exclude='build*/' ./ user@SENDER:/home/pi/Downloads/MServer/
rsync -az build/VCodecLibva/ user@SENDER:/home/pi/Downloads/MServer/VCodecLibva/
ssh user@SENDER 'cd /home/pi/Downloads/MServer &&
    cmake -S . -B fieldbuild -DCMAKE_BUILD_TYPE=Release \
          -DMSERVER_VCODEC_LIBVA_DIR=$PWD/VCodecLibva &&
    cmake --build fieldbuild -j"$(nproc)"'

The receiver needs only ffmpeg, ffprobe, curl and openssl.

The sections

Section What it proves, at the far end
rtsp H.264 over TCP and over UDP; geometry; a frame count over a window
codecs H.265 and JPEG arrive and decode as hevc and mjpeg
hls the playlist is well-formed and live, and ffmpeg plays it through
hls_aes segments are ciphertext, and ffmpeg still plays them – it fetches the key itself
selfsigned TLS with no certificate: the generated identity, and that RTSPS and HLS present the same one
rtsps a real certificate is preferred over the fallback, and plaintext RTSP coexists under optional
auth digest on RTSP and HLS, and that a live password rotation reaches both
direct the push leg, as RTP and as bare MPEG-TS
klv KLV as its own SDP track with its own control URL, and as a data PID in the transport stream
runtime every runtime setParam under a watching client, and that invalid values are refused rather than clamped
fps 10 to 120 fps in steps of 10, each one counted at the receiver
busy a source feeding flat out does not carry the rate with it
resolution the input geometry changes under a watching client and the output does not move
idle with nobody watching the encoder does no work; the first client to arrive gets a decodable picture
stability a 60-second window: rate held, no send failure

The script has four more sections:

  • strict: every protocol through ffmpeg at debug level. Any error or warning line fails, apart from a few exceptions allowed by name.
  • telemetry: KLV and ONVIF metadata in every carriage, and KLV switched off and on again on the live object.
  • online_all: online parameter changes on a live object, with a client attached.
  • input_fps: the input rate swept around a fixed output rate, and the output rate swept against a fixed input rate.

With no arguments, the script runs all nineteen sections.

How the control channel works

The server reads one command per line on stdin. fs_launch.sh attaches that stdin to a FIFO, so the driver can send commands from separate ssh sessions without keeping one open. Commands: set/sets <PARAM> <value>, stats, rate, source <fps|busy>, res <W>x<H>, klv <0|1>, quit. Each one answers with exactly one line, so the driver reads one line per command.

Two things about that FIFO are worth knowing before you write anything against it; both cost real debugging time here:

  • A FIFO with no reader blocks the writer. A printf > /tmp/fs.cmd against a server that has already exited never returns, so every write in the scripts is wrapped in timeout.
  • pkill -f MServerFieldServer kills the ssh session that invoked the script, because that command line contains the name. And pkill -x MServerFieldServer matches nothing, because Linux truncates the process name to 15 characters and the binary’s name has 18. The scripts kill by the recorded PID, with a sweep by the truncated name as a fallback.

Troubleshooting

Each failure below is one that people meet on their first afternoon with the library. None of them is a bug.

The player says 404 Not Found, and the server is clearly running. A path exists for clients only once MServer can describe it, which for H.264 and H.265 means once the encoder has produced the parameter sets. Until then DESCRIBE is answered 404. Call getStats(). If framesIn climbs but framesEncoded stays at 0, your frames arrive and the encoder produces nothing. The usual cause is type, which selects the encoder backend and defaults to 0, hardware. On a machine with no usable GPU nothing is ever encoded; set params.type = 1 for the software encoder. MServer prints nothing about this itself, by design: it is a library, writes no log, and reports through getStats().

The player says 401 Unauthorized. Authentication is on for that stream. Put the credentials in the URL (rtsp://admin:admin@127.0.0.1:8554/live), or turn authentication off with user and password set to "no". That takes effect at once and does not disturb any other stream on the port. A stream that was never given credentials is served without a challenge, even while its neighbour is protected. A name that does not exist is answered 404, not 401. For WebRTC from your own page there is no way to put credentials in the URL, because a page’s fetch() cannot answer a digest challenge; switch authentication off for that stream, or use the built-in player page, which works with authentication on.

The TLS client refuses the certificate. Nothing signed it. Either MServer generated the certificate itself (an in-memory one, or one it wrote to the files you named, as for the test application), or you installed a self-signed one. getStats() tells the generated cases apart: tlsSelfSigned for the in-memory identity, tlsGeneratedToDisk for one written to disk. In each case the problem is the certificate, not the server: use curl -k, “proceed anyway” in a browser, or -tls_verify 0 for an ffmpeg build that verifies. In production, pass your own rtspCert / rtspKey. (Windows has no TLS at all.)

The HLS playlist is served but names no segments. The stream keeps encoding until it has closed its first segment, even with nobody watching, precisely so that this does not happen. A playlist that stays empty therefore means no segment can close. There are two causes: the codec is JPEG, for which the leg produces no segments at all (MPEG-TS has no stream type for it); or the encoder has stopped producing key frames. getStats().hlsSegments tells them apart: if it is 0 with a non-JPEG codec, look at framesEncoded next.

initVStreamer() fails with port conflicts. The VStreamer interface defaults hlsEnable and webRtcEnable to true. So a caller that never mentioned them still opens listeners on hlsPort (8080) and on webRtcPort (7000, with media on 7001). If another service already uses one of those ports, initialisation fails. Set params.hlsEnable = false or params.webRtcEnable = false if you do not want those legs.

ffplay -rtsp_transport http rtsps://... fails with an I/O error. ffmpeg tunnels RTSP over plain HTTP only. The server side is correct – see the note under How to play the stream.

WHEP answers 400, mentioning the payload type. The offer named no codec MServer can send. For H.264 the offer must carry packetization-mode=1. Browsers always do; hand-written offers often do not.

The browser negotiates, then shows black. Signalling succeeded and media did not arrive. Media does not travel on the signalling port – it is UDP on webRtcMediaPort, or on webRtcPort + 1 when that parameter is left at 0, and that port has to be reachable from wherever the browser runs.

initVStreamer() returns false. Start with the parameters: initVStreamer() checks them before it binds anything. The frequent causes are:

  • a port that another process already uses;
  • a value outside the documented range in VStreamerParams;
  • a strict security profile with no credentials or no certificate, which those profiles refuse by design rather than start an open server;
  • JPEG over MPEG-TS, for which STANAG 4609 and MISB ST 1402 define no stream type;
  • on Windows, any leg that asks for TLS (see Windows build with MSVC).

The full list is under initVStreamer method.

Two instances, and the second one changed the first one’s stream. First give them different suffix values: a second instance that publishes the same suffix publishes the same stream. If the suffixes differ, check what you changed against What the listener owns, and what the stream owns. The RTSP port really is shared, so changing it anywhere moves it everywhere; the bind address and the RTP port range must agree between instances. Credentials, bitrate and multicast are not shared, and a change to one stream’s copy of them cannot reach another stream.

The stream delivers fewer frames per second than the source produces. That is fps doing its job: the stream carries the configured rate, not the source’s. getStats() says which side of it you are on – framesDropped climbing means the source is faster, framesDuplicated climbing means it is slower.

A client reports missing packets and the network is clean. Read rtpSendFailures and directSendFailures before suspecting the network: a non-zero value means the kernel refused datagrams this server had already numbered, which a receiver reports as loss. Both stay at zero unless a socket is genuinely congested.

The push leg sends nothing after directStreamEnable was set. Check that directStreamIp and directStreamPort are usable. An address of "no", an empty address or a port of 0 leaves the leg with nowhere to go, and that is the one case where enabling it does nothing. (A strict compliance profile refuses the leg; then setParam() returns FALSE.) Everything else about the leg applies immediately.

Multicast works on one stream and not on another. The multicast configuration is per stream: a pool set on one instance is not inherited by the others. And a group is silent until a client sets it up – configuring a pool does not put anything on the wire.

The HLS playlist stays empty while RTSP works, and the codec is JPEG. MPEG-TS has no stream type for motion JPEG, so the HLS leg produces no segments for it rather than segments a player would decode as the wrong codec. Serve JPEG over RTSP, or switch the codec to H.264 or H.265 for the HLS leg.

Limitations

  • IPv4 only. Every inherited helper is AF_INET, and SETUP, SDP and multicast scoping have no IPv6 path.
  • The metadata track needs its own SETUP. On the codec-RTP path the metadata is declared as m=application 0 RTP/AVP 98 with an a=control:.../trackID=1 of its own. The port is 0 in a unicast presentation, because SETUP negotiates the transport and gives the track its own socket pair or interleaved channel pair. In a multicast presentation the port is real: the pair above the video one. KLV is never pushed onto the video transport, so a unicast or interleaved client that sets up only trackID=0 receives no metadata at all. A multicast group carries the metadata whether or not any member set the track up, but a member still has to listen on that second port to see it.
  • One MTU per stream, per leg. All RTSP/RTSPS/WebRTC clients of a stream share one packetization, video and metadata track alike. Both are sized from serverStreamMaxPayloadSize, and both are capped when WebRTC is enabled. A client that needs a smaller MTU must therefore be served by a separate instance. The direct push leg is the exception: it has its own packet size and packetizes separately when that size differs, its own metadata track included.
  • Multicast drops apply to the whole group. One datagram serves every member, so per-member queueing does not exist.
  • RTCP RR cannot report server-side drops. Because a renumbered sequence hides them by design, Stats counters are the only signal that a client is starving; never wire rate control to RR fraction lost.
  • SRTP key derivation is outside the validated module. See the compliance modes above. SRTP is negotiated with Transport: RTP/SAVP and is offered only on the TLS listener, because ONVIF forbids returning SAVP or MIKEY over an unprotected control channel.
  • MIKEY is built but not advertised. Only the pre-shared-key form is implemented (the certificate and Diffie-Hellman modes of RFC 3830 are not), and it has never been checked against a real ONVIF client, so no mode offers it. SDES carries the keys instead. ONVIF Core sec. 5.1.1.4 makes MIKEY mandatory for SecureRTSPStreaming, so this is a known gap against ONVIF, not a design choice: offering an unverified key exchange would let a client negotiate keys it cannot derive and fail silently, which is worse.
  • ONVIF conformance belongs to the device, not the library. MServer implements the ONVIF Streaming Specification and exposes the control points a host’s SOAP layer needs; it contains no SOAP service itself.
  • MJPEG above 2040 px, or with a size that is not a multiple of 8, is refused (RFC 2435 limit).
  • RTSP over HTTPS is verified server-side with a TLS client and by the VSourceStream test suite (the tunnel over TLS, also carrying SRTP and MPEG-TS, with digest authentication), but not against ffmpeg, which does not drive that combination.
  • ffmpeg cannot reach the strict compliance modes. ffmpeg 6.1.1 implements MD5 digest only, and every mode except ONVIF refuses MD5. Those modes are covered by MServerStrictModesTest, which brings its own RFC 7616 SHA-256 client over TLS.
  • The validated FIPS provider is not exercised here. It is absent from the development environment. What is tested is that STRICT_FIPS and FIPS_ONVIF_BRIDGE refuse to start without it (2.1.3) and that CRA_STRICT runs the hardened configuration on the default provider.
  • No TLS, DTLS or SRTP on Windows. RTSPS, HTTPS, the generated TLS identity, SRTP over RTSP and WebRTC are Linux-only, and the FIPS statements do not apply to a Windows build. See Windows build with MSVC.
  • A compliance mode tightens the port it shares. The plaintext listener is process-wide. Its transport policy (whether MD5 digest is offered, whether SRTP may be negotiated, whether interleaved-inside-TLS is required) is therefore the most restrictive combination of all attached streams, not each stream’s own. Credentials are the exception: they are per path. So a claim about who may read a stream is about that stream, while a claim about how the challenge is computed is about the port.
  • The container of the served leg cannot change without rebuilding the stream. MServer does the rebuild itself rather than refusing the command, but clients are dropped and reconnect; see Re-initialising instead of refusing.
  • fps is held by dropping and duplicating, not by asking the source for more. A source slower than fps produces a stream at the configured rate with repeated pictures. That is right for a receiver that needs a stable timeline, and wrong for a caller who wanted to know that the source had stalled. framesDuplicated is where that shows.
  • Configuring writes MServerVersion.h into the source tree. That is the convention every library in this family follows, so a fully read-only source checkout cannot be configured.
  • The nested submodules of the test codec are not de-duplicated. The VCodecLibav target is configured once, but the VCodec and Frame checkouts it and VStreamer each carry are separate directories on the include path. They are pinned independently upstream, so this is a property of the dependency graph rather than of this build.
  • Not verified against VLC or the ONVIF Device Test Tool. Neither is available in the development environment; the tool is a Windows-only proprietary application and conformance is granted to a device, not to a library.
  • WebRTC is not verified against a real browser. No browser is available in the development environment. What is verified is the protocol: the peer in MServerWebRtcTest performs the same exchange a browser does, with OpenSSL on the other side of the DTLS handshake and an independent SRTP implementation decrypting the media. Interoperability with a specific browser’s SDP dialect is therefore untested.
  • HLS is high-latency by construction. A player needs about three target durations of media before it can start, so roughly 6 s at the 2 s default. Low-latency HLS – partial segments and blocking playlist reload – is not implemented. Use RTSP or WebRTC where the delay matters.
  • HLS carries no metadata track. KLV travels inside the segments’ transport stream on its own PID, as on the push leg. But HLS defines no way to signal it in the playlist, so a player reads the video and ignores the KLV. To consume metadata, use the RTSP leg or the push leg.
  • HLS serves H.265 as MPEG-TS, which ffmpeg, VLC and GStreamer play and Safari does not: Apple devices need HEVC in fragmented MP4 and there is no fMP4 muxer here.
  • HLS produces no segments for JPEG. Neither STANAG 4609 nor MISB ST 1402 defines an MPEG-TS stream type for motion JPEG, so the playlist stays valid and empty rather than naming segments a player would decode as the wrong codec.
  • An HLS stream costs one segment of encoding before it may idle. The leg has nothing to serve until a segment exists. So the pipeline runs, with nobody watching, until it has closed one segment, and then parks like any other stream. This happens once, and never for JPEG, which produces no segments at all.
  • The generated TLS identity is not authenticated and does not survive a restart. See Certificates and keys. It is a first-run convenience: a deployment that must be verifiable installs a certificate.
  • WebRTC carries no audio, no data channel and no simulcast, offers no TURN or STUN server, and does no retransmission or congestion control. A host candidate only, so the media port must be reachable from the viewer.
  • The WebRTC media port defaults to webRtcPort + 1 and can be pinned with webRtcMediaPort. It is read when the endpoint starts, and the endpoint is a process-wide singleton that lives until the last published stream goes away, so changing it while any stream is using it has no effect until the next service start.

Table of contents