mserver_web_logo

MServer C++ library

v1.0.0

Table of contents

Overview

MServer is a C++ library that implements the VStreamer interface and serves H.264 (RFC 6184), H.265 (RFC 7798) and MJPEG (RFC 2435) video over RTSP, RTSPS, RTSP over HTTP/HTTPS, UDP multicast, direct RTP push and WebRTC. MServer spawns no external process and links exactly one external library – OpenSSL, which is required in order to make FIPS 140-3 statements at all. The library supplies RAW frames; everything else – pixel-format conversion, letterbox/crop scaling, overlay, encoding, Annex-B parsing, parameter-set extraction, RTP packetization, MPEG-TS multiplexing, KLV carriage, RTCP, TLS, digest authentication, ICE, DTLS-SRTP, WHEP signalling, session management and pacing – is handled internally. Note: B-frames are not supported. This is deliberate and is what allows PTS to equal DTS throughout, so RTP timestamps are non-decreasing and no reordering buffer is needed anywhere in the chain.

Implemented protocols and standards

Area Standard Status
Control RTSP 1.0 – RFC 2326 full server: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET/SET_PARAMETER, state machine with 404/405/454/455/457/461/501/551
  RTSPS (RTSP over TLS) yes, own listener, TLS 1.2 minimum
  RTSP over HTTP / HTTPS tunnel (Apple/QuickTime) yes, GET+POST paired by x-sessioncookie
  SDP – RFC 4566 a=control, a=rtpmap, a=fmtp, a=range, multicast c= with TTL
  Digest auth – RFC 7616 (SHA-256), RFC 2617 (MD5) both, selectable by compliance mode; rotating nonces, stale=TRUE re-challenge, per-cnonce nonce-count replay rejection
WebRTC WHEP signalling over HTTP/1.1 and HTTPS one POST carrying the offer, one DELETE ending the session; same digest authentication as RTSP
  ICE-lite – RFC 8445 sec. 2.4 host candidate only, never probes back; STUN RFC 5389 with MESSAGE-INTEGRITY and FINGERPRINT
  DTLS-SRTP – RFC 5764 DTLS 1.2 server role, self-signed certificate bound by a=fingerprint, keys from SSL_export_keying_material
  Demultiplexing – RFC 7983 STUN, DTLS, SRTP and SRTCP on one port; a=rtcp-mux
  Feedback – RFC 4585 / RFC 5104 PLI and FIR are honoured as key-frame requests
Transport RTP / RTCP – RFC 3550 RTP + Sender Reports with NTP<->RTP mapping, SDES, RR reception as keep-alive
  UDP unicast, UDP multicast, TCP interleaved all three
  Direct RTP push (no RTSP) yes, list of destinations, unicast or multicast
  SRTP – RFC 3711, RFC 7714 AES_CM_128_HMAC_SHA1_80, AEAD_AES_128_GCM
  SRTCP – RFC 3711 sec. 3.4, RFC 7714 sec. 9.1 both profiles, own KDF labels, 31-bit index, replay window; applied on WebRTC and on SAVP RTSP subscribers
  Key management – SDES RFC 4568 a=crypto in the SDP of the secure presentation, TLS listener only; proved end to end by a test that decrypts the media with the key it advertised
  Key management – MIKEY RFC 3830 pre-shared-key form is built (HDR/T/RAND/KEMAC+MAC) but not advertised: it has never been checked against a real ONVIF client, and offering an unverified key exchange fails worse than not offering one
Video H.264 over RTP – RFC 6184 single NAL + FU-A, packetization-mode=1
  H.265 over RTP – RFC 7798 single NAL + FU, no DONL
  MJPEG over RTP – RFC 2435 in-line quantization tables, restart markers; <= 2040x2040
  No B-frames enforced and proven by an automated slice_type census
Container MPEG-TS over RTP – RFC 2250 + MISB ST 0804 PT 33, 7x188 per packet, PAT/PMT/PCR/PES
  MPEG-TS over UDP – MISB ST 1402 on the direct-push leg (7x188 per datagram, no RTP header)
Metadata MISB ST 0601 (UAS Datalink Local Set) byte-exact pass-through, optional validation, optional restamp
  KLV over RTP – RFC 6597 smpte336m/90000 on its own payload type
  KLV in MPEG-TS – MISB ST 1402.1 asynchronous (0x06 + KLVA) and synchronous (0x15 + metadata descriptors)
  MISP precision timestamp – MISB ST 0604.6 emitted as an SEI, H.264 and H.265

Regulatory compatibility

Standard Status How it is achieved
ONVIF Profile S / T Streaming Specification implemented; conformance belongs to the device all mandated transports, digest auth, keep-alive by any request and by RTCP RR, SetSynchronizationPoint via GENERATE_KEYFRAME, multicast with real group address. MServer has no SOAP layer – it exposes the control points a host’s SOAP service drives
FIPS 140-3 consumes CMVP #4985 (OpenSSL FIPS Provider 3.1.2); MServer itself is not a validated module private OSSL_LIB_CTX, RAND_bytes_ex for every secret, provider enforcement proved at load (an approved fetch must succeed and MD5 must fail), and a mode that runs without the module says so instead of claiming validation
EU CRA (Reg. 2024/2847) engineering requirements implemented secure-by-default mode, bounded parsers, no UDP amplification, threat model (docs/THREAT_MODEL.md), fuzzing. Note: the coordinated vulnerability-disclosure policy Article 13(8) expects is not in this repository and must be supplied by the product that ships it.

Why compliance is switchable

These three standards contradict each other on the wire, so no single configuration satisfies all of them:

  • ONVIF Core sec. 5.9.3 makes MD5 the default RTSP digest, and nearly every deployed client implements only MD5. The FIPS provider does not implement MD5 at all.
  • ONVIF sec. 5.1.1.4 requires SRTP, but both SRTP profiles derive keys with the SRTP KDF, which is absent from the approved-algorithm table of CMVP #4985 – so any SRTP puts key derivation outside the validated module.
  • AES-GCM SRTP builds its IV externally, which the module’s security policy names as a non-conformance.
  • The CRA wants encryption and authentication on by default; ONVIF interoperability testing expects a device to stream with no prior configuration.

MServer therefore makes the choice a runtime switch (VStreamerParams::securityProfile), so one product can be certified against several standards: each evaluation runs in the mode it requires, and the mode is a documented configuration item rather than a separate firmware image. A FIPS mode still starts when the validated module is absent – its restrictions are MServer’s own policy, not the module’s – but it then reports itself as (unvalidated provider), so a FIPS claim can never be made by accident. See Compliance summary.

See Security and compliance modes for the full matrix and docs/THREAT_MODEL.md for the analysis.

Versions

Version Release date What’s new
1.0.0 23.08.2026 - First version.

Library files 

src/
  MServer.h                   the ONLY public header (pimpl)
  MServer.cpp                 VStreamer implementation and frame pipeline
  impl/
    MServerBitstream.{h,cpp}  Annex-B parsing, parameter sets, slice_type
    MServerRtp.{h,cpp}        RTP packetizers, 90 kHz clock, RTCP
    MServerRtsp.{h,cpp}       RTSP server, SDP, tunnel, endpoint registry
    MServerCrypto.{h,cpp}     OpenSSL: providers, digests, DRBG, TLS contexts
    MServerSrtp.{h,cpp}       SRTP, SDES, MIKEY
    MServerTs.{h,cpp}         MPEG-TS muxer, MISB ST 0601 KLV
    MServerIce.{h,cpp}        STUN, ICE-lite agent
    MServerDtls.{h,cpp}       DTLS-SRTP handshake and key export
    MServerHttp.{h,cpp}       HTTP/1.1 and HTTPS signalling listener
    MServerWebRtc.{h,cpp}     WHEP endpoint, sessions, SRTP send loop
    MServerCompliance.{h,cpp} compliance-mode policy
example/                      smallest working program
test/                         interactive test application for users
harness/                      full automated functional and performance suite
  fuzz/                       libFuzzer targets for the untrusted parsers

Consumers include MServer.h and nothing else; the impl/ headers are not installed.

Internal architecture

Frame pipeline

Two stages decoupled by a single slot, so sendFrame() never blocks on encoding or on the network:

sendFrame() -> [slot] -> Stage A -> [slot] -> Stage B -> packetizer -> subscribers
                       convert           encode
                       resize            parse NAL
                       overlay           cache SPS/PPS/VPS
                       -> NV12

The helper libraries fix the route when scaling or overlay is in play: ImageResizer accepts only 3-byte interleaved data and VOverlay draws on it, so that chain is any RAW -> YUV24 -> resize -> overlay -> NV12 -> encoder. All of it runs on the pipeline thread unless custom1 grants it more – see Compute threads for preprocessing. When neither is needed, the pipeline does not walk that chain. If the stream size equals the input size and the overlay is off, the only requirement left is the encoder’s input format, and FormatConverter reaches NV12 from every RAW format in a single pass:

Input Stream size = input size, overlay off Otherwise
NV12 nothing at all – the frame goes to the encoder untouched RAW -> YUV24 -> resize -> overlay -> NV12
YUV24, RGB24, BGR24, YUYV, UYVY, GRAY, … one conversion, straight to NV12 RAW -> YUV24 -> resize -> overlay -> NV12
H.264 / H.265 / JPEG pass-through: no conversion, no scaling – re-encoding compressed input is out of scope same

So a caller that already produces NV12 at the stream resolution pays for no conversion, no scaling and no copy in preprocessing. Both helpers are stateful and not thread-safe, so each pipeline thread owns its own instance.

Holding the configured frame rate

sendFrame() is called at the source’s rate, which is not necessarily the configured one. Holding fps is the server’s job, and it is done in the preprocess stage, between the input slot and the encoder:

Source What happens Counter
faster than fps (a 60 fps camera on a 30 fps stream) the frame waits in the input slot until its deadline; sendFrame() overwrites the slot with anything newer, so the deadline is met with the newest picture framesDropped counts the superseded ones
on rate nothing: a frame up to 1/8 of a period late is still used for its period, so two unsynchronised clocks do not cost a fifth of the frames –
slower than fps (25 fps on a 30 fps stream) the previous picture is repeated to fill the period framesDuplicated

The deadline grid advances by whole periods, so the long-run rate is exactly the configured one whatever the source does. This is not cosmetic: the encoder allocates bits per frame for fps, so a 60 fps source on a 30 fps stream used to produce twice the configured bitrate, overflow the subscriber queues and freeze every RTSP client for a second at a time.

Four cases deliberately do not repeat a picture:

  • nothing has arrived yet – there is nothing to repeat;
  • the source has been silent for two seconds – a frozen picture transmitted for as long as the service runs hides the failure, so the stream stops instead and resumes when the source does;
  • nobody is watching – repeating for no one would undo Idle streams cost nothing for every stream that ever had a client. A stream switched off with VStreamerCommand::OFF stops repeating at once, not after the grace window;
  • already-compressed input (pass-through) – re-sending an encoded P-frame makes the decoder apply the same residual twice, which is a corrupt picture rather than a still one, so a pass-through source keeps its own rate.

framesDuplicated is the diagnostic worth watching: a stream whose source matches its fps never duplicates, and a rising count means the source is short of the rate the SDP advertises.

Shared-stream model

Many clients may be connected at once and all receive the same stream. An access unit is packetized exactly once and the resulting payload bytes are shared by every subscriber; only the 12-byte RTP header differs. UDP unicast uses a two-element iovec (private header + shared payload) so the media bytes are never copied.

Sequence numbering and drop policy

The RTP sequence number is assigned per subscriber at transmission time, not at packetization time. This matters: each subscriber has its own bounded queue and may drop when it falls behind. If the number were baked into the shared packet, a drop would punch a hole in that subscriber’s sequence space, and a conformant receiver reads a hole as packet loss – its jitter buffer then waits out the reorder timeout on every drop. Because a packet that is never transmitted never consumes a number, a subscriber that drops sees a contiguous sequence with a jump in the RTP timestamp, which is indistinguishable from a source that lowered its frame rate. Dropping is always by whole access unit, followed by a hold until the next key frame. Truncating a unit mid-way would let the receiver reassemble a corrupt NAL from surviving fragments, so RtpSequencer detects that case and deliberately inserts a gap instead; truncatedUnits() must always read zero.

Shared listening port

All instances of MServer in a process share one RTSP listener, owned by a process-wide registry. Requirements this satisfies:

  • a second instance joins the existing port rather than opening its own;
  • changing the port through any instance re-binds the shared listener, so every instance follows;
  • re-binding is open-before-close with rollback – a failed change leaves the previous listener serving;
  • instances are told apart by the URL path (the stream suffix); a duplicate suffix is refused;
  • the last instance to detach releases the port.

Only the plaintext listener is shared. The RTSPS listener is each instance’s own socket, so rtspsPort is a per-instance value and two instances must not name the same one: the second bind() fails, and its initVStreamer() with it. Give every TLS stream a port of its own. It is also why the TLS listener takes its credentials as a listener default rather than per path – it serves exactly one stream.

What the listener owns, and what the stream owns

A shared listener does not make every parameter shared. The dividing line is whether the value describes the socket or the resource, and it decides what an operator can expect when they change one stream:

Owned by the listener – one value for the whole process Owned by the stream – its own value, independent of its neighbours
rtspPort (re-binds for every instance), the WHEP signalling and media ports user / password
bindAddress, rtpPortMin / rtpPortMax – refused when they disagree with a running listener, rather than merged bitrateKbps and the send budget derived from it
the HTTP tunnel, and the compliance policy, which is the most restrictive combination of every attached stream and is lifted when a stream detaches rtspMulticastIp / rtspMulticastPort / multicastTtl
corsAllowedOrigin, which the shared signalling endpoint holds one of rtspsPort – the TLS listener is the stream’s own socket, so two streams cannot share a value
  everything about the picture, the encoder, the metadata and the push leg

Two of those rows used to be the other way round, and both were operator-visible faults:

  • Credentials are per path, not per listener. The listener answers a challenge for the path the request names, so two streams on one port may demand different passwords and a stream that wants none is served without a challenge next to a protected one. A single listener-wide pair meant that a password set on one stream challenged every other stream on the port – with a password their clients did not have – and that a second stream asking for a different one was refused outright. A credential change closes the sessions of that path alone: a client that is already PLAYING sends no further request, so without that it would keep receiving a stream whose password had just been revoked.
  • The send budget is per path. Each subscriber is paced against its own stream’s ceiling; a single process-wide value meant the last stream to initialise paced everybody, so a 2500 kbps stream – even a disabled one, which still initialises – throttled a 4096 kbps stream’s subscribers to 2500, overflowed their queues and dropped them to the next key frame every few seconds.

Threading model

Thread Count Responsibility
caller external sendFrame() copies into the input slot and returns
preprocess 1 per instance convert, resize, overlay, NV12
encoder 1 per instance encode, parse NAL, publish access unit
reactor 1 per listener poll() over the listener and control connections, TLS handshakes, RTSP parsing
sender 1 per listener drains subscriber queues, paces, sends RTP and RTCP
signalling 1 per process, only with WebRTC poll() over the WHEP listener, TLS handshakes, HTTP parsing
WebRTC media 1 per process, only with WebRTC one UDP socket: STUN checks, DTLS handshakes, SRTP out, RTCP in
direct push 1 per instance, whenever the push leg has a destination drains the push queue, applies directStreamBitrateKbps pacing, sends RTP/UDP and its Sender Reports. Separate on purpose: pacing means waiting, and waiting on the encoder thread would hold every RTSP, RTSPS and WebRTC client back to the push leg’s rate. Started even when directStreamEnable is FALSE – it waits on an empty queue, which costs nothing and is what makes switching the leg on at runtime possible

Media is never written from the encoder thread: a blocking write to one slow client would otherwise delay every other client and the control channel.

The sender thread never waits for a socket, either. Every media socket is given a 4 MB send buffer, and a datagram the kernel still refuses is counted (rtpSendFailures, directSendFailures) and skipped – deliberately not retried. One sender thread serves every subscriber of a listener, so pausing for a congested socket would stall every healthy client behind it. Before that buffer existed, a 1080p key frame handed to a default-sized socket had its tail refused with ENOBUFS, and because the sequence number had already been consumed the receiver reported it as loss while the server reported nothing.

Idle streams cost nothing

Conversion, scaling, overlay and encoding are the expensive part of this library. With nobody watching, every cycle of it is waste — so MServer ignores input while a stream has no consumer, and picks it up again the instant it gains one.

The frame is dropped in sendFrame(), before it is even copied, so the whole chain behind it is skipped. sendFrame() still returns TRUE: the frame was accepted and dealt with, and the caller has no decision to make about it.

Measured on a 640×480 H.264 stream at 25 fps: 8.5 % of a core with one viewer, 0.6 % with none. What is left is the caller handing frames over and a four-times-a-second consumer check — on a real device it is whatever the capture path costs, which MServer does not control.

Nothing is asked of the calling code. Keep sending frames exactly as before. getStats() reports what happened: framesIgnored counts the skipped frames and idle says whether the stream is currently ignoring input. A watchdog that reads framesEncoded to decide the pipeline is alive must consult idle too — a healthy unwatched stream encodes nothing.

framesIgnored is deliberately separate from framesDropped. The latter counts frames a NEWER one superseded in a hand-off slot — usually because the source runs faster than the configured fps, where it is exactly what holding the rate means, and only sometimes because encoding fell behind. Reading one for the other would turn a healthy idle server into an overloaded one, or hide a rate mismatch behind an idle count.

What counts as a consumer

Wider than “client”, because two of them have no client to count:

  Consumer while
RTSP / RTSPS a session is in PLAYING
WebRTC a session has completed DTLS and media is flowing — a page that fetched an SDP answer but never finished ICE is not watching
UDP multicast the group is transmitting. Members join without telling the server and there is no back channel, so once a group has been set up the stream stays awake for as long as it is assigned
Direct RTP push the leg is enabled. It has a destination rather than a client, and nothing on the far end reports back

The census is per path, not per listener. Instances that bind the same port share one listener, so counting its sessions would keep every stream in the process encoding because one of them has a viewer.

Waking up

Every transition — a PLAY, a WebRTC session becoming ready, a connection closing — bumps a process-wide counter. sendFrame() compares one atomic against the value it last acted on, and recounts only when it has moved, so the steady-state cost is a load and a comparison. A 250 ms poll backs it up for the states nothing announces, such as a session swept away by the idle timeout.

The first client gets a key frame immediately. Waking asks the encoder for one and holds the publish gate shut until it arrives, so the client receives SPS, PPS and an IDR rather than a GOP of undecodable inter frames. Measured on a 25-frame GOP at 25 fps, where a client left to wait for the next natural key frame takes up to a second: PLAY to a complete IDR in 9–30 ms, which is faster than a client joining an already-running stream.

What stays awake regardless

A stream keeps encoding until its description exists, because the description is made of encoder output: the SDP’s sprop-parameter-sets come from the first encoded frames. Idling before that would answer every DESCRIBE with 404 and nothing could ever clear it. In practice that is a handful of frames at start-up. A runtime codec change puts the stream back in that state until the new parameter sets arrive.

MPEG-TS is the exception in the other direction: the codec is signalled in the PMT rather than in SDP, so there is nothing to stay awake for and such a stream idles from the very first frame, having encoded nothing at all.

DESCRIBE and WHEP are answered normally while idle — the description is cached, so signalling never waits for the pipeline.

Compute threads for preprocessing

Conversion, scaling and the overlay are the only work MServer parallelises, and by default it does not: they run on the pipeline thread. custom1 is how many threads they may use.

params.custom1 = 4;   // the converter and the scaler may take up to 4 threads
Value Meaning
0 the same as 1. It is the interface’s own default for the field, so a caller who never touched custom1 must not silently be given the whole machine
1 the default. Conversion, scaling and the overlay stay on the pipeline thread
n up to n threads, capped at what the machine has
above the hardware thread count refused by setParam() – a caller asking for more cores than exist has misconfigured something, and clamping would hide it – but clamped by initVStreamer(), which does not range-check the structure it is given. A large value passed at initialisation therefore does not fail: it quietly buys a worker team as wide as the machine

The bound is std::thread::hardware_concurrency(). setParam(CUSTOM1, …) range-checks against it; initVStreamer() applies whatever the structure holds, clamped to that bound. A refused value leaves the stored one alone.

A warning for callers that reuse custom1 for something else. Before VStreamer 3.2 custom1 was a deprecated alias for other fields, and an application that still mirrors a value into it - a payload size, for instance - is not passing an opaque number: it is asking for that many threads, and initVStreamer() will clamp rather than complain. If custom1 carries application data in your code, set the thread count last, immediately before initVStreamer().

It changes live. The pipeline thread picks the new limit up on its next frame, so a deployment can trade latency against cores without restarting the stream. Reducing the limit takes effect on the next frame as well, although the worker threads themselves stay parked until the stream closes – that is OpenMP’s pool, not MServer’s, and a parked thread costs nothing but its stack.

Why the pipeline thread applies it. OpenMP keeps a worker team per thread, and its thread-count control is per task rather than per process. Setting the limit from whichever thread happened to call setParam() would warm a team that the conversions never touch, leave a second one to be built on first use, and race with the pipeline thread reading the value. So the request is recorded and the thread that owns the team applies it.

What to set it to. Leave it at 1 unless measurement says otherwise. The pipeline is already a chain of threads – source, preprocessing, encoding, sending – and a stream that meets its frame rate gains nothing from more. It earns its keep on large frames with a scale factor: 4K input scaled to 1080p, several streams on one machine, or a CPU too slow to convert a frame inside its frame period. Measured cost is visible in getStats().cycleTimeUs, which is the interval between encoded frames.

Transports

Transport SETUP form Notes
RTP/UDP unicast RTP/AVP;unicast;client_port=n-n+1 server port pair is even/odd per RFC 3550 sec. 11
RTP/RTSP/TCP interleaved RTP/AVP/TCP;interleaved=0-1 $-framed, control prioritised over media
RTP/UDP multicast RTP/AVP;multicast one sequence space for the group, TTL bounded
RTSPS TLS on its own port media is protected only when interleaved inside TLS
RTSP over HTTP/HTTPS GET+POST tied by x-sessioncookie Apple/QuickTime de-facto tunnel
Direct RTP push not RTSP at all UDP only, fixed destination list, unicast or multicast; RTP or bare MPEG-TS, see stream type values. Container, payload size and bitrate are independent of the RTSP/RTSPS/WebRTC legs, and the leg transmits on its own thread so its pacing cannot hold the other transports back
WebRTC not RTSP at all WHEP over HTTP or HTTPS, then DTLS-SRTP on one UDP port; see WebRTC

What each transport carries, and to which standard

Every delivery path and the specification it follows, so a receiver can be matched to a mode without reading the source:

Path Container / framing Standard Metadata
RTSP / RTSPS, rtp codec-specific RTP H.264 RFC 6184, H.265 RFC 7798, MJPEG RFC 2435 none
RTSP / RTSPS, rtp-klv* codec-specific RTP + a muxed metadata track as above, plus KLV over RTP RFC 6597 m=application PT 98, smpte336m/90000 or vnd.onvif.metadata/90000
RTSP / RTSPS, mpegts-rtp-klv* MPEG-TS inside RTP RFC 2250 PT 33, MISB ST 1403 KLV on its own PID
Direct push, rtp codec-specific RTP over UDP RFC 6184 / 7798 / 2435 none
Direct push, rtp-klv* codec-specific RTP over UDP + its own metadata track RFC 6184 / 7798 / 2435, plus KLV over RTP RFC 6597 PT 98 with its own SSRC and sequence space, muxed onto the same destination port and told apart by payload type. Selected by directStreamType independently of the served leg; there is no SDP on this leg, so the receiver has to be told out of band
Direct push, mpegts-klv-* MPEG-TS straight into UDP, 7x188 per datagram STANAG 4609, MISB ST 1402 KLV PID, async or sync
Direct push, mpegts-rtp-klv-* MPEG-TS inside RTP RFC 2250 PT 33, MISB ST 1403 KLV PID, async or sync
WebRTC codec-specific RTP inside SRTP RFC 5764 DTLS-SRTP, RFC 8445 ICE-lite, WHEP none

Asynchronous vs synchronous KLV (the -klv-async / -klv-sync suffixes): asynchronous carriage is PES private_data with stream type 0x06 and a KLVA registration descriptor – raw KLV, no wrapper. Synchronous carriage is stream type 0x15 with a metadata_descriptor, the KLV wrapped in a metadata_AU_cell and aligned to PTS, which is what MISB ST 0604 requires and what a STANAG 4609 receiver expects when the metadata must correlate to a frame. mpegts-rtp-klv selects the synchronous form.

How a client learns the metadata format. On the MPEG-TS paths it is in the PMT. On the codec-RTP paths it is in the SDP, which is the only thing an ONVIF client can read:

m=video 0 RTP/AVP 96
a=rtpmap:96 H264/90000
a=control:rtsp://host:8554/live/trackID=0
m=application 0 RTP/AVP 98
a=rtpmap:98 SMPTE336M/90000
a=control:rtsp://host:8554/live/trackID=1

The second section is emitted whenever the stream carries metadata on a codec-RTP container. It is a real second track: it has its own control URL, the client SETUPs it separately, and the KLV then arrives on its own transport with its own SSRC and sequence space. A client that wants only video sets up trackID=0 and is never sent metadata packets it did not negotiate. metadataSuffix chooses the name: SMPTE336M for MISB ST 0601 KLV, VND.ONVIF.METADATA for an ONVIF metadata stream.

The a=control line is not decoration. A media section without one is aggregate-controlled, and both ffmpeg and GStreamer then SETUP the session URL for it — which replaced the video transport and left the stream silent on the channel the client was listening to. An earlier version omitted it and no standard client could play a rtp-klv stream at all; the SDP text was checked, a real client was not. MServerDirectPushTest now plays one.

SETUP of trackID=1 opens a session of its own when none exists, and joins the video session when one does. RFC 2326 s10.4 lets a client set up any single media stream of a presentation, and an ONVIF metadata client does exactly that – it never touches the video track. This used to be refused with 455 Method Not Valid In This State, which made every ONVIF metadata-streaming test fail at SETUP.

A metadata-only session carries no video transport, so:

  • the video fan-out skips it – it is never sent packets it did not negotiate;
  • the key-frame gate does not apply to it: that gate exists so a client is not shown pictures it cannot decode, and there are no pictures here;
  • RTP-Info names trackID=1, the track the session actually plays, and the sequence number comes from the metadata space;
  • interleaved channels 0-1 are available to it, because no video track has claimed them;
  • its metadata sequencer is configured here, with its own SSRC, its own initial sequence number and payload type 98. Every video SETUP configures both sequencers, so a session that took the video track first arrived with a correct metadata source; this one never passes through those branches, and an unconfigured sequencer defaults to SSRC 0, sequence 0 and payload type 96 – the H.264 one. The SDP announces the track as RTP/AVP 98, so a client binds to 98 and discards every packet stamped 96: the server sees SETUP and PLAY answered, datagrams leaving and its counters rising, while the client reports no metadata at all, identically over UDP, interleaved TCP and the HTTP tunnel;
  • clientAddr is recorded here too. The video SETUP records it in its UDP-unicast branch, which this session never reaches, so it used to stay 0 and every metadata datagram was addressed to 0.0.0.0 – which the kernel resolves to the local host. The server sent the stream to itself and sendto() reported success, so nothing counted a failure. Interleaved TCP and the HTTP tunnel write down the RTSP connection and never read the field, which is why only the UDP-unicast metadata tests failed once the payload type was right;
  • SRTP gets its own context. The metadata track is a second RTP source and so a second cryptographic context (RFC 3711 s3.2.1), keyed from the same master material. Without one the send path – which encrypts only when the context pointer is non-null – emitted the KLV in the clear, to a client that had been told the track is SRTP. Under requireEncryptedMedia that was worse than a downgrade: the transport guard passes because the client asked for /SAVP, so the profile that promises never to emit unprotected media emitted unprotected metadata. The Transport reply now names RTP/SAVP when it did select SRTP, as RFC 2326 s12.39 requires.

The transport policy is the same one a video SETUP faces: with requireInterleaved or requireEncryptedMedia set – the STRICT_FIPS, FIPS_ONVIF_BRIDGE and CRA_STRICT profiles – a metadata-only SETUP on plain UDP is refused 461 exactly as a video one is. The metadata track is not a way around the profile.

A SETUP of trackID=1 on a stream whose SDP advertises no metadata track answers 404 Not Found: it names a stream that does not exist.

When the session already has a video track, the metadata track follows its transport family:

Video transport Metadata transport Where it arrives
interleaved over TCP/TLS interleaved its own channel pair, default 2-3
UDP unicast UDP unicast its own server/client port pair
UDP multicast UDP multicast the same group, on port + 2

Multicast is the one case where the transport is not per client: a group is one datagram shared by every member, so the metadata stream belongs to the group and goes out once, with its own SSRC, on the port pair above the video one. Each path holds a group address of its own, so nothing else is listening there. The multicast SDP names that real port, exactly as it names the real group.

Units larger than one packet are fragmented (RFC 6597 section 4.2), with the marker bit on the last packet of a unit. Arbitrary metadata is not bounded by anything MServer controls, and sending an oversized unit as a single RTP packet built a datagram past the path MTU – IP fragmentation on a good link, silent loss on a real one.

Over SRTP the metadata track has its own cryptographic context. RFC 3711 section 3.2.1 scopes a context to one SSRC: the IV is built from the SSRC and the rollover counter tracks that SSRC’s sequence space. Sharing the video context encrypted metadata under the wrong SSRC, and the failure was silent – the auth tag still verified, because HMAC covers the ciphertext and never touches the SSRC, so the packets looked authentic and decrypted to noise. Both contexts derive from the same master key, because the SRTP KDF does not take the SSRC.

Which presentation DESCRIBE returns

The SDP describes what the client asked for, decided per request:

Request URL c= line For
rtsp://host/live c=IN IP4 0.0.0.0, m=video 0 any client; SETUP then negotiates unicast, interleaved or multicast
rtsp://host/live?multicast c=IN IP4 <group>/<ttl>, real group port ONVIF multicast streaming
rtsps://host/live?srtp m=video 0 RTP/SAVP, plus the RFC 4568 a=crypto key SRTP – see below

A multicast presentation names the metadata port too, because that one is real as well: m=application <group port + 2>.

Advertising the group unconditionally would make the presentation multicast-only, which GStreamer’s rtspsrc correctly rejects for a unicast request (“no protocols left”); never advertising it breaks ONVIF, which requires the real group address in a multicast DESCRIBE. Making it a property of the request satisfies both, and mirrors how a host’s ONVIF media service hands out a separate stream URI for multicast. ;multicast and ?mcast are accepted as well, because clients differ in which form they preserve.

The secure presentation, and why SRTP needs one

SRTP keys live in the SDP. a=crypto (RFC 4568) and a=key-mgmt (RFC 4567) are SDP attributes; neither is a valid RTSP header field, so the description is the only place a client will find them. MServer therefore mints the master key and salt while building the SDP, remembers them on that connection, and keys the session that follows with exactly the bytes the client was told:

DESCRIBE rtsps://host/live?srtp   ->  m=video 0 RTP/SAVP 96
                                      a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:<key+salt>
SETUP    ... Transport: RTP/SAVP  ->  Transport: RTP/SAVP;unicast;...

It is a per-request presentation for the same reason multicast is: an SDP whose media line says RTP/SAVP is unplayable by a client that cannot do SRTP, so a plain DESCRIBE must keep describing RTP/AVP and carry no keys. This is the analogue of the separate stream URI an ONVIF media service hands out for SecureRTSPStreaming – MServer has no SOAP layer, so the host’s service publishes this URL form instead. ;srtp and ?savp are accepted too.

Three consequences worth knowing:

  • SETUP with RTP/SAVP is refused 461 unless that connection was described SRTP. Without the secure DESCRIBE the client holds no key, and a session that encrypts media the client cannot read is worse than a refusal: it looks established for as long as it runs.
  • The secure presentation requires TLS. ONVIF forbids returning RTP/SAVP or MIKEY on an unprotected control connection – keys readable by anyone on the path are worse than no encryption, because they look like some. Over plain rtsp:// the request is answered 461.
  • MIKEY accompanies the SDES attribute only when the mode advertises it, which no mode does by default: the message is structurally complete but has never been checked against a reference peer.

Multicast addressing

Multicast is the one transport where the server, not the client, decides the address – so the rules are worth stating in full.

Configuring the group

rtspMulticastIp accepts two forms, and they behave differently on purpose: some clients take whatever group the server names, others are provisioned with one fixed address and accept nothing else.

The configuration is per stream, not per listener: the pool, the base port and the TTL belong to the stream that set them, so one stream’s reconfiguration can neither move another’s group nor change the scope its datagrams leave with. (It used to be four listener-wide values, which meant the last stream to initialise chose them for every stream in the process, and reconfiguring one re-derived the groups of the others – including groups already published in an SDP.)

rtspMulticastIp Behaviour
239.1.0.0/16 (a CIDR range) a pool: each stream path is allocated its own group from the range, all on that stream’s rtspMulticastPort, so several streams never collide on one address
239.1.1.1 (a bare address) a fixed group: every stream uses that exact address, and successive streams take successive port ranges on it instead

Details that decide whether a configuration does what you meant:

  • The base address must be multicast (224.0.0.0/4). A unicast address is refused – setParam() returns FALSE and the previous value stands – because a server that sends RTP to whatever address it was configured with is an amplification reflector built from a typo.
  • The prefix must be between /4 and /31; /32 is a single address and therefore a fixed group. Anything else – /1, /33, 239.1.0.0/abc – is refused rather than silently degraded. It used to become a fixed group, which collapsed every stream of the process onto one address.
  • rtspMulticastPort is forced even: RTCP for the group uses port+1 (RFC 3550 sec. 11), and ONVIF restates this for multicast configuration.
  • A group reserves a range of ports, not one: port and port+1 for video RTP and RTCP, plus port+2 and port+3 when the path advertises a metadata track. Reservation is an overlap test over that range, so two streams whose base ports differ by less than their span still collide and the second is offered a different address (or, in fixed-group mode, the next free range). A plain equality test used to put stream N’s KLV on stream N+1’s video port.
  • The TTL comes from multicastTtl and is clamped to 32 for the served legs, not reset – asking for 64 gives 32. The push leg programs the value as given. Each group has its own socket, because IP_MULTICAST_TTL, IP_MULTICAST_LOOP and IP_MULTICAST_IF are socket-wide options: with one shared socket every group left with the TTL of whichever stream configured it last.
  • A transmitting group follows a live codec change. Its payload type is chosen where the group is activated, and that runs only once – a group stays active with no session behind it, deliberately, so it never reaches that code again. A later CODEC change therefore used to move the SDP and leave the group behind: the description advertised RTP/AVP 26 for JPEG while the group kept stamping 96, and a receiver that binds to what the SDP told it discards every datagram. Nothing looks wrong from the server’s side. Re-publishing the description now re-stamps an active group’s payload type and closes its key-frame gate, so the first thing it sends under the new codec is decodable on its own. Only the payload type moves – the SSRC and the sequence space continue, because to the receiver this is one uninterrupted RTP source and a restart reads as mass packet loss. Sessions that are already playing are re-stamped the same way, and for the same reason: a subscriber’s payload type was written once at its SETUP and never moved, so a unicast client that was playing H.264 when the encoder was switched went on being told 96 while JPEG or H.265 arrived. The group was fixed first only because it has no client to notice; a unicast client is no better able to guess. A metadata-only session is skipped – its sequencer is payload type 98 whatever the video codec is.
  • rtspMulticastIp, rtspMulticastPort and multicastTtl apply immediately through setParam(). A path already transmitting keeps its address and only its TTL follows the change; an assignment nobody has set up yet is dropped so the next DESCRIBE re-derives it. A live group is never moved, because a group has no back channel with which to tell its receivers.

How an address is chosen, and when

With a pool, the allocation is by order of first use, not by path name: the first path to be described or set up for multicast takes the base address, and the next advances to base+1 only if its port range would overlap an address already handed out. Two streams that share a pool but sit on different rtspMulticastPort values therefore both land on the base address – what a stream reserves is an address and a port range, not an address on its own. With a fixed group, the address never changes and the base port advances by the range the path needs: two ports, or four when it advertises a metadata track. The allocation happens on the first multicast DESCRIBE or SETUP for that path and is then stable for the life of the stream: the address named in the SDP is the address SETUP confirms and the address the media arrives on. A client that reconnects lands on the same group. It is released when the stream goes away (closeVStreamer(), or a suffix change, which makes a new stream on a new path), and the address returns to the pool. When the pool has no free address left – or, for a fixed group, no free range – SETUP is refused with 461 Unsupported Transport. Handing out an address that is already carrying another stream would put two RTP streams in one sequence space and break both.

A client asking for a particular group

A client may name the group it wants in SETUP with Transport: ...;destination=. It is honoured only when all three hold:

  • it falls inside the configured pool (or equals the configured fixed group) – another network’s group, or a unicast address, is refused, and refusing is an EU CRA Annex I 2(i) requirement rather than caution;
  • no other stream already holds it, since two streams on one address and port share a sequence space and break each other;
  • the stream is not already transmitting, unless the request names the address it is already on. A live group cannot be moved: its members have joined an address and there is no back channel with which to tell them otherwise.

Anything else is refused with 461 Unsupported Transport and no session is created. A recorder provisioned with one fixed address therefore asks for it on its first SETUP, which is what such clients do.

What is on the wire

  • One stream per group. Each path owns its own group, its own SSRC, its own sequence space and its own payload type. Two streams multicasting at once are two independent RTP streams; sharing a counter between them would make each of them look like the other’s packet loss.
  • A group nobody asked for is silent. A path is transmitted only after a client has set it up for multicast – configuring a pool does not put every stream on the wire.
  • Transmission is a property of the stream, not of a session. Once a group is live it keeps carrying media while the stream runs, regardless of how many members are currently joined; there is no per-member state to start or stop it, and TEARDOWN by one client cannot silence a group others may be watching.
  • No per-member queueing or dropping. One datagram serves every member, so a slow receiver cannot be served a reduced stream – it simply loses packets, which is what multicast is.
  • The group waits for a key frame before its first packet, so a receiver that joins at the start can decode; one joining later waits for the next key frame like any RTP receiver.
  • RTCP Sender Reports go to port+1 of the group once a second. Members have no back channel, so the SR is their only source of the RTP-to-wall-clock mapping (RFC 3550 sec. 6.4.1).

Which SDP a client is given

DESCRIBE returns a multicast presentation only when the request asks for one (?multicast); see Which presentation DESCRIBE returns. A multicast DESCRIBE allocates the group, so the c= line names the real address with its TTL suffix and m=video carries the real even port.

Scope and limits

  • Multicast is configured on both listeners, so an RTSPS-only deployment has it too. The two listeners keep independent allocations, so a path can sit on different addresses depending on which listener a client came through.
  • The multicast configuration is per stream even though the listener is shared, so two streams may sit on different pools with different scopes. The registry keeps its own copy of every stream’s policy in addition to forwarding it, and re-applies the copies when it creates the listener – so a stream configured while no listener existed yet, or one that outlives a port re-bind, is not left without a policy.
  • A suffix change moves the stream to a new path: the old path’s group is released – its socket closes and the address returns to the pool, and members already joined cannot be told it moved – while the policy is re-registered under the new name, so the next multicast DESCRIBE derives a fresh group for it from the same pool with the same port and TTL.
  • A multicast session counts against the 64-session limit and is subject to the same 60-second inactivity timeout as any other, though its expiry does not stop the group.
  • No SRTP over multicast. SRTP keys are per session; a group is one datagram shared by every member, so there is no key that could protect it. A SETUP for RTP/SAVP;multicast is refused with 461 Unsupported Transport rather than handing out keys and then sending the group in the clear.
  • When bindAddress is set, multicast egress is pinned to that interface (IP_MULTICAST_IF), so a group cannot leave through an interface the operator excluded. With bindAddress left at 0.0.0.0 the choice falls to the kernel’s route for 239/8, which on a multi-homed host is not necessarily the interface the receivers are on: the group leaves through one interface and a client on any other sees nothing, with SETUP and PLAY both answering 200. A host that serves multicast should name its address, not rely on the default route.
  • IPv4 only.

The direct push leg

RTSP, RTSPS and WebRTC all serve clients that connect. The push leg does the opposite: it sends to an address that was configured, with no session, no negotiation and no back channel. It is a stream in its own right, parallel to the served ones, and everything about it is its own.

Parameter Effect When
directStreamEnable transmits or does not immediately, both ways
directStreamIp one address, or a comma-separated list to push the same stream to several receivers immediately
directStreamPort destination port; RTCP for the leg uses port+1 immediately
directStreamType container and KLV signalling – see Which leg carries which container immediately
directStreamPacingMode 0 spends a token bucket at directStreamBitrateKbps; 1 sends on the frame boundary and lets the kernel’s send buffer provide the back-pressure immediately
directStreamBitrateKbps the rate mode 0 paces toward immediately
directStreamMaxPayloadSize fragmentation size for this leg, [576:9000] next initVStreamer()

Three things about it are worth knowing before configuring one:

  • Its container is independent of the served leg. A STANAG 4609 receiver can be fed raw MPEG-TS while RTSP clients get codec RTP: when the two legs disagree the push leg packetises and muxes for itself, with its own payload type, its own sequence space and its own SSRC. It used to be refused unless it matched, which made the two inseparable.
  • A container change is applied at once, with a clean start. The PSI tables are restarted and the leg waits for the next key frame, because a TS demuxer cannot be switched mid-stream. The served leg cannot do this – see Re-initialising instead of refusing.
  • It is built even when it is off. The socket, the sequencers and the sender thread exist whenever the leg has a usable address and port, so directStreamEnable is a genuine switch rather than something that needs a restart. A leg that was never given an address can still be enabled at runtime: the socket is created on demand.
  • It follows a live codec change, packetiser included. When the leg packetises for itself, a CODEC change re-configures its packetiser as well as the served leg’s, re-derives its payload type from it and holds the leg until the next key frame. This is the only signalling the receiver gets: there is no SDP on this leg, so the RTP payload type is the whole contract. Getting it wrong is silent and total – the served leg would announce RTP/AVP 26 for JPEG while this leg kept emitting H.264 FU-A fragments under payload type 96, and a receiver has no way back from that. ONVIF’s START AND STOP MULTICAST STREAMING – JPEG runs exactly that sequence: SetVideoEncoderConfiguration switches the encoder, then the group is joined with no RTSP in between. Only this leg’s container decides whether the re-stamp applies: a served leg in MPEG-TS with an RTP push leg is precisely the case where the push leg always packetises for itself, so it is the last case that may be skipped.

And two limits: a push destination counts as a consumer, so a leg pointed at an address nobody reads keeps the stream awake and encoding (What counts as a consumer); and there is no retransmission, so a datagram the kernel refuses is loss the receiver cannot repair – which is what directSendFailures counts.

Binding and port ranges

Two things a firewalled or multi-homed deployment needs, and which the library could not be told before VStreamer 3.2.

bindAddress confines every listener MServer opens – RTSP, RTSPS, the WHEP signalling port and the WebRTC media socket – to one local address. Every socket used to bind INADDR_ANY, so a host with a management interface and a video interface published the stream on both. A value that cannot be honoured – a hostname, an unparsable literal, an address that is not local – fails initialisation; falling back to every interface is how a stream ends up exactly where the operator excluded it.

It also moves the ICE candidate: with a bind address set and no explicit publicAddress, the candidate names the bound address rather than whatever the routing table would have offered. Otherwise the browser is handed a candidate for an interface the media socket is not listening on, and ICE never completes.

rtpPortMin / rtpPortMax put the server’s UDP-unicast RTP/RTCP pairs in a known range, so a firewall can be written for it. The RTP port stays even and RTCP takes port+1 (RFC 3550 sec. 11), and allocation starts from a rotating cursor so consecutive clients do not contend for the same numbers. Exhaustion fails the SETUP – falling back to an ephemeral port would hand a client a port the firewall does not admit, which looks to the operator like a stream that sometimes works.

The range covers only ports MServer allocates: multicast and the WebRTC media port are configured explicitly and are not drawn from it.

Both are properties of a shared listener, so an instance that names one constrains the instances sharing that port. A conflicting value is refused rather than merged: widening a bind address would expose a stream that asked to be confined, narrowing it would withdraw one that is already published, and a port range that covers only some of a port’s traffic is not a rule a firewall can be written against.

WebRTC

WebRTC puts the stream in a browser with no plugin and no player: a viewer opens a page, the page does one HTTP request, and video appears. Everything below the handshake is the machinery MServer already has – the same packetizers, the same packetize-once fan-out, the same SRTP – so a WebRTC viewer costs one encryption per packet and no extra packetization.

What a viewer connects to

Two ports, both fixed by configuration:

Port Protocol Carries
webRtcPort TCP, HTTP/1.1 or TLS WHEP signalling: the SDP offer in, the answer back
webRtcMediaPort, or webRtcPort + 1 when it is 0 UDP everything else – STUN, DTLS, SRTP and SRTCP on the one port

The media port defaults to signalling + 1 and can be pinned with webRtcMediaPort when a firewall was opened for a specific port. The two must differ, or initialisation fails. Both ports are shared by every MServer instance in the process, exactly as the RTSP listener is; instances are told apart by the URL path, which is suffix.

The signalling endpoints are:

Request Meaning
POST /<suffix>/whep offer in the body, 201 with the answer and a Location header
PATCH /<suffix>/whep/<id> trickle-ICE candidates as an SDP fragment (RFC 9725 sec. 4.6). Acknowledged 204 and discarded – an ice-lite agent never checks a remote candidate, but answering 405 makes the reference WHEP reader abandon a healthy session. An unknown session id gets 404, as DELETE does
DELETE /<suffix>/whep/<id> end that session
OPTIONS /<suffix>/whep CORS pre-flight and method discovery

The handshake, step by step

  1. The browser POSTs an SDP offer. MServer parses it, picks a payload type the offer named and the encoder can actually produce, and answers.
  2. The answer carries this server’s ICE credentials, its DTLS fingerprint and one host candidate naming the media port. The candidate address is detected from the routing table at startup and can be overridden for a NAT.
  3. The browser sends a STUN Binding Request to that candidate. It is answered only if MESSAGE-INTEGRITY verifies against the password in the answer, and the response is never larger than the request – an unauthenticated port that replies with more than it receives is an amplifier.
  4. The browser starts the DTLS handshake. MServer is always the server (a=setup:passive), and the handshake fails unless the browser’s certificate hashes to exactly the fingerprint its offer carried. That fingerprint is the only thing authenticating the media peer.
  5. SRTP keys are exported from the completed handshake with the label EXTRACTOR-dtls_srtp, and media starts at the next key frame – a viewer that joined mid-GOP cannot decode anything before it.
  6. A PLI or FIR from the browser is turned into a key-frame request for that stream, so a viewer that joins or loses a frame gets an IDR in milliseconds instead of waiting a whole GOP. The packet is verified as SRTCP first, and the whole compound packet is walked, so feedback behind a leading report is found whether or not a=rtcp-rsize was negotiated. One request per second per session is honoured; the rest are counted and refused, so even an authenticated peer cannot turn every frame into an IDR. A pass-through deployment has no encoder to ask, so nothing is honoured there.

Configuring it

Parameter Effect
webRtcEnable starts the endpoint at initVStreamer()
webRtcPort signalling port; the media port is this + 1 unless webRtcMediaPort names one
webRtcMediaPort UDP media port; 0 means webRtcPort + 1. Must differ from the signalling port, or initialisation fails. Read when the shared endpoint starts, so it never moves under a live stream
webRtcEncryption TLS policy for the signalling leg: no (default), optional, strict. Any other value fails initialisation, and optional/strict without a usable certificate and key fail too
webRtcCert, webRtcKey HTTPS signalling. Absent, the RTSPS certificate is used if there is one. Absent both, signalling is plaintext only when webRtcEncryption is no and the compliance mode does not mandate TLS; with optional or strict, or in a mode that requires TLS, initialisation fails – asking for encryption without supplying a certificate and key is a configuration error, not an invitation to exchange offers in the clear. Material given for this leg that fails to build a TLS context is always fatal; borrowed RTSPS material that fails is fatal only when TLS was demanded
user, password digest authentication on the signalling endpoint, per published path: a WHEP request names its stream in the URL, so each stream answers for its own credentials and clearing one does not unprotect the others
securityProfile compliance mode – STRICT_FIPS refuses to start the endpoint at all

Requesting WebRTC in a mode that forbids it fails initialisation rather than starting quietly without it: an operator who asked for WebRTC and got a running server would reasonably believe a viewer can connect.

Signalling works over HTTP and HTTPS both, and the media is encrypted either way – DTLS-SRTP is mandatory in WebRTC. But the SDP carries the ICE credentials and the DTLS fingerprint, which is everything an active attacker needs to take the session setup over, so the modes that require TLS refuse to open a plaintext signalling listener.

user/password follow the library-wide convention: "" or "no" means authentication is off, exactly as on the RTSP listener – and, exactly as there, they are keyed by path. The endpoint is a process-wide singleton, so a single pair used to mean that a password set on one camera challenged every camera’s WHEP endpoint, and that clearing it was impossible: the endpoint refused to drop a credential once any publisher had set one, which took a browser’s video away until the process was restarted. Setting a password now also drops that path’s live WHEP sessions, because a viewer authenticates once on the POST and the media then rides DTLS-SRTP with no further HTTP request – without that, revocation would not reach anyone already watching.

CORS. A page reaching WHEP is cross-origin by construction – MServer serves no HTML – and application/sdp is not a safelisted content type, so every real browser sends a pre-flight first. The endpoint therefore always answers one:

Endpoint Access-Control-Allow-Origin
no credentials configured *
credentials configured the request’s own origin, plus Vary: Origin

The wildcard is only ever used on an endpoint that has no credentials, where it grants nothing that reaching the port did not already grant. Access-Control-Allow-Credentials is never sent, so a page can only use a credential it supplies itself and never the ambient one the browser holds for the realm. The pre-flight itself is never challenged – a browser will not put credentials on one, so challenging it makes every cross-origin client fail before it can authenticate.

The endpoint is shared, and it is the streams’ – not the process’s. Like the RTSP listener it is created by the first stream that needs it and released by the last one that stops, so the ports do not stay bound after streaming ends. A second instance does not get its own endpoint, it gets this one, so its policy is merged restrictively: a permission survives only if every attached stream grants it. A requirement the running endpoint cannot satisfy – TLS on a listener already accepting plaintext, authentication where none is configured – makes that instance fail to initialise rather than be served by a laxer endpoint while believing otherwise.

What the answer offers

Attribute Value Why
a=sendonly always MServer is a source; it never receives media
a=setup:passive always the browser is the DTLS client
a=ice-lite always the server has a routable address and never probes back
a=rtcp-mux always one port for everything, which is what browsers require
a=fingerprint SHA-256 binds the DTLS certificate to this signalling exchange
a=fmtp packetization-mode=1, real profile-level-id taken from the actual SPS, so the description matches the bitstream
a=msid mserver mserver-video without it ontrack fires with an empty streams array, so the usual e.streams[0] is undefined and nothing renders (RFC 8830, RFC 8829 sec. 5.2.1)
a=ssrc cname and msid the per-SSRC form as well, for stacks that read RFC 5576 rather than a=msid

H.264 and H.265 are answered. MJPEG and MPEG-TS are refused, because neither has a WebRTC payload format and a browser has no decoder for either.

The codec parameters are checked against the offer, not merely stated. For H.264 the profile and constraint flags are read from the real SPS and compared with every payload type the browser named: an offer that names only a profile the encoder cannot produce is refused with a reason rather than answered with a profile-level-id the browser never asked for. Constraint-flag refinements within one profile_idc are treated as one decoder, so a High-profile stream is still answered on Chrome’s constrained-High payload type. level-asymmetry-allowed is echoed only when the offer set it, and a stream above the offered level is refused. For H.265 the answer’s profile-space/profile-id/tier-flag/level-id come from this stream’s own SPS – echoing the offer’s sprop-* back would have described the browser rather than the stream.

Scope and limits of the WebRTC leg

  • Send-only video. No audio, no data channels, no simulcast.
  • No TURN and no STUN server. A host candidate only, so the media port must be reachable from the viewer. A deployment behind NAT forwards the UDP port and sets the public address.
  • No retransmission (NACK/RTX) and no congestion control. The stream is paced by the encoder, as every other MServer transport is. nack alone is deliberately never offered – only nack pli – because advertising it would promise retransmissions that never come.
  • Enabling WebRTC caps packetization at 1200 bytes for the shared packetizer, so the RTSP and RTSPS legs are capped with it. SRTP appends an authentication tag after packetization, so a packet sized to fill the Ethernet MTU exactly becomes a 1510-byte IP datagram and is fragmented – and every full-size packet belongs to a key frame, so fragment loss costs key frames while small packets arrive fine. One packetization is shared by the served and WebRTC legs (that is what makes the fan-out cheap), so the smallest requirement has to win there. The push leg is not affected: when its container, KLV signalling or payload size differ from the served leg’s it packetises for itself at directStreamMaxPayloadSize. The cost is about 2 % more header overhead on the other legs.
  • A send failure is not silent. sendto() results are checked and counted (WebRtcStats::sendFailures); on failure the rest of the access unit is dropped and the session waits for a fresh key frame rather than delivering a half-written one. There is still no retransmission and no pacing within an access unit – a retry loop would block the encoder thread on one congested peer, which is the property every MServer transport is built to avoid.
  • RTCP is protected, in both directions. Sender Reports go out under SRTCP and incoming feedback is verified before it is acted on, so the NTP<->RTP anchor of RFC 3550 sec. 6.4.1 is available and a forged PLI is no longer possible. See SRTCP.
  • IPv4 only, like the rest of the library.
  • packetization-mode=0 and profile-level-id values the encoder does not produce are refused rather than answered wrongly.

SRTCP

RTCP on a protected session is SRTCP, not RTCP: RFC 3711 sec. 3.4 makes authentication mandatory there – unlike RTP, where it is only recommended – and RFC 5764 sec. 4.1 puts every RTCP packet of a UDP/TLS/RTP/SAVPF session under it. MServer applies it on both protected legs: WebRTC, and any RTSP subscriber that negotiated SAVP.

The 8-byte RTCP header stays in the clear so a receiver can still demultiplex; everything after it is encrypted, and a trailer carrying an E flag and a 31-bit index is authenticated along with the packet. The two profiles order that trailer differently, and getting it the wrong way round produces packets every receiver silently drops:

Profile Layout
AES_CM_128_HMAC_SHA1_80 (RFC 3711 sec. 3.4) header | encrypted | E+index | tag
AEAD_AES_128_GCM (RFC 7714 sec. 9.1) header | encrypted | tag | E+index

The session keys come from the same master key as the RTP ones but with the labels of RFC 3711 sec. 4.3.1 (0x03 encryption, 0x04 authentication, 0x05 salt), so RTP and RTCP never share a keystream. Received packets are checked against a 64-entry replay window (sec. 3.3.2) and the window only advances after the tag verifies, so a forged index cannot lock out the genuine sender.

What this buys, concretely:

  • PLI works with every browser, not only those that negotiated a=rtcp-rsize. Feedback inside a compound packet was previously unreadable.
  • Feedback is authenticated. An off-path attacker who could reach a session’s 4-tuple used to be able to force key frames; now it cannot.
  • Sender Reports carry the NTP<->RTP correspondence of RFC 3550 sec. 6.4.1, which is what a receiver needs to place the stream on a wall clock, and what inter-stream synchronisation would need the moment audio is added.
  • On the RTSP leg, a SAVP subscriber’s report no longer travels in the clear – which had leaked the sender’s packet and octet counts and violated the crypto suite the a=crypto line offered.

Two things it does not change: the SRTP key derivation is still outside the validated module (SP 800-135 sec. 5.3 is not in CMVP #4985), so STRICT_FIPS still refuses SRTP and WebRTC; and Receiver Reports still do not extend the ICE consent timer, because RFC 7675 sec. 5.1 is explicit that received traffic is not evidence of consent – an authenticated peer is authenticated, not trusted.

Codec packetization

Codec Payload type Format
H.264 96 (dynamic) single NAL + FU-A, packetization-mode=1
H.265 97 (dynamic) single NAL + FU, no DONL
MJPEG 26 (static) RFC 2435 with in-line quantization tables, 8-bit only
MPEG-TS 33 (static) RFC 2250, 7x188 bytes per RTP packet
KLV 98 (dynamic) RFC 6597: one unit per RTP timestamp, fragmented across packets with the marker on the last. On a codec-RTP container it is a real second track (m=application, smpte336m/90000 or vnd.onvif.metadata/90000, trackID=1) with its own SSRC and sequence space, delivered to a subscriber that set that track up (multicast: the port pair above the video one). On an MPEG-TS container the bytes travel on a TS PID instead.

Aggregation packets (STAP-A / AP) are deliberately not produced: they are an optimisation, and every receiver must accept the single-NAL and fragmentation forms.

a=fmtp fields are derived from the actual bitstream after removing emulation-prevention bytes (0x000003 -> 0x0000) – parsing the escaped bytes yields wrong profile and level values.

MJPEG is limited by RFC 2435 to 2040x2040 (dimensions are one byte each in 8-pixel units). Larger images are refused rather than truncated into a silently wrong header; use H.264 or H.265 instead. A picture carrying 16-bit quantization tables is refused for the same reason: the RFC 2435 quantization header has a precision bit per table, but receivers overwhelmingly assume 8-bit, so shipping one would decode to garbage.

MPEG-TS and KLV metadata

KLV is encoded by the caller and forwarded byte-exact by default. Two mutually exclusive carriage forms are supported (MISB ST 1402.1):

  Asynchronous (SMPTE RP 217) Synchronous (ISO/IEC 13818-1 sec. 2.12.4)
stream_type 0x06 private_data 0x15 metadata
stream_id 0xBD 0xFC
PTS absent (PTS_DTS_flags='00') mandatory on every PES
Descriptor registration_descriptor “KLVA” metadata_descriptor + metadata_std_descriptor
KLV wrapper raw metadata_AU_cell

Three KLV handling modes are available: PassThrough (default, byte-exact), Validate (checks the UL key, BER length and ST 0601 sec. 6 checksum) and Restamp (rewrites Tag 2 and recomputes Tag 1 – only for sources with no clock of their own, since it destroys the ST 0601.8 “time of birth” semantic). Those modes are selected by the codec-RTP stream types (rtp-klv-validate, rtp-klv-restamp); every mpegts-* value forwards the caller’s bytes unchanged. The mode is applied to the buffer before it is carried, whichever container carries it, and a unit that fails validation is counted in klvRejected and not sent. Only serverStreamType selects the mode – directStreamType chooses the push leg’s container, not its KLV handling.

Which leg carries which container

The two legs choose independently, and only one of them is restricted:

  Accepted streamType values Where the KLV goes
served leg (serverStreamType, i.e. RTSP, RTSPS, RTSP-over-HTTP) rtp, rtp-klv, mpegts-rtp-klv* rtp-klv: an RFC 6597 metadata track, payload type 98, on the same session. mpegts-rtp-klv*: inside the transport stream. rtp: video only
push leg (directStreamType) all of the above plus the raw forms mpegts-klv-sync / mpegts-klv-async same, and the raw forms put the KLV on a TS PID in datagrams with no RTP header at all

A transport stream reaches the served leg only inside RTP (MISB ST 1403 / RFC 2250, payload type 33): an RTSP client negotiates an RTP transport, so the raw mpegts-klv-* forms have nowhere to go there and are refused. They are exactly what the push leg exists for (MISB ST 1402).

Whether the served leg carries a metadata track follows serverStreamType live: adding klv to a running stream starts the track and updates the SDP the next DESCRIBE returns; removing it stops advertising a track that has stopped. The metadataEnable field is not read by MServer at all – carriage is decided by the stream type, and whether a KLV buffer is offered in the first place is the caller’s decision.

MServer also emits the MISP precision-timestamp SEI (ST 0604.6) carrying the same microsecond value the caller placed in ST 0601 Tag 2.

Security and compliance modes

Some obligations of FIPS 140-3, ONVIF and the EU CRA are mutually exclusive on the wire, so a single configuration cannot satisfy all of them. MServer makes the choice a runtime switch, which is what allows one device to be certified against several standards: each evaluation runs in the mode it requires, and the mode is a documented configuration item rather than a separate firmware image.

Selecting a mode

Set VStreamerParams::securityProfile:

Value Mode
ONVIF maximum interoperability
STRICT_FIPS everything inside the validated module
FIPS_ONVIF_BRIDGE approved primitives, SRTP for ONVIF clients
CRA_STRICT secure by default

"" and "no" select ONVIF. Anything else is refused – initVStreamer() and setParam() both fail rather than pick a default. That is deliberate: this field decides whether the validated provider is loaded, whether TLS and authentication are mandatory, whether MD5 digest is offered and whether WebRTC starts at all, so a typo silently relaxing all four is the failure it exists to prevent.

Before v3.2 of the interface the mode was carried in custom2, a field the interface documents as the application’s. That hijack is gone: custom2 is now opaque caller data like custom3, and setting it neither changes the compliance mode nor is lost on the getParams() round trip. Selecting the mode by number silently mapped any unrecognised value to ONVIF, so a caller using custom2 for its own bookkeeping could downgrade a strict deployment without noticing – exactly why the mode now lives only in securityProfile, which rejects a value it does not know. Pre-3.2 callers must migrate.

The mode is also what selects the OpenSSL provider. There is no build-time crypto switch: initVStreamer() asks OpenSSL for the validated module when the mode requires one. If it cannot be activated the library falls back to the default provider and still starts – what is withdrawn is the claim, not the behaviour: getStats().complianceMode gains an ` (unvalidated provider) suffix and fullModuleCoverage stays FALSE`, while the mode’s algorithm restrictions are enforced exactly as before. Initialisation fails only when no provider at all can be loaded or the DRBG is unusable. Activation is proved, not assumed – the library checks that an approved digest resolves and that a non-approved one does not, because “FIPS enabled” and “FIPS enforced” are not the same property.

Provisioning the module is the deployment’s job: MServer loads the system OpenSSL configuration and never a path of its own, so which provider is available is decided by openssl.cnf, not by this library.

Because the RTSP listener is shared by every instance in the process, the provider is a process-wide property: the first instance to initialise establishes it, and a later instance that asks for the validated module when it is not active is not refused – it runs the hardened configuration on the default provider and reports STRICT_FIPS (unvalidated provider) instead of a validated claim. Refusing meant the first stream a process created decided permanently whether any later one could use the hardened configuration at all. A mode that does not require the module runs perfectly well inside a process where it is active – what it may offer is then tightened automatically (ONVIF drops MD5, because the module does not implement it).

What each mode changes

  ONVIF STRICT_FIPS BRIDGE CRA_STRICT
MD5 digest (RFC 2617) yes no no no
SHA-256 digest (RFC 7616) yes yes yes yes
SRTP AES-CM yes no yes yes
SRTP AES-GCM yes no no only without the FIPS provider
SDES key exchange yes no yes yes
MIKEY key exchange no – unverified, see Limitations no no no
TLS required no yes yes yes
Authentication required no yes yes yes
Plaintext RTSP listener yes no no no
Unprotected media on request yes no no no
Multicast yes no no no
WebRTC yes no yes yes
Plaintext WebRTC signalling yes – no no
Full module coverage no yes no no

STRICT_FIPS and FIPS_ONVIF_BRIDGE prefer the validated module but do not require it. Every restriction they enforce – no MD5, SHA-256 digest only, mandatory TLS and authentication, no SRTP in STRICT_FIPS, no plaintext listener – is MServer’s own policy, and the default OpenSSL provider implements every primitive involved, so the hardened configuration is usable on a stock OpenSSL. What the module decides is not whether the mode works but whether its claim is a validated one.

That distinction is made visible rather than assumed:

  validated module active default provider
initVStreamer() succeeds succeeds
algorithm restrictions enforced enforced
getStats().complianceMode STRICT_FIPS STRICT_FIPS (unvalidated provider)
full module coverage claimed yes (STRICT_FIPS only) no
compliance statement names the validated module says the module is NOT active and that no FIPS 140-3 claim applies

The statement is resolved inside the library and is not exposed through getStats(); the four texts are reproduced verbatim in Compliance summary so they can be copied into certification paperwork without reading the source.

So a deployment can run the FIPS configuration anywhere, and a deployment that needs the certified claim checks complianceMode – a bare STRICT_FIPS is the only string that carries it. Earlier versions refused to initialise instead; that bought no security, it only meant a device without the module could not use the hardened configuration at all, and it made the FIRST stream created in a process decide permanently whether any later one could ask for FIPS.

CRA_STRICT is about configuration rather than algorithm choice, so it has never needed the module.

Why STRICT_FIPS refuses SRTP: both profiles derive session keys with the SRTP KDF, which is not in the approved-algorithm table of CMVP #4985. Any SRTP therefore performs key derivation outside the cryptographic boundary. In strict mode media travels as RTP interleaved inside TLS instead, which keeps the entire media path inside module-provided primitives.

Why STRICT_FIPS refuses WebRTC: for the same reason it refuses SRTP. WebRTC media is SRTP by definition, so the session keys are derived by the SRTP KDF outside the validated module, and there is no interleaved-inside-TLS alternative the way there is for RTSP. The endpoint therefore does not start at all in that mode.

Why CRA_STRICT does not claim full module coverage: it permits SRTP, whose key derivation is the same out-of-module KDF that STRICT_FIPS refuses SRTP over. Two modes with the same media path cannot honestly make different coverage claims, so only STRICT_FIPS – the mode with no SRTP at all – makes the unqualified one.

Why a strict mode refuses the direct RTP push leg: that leg carries media with no TLS, no SRTP and no handshake. A mode that will not open a plaintext RTSP listener cannot consistently emit the same media unprotected to a fixed destination, so initVStreamer() refuses the combination. Note that the interface defaults directStreamEnable to true with a default destination, so a configuration that never mentioned the push leg still requests one: set directStreamEnable = false explicitly when using CRA_STRICT, FIPS_ONVIF_BRIDGE or STRICT_FIPS.

Why ONVIF enables MD5: ONVIF Core sec. 5.9.3 makes MD5 the default digest algorithm for RTSP, and virtually every deployed client implements only MD5. The FIPS provider does not implement MD5 at all – EVP_MD_fetch returns NULL – so without a switch the choice would be between breaking ONVIF and having no FIPS mode.

Cryptographic module boundary

All OpenSSL usage is confined to six translation units: src/impl/MServerCrypto.cpp (providers, digests, DRBG, TLS contexts), src/impl/MServerSrtp.cpp (AES and HMAC for SRTP), src/impl/MServerRtsp.cpp (SSL_read/SSL_write on an accepted connection), src/impl/MServerIce.cpp (HMAC-SHA1 for STUN MESSAGE-INTEGRITY), src/impl/MServerDtls.cpp (the DTLS-SRTP handshake and the key export) and src/impl/MServerHttp.cpp (TLS on the signalling listener). No OpenSSL type appears in any header. A private OSSL_LIB_CTX is used so MServer never competes with the host application for global OpenSSL state, and every random value – SSRC, initial sequence number, initial RTP timestamp, session identifiers, digest nonces, SRTP keys – comes from RAND_bytes_ex() bound to that context.

Exactly one cryptographic primitive is implemented in-tree rather than fetched from OpenSSL: MD5, for RFC 2617 digest interoperability. It is documented as a non-approved legacy interoperability function and is unreachable in every FIPS mode. See docs/CERTIFICATION-CUSTOM-CRYPTO.md.

Compliance summary: FIPS 140-3, EU CRA, ONVIF

A condensed, decision-oriented view of what MServer does and does not give a certification effort. The mechanics are in Security and compliance modes; this section is what to put in front of an auditor, and what not to promise them.

What MServer is, in certification terms

MServer is not a cryptographic module and does not aspire to be one. It is a consumer of one. The validated boundary is the OpenSSL FIPS provider (CMVP #4985, OpenSSL FIPS Provider 3.1.2), and every line of MServer is outside it. Nothing in this repository can invalidate that module, and nothing in this repository is itself validated.

That distinction decides the shape of every claim below. A FIPS statement is about which module performed the cryptography; a CRA statement is about the product placed on the market; an ONVIF statement is about the device’s interfaces. MServer supplies evidence for all three and is the subject of none.

Two further scoping facts:

  • A claim is about a mode, not about the library. securityProfile selects one of four mutually exclusive configurations, and what is on the wire differs between them. Every test report must name the mode it ran in.
  • The library has one external dependency, OpenSSL. FFmpeg is not linked – MServerNoFFmpegDependency asserts it – so the SBOM of the streaming layer is one entry, and the CVE surface is one project.

FIPS 140-3

Inside the boundary. TLS 1.2+ and DTLS 1.2+ contexts, SHA-256, HMAC, AES-CTR and AES-GCM, and every random value the server produces – SSRC, initial sequence number and RTP timestamp, session identifiers, digest nonces, SRTP master keys – are fetched from a private OSSL_LIB_CTX that holds the FIPS provider, via EVP_*_fetch(libctx, ...) and RAND_bytes_ex(libctx, ...). The private context also means MServer never competes with the host application for global OpenSSL state.

Enforcement is proved, not assumed. Loading the provider is not accepted as evidence that it is in charge. After EVP_default_properties_enable_fips(), MServer asks the context two questions it can only answer correctly when the module really is enforcing: SHA2-256 must fetch, and MD5 must not. If MD5 still resolves, the provider is unloaded and the module is treated as absent. “FIPS enabled” and “FIPS enforced” are not the same statement, and only the second one is accepted.

What is outside the boundary, and what MServer does about it.

Operation Why it is outside Consequence
SRTP / SRTCP session-key derivation The SRTP KDF is approved by SP 800-135 Rev.1 sec. 5.3 but is not implemented by CMVP #4985 STRICT_FIPS refuses SRTP entirely and carries media as RTP interleaved inside TLS; FIPS_ONVIF_BRIDGE permits AES-CM and declares limited conformance
AES-GCM SRTP (RFC 7714) The IV is constructed externally from SSRC/ROC/SEQ, which the module’s security policy names as a non-conformance refused in both FIPS modes
WebRTC media SRTP by definition, and with no interleaved-inside-TLS alternative STRICT_FIPS refuses the endpoint outright; requesting it fails initialisation
RFC 2617 MD5 digest MD5 is absent from the module the only cryptographic primitive implemented in-tree; unreachable in STRICT_FIPS, FIPS_ONVIF_BRIDGE and CRA_STRICT, and refused on receipt of the Authorization header, not merely left out of the challenge

Running without the validated module. A FIPS mode still starts. Its restrictions are MServer’s own policy and the default OpenSSL provider implements every primitive involved, so the hardened configuration is usable on stock OpenSSL – but the claim is withdrawn, not the behaviour: getStats().complianceMode returns STRICT_FIPS (unvalidated provider) and fullModuleCoverage stays false. A bare STRICT_FIPS with fullModuleCoverage == true is the only runtime evidence that a validated claim applies. Wire it into a self-test; do not infer it from configuration.

What the laboratory will still need from you. MServer supplies the technical facts; the submission is the integrator’s. Expect to state the module boundary and that MServer is a consumer of it; map every cryptographic operation onto an approved algorithm; list the non-approved functions and prove they are unreachable in the approved mode – for MServer that is exactly one entry, MD5 for RFC 2617 interoperability; and show that entropy comes from the module’s DRBG. Background and the argument for the in-tree MD5 are in docs/CERTIFICATION-CUSTOM-CRYPTO.md.

EU CRA (Regulation 2024/2847)

The CRA applies to a product with digital elements, not to a library, so MServer can only carry part of the obligation. It carries the Annex I Part I engineering half.

Annex I Part I What MServer provides
2(a) secure by default CRA_STRICT: TLS mandatory, authentication mandatory, no plaintext RTSP listener, MD5 refused
2(b) protection from unauthorised access RFC 7616 SHA-256 digest, constant-time comparison, per-nonce replay rejection by nonce-count, per-path credentials on a shared listener
2(e) confidentiality in transit RTSPS / RTSP-over-HTTPS, DTLS-SRTP for WebRTC, SRTP and SRTCP for RTSP; in CRA_STRICT a SETUP that would put media in the clear is refused rather than served
2(f) minimise attack surface one dependency; no external process; compliance modes that remove reachable code paths rather than merely discouraging them
2(h) no unbounded allocation every parameter that bounds an allocation or names a port is range-checked and rejected, never clamped
2(i) protect against DoS 16 KiB request cap, 64 headers, 512-byte URL, 64 sessions, 1 MiB per-subscriber queue with whole-access-unit drop, 4 MiB output queue, idle and sessionless timeouts
2(j) minimise impact on other services no UDP amplification: unicast RTP goes only to the source address of the control connection, and a client-supplied destination= is honoured only for a multicast group inside the configured pool
2(k) recording of activity getStats() counters; the library writes no log of its own by design
Annex VII documentation docs/THREAT_MODEL.md, this README, and the fuzzing corpus in harness/fuzz/ (seven libFuzzer targets)

What is deliberately not here. Annex I Part II – vulnerability handling – is a process obligation on the manufacturer: the coordinated vulnerability disclosure policy of Article 13(8), the SBOM, the security-update channel and the reporting duties of Article 14. None of that lives in this repository, and the product that ships MServer must supply it.

ONVIF

MServer implements the streaming half of ONVIF Profile S / T and nothing else. There is no SOAP layer and no device management: MServer exposes the control points a host’s ONVIF media service drives – GENERATE_KEYFRAME is what SetSynchronizationPoint should call, securityProfile is what a security configuration should switch, and multicast DESCRIBE returns the real group address. Conformance belongs to the device, not to this library.

What the ONVIF mode changes, and why: ONVIF Core sec. 5.9.3 makes MD5 the default RTSP digest and nearly every deployed client implements only MD5, so the mode offers it – but only while the validated module is absent; if another instance in the same process has put the module in charge, MD5 is withdrawn automatically, because a process running on the validated module must not answer challenges with a digest that module does not contain.

Do not claim ONVIF SecureRTSPStreaming. Streaming sec. 5.1.1.4 requires MIKEY key exchange, and MServer does not advertise it: the pre-shared-key message is structurally complete but has never been checked against a reference peer, and an unverified a=key-mgmt makes a client fail the whole SETUP. SRTP itself works and is proved end to end – see the secure presentation – so what is missing is the exchange, not the media protection.

Nuances that decide a certification outcome

  1. A FIPS mode running without the module is a configuration, not a claim. Check the complianceMode string, not the requested profile.
  2. STRICT_FIPS is narrow on purpose. No SRTP, no WebRTC, no UDP unicast and no multicast – every non-interleaved transport is answered 461. Media exists only as RTP interleaved inside TLS. Budget for that before choosing the mode.
  3. Every mode that mandates TLS also refuses unprotected media. Withholding the plaintext listener only protects the control channel; without more, a client authenticated over TLS could still ask for RTP/AVP;unicast or a multicast group and be served unencrypted video. In CRA_STRICT, FIPS_ONVIF_BRIDGE and STRICT_FIPS such a SETUP is answered 461. Protected means one of exactly two things: RTP interleaved inside the TLS control connection, or SRTP. Multicast is therefore unavailable in those modes – a group is one datagram shared by every member and has no per-session key – so a deployment that needs multicast must run ONVIF and protect the segment by other means. The refusal is decided before a group is assigned: a refused SETUP leaves no address reserved and, more importantly, does not mark the group transmitting, which would have put plain RTP on the wire from a mode that documents itself as never emitting any.
  4. SRTP over RTSP needs the secure presentation. The keys are in the SDP, where the standards put them, so a client asks for rtsps://host/live?srtp, reads a=crypto, and then SETUPs RTP/SAVP; the reply names RTP/SAVP and the media decrypts with the advertised key. A SETUP for RTP/SAVP on a connection that was not described SRTP is refused 461 rather than served, because that client has no key. See the secure presentation. ONVIF SecureRTSPStreaming additionally requires MIKEY, which is built but not advertised – that claim still has to be withheld.
  5. The direct RTP push leg is refused by every strict mode, because it carries media with no TLS, no SRTP and no handshake. The VStreamer interface defaults directStreamEnable to true with a default destination, so a configuration that never mentioned the push leg still requests one: set it to false explicitly under CRA_STRICT, FIPS_ONVIF_BRIDGE or STRICT_FIPS, or initialisation fails.
  6. The compliance statement is not exposed at runtime. getStats() reports the mode, the provider state and the coverage flag, but not the sentence; the four texts are reproduced below so they can be quoted directly.
  7. On a shared listener the most restrictive policy wins. Instances that bind the same port share one listener, and the effective policy is the conjunction of every attached path’s policy – one strict stream tightens the digest and SRTP rules for the port. Credentials, by contrast, stay per path.
  8. An unrecognised securityProfile is refused, never defaulted. A typo cannot silently re-certify a device as the most permissive mode.

The four compliance statements, verbatim

Copy these into the security documentation rather than paraphrasing them.

ONVIF – “ONVIF interoperability mode. MD5 digest and AES-GCM SRTP are enabled; neither is FIPS-approved, so no FIPS claim applies to this configuration.”

STRICT_FIPS, module active – “Strict FIPS 140-3 mode. All cryptography is performed by the validated module; media travels as RTP interleaved inside TLS. MD5 digest and SRTP are refused.” Module absent – “Strict FIPS 140-3 CONFIGURATION on the default OpenSSL provider. The algorithm restrictions are enforced – MD5 digest and SRTP are refused, TLS and authentication are mandatory – but the validated module is NOT active, so NO FIPS 140-3 claim applies to this deployment.”

FIPS_ONVIF_BRIDGE, module active – “FIPS/ONVIF bridge. Only FIPS-approved primitives are used, and SRTP is limited to AES_CM_128_HMAC_SHA1_80. LIMITED CONFORMANCE: the SRTP key derivation (SP 800-135 sec. 5.3) is not implemented by the validated module.” Module absent – “FIPS/ONVIF bridge CONFIGURATION on the default OpenSSL provider. Only FIPS-approved primitives are selected and SRTP is limited to AES_CM_128_HMAC_SHA1_80, but the validated module is NOT active, so NO FIPS 140-3 claim applies to this deployment.”

CRA_STRICT – “EU CRA secure-by-default mode. TLS and authentication are mandatory, MD5 digest is disabled, no plaintext listener is opened, and media is refused unless it is protected – SRTP, or RTP interleaved inside the TLS connection. SRTP remains available, so the media key derivation is outside the validated module and coverage is not claimed to be complete.”

Verdict

FIPS 140-3 – yes, as a consumer of CMVP #4985, in STRICT_FIPS. There is no blocker. MServer performs no approved-mode cryptography outside the module, proves the module is enforcing rather than merely loaded, takes all entropy from its DRBG, and carries exactly one non-approved primitive (MD5) that is provably unreachable in the mode. The conditions are: provision the module on the target; run STRICT_FIPS; assert at start-up that complianceMode is the bare string and fullModuleCoverage is true; document MD5 as a non-approved legacy interoperability function; and accept that the mode has no SRTP, no WebRTC and no UDP transport. FIPS_ONVIF_BRIDGE is not a route to an unqualified claim – its SRTP key derivation is outside the module by construction, which is why it states limited conformance itself.

EU CRA – yes for the engineering requirements, provided the product supplies the process half. Annex I Part I is met in CRA_STRICT, with the deployment decisions in nuances 3 and 5 made explicitly rather than by default. Annex I Part II is out of scope of this repository: the manufacturer must add the coordinated vulnerability disclosure policy (Article 13(8)), the SBOM, the security-update mechanism and the Article 14 reporting process. The single external dependency makes that tractable, but it is not done here.

The one claim still to withhold is ONVIF SecureRTSPStreaming, and only because of MIKEY. The SRTP media path itself is exercised end to end by MServerStrictModesTest, which takes the master key from the SDP, derives the session keys with its own RFC 3711 implementation, verifies every HMAC-SHA1-80 tag and recovers the H.264 – so FIPS_ONVIF_BRIDGE’s media path carries evidence, not an assertion. What it does not carry is the MIKEY exchange ONVIF Streaming sec. 5.1.1.4 mandates: the message is built but has never met a reference peer, so it is not advertised and the ONVIF secure-streaming claim stays unmade until it is.

MServer class description

MServer class declaration 

class MServer : public VStreamer
{
public:

    /// Get library version.
    static std::string getVersion();

    /// Init video server.
    bool initVStreamer(VStreamerParams& params, VCodec* codec = nullptr,
                       VOverlay* overlay = nullptr) override;

    /// Check initialization status.
    bool isVStreamerInit() override;

    /// Close video server.
    void closeVStreamer() override;

    /// Send frame for streaming.
    bool sendFrame(Frame& frame, uint8_t* userData = nullptr,
                   int userDataSize = 0) override;

    /// Set parameter.
    bool setParam(VStreamerParam id, float value) override;

    /// Set parameter.
    bool setParam(VStreamerParam id, std::string value) override;

    /// Get all parameters.
    void getParams(VStreamerParams& params) override;

    /// Execute action command.
    bool executeCommand(VStreamerCommand id) override;

    /// Get statistics.
    struct Stats { /* ... */ };
    void getStats(Stats& stats) const;
};

getVersion method

The getVersion() method returns a string of the current version of the MServer class. Method declaration:

static std::string getVersion();

The method can be used without an MServer class instance:

std::cout << "MServer class version: " << cr::video::MServer::getVersion();

Console output:

MServer class version: 1.0.0

Returns: the library version as a string, "Major.Minor.Patch".

initVStreamer method

The initVStreamer(…) method initialises the streamer: it validates the whole parameter structure, binds the listeners the configuration asks for, starts the pipeline threads and publishes the stream path. Method declaration:

bool initVStreamer(VStreamerParams& params,
                   VCodec* codec = nullptr,
                   VOverlay* overlay = nullptr) override;
Parameter Value
params VStreamerParams class object. Every field MServer reads is listed in Parameters; fields belonging to protocols MServer does not serve are stored unchanged so a getParams() round trip is faithful. Values are range-checked, not clamped: an out-of-range port, size, rate or enumeration fails the call rather than being silently corrected.
codec Pointer to a VCodec object, used to encode RAW input frames. Required whenever RAW frames will be supplied. A caller that feeds already-compressed H.264, HEVC or JPEG frames may pass nullptr: such frames bypass the encoder entirely. The pointer is kept for the lifetime of the stream, so the object must outlive the MServer instance or be released only after closeVStreamer().
overlay Pointer to a VOverlay object, used to draw over RAW frames when overlayEnable is TRUE. With nullptr nothing is drawn, whatever overlayEnable says. Compressed input is never overlaid – drawing on it would require a decode. Same lifetime rule as codec.

Returns: TRUE if the streamer was initialised, or FALSE if it was not. It fails when a parameter is out of range; when the requested securityProfile name is not recognised; when a strict compliance mode cannot be honoured by the configuration (TLS or credentials missing, WebRTC or the direct push leg requested where the mode forbids them); when encryption is asked for without both a certificate and a key; when the stream suffix is already claimed by another instance on the shared listener; when a listening port cannot be bound; or when the codec and container are incompatible (JPEG with any MPEG-TS container, which STANAG 4609 and MISB ST 1402 define no stream type for).

isVStreamerInit method

The isVStreamerInit() method returns the initialisation status. Method declaration:

bool isVStreamerInit() override;

Returns: TRUE if the streamer is initialised, or FALSE if it is not.

closeVStreamer method

The closeVStreamer() method stops the streamer: it closes every session, stops the pipeline and sender threads, releases the listening ports this instance holds and drops its path from the shared listener. It is safe to call on an instance that was never initialised, and safe to call twice. The destructor calls it. Method declaration:

void closeVStreamer() override;

sendFrame method

The sendFrame(…) method hands one frame to the streamer. To produce a video stream the caller must call it for every frame coming from the video source. Method declaration:

bool sendFrame(Frame& frame, uint8_t* userData = nullptr,
               int userDataSize = 0) override;
Parameter Value
frame Frame class object. Accepted formats are listed below. The frame is copied and the method returns at once, so the caller may reuse or destroy the buffer immediately; nothing here blocks on the network or on the encoder.
userData Pointer to user data (telemetry) belonging to this frame – MISB ST 0601 KLV, or an ONVIF metadata document, according to metadataSuffix. The bytes are encoded by the caller and forwarded byte-exact; MServer never re-encodes them. nullptr sends no telemetry with this frame. See How to extract KLV from the stream.
userDataSize Size of the user data in bytes. 0 sends no telemetry with this frame. A unit larger than one RTP packet is fragmented across packets with the marker bit on the last, so there is no size limit imposed here.

Accepted input formats. Two families, handled quite differently:

Input Handling
RAW: YUV24, NV12, NV21, YU12, YV12, RGB24, BGR24, YUYV, UYVY, GRAY Converted, scaled and overlaid as the parameters require, then encoded. Needs the codec given to initVStreamer().
Compressed: H264, HEVC, JPEG Passed straight through to the packetiser. No encoder, no scaler and no overlay – all three would require a decode first. width, height, fitMode and overlayEnable are therefore ignored for such input, and codec may be nullptr.

Which RAW format to send. Short answer: with an encoder in use, send YUV24 unless you know the frame needs neither scaling nor overlay, in which case send NV12. Any of the ten formats above is accepted either way – this is about cost per frame, not correctness. The reason there are two answers is that the pipeline has two routes:

Situation Send this Why
Scaling or overlay is in use (width/height differ from the source, or overlayEnable is on with an overlay object) YUV24 The scaler and the overlay accept only 3-byte interleaved data, so the pipeline works in YUV24: the route is RAW → YUV24 → resize → overlay → NV12. Supplying YUV24 skips the first conversion.
Neither is in use, and the frame is already at the stream resolution NV12 The encoder takes NV12, and FormatConverter reaches it from any RAW format in one pass. Supplying NV12 at the stream size means no conversion and no copy at all – the frame goes to the encoder untouched.

Neither is a requirement: any listed format is accepted in either situation. The recommendation is about cost, not correctness – sending RGB24 into a scaling pipeline simply pays for one extra conversion per frame.

Returns: TRUE if the frame was accepted, or FALSE if it was not. FALSE means the streamer is not initialised, or frame.data is nullptr, or frame.size is not positive. TRUE does not mean the frame was encoded or transmitted: a frame is legitimately discarded when the stream is disabled (enable is FALSE), when nobody is watching (see Idle streams cost nothing), when a newer frame supersedes it in the input slot, or when the rate governor drops it because the source runs faster than fps. getStats() reports each of those separately.

setParam method

The setParam(…) method changes one parameter on a running or a not-yet-initialised streamer. Two overloads exist because the interface carries both numeric and string parameters. Method declaration:

bool setParam(VStreamerParam id, float value) override;
bool setParam(VStreamerParam id, std::string value) override;
Parameter Value
id Parameter identifier from the VStreamerParam enumeration. Use the numeric overload for numeric parameters and the string overload for string ones; the wrong overload for a given id is rejected.
value New value. Which parameters take effect at once, which are stored until the next initialisation, and which rebuild the stream is set out in When a parameter takes effect, and how far it reaches.

Returns: TRUE if the parameter was accepted, or FALSE if it was not. Everything that bounds an allocation or names a port is range-checked, because silently accepting one is an unbounded allocation primitive (EU CRA Annex I 2(h)): sizes, frame rate, GOP, bitrates, quality, ports and the mode enumerations. The multicast port must also be even, because RTCP uses port+1. A value is rejected, never clamped – a caller that asked for something impossible is told so rather than being given something else.

FALSE therefore means one of: the value is out of range; the id is not recognised, or was given through the wrong overload; the parameter is output-only (CYCLE_TIME_USEC) or names a listener that does not exist (METADATA_PORT); or the change cannot be honoured by the running configuration – switching to a compliance mode that demands TLS on a stream that has no TLS listener, for instance. Parameters belonging to protocols MServer does not serve are accepted and stored, so a setParam() / getParams() round trip returns exactly what the caller set even though nothing is started for them.

getParams method

The getParams(…) method returns the streamer’s current parameters. Method declaration:

void getParams(VStreamerParams& params) override;
Parameter Value
params VStreamerParams object to fill. Every field is overwritten.

Three fields deserve attention, because what they report is not simply what was set:

  • custom1..3 come back unmodified. A getParams() → initVStreamer() round trip must not corrupt them, so MServer writes nothing of its own into them. custom1 carries the preprocessing thread count, but it is stored exactly as given – not normalised, not clamped – so the round trip is faithful there too.
  • cycleTimeUs is measured, not set: it is the observed interval between encoded frames, which is what the interface documents it to be. setParam() refuses to write it.
  • rtspPort reports the port that is actually bound, which is not always the one requested. Instances sharing a process share one plaintext listener, and the first port bound wins; an instance that asked for a different one is told the truth here rather than the wish. See Reach: the stream, or the whole process.

getStats method

The getStats(…) method returns the runtime counters. It is an MServer extension: the VStreamer interface has no read direction of its own, and putting statistics into the custom fields would corrupt the parameter round trip described above. Method declaration:

void getStats(Stats& stats) const;
Parameter Value
stats MServer::Stats object to fill. Every field is overwritten. All counters are per instance and are reset by initVStreamer(); the fields describing compliance are read from live state.

The structure:

| Field | Meaning | |——-|———| | framesIn / framesDropped / framesEncoded | pipeline counters. framesDropped includes the frames superseded in the input slot while the source runs faster than fps | | framesDuplicated | pictures repeated to hold fps while the source ran slower than it. A stream whose source matches its fps never duplicates; a rising count means the source is short of the rate the SDP advertises. Reset by initVStreamer(), like the other pipeline counters | | framesIgnored | frames skipped because nobody was watching – see Idle streams cost nothing | | idle | TRUE while input is being ignored for want of a consumer | | accessUnits / rtpPackets | published units and packets produced | | klvUnits / klvRejected | KLV accepted and rejected | | clients | consumers of this stream: its RTSP and RTSPS sessions in PLAYING state, its WebRTC sessions, and its multicast group if one is transmitting (a group counts as one consumer with no session behind it). Per path, not per process – a client on another stream, or the page watching a different camera, is not counted here | | directPackets | packets on the direct push leg | | rtpSendFailures / directSendFailures | datagrams the kernel refused, on this stream’s RTSP legs and on its push leg. Each one is a hole in a sequence space the receiver reports as loss and neither leg can retransmit, so a non-zero value is the server’s own contribution to a client’s “missed packets” – not the network’s. The RTSP figure includes this path’s departed subscribers, so it never falls | | complianceMode | active mode name, with ` (unvalidated provider) appended when a FIPS mode (STRICT_FIPS, FIPS_ONVIF_BRIDGE) is running without the validated module. A bare STRICT_FIPS together with fullModuleCoverage TRUE is the only pair that carries a validated claim, so never compare the whole field for equality without accounting for the suffix | | fipsActive | FIPS provider in use | | fullModuleCoverage | FALSE` means any FIPS claim must be qualified |

executeCommand method

The executeCommand(…) method executes one command on the streamer. Method declaration:

bool executeCommand(VStreamerCommand id) override;
Parameter Value
id Command identifier from the VStreamerCommand enumeration.
Command Effect
ON Resume streaming. Equivalent to setting enable to TRUE: frames handed to sendFrame() are processed again.
OFF Stop streaming without closing anything. Listeners stay bound and sessions stay open; frames are accepted and discarded.
RESTART Rebuild the stream with the parameters currently held: the same sequence initVStreamer() performs. Clients are dropped and reconnect, and everything a re-initialisation resets – the counters, the sequence spaces, the multicast assignments – is reset.
GENERATE_KEYFRAME Ask the encoder for a key frame with fresh parameter sets on the next frame. This is what a host’s ONVIF SetSynchronizationPoint should call. Measured latency from the command to a complete IDR on the wire: 19 ms at 25 fps with a GOP of 3000, i.e. where no key frame was due.

Returns: TRUE if the command was accepted, or FALSE if it was not. GENERATE_KEYFRAME returns FALSE when there is no encoder to ask – either none was given to initVStreamer(), or the pipeline is in pass-through because the caller is supplying already-compressed frames, in which case the key frames are whatever the source produces.

Parameters

MServer implements the VStreamer interface, so its whole configuration surface is the interface’s own: the VStreamerParams structure passed to initVStreamer(), the VStreamerParam enumeration used by setParam(), and the VStreamerCommand enumeration used by executeCommand().

The interface covers a family of streamers, so it declares parameters for protocols MServer does not serve. Those are listed below and marked not used: setParam() accepts and stores them so a setParam() / getParams() round trip returns exactly what the caller set, but nothing is started for them. One parameter is rejected outright rather than stored, because accepting it would imply a listener that does not exist.

VStreamerCommand

executeCommand() takes one of four values. All four are implemented.

Command Value Description
RESTART 1 Stop and start the streamer with the current parameters. Client sessions are dropped and the shared listener is re-attached.
ON 2 Enable streaming. Equivalent to MODE = 1: frames are encoded and published again.
OFF 3 Disable streaming. Equivalent to MODE = 0: sendFrame() still accepts frames and discards them, sessions stay open, no media is published.
GENERATE_KEYFRAME 4 Force the encoder to emit a key frame with fresh parameter sets on the next frame. This is what a host’s ONVIF SetSynchronizationPoint should call. Returns FALSE when no encoder is attached, and also when the pipeline is in pass-through (the caller is supplying already-encoded frames, so there is no encoder to ask and the key frames are whatever the source produced).

VStreamerParam

setParam() has a float overload and a string overload; each parameter accepts one or the other. The Range column is what MServer actually enforces – a value outside it makes setParam() return FALSE and changes nothing.

Parameter Type Range enforced by MServer Description
MODE int 0 or 1 Streamer enable / disable. 0 keeps the pipeline and the sessions alive but publishes nothing.
WIDTH int [8:4096] Stream width. Applied live: the encoder is reconfigured and clients keep their session.
HEIGHT int [8:4096] Stream height. Applied live, as WIDTH.
DIRECT_STREAM_IP string any Destination of the direct RTP push, applied live: the destination list is rebuilt and the running leg follows without a restart. A comma-separated list sends the same packets to several receivers; entries that are not IPv4 literals are dropped.
RTSP_PORT int [500:65534] RTSP port. Shared by every MServer instance in the process, so a change re-binds the listener for all of them.
RTSPS_PORT int [1024:65535] RTSPS (RTSP over TLS) port. Takes effect at the next initVStreamer().
DIRECT_STREAM_PORT int [1024:65535] Destination port of the direct RTP push, applied live (the destination list is rebuilt inside setParam()). Its RTCP Sender Reports go to port+1 only while the leg carries RTP; a raw mpegts-klv-* leg has no RTP session and sends none.
WEBRTC_PORT int [1024:65534] WebRTC signalling port; the media port is this + 1 unless WEBRTC_MEDIA_PORT names one, which is why 65535 is out of range. Read at the next initVStreamer(). The endpoint is shared by the whole process: while another instance still holds it, an initialisation asking for a different port fails rather than moving the listener out from under the streams already using it.
HLS_PORT int any Not used. Stored only.
SRT_PORT int any Not used. Stored only.
RTMP_PORT int any Not used. Stored only.
RTMPS_PORT int any Not used. Stored only.
METADATA_PORT int – Rejected. setParam() returns FALSE. There is no separate metadata listener: KLV travels inside the media stream, so a port for it would be a promise MServer cannot keep.
RTSP_MODE int 0 or 1 RTSP protocol enable / disable. Takes effect at the next initVStreamer().
DIRECT_STREAM_ENABLE int 0 or 1 Direct RTP push enable / disable, applied live: 0 clears the flag so the leg stops with the next packet, 1 starts it at once – building the socket, the sequencers and the sender thread if initialisation did not. initVStreamer() builds the leg whenever DIRECT_STREAM_IP and DIRECT_STREAM_PORT name a destination even when the leg is disabled, which is what makes 0<->1 need no restart. It returns FALSE only when the leg would have to be built from nothing and cannot be: no destination configured, or a compliance mode that forbids a plaintext leg.
WEBRTC_MODE int 0 or 1 WebRTC enable / disable. Takes effect at the next initVStreamer().
HLS_MODE int any Not used. Stored only.
SRT_MODE int any Not used. Stored only.
RTMP_MODE int any Not used. Stored only.
METADATA_MODE int any Not used. Stored only; KLV carriage is selected by SERVER_STREAM_TYPE.
RTSP_MULTICAST_IP string empty (off), or a multicast base with an optional /prefix in [4:31] (/32 = one address) Multicast group or pool of this path, applied live. A /prefix makes it a pool, a bare address a fixed group. The value is validated as it is set: a base outside 224.0.0.0/4, or an unusable prefix, makes setParam() return FALSE and the previous value is kept – nothing is stored and silently ignored. The check is deferred only while no multicast port is configured yet. See Multicast addressing.
RTSP_MULTICAST_PORT int [1024:65534], even Multicast RTP port, applied live. Must be even because RTCP uses port+1 (RFC 3550 sec. 11).
USER string any RTSP user, applied live. "" or "no" disables authentication. Refused while a compliance mode requires authentication.
PASSWORD string any RTSP password, applied live.
SUFFIX string non-empty; before initVStreamer() it is only stored Stream name, i.e. the URL path. Applied live: the stream moves to the new path with its RTSP/RTSPS credential record, its 90 kHz RTP clock, its send budget and its multicast policy; RTSP sessions on the old path are dropped, and WebRTC viewers follow the stream to the new name. Returns FALSE and leaves the stream where it was when the new name is already claimed on the shared listener or refused by the WebRTC endpoint. The WHEP credential is keyed by the published path and is not re-keyed by the rename.
METADATA_SUFFIX string SMPTE336M, VND.ONVIF.METADATA Metadata format, not a URL path. SMPTE336M is MISB ST 0601 KLV (RFC 6597 encoding name smpte336m); VND.ONVIF.METADATA is an ONVIF metadata stream. On the codec-RTP paths the value is what the SDP’s a=rtpmap:98 names, which is how a client learns the format; see what each transport carries. "", no and the interface’s legacy default metadata all select SMPTE336M. Any other value is rejected.
MIN_BITRATE_KBPS int [1:1000000] Minimum bitrate for variable-bitrate mode. Forwarded to the encoder.
MAX_BITRATE_KBPS int [1:1000000] Maximum bitrate for variable-bitrate mode. Forwarded to the encoder.
BITRATE_KBPS int [1:1000000] Target bitrate. Forwarded to the encoder.
BITRATE_MODE int 0 or 1 0 – constant bitrate, 1 – variable. Forwarded to the encoder.
FPS float [1:240] Frame rate, applied live. Forwarded to the encoder, published in the SDP (a=framerate, so a later DESCRIBE sees the new value) and used as the rate the governor holds: a frame that arrives early waits in the hand-off slot, and a missed deadline repeats the last picture. It is not the pacing budget – that comes from BITRATE_KBPS / MAX_BITRATE_KBPS. See Holding the configured frame rate.
GOP int [1:65535] GOP size. Forwarded to the encoder; also how long a new client waits for its first key frame.
H264_PROFILE int [0:2] 0 – baseline, 1 – main, 2 – high. Forwarded to the encoder.
JPEG_QUALITY int [1:100] JPEG quality in percent. Forwarded to the encoder.
CODEC string H264, H265/HEVC, JPEG/MJPEG Codec, matched case-insensitively. Applied on the next access-unit boundary: packetizer, payload type, SDP and the encoder’s own output format move together. JPEG cannot be combined with an MPEG-TS container – the muxer has no stream type for it – and that pair is refused on a live change as well as at initialisation, on either leg’s container. Accepting it live was silent rather than loud: the muxer picks its stream type as “H.265 or else H.264”, so JFIF bytes went out under a PMT declaring 0x1B and every conformant demuxer handed them to an H.264 decoder. Move the container off MPEG-TS first.
FIT_MODE int 0 or 1 0 – fit (letterbox, aspect preserved), 1 – fill (stretch). Applied live.
CYCLE_TIME_USEC int – Read-only. Reported by getParams() as the measured pipeline cycle; setParam() does not accept it.
OVERLAY_MODE int 0 or 1 Overlay enable / disable. Applied live.
TYPE int any Streamer type, forwarded to the encoder as its backend selector (0 – hardware, 1 – software for VCodecLibav).
CUSTOM1 float [0 : hardware threads] Compute threads for preprocessing – the limit the format converter and the scaler work under. 1 by default, 0 means the same. The two entry points differ: setParam() range-checks the request and refuses anything above the machine’s thread count, while initVStreamer() does not range-check it and clamps - so a caller that passes a large number here at initialisation silently gets a worker team as wide as the machine. Applied live: the pipeline thread picks it up on its next frame. Stored and returned by getParams() unmodified. Carried the multicast TTL before v3.2 of the interface; use MULTICAST_TTL. See Compute threads for preprocessing.
CUSTOM2 float any Not used by MServer. Opaque caller data, returned by getParams() unmodified. Carried the compliance mode before v3.2; that hijack is gone – use SECURITY_PROFILE.
SECURITY_PROFILE string ONVIF, STRICT_FIPS, FIPS_ONVIF_BRIDGE, CRA_STRICT; ""/no – default Compliance mode. Matched case-insensitively. An unrecognised value is rejected, never defaulted. See Security and compliance modes.
MULTICAST_TTL int [0:255] Multicast TTL of this path’s RTSP/RTSPS groups, applied live: a group already transmitting keeps its address and only its TTL is re-programmed. 0 means the default of 1, and the served legs clamp to 32. The push socket is programmed the value as given, without the clamp, and keeps the TTL it was built with until the leg is rebuilt. A value the listeners refuse is rolled back and setParam() returns FALSE.
PUBLIC_ADDRESS string IPv4 literal, ""/no – auto Address advertised in SDP and ICE candidates instead of the detected one. Validated when it is set: anything that is not an IPv4 literal (or ""/no) makes setParam() return FALSE and changes nothing, so a host name is refused by the command instead of taking the stream down at the next restart. The accepted value is read at the next initVStreamer(), because it is written into candidate lines that sessions already hold; a bad value handed straight to initVStreamer() still fails initialisation.
BIND_ADDRESS string local IPv4 literal, ""/0.0.0.0 – all Interface every listener binds to. Validated when it is set, exactly as strictly as initVStreamer() validates it: a hostname, an unparsable value, or an address this host does not hold makes setParam() return FALSE rather than being stored and quietly falling back to every interface. The accepted value is read at the next initVStreamer() – a bound socket cannot be moved under its sessions.
RTP_PORT_MIN int [0:65534] Lowest server RTP/RTCP port. 0 – ephemeral.
RTP_PORT_MAX int [0:65535] Highest server RTP/RTCP port. 0 – ephemeral. Must be >= RTP_PORT_MIN + 1 when RTP_PORT_MIN is set.
WEBRTC_MEDIA_PORT int [0:65535] WebRTC media port. 0 keeps the convention of WEBRTC_PORT + 1.
CORS_ALLOWED_ORIGIN string one origin, ""/no – default policy Origin permitted on the signalling endpoint, applied live.
SERVER_STREAM_MAX_PAYLOAD int [576:9000] Maximum RTP payload of the server-delivered stream. Read at the next initVStreamer() and re-read on a live codec change. Clamped to 1200 bytes whenever the WebRTC leg is enabled with a port set, because one packetisation serves the served and WebRTC legs – see Scope and limits of the WebRTC leg.
CUSTOM3 float any Not used by MServer. Free for the application: stored and returned by getParams() unmodified.
RTSP_KEY string any Path to the private key for RTSPS. Read at initVStreamer().
RTSP_CERT string any Path to the certificate for RTSPS. Read at initVStreamer().
WEBRTC_KEY string any Path to the private key for HTTPS signalling. Read at initVStreamer(); falls back to RTSP_KEY when unset.
WEBRTC_CERT string any Path to the certificate for HTTPS signalling. Read at initVStreamer(); falls back to RTSP_CERT when unset.
HLS_KEY string any Not used. Stored only.
HLS_CERT string any Not used. Stored only.
RTMP_KEY string any Not used. Stored only.
RTMP_CERT string any Not used. Stored only.
RTSP_ENCRYPTION string no, optional, strict TLS policy, selecting which listeners open: no – plaintext only; optional – both, so an rtsp:// and an rtsps:// client are served side by side; strict – RTSPS only, and initialisation fails if the TLS context cannot be built. Any other value is rejected.
WEBRTC_ENCRYPTION string no, optional, strict TLS policy for signalling (WHEP over HTTP/HTTPS). The media leg is always DTLS-SRTP. optional and strict both demand a certificate; strict fails initialisation if TLS cannot be built. "yes" is rejected – it is not one of the three values.
RTMP_ENCRYPTION string any Not used. Stored only.
HLS_ENCRYPTION string any Not used. Stored only.
LOG_LEVEL int any Not used. Stored only: MServer writes no log of its own, it reports through getStats() and the return codes.
DIRECT_STREAM_TYPE string see Stream type values Container and framing of the direct push leg, independent of SERVER_STREAM_TYPE and switchable live: unlike the served leg, a container change here needs no re-initialisation. The framing (raw MPEG-TS in datagrams versus TS inside RTP), the KLV signalling and the leg’s payload type (33 for a transport stream, the dynamic type otherwise) are re-derived on the spot, and the receiver is given a clean start by a wait for the next key frame. The leg packetizes for itself when its container, KLV signalling or payload size differ from the served leg’s, and then its packetizer, TS muxer and PSI tables are rebuilt from scratch. JPEG cannot be combined with an MPEG-TS container, and an unknown value is rejected.
DIRECT_STREAM_BITRATE_KBPS int any Channel bandwidth for the push leg’s packet pacer, in kbit/s. Honoured when DIRECT_STREAM_PACING_MODE is 0: a token bucket refilled at this rate, with a quarter-second burst allowance, so a burst from the encoder leaves the interface at the rate the channel was told it can carry.
DIRECT_STREAM_MAX_PAYLOAD int [576:9000] Maximum RTP/UDP payload of the push leg’s own packetizer; it never shrinks the packets the served leg fans out (initVStreamer() and the live codec-change path size the shared packetizer from SERVER_STREAM_MAX_PAYLOAD alone, capped only by the WebRTC limit). Read at the next initVStreamer(), and re-read at once by a live DIRECT_STREAM_TYPE change or when the leg is enabled at runtime; on its own the field only stores the value.
DIRECT_STREAM_PACING_MODE int 0 or 1 0 – token-bucket pacing toward DIRECT_STREAM_BITRATE_KBPS; 1 – push, sending on the frame boundary and leaving the back-pressure to the kernel’s send buffer.
SERVER_STREAM_TYPE string see Stream type values Container and KLV mode of the served (RTSP / RTSPS / WebRTC) leg. An unknown value is rejected, and so are the raw mpegts-klv-* forms without the mpegts-rtp infix – the served leg carries a transport stream only inside RTP – and any MPEG-TS form while the codec is JPEG. The KLV mode applies live, and the metadata track and the SDP follow it at once. A container change cannot be handed to a client that is already playing, so setParam() re-initialises this instance automatically (the RESTART sequence: clients are dropped and reconnect to an SDP that matches what they are sent) instead of refusing.

Stream type values

serverStreamType and directStreamType select the container and whether KLV is carried:

Value Container and framing KLV
rtp codec-specific RTP (RFC 6184 / 7798 / 2435) accepted from sendFrame(), carried only if the container is MPEG-TS
rtp-klv same as rtp muxed metadata track, PT 98, declared in the SDP (RFC 6597) – pass-through
rtp-klv-validate same as rtp same track, KLV validated before it is sent
rtp-klv-restamp same as rtp same track, KLV timestamp restamped
mpegts / mpegts-klv-async MPEG-TS straight into UDP, MISB ST 1402 asynchronous
mpegts-rtp / mpegts-rtp-klv-async MPEG-TS inside RTP, RFC 2250 PT 33 asynchronous
mpegts-klv-sync MPEG-TS straight into UDP synchronous
mpegts-rtp-klv-sync MPEG-TS inside RTP synchronous
mpegts-rtp-klv MPEG-TS inside RTP synchronous – the STANAG 4609 value the reference implementation uses; accepted as a synonym of mpegts-rtp-klv-sync

The -rtp- infix is the framing switch: the forms without it put the transport stream directly in the datagram (MISB ST 1402), the forms with it wrap the same stream in RTP (RFC 2250, PT 33).

The two legs choose independently. serverStreamType describes what the RTSP, RTSPS and WebRTC clients receive; directStreamType describes what the push destination receives. A STANAG receiver can be given mpegts-rtp-klv while the RTSP clients keep codec-specific RTP – the push leg runs its own packetizer and its own MPEG-TS muxer when the two differ, and reuses the shared packets (costing nothing extra) when they agree.

serverStreamType configures the served leg and directStreamType the direct-push leg, and they are independent in container and in framing both. The served leg carries a transport stream only inside RTP, so the raw mpegts-klv-* forms are refused there – by initVStreamer() and by setParam() alike – rather than being silently reframed as RTP. The push leg honours the value literally, raw TS included. Asking the two legs for different containers is supported: the push leg re-derives its own framing, KLV signalling, payload type, packetizer and MPEG-TS muxer, keeps its own continuity counters and PSI tables, and falls back to reusing the shared packets only when container, KLV signalling and payload size all agree.

VStreamerParams class

initVStreamer() takes this structure and getParams() fills it. It carries the same information as the VStreamerParam enumeration, so the Not used marks are identical; the defaults below are the interface’s own.

class VStreamerParams
{
public:
    bool        enable{true};
    int         width{1280};
    int         height{720};
    std::string directStreamIp{"127.0.0.1"};
    int         rtspPort{8554};
    /* ... see the table below for the full list ... */
};
Field Type Default Description
enable bool true Streaming enable / disable. false keeps sessions open and publishes nothing.
width int 1280 Stream width, [8:4096]. The input frame may be any size; it is scaled to this.
height int 720 Stream height, [8:4096].
directStreamIp string 127.0.0.1 Destination of the direct RTP push; comma-separated for several receivers.
rtspPort int 8554 RTSP port, shared by every instance in the process.
rtspsPort int 8555 Port of this instance’s own RTSPS listener – unlike rtspPort, it is not shared, so two instances must not name the same one. Opened only when rtspEnable is set, the port is non-zero, rtspEncryption is optional or strict, and a certificate and key are supplied; under those settings a missing certificate or key fails initialisation rather than falling back to plaintext. With the default rtspEncryption no a supplied certificate opens nothing.
directStreamPort int 5004 Destination port of the direct push; RTCP on port+1.
webRtcPort int 7000 WebRTC signalling port, [1024:65534]. The media port is this + 1. See WebRTC.
hlsPort int 8080 Not used.
srtPort int 6000 Not used.
rtmpPort int 1935 Not used.
rtmpsPort int 1936 Not used.
metadataPort int 9000 Not used. KLV travels inside the media stream, not on a port of its own.
rtspEnable bool true Open the RTSP listener.
directStreamEnable bool true Enable the direct RTP push leg.
webRtcEnable bool true Start the WebRTC endpoint. Refused by STRICT_FIPS, which fails initialisation rather than starting without it.
hlsEnable bool true Not used.
srtEnable bool true Not used.
rtmpEnable bool true Not used.
metadataEnable bool false Not used.
rtspMulticastIp string 224.1.0.1/16 Multicast group or CIDR pool. Must be inside 224.0.0.0/4, otherwise multicast stays disabled.
rtspMulticastPort int 18000 Multicast RTP port, [1024:65534] and required even so RTCP can use port+1 (RFC 3550 sec. 11). An odd or out-of-range non-zero value fails initialisation and is refused by setParam(); it is not coerced. 0 means multicast is not configured.
user string no RTSP user. "" or "no" disables authentication.
password string no RTSP password.
suffix string live Stream name, used as the URL path. Must be unique across instances sharing the port.
metadataSuffix string metadata Metadata format: SMPTE336M or VND.ONVIF.METADATA. The interface’s default metadata names no format and is normalised to SMPTE336M at initialisation, so getParams() reports the format actually in force.
minBitrateKbps int 1000 Minimum bitrate for variable-bitrate mode; forwarded to the encoder.
maxBitrateKbps int 5000 Maximum bitrate for variable-bitrate mode; forwarded to the encoder.
bitrateKbps int 3000 Target bitrate; forwarded to the encoder.
bitrateMode int 0 0 – constant, 1 – variable; forwarded to the encoder.
fps float 30.0 Frame rate; forwarded to the encoder and used for pacing.
gop int 30 GOP size; forwarded to the encoder.
h264Profile int 0 0 – baseline, 1 – main, 2 – high; forwarded to the encoder.
jpegQuality int 80 JPEG quality in percent; forwarded to the encoder.
codec string H264 H264, H265/HEVC or JPEG/MJPEG, matched case-insensitively.
fitMode int 0 0 – fit (letterbox), 1 – fill (stretch).
cycleTimeUs int 0 Output only. getParams() reports the measured pipeline cycle here.
overlayEnable bool true Call the overlay renderer on each frame.
type int 0 Forwarded verbatim to the encoder as its backend selector; the meaning belongs to the encoder, not to MServer. For the bundled VCodecLibav: 0 = hardware (VAAPI, then QSV), 1 = software. The default 0 therefore needs a working GPU – set 1 if you have none.
custom1 float 0.0 Compute threads for preprocessing: the upper limit the format converter and the scaler may use, [0 : hardware threads]. 1 by default, which keeps conversion, scaling and the overlay on the pipeline thread. 0 means the same as 1 – it is the interface’s own default for the field, so it cannot be read as a request for the whole machine. To use more cores, ask for them by number. Range-checked, not clamped. See Compute threads for preprocessing.
custom2 float 0.0 Not used by MServer. Opaque caller data; the compliance mode moved to securityProfile in v3.2.
multicastTtl int 0 Multicast TTL for every multicast leg; clamped to 32, 0 means the default of 1.
securityProfile string no Compliance mode by name. An unrecognised value is refused.
publicAddress string no IPv4 literal advertised in SDP and ICE candidates; no means auto-detect.
bindAddress string 0.0.0.0 Interface every listener binds to.
rtpPortMin int 0 Lowest server RTP/RTCP port; 0 – ephemeral.
rtpPortMax int 0 Highest server RTP/RTCP port; 0 – ephemeral.
webRtcMediaPort int 0 WebRTC media port; 0 keeps webRtcPort + 1.
corsAllowedOrigin string no Origin permitted on the signalling endpoint.
serverStreamMaxPayloadSize int 1472 Maximum RTP payload of the server-delivered stream, video and metadata track alike. Clamped to 1200 while the WebRTC leg is enabled.
custom3 float 0.0 Not used by MServer. Free for the application; stored and returned unmodified.
rtspKey string no Path to the RTSPS private key.
rtspCert string no Path to the RTSPS certificate.
webRtcKey string no Private key for HTTPS signalling. no or empty falls back to rtspKey.
webRtcCert string no Certificate for HTTPS signalling. no or empty falls back to rtspCert.
hlsKey string no Not used.
hlsCert string no Not used.
rtmpKey string no Not used.
rtmpCert string no Not used.
rtspEncryption string no no, optional or strict. Any other value fails initVStreamer().
webRtcEncryption string no TLS policy for WHEP signalling: no, optional or strict. The media leg is always DTLS-SRTP. "yes" is rejected.
rtmpEncryption string no Not used.
hlsEncryption string no Not used.
logLevel int 0 Not used. MServer writes no log of its own.
directStreamType string rtp Container and framing of the push leg; see Stream type values.
directStreamBitrateKbps int 5000 Channel bandwidth for the push leg’s packet pacer, kbit/s. Used when directStreamPacingMode is 0.
directStreamMaxPayloadSize int 1472 Maximum RTP/UDP payload of the push leg’s own packetizer; it never shrinks the packets served to RTSP or WebRTC clients. 0 means the default size, and the [576:9000] range is enforced only when the leg is enabled.
directStreamPacingMode int 0 0 – pace toward directStreamBitrateKbps; 1 – push on the frame boundary.
serverStreamType string rtp Container and KLV mode of the RTSP leg; see Stream type values.

custom1..3 are returned by getParams() unmodified, so a getParams() -> initVStreamer() round trip never corrupts them. custom1 is stored verbatim rather than normalised, so this holds even though MServer reads it.

When a parameter takes effect, and how far it reaches

Every parameter answers two questions, and the answers are independent: when does a change apply, and who does it apply to. A value that needs a restart is not a defect as long as the caller is told which ones those are.

Applied immediately

setParam() returns and the change is already in force.

Parameter Notes
bitrateKbps, minBitrateKbps, maxBitrateKbps, bitrateMode pushed into the codec; the two ceilings also set the RTSP send budget of this stream’s subscribers
fps the rate the stream carries, held by dropping and duplicating – see Holding the configured frame rate
gop, h264Profile, jpegQuality, type codec parameters
width, height, fitMode, overlayEnable see Resolution changes – no client is disturbed
codec new dynamic payload type and fresh parameter sets at the next access unit
user, password per path; closes that path’s sessions so a revoked password stops reaching a client that is already playing
suffix moves the stream to the new URL path, carrying its credentials, its send budget, its clock and its multicast policy with it
rtspPort re-binds the shared listener, so every instance follows
rtspMulticastIp, rtspMulticastPort, multicastTtl a group already transmitting keeps its address and only its TTL follows; an unused assignment is re-derived
directStreamEnable, directStreamIp, directStreamPort, directStreamType, directStreamPacingMode, directStreamBitrateKbps the whole push leg – see The direct push leg
serverStreamType without a container change the KLV mode moves freely; the metadata track and the SDP follow at once
custom1 the thread limit the format converter and the scaler work under. Picked up by the pipeline thread on its next frame, so a running stream changes its parallelism without a restart
securityProfile, corsAllowedOrigin, custom2, custom3  

Re-initialising instead of refusing

One change cannot be applied to a running stream: the container of the served leg. The RTP payload type, the SDP and the packetiser have already been given to whoever is playing, and a subscriber cannot be told mid-stream that H.264 has become a transport stream. So setParam(SERVER_STREAM_TYPE) with a container change does not refuse – it rebuilds the stream: the value is stored, the instance re-initialises itself as soon as the parameter lock is released, and clients are dropped and reconnect to a stream whose SDP describes what they are actually being sent. The push leg needs none of this; it switches container in place.

Two consequences for a caller: setParam() may take as long as initVStreamer() for that one parameter, and everything a re-initialisation resets – the pipeline counters, the sequence spaces, the multicast assignments – is reset. It is the same sequence as VStreamerCommand::RESTART.

Read at the next initialisation

Stored when set, in force after the next initVStreamer() (or VStreamerCommand::RESTART): rtspEnable, rtspsPort, rtspEncryption, rtspKey, rtspCert, webRtcEnable, webRtcPort, webRtcMediaPort, webRtcEncryption, webRtcKey, webRtcCert, bindAddress, publicAddress, rtpPortMin, rtpPortMax, directStreamMaxPayloadSize (also re-read at once by a live directStreamType change, and when a live directStreamEnable 1 has to build a push leg that initialisation did not), serverStreamMaxPayloadSize (which also follows a codec change), metadataSuffix (also picked up by a live serverStreamType change, which republishes the metadata format the SDP describes). A socket cannot be moved and a listener that is already accepting plaintext cannot be given TLS underneath it, which is what puts most of that list here.

publicAddress is validated when it is set, not only when it is read: it used to accept a host name and then fail the next initialisation, long after the command that caused it.

Output only

cycleTimeUs is measured by the server and overwritten on every getParams(); setParam() refuses it. metadataPort is refused too – there is no separate metadata listener to point anywhere.

Reach: the stream, or the whole process

See What the listener owns, and what the stream owns. In short: the RTSP port, the WebRTC signalling and media ports, bindAddress and the RTP port range are properties of shared listeners and change for every instance. The RTSPS listener is this instance’s own, so rtspsPort reaches only this stream – and two instances cannot share a value, because the second one’s bind() fails and its initialisation with it. Everything else is the stream’s own, including credentials, bitrate and the whole multicast configuration – except corsAllowedOrigin, which is a property of the shared signalling endpoint and therefore applies process-wide, and securityProfile, which folds into the listener-wide compliance policy (the most restrictive combination of every attached stream).

The pair rule for the RTP port range

rtpPortMin and rtpPortMax are validated together at initialisation: 0/0 means “any ephemeral port”, and any other combination must be able to hold a pair (max >= min + 1), because RTCP takes the port after the RTP one. A lower bound with no upper bound describes a range nothing can allocate from and fails initialisation, so a caller that sets them one at a time should set the upper bound first.

Resolution changes

Two different things can change, and neither restarts the stream or asks a client to reconnect:

  • The input frame size changes while width/height stay put. The scaler absorbs it: conversion, letterbox/crop and overlay all re-target the new source size, the encoder never sees a change, and the SDP is untouched. A connected client observes nothing at all.

  • The stream resolution changes (width/height via setParam). The encoder is reconfigured and emits fresh parameter sets on the next key frame, and the SDP is updated for future DESCRIBEs. Clients already playing keep the same RTSP session, the same SSRC and a contiguous sequence space – they are never torn down and never asked to re-SETUP. MServerResolutionTest asserts exactly this.

    A geometry change also withdraws the description until the new parameter sets exist. The fmtp line carries the encoder’s parameter sets, and those encode the picture size, so between the change and the next key frame the published SDP would describe the previous geometry. DESCRIBE therefore fails for that window rather than answering with an SDP a client would decode against the wrong size. The window also restarts the pipeline: it is gated on consumersPresent(), and a DESCRIBE always arrives before any client has played, so a stream that stayed “describable” would never produce the parameter sets the new description needs and would serve the old geometry for ever. The same rule applies to a codec change, for the same reason.

Build and connect to your project 

git clone --recurse-submodules <repo>
cd MServer
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

Options:

Option Default Meaning
MSERVER_TARGET_ARCH empty value for -march=; deliberately not native, so builds stay reproducible
<PARENT>_MSERVER_TEST ON standalone the interactive test application
<PARENT>_MSERVER_EXAMPLE ON standalone the minimal example
<PARENT>_MSERVER_HARNESS ON standalone the full automated suite

There is deliberately no crypto build option. OpenSSL is always linked, and which provider backs it is a deployment decision: the compliance mode selected through VStreamerParams::securityProfile asks for the validated module when it wants one, and runs on the default provider when there is none – reporting (unvalidated provider) so the claim stays truthful. See Security and compliance modes.

To use as a submodule, add add_subdirectory(MServer) and link MServer. The library itself pulls in only 3rdparty/ (VStreamer, FormatConverter, ImageResizer). The FFmpeg-based test codec is a submodule of the directories that consume it – test/, harness/ and example/ – and is never part of the library. In a build that enables several of them only the first copy is configured; they must therefore be pinned to the same commit, which the root CMakeLists.txt checks and refuses to configure if it is not.

Example

The smallest program that streams video. It generates its own picture, so it needs no camera, no file and no decoder – build the project as above and run it:

./build/bin/MServerExample
# it prints: Streaming on rtsp://127.0.0.1:8554/live

Then point any player at that URL (see How to play the stream).

Two things in the code below are worth noticing before you copy it:

  • The input frame and the output stream are different sizes. The frame fed to sendFrame() is 1280x720 while params.width/params.height ask for 640x480. MServer scales for you; the frame you send does not have to match the stream you publish.
  • initVStreamer() takes the encoder, and MServer keeps using it until the streamer is closed, so it must outlive the MServer object.
#include <chrono>
#include <iostream>
#include <thread>
#include "MServer.h"
#include "VCodecLibav.h"

int main()
{
    // Prepare video server params.
    cr::video::VStreamerParams params;
    params.codec    = "H264"; // Codec type.
    params.width    = 640;    // Output video width.
    params.height   = 480;    // Output video height.
    params.fps      = 25.0f;  // Output video FPS.
    params.gop      = 25;     // Output video GOP size (key-frame interval).
    params.rtspPort = 8554;   // Output RTSP port.
    params.suffix   = "live"; // Stream name.
    params.type     = 1;      // Software codec type for VCodecLibav.

    // Init video server.
    cr::video::MServer server;
    if (!server.initVStreamer(params, new cr::video::VCodecLibav(), nullptr))
        return -1;

    std::cout << "Streaming on rtsp://127.0.0.1:8554/live" << std::endl;

    // Inout video frame.
    cr::video::Frame frame(1280, 720, cr::video::Fourcc::YUV24);

    // Stream loop.
    uint8_t color = 0;
    while (true)
    {
        // Reset input frame with color.
        memset(frame.data, color++, frame.size);
        memset(frame.data + color * frame.width * 3, 255 - color, frame.width * 3);

        // Send frame to video server.
        server.sendFrame(frame);

        // Wait for next frame.
        std::this_thread::sleep_for(std::chrono::milliseconds(35));
    }

    return 0;
}

How to play the stream 

# RTSP over TCP (interleaved)
ffplay -rtsp_transport tcp rtsp://127.0.0.1:8554/live

# RTSP over UDP
ffplay -rtsp_transport udp rtsp://127.0.0.1:8554/live

# UDP multicast
ffplay -rtsp_transport udp_multicast rtsp://127.0.0.1:8554/live

# RTSP tunnelled over HTTP
ffplay -rtsp_transport http rtsp://127.0.0.1:8554/live

# RTSPS. The test application's certificate is self-signed, so verification
# has to be switched off -- that is the certificate, not the server.
ffplay -rtsp_transport tcp -tls_verify 0 rtsps://user:pass@127.0.0.1:8555/live

# GStreamer
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/live protocols=tcp \
  ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink

A note on the HTTPS tunnel. MServer serves it – the GET leg answers with 200 OK and Content-Type: application/x-rtsp-tunnelled over TLS – but ffmpeg cannot consume it: its RTSP tunnel is built over plain HTTP only, so ffplay -rtsp_transport http rtsps://... fails with an I/O error no matter what the server does. Use a client that implements the tunnel over TLS (the Apple / QuickTime stack is the one it was designed for), or connect over plain RTSPS, which every client above supports.

WebRTC needs no player at all – a browser and about fifteen lines. The whole protocol is the one POST below. Save it as whep.html and open it in any browser, straight from disk – MServer answers with Access-Control-Allow-Origin: * by default, so a file:// page is allowed to talk to it.

Put the URL the server actually printed into WHEP_URL. The line below is the library default (webRtcPort = 7000, suffix live); MServerTest uses port 8600 and names its first stream live0, so from the test application the URL is http://127.0.0.1:8600/live0/whep. And start the test application with authentication off (no no) – a browser cannot answer a digest challenge, so otherwise this page gets 401 and nothing plays.

<video id="v" autoplay muted playsinline></video>
<script>
const WHEP_URL = 'http://127.0.0.1:7000/live/whep';   // <-- the printed URL

const pc = new RTCPeerConnection();
pc.addTransceiver('video', {direction: 'recvonly'});
pc.ontrack = e => document.getElementById('v').srcObject = e.streams[0];

pc.createOffer()
  .then(o => pc.setLocalDescription(o))
  .then(() => fetch(WHEP_URL, {
      method: 'POST',
      headers: {'Content-Type': 'application/sdp'},
      body: pc.localDescription.sdp
  }))
  .then(r => r.text())
  .then(sdp => pc.setRemoteDescription({type: 'answer', sdp}));
</script>

Serve that page over HTTPS if the signalling endpoint is HTTPS: a browser refuses a plaintext fetch from a secure page. The UDP media port (webRtcPort + 1) must be reachable from wherever the browser runs – it carries STUN, DTLS and the media itself, so a firewall that allows only the signalling port gives you a connection that negotiates and then stays black.

If you would rather check the endpoint without a browser, one curl does it – a 201 Created with an SDP answer means the whole WHEP leg is working:

curl -i -X POST -H "Content-Type: application/sdp" \
     --data-binary @offer.sdp http://127.0.0.1:8600/live0/whep

The offer must name a codec MServer can send. For H.264 that means packetization-mode=1 in an a=fmtp: line (RFC 6184 interleaved mode is not implemented); every browser offers it, but a hand-written offer that omits it is answered 400 with that exact explanation.

How to extract KLV from the stream

There are two carriages, and which one a client sees depends on serverStreamType.

MPEG-TS in RTP (mpegts-rtp*) – KLV rides its own PID inside the transport stream, which is what a STANAG 4609 receiver expects. This is the only transport-stream form the served leg accepts; the raw mpegts / mpegts-klv-* forms put TS straight into a datagram and exist for the direct push leg, where they are refused on the served one:

# ffprobe reports the KLV PID as a data stream
ffprobe -rtsp_transport tcp -i rtsp://127.0.0.1:8554/live \
  -show_entries stream=codec_name,codec_type

# Dump the KLV elementary stream
ffmpeg -rtsp_transport tcp -i rtsp://127.0.0.1:8554/live \
  -map 0:d -c copy -f data klv.bin

Codec RTP (rtp-klv*) – KLV is a second RTP source, payload type 98, declared in the SDP as its own m=application track with its own a=control:.../trackID=1. A client has to SETUP that track as well as the video one (and after it, since trackID=0 is what creates the session); the KLV then arrives on its own interleaved channel pair or its own UDP port pair, with its own SSRC and sequence space, and never on the video transport. This is the carriage a client without an MPEG-TS parser uses, an ONVIF client above all: it reads the format straight off the a=rtpmap line.

# The DESCRIBE response names the track and its format
ffprobe -rtsp_transport tcp -i rtsp://127.0.0.1:8554/live 2>&1 | grep -A1 "application"

A KLV unit that fits in one packet travels in one packet, and the payload after the 12-byte RTP header is then the whole unit. A larger one is fragmented across consecutive packets of the same timestamp with the marker bit on the last (RFC 6597 sec. 4.2), so a receiver reassembles up to the marker. The fragment size follows the video’s: serverStreamMaxPayloadSize, capped with it when the WebRTC leg imposes its 1200-byte limit.

Testing

There are three separate things, and they serve different audiences.

example/ – the smallest program that streams video. Read this first.

test/ – an interactive application for you to point your own software at. It generates its own animated test pattern, so it needs no input file, no decoder and no OpenCV.

./build/bin/MServerTest [mode 1-8] [profile] [codec] [streams] [user password]

Run it with no arguments and it asks the two questions that change what goes on the wire – which protocol, and which security profile – listing the choices, so there is no order to remember. The other two are arguments only: codec defaults to H264 and streams to 2, which is why the plain interactive run publishes live0 and live1. The mode chooses what to exercise:

Mode Exercises
1 RTSP – RTP, plaintext
2 RTSPS – RTP over TLS
3 MPEG-TS over RTSP
4 MPEG-TS over RTSPS
5 RTSP over HTTP (tunnel)
6 RTSP over HTTPS (tunnel over TLS)
7 WebRTC (WHEP)
8 everything at once – the plaintext, TLS and WebRTC legs of one server, serving concurrently

profile is the security profile (ONVIF, CRA_STRICT, FIPS_ONVIF_BRIDGE, STRICT_FIPS), because what is on the wire differs between them. codec is H264, H265 or JPEG. streams is how many independent streams to publish from the one server; they are named live0, live1, … and each is a real separate stream, not the same one under two names.

Authentication is on by default with admin / admin. Pass no no as the last two arguments to switch it off:

./build/bin/MServerTest 7 ONVIF H264 1 no no     # WebRTC, reachable from a browser

Mode 7 (WebRTC) needs no no to be usable from a web page. A browser’s fetch() cannot answer an HTTP digest challenge, so with authentication left on the WHEP request is answered 401 Unauthorized and the video never starts. That is correct behaviour, not a defect – but it is not what you want while trying the demo. Modes 1-6 are different: ffplay and VLC do speak digest, so leave the credentials on and put them in the URL.

Any mode needing TLS generates a throwaway self-signed certificate through the OpenSSL library if one is not already present – no openssl command line required. Because it is self-signed, clients must be told to skip verification (ffplay -tls_verify 0, curl -k, or “proceed anyway” in a browser).

It publishes the streams, prints the exact client command for the mode chosen, and reports live statistics for the first stream – frames encoded, access units, RTP packets, connected clients, frames dropped and the compliance mode – so you can watch your client attach and see whether media is really flowing. The client count is per stream, so attach to live0, or run with streams = 1, if you want to watch it move.

Mode 8 is the one worth running if you only run one: the same server answers rtsp://127.0.0.1:8554/live0, rtsps://127.0.0.1:8555/live0 and WHEP at the same time, which is how a real device is usually deployed. Note that with TLS in the picture the signalling endpoint is TLS too, so its URL is https:// – the application prints the right one for the mode you chose.

The application deliberately leaves directStreamEnable off, so the direct push leg is not exercised here; it needs a destination address to push to, which only you can supply. It is covered by MServerDirectPushTest in the harness, and described under What each transport carries and configured through the DIRECT_* parameters.

Not every combination is legal, and the application says why rather than failing silently. JPEG with either MPEG-TS mode is refused, for instance, because STANAG 4609 and MISB ST 1402 define no MPEG-TS stream type for motion JPEG – pick H264 or H265 for modes 3, 4 and 8.

harness/ – the full automated suite. In a standalone build (the one above) it is already on, together with the example and the test application, so there is nothing extra to enable:

cd build && ctest --output-on-failure

It is off only when MServer is consumed as a submodule, where a parent project has no reason to build somebody else’s tests. To switch it on there, set the option named in the table above – the prefix is the parent project’s name, which is empty in a standalone build, hence the bare leading underscore:

cmake -S . -B build -D_MSERVER_HARNESS=ON      # standalone spelling

FFmpeg is what the test applications need, not the library. libMServer links OpenSSL and nothing else; the example, the test application and the harness all need VCodecLibav, which is a wrapper around FFmpeg. When the FFmpeg development libraries or headers are missing, those three subprojects are excluded from the build and the library still configures and builds — they are dropped, never failed:

-- VCodecLibav: FFmpeg development libraries/headers not found; test/example/harness disabled
-- MServer: test, example and harness are excluded from the build

The decision is taken at the top level, before add_subdirectory(), because the vendored VCodecLibav creates its target unconditionally and reports a missing FFmpeg with message(SEND_ERROR) — which fails the whole configuration. Deciding first means that code is never reached.

To check that the exclusion really works on a machine that does have FFmpeg:

cmake -S . -B build-noffmpeg -DMSERVER_FORCE_NO_FFMPEG=ON
cmake --build build-noffmpeg -j        # builds libMServer alone

MServerFieldTest is the exception and is built anyway: it encodes in hardware through VCodecLibva and needs libva, not FFmpeg. It excludes itself when libva or the VCodecLibva sources are missing. See Field test rig.

Test Covers Needs
MServerUnitTest Annex-B parsing, parameter sets, slice_type, lifecycle –
MServerRtpTest packetizers, sequencing, drop resilience, clock, RTCP –
MServerSrtpTest SRTP profiles, ROC, SDES/MIKEY, compliance modes, and SRTCP – both profiles’ trailer layouts checked against an independent RFC 3711 implementation, replay window, tampering in each region of the packet –
MServerTsTest PSI CRC, continuity counters, async/sync KLV, ST 0601 –
MServerStunTest STUN parsing against the RFC 5769 test vectors, MESSAGE-INTEGRITY, FINGERPRINT, ICE role attributes, the amplification cap, malformed input –
MServerDtlsTest DTLS-SRTP against a real OpenSSL client, key export equality, fingerprint mismatch refusal, GCM refusal, retransmission after a dropped flight –
MServerHttpTest HTTP/1.1 parsing, request framing and smuggling, the request deadline, TLS, digest authentication, nonce replay rejection, CORS –
MServerWebRtcTest the whole WebRTC leg end to end: WHEP, ICE, DTLS-SRTP and media the test decrypts with keys it derives itself, plus PLI and teardown –
MServerWebRtcIntegrationTest Everything reachable through the public API only: the default configuration answers a browser, CORS, the answer’s a=msid, packet size on the wire, endpoint release on the last stream, compliance modes that must refuse, bindAddress confining every listener (and failing when it cannot be honoured), and RTP port ranges including exhaustion and cross-instance conflicts test codec
MServerCodecTest real encoder, B-frame absence proof test codec
MServerRtspTest RTSP status codes, full session, RTCP reception, malformed input test codec
MServerRuntimeParamsTest live parameter changes, validation test codec
MServerSharedPortTest shared port across instances test codec
MServerResolutionTest input and stream resolution changes without restarting the stream test codec
MServerMulticastTest pool vs fixed group allocation, client-requested destination=, per-request multicast SDP test codec
MServerStrictModesTest the modes ffmpeg cannot reach: RTSPS + RFC 7616 SHA-256 digest, MD5 refusal, replayed-Authorization refusal, profile names rejected rather than defaulted, a FIPS mode starting on the default provider while reporting (unvalidated provider), SRTP over RTSP end to end – including the metadata track, whose packets must decrypt byte-exact under their OWN SSRC – the key taken from the SDP, session keys derived by the test’s own RFC 3711 implementation, every tag verified and the H.264 recovered – and a TLS-mandating mode refusing every transport that would put media in the clear test codec
MServerDirectPushTest the push leg with no RTSP client: RTP framing, MPEG-TS in RTP, bare MPEG-TS in UDP, a push container independent of the server leg proven on the wire, directStreamBitrateKbps pacing measured against its cap, and the codec-RTP metadata track end to end – declared in the SDP with its own control URL for both metadata formats, played by a real client, delivered over interleaved and over UDP on a transport of its own, never leaking onto the video channel, and fragmented and reassembled byte-exact when a unit exceeds one packet test codec
MServerFastPathTest every input format publishes every frame it encodes at the same pacing, so a broken preprocessing fast path shows up as loss test codec
MServerIdleTest idle suppression: the encoder stops entirely with no consumer, DESCRIBE is still answered from the cached description, the first client is served SPS/PPS/IDR within a fraction of a GOP, the stream idles again when the last client leaves, the push leg keeps it awake, MPEG-TS idles from the first frame, and one stream’s viewer never wakes another sharing the port test codec
MServerNoFFmpegDependency asserts FFmpeg is not linked into the library –
MServerInteropFfmpeg H.264 and H.265 over TCP interleaved and UDP unicast, multicast, the HTTP tunnel and MPEG-TS, decoded by ffmpeg with zero warnings ffmpeg
MServerInteropGstreamer rtspsrc over TCP, UDP and multicast gst-launch-1.0
MServerPerfFrameRate measured frame rate within +/-5 % of the configured rate ffmpeg

Every harness target excludes itself automatically when its prerequisite is missing, so the suite builds and runs on a machine without FFmpeg, without GStreamer or without clang – it simply covers less and says so at configure time.

Each test binds ports of its own rather than the default 8554, so an unrelated RTSP server already running on the machine does not turn into a false failure. Within the suite the tests still share the machine’s port space, so every test that listens takes the same RESOURCE_LOCK: ctest -j is safe, it simply does not run two listening tests at once.

The two WebRTC tests are deliberately different, and the difference is the point.

MServerWebRtcTest is not a mock: it builds a browser-shaped peer out of sockets and OpenSSL, runs the real WHEP exchange, the real STUN check and the real DTLS handshake, and then re-implements the RFC 3711 key derivation to decrypt the media. A test that derived the keys with MServer’s own SRTP code would pass even if both sides agreed on the wrong thing; deriving them independently from the DTLS export proves the bytes on the wire are what the RFCs say they should be.

What it does not do is call initVStreamer() – it drives the endpoint directly. MServerWebRtcIntegrationTest exists because that gap is where the defects were: the credential sentinel, CORS, what the answer actually contains, the packet size, the endpoint’s lifetime. None of them is reachable except through the public API, so none of them was covered, and the suite stayed green on a configuration no browser could use. That test therefore includes no impl/ header at all – only MServer.h and a socket, which is everything a deployment has.

Sanitizers

The build files carry no sanitizer options – they are a developer concern, not a property of the library, so they are passed on the command line into a separate build directory:

cmake -S . -B build-asan -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -g"
cmake -S . -B build-tsan -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_CXX_FLAGS="-fsanitize=thread -fno-omit-frame-pointer -g"

Run the suite with the leak suppressions, or the H.265 tests fail on leaks that belong to libx265, not to MServer:

cd build-asan
LSAN_OPTIONS=suppressions=$PWD/../harness/lsan.supp ctest -E "Interop|Perf"

The suppression path must be absolute: the test binaries run from build-asan/bin, so a relative one is resolved against that directory and the sanitizer aborts every test with “failed to read suppressions file”.

harness/lsan.supp is deliberately narrow – it names libx265, libx264 and libav* only – so a leak in MServer itself still fails the run. Those libraries reach the build through the test codec and never through the library, which MServerNoFFmpegDependency asserts.

ThreadSanitizer needs both a suppression file and ASLR off on recent kernels:

cd build-tsan
TSAN_OPTIONS=suppressions=$PWD/../harness/tsan.supp setarch $(uname -m) -R ctest -E "Interop|Perf"

harness/tsan.supp covers FormatConverter, ImageResizer and the OpenMP runtime. Those libraries parallelise their inner loops with OpenMP when custom1 asks them to, and ThreadSanitizer cannot see the OpenMP runtime’s own synchronisation when that runtime is not instrumented, so it reports the worker threads’ stores as races. Each converter instance is owned by exactly one pipeline thread, which is the contract those libraries document. As with the leak file, the scope is narrow enough that a real race in MServer still fails the run.

One more note: the SIMD submodules compile their AVX2 paths only in Release, so layer sanitizer builds on Release rather than Debug.

Fuzzing

harness/fuzz/ holds seven libFuzzer targets covering everything that touches untrusted input: fuzz_rtsp.cpp drives the URL and base64 helpers directly (fast, narrow); fuzz_rtsp_server.cpp speaks to a real listener over a real socket, so request framing, method dispatch, Transport parsing, digest parsing and the session state machine are all on the path (slow, broad); fuzz_klv.cpp walks the MISB ST 0601 parser; fuzz_bitstream.cpp covers Annex-B parsing and the packetizers.

The three parsers WebRTC added have their own: fuzz_stun.cpp drives the STUN parser, the integrity check and the response builder directly; fuzz_http.cpp and fuzz_sdp.cpp speak to a real listener over a real socket, the first exploring request framing and the second the SDP offer behind a fixed, valid WHEP envelope. All three reach the network before anything has authenticated, which is why they exist.

They are built directly with clang rather than through CMake, because libFuzzer is inseparable from -fsanitize and the build files stay free of it. The library must carry the same instrumentation, or the fuzzer runs blind – without coverage feedback from inside the library the input is effectively random, and without ASAN there a memory error that is reached goes undetected:

cmake -S . -B build-fuzz -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
      -DCMAKE_CXX_FLAGS="-fsanitize=fuzzer-no-link,address,undefined -g -O1"
cmake --build build-fuzz --target MServer

mkdir -p corpus/rtsp
clang++ -std=c++17 -fsanitize=fuzzer,address,undefined -g -O1 \
        -Isrc -I3rdparty/VStreamer/src \
        harness/fuzz/fuzz_rtsp.cpp build-fuzz/src/libMServer.a \
        -lssl -lcrypto -o build-fuzz/fuzz_rtsp
./build-fuzz/fuzz_rtsp corpus/rtsp -max_total_time=600

Field test rig

MServerFieldTest drives the library on real hardware from another machine. It exists because the harness proves the protocols against a software encoder on one host, and a product does neither of those things.

  • Hardware encode. Frames go to VCodecLibva, which encodes on the Intel GPU through libva. Its prerequisite is libva, not FFmpeg, so it is built even where the other test applications are excluded.
  • Content that a codec cannot cheat on. Drifting gradients, blocks on Lissajous paths, a sweeping bar and a scrolling comb — a still picture compresses to nothing and makes a broken scaler look like a working one. The frame number is burned in as blocks so a decoded picture can be tied back to the frame that produced it.
  • A source rate beyond suspicion. A busy loop, not a sleep: measured 120.05 Hz with a maximum interval error of 0.5 microseconds. The server’s own rate can only be judged once the source is not in question.
  • A control channel. One TCP port, one command per line. The parameter tables are generated from VStreamer.h, so all 67 VStreamerParam values and all 69 VStreamerParams fields are reachable by name.
./build/bin/MServerFieldTest --control 9099
Command Effect
new <id> / del <id> create or destroy an instance
setp <id> <field> <value> set a VStreamerParams field before init
init <id> / close <id> initVStreamer() / closeVStreamer()
set <id> <PARAM> <value> setParam() on a running stream
cmd <id> <ON\|OFF\|RESTART\|GENERATE_KEYFRAME> executeCommand()
params <id> / stats <id> getParams() / getStats(), plus the rig’s own source-rate measurements
input <id> <w> <h> change the generated INPUT resolution
srcfps <id> <hz> change the source feed rate
klv <id> <none\|klv\|onvif> [size] attach ST 0601 KLV or ONVIF XML metadata

Everything the rig reports about itself — srcFps, srcJitterUs, srcMaxErrUs, tNs — describes the rig, not the server. They are there so a measurement can be shown to rest on a steady source and an exact clock before any conclusion is drawn about MServer.

Troubleshooting

Every failure below is one somebody meets on their first afternoon with the library. None of them is a bug.

The player says 404 Not Found, and the server is clearly running. A path exists for clients only once MServer holds an encoded frame for it – until then there is nothing to describe, so DESCRIBE is answered 404. Call getStats(): if framesIn is climbing but framesEncoded stays at 0, your frames are arriving and the encoder is refusing them. The usual cause is type, which selects the encoder backend and defaults to 0 – hardware – so on a machine with no usable GPU nothing is ever encoded. Set params.type = 1 for the software encoder. MServer prints nothing about this itself, by design: it is a library and writes no log, and getStats() is the channel it reports through.

The player says 401 Unauthorized. Authentication is on for that stream. Put the credentials in the URL – rtsp://admin:admin@127.0.0.1:8554/live – or turn it off with user and password set to "no", which takes effect at once and does not disturb any other stream on the port. A stream that was never given credentials is served without a challenge even while its neighbour is protected; a name that does not exist is answered 404, not 401. For WebRTC there is no way to put them in the URL: a browser’s fetch() cannot answer a digest challenge, so a stream a browser must reach has to have them off.

The TLS client refuses the certificate. The test application generates a self-signed one, and nothing signed it. That is the certificate, not the server: ffplay -tls_verify 0, curl -k, or “proceed anyway” in a browser. In production pass your own rtspCert / rtspKey.

ffplay -rtsp_transport http rtsps://... fails with an I/O error. ffmpeg tunnels RTSP over plain HTTP only. The server side is correct – see the note under How to play the stream.

WHEP answers 400, mentioning the payload type. The offer named no codec MServer can send. For H.264 the offer must carry packetization-mode=1. Browsers always do; hand-written offers often do not.

The browser negotiates, then shows black. Signalling succeeded and media did not arrive. Media does not travel on the signalling port – it is UDP on webRtcMediaPort, or on webRtcPort + 1 when that parameter is left at 0, and that port has to be reachable from wherever the browser runs.

initVStreamer() returns false. It validates before it binds, so the cause is in the parameters, not the network. The frequent ones: a port already taken by another process; a value outside the documented range in VStreamerParams; a strict security profile with no credentials or no certificate, which those profiles refuse by design rather than starting an open server; or JPEG asked for over MPEG-TS, which STANAG 4609 and MISB ST 1402 define no stream type for.

Two instances, and the second one changed the first one’s stream. Give them different suffix values first – a second instance publishing the same suffix is publishing the same stream. If the suffixes differ, check what you changed against What the listener owns, and what the stream owns: the RTSP port, the bind address and the RTP port range really are shared, and changing them anywhere changes them everywhere. Credentials, bitrate and multicast are not shared, and a change to one stream’s copy of them cannot reach another.

The stream delivers fewer frames per second than the source produces. That is fps doing its job: the stream carries the configured rate, not the source’s. getStats() says which side of it you are on – framesDropped climbing means the source is faster, framesDuplicated climbing means it is slower.

A client reports missing packets and the network is clean. Read rtpSendFailures and directSendFailures before suspecting the network: a non-zero value means the kernel refused datagrams this server had already numbered, which a receiver reports as loss. Both stay at zero unless a socket is genuinely congested.

The push leg sends nothing after directStreamEnable was set. Check directStreamIp and directStreamPort are usable – an address of "no", an empty one or a port of 0 leaves the leg with nowhere to go, and that is the one case where enabling it does nothing. Everything else about the leg applies immediately.

Multicast works on one stream and not on another. The multicast configuration is per stream: a pool set on one instance is not inherited by the others. And a group is silent until a client sets it up – configuring a pool does not put anything on the wire.

Limitations

  • IPv4 only. Every inherited helper is AF_INET, and SETUP, SDP and multicast scoping have no IPv6 path.
  • The metadata track needs its own SETUP. On the codec-RTP path the metadata is declared as m=application 0 RTP/AVP 98 with an a=control:.../trackID=1 of its own. The port is 0 in a unicast presentation because the transport is negotiated by SETUP, which gives the track its own socket pair or its own interleaved channel pair; in a multicast presentation the port is real – the pair above the video one. KLV is never pushed onto the video transport, so a unicast or interleaved client that sets up trackID=0 alone receives no metadata at all. A multicast group carries the metadata whether or not any member set the track up, but a member still has to listen on that second port to see it.
  • One MTU per stream, per leg. All RTSP/RTSPS/WebRTC clients of a stream share one packetization – video and metadata track alike, both sized from serverStreamMaxPayloadSize and both capped when WebRTC is enabled – so a client needing a smaller MTU must be served by a separate instance. The direct push leg is the exception: it has its own packet size and packetizes separately when it differs, its own metadata track included.
  • Multicast drops apply to the whole group. One datagram serves every member, so per-member queueing does not exist.
  • RTCP RR cannot report server-side drops. Because a renumbered sequence hides them by design, Stats counters are the only signal that a client is starving; never wire rate control to RR fraction lost.
  • SRTP key derivation is outside the validated module. See the compliance modes above. SRTP is negotiated with Transport: RTP/SAVP and is offered only on the TLS listener, because ONVIF forbids returning SAVP or MIKEY over an unprotected control channel.
  • MIKEY is built but not advertised. Only the pre-shared-key form is implemented (the certificate and Diffie-Hellman modes of RFC 3830 are not), and it has never been checked against a real ONVIF client, so no mode offers it. SDES carries the keys instead. ONVIF Core sec. 5.1.1.4 makes MIKEY mandatory for SecureRTSPStreaming, so this is a known gap against ONVIF, not a design choice: offering an unverified key exchange would let a client negotiate keys it cannot derive and fail silently, which is worse.
  • ONVIF conformance belongs to the device, not the library. MServer implements the ONVIF Streaming Specification and exposes the control points a host’s SOAP layer needs; it contains no SOAP service itself.
  • MJPEG above 2040 px is refused (RFC 2435 limit).
  • RTSP over HTTPS is verified server-side with a TLS client but not against ffmpeg, which does not drive that combination.
  • ffmpeg cannot reach the strict compliance modes. ffmpeg 6.1.1 implements MD5 digest only, and every mode except ONVIF refuses MD5. Those modes are covered by MServerStrictModesTest, which brings its own RFC 7616 SHA-256 client over TLS.
  • STRICT_FIPS and FIPS_ONVIF_BRIDGE cannot be exercised here. Both require the validated FIPS provider, which is absent from the development environment; what is tested is that they refuse to initialise without it.
  • A compliance mode tightens the port it shares. The plaintext listener is process-wide, so its transport policy – whether MD5 digest is offered, whether SRTP may be negotiated, whether interleaved-inside-TLS is required – is the most restrictive combination of every attached stream, not each stream’s own. Credentials are the exception: they are per path, so a claim about who may read a stream is about that stream, while a claim about how the challenge is computed is about the port.
  • The container of the served leg cannot change without rebuilding the stream. MServer does the rebuild itself rather than refusing the command, but clients are dropped and reconnect; see Re-initialising instead of refusing.
  • fps is held by dropping and duplicating, not by asking the source for more. A source slower than fps produces a stream at the configured rate with repeated pictures, which is the right answer for a receiver that needs a stable timeline and the wrong one for a caller who wanted to know the source had stalled – framesDuplicated is where that shows.
  • Configuring writes MServerVersion.h into the source tree. That is the convention every library in this family follows, so a fully read-only source checkout cannot be configured.
  • The nested submodules of the test codec are not de-duplicated. The VCodecLibav target is configured once, but the VCodec and Frame checkouts it and VStreamer each carry are separate directories on the include path. They are pinned independently upstream, so this is a property of the dependency graph rather than of this build.
  • Not verified against VLC or the ONVIF Device Test Tool. Neither is available in the development environment; the tool is a Windows-only proprietary application and conformance is granted to a device, not to a library.
  • WebRTC is not verified against a real browser. No browser is available in the development environment. What is verified is the protocol: the peer in MServerWebRtcTest performs the same exchange a browser does, with OpenSSL on the other side of the DTLS handshake and an independent SRTP implementation decrypting the media. Interoperability with a specific browser’s SDP dialect is therefore untested.
  • WebRTC carries no audio, no data channel and no simulcast, offers no TURN or STUN server, and does no retransmission or congestion control. A host candidate only, so the media port must be reachable from the viewer.
  • The WebRTC media port defaults to webRtcPort + 1 and can be pinned with webRtcMediaPort. It is read when the endpoint starts, and the endpoint is a process-wide singleton that lives until the last published stream goes away, so changing it while any stream is using it has no effect until the next service start.

Table of contents