This puts a lot of state into a new `LockedStream` that was formerly guarded more coarsely by the recording lock or more granularly by `RecordingToInsert` locks. And it restructures `LockedDatabase::flush` to avoid holding this lock while performing SQLite operations. Streamers write only into a recent frames buffer in RAM, which the directory pool workers asynchronously write to disk approximately once per GOP, where formerly there was a write call per frame. Before, streamers could fall behind due to slow sample file directory disk I/O calls or if writing got stuck in a retry loop, for example if out of disk space. If Moonfire falls too far behind in receiving, cameras tend to disconnect or (sadly) mis-frame messages because they do not handle this case well. This removes a confusing potential cause of such errors. In the new model, streamers never wait for disk I/O. Frames are buffered up to a limit. Recordings are aborted if the I/O pool falls too far behind. There's no retry loop. This roughly halves CPU usage as thread hand-offs and syscalls are less common. It also enables several desirable future changes: * audio recording (which radically increases the number of frames and thus would otherwise significantly increase CPU usage). * "monitor mode" (allowing live view without recording to disk), * further CPU reductions for live view (by skipping all SQLite ops and disk I/O). In the new model, streamers also rarely wait for SQLite database operations either—now only on video parameter changes. That might go away in an upcoming change too. The "syncer" (which synced files/dirs and kept track of what has to be flushed into the SQLite database when) has mostly gone away too. The directory pool workers handle the syncing now, and then they notify a new "flusher" for the latter portion. I'd like to go further still and have the database as a whole be owned by a dedicated thread, the only one that uses the SQLite connection. This prevents the tokio threads from blocking on SQLite operations. I tried out doing so before making the streamers less dependent on the db, but the extra thread handoffs were awful for performance.
5.2 KiB
Building Moonfire NVR
This document has notes for software developers on building Moonfire NVR from source code for development. If you just want to install precompiled binaries, see the installation instructions instead.
This document doesn't spell out as many details as the installation instructions. Please ask on Moonfire NVR's issue tracker or mailing list when stuck. Please also send pull requests to improve this doc.
Downloading
See the github page (in case you're not reading this text there already). You can download the bleeding-edge version from the commandline via git:
$ git clone https://github.com/scottlamb/moonfire-nvr.git
$ cd moonfire-nvr
Building
Moonfire NVR should run natively on any Unix-like system. It's been tested on Linux, macOS, and FreeBSD. (In theory Windows Subsystem for Linux should also work. Please speak up if you try it.)
To build the server, you will need SQLite3. You
can skip this if compiling with --features=rusqlite/bundled and don't
mind the moonfire-nvr sql command not working.
To build the UI, you'll need a nodejs release in "Maintenance", "LTS", or "Current" status on the Release Schedule: currently v20, v22, v24, or v25.
On recent Ubuntu or Raspbian Linux, the following command will install most non-Rust dependencies:
$ sudo apt-get install \
build-essential \
libsqlite3-dev \
pkgconf \
sqlite3 \
tzdata
Ubuntu 20.04 LTS (still popular, supported by Ubuntu until April 2025) bundles
node 10, which has reached end-of-life (see
node.js: releases).
So rather than install the nodejs and npm packages from the built-in
repository, see Installing Node.js via package
manager.
On macOS with Homebrew and Xcode installed, try the following command:
$ brew install node
Next, you need Rust 1.91+ and Cargo. The easiest way to install them is by following the instructions at rustup.rs. Avoid your Linux distribution's Rust packages, which tend to be too old. (At least on Debian-based systems; Arch and Gentoo might be okay.)
Once prerequisites are installed, you can build the server and find it in
target/release/moonfire-nvr:
$ cd server
$ cargo test
$ cargo build --release
$ sudo install -m 755 target/release/moonfire-nvr /usr/local/bin
$ cd ..
You can build the UI via pnpm and find it in the ui/build directory:
$ cd ui
$ pnpm install
$ pnpm run build
$ sudo mkdir /usr/local/lib/moonfire-nvr
$ cd ..
$ sudo rsync --recursive --delete --chmod=D755,F644 ui/dist/ /usr/local/lib/moonfire-nvr/ui
For more information about using pnpm, check out the Developing UI Guide.
If you wish to bundle the UI into the binary, you can build the UI first and then pass
--features=bundled-ui when building the server. See also the
release workflow which statically links SQLite and
(musl-based) libc for a zero-dependencies binary.
Running interactively straight from the working copy
The author finds it convenient for local development to set up symlinks so that
the binaries in the working copy will run via just nvr:
$ sudo mkdir /usr/local/lib/moonfire-nvr
$ sudo ln -s `pwd`/ui/dist /usr/local/lib/moonfire-nvr/ui
$ sudo mkdir /var/lib/moonfire-nvr
$ sudo chown $USER: /var/lib/moonfire-nvr
$ ln -s `pwd`/server/target/release/moonfire-nvr $HOME/bin/moonfire-nvr
$ ln -s moonfire-nvr $HOME/bin/nvr
$ nvr init
$ nvr config
$ nvr run
(Alternatively, you could symlink to target/debug/moonfire-nvr and compile
with cargo build rather than cargo build --release, for a faster build
cycle and slower performance.)
Release procedure
Releases are currently a bit manual. From a completely clean git work tree,
- manually verify the current commit is pushed to github's master branch and has a green checkmark indicating CI passed.
- update versions:
- update
server/Cargo.tomlversion by hand; runcargo test --workspaceto updateCargo.lock. - ensure
README.mdandCHANGELOG.mdrefer to the new version.
- update
- run commands:
VERSION=x.y.z git commit -am "prepare version ${VERSION}" git tag -a "v${VERSION}" -m "version ${VERSION}" git push origin "v${VERSION}"
The rest should happen automatically—the tag push will fire off a GitHub Actions workflow which creates a release, cross-compiles statically compiled binaries for three different platforms, and uploads them to the release.