Run a community / Looking after the server

How big a server

Short answer: a lot more than a small group needs. On a server with half a CPU and 512 MB of memory, Sideporch kept messages arriving within a fraction of a second for 4,800 people online at once, each of them posting every two minutes. Most teams are nowhere near that.

“Fast” means 95% of messages are posted, and reach everyone looking at the channel, within one second. Nothing may fail, and nobody may fall behind.

What to run

These numbers came from a fast machine, and real servers’ CPUs are often slower. To leave room, plan with a quarter of them:

• Up to 1,000 people online at once: the smallest VPS you can find, a Raspberry Pi 4, or any home server. Half a CPU and 512 MB are plenty.
• Up to 2,500: 1 CPU and 1 GB.
• Up to 5,000: 2 CPUs and 2 GB.
• More than that: it will likely work on bigger machines, but it wasn’t tested. Sideporch runs on one server with SQLite, which is not built for organisations of tens of thousands.

People online at once are usually a fraction of everyone with an account. Accounts themselves cost almost nothing: the tests had 12,800.

Reading aloud and dictation are the exception: while in use, their models take 0.6 to 3.5 GB of memory, and they run one at a time. Small servers are better off with Whisper tiny, or no speech models.

Signing in is deliberately slow work. Passwords are checked with Argon2, about 6 sign-ins a second on half a CPU. Sessions last 30 days, so this only matters if hundreds of people sign in at the same moment.

Disk grows with what people post and upload; text is small, and files take their own size.

How it was measured

The load generator in tools/loadtest/ simulates people the way browsers use Sideporch:

• Each person keeps a live connection open.
• Each person looks at one of 10 channels, spread evenly.
• Each person posts a message there every two minutes on average (random intervals), and reloads the channel page every five minutes.
• It measures how long each post takes, how long until everyone else looking at that channel has the message, and how long pages take.

Posting every two minutes is far busier than most teams: 30 messages an hour, per person. With 10 channels, every channel has hundreds or thousands of people looking at it. Both make the test harsher than real use.

Each server size is a container with those CPU and memory limits, running the same image people install. It ran in Colima on an Apple silicon Mac, next to the load generator, and started from a database with 12,800 accounts. Each step ran for a minute after everyone had connected; server CPU and memory were sampled every two seconds.

To run it yourself:

# Build the load generator for Linux, once:
docker run --rm -v "$PWD/tools/loadtest":/src -w /src rust:1-slim-bookworm \
  cargo build --release --target-dir /src/target-linux
# Then pick sizes (CPUs:memory) and numbers of people:
tools/loadtest/run.sh "0.5:512m 1:1g" "100 400 1600 3200"
tools/loadtest/report.sh

What limits it

For most of the range, the limit is CPU: fanning messages out to everyone who sees them. Memory stays small, at about 20 KB per connected person plus a few megabytes.

Version 0.3.0 did much worse:

These load tests found four problems, all fixed now:

• Every browser got every message in full. Sideporch 0.3.0 sent every new message, fully rendered, to every connected browser, whether or not it showed that channel. The work grew with the square of the people online. Browsers now say which channel they show; everyone else gets a short “something new” notice, once until they read it.
• Rendering rebuilt everyone’s usernames. Every message and page rebuilt a list of all usernames and custom emoji. It is now kept until accounts or emoji change.
• Mentions checked every account. Finding who a message mentions scanned every account; it now looks up only the names in the message.
• WebSocket buffers took most of the memory. Each connection reserved 128 KB buffers, about 145 KB per person online. That filled half a gigabyte at 4,800 people. They are now sized for the small messages Sideporch sends.

All results

Latencies are the 95th and 99th percentiles over the minute. “Seen” is from posting until the message reached everyone else looking at the channel. Server CPU is the average share of one core. The 0.3.0 runs started from 3,200 accounts, the others from 12,800.

0.3.0: ½ CPU, 512 MB

0.3.0: 1 CPU, 1 GB

0.3.0: 2 CPUs, 2 GB

Now: ½ CPU, 512 MB

Now: 1 CPU, 1 GB

Now: 2 CPUs, 2 GB