JoinGG

The staging server nobody has

A few pounds a month against the update that breaks in front of a hundred people. What twenty minutes of testing actually catches.

SweetMask · 3 min read

Every server owner has, at some point, updated a plugin on the live server because it was a small change and they were fairly sure it would be fine. Most of the time it is fine. The times it is not are the times a hundred people are watching.

A staging server is a second copy of your server that nobody plays on. It costs a few pounds a month and it is the most-skipped piece of infrastructure in community hosting, because its entire value is in events that have not happened yet.

What it is, minimally

A second instance running the same server software, the same plugin versions and a copy of the world. It does not need the same hardware, it does not need to be always on, and it does not need to be reachable by anyone but you.

Started before a change, used for twenty minutes, stopped afterwards. On an hourly-billed provider that is pennies — genuinely less than a rounding error against what the live server costs, which is the detail that makes the usual objection to it hard to sustain.

What it catches that reading the changelog does not

Plugin conflicts. Version 3 of one plugin and version 7 of another have not necessarily been run together by anybody before you. A changelog cannot tell you this; starting both can.

Config migrations. Many plugins rewrite their config on first run with a new version. If that migration loses a setting, you find out on staging, where the config you lost is a copy.

Startup failures. The simplest and most common. A version that will not load at all is a five-minute discovery on staging and a very public one on live.

World compatibility. A Minecraft version upgrade touches the world format. Some upgrades are one-way, and that is exactly the sentence you want to read while looking at a copy.

The copy is the whole trick

Staging is only useful if it is the same. A staging server with a fresh world and three plugins tests nothing, because your problems live in the interaction between your specific plugins and your specific world.

So the workflow is: stop live briefly or take a consistent snapshot, copy the world and the config tree, start staging from it. That is the same operation as taking a backup, which is why the two habits reinforce each other — a server that can produce a staging copy on demand is a server that has a restore path that works, because it has just used it.

Twenty minutes of actual testing

Starting it is not testing it. The checklist worth running:

  • Does it start clean? Read the startup block.
  • Do the four or five things players do most still work? Log in, teleport, open a shop, use the thing your economy runs on.
  • Does the permissions plugin still recognise your groups?
  • Does the console go quiet after startup, or is something warning repeatedly?

Twenty minutes. Write the list down once and reuse it, because a checklist you improvise each time is a checklist that skips whatever you forgot, and what you forgot is where the problem is.

The version you keep

After a successful update, the useful artefact is the record: which versions, together, on which date, working. A file with fifteen lines in it.

When something breaks in three weeks, that file is the difference between rolling back to a known state and guessing. Servers that update confidently are not the ones with better luck; they are the ones that can answer "what changed" without archaeology: the same question that makes any crash diagnosable.

Why people skip it

Because the cost is certain and the benefit is probabilistic. You pay a few pounds and twenty minutes every time, and most of the time nothing would have gone wrong anyway.

That trade looks bad until the first time it saves you, and then it looks obvious forever. The servers that have one almost always got it after an incident rather than before, which is a fair description of most operational discipline and the same reason nobody sets up monitoring until after the outage they could not explain.

If you run anything more than a handful of friends play on, spin one up before your next update rather than after your next disaster.

Tags
operationstestingmaintenance
Share

SweetMask

Published · 3 min read

All articles

Keep reading

Guides

Whitelisting, and who it keeps out

The most effective moderation tool and the most expensive one. The cost is your growth rate, and nobody evaluates that part.

· 3 min read
Guides

Running several game servers on one box

They cannot see each other and they are competing. CPU contention, the memory arithmetic, and the port map nobody writes down.

· 4 min read
Guides

Reading server logs without drowning

Four categories, two that need you. Start at the top rather than the bottom, and learn what a quiet log looks like.

· 4 min read