The first hour of an incident

Something has gone wrong. The server is down, or duplicating items, or a griefer is three minutes into a base that took a month. What you do in the next hour decides whether this is a bad evening or the thing people remember the server for.

The instinct is to start fixing. That is the wrong first move, and it is wrong for a reason worth internalising: you cannot fix what you have not stopped, and you cannot investigate what you have already overwritten.

Stop the bleeding first

Before diagnosis, before communication, before anything: stop the damage from growing.

For a griefing incident that is a ban and a rollback boundary. For an exploit it is removing the plugin or disabling the command. For a crash loop it is stopping the service rather than letting it restart into the same crash forty more times.

A server that is off is not failing. A server that is restarting every fifteen seconds is generating a log you cannot read, contending for the disk, and possibly corrupting the thing you are about to need.

Take a copy before you touch anything

This is the step people skip and the one they regret.

Before you fix, restore, roll back or delete anything, copy the current state somewhere else. The broken state is evidence, and the moment you start repairing you lose the ability to work out what happened.

It is also insurance against the fix making things worse, which happens more often than anyone admits. A rollback to the wrong point, applied to a world with no copy of its pre-rollback state, turns one bad hour into two irrecoverable ones, and this is precisely the moment when a backup regime you have never tested turns out to have been theoretical.

Say something within fifteen minutes

Not an explanation. An acknowledgement.

"The server is down, we know, we are looking at it, update in thirty minutes." That is the whole message and it should go wherever your community actually reads, which for most servers is Discord rather than the game.

Silence is what turns an outage into a rumour. Within an hour of an unexplained outage, somebody in your community will have decided the server is closing, and other people will believe them. Fifteen minutes of honesty prevents a week of that.

Then keep the promise about the update. If you said thirty minutes and know nothing new, say that — "still working on it, no news" is a real update.

Work out what changed

Almost every incident has a change behind it: a plugin updated, a config edited, a version bumped, a player given permissions they should not have.

So the first diagnostic question is not "what is broken" but "what changed in the last 48 hours". If you keep a record of updates, this takes a minute. If you do not, this is the moment you start keeping one.

Read the first error rather than the last, because the last one is the consequence — the same principle as reading any log under pressure.

Decide the rollback boundary deliberately

If a rollback is needed, the question is to when, and it is a trade: further back loses more legitimate play, closer in risks keeping the damage.

Pick the point, state it publicly, and be specific: "rolling back to 19:00, you will lose about two hours" is manageable. Rolling back an unstated amount is how people discover their evening is gone while reading chat.

Some things can be restored selectively. Most cannot, under time pressure, and attempting surgical restoration during an incident is how a one-hour outage becomes a five-hour one.

The decisions that are harder than they look

Three calls come up in almost every incident and all three are made badly under pressure, which is why they are worth deciding in advance.

How far back to roll. The instinct is the most recent backup. The correct answer is the most recent backup taken before the problem started, and those are frequently not the same thing — a griefing incident discovered on Sunday may have begun on Friday, and restoring Saturday night restores the damage. Establish when it started before choosing what to restore.

Whether to keep the server up. Leaving it running while you investigate is comfortable and is usually wrong when the damage is ongoing or the evidence is being overwritten. It is usually right when the problem is contained and taking the server down costs more than the problem does.

Who is allowed to act. More than one person making changes during an incident is how a recoverable situation becomes an unexplainable one. Say who is driving, in the channel, even if the team is two people.

What to preserve before you fix anything

Fixing destroys evidence, and the order that protects both is copy first, then repair.

The copy that matters is not another backup of the world. It is the state at the moment of discovery: the current world as it stands damaged, the logs covering the window, the database if there is one, and whatever the panel or the console is showing. Ten minutes of copying is the difference between knowing what happened and having a theory.

Servers that skip this are not careless. They are in a hurry, which is exactly the condition the rule exists for.

Afterwards, write it down

Same day, while it is fresh. What happened, what caused it, what you changed, what would have prevented it.

Half of these end with a preventable cause you already knew about: an unmaintained plugin, an untested backup, a permission nobody audited. The write-up is what turns that from a thing you knew into a thing you fixed, and it is the difference between a server that gets more reliable and one that has the same incident twice.

It is also the honest record to point at when somebody asks what happened, which is worth more to a community's trust than a confident explanation invented on the night: the same currency as everything else that keeps regulars through a bad week.