Field Note · Recovery
A backup is not real until you can restore it.
Earlier in my career, a Friday release corrupted the SQL Server database behind a live e-commerce store. The recovery took less than an hour. The more important result was what changed afterward.
The situation
A live store, one new DBA, and no proven recovery process.
I had joined a startup as a contractor and was its only DBA. The store depended on a small set of production databases, but backups, monitoring, restore tests, and documentation had not yet become routine.
I began scheduling backups because the absence of a process did not reduce the business risk. It only made the risk less visible.
The incident
The new release damaged database pages.
A Friday application rollout included changes that had not been validated well enough. The application stopped functioning normally, and SQL Server's consistency checks reported page corruption. The store was already live, so waiting for a perfect explanation was not a neutral choice.
The decision
Restore a known state instead of improvising on the damaged one.
I chose the most deterministic recovery path available: restore the database to the pre-release backup and return the store to a known operating state. The recovery completed in under an hour.
We lost only the activity between that backup and the failed rollout—a window of minutes to a few hours. That loss was small compared with preserving the essential state of the business and restoring service predictably.
What changed
The emergency became operating practice.
Scheduled backups, monitoring, documented recovery steps, and backup-and-restore testing became standard DBA practices. I was later promoted from contractor to full-time DBA, but the more useful evidence of trust was that recovery stopped depending on a single Friday improvisation.
DBRE lesson
Backup success is an input. Restore evidence is the outcome.
A completed backup job proves that bytes were written somewhere. A rehearsed restore proves that the organization can recover the system it depends on—within understood time and data-loss boundaries.