Serious recent errors.

What this check looks for

Error log entries matching Severity: 19, through Severity: 25,. Each severity is reported separately, so a log containing several kinds produces several findings rather than one.

The check also works out whether the instance restarted in the last two hours, by reading the creation date of tempdb, which is recreated at every startup. That matters for reading the result: entries around a restart are often the explanation for it.

Why it matters

Severity 19 and above are the fatal severities. SQL Server terminated something in every one of these cases, and the higher numbers mean it did so because it no longer trusted its own state.

Severity Meaning
19 A fatal error in a resource. A non-configurable internal limit was exceeded.
20 A fatal error in the current process. The statement was terminated.
21 A fatal error in the database process, affecting every process in that database.
22 Table integrity suspect. The table or index is probably damaged.
23 Database integrity suspect. The whole database is probably damaged.
24 Hardware error. Usually the storage, usually corruption.
25 A fatal system error. The instance normally terminates.

The split is the thing to read first. 19 through 21 are the engine or a resource giving up on one operation, often recoverable and often pointing at configuration or a bug. 22, 23 and 24 are corruption, and belong to the corruption response rather than to error log triage.

Severity 24 in particular means SQL Server met a hardware problem and said so plainly. Almost everything that raises it also shows up as an 823, 824 or 825, which has its own check on this report.

These are also exactly the severities an Agent alert exists for. If this check is the first you knew about them, nothing on this instance is configured to tell you when they happen, which is the missing alerts finding.

How to confirm it yourself

EXEC sp_readerrorlog 0, 1, N'Severity: 19';
EXEC sp_readerrorlog 0, 1, N'Severity: 20';
EXEC sp_readerrorlog 0, 1, N'Severity: 21';
EXEC sp_readerrorlog 0, 1, N'Severity: 22';
EXEC sp_readerrorlog 0, 1, N'Severity: 23';
EXEC sp_readerrorlog 0, 1, N'Severity: 24';
EXEC sp_readerrorlog 0, 1, N'Severity: 25';

Read the previous log as well. A severity 25 usually terminates the instance, and the restart cycles the log, so the entry explaining it is in the previous file:

EXEC sp_enumerrorlogs;
EXEC sp_readerrorlog 1, 1, N'Severity: 2';

Whether the instance restarted, which the check also reports:

SELECT [sqlserver_start_time] FROM sys.dm_os_sys_info WITH (NOLOCK);
SELECT [crdate] AS [tempdb_created] FROM sys.sysdatabases WITH (NOLOCK) WHERE [name] = 'tempdb';

What a specific error number means, once you have one:

SELECT [message_id], [severity], [is_event_logged], 
  FROM sys.messages WITH (NOLOCK)
 WHERE [message_id] = 823 AND [language_id] = 1033;

And whether anything was recorded permanently:

SELECT * FROM msdb..suspect_pages WITH (NOLOCK) ORDER BY [last_update_date] DESC;

How to fix it

Sort by severity first. The response is different above and below 22.

Severity 22, 23 or 24: treat as corruption.

  1. Run DBCC CHECKDB on the named database immediately.
  2. Do not restart the instance, detach the database, or run a repair option as a first move.
  3. Find the last clean backup and verify it elsewhere.
  4. For severity 24, escalate to whoever owns the storage with the timestamps.

See the suspected corruption and CHECKDB errors checks, which cover this properly.

Severity 19, 20 or 21: read the message. These carry a specific error number and the text is usually actionable:

  • A resource limit, such as running out of locks or worker threads. The worker thread check covers one common case.
  • An internal error or assertion, which usually means a bug. Get current on cumulative updates; most are fixed. A memory dump written at the same time confirms it, and that has its own check.
  • A stack overflow from deeply nested code, usually a recursive view or function.

Severity 25: the instance terminated. Look for a memory dump written at the same time, and read the previous error log for what preceded it.

In every case, set up alerts. An Agent alert on each of severities 19 through 25 turns this from something a scan finds into an email when it happens. It is an hour of work and it is the single highest value thing on this page.

How long it takes

About an hour to triage what is in the log and set up the alerts. Acting on an individual finding varies from a patch to a restore.

Report Why you would go there
Error Log The full log rather than the matched lines.
Suspect Pages Whether corruption was recorded.
Alerts and Operators Whether anything would have told you.
Last DBCC CheckDB Known Good by Database When the data was last verified.
Performance History What the instance was doing at the time.
Failed Jobs Jobs that failed in the same window.
Check
Serious errors in log The 823, 824 and 825 subset, on its own terms.
Missing Alerts Alerts on exactly these severities.
Memory dumps detected Usually written alongside a severity 20 or above.
Suspected Corruption Where severities 22 to 24 lead.
SQL Server recently restarted Often the same event from the other side.
Error Log Issues The general error log sweep.

Frequently asked questions

We get severity 19 regularly from one application. Then the message text is the thing to read; a repeating resource error is a fixable pattern rather than an incident. It is also training people to ignore the alert, which is worse.

The instance restarted. Does that explain these? It may be the consequence rather than the cause. A severity 25 terminates the instance, so the restart is the result. Read the previous log.

Should I alert on severity 17 and 18 too? Worth it on a quiet instance and noisy on a busy one. 17 is a resource problem such as a full database, 18 a non-fatal internal error. Start at 19 and add lower ones deliberately.

Nothing is reported but I know we had an error. The check reads the error log as collected. If error log reading was skipped for the scan, or the entry is older than the log retained, it will not appear. sp_enumerrorlogs shows what is still there.