DB2 z/OS - Message id - DSNJ110E - Additional alerts for active log monitoring and highlighted new message id when DB2 logging was suspended

We had a major incident due to suspended logging for one of the critical customer.

One of the critical production DB2 subsystem logging was suspended , caused by stalled archive log offloading process due to insufficient storage space on archive log volume pool. DB2 utilized all available active logs and stalled when it utilized the last available active log.

We have implemented several measures to eliminate the re-occurrence of this scenario.

a) Added additional active logs.

b) Increased the storage space allocation of archive logs.

c) Routing the backup copy of archive logs to tape etc.

We have a concern with respect to the active log alert monitoring. DB2 z/OS issues warning message for every 5% when the last copy of active log is utilized.

When a junior resource is involved in debugging situation, it would be very difficult to understand the problem especially during conference call and there is no highlighted message available today to indicate the DB2 logging is suspended.

To get additional attention from the system operator and technical teams, we recommend a DB2 code change and have DB2 to issue additional warning messages when it hits 90% for every 1% (i.e., 90,91,92 etc.,) and display highlighted warning message on the console when DB2 logging is suspended.

Pain points:

a) The technical team and operator missed the DSNJ110E alert . The suspended logging had major impact on the system. The team was diverted fixing individual issues, causing huge delay in resolution.

Proposal / Idea:

a) We recommend a DB2 code change and have DB2 to issue additional warning messages when it hits 90% for every 1% (i.e., 90,91,92 etc.,)

b) We recommend a highlighted message to stay on console when DB2 logging got suspended and the message should go away once the logging is resumed.

Benefits/Value:

a) The highlighted message will get more attention from system operator and DB2 technical teams.

b) Junior resources can quickly identify if there is an message indicating suspended logging , which will aid quick resolution.

Thanks,

V Prasanna