Evaluating the Effectiveness of Risk Assessments Post-Incident

[Lillian Whitehead]

How do you evaluate the effectiveness of risk assessments post-incident?

 

[Marzena STC & QTC]

When evaluating the effectiveness of a risk assessment post-incident, the first step is to compare the incident with the original risk assessment. Was the risk identified? If it was, did the controls in place work as expected, or did they fail? If the risk wasn’t considered at all, it’s a clear sign that the assessment needs to be updated.
Next, the control measures should be reviewed. If they were in place but the incident still occurred, were they properly implemented, or did they need strengthening? A root cause analysis is also essential to determine whether the issue stemmed from unforeseen risks, or gaps in existing controls. Please remember: human error is not the root cause, as many root cause analyses conclude.
Feedback from staff involved in the incident is crucial, as they can often highlight whether the controls were practical and understood. If similar incidents continue to happen, it suggests that the risk assessment wasn’t entirely effective, which calls for adjustments.
Finally, the changes made should be monitored over time to ensure they are working. Ongoing reviews and possibly additional training might be necessary to improve the situation.
@Cathy Gladwin any other ideas?

 

[Cathy Gladwin]

@Marzena STC & QTC Fantastic advice there Marzena!
I always recommend that sites update their RCA form to include an additional step which is "now go and update your risk assessment to include the additional controls", (from corrective and preventive actions). It closes that loop. Risk assessment was supposed to prevent the issue - but for the reasons you mentioned, it did not prevent the issue. So after all the work done "after the horse has bolted", we can make sure that at least horses won't bolt in the future because the "stable door will be bolted" at the right time (as the saying goes). We can't always prevent issues, but hindsight will help our overall Business Risk Mangement / Mitigation.

 

[Steph]

This is a really interesting thread. I would be tempted to attach a report to the new RA, which provides details of how the incident occurred. A strengths-based analysis which commended good practice but with recommendations for future actions.

 

[Cathy Gladwin]

This is a really interesting thread. I would be tempted to attach a report to the new RA, which provides details of how the incident occurred. A strengths-based analysis which commended good practice but with recommendations for future actions.

Actually your point there is a great idea @Steph. I wouldn't have thought to specifically include the incident as an attachment. But there is no reason not to - and you have reminded me of the importance of "change history" for all documents - including risk assessments. The incident should really be documented as the reason for the risk assessment update - there's the link to the incident itself - just as you have suggested and we end up with a great audit trail.

 

[Harrison]

How often do you reassess your product risk classifications, and what triggers a change?

 

[Cathy Gladwin]

How often do you reassess your product risk classifications, and what triggers a change?

Ah, actually this is something that a lot of people who attend the BRCGS risk assessment course ask us - because the course offers various flexible examples of how to classify or how to score risk - likelihood and severity (and optionally detection), that very flexibility leaves people indecisive, instead of secure in the knowledge that they can pick a way of scoring the risk that works for them. Furthermore, if what you picked ends up not working as well as you'd have liked, go ahead and change it!

So back to the question, reassess how you classify your risk factors whenever you like. But that's slightly different to the question of when to review your risk assessment - that's got to be regularly (usually at least annually) and when there's a problem - that's a trigger. Something within the scope of the risk assessment has gone wrong and it's either serious or there's a trend of issues. Since the whole point of a risk assessment is to prevent issues - if we experience a problem - then the risk assessment might have failed in it's purpose (other option is that the root cause of the issue is based in a lack of compliance). So of course, we need to review the risk assessment when triggered. This relates to the above conversations in this post - for reference: https://1stc.uk/forum/threads/evalu...of-risk-assessments-post-incident.20/post-152