The Crowdstrike incident from a few weeks ago was remarkable. With the push towards lean and agile development, it is surprising that more incidents like this have not occurred. Teams now have more autonomy and control over systems, leading to stuff getting released that probably isn’t ready. While you may not have caused an endless reboot loop, consider the thousands of micro-failures your users experience daily.

Don’t believe me? Think about this. A paying customer visits your site or app, tries to access something, and it takes forever. Micro-failure #1. They enter something in the search box and hit enter, but nothing happens until they click on an icon they don't recall having to do before. Micro-failure #2. They access some content, and the quality is bad. Micro-failure #3. None of these issues are enough for them to contact customer support. Even if they did, your support options are limited to a help portal with no relevant information or chatting with an AI help agent. Maybe your system is better and has extensive logging. Your product manager reviews a weekly report, and just maybe they find something interesting to pass to engineering. We all know how that usually turns out.

So what's the answer? Apologize to the user in real time. Use a messaging system that automatically displays a message to the end user as they are experiencing the micro-failure. “Jackie, something’s wrong. Sorry about that, we are taking a look.” The user will be less annoyed, and more importantly, you will see in real-time how often that message is triggered. It's one thing for engineering to log something, but it's a whole other thing when the VP of Growth or Product sees how often they are apologizing to the end user. This is your canary. Simple, quick, and highly effective.

The team at Redfast has helped dozens of online businesses respond to events in real time. Reach out to one of our team members to learn more or just reply to this message. We will be happy to share our best practice playbook with you!