As of 12/17/24, this article is no longer accurate. Please refer to the Trending Incident Process Guide that is applicable to ALL support teams across ALL brands.
Detection: All Tier 1 Support Technicians, Senior Technicians, and Supervisors will monitor incoming tickets and look to identify any potential trending incidents with a large customer impact.
-
Escalation: Once a Trending Incident is suspected, escalate the incident to higher-level Support or Engineering team members for further investigation.
Tier 1 Support Technicians will escalate to a Senior Technician or Supervisor
Senior Technicians or Supervisors will post in the “Ask Apteryx Developers” channel for confirmation
-
Confirmation and Messaging: Once Engineering teams have confirmed a trending incident is occurring:
-
Engineering Leadership will:
Determine and post messaging for the Planet DDS Status page
Determine and post internal messaging in the “Apteryx - Incident Log” channel in Teams
-
Support Leadership will use this to:
Craft messaging for our “Apteryx Trending Incident - Initial Response” macro
Include any possible workarounds
Create a unique tag for the macro such as “Apteryx_Trending_Incident_1_12_1990” for accurate reporting and monitoring (for 1/12/90 in this example)
Macro will be configured to assign tickets to the Apteryx Trending Incidents group in Zendesk
-
Triage: Once messaging has been confirmed, ALL Apteryx Support Technicians will collaborate to grab all related Trending Incident tickets from the support queue and apply the “Apteryx Trending Incident - Initial Response” macro. Once completed, Tier 1 technicians will return to working the Support queue and monitoring for new tickets related to the trending incident. New related tickets will be updated with the existing Initial Macro.
-
Ongoing Updates:
Apteryx Engineering will provide updates on the Apteryx Status page.
-
Support Leadership and Senior Technicians will:
Monitor the “Apteryx - Incident Log” group for internal updates
Coordinate on any relevant update messaging that should be crafted and sent for “Apteryx Trending Incidents - Update” or “Apteryx Trending Incidents - Resolved” macros
-
Resolution:
-
Apteryx Engineering will:
Update the Apteryx Status page with a notification of the resolution and steps taken to prevent recurrence.
Update the Apteryx - Incident Logs channel for Internal Stakeholders and provide an RCA for escalated responses
-
Support Leadership and/or Senior Technicians will:
Use Engineering’s messaging of the resolution and steps to mitigate further disruption to assist in crafting a Customer-facing response explaining the time the issue was identified, the time the issue was resolved, and any steps taken to resolve or mitigate further disruption.
Update the “Apteryx Trending Incidents - Resolved” macro
Apply the “Apteryx Trending Incidents - Resolved” macro on all tickets in the “Apteryx Trending Incidents” group in Zendesk.
If customers request an official RCA, Support Leadership and/or Senior Technicians will work with Engineering to obtain and provide one.
-
Macro Links:
Initial - Macros - Agent tools - Workspaces - Zendesk Admin Center
Update - Macros - Agent tools - Workspaces - Zendesk Admin Center
Closed - Macros - Agent tools - Workspaces - Zendesk Admin Center