Monitoring Failures Across Zapier, Make, n8n, and Apps Script

The Business Situation

Alder Peak Equipment Services is a fictional 86-person business that maintains industrial equipment for commercial customers. Its finance, customer service, field operations, sales, and IT teams rely on automations built with Zapier, Make, n8n, Google Apps Script, and Microsoft Power Automate.

The internal IT and infrastructure function consists of an IT manager and a systems analyst. An operations analyst maintains several department-specific workflows but does not administer the automation platforms.

Note: This case study is provided as a representative example of the types of AI integration and digital transformation solutions Intelligex designs and delivers. Actual engagements are tailored to each client’s goals, constraints, existing systems, timeline, and available resources, so the approach, tools, and outcomes may vary.

The business had 46 active workflows:

• 12 Zapier Zaps for sales, customer service, and CRM updates
• 9 Make scenarios for document routing and data synchronization
• 8 n8n workflows for API integrations and scheduled data processing
• 13 Google Apps Script jobs for spreadsheet, email, and file operations
• 4 Power Automate flows for Microsoft 365 approvals

Together, these workflows produced approximately 3,200 terminal run events each month. A terminal event is the final outcome of an execution, such as success, failure, timeout, or cancellation. Selected critical workflows also sent periodic heartbeat events to confirm that scheduled processes were still running.

Each platform retained its own execution history, but there was no shared operational view. The IT team often learned about failures when a user reported a missing document, an unprocessed form, an absent notification, or an outdated spreadsheet.

The immediate goal was not to replace the automation platforms. It was to create a common reliability layer that could answer six questions:

• Which workflows are healthy?
• Which workflows have failed?
• Who owns each failed workflow?
• Has someone acknowledged the incident?
• Was the failed work retried successfully?
• Which scheduled workflows have stopped reporting altogether?

The Existing Process

The original support process was reactive and followed this sequence:

1. A workflow ran within Zapier, Make, n8n, Apps Script, or Power Automate.
2. If it failed, the platform might send an email to the account administrator.
3. Platform emails were mixed with unrelated administrative messages.
4. A department user eventually noticed that an expected business action had not occurred.
5. The user contacted the IT manager or the workflow’s original builder.
6. The IT team searched several platforms to locate the relevant execution.
7. The team copied the error into Slack or an informal spreadsheet.
8. The workflow was retried, corrected, or left for later investigation.
9. There was no reliable record showing acknowledgement, resolution, repeat failures, or time to recover.

The platforms also handled failures differently. n8n supported dedicated error workflows. Make provided error-handling routes. Apps Script could catch runtime exceptions, but trigger failures or authorization problems could prevent the logging code from running. Zapier required a combination of final logging steps and a separate error-monitoring Zap. Power Automate required explicit success and failure scopes.

This meant that a single connector design could not simply be copied without adaptation. The central event schema could be shared, but each source platform needed a platform-specific reporting pattern.

What the New System Needed to Do

The system also needed to remain understandable to the two-person IT team. It could not require a full observability engineering function or a new custom application for basic administration.

Implementation Approaches Considered

Why email-only monitoring was rejected

Email alerts could be sent to a shared inbox, but they would not provide consistent correlation IDs, ownership, acknowledgement, retry status, or cross-platform reporting. An email could also be delayed, filtered, or ignored.

Why a manual register was rejected

A manual spreadsheet would improve incident notes but would not solve the initial detection problem. It would create another administrative task after every failure.

Why a managed platform was deferred

A managed observability or incident-management platform would provide more mature retention, alert routing, service maps, and on-call scheduling. Alder Peak’s 3,200 monthly executions and small support team did not yet require that level of infrastructure.

Why the Apps Script and Google Sheets approach was selected

The business already used Google Workspace, Apps Script, and Slack. Google Sheets provided an understandable system of record, while Apps Script could expose an HTTPS web application, validate incoming events, create incidents, and send alerts.

The design also established a platform-neutral event contract. If the business later replaced Google Sheets with a database or managed monitoring service, source workflows could continue sending nearly the same JSON payload.

The Selected Solution

The selected implementation used a protected Google Sheets workbook as the reliability register and a spreadsheet-bound Google Apps Script project as the event intake and monitoring service.

Existing automation tools were retained. The implementation removed manual failure transcription, manual owner lookup, and routine cross-platform checking. Technical diagnosis, retry approval, business-impact assessment, and final closure remained human-controlled.

Slack incoming webhooks were used to post into a restricted automation-operations channel. Owner and escalation Slack member IDs were stored in the workflow register so messages could include direct mentions.

The implementation used an Apps Script web application. Google documents this deployment model in its web application guidance. Slack webhook behavior and security considerations are documented in the Slack incoming webhook documentation.

System Architecture and Data Flow

• Intake: HTTPS POST requests containing a normalized workflow-run event.
• System of record: A protected Google Sheets workbook with Workflows, Runs, Incidents, and System Errors tabs.
• Automation layer: A spreadsheet-bound Google Apps Script web application and scheduled monitoring trigger.
• Document storage: No business documents are copied. Source run URLs point back to each platform’s authorized execution history.
• Notifications: Slack incoming webhook messages sent to a restricted operations channel.
• Reporting: Filtered Google Sheets views, formulas, pivot tables, and scheduled management summaries.
• AI layer: Optional error classification and remediation suggestions, introduced only after the rule-based monitoring works reliably.

1. A workflow starts. The source platform creates or exposes a run ID. Where possible, the workflow also sets a correlation ID that remains unchanged across retries.
2. The workflow finishes or fails. The source adapter converts the platform-specific result into the shared event schema.
3. The source posts an event. The request contains the source token, event ID, workflow ID, platform, status, timestamps, retry count, error details, and source run URL.
4. The web application authenticates the source. The platform name is matched to a source-specific secret stored in Apps Script Properties.
5. The payload is validated. Required identifiers, platform, status, dates, retry count, URL, and error fields are checked.
6. Duplicate delivery is checked. The event ID is searched in the Runs sheet. An existing ID returns a successful duplicate response without writing another row.
7. The workflow definition is loaded. The service retrieves the owner, Slack member ID, criticality, stale threshold, and escalation target.
8. The run is recorded. A normalized append-only event is written to the Runs sheet.
9. The workflow summary is updated. Last run, last event time, status, error, retry count, and open incident ID are refreshed.
10. A failure creates an incident. The service opens one incident per workflow and correlation ID. Repeated failure events update the existing incident instead of creating duplicates.
11. A critical incident alerts Slack. The assigned owner is mentioned in the restricted operations channel. High and standard incidents remain visible in operational queues unless escalation rules require a message.
12. A retry succeeds. A success event using the same correlation ID automatically resolves the related incident.
13. A watchdog checks freshness. Every five minutes, Apps Script compares the last event time with each active workflow’s stale threshold.
14. An overdue workflow creates a stale incident. Critical stale incidents alert the owner even though the source platform did not report an explicit failure.
15. An independent n8n sentinel checks the monitor. If the central Apps Script health endpoint is unavailable, n8n posts directly to the Slack fallback webhook.

The source receives a JSON response containing ok, duplicate, and any created incident_id. Apps Script Content Service responses do not provide the same status-code control as a conventional API framework, so connectors inspect the JSON ok property rather than relying only on the HTTP status.

Data Structure

Workflows table

Runs table

The Runs sheet is append-only for ordinary users. Each row represents one accepted event, not necessarily one unique business transaction. A workflow may generate RUNNING and SUCCESS events, while a retry may generate a second execution tied to the same correlation ID.

Incidents table

Incidents relate to Workflows through Workflow ID and to Runs through Correlation ID and Run ID. One workflow can have many historical incidents, but the design attempts to maintain only one open incident for the same workflow and correlation ID.

System Errors table

The System Errors sheet records endpoint validation problems, Slack delivery errors, malformed requests, AI failures, and internal script exceptions. Authentication tokens and unfiltered payloads are never written to this sheet.

Workflow Statuses and Ownership

A record moves backward when a retry fails, a recovered workflow becomes stale again, or an owner determines that the reported success did not complete the business transaction. Staff do not delete the previous incident. They reopen it or create a linked follow-up incident so the history remains visible.

Rejection is used for proposed recovery actions rather than for the run event itself. For example, a department owner may reject replaying an invoice synchronization because a manual accounting correction has already been posted. The incident is then closed with a documented reason.

Step-by-Step Implementation

Step 1: Prepare the Accounts and Permissions

1. Create a Google Sheets workbook named Automation Reliability Register in a restricted IT shared drive or controlled folder.
2. Grant edit access only to the IT manager, systems analyst, and approved backup administrator.
3. Grant read-only access to department managers only if the register contains no sensitive errors or restricted source links.
4. Create a spreadsheet-bound Apps Script project owned by a durable administrative account, not a departing employee’s personal account.
5. Create a private Slack channel such as #automation-operations.
6. Create a Slack incoming webhook for that channel and store the webhook URL in Apps Script Properties.
7. Collect the Slack member IDs for each workflow owner and escalation contact.
8. Confirm that Zapier can send outbound webhooks and expose an error-event trigger appropriate to the account.
9. Confirm that Make scenarios can use HTTP modules and error-handling routes.
10. Confirm that n8n credentials permit HTTP Request nodes and shared error workflows.
11. Confirm that selected Power Automate flows can use an outbound HTTP action or an approved equivalent connector. Required licensing varies, so verify the current tenant entitlements.
12. Create test workflows in each platform. Do not modify production workflows until event validation and duplicate handling have passed.

Create five independent random source tokens, one for each platform. Use a password manager or cryptographically secure secret generator. Do not place tokens in the workbook.

{
  "ZAPIER": "REPLACE_WITH_A_LONG_RANDOM_ZAPIER_TOKEN",
  "MAKE": "REPLACE_WITH_A_LONG_RANDOM_MAKE_TOKEN",
  "N8N": "REPLACE_WITH_A_LONG_RANDOM_N8N_TOKEN",
  "GOOGLE_APPS_SCRIPT": "REPLACE_WITH_A_LONG_RANDOM_APPS_SCRIPT_TOKEN",
  "POWER_AUTOMATE": "REPLACE_WITH_A_LONG_RANDOM_POWER_AUTOMATE_TOKEN"
}

Store this JSON as the Apps Script property API_TOKENS_JSON. Store the Slack incoming webhook as SLACK_WEBHOOK_URL. Store a separate random value as HEALTH_TOKEN.

The deployed web application must accept HTTPS requests from the external automation platforms. Depending on the Google Workspace configuration, this may require a deployment that permits unauthenticated access to the endpoint. The application still authenticates every event with a source-specific secret. An organization that prohibits public Apps Script deployments should place the endpoint behind an approved API gateway or use a managed API service instead.

Step 2: Build the Intake

There are two intake paths:

• Workflow registration: An IT administrator adds an approved workflow definition to the protected Workflows sheet.
• Run-event intake: Automation platforms post JSON to the Apps Script web application.

A representative event is:

{
  "token": "REPLACE_WITH_PLATFORM_TOKEN",
  "event_id": "N8N-84721-FAILED",
  "workflow_id": "WF-OPS-DOC-006",
  "platform": "N8N",
  "run_id": "84721",
  "correlation_id": "service-order-10584",
  "status": "FAILED",
  "started_at": "2026-07-15T14:21:58Z",
  "ended_at": "2026-07-15T14:22:08Z",
  "duration_ms": 10000,
  "retry_count": 1,
  "error_code": "HTTP_429",
  "error_message": "Destination API rate limit exceeded",
  "source_run_url": "https://YOUR_N8N_HOST/execution/84721",
  "metadata": {
    "environment": "production",
    "record_type": "service_order"
  }
}

The response does not echo the token or original payload:

{
  "ok": true,
  "duplicate": false,
  "event_id": "N8N-84721-FAILED",
  "incident_id": "INC-20260715-7A8B2C1D"
}

Incomplete or invalid submissions return ok: false and a bounded error message. They are also written to System Errors when the workbook remains available.

Step 3: Create the System of Record

The setup script creates four principal sheets:

• Workflows: One row per registered automation.
• Runs: Append-only normalized events.
• Incidents: Failure, timeout, and stale-workflow records.
• System Errors: Intake, notification, script, and optional AI failures.

Workflow IDs follow a stable naming convention:

WF-[DEPARTMENT]-[PROCESS]-[NUMBER]

Examples:
WF-FIN-AP-004
WF-OPS-DOC-006
WF-SALES-CRM-002
WF-IT-MONITOR-001

The workflow row is the authoritative source for ownership and criticality. Platform payloads cannot override those values. This prevents a malformed source request from changing who receives an alert.

The Runs sheet uses Event ID as its unique application-level key. Google Sheets does not provide a database unique constraint, so the script acquires a document lock and performs an exact search before appending.

Filtered views are created for:

• Active critical workflows
• Workflows with an open incident
• Workflows that have never reported
• Inactive workflows pending archive
• Incidents awaiting acknowledgement
• Incidents assigned to each owner
• Automation or notification failures

Quarterly archives keep the workbook responsive. Closed incidents and old run rows are exported to a date-labelled archive workbook before deletion from the active register.

Step 4: Connect the Tools

Zapier connection

1. Add a final webhook action to each monitored Zap for successful completion.
2. Use a custom POST request with a JSON body matching the central schema.
3. Store the workflow ID and source token as controlled configuration values rather than accepting them from untrusted input.
4. Create a separate monitoring Zap using the current Zapier-provided error event trigger available to the account.
5. Map the failed Zap identifier to the registered Workflow ID using a lookup table.
6. Post a FAILED event to the central endpoint.
7. Inspect the returned ok value. A false value must cause the monitoring Zap to fail visibly rather than appearing successful.

Zapier field labels and error-event fields can change by connector version. The required design is an error event containing the failed automation identity, execution identity, time, error text, and available run URL.

Make connection

1. Add an initial Tools module that establishes a run key and correlation ID.
2. Add a final HTTP module on the successful route.
3. Map a SUCCESS event into the request JSON.
4. Add error-handling routes to modules that can terminate the scenario.
5. Place the monitoring HTTP request before the error-handler directive.
6. Map the module error message, scenario run reference, retry count, and correlation ID.
7. Choose the final error directive according to the business process. For example, preserve an incomplete execution when a controlled retry is required.
8. Do not use a resume directive merely to make the scenario appear successful.

n8n connection

1. Add an HTTP Request node as the final successful node of each monitored workflow.
2. Create a shared error workflow beginning with the Error Trigger node.
3. Link monitored workflows to the shared error workflow.
4. Use a lookup node or controlled mapping to translate the n8n workflow ID into the central Workflow ID.
5. Post execution ID, workflow identity, error message, failed node, and execution URL to the register.
6. Use a separate HTTP Request node to post directly to the Slack fallback webhook if the register call fails for a critical monitoring event.

n8n documents the shared pattern in its error workflow guidance.

Google Apps Script connection

Each monitored Apps Script job calls a reusable wrapper. The wrapper posts SUCCESS after normal completion and FAILED after a caught exception. A time-driven heartbeat handles jobs where an execution event is not otherwise needed.

Because a syntax error, disabled trigger, authorization failure, or Google service outage can prevent the wrapper from running, the central stale-workflow scan remains essential.

Power Automate connection

1. Initialize Run ID, Correlation ID, Start Time, and Retry Count variables.
2. Place business actions inside a Try scope.
3. Add a Log Success scope configured to run after Try succeeds.
4. Add a Catch scope configured to run after Try fails or times out.
5. In each logging scope, send the normalized JSON to the register using an approved outbound HTTP action.
6. Inspect the returned ok property with a condition.
7. If the failure log cannot be delivered, post to the Slack fallback webhook or invoke the organization’s approved notification path.
8. After logging the original business failure, use a Terminate action so the flow remains visibly failed.

Run ID variable:
guid()

Start Time variable:
utcNow()

Success condition:
body('Post_monitoring_event')?['ok'] is equal to true

Action names in expressions must match the actual names assigned in the flow. Connector labels vary by tenant and product release.

Step 5: Build the Core Automation

Automation 1: Accept and record a terminal event

• Trigger: HTTPS POST to the Apps Script web application.
• Conditions: Valid JSON, valid source token, known active workflow, matching platform, allowed status, and unique event ID.
• Actions: Normalize values, find the workflow, append a run row, and update workflow health.
• Fields updated: Last Event At, Last Run ID, Last Status, Last Error, Last Retry Count, Open Incident ID, and Updated At.
• Notification: None for ordinary success events.
• Exception: Invalid requests return ok: false and create a bounded System Errors entry.

Automation 2: Create a failure incident

• Trigger: Accepted FAILED or TIMED_OUT event.
• Conditions: No existing open incident for the same workflow and correlation ID.
• Actions: Generate an incident ID, copy controlled ownership values, set OPEN, calculate escalation time, create the incident link, and connect the run row.
• Fields updated: Incident ID, Opened At, Status, owner fields, error details, Retry Count, Escalate At, Alert Status, and Updated At.
• Notification: CRITICAL incidents mention the owner in Slack. Other incidents enter the dashboard queue.
• Exception: Repeated events update the existing incident rather than opening another one.

Automation 3: Resolve a correlated incident

• Trigger: Accepted SUCCESS event.
• Conditions: An unresolved incident exists for the same Workflow ID and Correlation ID.
• Actions: Set the incident to RESOLVED, stamp Resolved At, add an automated resolution note, and clear the workflow’s matching open incident reference.
• Fields updated: Incident Status, Resolved At, Resolution Notes, Updated At, and workflow health fields.
• Notification: Optional recovery message for critical incidents.
• Exception: A success with a different correlation ID does not close an unrelated incident.

Automation 4: Detect stale workflows

• Trigger: Time-driven Apps Script trigger every five minutes.
• Conditions: Workflow is active, Stale After Minutes is configured, and Last Event At is older than the permitted threshold.
• Actions: Create one stale incident, set Last Status to STALE, and retain the existing last run information.
• Fields updated: Workflow status, last error, open incident, and a new incident row.
• Notification: Immediate Slack alert for critical workflows.
• Exception: Existing open stale incidents prevent duplicate incident creation.

Automation 5: Check the monitoring service

• Trigger: n8n schedule every five minutes.
• Conditions: Apps Script health response is absent, malformed, late, or returns ok: false.
• Actions: Retry the health request, then post directly to the Slack fallback webhook.
• Fields updated: No central fields when the service is unavailable.
• Notification: Slack message identifies the monitoring service as unavailable.
• Exception: The fallback alert includes a unique sentinel event ID so duplicate messages can be reconciled.

Step 6: Add Approvals, Reminders, and Escalations

Incident acknowledgement and recovery are operational approvals. The monitoring system detects a problem but does not decide whether replaying a business transaction is safe.

If the primary owner is unavailable, the IT manager changes the incident owner and records the reassignment in Resolution Notes. For predictable absences, the workflow inventory is updated before the absence begins.

Rejected retries are recorded in Resolution Notes. The incident may be resolved through manual correction, closed as an accepted known issue, or left open until a safe remediation is available.

The incident row, run row, source execution URL, acknowledgement timestamp, Slack alert state, and resolution note together form the approval evidence.

Step 7: Add Documents and File Management

The monitoring register does not copy business documents or full automation payloads. This reduces privacy exposure and avoids creating another uncontrolled file repository.

File management is limited to monitoring evidence:

• Source execution links point to authorized platform histories.
• Long diagnostic exports are stored in a restricted incident-evidence folder only when required.
• Evidence folders use Incident ID - Workflow ID as the name.
• Incident rows contain the folder link when evidence is collected.
• Screenshots and exports are not posted to Slack if they contain customer or employee information.
• Updated evidence replaces obsolete copies only when retention policy permits. Otherwise, date-labelled versions are retained.
• Failed evidence uploads are recorded in System Errors and do not prevent the incident itself from being created.

Large log files remain in the source platform or approved storage. The workbook stores only a short error message and a link.

Step 8: Add Reporting and Operational Views

The IT team created the following views:

• New incidents opened in the last 24 hours
• Critical incidents awaiting acknowledgement
• Open incidents older than their service target
• Workflows currently stale
• Incidents by owner
• Rejected or cancelled executions
• Retries awaiting confirmation
• Recently resolved incidents
• Slack alert failures
• Invalid monitoring requests
• Failure volume by platform and workflow
• Median and average recovery time
• Manual-review queue

A live open-incident view can use a formula similar to:

=FILTER(
  Incidents!A2:AD,
  Incidents!A2:A<>"",
  Incidents!I2:I<>"RESOLVED",
  Incidents!I2:I<>"CLOSED"
)

Incident age in hours is calculated as:

=IF(B2="","",(NOW()-B2)*24)

Recovery time in minutes is calculated as:

=IF(OR(B2="",R2=""),"",ROUND((R2-B2)*24*60,1))

Pivot tables summarize incident counts by platform, criticality, owner, error code, week, and workflow. The source data updates immediately after an accepted event. Time-based formulas such as incident age refresh according to Google Sheets recalculation behavior.

The systems analyst owns the daily operational view. The IT manager owns the monthly trend report and investigates any workflow with repeated failures, increasing duration, or unresolved ownership.

Step 9: Add Security and Governance Controls

• Limit workbook editors to the operational support team.
• Protect the Runs sheet against ordinary edits and deletions.
• Use a durable administrative owner for the workbook and script deployment.
• Store API tokens and webhook URLs in Apps Script Properties, not cells.
• Use a different source token for each platform.
• Rotate a single source token without changing the other integrations.
• Restrict Slack alerts to a private operations channel.
• Do not include credentials, customer documents, payment details, health information, or full message bodies in monitoring events.
• Truncate error messages before storage and Slack delivery.
• Limit source execution links to HTTPS.
• Review shared-drive access and Slack membership when employees leave.
• Retain run data according to operational and regulatory requirements.
• Export archives before removing old rows.
• Review Apps Script executions and trigger ownership monthly.
• Keep AI disabled for sensitive incident categories unless an approved data-processing agreement and policy permit it.
• Require human approval before replaying high-impact transactions.

The shared token pattern is appropriate only for controlled machine-to-machine event intake at this scale. Organizations requiring IP restrictions, signed requests, formal service identities, or centralized secret rotation should use an API gateway and managed secret service.

Step 10: Deploy and Test

1. Build the workbook and script in a non-production Google Workspace folder.
2. Run the setup function and verify all sheets, headers, validation rules, and triggers.
3. Deploy the script as a test web application.
4. Send valid and invalid sample requests from an API client or one test automation.
5. Confirm duplicate events do not create duplicate rows.
6. Register one test workflow for each platform.
7. Test success, failure, timeout, malformed payload, retry, and recovery events.
8. Temporarily set a test workflow’s stale threshold to ten minutes and verify stale detection.
9. Disable the central deployment temporarily and confirm the n8n sentinel posts directly to Slack.
10. Conduct user acceptance testing with the IT manager, systems analyst, and one department workflow owner.
11. Pilot five low-risk workflows for one week.
12. Compare source-platform histories with the central Runs sheet daily.
13. Add critical workflows only after event completeness is confirmed.
14. Retain the old platform email alerts during the pilot.
15. Document workflow IDs, source mappings, owners, retry procedures, and rollback steps.
16. Activate remaining workflows in groups by platform.

Rollback consists of disabling the source logging steps and the Apps Script time-driven trigger while leaving the existing platform automations unchanged. The monitoring components are designed not to be required for the underlying business workflow to complete.

Code and Configuration

Central Google Apps Script service

The following script belongs in the Apps Script project bound to the Automation Reliability Register workbook.

Open the workbook, open the Apps Script editor from the Extensions menu, replace the default editor contents, and paste the complete code below. Replace YOUR_SPREADSHEET_ID with the workbook ID from its URL.

The script requests permission to read and update the workbook, manage triggers, and make external HTTPS requests to Slack. The web application runs when a source posts an event. The reliability monitor runs every five minutes. The incident edit handler runs when an authorized user changes an incident status.

const CONFIG = Object.freeze({
  SPREADSHEET_ID: 'YOUR_SPREADSHEET_ID',
  VERSION: '1.0.0',
  SHEETS: {
    WORKFLOWS: 'Workflows',
    RUNS: 'Runs',
    INCIDENTS: 'Incidents',
    ERRORS: 'System Errors'
  },
  PLATFORMS: [
    'ZAPIER',
    'MAKE',
    'N8N',
    'GOOGLE_APPS_SCRIPT',
    'POWER_AUTOMATE'
  ],
  EVENT_STATUSES: [
    'RUNNING',
    'SUCCESS',
    'FAILED',
    'TIMED_OUT',
    'CANCELLED',
    'HEARTBEAT'
  ],
  INCIDENT_STATUSES: [
    'OPEN',
    'ACKNOWLEDGED',
    'RETRYING',
    'ESCALATED',
    'RESOLVED',
    'CLOSED'
  ],
  CRITICALITIES: ['CRITICAL', 'HIGH', 'STANDARD'],
  MAX_ERROR_LENGTH: 2000
});

const HEADERS = Object.freeze({
  WORKFLOWS: [
    'Workflow ID',
    'Workflow Name',
    'Platform',
    'Criticality',
    'Owner Name',
    'Owner Email',
    'Owner Slack ID',
    'Escalation Slack ID',
    'Expected Frequency Minutes',
    'Stale After Minutes',
    'Active',
    'Last Event At',
    'Last Run ID',
    'Last Status',
    'Last Error',
    'Last Retry Count',
    'Open Incident ID',
    'Updated At'
  ],
  RUNS: [
    'Event ID',
    'Received At',
    'Workflow ID',
    'Workflow Name',
    'Platform',
    'Run ID',
    'Correlation ID',
    'Status',
    'Started At',
    'Ended At',
    'Duration Milliseconds',
    'Retry Count',
    'Error Code',
    'Error Message',
    'Source Run URL',
    'Incident ID',
    'Payload JSON'
  ],
  INCIDENTS: [
    'Incident ID',
    'Opened At',
    'Workflow ID',
    'Workflow Name',
    'Platform',
    'Criticality',
    'Correlation ID',
    'Run ID',
    'Status',
    'Owner Name',
    'Owner Email',
    'Owner Slack ID',
    'Error Code',
    'Error Message',
    'Source Run URL',
    'Retry Count',
    'Acknowledged At',
    'Resolved At',
    'Escalate At',
    'Last Alert At',
    'Alert Count',
    'Alert Status',
    'Incident Link',
    'Resolution Notes',
    'AI Category',
    'AI Summary',
    'AI Suggested Action',
    'AI Confidence',
    'AI Review Status',
    'Updated At'
  ],
  ERRORS: [
    'Occurred At',
    'Component',
    'Severity',
    'Event ID',
    'Workflow ID',
    'Message',
    'Sanitized Context'
  ]
});

function setupMonitoringWorkbook() {
  const ss = getSpreadsheet_();

  const workflows = ensureSheet_(
    ss,
    CONFIG.SHEETS.WORKFLOWS,
    HEADERS.WORKFLOWS
  );
  ensureSheet_(ss, CONFIG.SHEETS.RUNS, HEADERS.RUNS);
  const incidents = ensureSheet_(
    ss,
    CONFIG.SHEETS.INCIDENTS,
    HEADERS.INCIDENTS
  );
  ensureSheet_(ss, CONFIG.SHEETS.ERRORS, HEADERS.ERRORS);

  const workflowRows = Math.max(workflows.getMaxRows() - 1, 1);
  const incidentRows = Math.max(incidents.getMaxRows() - 1, 1);

  workflows.getRange(2, 3, workflowRows, 1).setDataValidation(
    SpreadsheetApp.newDataValidation()
      .requireValueInList(CONFIG.PLATFORMS, true)
      .setAllowInvalid(false)
      .build()
  );

  workflows.getRange(2, 4, workflowRows, 1).setDataValidation(
    SpreadsheetApp.newDataValidation()
      .requireValueInList(CONFIG.CRITICALITIES, true)
      .setAllowInvalid(false)
      .build()
  );

  workflows.getRange(2, 11, workflowRows, 1).insertCheckboxes();

  incidents.getRange(2, 9, incidentRows, 1).setDataValidation(
    SpreadsheetApp.newDataValidation()
      .requireValueInList(CONFIG.INCIDENT_STATUSES, true)
      .setAllowInvalid(false)
      .build()
  );

  installTriggerIfMissing_('monitorReliability', 'time');
  installTriggerIfMissing_('onIncidentEdit', 'edit');

  console.log('Monitoring workbook setup completed.');
}

function doGet(e) {
  try {
    const expected = getRequiredProperty_('HEALTH_TOKEN');
    const supplied = e && e.parameter ? String(e.parameter.token || '') : '';

    if (!safeEquals_(supplied, expected)) {
      return jsonResponse_({
        ok: false,
        service: 'automation-reliability-monitor',
        error: 'Unauthorized health request'
      });
    }

    getSpreadsheet_().getSheetByName(CONFIG.SHEETS.WORKFLOWS)
      .getRange('A1')
      .getValue();

    return jsonResponse_({
      ok: true,
      service: 'automation-reliability-monitor',
      version: CONFIG.VERSION,
      checked_at: new Date().toISOString()
    });
  } catch (error) {
    return jsonResponse_({
      ok: false,
      service: 'automation-reliability-monitor',
      error: truncate_(error.message, 300)
    });
  }
}

function doPost(e) {
  let sanitizedContext = {};

  try {
    if (!e || !e.postData || !e.postData.contents) {
      throw new Error('Request body is required.');
    }

    const raw = e.postData.contents;
    if (raw.length > 50000) {
      throw new Error('Request body exceeds 50,000 characters.');
    }

    const input = JSON.parse(raw);
    sanitizedContext = sanitizePayload_(input);
    const event = validateEvent_(input);

    const lock = LockService.getScriptLock();
    lock.waitLock(25000);

    let result;
    try {
      result = recordEvent_(event, sanitizedContext);
    } finally {
      lock.releaseLock();
    }

    if (result.alert) {
      deliverIncidentAlert_(result.alert);
    }

    if (result.recoveryAlert) {
      deliverRecoveryAlert_(result.recoveryAlert);
    }

    return jsonResponse_({
      ok: true,
      duplicate: result.duplicate,
      event_id: event.event_id,
      incident_id: result.incidentId || ''
    });
  } catch (error) {
    try {
      recordSystemError_(
        'EVENT_INTAKE',
        'ERROR',
        sanitizedContext.event_id || '',
        sanitizedContext.workflow_id || '',
        error.message,
        sanitizedContext
      );
    } catch (loggingError) {
      console.error('Unable to record intake error: ' + loggingError.message);
    }

    return jsonResponse_({
      ok: false,
      error: truncate_(error.message, 500)
    });
  }
}

function validateEvent_(input) {
  if (!input || typeof input !== 'object' || Array.isArray(input)) {
    throw new Error('JSON body must be an object.');
  }

  const platform = requiredString_(input.platform, 'platform').toUpperCase();
  if (CONFIG.PLATFORMS.indexOf(platform) === -1) {
    throw new Error('Unsupported platform.');
  }

  const tokens = JSON.parse(getRequiredProperty_('API_TOKENS_JSON'));
  const expectedToken = String(tokens[platform] || '');
  const suppliedToken = requiredString_(input.token, 'token');

  if (!expectedToken || !safeEquals_(suppliedToken, expectedToken)) {
    throw new Error('Source authentication failed.');
  }

  const eventId = requiredString_(input.event_id, 'event_id');
  const idPattern = /^[A-Za-z0-9._:-]{8,200}$/;
  if (!idPattern.test(eventId)) {
    throw new Error('event_id has an invalid format.');
  }

  const workflowId = requiredString_(input.workflow_id, 'workflow_id');
  if (!/^[A-Za-z0-9._:-]{3,100}$/.test(workflowId)) {
    throw new Error('workflow_id has an invalid format.');
  }

  const runId = requiredString_(input.run_id, 'run_id');
  const correlationId = requiredString_(
    input.correlation_id,
    'correlation_id'
  );
  const status = requiredString_(input.status, 'status').toUpperCase();

  if (CONFIG.EVENT_STATUSES.indexOf(status) === -1) {
    throw new Error('Unsupported event status.');
  }

  const retryCount = Number(input.retry_count || 0);
  if (!Number.isInteger(retryCount) || retryCount < 0) {
    throw new Error('retry_count must be a non-negative integer.');
  }

  let duration = '';
  if (input.duration_ms !== undefined && input.duration_ms !== '') {
    duration = Number(input.duration_ms);
    if (!Number.isFinite(duration) || duration < 0) {
      throw new Error('duration_ms must be a non-negative number.');
    }
  }

  const errorMessage = truncate_(
    String(input.error_message || ''),
    CONFIG.MAX_ERROR_LENGTH
  );

  if (
    (status === 'FAILED' || status === 'TIMED_OUT') &&
    !errorMessage
  ) {
    throw new Error('error_message is required for failure events.');
  }

  const sourceUrl = String(input.source_run_url || '').trim();
  if (sourceUrl && !/^https:\/\//i.test(sourceUrl)) {
    throw new Error('source_run_url must use HTTPS.');
  }

  return {
    event_id: eventId,
    workflow_id: workflowId,
    platform: platform,
    run_id: truncate_(runId, 200),
    correlation_id: truncate_(correlationId, 200),
    status: status,
    started_at: parseOptionalDate_(input.started_at, 'started_at'),
    ended_at: parseOptionalDate_(input.ended_at, 'ended_at'),
    duration_ms: duration,
    retry_count: retryCount,
    error_code: truncate_(String(input.error_code || ''), 100),
    error_message: errorMessage,
    source_run_url: truncate_(sourceUrl, 1000),
    metadata: input.metadata && typeof input.metadata === 'object'
      ? input.metadata
      : {}
  };
}

function recordEvent_(event, sanitizedPayload) {
  const ss = getSpreadsheet_();
  const runs = ss.getSheetByName(CONFIG.SHEETS.RUNS);
  const workflows = ss.getSheetByName(CONFIG.SHEETS.WORKFLOWS);
  const incidents = ss.getSheetByName(CONFIG.SHEETS.INCIDENTS);

  const duplicateRow = findRowByExactValue_(runs, 1, event.event_id);
  if (duplicateRow) {
    return {
      duplicate: true,
      incidentId: runs.getRange(duplicateRow, 16).getValue() || '',
      alert: null,
      recoveryAlert: null
    };
  }

  const workflowRow = findRowByExactValue_(
    workflows,
    1,
    event.workflow_id
  );

  if (!workflowRow) {
    throw new Error('workflow_id is not registered.');
  }

  const workflowValues = workflows
    .getRange(workflowRow, 1, 1, HEADERS.WORKFLOWS.length)
    .getValues()[0];

  const workflow = workflowFromRow_(workflowRow, workflowValues);

  if (!workflow.active) {
    throw new Error('Workflow is inactive.');
  }

  if (workflow.platform !== event.platform) {
    throw new Error('Event platform does not match workflow registration.');
  }

  let incidentId = '';
  let alert = null;
  let recoveryAlert = null;
  let resolvedIncidentId = '';

  if (event.status === 'FAILED' || event.status === 'TIMED_OUT') {
    const incidentResult = createOrUpdateFailureIncident_(
      ss,
      incidents,
      workflow,
      event
    );
    incidentId = incidentResult.incidentId;
    alert = incidentResult.alert;
  } else if (event.status === 'SUCCESS') {
    const resolution = resolveCorrelatedIncident_(
      incidents,
      workflow,
      event
    );
    resolvedIncidentId = resolution.incidentId;
    recoveryAlert = resolution.alert;
  }

  if (event.status === 'SUCCESS' || event.status === 'HEARTBEAT') {
    const staleResolution = resolveStaleIncident_(
      incidents,
      workflow,
      event
    );

    if (staleResolution.incidentId) {
      resolvedIncidentId = staleResolution.incidentId;
      recoveryAlert = staleResolution.alert;
    }
  }

  runs.appendRow([
    event.event_id,
    new Date(),
    workflow.workflowId,
    workflow.workflowName,
    workflow.platform,
    event.run_id,
    event.correlation_id,
    event.status,
    event.started_at,
    event.ended_at,
    event.duration_ms,
    event.retry_count,
    event.error_code,
    event.error_message,
    event.source_run_url,
    incidentId || resolvedIncidentId,
    truncate_(JSON.stringify(sanitizedPayload), 10000)
  ]);

  let openIncidentId = workflow.openIncidentId;
  if (incidentId) {
    openIncidentId = incidentId;
  }
  if (resolvedIncidentId && openIncidentId === resolvedIncidentId) {
    openIncidentId = '';
  }

  const updatedWorkflow = workflowValues.slice();
  updatedWorkflow[11] = new Date();
  updatedWorkflow[12] = event.run_id;
  updatedWorkflow[13] = event.status;
  updatedWorkflow[14] = event.error_message;
  updatedWorkflow[15] = event.retry_count;
  updatedWorkflow[16] = openIncidentId;
  updatedWorkflow[17] = new Date();

  workflows
    .getRange(workflowRow, 1, 1, updatedWorkflow.length)
    .setValues([updatedWorkflow]);

  return {
    duplicate: false,
    incidentId: incidentId || resolvedIncidentId,
    alert: alert,
    recoveryAlert: recoveryAlert
  };
}

function createOrUpdateFailureIncident_(ss, sheet, workflow, event) {
  const values = sheet.getDataRange().getValues();

  for (let i = values.length - 1; i >= 1; i--) {
    const row = values[i];
    const sameWorkflow = String(row[2]) === workflow.workflowId;
    const sameCorrelation = String(row[6]) === event.correlation_id;
    const isOpen = ['RESOLVED', 'CLOSED'].indexOf(String(row[8])) === -1;

    if (sameWorkflow && sameCorrelation && isOpen) {
      sheet.getRange(i + 1, 8).setValue(event.run_id);
      sheet.getRange(i + 1, 13).setValue(event.error_code);
      sheet.getRange(i + 1, 14).setValue(event.error_message);
      sheet.getRange(i + 1, 15).setValue(event.source_run_url);
      sheet.getRange(i + 1, 16).setValue(event.retry_count);
      sheet.getRange(i + 1, 30).setValue(new Date());

      return {
        incidentId: String(row[0]),
        alert: null
      };
    }
  }

  const incidentId = createIncidentId_();
  const openedAt = new Date();
  const escalationMinutes = escalationMinutes_(workflow.criticality);
  const escalateAt = new Date(
    openedAt.getTime() + escalationMinutes * 60000
  );

  sheet.appendRow([
    incidentId,
    openedAt,
    workflow.workflowId,
    workflow.workflowName,
    workflow.platform,
    workflow.criticality,
    event.correlation_id,
    event.run_id,
    'OPEN',
    workflow.ownerName,
    workflow.ownerEmail,
    workflow.ownerSlackId,
    event.error_code,
    event.error_message,
    event.source_run_url,
    event.retry_count,
    '',
    '',
    escalateAt,
    '',
    0,
    workflow.criticality === 'CRITICAL'
      ? 'PENDING'
      : 'DASHBOARD_ONLY',
    '',
    '',
    '',
    '',
    '',
    '',
    '',
    openedAt
  ]);

  const rowNumber = sheet.getLastRow();
  const incidentLink =
    ss.getUrl() +
    '#gid=' +
    sheet.getSheetId() +
    '&range=A' +
    rowNumber;

  sheet.getRange(rowNumber, 23).setValue(incidentLink);

  const alert = workflow.criticality === 'CRITICAL'
    ? {
        incidentId: incidentId,
        workflowId: workflow.workflowId,
        workflowName: workflow.workflowName,
        platform: workflow.platform,
        criticality: workflow.criticality,
        ownerSlackId: workflow.ownerSlackId,
        escalationSlackId: workflow.escalationSlackId,
        errorMessage: event.error_message,
        sourceRunUrl: event.source_run_url,
        incidentLink: incidentLink
      }
    : null;

  return {
    incidentId: incidentId,
    alert: alert
  };
}

function resolveCorrelatedIncident_(sheet, workflow, event) {
  const values = sheet.getDataRange().getValues();

  for (let i = values.length - 1; i >= 1; i--) {
    const row = values[i];
    const sameWorkflow = String(row[2]) === workflow.workflowId;
    const sameCorrelation = String(row[6]) === event.correlation_id;
    const isOpen = ['RESOLVED', 'CLOSED'].indexOf(String(row[8])) === -1;

    if (sameWorkflow && sameCorrelation && isOpen) {
      const resolvedAt = new Date();
      sheet.getRange(i + 1, 9).setValue('RESOLVED');
      sheet.getRange(i + 1, 18).setValue(resolvedAt);
      sheet.getRange(i + 1, 24).setValue(
        'Resolved automatically by successful event ' + event.event_id
      );
      sheet.getRange(i + 1, 30).setValue(resolvedAt);

      return {
        incidentId: String(row[0]),
        alert: workflow.criticality === 'CRITICAL'
          ? recoveryAlertFromRow_(row, event)
          : null
      };
    }
  }

  return { incidentId: '', alert: null };
}

function resolveStaleIncident_(sheet, workflow, event) {
  const values = sheet.getDataRange().getValues();

  for (let i = values.length - 1; i >= 1; i--) {
    const row = values[i];
    const sameWorkflow = String(row[2]) === workflow.workflowId;
    const isStale = String(row[12]) === 'STALE_HEARTBEAT';
    const isOpen = ['RESOLVED', 'CLOSED'].indexOf(String(row[8])) === -1;

    if (sameWorkflow && isStale && isOpen) {
      const resolvedAt = new Date();
      sheet.getRange(i + 1, 9).setValue('RESOLVED');
      sheet.getRange(i + 1, 18).setValue(resolvedAt);
      sheet.getRange(i + 1, 24).setValue(
        'Resolved automatically after event ' + event.event_id
      );
      sheet.getRange(i + 1, 30).setValue(resolvedAt);

      return {
        incidentId: String(row[0]),
        alert: workflow.criticality === 'CRITICAL'
          ? recoveryAlertFromRow_(row, event)
          : null
      };
    }
  }

  return { incidentId: '', alert: null };
}

function monitorReliability() {
  const lock = LockService.getScriptLock();
  lock.waitLock(25000);

  let newAlerts = [];
  let escalations = [];

  try {
    const ss = getSpreadsheet_();
    const workflows = ss.getSheetByName(CONFIG.SHEETS.WORKFLOWS);
    const incidents = ss.getSheetByName(CONFIG.SHEETS.INCIDENTS);
    const workflowValues = workflows.getDataRange().getValues();
    const now = new Date();

    for (let i = 1; i < workflowValues.length; i++) {
      const workflow = workflowFromRow_(i + 1, workflowValues[i]);

      if (!workflow.active || !workflow.staleAfterMinutes) {
        continue;
      }

      const referenceDate = asDate_(
        workflow.lastEventAt || workflow.updatedAt
      );

      if (!referenceDate) {
        continue;
      }

      const ageMinutes =
        (now.getTime() - referenceDate.getTime()) / 60000;

      if (ageMinutes <= workflow.staleAfterMinutes) {
        continue;
      }

      if (hasOpenStaleIncident_(incidents, workflow.workflowId)) {
        continue;
      }

      const staleEvent = {
        correlation_id: 'STALE',
        run_id: workflow.lastRunId || 'NO_RECENT_RUN',
        error_code: 'STALE_HEARTBEAT',
        error_message:
          'No event received for ' +
          Math.floor(ageMinutes) +
          ' minutes. Stale threshold is ' +
          workflow.staleAfterMinutes +
          ' minutes.',
        source_run_url: '',
        retry_count: 0
      };

      const result = createOrUpdateFailureIncident_(
        ss,
        incidents,
        workflow,
        staleEvent
      );

      workflows.getRange(i + 1, 14).setValue('STALE');
      workflows.getRange(i + 1, 15).setValue(staleEvent.error_message);
      workflows.getRange(i + 1, 17).setValue(result.incidentId);
      workflows.getRange(i + 1, 18).setValue(now);

      if (result.alert) {
        newAlerts.push(result.alert);
      }
    }

    escalations = collectDueEscalations_(incidents, now);
  } finally {
    lock.releaseLock();
  }

  newAlerts.forEach(function(alert) {
    deliverIncidentAlert_(alert);
  });

  escalations.forEach(function(alert) {
    deliverEscalationAlert_(alert);
  });
}

function collectDueEscalations_(sheet, now) {
  const values = sheet.getDataRange().getValues();
  const results = [];

  for (let i = 1; i < values.length; i++) {
    const row = values[i];
    const criticality = String(row[5]);
    const status = String(row[8]);
    const escalateAt = asDate_(row[18]);

    if (
      criticality !== 'CRITICAL' ||
      status !== 'OPEN' ||
      !escalateAt ||
      now.getTime() < escalateAt.getTime()
    ) {
      continue;
    }

    results.push({
      incidentId: String(row[0]),
      workflowId: String(row[2]),
      workflowName: String(row[3]),
      ownerSlackId: String(row[11]),
      escalationSlackId: findWorkflowEscalationSlackId_(
        String(row[2])
      ),
      errorMessage: String(row[13]),
      sourceRunUrl: String(row[14]),
      incidentLink: String(row[22])
    });
  }

  return results;
}

function deliverIncidentAlert_(alert) {
  const ownerMention = alert.ownerSlackId
    ? '<@' + alert.ownerSlackId + '>'
    : 'Unassigned owner';

  const text = [
    ':rotating_light: CRITICAL automation failure',
    'Workflow: ' + alert.workflowName + ' (' + alert.workflowId + ')',
    'Platform: ' + alert.platform,
    'Owner: ' + ownerMention,
    'Error: ' + truncate_(alert.errorMessage, 500),
    alert.sourceRunUrl ? 'Source run: ' + alert.sourceRunUrl : '',
    'Incident: ' + alert.incidentLink
  ].filter(Boolean).join('\n');

  try {
    sendSlack_(text);
    updateIncidentAlertState_(alert.incidentId, 'SENT', false);
  } catch (error) {
    updateIncidentAlertState_(alert.incidentId, 'FAILED', false);
    recordSystemError_(
      'SLACK_ALERT',
      'ERROR',
      '',
      alert.workflowId,
      error.message,
      { incident_id: alert.incidentId }
    );
  }
}

function deliverEscalationAlert_(alert) {
  const ownerMention = alert.ownerSlackId
    ? '<@' + alert.ownerSlackId + '>'
    : 'Unassigned owner';
  const escalationMention = alert.escalationSlackId
    ? '<@' + alert.escalationSlackId + '>'
    : 'No escalation contact configured';

  const text = [
    ':warning: Unacknowledged critical automation incident',
    'Workflow: ' + alert.workflowName + ' (' + alert.workflowId + ')',
    'Owner: ' + ownerMention,
    'Escalation: ' + escalationMention,
    'Error: ' + truncate_(alert.errorMessage, 500),
    alert.sourceRunUrl ? 'Source run: ' + alert.sourceRunUrl : '',
    'Incident: ' + alert.incidentLink
  ].filter(Boolean).join('\n');

  try {
    sendSlack_(text);
    updateIncidentAlertState_(alert.incidentId, 'ESCALATED', true);
  } catch (error) {
    updateIncidentAlertState_(alert.incidentId, 'FAILED', false);
    recordSystemError_(
      'SLACK_ESCALATION',
      'ERROR',
      '',
      alert.workflowId,
      error.message,
      { incident_id: alert.incidentId }
    );
  }
}

function deliverRecoveryAlert_(alert) {
  const text = [
    ':white_check_mark: Critical automation recovered',
    'Workflow: ' + alert.workflowName + ' (' + alert.workflowId + ')',
    'Successful event: ' + alert.eventId,
    'Incident: ' + alert.incidentLink
  ].join('\n');

  try {
    sendSlack_(text);
  } catch (error) {
    recordSystemError_(
      'SLACK_RECOVERY',
      'WARNING',
      alert.eventId,
      alert.workflowId,
      error.message,
      { incident_id: alert.incidentId }
    );
  }
}

function sendSlack_(text) {
  const url = getRequiredProperty_('SLACK_WEBHOOK_URL');
  let lastError;

  for (let attempt = 1; attempt <= 3; attempt++) {
    try {
      const response = UrlFetchApp.fetch(url, {
        method: 'post',
        contentType: 'application/json',
        payload: JSON.stringify({ text: text }),
        muteHttpExceptions: true
      });

      const status = response.getResponseCode();
      if (status >= 200 && status < 300) {
        return;
      }

      lastError = new Error(
        'Slack returned HTTP ' +
        status +
        ': ' +
        truncate_(response.getContentText(), 300)
      );

      if (status !== 429 && status < 500) {
        break;
      }
    } catch (error) {
      lastError = error;
    }

    Utilities.sleep(attempt * 1000);
  }

  throw lastError || new Error('Slack notification failed.');
}

function updateIncidentAlertState_(incidentId, state, escalated) {
  const sheet = getSpreadsheet_()
    .getSheetByName(CONFIG.SHEETS.INCIDENTS);
  const row = findRowByExactValue_(sheet, 1, incidentId);

  if (!row) {
    return;
  }

  const alertCount = Number(sheet.getRange(row, 21).getValue() || 0);
  sheet.getRange(row, 20).setValue(new Date());
  sheet.getRange(row, 21).setValue(alertCount + 1);
  sheet.getRange(row, 22).setValue(state);
  sheet.getRange(row, 30).setValue(new Date());

  if (escalated) {
    sheet.getRange(row, 9).setValue('ESCALATED');
  }
}

function onIncidentEdit(e) {
  if (!e || !e.range) {
    return;
  }

  const sheet = e.range.getSheet();
  if (
    sheet.getName() !== CONFIG.SHEETS.INCIDENTS ||
    e.range.getRow() < 2 ||
    e.range.getColumn() !== 9
  ) {
    return;
  }

  const status = String(e.value || '').toUpperCase();
  if (CONFIG.INCIDENT_STATUSES.indexOf(status) === -1) {
    return;
  }

  const row = e.range.getRow();
  const now = new Date();

  if (status === 'ACKNOWLEDGED' || status === 'RETRYING') {
    if (!sheet.getRange(row, 17).getValue()) {
      sheet.getRange(row, 17).setValue(now);
    }
  }

  if (status === 'RESOLVED' || status === 'CLOSED') {
    if (!sheet.getRange(row, 18).getValue()) {
      sheet.getRange(row, 18).setValue(now);
    }
  }

  sheet.getRange(row, 30).setValue(now);
}

function hasOpenStaleIncident_(sheet, workflowId) {
  const values = sheet.getDataRange().getValues();

  for (let i = values.length - 1; i >= 1; i--) {
    const row = values[i];
    const sameWorkflow = String(row[2]) === workflowId;
    const stale = String(row[12]) === 'STALE_HEARTBEAT';
    const open = ['RESOLVED', 'CLOSED'].indexOf(String(row[8])) === -1;

    if (sameWorkflow && stale && open) {
      return true;
    }
  }

  return false;
}

function workflowFromRow_(rowNumber, row) {
  return {
    rowNumber: rowNumber,
    workflowId: String(row[0]),
    workflowName: String(row[1]),
    platform: String(row[2]).toUpperCase(),
    criticality: String(row[3]).toUpperCase(),
    ownerName: String(row[4]),
    ownerEmail: String(row[5]),
    ownerSlackId: String(row[6]),
    escalationSlackId: String(row[7]),
    expectedFrequencyMinutes: Number(row[8] || 0),
    staleAfterMinutes: Number(row[9] || 0),
    active: row[10] === true || String(row[10]).toUpperCase() === 'TRUE',
    lastEventAt: row[11],
    lastRunId: String(row[12] || ''),
    lastStatus: String(row[13] || ''),
    lastError: String(row[14] || ''),
    lastRetryCount: Number(row[15] || 0),
    openIncidentId: String(row[16] || ''),
    updatedAt: row[17]
  };
}

function recoveryAlertFromRow_(row, event) {
  return {
    incidentId: String(row[0]),
    workflowId: String(row[2]),
    workflowName: String(row[3]),
    incidentLink: String(row[22]),
    eventId: event.event_id
  };
}

function findWorkflowEscalationSlackId_(workflowId) {
  const sheet = getSpreadsheet_()
    .getSheetByName(CONFIG.SHEETS.WORKFLOWS);
  const row = findRowByExactValue_(sheet, 1, workflowId);
  return row ? String(sheet.getRange(row, 8).getValue() || '') : '';
}

function escalationMinutes_(criticality) {
  if (criticality === 'CRITICAL') {
    return 15;
  }
  if (criticality === 'HIGH') {
    return 60;
  }
  return 240;
}

function createIncidentId_() {
  const date = Utilities.formatDate(
    new Date(),
    'UTC',
    'yyyyMMdd'
  );
  const suffix = Utilities.getUuid()
    .replace(/-/g, '')
    .substring(0, 8)
    .toUpperCase();

  return 'INC-' + date + '-' + suffix;
}

function sanitizePayload_(input) {
  if (!input || typeof input !== 'object') {
    return {};
  }

  return {
    event_id: truncate_(String(input.event_id || ''), 200),
    workflow_id: truncate_(String(input.workflow_id || ''), 100),
    platform: truncate_(String(input.platform || ''), 50),
    run_id: truncate_(String(input.run_id || ''), 200),
    correlation_id: truncate_(String(input.correlation_id || ''), 200),
    status: truncate_(String(input.status || ''), 50),
    started_at: truncate_(String(input.started_at || ''), 100),
    ended_at: truncate_(String(input.ended_at || ''), 100),
    duration_ms: input.duration_ms || '',
    retry_count: input.retry_count || 0,
    error_code: truncate_(String(input.error_code || ''), 100),
    error_message: truncate_(
      String(input.error_message || ''),
      CONFIG.MAX_ERROR_LENGTH
    ),
    source_run_url: truncate_(
      String(input.source_run_url || ''),
      1000
    ),
    metadata: sanitizeMetadata_(input.metadata)
  };
}

function sanitizeMetadata_(metadata) {
  if (!metadata || typeof metadata !== 'object' || Array.isArray(metadata)) {
    return {};
  }

  const blocked = [
    'token',
    'password',
    'secret',
    'authorization',
    'api_key',
    'apikey',
    'document',
    'body'
  ];
  const result = {};

  Object.keys(metadata).slice(0, 20).forEach(function(key) {
    if (blocked.indexOf(String(key).toLowerCase()) !== -1) {
      return;
    }

    const value = metadata[key];
    if (
      typeof value === 'string' ||
      typeof value === 'number' ||
      typeof value === 'boolean'
    ) {
      result[key] = truncate_(String(value), 500);
    }
  });

  return result;
}

function recordSystemError_(
  component,
  severity,
  eventId,
  workflowId,
  message,
  context
) {
  const sheet = getSpreadsheet_()
    .getSheetByName(CONFIG.SHEETS.ERRORS);

  sheet.appendRow([
    new Date(),
    component,
    severity,
    eventId,
    workflowId,
    truncate_(String(message || ''), 1000),
    truncate_(JSON.stringify(context || {}), 5000)
  ]);
}

function ensureSheet_(ss, name, headers) {
  let sheet = ss.getSheetByName(name);
  if (!sheet) {
    sheet = ss.insertSheet(name);
  }

  sheet.getRange(1, 1, 1, headers.length).setValues([headers]);
  sheet.setFrozenRows(1);
  return sheet;
}

function installTriggerIfMissing_(functionName, type) {
  const exists = ScriptApp.getProjectTriggers().some(function(trigger) {
    return trigger.getHandlerFunction() === functionName;
  });

  if (exists) {
    return;
  }

  if (type === 'time') {
    ScriptApp.newTrigger(functionName)
      .timeBased()
      .everyMinutes(5)
      .create();
    return;
  }

  if (type === 'edit') {
    ScriptApp.newTrigger(functionName)
      .forSpreadsheet(getSpreadsheet_())
      .onEdit()
      .create();
  }
}

function findRowByExactValue_(sheet, column, value) {
  if (sheet.getLastRow() < 2) {
    return 0;
  }

  const match = sheet
    .getRange(2, column, sheet.getLastRow() - 1, 1)
    .createTextFinder(String(value))
    .matchEntireCell(true)
    .findNext();

  return match ? match.getRow() : 0;
}

function parseOptionalDate_(value, fieldName) {
  if (value === undefined || value === null || value === '') {
    return '';
  }

  const date = new Date(value);
  if (isNaN(date.getTime())) {
    throw new Error(fieldName + ' must be a valid ISO 8601 timestamp.');
  }

  return date;
}

function asDate_(value) {
  if (!value) {
    return null;
  }

  const date = value instanceof Date ? value : new Date(value);
  return isNaN(date.getTime()) ? null : date;
}

function requiredString_(value, fieldName) {
  const result = String(value === undefined ? '' : value).trim();
  if (!result) {
    throw new Error(fieldName + ' is required.');
  }
  return result;
}

function safeEquals_(left, right) {
  const a = String(left || '');
  const b = String(right || '');

  if (a.length !== b.length) {
    return false;
  }

  let difference = 0;
  for (let i = 0; i < a.length; i++) {
    difference |= a.charCodeAt(i) ^ b.charCodeAt(i);
  }

  return difference === 0;
}

function truncate_(value, maximumLength) {
  const text = String(value === undefined ? '' : value);
  return text.length > maximumLength
    ? text.substring(0, maximumLength)
    : text;
}

function getRequiredProperty_(name) {
  const value = PropertiesService.getScriptProperties().getProperty(name);
  if (!value) {
    throw new Error('Missing required script property: ' + name);
  }
  return value;
}

function getSpreadsheet_() {
  if (
    !CONFIG.SPREADSHEET_ID ||
    CONFIG.SPREADSHEET_ID === 'YOUR_SPREADSHEET_ID'
  ) {
    throw new Error('Replace YOUR_SPREADSHEET_ID in CONFIG.');
  }

  return SpreadsheetApp.openById(CONFIG.SPREADSHEET_ID);
}

function jsonResponse_(body) {
  return ContentService
    .createTextOutput(JSON.stringify(body))
    .setMimeType(ContentService.MimeType.JSON);
}

Authorization and deployment

1. Replace YOUR_SPREADSHEET_ID.
2. Add API_TOKENS_JSON, SLACK_WEBHOOK_URL, and HEALTH_TOKEN
3. Run setupMonitoringWorkbook manually.
4. Review and accept the requested permissions.
5. Confirm that one time-driven trigger and one edit trigger were installed.
6. Deploy the script as a web application that executes as the deployment owner.
7. Select the access setting permitted by the organization that allows source platforms to reach the endpoint.
8. Copy the deployed URL and store it in each source platform as YOUR_MONITORING_WEB_APP_URL.
9. Do not use the temporary development URL for production integrations.

Execution logs are available in the Apps Script execution history. Failed events that reach the application are also recorded in System Errors. Duplicate execution is prevented with Event ID matching under a script lock.

Reusable Apps Script client

This client can be placed in an Apps Script project that needs monitoring. It includes a complete test job that counts rows in a Monitor Test Input sheet. Replace the configuration values before running it.

const MONITOR_CLIENT = Object.freeze({
  ENDPOINT: 'YOUR_MONITORING_WEB_APP_URL',
  TOKEN: 'YOUR_APPS_SCRIPT_SOURCE_TOKEN',
  WORKFLOW_ID: 'WF-IT-MONITOR-TEST-001',
  PLATFORM: 'GOOGLE_APPS_SCRIPT',
  FALLBACK_SLACK_WEBHOOK_URL: 'YOUR_FALLBACK_SLACK_WEBHOOK_URL'
});

function runMonitoredExample() {
  const runId = Utilities.getUuid();
  const correlationId = runId;
  const startedAt = new Date();
  const runningEventId = runId + '-RUNNING';

  try {
    postMonitoringEvent_({
      event_id: runningEventId,
      workflow_id: MONITOR_CLIENT.WORKFLOW_ID,
      platform: MONITOR_CLIENT.PLATFORM,
      run_id: runId,
      correlation_id: correlationId,
      status: 'RUNNING',
      started_at: startedAt.toISOString(),
      retry_count: 0,
      metadata: {
        environment: 'production',
        task: 'count_test_rows'
      }
    });
  } catch (monitorError) {
    console.warn(
      'RUNNING event could not be recorded: ' + monitorError.message
    );
  }

  try {
    const result = countMonitorTestRows_();
    const endedAt = new Date();

    postMonitoringEvent_({
      event_id: runId + '-SUCCESS',
      workflow_id: MONITOR_CLIENT.WORKFLOW_ID,
      platform: MONITOR_CLIENT.PLATFORM,
      run_id: runId,
      correlation_id: correlationId,
      status: 'SUCCESS',
      started_at: startedAt.toISOString(),
      ended_at: endedAt.toISOString(),
      duration_ms: endedAt.getTime() - startedAt.getTime(),
      retry_count: 0,
      metadata: {
        environment: 'production',
        task: 'count_test_rows',
        rows_processed: result.rowsProcessed
      }
    });

    console.log(JSON.stringify(result));
    return result;
  } catch (error) {
    const endedAt = new Date();
    const failurePayload = {
      event_id: runId + '-FAILED',
      workflow_id: MONITOR_CLIENT.WORKFLOW_ID,
      platform: MONITOR_CLIENT.PLATFORM,
      run_id: runId,
      correlation_id: correlationId,
      status: 'FAILED',
      started_at: startedAt.toISOString(),
      ended_at: endedAt.toISOString(),
      duration_ms: endedAt.getTime() - startedAt.getTime(),
      retry_count: 0,
      error_code: error.name || 'APPS_SCRIPT_ERROR',
      error_message: String(error.message || error),
      metadata: {
        environment: 'production',
        task: 'count_test_rows'
      }
    };

    try {
      postMonitoringEvent_(failurePayload);
    } catch (monitorError) {
      sendFallbackSlack_(
        'Apps Script task failed and central monitoring delivery also failed.\n' +
        'Workflow: ' + MONITOR_CLIENT.WORKFLOW_ID + '\n' +
        'Run: ' + runId + '\n' +
        'Task error: ' + failurePayload.error_message + '\n' +
        'Monitoring error: ' + monitorError.message
      );
    }

    throw error;
  }
}

function sendMonitoringHeartbeat() {
  const eventId = Utilities.getUuid() + '-HEARTBEAT';

  return postMonitoringEvent_({
    event_id: eventId,
    workflow_id: MONITOR_CLIENT.WORKFLOW_ID,
    platform: MONITOR_CLIENT.PLATFORM,
    run_id: eventId,
    correlation_id: eventId,
    status: 'HEARTBEAT',
    started_at: new Date().toISOString(),
    ended_at: new Date().toISOString(),
    duration_ms: 0,
    retry_count: 0,
    metadata: {
      environment: 'production',
      task: 'heartbeat'
    }
  });
}

function countMonitorTestRows_() {
  const ss = SpreadsheetApp.getActiveSpreadsheet();
  let sheet = ss.getSheetByName('Monitor Test Input');

  if (!sheet) {
    sheet = ss.insertSheet('Monitor Test Input');
    sheet.getRange(1, 1, 3, 1).setValues([
      ['Test Value'],
      ['Alpha'],
      ['Beta']
    ]);
  }

  return {
    ok: true,
    rowsProcessed: Math.max(sheet.getLastRow() - 1, 0)
  };
}

function postMonitoringEvent_(event) {
  const payload = Object.assign({}, event, {
    token: MONITOR_CLIENT.TOKEN
  });

  let lastError;

  for (let attempt = 1; attempt <= 3; attempt++) {
    try {
      const response = UrlFetchApp.fetch(MONITOR_CLIENT.ENDPOINT, {
        method: 'post',
        contentType: 'application/json',
        payload: JSON.stringify(payload),
        muteHttpExceptions: true,
        followRedirects: true
      });

      const status = response.getResponseCode();
      const text = response.getContentText();
      let body;

      try {
        body = JSON.parse(text);
      } catch (parseError) {
        throw new Error(
          'Monitoring endpoint returned non-JSON content: ' +
          text.substring(0, 300)
        );
      }

      if (status >= 200 && status < 300 && body.ok === true) {
        return body;
      }

      lastError = new Error(
        'Monitoring endpoint rejected the event: ' +
        String(body.error || 'HTTP ' + status)
      );
    } catch (error) {
      lastError = error;
    }

    Utilities.sleep(attempt * 1000);
  }

  throw lastError || new Error('Monitoring delivery failed.');
}

function sendFallbackSlack_(text) {
  if (
    !MONITOR_CLIENT.FALLBACK_SLACK_WEBHOOK_URL ||
    MONITOR_CLIENT.FALLBACK_SLACK_WEBHOOK_URL.indexOf('YOUR_') === 0
  ) {
    console.error(text);
    return;
  }

  const response = UrlFetchApp.fetch(
    MONITOR_CLIENT.FALLBACK_SLACK_WEBHOOK_URL,
    {
      method: 'post',
      contentType: 'application/json',
      payload: JSON.stringify({ text: text }),
      muteHttpExceptions: true
    }
  );

  if (response.getResponseCode() >= 300) {
    console.error(
      'Fallback Slack delivery failed: ' + response.getContentText()
    );
  }
}

To test the client, create and register WF-IT-MONITOR-TEST-001, run runMonitoredExample, and verify the RUNNING and SUCCESS rows. Rename the test sheet temporarily to test its automatic recreation. To test failure handling, revoke the script’s spreadsheet permission in a test project or use an intentionally invalid workbook context.

For scheduled operation, create an installable time-driven trigger for the monitored function or heartbeat function. Review execution logs in both the monitored project and central project. Manual recovery consists of correcting the source configuration and resending the same event ID. If the original event was accepted, the central endpoint returns a duplicate response rather than writing another row.

n8n sentinel configuration

1. Create a workflow named Automation Monitor Sentinel.
2. Add a Schedule Trigger that runs every five minutes.
3. Add an HTTP Request node using GET.
4. Request YOUR_MONITORING_WEB_APP_URL?token=YOUR_HEALTH_TOKEN.
5. Set a practical request timeout and enable limited retries for transient failures.
6. Add an IF node that requires the response property ok to equal true.
7. Connect the false and error paths to an HTTP Request node that posts directly to the Slack fallback webhook.
8. Include the sentinel execution ID, timestamp, and monitor URL in the fallback message.
9. Activate the sentinel only after testing it with the monitoring deployment temporarily disabled.

The sentinel should run on infrastructure that is operationally independent of the Apps Script endpoint. It cannot write to the central register while that register is unavailable, so the fallback Slack message is the temporary incident record. Staff reconcile it when service returns.

Failure Handling and Operational Reliability

Idempotency depends on stable event IDs. A source must reuse the same Event ID when retrying delivery of the same status event. A new execution attempt receives a new Event ID but retains the original Correlation ID.

There is no separate database dead-letter queue in this implementation. System Errors functions as the manual-review queue for requests that reached the service but could not be accepted. Source-platform failed execution histories and the n8n sentinel provide the fallback evidence for requests that never reached Apps Script.

Daily reconciliation compares terminal source executions with central run counts for critical workflows. A missing central event is investigated even if the underlying business workflow succeeded, because missing telemetry weakens future detection.

A Complete Example

Alder Peak has a critical n8n workflow named Approved Service Order to Dispatch. Its central Workflow ID is WF-OPS-DISPATCH-003. The workflow normally receives an approved service order, creates a dispatch record through an API, and notifies the operations channel.

1. A service order with business key SO-10584 enters n8n.
2. n8n assigns execution ID 84721.
3. The workflow uses SO-10584 as the Correlation ID.
4. The destination API returns HTTP 429 after its request allowance is reached.
5. The n8n shared error workflow receives the workflow name, workflow ID, execution ID, failed node, error message, and execution URL.
6. The error workflow maps the n8n workflow to WF-OPS-DISPATCH-003.
7. It generates Event ID N8N-84721-FAILED.
8. It posts a FAILED event to the Apps Script endpoint.
9. The endpoint authenticates the N8N token and confirms that the registered platform is N8N.
10. The Event ID is not already present, so the event passes duplicate validation.
11. The register appends a Runs row and creates INC-20260715-7A8B2C1D.
12. The workflow definition supplies CRITICAL priority, the systems analyst as owner, and the IT manager as escalation contact.
13. Slack receives a message mentioning the systems analyst and linking to the n8n execution and incident row.
14. The systems analyst changes the incident to ACKNOWLEDGED.
15. The analyst verifies that the destination did not create the dispatch record before returning HTTP 429.
16. After the destination allowance resets, the analyst starts a controlled retry.
17. The new n8n execution ID is 84736, but the Correlation ID remains SO-10584.
18. The retry succeeds and posts Event ID N8N-84736-SUCCESS with Retry Count 1.
19. The central service finds the unresolved incident with the same Workflow ID and Correlation ID.
20. The incident changes to RESOLVED, Resolved At is stamped, and the successful event ID is added to Resolution Notes.
21. A recovery message is posted to Slack because the workflow is critical.
22. The IT manager later confirms the dispatch record and changes the incident to CLOSED.

If the retry had failed again, the existing incident would have been updated with the new run ID, retry count, source URL, and error. It would not have created a second incident for SO-10584.

Implementation Cost

All amounts below are representative assumptions for this case study. They are not verified client results or vendor quotations. Current vendor licensing, webhook features, execution allowances, and connector entitlements must be checked before implementation.

The $90 tool allowance is deliberately separate from labour. Existing software is not treated as having no cost, but only incremental costs are used in the payback calculation.

An internal team could perform more implementation work, while a professional implementation team could handle architecture, coding, connector configuration, testing, and documentation. The appropriate split depends on internal platform knowledge and risk.

Estimated Time and Cost Savings

The case-study calculation uses these representative assumptions:

• 3,200 workflow executions per month
• 0.55 minutes of current monitoring and reactive handling allocated per execution
• 0.08 minutes of new dashboard-review handling allocated per execution
• 64 incidents or exceptions per month
• 8 minutes of manual investigation and review per exception after automation
• 3 hours of monthly maintenance
• $52 loaded hourly labour cost
• $90 incremental recurring software and execution cost
• $7,216 one-time implementation cost

The current 0.55-minute figure does not mean that an employee opens every execution. It allocates weekly platform checks, complaint handling, cross-platform searching, error transcription, and reconciliation across the monthly execution volume.

Current monthly labour hours: Monthly volume × current minutes per record ÷ 60

New monthly labour hours: Monthly volume × new minutes per record ÷ 60, plus exception handling and maintenance

Monthly hours recovered: Current monthly labour hours minus new monthly labour hours

Estimated monthly labour value: Monthly hours recovered × loaded hourly labour cost

Net estimated monthly value: Monthly labour value minus recurring tool costs

Estimated payback period: One-time implementation cost ÷ net estimated monthly value

Recovered time does not automatically reduce payroll. It may provide additional support capacity, quicker turnaround, less overtime, fewer administrative checks, or the ability to support more workflows without adding equivalent monitoring effort.

Non-financial benefits include clearer ownership, quicker failure detection, fewer user follow-ups, more consistent incident handling, better audit evidence, improved platform reporting, and reduced dependence on the employee who originally built each workflow.

Readers should replace the workflow volume, current handling time, exception rate, review time, maintenance hours, labour rate, recurring platform cost, and implementation cost with their own figures.

Adding AI to the Automation

AI is optional and is introduced only after deterministic event validation, duplicate prevention, incident creation, ownership, escalation, and retry correlation operate reliably.

The core monitoring benefits do not require AI. Required fields, exact workflow lookups, timestamp comparisons, retry counts, threshold rules, and permissions are more reliable and less expensive when implemented with normal code.

Potential AI uses include:

• Classifying unstructured error messages into operational categories
• Summarizing long source errors
• Suggesting likely diagnostic checks
• Grouping semantically similar incidents with different error wording
• Identifying missing diagnostic information
• Drafting a technical incident summary for human review

AI should not decide whether to replay a payment, alter an accounting record, send a customer communication, approve access, or close a critical incident.

The Recommended AI Enhancement

The recommended enhancement classifies new incident error messages and proposes a short diagnostic action. It runs asynchronously on open incidents, so an AI outage cannot block incident creation or Slack alerting.

• Trigger: A five-minute scheduled job finds open incidents whose AI Review Status is blank.
• AI input: Platform, workflow name, criticality, error code, and sanitized error message.
• System instruction: Act as an automation operations triage assistant and never make final operational decisions.
• Expected output: Strict JSON with category, summary, suggested action, confidence, and human-attention flag.
• Validation: Category must be allowed, confidence must be between 0 and 1, and text fields are length-limited.
• Record update: AI fields are written to the incident row with PENDING_HUMAN_REVIEW.
• Human review: The systems analyst marks the output ACCEPTED, CORRECTED, or REJECTED.
• Low confidence: Confidence below 0.70 remains in the manual-review queue and is not used for routing.
• Prohibited data: Credentials, customer documents, full payloads, personal data, payment data, and source tokens.
• Failure behavior: Incident processing continues normally and the AI failure is written to System Errors.

The reusable system instruction is:

You are an automation operations triage assistant.

Analyze only the supplied sanitized incident fields. Do not assume facts that are not present. Do not make a final decision about retries, payments, accounting entries, access, customer communications, or incident closure.

Select exactly one allowed category. Produce a concise factual summary and one suggested diagnostic action for a human operator. If evidence is incomplete, say what is missing. Set needs_human_attention to true for ambiguous, security-related, data-integrity, or high-impact errors.

Return only JSON matching the supplied schema.

The user prompt template is:

Classify this sanitized automation incident:

Platform: {{PLATFORM}}
Workflow: {{WORKFLOW_NAME}}
Criticality: {{CRITICALITY}}
Error code: {{ERROR_CODE}}
Error message: {{ERROR_MESSAGE}}

Allowed categories:
AUTHENTICATION
RATE_LIMIT
TIMEOUT
INVALID_DATA
MISSING_DATA
PERMISSION
FILE_OPERATION
NOTIFICATION
DESTINATION_UNAVAILABLE
SCRIPT_ERROR
CONFIGURATION
UNKNOWN

The required structured output is:

{
  "category": "RATE_LIMIT",
  "summary": "The destination API rejected the request because its request allowance was exceeded.",
  "suggested_action": "Check the destination retry window and confirm that replaying the correlated business record will not create a duplicate.",
  "confidence": 0.94,
  "needs_human_attention": true
}

If an approved AI provider supports strict structured output, the scheduled process should enforce this JSON schema:

{
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "category": {
      "type": "string",
      "enum": [
        "AUTHENTICATION",
        "RATE_LIMIT",
        "TIMEOUT",
        "INVALID_DATA",
        "MISSING_DATA",
        "PERMISSION",
        "FILE_OPERATION",
        "NOTIFICATION",
        "DESTINATION_UNAVAILABLE",
        "SCRIPT_ERROR",
        "CONFIGURATION",
        "UNKNOWN"
      ]
    },
    "summary": {
      "type": "string",
      "maxLength": 500
    },
    "suggested_action": {
      "type": "string",
      "maxLength": 700
    },
    "confidence": {
      "type": "number",
      "minimum": 0,
      "maximum": 1
    },
    "needs_human_attention": {
      "type": "boolean"
    }
  },
  "required": [
    "category",
    "summary",
    "suggested_action",
    "confidence",
    "needs_human_attention"
  ]
}

Provider credentials must be stored in an approved secret store or Apps Script Properties. The model name, endpoint, retention settings, regional processing, and data-use terms must be confirmed against the selected provider’s current documentation before deployment.

Benefits of the AI Enhancement

• Operators spend less time reading repetitive error text.
• Different platform messages can be grouped into consistent categories.
• Monthly reports can show themes such as authentication, rate limits, and invalid data.
• Suggested checks can help backup staff begin investigation.
• Low-confidence and ambiguous incidents can be identified quickly.
• Similar error wording can be found even when exact error codes differ.

These are specifically AI-assisted benefits. Detection, logging, incident creation, owner assignment, alerts, retries, and escalation are provided by the core rule-based system.

What Remains Rule-Based or Human-Controlled

Estimating the Additional Value of AI

The representative comparison assumes 64 monthly incidents. Without AI, an operator spends approximately three of the eight incident-handling minutes reading and categorizing error text. With AI, the operator spends one minute reviewing the proposed category and summary.

Gross additional time recovered: 64 incidents × 2 minutes ÷ 60 = 2.13 hours per month

Correction allowance: 64 × 15% × 3 minutes ÷ 60 = 0.48 hours per month

AI failure allowance: 64 × 3% × 3 minutes ÷ 60 = 0.10 hours per month

Net additional capacity: 2.13 – 0.48 – 0.10 = 1.55 hours per month

At a $52 loaded hourly rate, the representative labour value is $80.60 per month before AI usage cost. If the organization budgets $12 per month for model usage and monitoring, the net estimated additional value is $68.60 per month.

The value is modest at this volume. The stronger reason to add AI would be consistent categorization and easier trend reporting, not removal of human involvement.

Testing Checklist

Use synthetic sample data and test accounts before processing real operational information.

Ongoing Maintenance

Documentation must include the event schema, workflow inventory, deployment owner, source token rotation procedure, Slack webhook rotation procedure, archive process, retry policy, sentinel configuration, and manual recovery instructions.

If AI is enabled, the systems analyst samples accepted and corrected outputs monthly. Category drift, repeated low-confidence results, unexpected sensitive data, and increasing usage cost are reasons to pause the AI process without disabling core monitoring.

When to Move to Dedicated Software

The implementation should not be replaced solely because it uses Google Sheets. It remains appropriate while the run volume, access model, retention requirements, and support capacity remain manageable.

Migration to a managed observability, incident-management, integration-governance, or custom database platform becomes more appropriate when:

• Execution volume causes slow searches, locking, or archive work.
• The business needs multi-year searchable run retention.
• Multiple locations require separate permissions or data residency.
• Formal audit requirements prohibit spreadsheet-based operational records.
• On-call schedules, paging, acknowledgement timers, and mobile escalation become necessary.
• Customer-facing service status or incident portals are required.
• Complex service dependencies require parent and child incident relationships.
• Logs must be searched across high-volume application and infrastructure sources.
• Security policy requires signed requests, private network access, managed identities, or centralized secret rotation.
• Exception rates create excessive manual maintenance.
• Integration teams need deployment promotion, version control, and formal change approval.
• Offline or mobile incident response becomes an operational requirement.
• The organization requires vendor support commitments for the monitoring layer.
• Spreadsheet size or Apps Script execution limits interfere with reliable event intake.

The shared event schema reduces migration effort. A future database or monitoring API can accept the same Workflow ID, Event ID, Run ID, Correlation ID, status, error, owner, and source URL structure.

Implementation Checklist

• Document workflow volume, criticality, support hours, and escalation targets.
• Inventory Zapier, Make, n8n, Apps Script, and relevant Power Automate workflows.
• Select the protected Google account, workbook owner, and Slack channel.
• Confirm required webhook, HTTP, error-workflow, and trigger capabilities.
• Create platform-specific source tokens and a separate health token.
• Create the Workflows, Runs, Incidents, and System Errors structures.
• Assign stable Workflow IDs and Correlation ID rules.
• Register owners, Slack member IDs, criticality, and stale thresholds.
• Deploy the Apps Script web application.
• Configure Apps Script Properties without exposing credentials in cells.
• Map every source field into the normalized event schema.
• Add success and failure reporting to each monitored platform.
• Inspect the returned ok value in every connector.
• Implement duplicate prevention with stable Event IDs.
• Create correlated incident resolution for successful retries.
• Configure acknowledgement, reassignment, retry, and closure procedures.
• Configure critical Slack alerts and escalation timing.
• Add the n8n sentinel and direct Slack fallback route.
• Restrict workbook, script, source platform, and Slack permissions.
• Exclude credentials and sensitive business payloads from logs.
• Create open, overdue, stale, owner, failure, and recovery views.
• Test valid, invalid, duplicate, failed, stale, retry, and recovery events.
• Test unavailable approvers, failed notifications, and endpoint outages.
• Run a pilot with low-risk workflows before critical rollout.
• Document rollback, archive, token rotation, and manual recovery procedures.
• Validate representative implementation and recurring cost assumptions.
• Replace savings assumptions with actual workflow and labour data.
• Introduce AI only after core monitoring is stable.
• Require human review of every AI category and suggested action.
• Assign a primary maintenance owner and backup owner.
• Define volume, security, audit, and maintenance criteria for migration to dedicated software.