Bot Failure Report
Bot Failure Report is one of the crucial reports for improving Chatbot performance and reducing the number of handover happening to Human Agents.
You can download bot failure report from AmplifyReach Dashboard -> Analytics -> Left Panel -> Dashboard -> Scroll down to Bot Failure Section -> Download. You can select department and date range before downloading the report.
It provides a snapshot of information providing all the details regarding the chatbot failure and why the Chatbot transferred the chat to a Human agent. This report can be downloaded in xlsx format.
There are four high level reasons for a chatbot failure and transferring chat to Human Agent.
- Trigger Failure – Chatbot can integrate external Rest/SOAP API Calls (Trigger). If API Call fails for any reason like Timeout, Response Validation condition or HTTP failure, it will be marked as Trigger Failure in Bot Failure Report. These entries can be used to improve API Handling in your system.
- Input Validation Error – Chatbot can be configured to validate user input. If this validation fails, then it is marked as Input Validation Error in Bot Failure Report.
- Intent Identification Failure – Chatbot can be configured to detect Intent from user input. If chatbot fails to understand intent, it will be marked in Bot Failure Report. These entries can be used to improve Intent Training of Chatbot or introduce new intents
- Info Not Available – When a customer says that I don’t have a particular information handy or available, those failures are marked in Bot Failure Report.
Details in Bot Failure Report
The Bot failure reports consists of following information
| Name | Description |
|---|---|
| Date | It gives information of date and time of failure. |
| Chat ID | It describes the unique Chat ID of the relevant chat. It can be used to check chat transcript. |
| Type | Type indicates the type of following failure. It has one of the following values. – Trigger – Input Variable – Intent |
| Description | This provides the sub-type for the failure |
| Failed On | This provides the conversation details where the failure occurred depending on the Type of failure. |
| User Message | The message provided by the visitor after which the failure occurred. 1. In case of Input Variable Failure, the user message shows the wrong input entered by the user. 2. In case of Intent Failure, user message is useful to check if chatbot intent training needs to be improved, or it also provides opportunity to understand customer demand and train new intents. 3. In case of trigger failure the user message is blank. |
| Number of Retries | Every failure, even if it doesn’t result into handover is captured as per retry count. |
| Request Type | Request type for which the failure occurred. It contains either intent name or marker name. This field would be empty if the Bot failed to identify the intent. |
| Department Name | Name of the department for which the request was being served. |
| Conversation Name | Name of the conversation for which the request was being served. |
| Deployment ID | This helps to trace which deployment of the failure. This is helpful mainly if you have multiple deployments. |
Understanding Failure Conditions
The Type and Description fields are important fields in Bot Failure Report. Following table explains how Type and Description values map to error description.
| Type | Description | Error Description |
|---|---|---|
| Input Variable | Invalid variable value | The user entered the wrong input. Input validation failed for the given entry. |
| Input Variable | Handover – Invalid variable value retry limit reached | Max retry limit is reached and the handover is initiated by Bot to Human Agent. |
| Input Variable | ||
| Input Variable | Handover – Info not available | Visitor said that I don’t have information. These failures are captured here. |
| Input Variable | Chat Closed – Invalid variable value retry limit reached | Max retry limit is reached and the chat is closed because Human Agent support is off. |
| Input Variable | Chat Closed – Info not available | When a customer says that I don’t have information and Chat is closed by Bot because Human Agent support is off. |
| Input Variable | Foul message | If visitors talk in abusive language. Such messages fall into the Foul message category. |
| Intent | Intent detection failed | ChatBot failed to understand customer intent from input sentence or the intent is not trained in the bot. |
| Intent | Handover – Intent detection failed and retry limit reached | Max retry limit is reached and the handover is initiated by Bot to Human Agent. |
| Intent | Chat Closed – Intent detection failed and retry limit reached | Max retry limit is reached and the chat is closed as Human Agent support is off. |
| Intent | Foul message | If visitors talk in abusive language. Such messages fall into the Foul message category. |
| Trigger | Handover – Trigger status failure | Status check for the trigger failed and handover to human agent is initiated. |
| Trigger | Handover – Trigger validation failure | Validation check provided for trigger failed |
| Trigger | Chat Closed – Trigger status failure | Status check failed, chat closed as human agent support is off. |
| Trigger | Chat Closed – Trigger validation failure | Validation provided under the trigger failed. Chat closed indicates that Human Agent support is turned off. |
Checking Trigger (API) failures:
Bot failure report provides additional details on Trigger (API) failures. The details are present under “Trigger Failure Details” tab. It provides following details about each trigger failure.
| Name | Description |
|---|---|
| Date | The date and time of trigger failure |
| Chat ID | Unique Chat ID of conversation. This can be used to check chat trnascript |
| Department- | Department of the conversation |
| Conversation Name | Conversation details in which the failure was detected |
| Failed On | It contains the trigger name where the trigger was failed. |
| Request Type | Request type for which the failure occurred. It contains either intent name or marker name. This field would be empty if the Bot failed to identify the intent. |
| API URL | This contains the end point of Rest/Soap Request |
| API Method | Request Method like GET, POST, PUT, DELETE |
| API Request Header | It contains HTTP headers set in the request |
| API Request Payload | It contains payload sent at the time of request |
| API Response Header | It contains detailed information about response header. |
| API Response Status Code | It contains HTTP Response code received |
| API Response Payload | This contains the HTTP Response body. |
| Deployment ID | This contains Deployment ID of the conversation. This is helpful mainly if you have multiple deployments. |
