Resuming A Dialog Session Following A Human to Computer Dialog

The user may intend for a task or action to be performed/completed in the conversation.

The user can engage with the automated assistant application using a first client device of client devices operated by the user. The method can also include determining, based on the analysis, that the user did not complete a task raised during the human-to-computer dialog session.

Subsequently, the user may express interest, directly or indirectly, in completing the dialog session, and the automated assistant can provide the user with a selectable element that, when selected, causes the dialog session to be reopened.

This patent is intended to continue conversations and has a phone call as an action that can further that activity:

However, if the user does not select the conversation suggestion, then the rank associated with the conversation suggestion can be decreased (or, in some cases, unaltered).

The user can then continue the dialog session with the automated assistant so that the initially intended action can be performed by the automated assistant.

In this way, other higher-ranked conversation suggestions can be presented to the user so that the user might be presented with conversation suggestions that the user would be more interested in continuing to the point of completion.

Suggestions for Completing Human to Computer Dialogs

This doesn’t seem like a big problem, but the patent expands on the troubles involved:

The automated assistant can respond via the mechanical assistant interface with options of who exactly the user is referring to (e.g., “Would you like to call Sally Smith, Sally Beth, or Sally O’Malley?”).

The task or action can be:

• Placing a call
• Booking an event
• Sending a message
• Controlling a device
• Aaccessing information
• Any other action that can be performed by a computing device

The Focus Here Is On Completing An Action

This patent seems to make sense to broaden suggestions beyond the person interacting with the automated assistant. The actions suggested might come from outside sources such as a video.

The home page can provide multiple suggestions related to various applications on the client device.

The method can further include, based on the determining, storing a state of the human-to-computer dialog session in which the task is primed for completion.

The conversation between the automated assistant and the user can be stored in memory, which the computerized assistant can access at a later time when the user is determined to be interested in having the robotic assistant perform the action.

This change of suggestion elements could be based on the assumption that the user was not interested in the previously provided suggestion elements if the user did not select those previously presented suggestion elements.

Suggestions Based on Interests of Others

The selectable element can include the phrase “Call Sally.” In response to the user selecting the selectable element, the automated assistant can provide an output corresponding to where the previous conversation ended (e.g., “Would you like to call Sally Smith, Sally Beth, or Sally O’Malley?”). In this way, the user does not have to repeat past commands to the automated assistant, thereby streamlining the path to completing the intended action (e.g., placing a phone call to Sally).

Is the Human to Computer Dialog Session Complete or Incomplete?

The task can include dialing a telephone number, and the stored state can identify the incomplete job. The method can also include assigning a rank to the stored state and comparing the position to other ranks associated with other stored conditions of human-to-computer dialogs.

Conversation suggestions can be presented to the user based on ranking and weights established based on other users’ aggregate interests.

Suggestions for completing conversations can be ranked and weighted according to certain computer-based activities of the user. These can go beyond the actions of having conversations.

Furthermore, tracking incomplete interactions allows the user to complete an exchange by their election should the user be interrupted during an interaction or choose not to continue at some point during the interaction.

The suggestions can be presented as selectable elements at a conversational user interface of the client device, along with other selectable factors that can be associated with the conversation suggestion.

We get told that:

For example, if after the incomplete conversation, other people watch the video, and after that, the user searches for the video, the user can be presented with a conversation suggestion for completing the conversation to watch the video.

The Summary Finally Focuses on Interaction with a Person and an Automated Assistant in a Human to Computer Dialog

Subsequently, while viewing a food website, the user can be presented with a conversation suggestion corresponding to the incomplete food order conversation.

Increasing the Rank of that Human to Computer Dialog

The conversation can be stored as complete even when a performed task was not necessarily completed. For example, the user can engage in a few rounds of discussion with the automated assistant to get the computerized assistant to start a music application for playing music.

Some of the other patents about automated assistants that I have written about include:

This can be a waste of computational resources and human time. The automated assistant would reprocess the user’s commands that were already processed during the previous interaction.

Automated assistants can be used to track incomplete interactions with the automated assistant so that they can be subsequently completed without repeating previous commands.

The automated assistant can store specific conversations according to whether the discussions were complete or incomplete. Conversations between the user and the robotic assistant can include multiple different spoken or typed commands from the user and multiple different responsive outputs from the automated assistant.

For example, the stored conversation can be ranked higher if people have recently been searching for the video than when people have not been searching for the video.

Automated assistants may use several computing devices, such as smartphones, tablet computers, wearable devices, automobile systems, and standalone personal assistant devices.

Furthermore, the selectable element can invoke the automated assistant application in the state to resume the human-to-computer dialog session and can be selectable to cause the task to be completed.

While interacting with an automated assistant, a user can become distracted and not complete the interaction to the point of the automated assistant performing some action.

For example, a previous incomplete conversation can be associated with a food order that the user was attempting to place but ultimately did not complete because the user did not provide an address for the food to be delivered.

For example, the home page can provide reminders about events stored in the calendar application of the client device and provide news article summaries from a news application on the client device.

If the user selects the conversation suggestion, a rank associated with the incomplete conversation can be increased.

Additionally, after the storing, the method can provide to the client devices data indicative of a selectable element that is selectable to enable the user to complete the task.

Systems, methods, and apparatuses for resuming dialog sessions via automated assistant
Inventors: Vikram Aggarwal, Jung Eun Kim, and Deniz Binay
Assignee: Google LLC
US Patent: 11,264,033
Granted: March 1, 2022
Filed: March 20, 2019

I have a Google Speaker at home to perform some searches and listen to some music.

During the previous conversation, the user may have provided the number of guests and the dates for the hotel booking but may not have paid for the hotel booking. Therefore the conversation was not complete as a hotel was not booked. The hotel conversation can be stored as incomplete and subsequently provided in association with a selectable element when the user uses a search application to find places to vacation.

Where Conversation Suggestions Might Take Place

For example, humans (who, when they interact with automated assistants, may be referred to as “users”) may provide commands and requests using spoken natural language input (i.e., utterances) which may, in some cases, be converted into text and then processed, and by providing textual (e.g., typed) natural language input.

Yet another implementation may include a system of computers and robots that include processors operable to execute stored instructions to perform a method such as the methods described above and elsewhere herein.

This Resuming Dialog Sessions In A Human to Computer Dialog Patent Can Be Found At

As with most patents, this one introduces the problem that the patent gets intended to help solve. The algorithm behind the patent gets used to solve issues anticipated by the inventors of the patent.

When a task is completed as a result of the conversation, the conversation can be stored with a field or slot that includes a parameter indicating that the conversation resulted in an action (e.g., STORE_CONVERSATION=(content=” call sally; Would you like to call . . . ;”, action=”call,” complete=”1″).

This is exciting because Google is fleshing out different aspects of how Automated Assistants work in the Google search ecosystem. It allows us to look behind the curtain at Google and see what steps they are taking to get behind the curtain. I will focus on the summary of the patent rather than the full description.

The data can be generated based on the state.

Because of this, the searcher may have to repeat inputs to the automated assistant to have the computerized assistant complete the task or action.

Sharing is caring!