Game-changer: Using New AI Tools to Analyze Guest Experience Feedback

Introduction

If you've been following our recent blog posts, you know that we've been experimenting with use cases for new Artificial Intelligence (AI) tools, like ChatGPT, within the context of our business; collecting, analyzing and reporting guest experience feedback for hospitality operators.

‍

Our latest AI project involves exploring ChatGPT's capacity to process and manage the most complex data that we gather from guest experience surveys; typed-in guest comments.

‍

During a Guestinsight survey, we provide guests with three prompts to share their open-ended thoughts: 1) We ask if they have specific comments about their guestroom, 2) We ask if they would like to acknowledge any hotel staff members, and 3) We request any general comments, including their rationale for rating their stay as exceeding, meeting, or falling short of their expectations (which was the initial question posed in the survey).

‍

Handling and interpreting the responses can pose a challenge because of their unstructured format and the vast amount of feedback we receive. It’s a labor intensive and time-consuming task for hotel teams to pore through their guests’ comments, weigh the insights they glean against the quantitative metrics that our surveys also provide, and take appropriate actions, when necessary. 

‍

We are hopeful that the new AI tools can help ease the burden placed on hotel teams to process all of their guests’ comments. These are the specific capabilities of AI we hope to leverage in order to save hotel teams time and be able to react to all guest feedback faster than ever before:

• Sentiment Analysis: We are looking at ways to expand our use of Natural Language Processing (NLP) AI models to assess the sentiment of guest comments. In this previous blog post, we took a look at how the new NLP models stack up to what we’ve already been using (we’ve been using NLP for quite some time to gauge the sentiment of guest comments, for among other things, determining whether a comment might be suitable to push to a hotel’s testimonial feed). 

• Distillation and Summarization of Guest Comments: Here’s where labor and time saving will really start kicking in; AI can distill and summarize guest comments, condensing the free-form typed in comments into meaningful summaries. This will allow hotel team members to quickly grasp key highlights without having to painstakingly read every comment.

• Categorization and Prioritization: Another promising time and cost savings of AI in guest survey analysis is the ability to categorize and prioritize comment text. AI algorithms can automatically group guest comments into categories such as room comfort, staff service, cleanliness, amenities, and more. This will allow hotels to easily identify recurring themes and prioritize areas for improvement based on factors like frequency, impact, and feasibility. It will streamline and automate the analysis process, enabling efficient decision-making and resource allocation.
  

• ‍Actionable Insights for Different Teams: We already send guest survey alert messaging to hotel team distribution lists based on various criteria. But currently, messages go to everyone on the distribution list and “ownership” of the content of the message is up to the recipients to decide (time and energy expended). Imagine if messages only went to team members on the distribution list that were appropriate, based on the content of the comments. By mining the guest feedback, AI can provide specific recommendations and suggestions to the relevant departments or teams responsible for addressing the identified issues. This targeted approach ensures that the right teams receive actionable recommendations, empowering them to take prompt action and drive improvements.

‍

The Test

To put ChatGPT’s capabilities in these areas to the test, we uploaded comments from 100 guest experience surveys. The challenge for us was then to engineer prompts for ChatGPT to give us the output we were hoping for in handling the tasks listed above. Prompt engineering is an emerging field; to explain exactly what it is, I asked ChatGPT to define it. This was its response:

In AI models like ChatGPT, prompts are the input strings of text that the model generates a continuation or response for. The aim of prompt engineering is to create prompts that yield the most useful and accurate responses from the AI.

Prompt engineering can involve a variety of techniques, such as:

• Varying the format of the question
• Including specific keywords or phrases known to provoke certain responses
• Giving the AI model a role to play (e.g., "You are a helpful assistant...")
• Providing the model with a context or a goal
• The effectiveness of the prompts can significantly impact the quality of the AI's responses, making prompt engineering an important part of deploying language models like ChatGPT.

‍

The prompt engineering process is iterative, somewhat akin to going down a rabbit hole. As you refine prompts based on ChatGPT’s responses, you make new discoveries about what you may be looking for in the output. As an example, our first thought about how to use AI with our survey data was transactionally; simply appending AI generated columns to each survey data record to accomplish the goals mentioned above.

However, as part of the prompt engineering process, we discovered that we should also use the new AI tools for summary reports of batches of survey data. Keep reading to see how we’re planning to do this.

‍

The Results

In the table below you will find a few examples of the 100 records of guest comment input we gave ChatGPT in the blue columns A-E. 

ChatGPT’s output appears in the orange columns F-I.

Let’s look at how ChatGPT did with respect to the 3 tasks we outlined up front.

‍

Sentiment Analysis:

We asked ChatGPT to determine the sentiment of guest comments using its most current language model (GPT 4) because we wanted it to specifically consider the sentiment of the comments for the other tasks we asked of it. As mentioned earlier, we already tested the new Natural Language Processing models against what we’re currently using in an earlier blog post. You can see the results in column F of the table. ChatGPT certainly did a good job in assessing the tone of the comments, but sentiment analysis is just scratching the surface of its capabilities. There’s really not much difference in the results here from what we’ve seen in the past.

‍

Distillation and Summarization of Guest Comments:

Wow– take a look at column G in the table. ChatGPT did a great job of distilling each guest’s comments down to the important points for the hotel’s team.

‍

And, as mentioned earlier, the use cases for this capability extends beyond looking at each guest’s feedback individually. ChatGPT created a summary analysis of the 100 guests’ comments (including the examples in the table) …in a few seconds:

‍

Here's a concise summary of the key points extracted from the guest comments:

‍

Positive Comments:

Guests expressed their appreciation for exceeding their expectations.

Many guests found the rooms pleasant, quiet, and in an excellent location.

Staff members were commended for their professionalism, kindness, and hard work.

The hotel was praised for being clean, safe, and well-managed.

Guests highlighted the convenience of having a microwave and fridge in the room.

The helpfulness and friendliness of the staff were consistently mentioned.

Some guests mentioned the hotel as their favorite place to stay and expressed gratitude for the positive experiences over the years.

Areas for Improvement:

Some guests mentioned issues with room amenities, such as loud AC units, noise from neighboring rooms, and small room sizes.

Guests requested more food options due to limited choices in the vicinity.

Some guests faced difficulties with the reservation process and check-in procedures.

The absence of breakfast or changes in breakfast offerings was noted by a few guests.

Issues with ironing boards, Wi-Fi connectivity, and bathroom facilities were raised by specific guests.

Staff Recognition:

Specific staff members were recognized for their exceptional service and housekeeping efforts.

‍‍

The concise summary captures the key points mentioned in the guest comments, providing an overview of both positive feedback and areas for improvement.

‍

Wow, again. We think this is a game-changer, offering incredible savings of time for hotel teams and the speed in which teams can react to all guest feedback.

In fact, the first use case for integrating  new AI tools into the GuestInsight platform is based on this example. One of the features of our platform is a snapshot Guest Experience Scorecard that is emailed to a property's designated email distribution list on a weekly basis. At a glance, our clients see how their guests are rating the guest experience. However, that weekly scorecard only looks at important metrics calculated from the survey responses. It does not take into account the open-ended guest comments- until now. We’re beta testing a summary report, like the one that ChatGPT generated for our exercise, and hope to have it rolled out to our clients by the end of the summer.

‍

Categorization and Prioritization and Actionable Insights for Different Teams:

Let’s look at how ChatGPT performed these tasks together, because in practice, these functions will be used together. You can see ChatGPTs results in column I of the table, as well as in the batch summary of all the survey comments above.

Again, ChatGPT did a great job in these areas and we foresee great savings of time and speed to react to feedback for hotel teams. Certainly, there will be some customization required for each property to match operational departments with appropriate categories for the AI to sort comments by. This will simply become part of our onboarding process for new clients, just as we have to collect team email addresses for our distribution lists.

‍

Final Thoughts

We are excited to begin offering these new AI capabilities to our clients. AI tools like ChatGPT, when integrated with current survey tools, will offer hotel operating teams amazing savings in time and labor, as well as the ability to react quicker than ever before to all guest feedback. 

That’s our take on it. By comparison, we asked ChatGPT what it thought about where its capabilities fit into what we do. Here’s its conclusion:

The future of guest survey offerings lies in AI-powered platforms that revolutionize the way hoteliers analyze feedback and drive actionable insights. By embracing AI's advanced capabilities for sentiment analysis, comment distillation/summarization, categorization, and actionable recommendations, hoteliers can unlock unprecedented efficiencies, elevate guest experiences, and save valuable time and resources.

With AI as their ally, hoteliers can swiftly uncover guest sentiments, distill insights, and prioritize action items, resulting in enhanced operational efficiency and significant time and labor savings. Investing in a guest survey platform that harnesses the power of AI is a game-changer, propelling hotels towards exceptional guest experiences while optimizing their operations and resources.

‍