How to save time with AI-generated minutes in online meetings!

October 23, 2024 – Reading time: 8 minutes

AI-supported tools like Microsoft Copilot can significantly enhance the efficiency and quality of meeting minutes in project management. AI enables accurate transcription of online meetings, with a 98% accuracy rate, reducing report creation time by 70-90% and saving up to seven hours per week. While AI improves the completeness and transparency of meeting records, human oversight is still required for complex information. Automated minutes allow participants to focus more on discussions, improving decision-making, and practical tips highlight the importance of addressing both technical and cultural factors for successful AI integration.

Effective project management requires regular meetings to discuss progress, solve problems, and make decisions, with meeting minutes being essential to capture key points, results, and tasks. However, taking minutes can be challenging, requiring active participation while accurately recording and organizing information. AI tools, such as Microsoft’s Copilot app integrated into Teams, use speech recognition and processing (via OpenAI’s GPT-4) to automate this process, improving meeting efficiency and the quality of project management.

The theoretical potential of AI support for everyday project work presented in many publications raises high expectations. We wanted to know what is actually already possible in practice and how great the demonstrable benefits are. This practical perspective is particularly important in order to make both advantages and limitations tangible.

That is why we did not test under «laboratory conditions», but in real situations that project managers experience in their day-to-day work. We proceeded as follows and documented the respective results in detail in order to make a realistic and comprehensible assessment based on practical experience:

• We tested the automatic meeting protocols in twelve different real-life appointments with a total duration of 6.5 hours.
• The meetings covered various topics, participants and languages.
• We took a closer look at special situations, such as a change of language during the appointment or background noise, to test how well the artificial intelligence can handle them.
• We also tested hybrid appointments in which several participants sat together in person at a video terminal. Pure face-to-face meetings, where all participants are on site and the AI listens in via microphone, did not take place.
• We used transcripts taken by a minute-taker during the meeting as a reference. These contained the most important discussion points, decisions and agreed tasks of the respective meeting.

The quality criterion for the comparison was ultimately whether the transcripts generated by the AI were complete and correctly reflected the content, results and decisions of the meeting.

For a quantitative test, we took a sample of around 1,600 words in total, switching from German to English after 1,044 words without adjusting the language setting of the meeting. This corresponds, for example, to the situation when an English-speaking person joins the meeting. 566 words were therefore spoken in English, although the AI expected German.

We manually transcribed this sample as a reference and compared the two sections with the automatically generated transcript.

In the first section, where language and language setting matched, the AI transcription contained only 21 errors, meaning that the accuracy of the speech recognition was 98%. Deviations in punctuation marks (e.g. question marks at the end of a statement) and different sentence separations were not taken into account. The automatic transcript was consistently easy to understand and met the requirements for a conversation transcript.

In the second section, however, the error rate was around 20%: 118 of the 566 words were interpreted incorrectly. The AI did not really «understand» that the participants spoke English despite the language setting «German».

The AI correctly assigns the statements to the participants, provided that everyone is connected via their own account with their own microphone. If two or more people share a microphone in a meeting room, only the person whose account was connected to the group will appear in the transcript and what was said will be assigned to that person.

The fact that speech recognition works satisfactorily with the help of AI is already a major step forward. It is also convenient that the transcript is automatically available in the appointment and does not have to be downloaded manually as a text file.

However, the greatest strength and also the greatest benefit of the automatic creation of minutes with AI lies in the processing of the transcript: How well is the AI now able to derive important information from the verbatim transcript, such as decisions made and measures adopted?

To do this, we compared the follow-up tasks recorded manually at each meeting with the tasks recognized by the AI. On average, there were five to six tasks per appointment.

The result was consistently positive: the AI correctly identified all the tasks mentioned in each appointment and highlighted them in the summary. The accuracy of the task descriptions was also very good at almost 100%: all tasks were assigned to the correct person; only one task mentioned was only partially correctly logged in terms of content.

The most obvious benefit we tried to estimate was the time saved. To this end, we measured the time required for the manual preparation of the minutes. We only took into account the time required to produce the results (meeting points, decisions, tasks), but not the time required to transcribe the entire conversation.

For the twelve appointments in question, the manual preparation and follow-up time was around 20 minutes per appointment. These were shorter meetings and therefore required less documentation. In order to make a representative statement for all types of meetings, we supplemented these measurements with empirical values from our internal company database. This also includes data points from longer meetings and workshops. This data resulted in a slightly higher mean value of around 25 minutes per meeting. Assuming that a project manager attends four meetings per day on average, this adds up to seven to eight hours per week.

In busy times, more than four appointments per day are not uncommon, but realistically this does not increase the effort required for logging – it simply falls by the wayside due to a lack of time. In our view, this represents a high risk because neglecting this communication task can lead to serious errors. This is a strong argument for the use of automated meeting minutes.

In contrast to the reduction in workload, qualitative aspects can only be evaluated in monetary terms to a limited extent. We see the following qualitative advantages that speak in favor of the use of AI protocols, depending on the project and company environment:

• Participants can concentrate on actively taking part in the meeting. This increases the quality of discussion and decision-making.
• The complete and objective documentation of meetings significantly increases the completeness of information and the transparency of decisions.
• The use of AI makes it possible to quickly find specific information in the large amounts of data, e.g. to search for statements on a recurring topic in the meeting history.
• The AI software can also create the protocol in a more consistent and standardized way by using uniform formats, structures and terminology.

The lessons we learned from the experiment with AI-generated meeting notes range from technical, content-related and administrative to cultural aspects. In the following, we list the specific recommendations for action that have emerged from our application experience:

• Clarify the legal and administrative aspects, especially for companies with a works council. Section 87 of the Works Constitution Act regulates cases in which the works council may co-determine and, in case of doubt, also give a veto. This should definitely be communicated and checked together in advance.
• The company’s IT department should check the relevant default settings in the meeting software. Among other things, organizations must allow the transcription of appointments and, if necessary, adjust guidelines or rights for users with the IT department.
• Don’t underestimate the impact on the company culture. Employees also need to be comfortable with the technology and adapt their meeting culture. In addition to general etiquette, letting each other speak out becomes all the more important to ensure high quality AI-generated minutes.
• Organize the meetings so that all participants are logged in with their own account and their own technical equipment. If this is not possible, bear in mind that the AI cannot clearly assign the conversation contributions to the individual persons in shared accounts.
• Stick to one language within a meeting. If a language change is required, adjust the settings in the options.
• Avoid project- or company-specific jargon or explain the specific terms or abbreviations in each case. One workaround is to pronounce the abbreviations literally.
• If possible, use the suggested prompts that can be selected directly in the Copilot Teams connection. In our experience, these often deliver better results than self-formulated prompts. It would therefore appear that the model is specialized in these ready-made commands.
• Speak clearly, simply and distinctly – this is not only an advantage for speech recognition.
• Apparently, MS Copilot prefers certain terms for its prompts. For example, it likes to use the terms «Key Topics» and «Action Items» and understands «Meeting Notes» better than «Meeting Minutes». Gather your own experience here.

The tips listed here are naturally not exhaustive, as we have not been able to examine all conceivable use cases and AI applications are constantly evolving. We would be delighted if you could share your own experiences and recommendations with us in the comments below.

We rate the use of AI for automatic meeting recording as overwhelmingly positive and see it as an innovative and useful application of artificial intelligence in project management. In our experience, there are several advantages for project managers and organizations in general, which are offset by only a few disadvantages and challenges.

In our view, the three most important arguments in favor of using this function are:

• Communication, tracking and transparency of meeting minutes no longer depend on the diligence and precision of individuals.
• The time and effort required to prepare and follow up meeting minutes as well as the time and effort required during the meetings is considerably reduced, resulting in a significant efficiency advantage.
• Project managers can focus more on the content and strategic aspects of their projects and thus make optimal use of their time and energy.

If you would like to learn more about the usage of AI for your business, you can check out our page about Artificial Intelligence or approach our Head of Artificial Intelligence Dr. Marc Großerüschkamp directly. For business inquiries regarding Project Management please contact our Senior Consultant in Project Management Marc Bollmann.

To read the full article in Project Management Magazine, please visit this site.

Author

Contact person

Resources