Improving the pronunciation prompt in my Language Coach Gem (and my Prompt for Gemini)

 

While attending the Automated L2 Speaking Assessment (AL2SA) International Workshop 2026, I asked Gemini to give me a verbatim transcription of a recording and there ensued a conversation about how Gemini could help learners with very specific pronunciation problems. I liked what I read and asked Gemini to design a prompt to apply the ideas we had discussed to improve my Language Coach prompt for use with Gemini. This is Gemini’s suggestion:

3 Pronunciation (The Sound Spotlight): Identify the top 2 issues.

Show the "Sound Spotlight": Pick a word I said and show how I said it vs how a native speaker says it (e.g., "You said 'so' but it sounds like 'saw'").

Give a "Physical Tip": Use simple descriptions of how to move the mouth or tongue (e.g., "Keep your jaw low and your mouth open wide") instead of technical terms.

 

This was to replace the previous version’s section on pronunciation, which was:

3 Pronunciation: Identify the top 2 issues. Use simple descriptions (e.g., "The 'H' sounds like a breath") instead of technical terms.

Here are three examples of the feedback Gemini produced with this improved pronunciation step:

1 Rob and Jenny and the chocolates
https://youtu.be/o-6RzZqxskU

Pronunciation (The Sound Spotlight)

Here are the top two areas to focus on from your recording:

The "S" at the end of words: In English, we use the -s for plurals (chocolates) and third-person verbs (she says). Sometimes this sound was missing in your recording.

The "O" in "Chocolates": This word can be tricky because we don't pronounce every letter.

 

The Sound Spotlight: "Chocolates"

What you said: It sounded like you were trying to say every syllable, like "chock-oh-lay-tes."

How a native speaker says it: It sounds like "chock-letz." The middle "o" is silent, and the "a" sounds like a short "e" or "i."

Physical Tip

To say "chock-letz" naturally:

The first part: Open your mouth wide and drop your jaw for the "chock" sound.

The jump: Quickly move your tongue to the roof of your mouth for the "l" and don't make any sound for that middle "o."

The end: Keep your teeth close together and blow air out for the "ts" sound at the end. It should sound like a little hiss.

2 Bargain Sweater

https://youtu.be/TL1nkW0s9YY

Let's look at how you pronounced some of the words. Overall, you were very clear, but here is your Pronunciation Spotlight.

3. Pronunciation (The Sound Spotlight)

• The Issue: Many Spanish speakers add an "e" sound before words starting with "s," and I noticed this with "sweater" and "sales."
• The Issue: The word "bought" was a bit tricky. It sounded a little like "boat."

---

Sound Spotlight: "Bought"

·         You said: It sounded like "boat" (with a long 'o' sound like in "no").

·         A native speaker says: /bɔːt/. It should have a deep, open "aw" sound, like in the word "law."

Physical Tip

To get the "aw" sound in bought:

·         Drop your jaw down low.

·         Keep your tongue flat at the bottom of your mouth.

·         Make your mouth into a tall O-shape (not a wide one).

·         It should feel much "deeper" in your throat than the word "boat."

Sound Spotlight: "Sweater"

·         You said: "Es-weater" (adding an 'e' at the start).

·         A native speaker says: /ˈswetə/. It starts directly with the "s" sound, like a snake hissing.

Physical Tip

To avoid the extra "e" in sweater:

·         Close your teeth and let the air hiss out (sssss) before you move your lips to make the "w" sound.

·         Don't let your throat vibrate until you get to the "w."

·         Try saying "ssss-sweater" very slowly to practice.

3 My Plans for the Weekend
https://youtu.be/AlwycCUTQ4I

Great choice! Let's focus on the sounds of your speech. Here is your Sound Spotlight for the recording:

Top 2 Issues

·         The "th" sound: In the word "this," the beginning sounded a bit more like a "d" or a "z."

·         The "-ing" ending: In the word "staying," the end of the word was a bit short.

---

Sound Spotlight: "This"

·         How you said it: You said something like "dis."

·         How a native speaker says it: The "th" is soft and uses air.

·         Physical Tip: Put the tip of your tongue between your front teeth. Blow a little bit of air out while you make a sound with your throat. It should feel like a small vibration on your tongue.

---

Sound Spotlight: "Watching"

·         How you said it: You said "watch."

·         The "-ing" ending: In the word "staying," the end of the word was a bit short.

·         How a native speaker says it: We add a "ing" sound at the end.

·         Physical Tip: To make the "ng" sound, lift the back of your tongue to touch the top-back of your mouth. The air should come out of your nose, not your mouth.

 

Language Coach Prompt - letting students decide what kind of feedback they want

 

This is the prompt that is referred to in the video. It works with any language and the Listen or Read  Aloud option works well for English, Spanish and Italian, but not for Catalan, but there are workarounds. I cannot say how well Listen or Read Aloud works with other languages,  but would love to hear either way.

Language Coach Prompt

I am a language learner. I will upload audio files for you to analyse. Your goal is to be a helpful coach.

Core Communication Rules (Apply to EVERYTHING you say):

• Match My Level: You must use vocabulary and sentence structures that match the CEFR level of the audio I upload. If the audio is A2, your explanations and instructions must be A2.
• Language: If the audio is in English, use British English spelling and vocabulary at all times. If the audio is in another language, all responses should be in that language at the level of the recording from the very beginning
• No Jargon: Do not use academic or formal words (e.g., avoid "transitioned", "lexical", or "syntax"). Use simple, natural words a native speaker uses in daily life.
• Scannability: Use bullet points for clarity. Never use tables. Avoid long walls of text.
• Wait for Audio: Do not give any feedback or assessments until I upload a file and you know which language I speak and have heard my level.

The Process: When I upload a file, first ask if I want "Quick Feedback" or the "7-Step Sequence." If I choose the sequence, ask which step I want first. After every step, list the remaining options as briefly as possible.

The 7 numbered Steps:

1 Verbatim Transcript: Provide a transcript of exactly what I said in continuous prose.

2 Error Identification: Rewrite my text exactly as it is, but put brackets [ ] around any errors. Do not correct them yet.

3 Pronunciation: Identify the top 2 issues. Use simple descriptions (e.g., "The 'H' sounds like a breath") instead of technical terms.

4 Natural Correction: Provide a corrected version that is natural but NOT more sophisticated than my original.

5 Colloquial Version: Create a version that is slightly more casual/conversational. It should be less than half a CEFR level higher than my original. List 3 changes and explain them simply.

6 Advanced Version (Level +0.5): Create a version that is roughly half a CEFR level higher than my original. Focus on natural spoken language. List 3 specific changes and explain why they are a better "bridge" to the next level.

7 More Advanced Version (Level +1.0): Create a version of spoken language that is roughly half a CEFR level higher than the previous "Advanced" version (one full level above my original). List 3 changes and explain how they help me reach this higher level.

But if you want to see if there have been any updates, use this link:

Feel free to copy and paste this prompt into a free account with Gemini or experiment with your own variations. All you will need then are some audio files in one of these formats:

• MP3 (The standard format for most phones and voice recorders)

• WAV (High quality, but larger file sizes)1

• AAC / M4A (Common for iPhone "Voice Memos")2

• OGG

• FLAC

If a student records themselves on camera, these formats are supported:

• MP4 (The most common format for smartphones)

• MOV (Standard for Apple devices)

• AVI

• WMV

• WebM

Key Technical Limits

• File Size: Generally, files should be under 20MB for the best performance. If a video is too large, it is often better to convert it to audio (MP3) before uploading.

• Duration: For a detailed analysis, recordings between 1 and 3 minutes are ideal. This gives the AI enough data to find patterns without becoming overwhelmed.

SpeakSmart AI by Google's AI Studio looks good, but is it really?

I wondered how SpeakSmart AI arrived at the scores in its Analysis, so I asked it in the chat:

after 24 seconds, I got this reply:

The scores in the SpeakSmart AI analysis are generated by the Gemini 3 Pro model through a multi-modal analysis of your audio recording.

When you upload or record audio, the raw audio data is sent to the model along with a specific set of instructions (a prompt). The AI arrives at these scores by evaluating several characteristics of the speech:

1.      Pronunciation Score: The AI analyses the phonetic accuracy of the words spoken. it compares the phonemes (individual sounds) in your recording against standard models of spoken English to identify deviations, substitutions, or omissions.

2.      Clarity Score: This measures "articulatory precision"—how well-defined the boundaries between words are. The model looks for signs of mumbling, slurring, or audio interference that might make the speech difficult for a listener to parse.

3.      Pacing Score: The model evaluates the tempo and rhythm of your speech. It looks for a natural "flow," checking if the words are spoken too quickly (making it hard to follow) or too slowly (which can sound unnatural), and whether pauses are placed logically between sentences.

4.      Overall Score: This is a holistic assessment derived from the combination of the metrics above, reflecting the general effectiveness and professional quality of the communication.

Because Gemini is a "native multi-modal" model, it doesn't just transcribe the text and then grade it; it "listens" to the nuances of the audio signal itself—including tone, stress, and intonation—to provide feedback that goes beyond just the words used.

But I wasn't 100% convinced, so I used it to evaluate my own recordings in Catalan over a period of time, hoping that it would show a clear improvement, But It didn't.

I refuse to believe that that my speaking hasn't improved over these years.

Then I made a chance mistake - I asked for the same recording to be evaluated again and was disconcerted to see that the analysis was different, and what's more The Strengths, General Advice, Specific Improvements were also all different.

 

So, it seems that the scores are quite arbitrary. The Strengths always covered three from a total of four different points, but they were all useful comments. the General Advice always started with a positive comment and then showed two of the three variations of points to concentrate on.

Similarly, under Specific Improvements there was only one point covered by all three and then two covered the same problem with past tenses, but the third didn't. Two of them corrected a misused word from Spanish, which the third ignored, There were five other points that were covered in one or other version of the feedback, and the most remarkable correction was pointing out that the famous Catalan actor is in fact called Juanjo Puigcorbé and not Juan Puigcorbé!

In the Transcript, there were also a number of differences. The first version included five words that had been corrected including the misused word from Spanish, which had been rendered as the Catalan equivalent and so didn't appear in the list of Specific Improvements.

This was all done in the Catalan adaptation of the SpeakSmart AI app, so decided to check whether the same things happens with a recording in English with the original app.

I used the same recording made in class by a pre-intermediate student three times.

The first and most important problem is the arbitrary nature of the Analysis Results. They are not helpful at all and may give a totally false assessment of the student’s pronunciation.

First time	Second Time	Third Time
Analysis Results 45 Overall 50 Pronunciation 50 Clarity 35 Pacing	Analysis Results 52 Overall 55 Pronunciation 50 Clarity 45 Pacing	Analysis Results 40 Overall 45 Pronunciation 40 Clarity 35 Pacing

The third Transcript was very different from the first two as it included all the ums and uhs and so was like a verbatim transcript. Here is an extract:

"Hana was driving her car and suddenly one men across the... across round. Uh, after that, eh, she eh stop... the car just... [Spanish interaction: She stopped the car?]... Hana stopped eh the car and eh... she eh... see a man... uh, and the man eh was Jamie.

In the Strengths, there was only one similar phrase: “You successfully conveyed the main plot points of the story” together with some reference to self-correction.

In General Advice all three versions insisted on not translating (word-for-word) from Spanish, and on using the correct past forms of irregular (or regular) verbs. They each included one or two other useful suggestions.

The Specific Improvements were very similar and covered, albeit in different orders: 

• the pronunciation of the -ed ending in ‘stopped’
• the use of the false friend ‘history’ instead of ‘story’
• the fact that ‘explained’ is followed by ‘to’ if the person is included
• the correct past tense of ‘see’ is ‘saw’
• the use of the preposition ‘across’ instead of the verb ‘cross’

Despite the variations in the feedback, they were all valid and potentially useful, with the exception of the Analysis Results, which it would be better to remove.

Conclusions

I will challenge Google's AI Studio about the arbitrariness of the scores in the two versions of the app and prompt that the transcript should be verbatim. After these changes, I will repeat the test for consistency with another recording

I want to build a web app where i can upload an audio file and get feedback on my speaking including my pronunciation

SpeakSmart AI

Try it here your self

Try uploading an audio file made by a student and see what you think of the feedback.

My prompt for Google's AI Studio was, "I want to build a web app where i can upload an audio file and get feedback on my speaking including my pronunciation"

What happens when you ask AI Language Coach, "How would you grade my speaking on the CEFR scale?"

Learner:

How would you grade my speaking on the CEFR scale?

The video is only 1 1/2 minutes long

Coach:

That's a very natural question to ask. However, my specific role here as a language coach is to focus entirely on the process of improvement rather than assigning a formal CEFR grade.

There are a couple of key reasons for this approach:

1. Text vs. Audio: I am working from a written transcript of your speech. A true speaking assessment would require me to hear your actual voice to evaluate crucial elements like pronunciation, intonation, stress, rhythm, and fluency (how smoothly you speak, your use of pauses, etc.). These are just as important as grammar and vocabulary.
2. Coaching, Not Examining: My purpose is to guide you step-by-step on *how* to improve. The 9-step analysis is designed to show you a clear pathway, from fixing basic errors all the way up to using highly sophisticated language. The aim is to give you practical tools to level up, rather than just a label.

The best way to think of our sessions is that they demonstrate your progress. We start with your original text, and by the end, you have a version that operates at a much higher CEFR level. The journey through those nine steps is the most valuable part of the exercise.

I felt as if I was being told off for asking, "How would you grade my speaking on the CEFR scale?" And as I gave the instructions myself to Google's AI Studio about how it should design the AI Language Coach, maybe I shouldn't have been so surprised. Line 3 was, "Decide CEFR level (+ for half bands) but don’t reveal it. These are the specifications that I asked for: Instructions: Remove timestamps. Decide CEFR level (+ for half bands) but don’t reveal it. After each step (1–9), wait for “next.” Write in continuous prose. Answer questions at the learner’s level. Steps: 1 Identify errors in bold and enclose in [brackets]. 2 Correct errors, showing corrections in bold next to (errors). 3 Write the corrected version without mistakes. 4 Make it more colloquial and conversational, like two friends chatting. Mark changes next to original phrasing. 5 Write the changed version in full. 6 Rewrite it half a CEFR level higher, keeping it colloquial. Use more advanced vocabulary and idioms. Mark changes. 7 Write the updated version in full. 8 Rewrite it one CEFR level higher, keeping it lively, idiomatic, and expressive. Mark changes. 9 Write the final version in full.

AI Language Coach - Feedback on a B1 student retelling a story about the Maxwell Wildlife Park

I've made another video about how the AI Language Coach, which is still changing almost every day, could have provided valuable feedback for a student who made a recording about the Maxwell Wildlife Park. Sadly, the feedback is 10 years too late for her, but I think she would have benefitted from it, or at least the first seven steps.

The video is 13 minutes long and shows many of the 9 steps the AI Language Coach was asked to go through.

Step 1 - identifying the errors (not shown)

Step 2 - correcting the errors

              It is often useful and necessary to ask, "Can you explain these errors?"

Step 3 - reading and listening the corrected transcript Read Aloud

Step 4 - ways to make the transcript more colloquial (not shown)

Step 5 - reading and listening to a more colloquial version Read Aloud

Step 6 - ways to make the transcript 1/2 a CEFR level higher (not shown)

Step 7 - reading and listening to a more sophisticated version

Step 8 - ways to make the transcript another CEFR level higher (not shown)

Step 9 - reading and listening to an even more sophisticated version 

            - Asking for help with vocabulary during any of the nine steps (overarching)

            - Interacting with the AI Language Coach (I particularly like ...)

At the end I emphasise that, "It is only fair to say that this video has been edited to remove long pauses before the Read Aloud begins"

You can use and adapt the Google AI Studio app called 'AI Language Coach' I am working on

I've just uploaded two more videos about the Google AI Studio app called 'AI Language Coach' that I am working on. The great thing is that you can play with it, make a copy of it and make any changes you like to your copy. Here are the two videos:

And here is the link to the latest version which saves the feedback to your local device:

https://ai.studio/apps/drive/1qF_GxF-3cZJ-TuZBDBd8SMKeR5Yb5-Wa 

If you would prefer it not to save feedback to the local device, you can try this version:

https://ai.studio/apps/drive/1zNZ7V5ocmpVAnBi7xE1GbvTaI519eICD 

I've just added another feature that makes it possible for users to ask about the meaning of a word, which was possible before but was lost with the change to a button to move to the next step

Google's AI studio - you can make a sharable app in five minutes

I adapted Darren Coxon’s similar app here which gave very detailed feedback

AI Language Coach

Practice your English conversation skills with an AI-powered language coach. Get real-time spoken responses, live transcriptions, and receive brief, constructive feedback on your performance.

Last week I used Google's AI studio to produce this app. It isn't perfect but it only took about 10 minutes. My first version allowed you to have a conversation with a chatbot and gave you feedback on your English. The second version also allowed you the option to upload an audio file and get brief feedback on the English. I asked it to add an option to ulpload an audio file and went away to make some toast and a cup of coffee and it was ready when I got back!

These were the instructions I gave it originally:

Here are two examples of feedback it provided:

I'll try and find time to improve the feedback one day, but AI Studio is full of suggestions for ways to improve the app:

 

My 9-step prompt with Copilot + an interesting surprise

 

Bargain Sweater – experimenting with Turboscribe and Copilot

After Matthew Wemyss said on a webinar I was watching entitled AI Literacy – What Is It, Really? that ChatGPT was the worst LLM for young students to be using unsupervised, I decided to see if my 9-step prompt would work just as well on Copilot as on ChatGPT.

I used my ‘Bargain Sweater’ example on my Android phone, on my iPad and on my computer. It was so interesting that I have already made two videos of how that worked out. The one on my Android phone is already on YouTube, the iPad one just needs editing and the one on my PC, I haven’t even made yet.

There is ONE HUGE difference between how my 9-step prompt works with ChatGPT and Copilot. With Copilot when you ask for the different versions to be read aloud, you never know what you’ll get! Admittedly, Copilot usually reads out the written version verbatim prefaced by notes like (With an animated and bubbly tone), but sometimes it produces a much more colloquial version of the same text and even, on one occasion it reenacted the dialogue between the two women. Sadly, students don’t see these alternative versions written down and if they want to listen to them again, they may well just get the verbatim text or another colloquial version or even a recreation of the two women speaking!

To give you a flavour of the spoken language that Copilot sometime produces, here are a few examples:

1.      if you don't mind me asking?

2.      Go on have a guess.

3.      they were just chatting away

4.      let me tell you the compliments were flying.

5.      absolutely adored it

6.      they got on to the topic of how much it cost

7.      … was like oh why don't you guess

8.      nifty little 25% discount

      Here's the 11-minute video I made by recording the screen on my Android phone and commenting on it:

----------------------------------------------------------------------------------------------------------------------

NotebookLM now offers a 'video overview' as well as an 'audio overview'

The moment I heard this I wanted to try it out for myself so I tried it with the video on YouTube I had made when rehearsing for the JALT/EUROCALL joint webinar.

Here is the video of the rehearsal:

This is the summary created by NotebookLM:

Summary

This webinar rehearsal, led by Chris Fry, introduces educators to a method for providing instant speaking feedback to language students using AI tools. The primary focus is on Turboscribe AI for transcription and ChatGPT for comprehensive error analysis and improvement. Students, aged 13 and over with parental consent, can use free accounts to record their speech, receive a transcript, and then engage with a nine-step AI-powered feedback process. This process incrementally guides them from identifying and correcting basic errors to generating and listening to increasingly sophisticated versions of their spoken output, facilitating a gradual improvement in their language proficiency.

Here is the video overview created by NotebookLM based on just this one source:

Using an average of scores from Pearson's GSE Text analyzer and Text Inspector the transcript is about B2/B2+ 

Summary by NotebookLM

This source introduces an innovative, free two-part AI system that acts as a personal language tutor, designed to help learners achieve fluency in any language. The system addresses the common problem of lacking immediate feedback when practicing speaking independently, enabling users to receive instant, detailed feedback on their spoken language. This "AI language tutor" combines a transcription tool, like Turboscribe AI, to accurately convert speech into text, with a language model, such as ChatGPT, to provide comprehensive corrections and improvements. The process unfolds across three progressive levels: firstly, fixing obvious errors in grammar and structure; secondly, polishing phrasing to ensure natural, fluent expression; and thirdly, upgrading vocabulary to enhance expressiveness and sophistication. By systematically addressing these areas, the AI not only corrects mistakes but also significantly boosts a learner's confidence and control over their language acquisition journey, working for over 90 languages.

I then wondered how the video overview might change if I added two more sources.

1. My nine-step prompt revised to produce a smoother transition between the different levels of suggested improvements. 
   
   Summary of source by NotebookLM

   This source outlines a nine-step process for refining transcriptions, likely from an AI tool like Turboscribe, by systematically improving their quality. The core purpose is to elevate a text from its initial flawed state to a highly sophisticated version, moving through incremental levels of linguistic complexity and coherence as defined by the GSE and CEFR frameworks. Each step involves identifying and correcting errors, then progressively rewriting the text to incorporate more advanced grammar, richer vocabulary, enhanced logical structure, and greater detail, with all changes meticulously marked to show the evolution of the transcription.

2. A pdf showing the different steps and how to introduce them in different stages:

Summary of source by NotebookLM

This document outlines a nine-step process designed for improving transcription quality by systematically refining text. The methodology progresses through several "stages," beginning with identifying and correcting errors in an initial transcription. Subsequent stages focus on enhancing the language and structure of the text, first by rewriting half a CEFR level up with improved cohesion and vocabulary, then by further elevating the complexity, detail, and logical flow of the prose. The overarching purpose is to provide a structured approach for achieving increasingly sophisticated and error-free transcriptions.

Perhaps surprisingly the video overview based on three sources was actually shorter:

Using an average of scores from Pearson's GSE Text analyzer and Text Inspector the transcript is about B2 

Summary by NotebookLM

This source outlines an innovative method for transforming a personal device into an AI language tutor, offering immediate and tailored feedback on spoken language. It details a system leveraging two free tools, Turboscribe for accurate transcription and ChatGPT as the expert AI coach, to help learners progress from basic corrections to sophisticated expression. The core of this method is a "magic prompt" – a series of instructions that guides the AI through three stages: fixing errors, improving natural style, and levelling up language to more advanced forms. This personalised approach aims to build confidence and enhance learning by providing instant, customised feedback based directly on the user's spoken input.