AudioPen is one of the better tools for giving learners feedback on their speaking

I was listening to this Podcast yesterday and picked up on one thing in particular. (It's also available on YouTube )

Like me, Luke, quickly latched on to the idea of students using AudioPen to get feedback on their speaking. I've been experimenting with 15 systems that offer to do the same thing, using the same recording made by one of my students with all of them, with a view to writing up a comparison. Meanwhile, here is the data for AudioPen, which is one of the better ones. Here is what it did:

Original Transcript AudioPen	AudioPen polished version
Dolphins save swimmers from a shark attack. A lakeward rope lives when he is with his daughter and with daughter's friends and they were swimming in front of the beach and seven dolphins appear and start to swim with us around them but there are two dolphins that push Bob and far away there are two girls swimming with the rest of dolphins suddenly he could see another huge fish 3 meters long it was a shark and it feels nervous but he decided don't tell anything to two girls but he realized that the dolphins push the two girls together and start to swim around them 40 minutes at the end the shark swam away and all the people were safe Bob said that he swam with dolphins 3 or 4 times but he couldn't see this behaviour in dolphins and an expert told him that there are other things that in other seas happen this expert explained that when the dolphins feel that the humans are in danger have the sense of protecting them (176 words. errors not included in the polished version	Dolphins Shield Swimmers from Shark Dolphins save swimmers from a shark attack. A man named Bob was swimming with his daughter, her friends, and seven dolphins at the beach. Two of the dolphins pushed Bob away while two girls continued to swim with the remaining dolphins. Suddenly, Bob saw a big 3-meter-long shark nearby. He felt scared but did not tell the girls about it. The dolphins brought the two girls closer together and swam around them for 40 minutes until the shark left. Bob had swum with dolphins before but never saw this kind of behaviour from them. An expert told him that when humans are in danger, some friendly sea creatures like these intelligent animals will try to protect people by keeping any threats away. (127 words B2+ 54%) corrections suggested improvements
An interested student might learn quite a lot from the corrections and the suggested improvements. For example, in this corrected and improved sentence,  "Two of the dolphins pushed Bob away while two girls continued to swim with the remaining dolphins.”  a student could learn the items listed below. All of these would be useful for the B1 student who made the original recording and if they use the Edge browser they can use Read Aloud to listen to the improved version, as can you. ·         Two of the dolphins ·         pushed ·         pushed Bob away ·         while ·         continued to swim ·         the remaining dolphins

Some of the other 14 systems offer improved versions of this student's transcript that are too sophisticated for this student, who was at level B1 on the CEFR scale. AudioPen produced a version that Text Inspector rated at B2+ 54%. This is more likely to be of use to this student than all the ones producing versions at C1 or C1+.

I have a variant of Dictogloss where students speak instead of writing the story

My take on Dictogloss is this:

Instructions to students:

1.       Listen

2.       Listen and take notes

3.       If necessary, listen again while seeing the transcript

4.       Record yourself retelling the story to a partner

5.       Get some feedback on your recording from AI (or the teacher)

6.       Record yourself retelling the story to a new partner who hasn’t heard the story

7.       Get some more feedback on your recording from AI (or the teacher)

8.       Record some reflections on your performance

9.       Share the two last recordings with others (peers, teacher, parents)

I drafted this while I was watching Rachel talking about Dictogloss as there is a clear parallel between the original idea of Dictogloss and my variant, which I used every day in my pre-intermediate and intermediate classes for years. Here's a link to the recording:

Rachel Tsateri - Dictogloss

Posts of mine from 2015 on this kind of 'Dictogloss':

Students recording themselves in class about 200 times a year

Six steps to oral fluency

I have posted earlier this year about how AI can provide useful feedback based on machine-made transcripts of students' recordings:

Speak&Improve and transcripts made and tidied up using AI

How to use AI to get feedback on your speaking and learn from it

Further experiments with students recording themselves, getting a transcript and getting AI to tidy it up

Further experiments with students recording themselves, getting a transcript and getting AI to tidy it up

I want to be able to suggest a series of steps language learners can follow using just their mobile phones to get feedback on their speaking. 

Over the last four days I have been practising the steps using our granddaugher as a pre-intermediate learner of English. The procedure involved combining two of my obsessions: getting students to do extensive reading and getting them to record themselves retelling stories. 

1. I asked her to read an Oxford Bookworm Starter level reader on the Oxford Reading Club app
2. She then told me the story in her own words without recording herself
3. She then recordered herself retelling the story using 'Voice Capture' on Microsoft 365
4. Voice Capture generates a transcript automatically. I Shared the recording (and transcript) as a Word document
5. I copied and pasted the transcript into three AI webpages and asked for the transcript to be tidied up
6. I copied these 'improved' transcripts back into the Word document where with 'Read Aloud' she could listen to the suggested changes.
7. She selected which of the improved transcripts was the best and I copied and pasted it into WhasApp for her.
8. I then used WhatsApp to record 'Read Aloud' and add that so she could listen to it again.

Here's a short video I made and posted on YouTube:

My original idea was that this sequence would be followed by a repetition of the task of retelling the stories. However, I didn't use this idea, partly because my granddaugher's original recordings were about 9 minutes long, which is much much longer than my students' typical recordings of two to three minutes, but also because I was beginning to think that the exposure to up to five versions of her own story might be of sufficient benefit. I'm still open to suggestions on this.

Here are two of the AI suggestions - (with the Microsoft Edge browser you can select and right-click and choose 'Read Aloud' to listen to these versions)

ChatGPT suggested:

"The Mystery of Manor Hall" is about two brothers visiting their uncle's dark and mysterious house, which is very big. The brothers visit their uncle's house while their mother is buying a new house and getting everything in order. When they arrive at their uncle's house, he allows them to explore and enter all the places, except for the music room, which makes them curious.

During supper, the uncle points out the new moon and tells them they can't leave their room that night. Later, the sister wakes up and hears a strange sound. The brother listens and convinces his sister to go to the music room. They think they hear their uncle singing, but upon entering, they see a picture of a pirate. The pirate notices them, and they become frightened and attempt to leave the room, but the pirate prevents them from doing so.

The pirate explains his story to the brothers. He used to have a magic picture that allowed him to go sailing with his ship every new moon, but now the house is far from the sea, so he's unable to do so. Meanwhile, the uncle arrives and reveals his story. He had bought the house from an old man who warned him not to enter the music room due to a ghostly presence. The uncle had been afraid and avoided the room ever since.

Eventually, the two brothers move with their mom to the new house, which is located near the sea and took the magic picture. Every new moon, the pirate can now sail and sing quietly. And this is the end of the story.

Bing suggested:

The story is about two brothers who go to their uncle's house. The uncle's house is very dark, mysterious, and big. The brothers go to their uncle's house because their mother is buying a new house and needs to get everything in order. When the brothers arrive at their uncle's house, their uncle tells them that they can look for things and enter all the places in the house, except for the music room.

The brothers are confused because they can't go to the music room. Their curiosity gets the better of them and they try to enter the music room, but their uncle sees them. In the evening, while they are having supper, their uncle tells them to look at the new moon and that they can't leave their room that night.

In the middle of the night, the sister wakes up and tells her brother that she hears a strange sound. The brother listens and they both go outside their room. They don't want to, but they finally go to the music room. They think it's their uncle singing, but when they enter, they don't see anyone. When they turn on the lights, they see a picture of a pirate.

The pirate sees them and comes out of the picture. The brothers are very afraid and want to leave the room, but they can't because the pirate won't let them. Finally, the pirate explains his story to the brothers. He tells them that one day an old woman painted a picture of his ship and said it was a magic picture because every new moon he can come out of the picture and sail with his ship.

He could only do this if the picture was near the sea, but now the house isn't near the sea and he is sad because he can't sail. Then their uncle arrives and explains his story. He bought the house from an old man who told him not to enter the music room because 

How to use AI to get feedback on your speaking and learn from it

•  Record yourself doing a speaking task

• Use Word to get a transcript and add it to Column 1

• Manually correct the transcript as far as you can and add it to Column 2 (use tracking)

• Ask Bing AI Chat to "tidy up" your transcript and copy it to column 3

• Mark the mistakes in column 2 in red and the corrections in column 3 in green

• Copy what you have learnt into your notes (see below)

• Select the transcript in column 3, right-click on it and click on Read Aloud
• Practise reading this transcript aloud
• Try recording the story again without reading the transcripts
• Check you have got the five things to learn right this time

5 things to learn

• wait for someone

• Jenny is waiting for Rob in Central Park.

• have something to share/do/wear

• Both have good news to share.

• tell someone to do something
• Rob tells her to speak first.
• she/he doesn't need ...
• She doesn't need to move to London
• she tells her ... third person -s
• Jenny tells Barbara to delete her email.

Speak&Improve and transcripts made and tidied up using AI

Speak&Improve

Speak&Improve is a sister project to Write&Improve and is a research project that is looking for data from non-native speakers. I hadn’t used it because I didn’t realise that I would be able to use it to get grades for some of the hundreds, if not thousands, of EFL student recordings I have. https://beta.speakandimprove.com/

Grading audio files

In his talk at InnovateELT 2023, Paul Ricketts said that there wasn’t a way to upload audio files for evaluation, but I knew there would be a way round it and soon discovered that by playing the audio file on my computer with the microphone in front of the speaker I was able to get grades for my students’ recordings. However, the quality was very poor, so I experimented with playing the audio file on one device and using Speak&Improve on another. The sound was much better and the grades went up!

Grading after 60 seconds

Speak&Improve only asks students to speak for sixty seconds before trying to assess their speaking, and I can confirm that no more time is needed or practical for a teacher assessing speaking by listening to it. I only listened to the first minute of my students recordings as otherwise it would have been too much work for me (I aimed at 5 minutes per recording) and too much feedback on problems for the students.

Reluctance to speak in class

Both Paul Ricketts and Georgie Thorman, who spoke on “Reducing foreign language speaking anxiety in our learners”, referred to this as a problem for language learners, but my own experience was quite the opposite as I asked my students to record themselves speaking to someone else from the very first day of class. The result was that many of them recorded themselves for more than an hour a month. Listening to a few of them the other day I was delighted that the recording caught the listeners saying, “Ah ha.” or “That’s typical.” – a sure sign that it was a genuinely communicative activity that was being recorded.

Georgie spoke about her own panic about speaking in a foreign language, but I remembered that my own experience was quite the opposite. After studying French for nine years in a fairly conventional way, which never involved expressing ourselves in the language, I found myself hitch-hiking around France and having to speak French to the drivers. I was delighted that I could communicate perhaps because it was a one-to-one situation, I don’t remember any FLSA. I just did it, and years later as my students were speaking in pairs 95% of the times, I’m sure that helped them get over any possible nerves about speaking.

Monologues are easier to grade

Although my students recorded themselves interviewing each other sometimes, I always asked them to suggest their best monologue of the week/fortnight for me to grade and Speak&Improve only grades monologues, so students can do it at home. However, I thought that using it in class a couple of times at the beginning might help them get into the habit of doing it out of class, and by asking everyone to write a reflection journal on something like SpacesEDO, or SeeSaw including a picture of the task and grade, it might be possible to make it simply part of their “Speaking Homework”, which was part of the title of Paul’s talk.

Alternatives to Speak&Improve

I was disappointed that S&I didn’t show students transcripts of their recordings as I feel seeing it might make it easier to repeat the same task better a second or third time. I had experimented six months ago with using Otter.ai to generate transcripts from my students’ recordings. I have also used Dictate, Transcribe in Word to generate transcripts. Then asking ChatGPT or Bing to “tidy up” the transcript resulted in excellent versions that could serve as a model for future attempts at the task, particularly after listening to it read aloud by Word for example. In an earlier post I suggested a way students could practise voice-typing each sentence

I did use Speak&Improve to grade the recordings as well because as far as I know there isn’t any other website that offers to do this for free.

The six recordings I used to experiment with

There are six recordings below that are in chronological order for each of the two students, Dolors, and Lidia. For each there is:

• ·         the Title,
• ·         the date,
• ·         the grade assigned by Speak&Improve,
• ·         the word-for-word transcript of the first 60 seconds (the part that was graded),
• ·         the rest of the original transcript,
• ·         the tidied-up transcript of the first minute,
• ·         the rest of the improved transcript,
• ·         some comments about the differences I have highlighted.

My talk on Extensive Reading - the solution for everything! at InnovateELT2023

I really wanted to take advantage of the fact that I was going to give a presentation at a conference for the first time in a long time to try out some proofs of concept for an idea I got while watching some sessions from TESOL Spain 2023 which were available to watch live online. My original idea was to try to create a live (audio) podcast using Podbean, that was soon superseded by the idea of livestreaming it as a video using Vimeo. That turned out to involve paying for a premium account. So, in the end I opted for the idea of recording my PowerPoint presentation with only audio. But what could I do about the website I wanted to demonstrate? I downloaded ‘web-viewer’, an add-on for PowerPoint, but it didn’t allow me to show the pop-up windows.

In the end I decided to use Vimeo to create a recording of my slides and my demo of the website by recording my screen and I rehearsed doing it at home a couple of times.

 

Not wanting to waste the opportunity I bought a wireless clicker and a pair of wireless mics. The price of two mics was so little more than the price for one I thought it was worth it.

 

Sadly, on the day of my talk I didn’t complete all the steps to set up the screen recording using Vimeo, so at the end of my presentation I discovered that I hadn’t recorded it. So, my first proof of concept was a failure with the warning that it takes time and calm to set up screen recording when doing a live presentation.

The second ‘proof of concept’ was to see if I could share my slides and other documents I showed with everyone who came to my session, everyone who would have liked to come but chose a parallel session to go to, anyone in the organisation who wasn’t able to attend the conference and eventually anyone in the world who was interested.

I created a folder on my Google drive and shared the folder and its contents read only and created a short link  https://bit.ly/42SBkNS and a QR code using Bitly and created a folder with the name of my session. What I wanted to suggest was that presenters could be asked to copy their presentation materials into folders created by the conference organisers even if they were working on them up until the last minute.

In the folder for my talk, I put: the PowerPoint presentation, the catalogues of readers grouped by genres of a number of levels, a list of the genres I used and the ones OUP use, a questionnaire for students about their favourite genres and a link to Oxford Graded Readers Catalogue 2023, which is what I used to classify the readers on Oxford Reading Club by genres. The labelling with genres only applies to Domino and Bookworm readers.

I promised that I would include a link to the recording of my presentation on YouTube as well. Unfortunately, as I explained above, this will not be possible. However, I have a number of dry runs that I recorded to try to judge the timing right (unsuccessfully, as it turned out!) and I have added a link to the last of these. I also recorded a number of versions of my PowerPoint presentation and I have created another video including the slides I decided to eliminate to get the length down to 45 minutes. I have put a link to this in the folder, too.

My talk on the day didn’t attract as many people as I had hoped, but at least more that I had feared! I hope those who came, felt they had benefitted in some way. For my own part, preparing for and giving the talk was a wonderful learning experience. I researched more about extensive reading than ever before and learnt a huge amount, some of which I was able to share. But, in addition, I discovered how PowerPoint has evolved and now offers coaching (it seems I speak at the right speed – 125 words per minute and say ‘um’ and ‘er’ too often). I also learnt that it was possible to insert commentaries in the form of audio or video on any slide as you practise the presentation or afterwards.

ChatGPT - can check your pronunciation better than Bing with ChatGPT!

A further activity for Dolors would be dictate the same sentences herself and to do this I have asked Chat GPT to print the sentences one by one and wait for ‘me’ to voice-type the exact same sentence, before evaluating it and presenting the next sentence for me. (Remember I am using Microsoft Edge and the Edge extension Speech Agent)

Could you please print each sentence in the text below one by one and wait for me to try to voice-type the exact same sentence before evaluating it and presenting me with the next?

Could you please print each sentence in the text below one by one and wait for me to try to voice-type the exact same sentence before evaluating it and presenting me with the next? When Jane arrived at Rob’s house, she didn't find Rob, but she found Paul. She wasn't expecting him because she thought that he was leaving. She asked him what had happened and why he was now at Rob’s house. He said that he missed the company and the relationship with Paul, but that's no problem because he thinks that he will soon go back to London. Jane was very angry because of this answer and at this moment, Rob arrived with breakfast. He asked her if she wanted to have breakfast with Paul and him, but she was very angry and declined. After that, Rob asked Paul what had happened, and Paul said that he had only told Jane that he was soon coming back to London. Rob said that he never said that, and Paul responded that he thought it would be better for him to come back to London because he believed that his life in New York had changed him. Rob argued that it was his life and that he wanted to live in New York, and that he loved Jane. Paul bought a bus ticket for Rob for that afternoon because he thought it was better for him to leave.

Sure, here's the first sentence:

"When Jane arrived at Rob’s house, she didn't find Rob, but she found Paul."

When Jane arrived at Robs house, she didn't find Rob, but she found Paul.

Great! Here's the next sentence:

"She wasn't expecting him because she thought that he was leaving."

She wasn't expecting him because she thought he was leaving.

Good job! Here's the next sentence:

"She asked him what had happened and why he was now at Rob’s house."

She asked him what had happened and why he was now at robs House. (I've highlighted a mistake it missed, but...)

Great! Here's the next sentence:

"He said that he missed the company and the relationship with Paul, but that's no problem because he thinks that he will soon go back to London."

He said that he missed the company and relationship with Paul, but that's no problem because he thinks he will soon go back to London.

Excellent! Here's the next sentence:

"Jane was very angry because of this answer and at this moment, Rob arrived with breakfast."

Jane was very hungry because of this answer and at this moment Rob arrived with breakfast.

Close! The correct sentence is "Jane was very angry because of this answer and at this moment, Rob arrived with breakfast." (Speech Agent doesn't alawys get the commas right and ChatGPT is strict about this!) (ChatGPT detected this problem hungry/angry which I made a deliberate mistake with, but see later on...)

Here's the next sentence:

"He asked her if she wanted to have breakfast with Paul and him, but she was very angry and declined."

He asked her if she wanted to have breakfast with Paul and him, but she was very angry and declined.

Perfect! Here's the next sentence:

"After that, Rob asked Paul what had happened, and Paul said that he had only told Jane that he was soon coming back to London."

After that, Rob asked Paul what had happened and Paul said that he had only told Jane that he would soon coming back to London.

Almost perfect! Just a minor mistake. The correct sentence is "After that, Rob asked Paul what had happened, and Paul said that he had only told Jane that he was soon coming back to London." (Speech Agent doesn't alawys get the commas right and ChatGPT is strict about this!)

Here's the next sentence:

"Rob said that he never said that, and Paul responded that he thought it would be better for him to come back to London because he believed that his life in New York had changed him."

Rob said that he never said that.And Paul responded that he thought it would be better for him to come back to London because he believed that his life in Njc had changed him.

Good job! Here's the next sentence: (Strangely, New York has been dictated as Njc by Speech Agent and not detected by ChatGPT)

"Rob argued that it was his life and that he wanted to live in New York, and that he loved Jane."

Bob argue that it was his life and that he wanted to live in New York and that he loved Jane.

Close! The correct sentence is "Rob argued that it was his life and that he wanted to live in New York, and that he loved Jane."

Here's the next sentence:

"Paul bought a bus ticket for Rob for that afternoon because he thought it was better for him to leave."

Paul bought a bus ticket for Rob for that afternoon because he thought it was better for him to leave.

Great! That's correct. Is there anything else I can help you with?

Please write the complete text again and I will try to voice type it all by reading it as naturally as possible. When I have finished could you point out any errors you find and suggest why Speech Agent may have had a problem understanding me?

Sure, here's the full text:

"When Jane arrived at Rob’s house, she didn't find Rob, but she found Paul. She wasn't expecting him because she thought that he was leaving. She asked him what had happened and why he was now at Rob’s house. He said that he missed the company and the relationship with Paul, but that's no problem because he thinks that he will soon go back to London. Jane was very angry because of this answer and at this moment, Rob arrived with breakfast. He asked her if she wanted to have breakfast with Paul and him, but she was very angry and declined. After that, Rob asked Paul what had happened, and Paul said that he had only told Jane that he was soon coming back to London. Rob said that he never said that, and Paul responded that he thought it would be better for him to come back to London because he believed that his life in New York had changed him. Rob argued that it was his life and that he wanted to live in New York, and that he loved Jane. Paul bought a bus ticket for Rob for that afternoon because he thought it was better for him to leave."

Go ahead and try to voice-type it, and I'll let you know if there are any errors and provide feedback.

When Jane arrived at Robs house, she didn't find Rob, but she found Paul.She wasn't expecting him because she thought that he was leaving.She asked him what had happened and why he was now at robs House.He said that he missed the company and the relationship with Paul, but that's no problem because he thinks he will soon go back to London.Jane was very angry because this answer and at this moment Rob arrived with breakfast.He asked her if she wanted her breakfast with Paul and him, but she was very angry and declined.After that, Rob asked Paul what had happened and Paul said that he had only told Jane that he was coming back to London.Rob said that he never said that, and Paul responded that he thought it would be better for him to come back to London because he believed that his life in Njc had changed him.Robe argued that it was his life and that he wanted to live in New York and that he loved Jane.Paul bought a bus ticket for Rob for that afternoon and.Said it was better for him to leave.

Here are the corrections to your transcription:

• Instead of "hungry" it should be "angry" in the sentence "Jane was very angry because of this answer..." ( This is strange, because I didn't make that deliberate mistake in THIS exercise!)
  
• Instead of "her breakfast" it should be "to have breakfast" in the sentence "He asked her if she wanted her breakfast with Paul and him..."
• In the sentence "Paul said that he had only told Jane that he was coming back to London" it should be "soon coming back to London" instead of just "coming back to London".
• Instead of "Njc" it should be "New York".
• At the end of the text it should be "because he thought it was better for him to leave" instead of "and.Said it was better for him to leave".

It's possible that some of the errors were caused by misheard words or incorrect pronunciation of certain sounds. Also, speaking too quickly or not enunciating clearly can make it harder for speech recognition software to accurately transcribe the words. Keep practicing and speaking clearly, and you'll get better at voice typing!

 I found the advice given at the end absolutely heart-melting! Who could ask for a more charming teacher?