Microsoft Copilot v. ChatGPT

ChatGPT and Microsoft Copilot are simply different

I find it fascinating that the two AI systems I've used most understand different things when I use the same prompts. I wanted to check that my instructions for students on how to use SoundType AI were correct, so I set up my Android phone and my iPad to simulate a student recording themselves in class and played the 10-year-old recording made by a student on my PC.

The recording was 3 minutes long and I was worried that when the screens went dark that the recordings would be interrupted, but this didn't happen.

The transcripts were ready on both systems in under 2 minutes, but I needed to correct some errors in the transcriptions. For example, Jenny's was written as Janis. Editing is very easy in SoundType AI.

I then copied each transcript and pasted them into ChatGPT and Copilot and asked at the beginning "please correct my mistakes."

Below are the two conversations with the two flavours of AI I am most familiar with.

The two transcripts were different and I’ve underlined the differences. My Android phone was more verbatim and my iPad was more practical, but less exact.

ChatGPT 3.5		Microsoft Copilot
on my Android phone		on my iPad
Please correct my mistakes. Okay, the movie is a story for two persons. One of them is Jenny. She's from New York and she works in a magazine in New York. And the other person is Rob. He works in the same magazine but he lives in London. And Jenny goes to London to meet Rob. She has a very good relationship. And then when Jenny comes back to New York, she offers to Rob a job for one month. A temporary job. And Rob goes to New York and this is an opportunity for him because they can meet better each other. I don't know if it's right this sentence. And after this month, the magazine offers a job. A permanent job Uh, yes, I did a job. And Jenny helped Rob to rent an apartment for a friend, I don't know. A... They fall in love? They fall in love. Fall in love? They fall in love. Fall in love? They fall in love. They fall in love. They fall in love. We love. We love. No, together. I don't know what is the word. They are together. We live together. No, we live together. They live alone. He lives alone. But at the finale of the movie, they go to know the Jenny's parents, to meet the Jenny's parents. They go to meet. To meet. Okay. Not to know. To meet. That's enough.		please correct my mistakes. The movie is a story for two persons. One of them is Jenny. She is from New York and she works in a magazine in New York. And the other person is Rob. He works in the same magazine but he lives in London. Jenny goes to London and meets Rob. She has a very good relationship. And when she comes back to New York, she offers to go to work for one month. Rob goes to New York and this is an opportunity for him. Because they can meet each other better. And after this month, the magazine offers him a job. Yes, a permanent job. And Jenny helps Rob to rent an apartment or a flat. And when they fall in love, they fall in love. No, together. I don't know what is the word. They are in couple. Yes. They live alone. She lives alone. But at the finale of the movie, they go to meet the Jenny's parents. They meet with the parents. They go to meet. To meet. Ok. Not alone. Yes. Yes. That's enough.
Certainly. Here's a corrected version of your text: Okay, the movie is a story about two people. One of them is Jenny. She's from New York and works in a magazine there. The other person is Rob. He also works in the same magazine but lives in London. Jenny goes to London to meet Rob, and they have a very good relationship. When Jenny returns to New York, she offers Rob a temporary job for one month. Rob goes to New York, and this is an opportunity for them to get to know each other better. After this month, the magazine offers him a permanent job. Yes, I did a job. Jenny helped Rob to rent an apartment for a friend, I don't know. And... Do they fall in love? Yes, they fall in love. Fall in love? Yes, they fall in love. They fall in love? Yes, they fall in love. We love. We love. No, together. I don't know what the word is. They are together. They live together. No, we live together. They live alone. He lives alone. But at the end of the movie, they go to meet Jenny's parents, not to know. To meet. Okay. Not to know. To meet. That's enough.		Here is a corrected version of your text: “The movie is a story about two people. One of them is Jenny, who works for a magazine in New York. The other person is Rob, who also works for the same magazine but lives in London. Jenny goes to London and meets Rob, and they develop a good relationship. When she returns to New York, she offers to let Rob work there for one month, which is an opportunity for them to get to know each other better. After this month, the magazine offers Rob a permanent job, and Jenny helps him find an apartment. They fall in love and become a couple. At the end of the movie, they go to meet Jenny’s parents together.” I hope this helps! B1+ on TextInspector
The two interpretations of my identical prompts were very different and this may be due to the Custom instructions I use in ChatGPT. But I wanted to get from ChatGPT something like what Copilot had produced, so I asked, “Can you make that into a coherent story for a student at B1 level on the CEFR scale?”
Certainly. Here's a simplified and coherent version for a B1 level student: The movie tells a story about two people, Jenny and Rob. Jenny is from New York and works at a magazine there, while Rob also works in the same magazine but lives in London. Jenny goes to London to meet Rob, and they develop a good relationship. When Jenny returns to New York, she offers Rob a temporary job for one month. Rob comes to New York, and during this time, they have the chance to know each other better. After the month, the magazine offers Rob a permanent job. Jenny helps Rob find an apartment for a friend. Do they fall in love? Yes, they fall in love. They decide to live together. Although they initially live alone, towards the end of the movie, they visit Jenny's parents to meet them. And that's the story! B1+ on TextInspector		I’ve copied Copilot’s version here to make it easier to compare the two. Here is a corrected version of your text: “The movie is a story about two people. One of them is Jenny, who works for a magazine in New York. The other person is Rob, who also works for the same magazine but lives in London. Jenny goes to London and meets Rob, and they develop a good relationship. When she returns to New York, she offers to let Rob work there for one month, which is an opportunity for them to get to know each other better. After this month, the magazine offers Rob a permanent job, and Jenny helps him find an apartment. They fall in love and become a couple. At the end of the movie, they go to meet Jenny’s parents together.” I hope this helps! B1+ on TextInspector
Which works better? I liked the fact that ChatGPT made it easier to see the errors, and it used paragraphs, but it took an extra step to get an improved version at the student’s level. Sadly, ChatGPT also included some erroneous information. I wanted to listen to the improved version on my phone so I said (spoken) “Can you do that again?” and I was able to listen to another variation of the story but that still included the erroneous information.  So, I said (spoken) “That was great. Actually, they didn't live together. Rob lived in the flat that Jenny had helped him find, and she lived in her flat. They lived independently.”
This was the final version of the story from ChatGPT, which I was able to listen to.
Got it, thanks for clarifying. Here's the adjusted version: In the movie, we meet Jenny and Rob. Jenny, from New York, works at a magazine, and Rob, who also works at the same magazine, lives in London. Jenny travels to London to meet Rob, and they become good friends. When Jenny returns to New York, she offers Rob a temporary job for one month. Rob comes to New York, and during this time, they learn more about each other. After a month, the magazine offers Rob a permanent job. Jenny helps him find an apartment for a friend. Do they fall in love? Yes, they fall in love. They decide to continue living independently. Rob lives in the flat that Jenny helped him find, and she stays in her own flat. Although they stay separate, towards the end of the movie, they visit Jenny's parents to meet them. And that's the story! B1+ on TextInspector	I’ve copied Copilot’s version here again Here is a corrected version of your text: “The movie is a story about two people. One of them is Jenny, who works for a magazine in New York. The other person is Rob, who also works for the same magazine but lives in London. Jenny goes to London and meets Rob, and they develop a good relationship. When she returns to New York, she offers to let Rob work there for one month, which is an opportunity for them to get to know each other better. After this month, the magazine offers Rob a permanent job, and Jenny helps him find an apartment. They fall in love and become a couple. At the end of the movie, they go to meet Jenny’s parents together.” I hope this helps! B1+ on TextInspector

I plan to make two videos on YouTube showing all the steps in the process of using SoundType AI using each of my devices.

 

AI can help students improve their speaking

AI can help students improve their speaking and the first step is to get an accurate transcription.

Since last June (2023) I have studied more than 70 apps and webpages trying to find the best systems to help students get accurate transcripts for the recordings they do in class.

Students should be able to:

Use an iPhone or an Android phone to:

•          Record themselves
•          Listen to the recording
•          See the transcript quickly
•          Edit the transcript
•         Copy, export or share it
•       To Copilot or ChatGPT

In this final report, I only include systems that offer the above with a free account and note what exactly they offer along with the number of languages that could be used because, although I only taught English, I know that teachers of other languages might be interested in doing the same thing with their students.

I decided to include in this post only those systems (apps or web pages) that would provide transcript accuracies of 90% and above. Students can correct obvious errors in the transcripts before asking for help from Copilot or ChatGPT, which will mean transcripts will often be very accurate records indeed of what the student said.

All four offer a reasonable length for each recording and the total length of all recordings with a free account. They all also offer transcriptions in a wide range of languages. I have put them in order of accuracy in their transcriptions.

Here are the main details I noted for the four best speech to text systems I found:

The ‘Final Four’:

Address	https://soundtype.ai/en
Accuracy	96%
Number of Languages	90
What you get free	180 minutes/month Only the first 8 minutes of transcription per audio are accessible
Android 7.0 and up, iPhone iOS 12.1 or later

 

Address	https://app.transkriptor.com
Accuracy	96%
Number of Languages	60
What you get free	90 minutes of free transcription 5 minute limit + Earn 120 minutes if you install Transkriptor and leave a review on Google Play, App store, Chrome store, TrustPilot + Earn 90 minutes for each platform you share: Twitter or Facebook +Earn 180 minutes for each review on Capterra or G2
Android 5.0 and up, iPhone iOS 13.0 or later

 

Address	https://www.notta.ai/en
Accuracy	94%
Number of Languages	104
What you get free	120 minutes / month + 10 prompts
Android 7.0 and up, iPhone iOS 11.0 or later

 

Address	https://app.fireflies.ai
Accuracy	92%
Number of Languages	69
What you get free	800 minutes storage Unlimited transcription
Android 8.0 and up, iPhone iOS 13.0 or later

Many of these systems offer to provide summaries of one sort or another as well, but in view of the very limited control students have over the level of complexity of the language used by these systems, I decided that it was better to use Copilot or ChatGPT to ask for help with their language.

1. I first asked Copilot or ChatGPT for a corrected transcript to compare with the original by displaying them both side by side in columns of a table.
2. I also asked it to provide an improved version suitable for the level of the learner. If this prompt is spoken, I could hear it read aloud, too.
3. Finally, I asked for some brief notes to be used when retelling the story and recording it again.

AudioPen is one of the better tools for giving learners feedback on their speaking

I was listening to this Podcast yesterday and picked up on one thing in particular. (It's also available on YouTube )

Like me, Luke, quickly latched on to the idea of students using AudioPen to get feedback on their speaking. I've been experimenting with 15 systems that offer to do the same thing, using the same recording made by one of my students with all of them, with a view to writing up a comparison. Meanwhile, here is the data for AudioPen, which is one of the better ones. Here is what it did:

Original Transcript AudioPen	AudioPen polished version
Dolphins save swimmers from a shark attack. A lakeward rope lives when he is with his daughter and with daughter's friends and they were swimming in front of the beach and seven dolphins appear and start to swim with us around them but there are two dolphins that push Bob and far away there are two girls swimming with the rest of dolphins suddenly he could see another huge fish 3 meters long it was a shark and it feels nervous but he decided don't tell anything to two girls but he realized that the dolphins push the two girls together and start to swim around them 40 minutes at the end the shark swam away and all the people were safe Bob said that he swam with dolphins 3 or 4 times but he couldn't see this behaviour in dolphins and an expert told him that there are other things that in other seas happen this expert explained that when the dolphins feel that the humans are in danger have the sense of protecting them (176 words. errors not included in the polished version	Dolphins Shield Swimmers from Shark Dolphins save swimmers from a shark attack. A man named Bob was swimming with his daughter, her friends, and seven dolphins at the beach. Two of the dolphins pushed Bob away while two girls continued to swim with the remaining dolphins. Suddenly, Bob saw a big 3-meter-long shark nearby. He felt scared but did not tell the girls about it. The dolphins brought the two girls closer together and swam around them for 40 minutes until the shark left. Bob had swum with dolphins before but never saw this kind of behaviour from them. An expert told him that when humans are in danger, some friendly sea creatures like these intelligent animals will try to protect people by keeping any threats away. (127 words B2+ 54%) corrections suggested improvements
An interested student might learn quite a lot from the corrections and the suggested improvements. For example, in this corrected and improved sentence,  "Two of the dolphins pushed Bob away while two girls continued to swim with the remaining dolphins.”  a student could learn the items listed below. All of these would be useful for the B1 student who made the original recording and if they use the Edge browser they can use Read Aloud to listen to the improved version, as can you. ·         Two of the dolphins ·         pushed ·         pushed Bob away ·         while ·         continued to swim ·         the remaining dolphins

Some of the other 14 systems offer improved versions of this student's transcript that are too sophisticated for this student, who was at level B1 on the CEFR scale. AudioPen produced a version that Text Inspector rated at B2+ 54%. This is more likely to be of use to this student than all the ones producing versions at C1 or C1+.

I have a variant of Dictogloss where students speak instead of writing the story

My take on Dictogloss is this:

Instructions to students:

1.       Listen

2.       Listen and take notes

3.       If necessary, listen again while seeing the transcript

4.       Record yourself retelling the story to a partner

5.       Get some feedback on your recording from AI (or the teacher)

6.       Record yourself retelling the story to a new partner who hasn’t heard the story

7.       Get some more feedback on your recording from AI (or the teacher)

8.       Record some reflections on your performance

9.       Share the two last recordings with others (peers, teacher, parents)

I drafted this while I was watching Rachel talking about Dictogloss as there is a clear parallel between the original idea of Dictogloss and my variant, which I used every day in my pre-intermediate and intermediate classes for years. Here's a link to the recording:

Rachel Tsateri - Dictogloss

Posts of mine from 2015 on this kind of 'Dictogloss':

Students recording themselves in class about 200 times a year

Six steps to oral fluency

I have posted earlier this year about how AI can provide useful feedback based on machine-made transcripts of students' recordings:

Speak&Improve and transcripts made and tidied up using AI

How to use AI to get feedback on your speaking and learn from it

Further experiments with students recording themselves, getting a transcript and getting AI to tidy it up

Further experiments with students recording themselves, getting a transcript and getting AI to tidy it up

I want to be able to suggest a series of steps language learners can follow using just their mobile phones to get feedback on their speaking. 

Over the last four days I have been practising the steps using our granddaugher as a pre-intermediate learner of English. The procedure involved combining two of my obsessions: getting students to do extensive reading and getting them to record themselves retelling stories. 

1. I asked her to read an Oxford Bookworm Starter level reader on the Oxford Reading Club app
2. She then told me the story in her own words without recording herself
3. She then recordered herself retelling the story using 'Voice Capture' on Microsoft 365
4. Voice Capture generates a transcript automatically. I Shared the recording (and transcript) as a Word document
5. I copied and pasted the transcript into three AI webpages and asked for the transcript to be tidied up
6. I copied these 'improved' transcripts back into the Word document where with 'Read Aloud' she could listen to the suggested changes.
7. She selected which of the improved transcripts was the best and I copied and pasted it into WhasApp for her.
8. I then used WhatsApp to record 'Read Aloud' and add that so she could listen to it again.

Here's a short video I made and posted on YouTube:

My original idea was that this sequence would be followed by a repetition of the task of retelling the stories. However, I didn't use this idea, partly because my granddaugher's original recordings were about 9 minutes long, which is much much longer than my students' typical recordings of two to three minutes, but also because I was beginning to think that the exposure to up to five versions of her own story might be of sufficient benefit. I'm still open to suggestions on this.

Here are two of the AI suggestions - (with the Microsoft Edge browser you can select and right-click and choose 'Read Aloud' to listen to these versions)

ChatGPT suggested:

"The Mystery of Manor Hall" is about two brothers visiting their uncle's dark and mysterious house, which is very big. The brothers visit their uncle's house while their mother is buying a new house and getting everything in order. When they arrive at their uncle's house, he allows them to explore and enter all the places, except for the music room, which makes them curious.

During supper, the uncle points out the new moon and tells them they can't leave their room that night. Later, the sister wakes up and hears a strange sound. The brother listens and convinces his sister to go to the music room. They think they hear their uncle singing, but upon entering, they see a picture of a pirate. The pirate notices them, and they become frightened and attempt to leave the room, but the pirate prevents them from doing so.

The pirate explains his story to the brothers. He used to have a magic picture that allowed him to go sailing with his ship every new moon, but now the house is far from the sea, so he's unable to do so. Meanwhile, the uncle arrives and reveals his story. He had bought the house from an old man who warned him not to enter the music room due to a ghostly presence. The uncle had been afraid and avoided the room ever since.

Eventually, the two brothers move with their mom to the new house, which is located near the sea and took the magic picture. Every new moon, the pirate can now sail and sing quietly. And this is the end of the story.

Bing suggested:

The story is about two brothers who go to their uncle's house. The uncle's house is very dark, mysterious, and big. The brothers go to their uncle's house because their mother is buying a new house and needs to get everything in order. When the brothers arrive at their uncle's house, their uncle tells them that they can look for things and enter all the places in the house, except for the music room.

The brothers are confused because they can't go to the music room. Their curiosity gets the better of them and they try to enter the music room, but their uncle sees them. In the evening, while they are having supper, their uncle tells them to look at the new moon and that they can't leave their room that night.

In the middle of the night, the sister wakes up and tells her brother that she hears a strange sound. The brother listens and they both go outside their room. They don't want to, but they finally go to the music room. They think it's their uncle singing, but when they enter, they don't see anyone. When they turn on the lights, they see a picture of a pirate.

The pirate sees them and comes out of the picture. The brothers are very afraid and want to leave the room, but they can't because the pirate won't let them. Finally, the pirate explains his story to the brothers. He tells them that one day an old woman painted a picture of his ship and said it was a magic picture because every new moon he can come out of the picture and sail with his ship.

He could only do this if the picture was near the sea, but now the house isn't near the sea and he is sad because he can't sail. Then their uncle arrives and explains his story. He bought the house from an old man who told him not to enter the music room because 

How to use AI to get feedback on your speaking and learn from it

•  Record yourself doing a speaking task

• Use Word to get a transcript and add it to Column 1

• Manually correct the transcript as far as you can and add it to Column 2 (use tracking)

• Ask Bing AI Chat to "tidy up" your transcript and copy it to column 3

• Mark the mistakes in column 2 in red and the corrections in column 3 in green

• Copy what you have learnt into your notes (see below)

• Select the transcript in column 3, right-click on it and click on Read Aloud
• Practise reading this transcript aloud
• Try recording the story again without reading the transcripts
• Check you have got the five things to learn right this time

5 things to learn

• wait for someone

• Jenny is waiting for Rob in Central Park.

• have something to share/do/wear

• Both have good news to share.

• tell someone to do something
• Rob tells her to speak first.
• she/he doesn't need ...
• She doesn't need to move to London
• she tells her ... third person -s
• Jenny tells Barbara to delete her email.