Tuesday 6 June 2023

Speak&Improve and transcripts made and tidied up using AI


Speak&Improve

Speak&Improve is a sister project to Write&Improve and is a research project that is looking for data from non-native speakers. I hadn’t used it because I didn’t realise that I would be able to use it to get grades for some of the hundreds, if not thousands, of EFL student recordings I have. https://beta.speakandimprove.com/

Grading audio files

In his talk at InnovateELT 2023, Paul Ricketts said that there wasn’t a way to upload audio files for evaluation, but I knew there would be a way round it and soon discovered that by playing the audio file on my computer with the microphone in front of the speaker I was able to get grades for my students’ recordings. However, the quality was very poor, so I experimented with playing the audio file on one device and using Speak&Improve on another. The sound was much better and the grades went up!

Grading after 60 seconds

Speak&Improve only asks students to speak for sixty seconds before trying to assess their speaking, and I can confirm that no more time is needed or practical for a teacher assessing speaking by listening to it. I only listened to the first minute of my students recordings as otherwise it would have been too much work for me (I aimed at 5 minutes per recording) and too much feedback on problems for the students.

Reluctance to speak in class

Both Paul Ricketts and Georgie Thorman, who spoke on “Reducing foreign language speaking anxiety in our learners”, referred to this as a problem for language learners, but my own experience was quite the opposite as I asked my students to record themselves speaking to someone else from the very first day of class. The result was that many of them recorded themselves for more than an hour a month. Listening to a few of them the other day I was delighted that the recording caught the listeners saying, “Ah ha.” or “That’s typical.” – a sure sign that it was a genuinely communicative activity that was being recorded.

Georgie spoke about her own panic about speaking in a foreign language, but I remembered that my own experience was quite the opposite. After studying French for nine years in a fairly conventional way, which never involved expressing ourselves in the language, I found myself hitch-hiking around France and having to speak French to the drivers. I was delighted that I could communicate perhaps because it was a one-to-one situation, I don’t remember any FLSA. I just did it, and years later as my students were speaking in pairs 95% of the times, I’m sure that helped them get over any possible nerves about speaking.

Monologues are easier to grade

Although my students recorded themselves interviewing each other sometimes, I always asked them to suggest their best monologue of the week/fortnight for me to grade and Speak&Improve only grades monologues, so students can do it at home. However, I thought that using it in class a couple of times at the beginning might help them get into the habit of doing it out of class, and by asking everyone to write a reflection journal on something like SpacesEDO, or SeeSaw including a picture of the task and grade, it might be possible to make it simply part of their “Speaking Homework”, which was part of the title of Paul’s talk.

Alternatives to Speak&Improve

I was disappointed that S&I didn’t show students transcripts of their recordings as I feel seeing it might make it easier to repeat the same task better a second or third time. I had experimented six months ago with using Otter.ai to generate transcripts from my students’ recordings. I have also used Dictate, Transcribe in Word to generate transcripts. Then asking ChatGPT or Bing to “tidy up” the transcript resulted in excellent versions that could serve as a model for future attempts at the task, particularly after listening to it read aloud by Word for example. In an earlier post I suggested a way students could practise voice-typing each sentence

I did use Speak&Improve to grade the recordings as well because as far as I know there isn’t any other website that offers to do this for free.

The six recordings I used to experiment with

There are six recordings below that are in chronological order for each of the two students, Dolors, and Lidia. For each there is:

  • ·         the Title,
  • ·         the date,
  • ·         the grade assigned by Speak&Improve,
  • ·         the word-for-word transcript of the first 60 seconds (the part that was graded),
  • ·         the rest of the original transcript,
  • ·         the tidied-up transcript of the first minute,
  • ·         the rest of the improved transcript,
  • ·         some comments about the differences I have highlighted.