by Ana Paula Biazon Rocha
PronSIG’s Conference, Exploring effective pronunciation assessment practices in the English language classroom, on 01 Oct 2022, was an extremely successful and enriching event which shed light on the relevance of pronunciation assessment and how it can take place in our English lessons. The high level of presentations as well as participants’ engagement throughout the day were remarkable. Now it’s time to reflect on some key takeaways from it.
1. Intelligibility as the main focus of pronunciation assessment
Both plenary speakers, Talia Isaacs and Tanja Angelovska, highlighted that intelligibility has become the turning point in pronunciation assessment, guiding teachers and assessors into considering how intelligible a student/assessee is rather than how similar or different from a native speaker of English they might sound. This has to do with a fundamental change in paradigm in the field of L2 pronunciation where the Nativeness principle has been replaced by the Intelligibility principle (Levis, 2018). As Derwing and Munro (2015) explain, one can have a heavy accent and yet be extremely intelligible.
Similarly, Okim Kang and Kevin Hirschi reinforced that the focus on accuracy in L2 pronunciation is not a valid approach to pursue. Native speakers of English are not all the same and do not have the very same accent. Varieties among them abound, so why should one be assessed by how close their pronunciation sounds to a native speaker’s? Native speaker to whom? Under which circumstances?
Consequently, when assessing pronunciation, we should consider whether messages are getting across effectively, not whether one accent sounds easier or different than another. However, this does not mean that everything is to be accepted. It is fundamental to consider breaches in communication and how to tackle them.
2. Focus on intelligibility but measure comprehensibility
In the opening plenary, Talia Isaacs pointed out that in pronunciation assessment there might be an overlap between intelligibility and comprehensibility. Based on Derwing and Munro (2015), intelligibility has to do with how listeners understand L2 speech, and can be measured by the listeners transcribing what they understood, whereas comprehensibility relates to how easy or difficult it is for listeners to understand L2 speech, and can be measured by listeners using a rating scale. She mentioned that in high-stake exams such as IELTS and TOEFL, the assessment criteria refer to intelligibility but actually measure comprehensibility. In other words, a speaking examiner tends to assess how much they believe they have understood from the assessee’s speech. Therefore, comprehensibility is much more connected to personal perception, and everyone perceives things differently.
For the classroom, this means that when preparing learners for exams, teachers should help students understand such nuances and concentrate on comprehensibility when assessing their oral performance, which can be done through the use of rating scales. For instance, Isaacs, Trofimovich and Foote (2018) developed some L2 English comprehensibility global and analytic scales (p. 214-6), which can definitely improve teachers’ understanding of how comprehensibility is measured and how they can use such scales with their learners.
3. Assessing pronunciation features…but which features?
A recurrent remark made by most of the conference presenters was that in the assessment criteria of high-stake exams, when it comes to pronunciation, intelligible use of pronunciation features is key. But what does that mean exactly? In fact, pronunciation features refer to much broader concepts and numerous examples.
As explained in a previous blog post, pronunciation features are usually divided into the segmental and the suprasegmental level, and within each level different elements play an essential role in terms of intelligible speech. For this reason, narrowing down the features that are more relevant to a certain type of assessment can facilitate the assessor’s job, and most importantly, help the assessee understand the assessment criteria. Subsequently, this aids teachers to better instruct their learners. For example, if the pronunciation criterion lists word and sentence stress, intonation and linking sounds as the main features to be assessed, teachers and students can focus on these when preparing for a test/exam.
The same applies to the concept of pronunciation assessment itself. It seems more commonplace to rely on the general idea that ‘pronunciation is/will be assessed’ in a speaking task or in one of the parts of a language exam than specifying what exactly about pronunciation is being evaluated.
As for some guidance, Okim Kang mentioned her Hierarchical Priority in Pronunciation Features (2013), where she analysed the different pronunciation features that contributed to differences in raters’ judgement of non-native speakers’ oral performance in Cambridge ESOL General English Examinations (p. 12):
1. Stress and Pitch (31%)
2. Fluency (27%)
3. Segmentals (8%)
4. Tone choices (5%)
She indicated that these features can provide teachers with a basis to draw learners’ attention to the importance of intelligibility and better prepare them for high-stake speaking exams. Moreover, ‘this knowledge can be further applied to develop scoring criteria for L2 oral assessment’ (Kang, 2013, p. 13).
4. Dealing with Classroom Pronunciation Assessment (CPA)
In the closing plenary, Tanja Angelovska analysed the different aspects of Classroom Pronunciation Assessment (CPA), which range from learner-dependent factors, such as L1 transfer, proficiency, degree of inhibition, self-monitoring and self-assessment, and metaphonological awareness, to teacher-independent factors such as awareness about L1 transfer, rating experience, (lack of) pedagogical training, context/exposure and elicitation tasks. She mentioned that teachers should focus on three different types of assessment in the classroom: individualised, diagnostic and continuous.
Angelovska also listed some challenges and possible solutions to aid teachers with CPA:
As you can see, there is much to reflect on in terms of pronunciation assessment. It might look a bit overwhelming, but it is absolutely vital. As Tanja Angelovska pointed out, we teachers need more training and support when assessing our own students and preparing them for high-stake exams. This is what PronSig’s October Conference offered: a bridge between research and classroom practice on pronunciation assessment to help teachers help their learners, and such help is certainly invaluable for both, and for us at PronSIG as well!
Don’t forget to check our previous blog posts about pronunciation teaching as well as to follow PronSIG on social media and leave your comments below.
References
Derwing, T., & Munro, M. (2015). Pronunciation fundamentals: Evidence-based perspectives for L2 teaching and research. Amsterdam: John Benjamins.
Isaacs, T., Trofimovich, P., & Foote, J. A. (2018). Developing a user-oriented second language comprehensibility scale for English-medium universities. Language Testing, 35(2), p. 193–216.
Levis, J. (2018) Intelligibility, Oral Communication, and the Teaching of Pronunciation. Cambridge University Press.
Kang, O. (2013). Relative impact of pronunciation features on ratings of non-native speakers’ oral proficiency. In J. Levis & K. LeVelle (Eds.). Proceedings of the 4th Pronunciation in Second Language Learning and Teaching Conference. Aug. 2012. (p. 10-15). Ames, IA: Iowa State University.
