Creation and validation of a bilingual test to estimate aural and written vocabulary size

  1. Aoiz, Martín
Journal:
Porta Linguarum: revista internacional de didáctica de las lenguas extranjeras

ISSN: 1697-7467

Year of publication: 2022

Issue: 38

Type: Article

DOI: 10.30827/PORTALIN.VI38.23606 DIALNET GOOGLE SCHOLAR lock_openDialnet editor

More publications in: Porta Linguarum: revista internacional de didáctica de las lenguas extranjeras

Sustainable development goals

Abstract

Language learners’ vocabulary size is a reliable predictor of their success in a second language as it clearly correlates with better performances in the target language. Being precise in those estimations is paramount to plan language teaching. However, the instruments employed by previous studies for those estimations might present validity and reliability issues that affect their research sensitivity and accuracy. This paper presents a step-by-step account of the creation of an aural and a written version of a bilingual vocabulary test. Test was delivered to 73 adult L1-Spanish students attending English classes. Their answers were analysed with Rasch model to determine the best performing items in the test so that the overall reliability of the instrument was enhanced. The final version of the test presents high levels of reliability: .89 for the listening vocabulary test and .82 for the written vocabulary test. Furthermore, descriptive statistics confirm that recognizing the words in their aural form is more challenging than in their written form: participants got 10.80% fewer correct answers in the listening vocabulary test. This finding confirms the claim that aural and written vocabulary are two separate dimensions, and impacts on how vocabulary should be taught in L2 classrooms.

Bibliographic References

  • Abbuhl, R., & Mackey, A. (2017). Second language acquisition research methods. In King, K. A., Lai, Y. J., & May, S. (Eds.). Research methods in language and education (3rd edition). (pp. 183-193). https://doi.org/10.1007/978-3-319-02249-9
  • Andringa, S., Olsthoorn, N., van Beuningen, C., Schoonen, R., & Hulstijn, J. (2012). Determinants of success in native and non‐native listening comprehension: An individual differences approach. Language Learning, 62(Suppl. 2), 49–78. https://doi.org/10.1111/j.1467- 9922.2012.00706.x
  • Beglar, D., & Nation, P. (2007). A vocabulary size test. The Language Teacher, 31(7), 9-13.
  • Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: fundamental measurement in the human sciences (3rd ed.). Routledge.
  • Cheng, J., & Matthews, J. (2018). The relationship between three measures of L2 vocabulary knowledge and L2 listening and reading. Language Testing, 35(1), 3-25. https://doi.org/10.1177/0265532216676851
  • Cobb, T. (2013). Frequency 2.0: Incorporating homoforms and multiword units in pedagogical frequency lists. In Bardel, C., Lindqvist, C., & Laufer, B. (Eds.) L2 vocabulary acquisition, knowledge and use: New perspectives on assessment and corpus analysis, (pp. 79-108). EUROSLA-the European Second Language Association. Retrieved from https://www.eurosla.org/
  • Cobb, T. (2019). Compleat Web VP v.2 [computer program]. Accessed on 16 Jan 2019 at https://www.lextutor.ca/cgi-bin/range/texts/index.pl
  • Crossley, S. A., Cobb, T., & McNamara, D. S. (2013). Comparing count-based and band-based indices of word frequency: Implications for active vocabulary research and pedagogical applications. System, 41(4), 965-981. https://doi.org/10.1016/j.system.2013.08.002
  • Hatch, E. M., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. Heinle & Heinle Publishers.
  • Hazenberg, S., & Hulstijn, J. H. (1996). Defining a minimal receptive second language vocabulary for non-native university students: An empirical investigation. Applied Linguistics, 17(2), 145-163. https://doi.org/10.1093/applin/17.2.145
  • Huang, H. T. (2010). How does second language vocabulary grow over time? A multi-methodological study of incremental vocabulary knowledge development (Doctoral dissertation, University of Hawai’i).
  • Karami, H. (2012). The development and validation of a bilingual version of the Vocabulary Size Test. RELC Journal, 43(1), 53-67. https://doi.org/10.1177/0033688212439359
  • Levitzky-Aviad, T., & Laufer, B. (2013). Lexical properties in the writing of foreign language learners over eight years of study: Single words and collocations. In Bardel, C., Lindqvist, C., & Laufer, B. (Eds.) L2 vocabulary acquisition, knowledge and use: New perspectives on assessment and corpus analysis, (pp. 127-148). EUROSLA-the European Second Language Association. Retrieved from https://www.eurosla.org/
  • Linacre, J. M. (1997). KR-20 / Cronbach alpha or Rasch person reliability: Which tells the "truth"? Rasch Measurement Transactions, 11(3), 580-581
  • Linacre, J. M. (2012). A user’s guide to Winsteps Ministeps Rasch-model computer programs [version 3.74.0]. Retrieved from http://www.winsteps.com/index.htm
  • Linacre, J. M. (2012, 2019). Winsteps® Rasch Measurement, version 4.4.3. [Computer software] Downloaded from http://www.winsteps.com
  • Martinez, R., & Schmitt, N. (2012). A phrasal expressions list. Applied Linguistics, 33(3), 299-320. https://doi.org/10.1093/applin/ams010
  • Masrai, A. (2020). Exploring the impact of individual differences in aural vocabulary knowledge, written vocabulary knowledge and working memory capacity on explaining L2 learners’ listening comprehension. Applied Linguistics Review, 11(3), 423-447. https://doi.org/10.1515/applirev-2018-0106
  • Mathias, C.W. (2010). Sensitivity. In Salkind, N. J. (Ed.). (2010). Encyclopedia of research design (Vol. 3), (pp. 1337-1338). Sage.
  • Matthews, J. (2018). Vocabulary for listening: Emerging evidence for high and mid-frequency vocabulary knowledge. System, 72, 23-36. https://doi.org/10.1016/j.system.2017.10.005
  • McLean, S., Kramer, B., & Beglar, D. (2015). The creation and validation of a listening vocabulary levels test. Language Teaching Research, 19(6), 741- 760. https://doi.org/10.1177/1362168814567889
  • Meara, P. M., & Miralpeix, I. (2006). Y_Lex: The Swansea advanced vocabulary levels test. v2. 05. Lognostics.
  • Nation, I. S. P. (2001) Learning vocabulary in another language. Cambridge University Press.
  • Nation, I. S. P. (2012, 2019). The BNC/COCA word family lists (17 September 2012). Unpublished paper. [online] Retrieved from http://www.victoria.ac.nz/lals/about/staff/paul-nation
  • Nation, I. S. P. (2016). Making and using word lists for language learning and testing. John Benjamins. https://doi.org/10.1093/applin/amx052
  • Nation, I. S. P., & Webb, S. A. (2011). Researching and analyzing vocabulary. Heinle, Cengage Learning.
  • Nguyen, L.T.C., & Nation, P. (2011). A bilingual vocabulary size test of English for Vietnamese learners. RELC journal, 42(1), 86-99. https://doi.org/10.1177/0033688210390264
  • Ockey, G. J., & Green, B. A. (Eds.). (2020). Another generation of fundamental considerations in language assessment: A festschrift in honor of Lyle F. Bachman. Springer Nature.
  • Ovtcharov, V., Cobb, T., & Halter, R. (2006). La richesse lexicale des productions orales: mesure fiable du niveau de compétence langagière. Canadian Modern Language Review, 63(1), 107-125.
  • Schmitt, N. (2008). Instructed second language vocabulary learning. Language Teaching Research, 12(3), 329-363. https://doi.org/10.1177/1362168808089921
  • Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212-226. https://doi.org/10.1017/S0261444815000075
  • Schmitt, N., Jiang, X., & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. The Modern Language Journal, 95(1), 26-43. https://doi.org/10.1111/j.1540-4781.2011.01146.x
  • Schmitt, N., Schmitt, D. (2012). A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484-503. https://doi.org/10.1017/S0261444812000018
  • Schmuckler, M. A. (2001). What is ecological validity? A dimensional analysis. Infancy, 2(4), 419-436. https://doi.org/10.1207/s15327078in0204_02
  • Silva, B. B., & Otwinowska, A. (2019). VST as a reliable academic placement tool despite cognate inflation effects. English for Specific Purposes, 54, 35-49. https://doi.org/10.1016/j.esp.2018.12.001
  • UCLES (2012). Cambridge English: Preliminary Wordlist. Retrieved from https://www.cambridgeenglish.org/Images/84669-pet-vocabulary-list.pdf
  • UCLES (2021). B1 Preliminary and B1 Preliminary for Schools Vocabulary List. Retrieved from https://www.cambridgeenglish.org/images/506887-b1-Preliminary-2020-vocabulary-list.pdf
  • van Zeeland, H. (2014). Second language vocabulary knowledge in and from listening (Doctoral dissertation, University of Nottingham).
  • van Zeeland, H., & Schmitt, N. (2013). Lexical coverage in L1 and L2 listening comprehension: The same or different from reading comprehension? Applied Linguistics, 34(4), 457-479. https://doi.org/10.1093/applin/ams074
  • Zhao, P., & Ji, X. (2018). Validation of the Mandarin version of the Vocabulary Size Test. RELC Journal, 49(3), 308-321. https://doi.org/10.1177/0033688216639761
Data source: Dialnet