ASSESSING EFL SPEAKING BASED ON RECONSTRUCTED CAF MEASURES

Author: Dr. Reem Fahad Alshalan

ABSTRACT

In applied linguistics research, complexity, accuracy, and fluency (CAF) were based on the importance of task-based activities, task familiarity, communicative competence, or sociolinguistic factors. However, investigating CAF as an independent variable to assess language performance is controversial in research. Some studies found that language cannot be assessed in these dimensions alone since language is complex, changing, and dependent on many factors. Further research shed light on the importance of reconstructing these variables. The purpose aims to test the validity of a speaking assessment scheme based on previous research suggestions to reconstruct CAF measures by comparing it to the TOEFL speaking rubric. There were no statistical differences in the results of the participants in both assessment schemes. The implications are to further design valid assessment schemes based on reconstructed CAF measures, using an automated application that can transcribe and compute these measures instead of calculating them manually.

Keywords: Assessment Schemes, CAF Measures, EFL Speaking, Language Performance.

REFERENCES

  • Allen, S., & Knight, J. (2009). A Method for Collaboratively Developing and Validating a Rubric. International Journal for the Scholarship of Teaching and Learning, 3(2), n2. https://doi.org/10.20429/ijsotl.2009.030210
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2e éd.). Newbury Park, É. U. Sage. https://doi.org/ 10.1016/b978-0-12-179060-8.50006-2
  • De Jong, N., & Vercellotti, M. L. (2016). Similar prompts may not be similar in the performance they elicit: Examining fluency, complexity, accuracy, and lexis in narratives from five picture prompts. Language Teaching Research, 20(3), 387-404. https://doi.org/10.1177/1362168815606161
  • Ellis, R. (2012). Language teaching research and language pedagogy. John Wiley & Sons. https://doi.org/ 10.1002/9781118271643
  • Ellis, R., & Barkhuizen, G. P. (2005). Analysing learner language. Oxford: Oxford University Press.
  • ETS. (2014). TOEFL iBT test-Independent Speaking Rubric (Scoring Standards). Priceton, NJ: Author. Retrieved from https://www.ets.org/s/toefl/pdf/toefl_speaking_rubrics.pdf.
  • Foster, P., & Skehan, P. (1996). The influence of planning and task type on second language performance. Studies in Second language acquisition, 18(3), 299-323. https://doi.org/ 10.1017/s0272263100015047
  • Foster, P., Tonkyn, A., & Wigglesworth, G. (2000). Measuring spoken language: A unit for all reasons. Applied linguistics, 21(3), 354-375. https://doi.org/10.1017/s0272263100015047
  • Housen, A., & Kuiken, F. (2009). Complexity, accuracy, and fluency in second language acquisition. Applied linguistics, 30(4), 461-473. https://doi.org/10.1093/applin/amp048
  • Housen, A., Kuiken, F., & Vedder, I. (Eds.). (2012). Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (Vol. 32). John Benjamins Publishing. https://doi.org/10.1075/lllt.32.01hou
  • Hunt, K. W. (1965). Grammatical Structures Written at Three Grade Levels. NCTE Research Report No. 3.
  • Jamieson, J., & Poonpon, K. (2013). Developing Analytic Rating Guides for Toefl Ibt’s Integrated Speaking Tasks. ETS Research Report Series, 2013(1), i-93. https://doi.org/ 10.1002/j.2333-8504.2013.tb02320.x
  • Larsen-Freeman, D. (2006). The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English. Applied linguistics, 27(4), 590-619. https://doi.org/10.1093/applin/aml029
  • Larsen-Freeman, D. (2009). Adjusting expectations: The study of complexity, accuracy, and fluency in second language acquisition. Applied linguistics, 30(4), 579-589. https://doi.org/10.1093/applin/amp043
  • Larson-Hall, J. (2015). A guide to doing statistics in second language research using SPSS and R. Routledge. https://doi.org/ 10.4324/9780203875964
  • Mackey, A., & Gass, S. M. (2016). Second language research: methodology and design. New York and London. https://doi.org/ 10.4324/9781410612564
  • Norris, J. M., & Ortega, L. (Eds.). (2006). Synthesizing research on language learning and teaching (Vol. 13). John Benjamins Publishing. https://doi.org/ 10.1075/lllt.13
  • Pallotti, G. (2009). CAF: Defining, refining and differentiating constructs. Applied linguistics, 30(4), 590-601. https://doi.org/10.1093/applin/amp045
  • Purpura, J.E. (2017). Assessing Meaning: Language Testing and Assessment. In Encycolpedia of Language and Education, pp. 33-61. https://doi.org/10.1007/978-3-319-02261-1_1
  • Robinson, P. (2001). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied linguistics, 22(1), 27-57. https://doi.org/10.1093/applin/22.1.27
  • Sample, E., & Michel, M. (2014). An exploratory study into trade-off effects of complexity, accuracy, and fluency on young learners’ oral task repetition. TESL Canada Journal, 23-23. https://doi.org/ 10.18806/tesl.v31i0.1185
  • Skehan, P. (1998). A cognitive approach to language learning. Oxford University Press. https://doi.org/10.1177/003368829802900209
  • Skehan, P. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied linguistics, 30(4), 510-532. https://doi.org/10.1093/applin/amp047
  • Vercellotti, M. L. (2012). Complexity, accuracy, and fluency as properties of language performance: The development of multiple subsystems over time and in relation to each other (Doctoral dissertation, University of Pittsburgh).
  • Wulff, S., & Gries, S. T. (2011). Corpus-driven methods for assessing accuracy in learner production. Second language task complexity: Researching the cognition hypothesis of language learning and performance, 61, 87. https://doi.org/10.1075/tblt.2.07ch3