Teaching English as a Second Language Quarterly (Formerly Journal of Teaching Language Skills)

Comparing confidence-based and conventional scoring methods: The case of an English grammar class

Document Type : Research Paper

Authors

1 Islamic Azad University, Shiraz Branch

2 Islamic Azad University, Shiraz Branch, English Department

Abstract

This study aimed at investigating the reliability, predictive validity, and self-esteem and gender bias of confidence-based scoring. This is a method of scoring in which the test takers receive a positive or negative point based on their rating of their confidence in an answer. The participants, who were 49 English-major students taking their grammar course, were given 8 multiple-choice tests during the semester. These tests were scored both conventionally and in a confidence-based manner, and the reliabilities of these two score sets were compared. Each score set was correlated with the final exam scores to compare their predictive validity. Gender and self-esteem bias of the confidence-based scores of the eight tests were also calculated. The results showed that there was no difference between the reliabilities of the two sets of scores. Confidence-based scores had better predictive validity than conventional scores, but this difference was not significant. Confidence-based scores were not biased against a specific gender and specific levels of self-esteem. The conclusion is that confidence-based scoring is as good as conventional scoring and the choice between these two scoring methods depends on the teacher’s discretion and the teaching context.

Keywords

References
Adams, T. M. & Ewen G. W. (2009). The importance of confidence in improving educational outcomes. Proceedings of the 25th annual conference on distance teaching and learning, pp. 1-5.
Ahlgren, A. (1969). Reliability, predictive validity, and personality bias of confidence-weighted scores. Proceedings from American Educational Research Association symposium “Confidence on Achievement Tests- Theory, Applications”. Retrieved from www.p-mmm.com/founders/AhlgrenBody.htm
Anderson, R. S. (1998). Why talk about different ways to grade? The shift from traditional assessment to alternative assessment. New Directions for Teaching and Learning, 74, 5-16.  
Barr, D. A. & Burke, J. R. (2013). Using confidence-based marking in a laboratory setting: A tool for student self-assessment and learning. The Journal of Chiropractic Education, 27(1), 21-26.
Ben-Shakhar, G. & Sinai, Y. (1991). Gender differences in multiple-choice tests: The role of differential guessing tendencies. Journal of Educational Measurement, 28(1), 23-35. 
Ben-Simon, A., Budescu, D. V. & Nevo, B. (1997). A comparative study of measures of partial knowledge in multiple-choice tests. Applied Psychological Measurement, 21(1), 65-88.
Bokhorst, F. D. (1986). Confidence weighting and the validity of achievement tests. Psychological Reports, 59, 383-386.
Cash, B., Mitchner, N. A., & Ravyn, D. (2011). Confidence-based learning CME: Overcoming barriers in irritable bowel syndrome with constipation. Journal of Continuing Education in the Health Professions, 31(3), 157-164.
Coopersmith, S. (1967). The antecedents of self-esteem. San Francisco: W. H. Freeman & Company.
Davies, P. (2002). There is no confidence in multiple-choice testing. Proceedings of the 6th International Computer-Aided Assessment Conference, Loughborough, pp. 119-130.
Ebel, R. L. (1979). Essentials of educational measurement (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
Echternacht, G. J., Boldt, R. F., & Sellman, W. S. (1972). Personality influences on confidence test scores. Journal of Educational Measurement, 9(3), 235-241.
Fahim, M., & Dehghankar, A. (2014). Towards fuzzy scores in language multiple-choice tests. International Journal of Language Learning and Applied Linguistics World, 6(2), 291-308.
Fani, T. (2009). The role of foreign language anxiety, motivation, and self-esteem in the accuracy of self-assessment of EFL students’ reading comprehension abilities. Unpublished master’s thesis, Allameh Tabataba’i University, Tehran, Iran.
Frary, R. B. (1988). Formula scoring of multiple choice tests (correction for guessing). Instructional Topics in Educational measurement, 7(2), 33-38.
Frederiksen, N., Glaser, R., Lesgold, A., & Shafto, M. G. (Eds.). (2013). Diagnostic monitoring of skill and knowledge acquisition. Routledge.‏
Gardner-Medwin, A. R. (1995). Confidence assessment in the teaching of basic science. Association for Learning Technology Journal, 3, 80-85.
Gardner-Medwin, A. R. (2006). Confidence-based marking: Towards deeper learning and better exams. In C. Bryan & K. Clegg (Eds.). Innovative assessment in higher education (pp. 141-149). London: Routledge.
Gardner-Medwin, A. R. & Gahan, M. (2003). Formative and summative confidence-based assessment. Proceedings of the 7th International Computer-Aided Assessment Conference, Loughborough, pp. 147-155.   
Gurney, P. W. (1988). Self-esteem in children with special educational needs. London: Routledge.
Hassmen, P. & Hunt D. P. (1994). Human self-assessment in multiple-choice testing. Journal of Educational Measurment, 31, 149-160.
Hopkins, K. D., Hakstian, A. R., & Hopkins, B. R. (1973). Validity and reliability consequences of confidence weighting. Educational and Psychological Measurement, 33(1), 135-141.
Hunt, D. P. (1993). Human self-assessment: Theory and application to learning and testing. In D. Leclercq & J. E. Bruno (Eds.). Item bank: Interactive testing and self-assessment (pp. 177-189). Berlin: Springer Verlag.
Hunt, D. P. (2003). The concept of knowledge and how to measure it. Journal of Intellectual Capital, 4(1), 100-113.   
Issroff, K. & Gardner-Medwin, A. R. (1998). Evaluation of confidence assessment within optional coursework. In M. Oliver (Ed.). Innovation in the evaluation of learning technology (pp. 169-179). London: London Press.
Jacobs, S. S. (1971). Correlates of unwarranted confidence in responses to objective test items. Journal of Educational Measurement, 8(1), 15-19.
Khan, K. S., Davies, D. A., & Gupta, J. K. (2001). Formative self-assessment using multiple true-false questions on the Internet: feedback according to confidence about correct knowledge. Medical Teacher, 23(2), 158-163.
Kurz, T. B. (1999). A review of scoring algorithms for multiple-choice tests. Paper presented at the annual meeting of the Southwest Educational Research Association, San Antonio. Retrieved from http://files.eric.ed.gov/fulltext/ED428076.pdf.
Lau, P. N. K., Lau, S. H., Hong, K. S., & Usop, H. (2011). Guessing, partial knowledge, and misconceptions in multiple-choice tests. Educational Technology and Society, 14(4), 99-110.
Lenney, E. (1977). Women’s self-confidence in achievement settings. Psychological Bulletin, 84(1), 1-13.
Omirin, M. S. (2007). Validity and reliability indices of three multiple-choice tests using the confidence scoring procedure. The Social Sciences, 2(1), 20-23. 
Pollock, C. W. (1997). Communicate what you mean: A concise advanced grammar (2nd ed.). New York: Longman.
Pugh, R. C., & Brunza, J. J. (1975). Effects of a confidence weighted scoring system on measures of test reliability and validity. Educational and Psychological Measurement, 35(1), 73-78.
Sazvar, A. (2003). The impact of self-esteem on authentic material use: A case of Iranian non-English major students/graduates. Unpublished doctoral dissertation. Allameh Tabataba’i University, Tehran, Iran.
Yen, Y. C., Ho, R. G., Chen, L. J., Chou, K. Y., & Chen, Y. L. (2010). Development and evaluation of a confidence-weighting computerized adaptive system. Educational Technology and Society, 13(3), 163-176.
 
Teaching English as a Second Language Quarterly (Formerly Journal of Teaching Language Skills)
Volume 33, Issue 4 - Serial Number 4
March 2015
Pages 123-152
Files
History
  • Receive Date: 25 November 2014
  • Revise Date: 11 July 2015
  • Accept Date: 12 July 2015
Share
How to cite
Statistics