The Impact of Item Difficulty on Judgment of Confidence—A Cross-Level Moderated Mediation Model
Abstract
:1. Introduction
2. Materials and Methods
2.1. Participants
2.2. Materials
2.3. Task Procedure
2.4. Statistical Analyses
3. Results
3.1. Calculating Variables
3.2. Descriptive Statistics and Correlational Analysis Results
3.3. Mediation Effect Analysis
3.4. Cross-Level Moderating Effect Inspection
4. Discussion
4.1. The Effect of Item Difficulty on JOC Magnitude
4.2. The Mediating Effects of Processing Fluency
4.3. Cross-Level Moderating Effect of Intelligence
4.4. Research Innovation and Deficiency
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Alter, Adam L., and Daniel M. Oppenheimer. 2009. Uniting the tribes of fluency to form a metacognitive nation. Personality and Social Psychology Review 13: 219–35. [Google Scholar] [CrossRef] [PubMed]
- Arnold, Michelle M., Kristin Graham, and Sinead Hollingworth-Hughes. 2017. What’s Context Got to Do with It? Comparative Difficulty of Test Questions Influences Metacognition and Corrected Scores for Formula-scored Exams. Applied Cognitive Psychology 31: 146–55. [Google Scholar] [CrossRef]
- Bahnik, Štěpán. 2019. Disfluent, But Fast. Experimental Psychology 66: 346–54. [Google Scholar] [CrossRef] [PubMed]
- Baranski, Joseph V., and William M. Petrusic. 1994. The calibration and resolution of confidence in perceptual judgments. Perception & Psychophysics 55: 412–28. [Google Scholar] [CrossRef]
- Boldt, Annika, and Sam J. Gilbert. 2022. Partially Overlapping Neural Correlates of Metacognitive Monitoring and Metacognitive Control. The Journal of Neuroscience 42: 3622–35. [Google Scholar] [CrossRef]
- Breit, Moritz, Martin Brunner, and Franzis Preckel. 2020. General intelligence and specific cognitive abilities in adolescence: Tests of age differentiation, ability differentiation, and their interaction in two large samples. Developmental Psychology 56: 364–84. [Google Scholar] [CrossRef]
- Brewer, Gene A., and Nash Unsworth. 2012. Individual differences in the effects of retrieval from long-term memory. Journal of Memory and Language 66: 407–15. [Google Scholar] [CrossRef]
- Çapan, Dicle, Terry Eskenazi, and Sami Gülgöz. 2022. Dishonesty in public reports of confidence: Metacognitive monitoring of memory conformity. Journal of Applied Research in Memory and Cognition. [Google Scholar] [CrossRef]
- Chen, Fangyan, Fengying Li, and Weijian Li. 2016. Effects of perceptual cues on metamemory monitoring and control. Advances in Psychological Science 24: 494–500. [Google Scholar] [CrossRef]
- Cichoń, Ewelina, Łukasz Gawęda, Steffen Moritz, and Remigiusz Szczepanowski. 2018. Experience-based knowledge increases confidence in discriminating our memories. Current Psychology 40: 840–52. [Google Scholar] [CrossRef] [Green Version]
- Clariana, Roy B., and Eunsung Park. 2021. Item-level monitoring, response style stability, and the hard-easy effect. Educational Technology Research and Development 69: 693–710. [Google Scholar] [CrossRef]
- De Martino, Benedetto, Stephen M. Fleming, Neil Garrett, and Raymond J. Dolan. 2013. Confidence in value-based choice. Nature Neuroscience 16: 105–10. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Dodonova, Yulia A., and Yury S. Dodonov. 2013. Faster on easy items, more accurate on difficult ones: Cognitive ability and performance on a task of varying difficulty. Intelligence 41: 1–10. [Google Scholar] [CrossRef]
- Griffin, Dale, and Lyle Brenner. 2004. Perspectives on Probability Judgment Calibration. In Blackwell Handbook of Judgment and Decision Making. Hoboken: Blackwell Publishing Ltd., pp. 158–77. [Google Scholar]
- Hertzog, Cheistopher, Jarrod C. Hines, and Dayna R. Touron. 2013. Judgments of Learning are Influenced by Multiple Cues In Addition to Memory for Past Test Accuracy. Archives of Scientific Psychology 1: 23–32. [Google Scholar] [CrossRef]
- Iida, Ryosuke, Yukio Itsukusima, and Eric Y. Mah. 2020. How do we judge our confidence? Differential effects of meta-memory feedback on eyewitness accuracy and confidence. Applied Cognitive Psychology 34: 397–408. [Google Scholar] [CrossRef]
- Ismat, Shaista, and Junaid Sagir Sidiqui. 2015. A study of intelligence measure using Raven Standard Progressive Matrices Test items by principal components analysis. FUUAST Journal of Biology 5: 169–73. [Google Scholar]
- Jackson, Simon A., Gregory D. Martin, Eugene Aidman, and Sabina Kleitman. 2017. Acute short-term sleep deprivation does not affect metacognitive monitoring captured by confidence ratings: A systematic literature review. Metacognition and Learning 13: 39–56. [Google Scholar] [CrossRef]
- Jeckeln, Géraldine, Pascal Mamassian, and Alice J. O’Toole. 2022. Confidence Judgments Are Associated with Face Identification Accuracy: Findings from a Confidence Forced-Choice Task. Available online: https://www.ncbi.nlm.nih.gov/pubmed/36513903 (accessed on 13 December 2022).
- Jensen, Arthur R. 2006. Clocking the Mind: Mental Chronometry and Individual Differences. Amsterdam: Elsevier. [Google Scholar]
- Juslin, Peter, Anders Winman, and Henrik Olsson. 2000. Naive empiricism and dogmatism in confidence research: A critical examination of the hard–easy effect. Psychological Review 107: 384–96. [Google Scholar] [CrossRef]
- Koriat, Asher. 1997. Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General 126: 349–70. [Google Scholar] [CrossRef]
- Kreft, Ita G. G. 1996. Are Multilevel Techniques Necessary? An Overview, Including Simulation Studies. Los Angeles: California State University. [Google Scholar]
- Lamotte, Mathilde, Sylvie Droit-Volet, and Marie Izaute. 2017. Confidence judgment in a temporal generalization task: Accuracy and sensitivity to task difficulty. L’Année Psychologique 117: 275–98. [Google Scholar] [CrossRef]
- Lee, Douglas G., and Jean Daunizeau. 2021. Trading mental effort for confidence in the metacognitive control of value-based decision-making. Elife 10: e63282. [Google Scholar] [CrossRef]
- Liao, Dandan. 2018. Modeling the Speed-Accuracy-Difficulty Interaction in Joint Modeling of Responses and Response Time. Ph.D. dissertation, University of Maryland, College Park, MD, USA, October 8. [Google Scholar]
- Lindquist, Kristina. 2019. Beyond Right or Wrong: The Influences of Thinking Disposition and Item Difficulty on Student Behavior During High-stakes Testing. Ph.D. dissertation, University of Nevada, Las Vegas, NV, USA, March 9. [Google Scholar]
- McNeish, Daniel M., and Laura M. Stapleton. 2014. The Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration. Educational Psychology Review 28: 295–314. [Google Scholar] [CrossRef]
- Nelson, Thomas O. 1984. A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin 95: 109–33. [Google Scholar] [CrossRef] [PubMed]
- Putnam, Adam L., Will Deng, and K. Andrew DeSoto. 2022. Confidence ratings are better predictors of future performance than delayed judgments of learning. Memory 30: 537–53. [Google Scholar] [CrossRef] [PubMed]
- Raven, John C., and John Hugh Court. 1938. Raven’s Progressive Matrices. Los Angeles: Western Psychological Services. [Google Scholar]
- Reber, Rolf, and Rainer Greifeneder. 2016. Processing Fluency in Education: How Metacognitive Feelings Shape Learning, Belief Formation, and Affect. Educational Psychologist 52: 84–103. [Google Scholar] [CrossRef]
- Reber, Rolf, Pascal Wurtz, and Thomas D. Zimmermann. 2004. Exploring “fringe” consciousness: The subjective experience of perceptual fluency and its objective bases. Consciousness and Cognition 13: 47–60. [Google Scholar] [CrossRef] [PubMed]
- Rhodes, Matthew G., and Alan D. Castel. 2008. Memory predictions are influenced by perceptual information: Evidence for metacognitive illusions. Journal of Experimental Psychology: General 137: 615–25. [Google Scholar] [CrossRef] [Green Version]
- Schmitz, Florian, and Oliver Wilhelm. 2016. Modeling Mental Speed: Decomposing Response Time Distributions in Elementary Cognitive Tasks and Correlations with Working Memory Capacity and Fluid Intelligence. Journal of Intelligence 4: 13. [Google Scholar] [CrossRef] [Green Version]
- Schraw, Gregory. 2008. A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning 4: 33–45. [Google Scholar] [CrossRef]
- Shulman, Hillary C., Matthew D. Sweitzer, Olivia M. Bullock, Jason C. Coronel, Robert M. Bond, and Shannon Poulsen. 2022. Predicting Vote Choice and Election Outcomes from Ballot Wording: The Role of Processing Fluency in Low Information Direct Democracy Elections. Political Communication 39: 652–73. [Google Scholar] [CrossRef]
- Susser, Jonathan A., Neil W. Mulligan, and Miri Besken. 2013. The effects of list composition and perceptual fluency on judgments of learning (JOLs). Memory & Cognition 41: 1000–11. [Google Scholar] [CrossRef]
- Undorf, Monika, Malte F. Zimdahl, and Daniel M. Bernstein. 2017. Perceptual fluency contributes to effects of stimulus size on judgments of learning. Journal of Memory and Language 92: 293–304. [Google Scholar] [CrossRef]
- Voodla, Alan, and Andero Uusberg. 2021. Do performance-monitoring related cortical potentials mediate fluency and difficulty effects on decision confidence? Neuropsychologia 155: 107822. [Google Scholar] [CrossRef] [PubMed]
- Wang, Jingyu, and Kejia Qu. 2019. Cue Integration and Its Validity in Judgment of Learning %J Psychology:Techniques and Applications. Psychology: Techniques and Applications 7: 433–40. [Google Scholar] [CrossRef]
- Wang, Qianyu. 2019. The Influence of Word Frequency and Motion Fluency on Judgement of Learning: A Perspective from multiple cues. Master’s dissertation, Liaoning Normal University, Liaoning, China, May 1. [Google Scholar]
- Willoughby, Emily A., and James J. Lee. 2021. Parsing information flow in speeded cognitive tasks: The role of g in perception and decision time. Journal of Experimental Psychology: Learning, Memory, and Cognition 47: 1792–809. [Google Scholar] [CrossRef]
- Winkielman, Piotr, Michal Olszanowski, and Mateusz Gola. 2015. Faces in-between: Evaluations reflect the interplay of facial features and task-dependent fluency. Emotion 15: 232–42. [Google Scholar] [CrossRef] [Green Version]
- Yang, Chien-Lin, Thomas R. O. Neill, and Gene A. Kramer. 2002. Examining item difficulty and response time on perceptual ability test items. Journal of Applied Measurement 3: 282–99. [Google Scholar]
- Zhang, Houcan, and Xiaoping Wang. 1989. Standardization Research on Raven’s Standard Processive: Matrices in China. Acta Psychologica Sinica 21: 3–11. [Google Scholar]
Variable | M | SD | 1 | 2 | 3 |
---|---|---|---|---|---|
Individual level | |||||
Age | 22.76 | 1.38 | |||
Gender | 1.46 | 0.51 | |||
Intelligence | 26.32 | 3.58 | |||
item level | |||||
Item difficulty | 0.74 | 0.22 | — | ||
Processing fluency | 18.14 | 12.54 | −0.624 ** | — | |
JOC magnitude | 4.24 | 1.35 | 0.383 ** | −0.364 ** | — |
Variable | Processing Fluency | JOC Magnitude | |||||
---|---|---|---|---|---|---|---|
M3 (Null) | M4 | M6 | M7 | M1 (Null) | M2 | M5 | |
Intercept (γ00) | 18.18 *** | 35.68 *** | 8.22 | 10.13 | 4.24 *** | 3.53 * | 4.89 ** |
Gender | −0.50 | −0.47 | −0.45 | −0.52 | −0.58 * | ||
Age | 0.44 | 0.38 | 0.38 | 0.01 | 0.01 | ||
Item difficulty | −36.15 *** | −2.48 *** | −36.12 *** | 2.19 *** | 1.23 ** | ||
Processing fluency | −0.03 *** | ||||||
Intelligence | −1.28 * | −0.14 | |||||
Intelligence × item difficulty | −1.26 * | ||||||
σ2 | 144.35 | 76.60 | 76.57 | 76.57 | 1.20 | 0.83 | 0.73 |
m00 | 13.07 *** | 106.63 *** | 95.69 *** | 14.60 *** | 0.66 *** | 3.32 *** | 4.28 *** |
Intermediary Variables | Intelligence | Coefficient | 95% Confidence Range | |
---|---|---|---|---|
Lower Limit | Upper Limit | |||
Processing fluency | High | −1.106 | −1.461 | −0.756 |
Low | −0.858 | −1.165 | −0.571 | |
Difference | 0.248 | 0.298 | 0.185 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhou, Y.; Jia, N. The Impact of Item Difficulty on Judgment of Confidence—A Cross-Level Moderated Mediation Model. J. Intell. 2023, 11, 113. https://doi.org/10.3390/jintelligence11060113
Zhou Y, Jia N. The Impact of Item Difficulty on Judgment of Confidence—A Cross-Level Moderated Mediation Model. Journal of Intelligence. 2023; 11(6):113. https://doi.org/10.3390/jintelligence11060113
Chicago/Turabian StyleZhou, Yuke, and Ning Jia. 2023. "The Impact of Item Difficulty on Judgment of Confidence—A Cross-Level Moderated Mediation Model" Journal of Intelligence 11, no. 6: 113. https://doi.org/10.3390/jintelligence11060113