Testing ActiveYou II: Applying Cognitive Interviews in Improving Item Quality and Applicability of a Web-Based, Self-Report Instrument on Participation in Children with Disabilities
Abstract
:1. Introduction
2. Methods
2.1. Participants and Procedure
2.2. ActiveYou II
2.3. Cognitive Interviews
2.4. Data Analysis
3. Results
3.1. Characteristics of the Sample
3.2. Item Adjustments Based on the Cognitive Interviews
4. Discussion
4.1. Adjustments to the Instrument
4.2. The Value of Cognitive Interviews in the Instrument Development Process
4.3. Limitations
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Chien, C.-W.; Rodger, S.; Copley, J.; Shorka, K. Comparative Content Review of Children’s Participation Measures Using the International Classification of Functioning, Disability and HealtheChildren and Youth. Phys. Med. Rehabil. 2014, 141–152. [Google Scholar] [CrossRef] [PubMed]
- Shikako-Thomas, K.; Kolehmainen, N.; Ketelaar, M.; Bult, M.; Law, M. Promoting Leisure Participation as Part of Health and Well-Being in Children and Youth With Cerebral Paly. J. Child. Neurol. 2014, 29, 1125–1133. [Google Scholar] [CrossRef] [PubMed]
- Imms, C.; Granlund, M.; Wilson, P.H.; Steenbergen, B.; Rosenbaum, P.L.; Gordon, A.M. Participation, both a means and an end: A concepyual analysis of processes and outcomes in childhood disability. Dev. Med. Child. Neurol. 2017, 59, 16–25. [Google Scholar] [CrossRef] [Green Version]
- Shields, N.; King, M.; Corbett, M.; Imms, C. Is participation among children with intellectual disabilities in outside school activities similar to their typically developing peers? A systematic review. Dev. Neurorehabilit. 2014, 17, 64–71. [Google Scholar] [CrossRef]
- King, M.; Shields, N.; Imms, C.; Black, M.; Ardern, C. Participation of children with intellectual disability compared with typically developing children. Res. Dev. Disabil. 2013, 34, 1854–1862. [Google Scholar] [CrossRef]
- Adair, B.; Ullenhag, A.; Rosenbaum, P.; Granlund, M.; Keen, D.; Imms, C. Measures used to quantify participation in childhood disability and their alignment with the family of participation-related constructs: A systematic review. Dev. Med. Child. Neurol. 2018, 1101–1117. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Babulal, G.M.; Bakhshi, P.; Kopriva, S.; Ali, S.A.; Goette, S.A.; Trani, J.F. Measuring participation for persons with mental illness: A systematic review assessing relevance of existing scales for low and middle income countries. Bmc Psychol. 2015, 3, 1–10. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Michelsen, S.I.; Flachs, E.M.; Uldall, P.; Eriksen, E.L.; McManus, V.; Parkes, J.; Parkinson, K.N.; Thyen, U.; Arnaud, C.; Beckung, E.; et al. Frequency of participation of 8–12-year-old children with cerebral palsy: A multi-centre cross-sectional European study. Eur. J. Pediatric Neurol. 2009, 13, 165–177. [Google Scholar] [CrossRef]
- Ullenhag, A.; Bult, M.K.; Nyquist, A.; Ketelaar, M.; Jahnsen, R.; Krumlinde-Sundholm, L.; Almqvist, L.; Granlund, M. An international comparison of patterns of participation in leisure activities for children with and without disabilities in Sweden, Norway and the Netherlands. Dev. Neurorehabilit. 2012, 15, 369–385. [Google Scholar] [CrossRef] [Green Version]
- Hoberg, A.; Nyquist, A. CAPE/PAC Prosjektrapport 2008-2010 [Projectreport]; Beitostølen Helsesportsenter: Beitostølen, Norway, 2011. [Google Scholar]
- Nordtorp, H.L.; Nyquist, A.; Jahnsen, R.; Moser, T.; Strand, L.I. Reliability of the Norwegian Version of the Children’s Assessment of Participation and Enjoyment (CAPE) and Preferences for Activities of Children (PAC). Phys. Occup. Ther. Pediatrics 2013, 33, 199–212. [Google Scholar] [CrossRef] [Green Version]
- Dalen, L.K.; Nyquist, A.; Shields, L.; Stanley, D.; Nyquist, T.E.; Jahnsen, R.; Ullenhag, A. ActiveYou I–a new web-based measure of activity preferences among children with disabilities. Scand. J. Occup. Ther. 2020, 1–11. [Google Scholar] [CrossRef] [PubMed]
- World Health Organisation. International Classification of Function, Disability, and Health; World Health Organisation: Geneva, Switzerland, 2001. [Google Scholar]
- Badley, E.M. Enhancing the conceptual clarity of the activity and partcipitation components of the International Classification of Functioning, Disability, and Health. Soc. Sci. Med. 2008, 66, 2335–2345. [Google Scholar] [CrossRef] [PubMed]
- King, G.; Law, M.; King, S.; Rosenbaum, P.; Kertoy, M.K.; Young, N.L. A Conceptual Model of Factors Affecting the Recreation and Leisure Participation of Children with Disabilities. Phys. Occup. Ther. Pediatrics 2003, 23, 63–88. [Google Scholar] [CrossRef]
- Appleton, J.J.; Christenson, S.L.; Kim, D.; Reschly, A.L. Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument. J. Sch. Psychol. 2006, 44, 427–445. [Google Scholar] [CrossRef]
- Spencer, N.L.I.; Bouffard, M.; Watkinson, E.J. Cognitive interviews with children as a research tool for instrument validation in adapted physical activity. Eur. J. Adapt. Phys. Act. 2020, 13. [Google Scholar] [CrossRef] [Green Version]
- Steinhardt, F.; Ullenhag, A.; Jahnsen, R.; Dolva, A.-S. Perceived facilitators and barriers for participation in leisure activities in children with disabilities: Perspectives of children, parents and professionals. Scand. J. Occup. Ther. 2019, 28, 121–135. [Google Scholar] [CrossRef]
- Pearlin, L.I.; Schooler, C. The Structure of Coping. J. Health Soc. Behav. 1978, 19, 2–21. [Google Scholar] [CrossRef] [Green Version]
- Havitz, M.E.; Dimanche, F. Leisure Involvement Revisited: Conceptual Conundrums and Measurement Advances. J. Leis. Res. 1997, 29, 245–278. [Google Scholar] [CrossRef]
- Kyle, G.; Absher, J.; Norman, W.; Hammitt, W.; Jodice, L. A Modified Involvement Scale. Leis. Stud. 2007, 26, 399–427. [Google Scholar] [CrossRef]
- Willis, G.B. Cognitive Interviewing. In Proceedings of the Meeting of the American Statistical Association, Alexandria, VA, USA, 1999. [Google Scholar]
- Tourangeau, R. Cognitive science and survey methods. In Cognitive Aspects of Survey Design: Building a Bridge Between Disciplines; Jabine, T., Straf, M., Tanur, J., Tourangeau, R., Eds.; National Academy Press: Washington, DC, USA, 1984; pp. 73–100. [Google Scholar]
- Peterson, C.H.; Peterson, N.A.; Gilmore Powell, K. Cognitive Interviewing for Item Development: Validity Evidence Based on Content and Response Processes. Meas. Eval. Couns. Dev. 2017, 50, 217–223. [Google Scholar] [CrossRef]
- Liljenquist, K.; Kramer, J.; Rossetti, Z.; Coster, W. Content development, accessibility and feasibility of a self-report tool for use in programmes serving youth with cognitive disabilities: The Participatory Experience Survey. Aust. Occup. Ther. J. 2019, 66, 490–499. [Google Scholar] [CrossRef]
- Hsieh, H.-F.; Shannon, S.E. Three approaches to qualitative content analysis. Qual. Health Res. 2005, 15, 1277–1288. [Google Scholar] [CrossRef]
- MAXQDA, Version 2018; VERBI Software: Berlin, Germany, 2018.
- Gustafsson, S.; Hörder, H.; Ottenvall Hammer, I.; Skoog, I. Face and content validity and acceptability of the Swedish ICECAP-O capability measure: Cognitive interviews with 70-year-old persons. Health Psychol. Res. 2018, 6, 6496. [Google Scholar] [CrossRef] [PubMed]
- UNICEF. Convention on the Rights of the Child; UNICEF: New York, NY, USA, 1989. [Google Scholar]
- United Nations. Convention on the Rights of Persons with Disabilities; United Nations: New York, NY, USA, 2006. [Google Scholar]
- Hammel, J.; Magasi, S.; Heinemann, A.; Whiteneck, G.; Bogner, J.; Rodriguez, E. What does participation mean? An insider perspective from people with disabilities. Disabil. Rehabil. 2008, 30, 1445–1460. [Google Scholar] [CrossRef] [PubMed]
Item | Cognitive Interviews | Adjustments for Further Development |
---|---|---|
1a | How often have did you do this activity during the last three month?
| How often do you do this activity?
|
1b | Are you satisfied as it is?
| Is this as often as you like it?
|
2a | With whom do you do the activity | |
|
| |
2b | Are you satisfied as it is? | Do you like it that way? |
|
| |
3 | How good do you think you can do the activity? | |
Smileys
| Smileys
| |
4 | It is fun to do this activity. | |
Smileys
| Smileys
| |
5 | The activity is important to me. | |
Smileys
| ||
6 | This makes it easier for me to participate | |
Somebody tells me where I can participate
|
| |
7 | This makes it difficult for me to participate | |
|
|
Original Quote | Extraction | Conceptual Theme |
---|---|---|
Interviewer: «How often do you go to the pool?» Child: «I go there every other Friday.» Interviewer: «Alright. What answer-alternative do you think you should cross than?» Barn: «I don’t know.» | Item 1a: The child is able to formulate an answer to the question but cannot translate the answer to the given categories in the questionnaire. | response |
Interviewer: «Do you think there is a difference between these three questions?» (refer to items 3, 4 and 5) Child: «No.» | Items 3, 4, 5: The child cannot distinguish between the meaning of the items 3, 4 and 5 | comprehension |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Steinhardt, F.; Jahnsen, R.; Dolva, A.-S.; Ullenhag, A. Testing ActiveYou II: Applying Cognitive Interviews in Improving Item Quality and Applicability of a Web-Based, Self-Report Instrument on Participation in Children with Disabilities. Int. J. Environ. Res. Public Health 2021, 18, 4768. https://doi.org/10.3390/ijerph18094768
Steinhardt F, Jahnsen R, Dolva A-S, Ullenhag A. Testing ActiveYou II: Applying Cognitive Interviews in Improving Item Quality and Applicability of a Web-Based, Self-Report Instrument on Participation in Children with Disabilities. International Journal of Environmental Research and Public Health. 2021; 18(9):4768. https://doi.org/10.3390/ijerph18094768
Chicago/Turabian StyleSteinhardt, Friedolin, Reidun Jahnsen, Anne-Stine Dolva, and Anna Ullenhag. 2021. "Testing ActiveYou II: Applying Cognitive Interviews in Improving Item Quality and Applicability of a Web-Based, Self-Report Instrument on Participation in Children with Disabilities" International Journal of Environmental Research and Public Health 18, no. 9: 4768. https://doi.org/10.3390/ijerph18094768