1. Introduction
Spatial intelligence, the capacity to construct mental representations of objects and people in space and perform activities based on spatial perception, to evaluate and modify those perceptions according to subsequent experience and recreate components of a perception even without the physical presence of the original stimulus, is one of seven intelligences described in the theory of multiple intelligences (MI).
Visual skills are mentioned here as a modality central to spatial intelligence that also incorporates other senses and cognitive operations. The creation of objects and images (from artworks to graphs) was indicated but not detailed, as Gardner mentioned artists, architects, engineers, surgeons, and sportspeople who require high degrees of spatial intelligence to act successfully. (
Gardner 1983, pp. 173–75).
In his reinterpretation of the theory of intelligences, Gardner discussed
visual intelligence, which utilizes perception and manipulation guided by sight; and
spatial intelligence, which involves creation and perception of space and does not necessarily require vision (as in the tactile exploration of persons with damaged eyesight). In the expanded MI model,
visual-spatial intelligence is activated during the creation, perception, interpretation, and use of spatial and two-dimensional representations. Gardner offers a broad definition of intelligence, involving high-level and ordinary applications: “An intelligence entails the ability to solve problems or fashion products that are of consequence in a particular cultural setting and community. The problem-solving skills allows us to approach a situation in which a goal is to be obtained and locate the appropriate route to that goal. The creation of cultural products allows one to capture and transmit knowledge or to express one’s conclusions, beliefs, or feelings. The problems to be solved range from creating an end for a story to anticipating a mating move in chess to repairing a quilt”. (
Gardner 2006, p. 12). The accurate perception of objects, spaces, people and images and their relations, and the construction of mental representations of visual information involves interrelated spatial and visual skills.
In related studies, visual-spatial intelligence is always mentioned in conjunction with acts of perception and creation. The creative subskills connected to visual-spatial intelligence were first reduced to representation, then creative use of visual language (e.g., symbolization and expression of moods, feelings, and thoughts) were identified as important manifestations of this intelligence (
Winner et al. 2013). Gardner’s definition of visual-spatial intelligence inspired studies to describe its components and prove its importance for other areas of learning. (An example:
Stavridou and Kakana 2008) compared the use of three graphic abilities related to visual-spatial intelligence with performance in science and mathematics and found significant correlations). In his phenomenologist critique of the interpretation of visual intelligence in the theory of MI,
Cary (
2004) called for an enrichment of the concept with considerations of the use of visual language for aesthetic expression and called for research on its genesis and evolution. Development of visual-spatial intelligence was one of the main objectives of Gardner’s work at
Harward Project Zero (
n.d.) (The research group was founded by Nelson Goodman at the Harvard Graduate School of Education in 1967 to explore learning in and through the arts). The “zero” in its name suggested that no previous studies were considered relevant in this area as they provided emphatic claims for more arts at schools, but little research evidence. Visual art education projects that Gardner and his associates initiated focused on the varied and flexible use of visual language, and represented a radical shift from contemporary, high-arts-based methodologies (e.g.,
Hetland et al. 2007). Visual-spatial intelligence was explained through educational interventions, but levels of the development of skills that constitute visual-spatial intelligence were not described.
In the 1990s, more and more studies on the development of certain visual skills were published and the need for a common framework to guide research for curriculum design was voiced. (
Boughton 2013;
Winner et al. 2013). Art education worldwide experienced a “
cognitive turn” and wanted to show its potentials (Rat für Kulturelle Bildung 2019). In response to the need for a research-based structure of knowledge, skills, and competencies constituting the use of visual language, art educators from 17 European countries established the
European Network of Visual Literacy (
ENViL n.d.,
https://envil.eu/, accessed on 4 June 2022). The consortium analyzed 27 European art and design education curricula to identify skills that these educational documents found important to teach in public education (
Kirchner et al. 2016). The prototype of the
Common European Framework of Reference for Visual Literacy, (CEFR-VL) was developed. In compliance with other frameworks defining various important literacies in the field of native language use or mathematics, the framework was named “literacy”, although dealing with images (not letters) and their use. The competency definition of Franz
Weinert (
2001) was employed, who claims that a competence requires the combined use of learnable knowledge, skills, and attitudes. It is demonstrated in specific (professional) situations; its outcome may be an object as well as a demonstrable behavior or disposition.
A series of (inter)national experiments to test its validity and usability for curriculum design and research on skills development were launched and published (
Wagner and Schönau 2016). While the concept of visual-spatial intelligence was defined as a construct that needed verification, ENViL offered a new approach to visual-spatial intelligence through the identification and description of skills and subskills identified in educational curricula that defined teaching practice. The Framework was revised to include more subskills and renamed to the
Common European Framework of Reference for Visual Competency, (CEFR-VC) (Schönau et al.,
Figure 1 and
Figure 2).
The framework identifies two major competence domains (skill clusters): producing (planning and realization of a visual creation) and responding (perception and response to a visual creation), while acknowledging their interrelated character in certain visual activities. Relationships of visual skills with basic competencies: self-, methodological, and social competences, are also indicated. Metacognition represented above the two domains or skill clusters indicate the importance of critical awareness of the thinking and learning processes of our own and those of others. Metacognition as part of visual competence indicates the importance of monitoring and assessing our creative and responsive performance.
Figure 2 represents the subskills identified in the revised version of the framework (
Schönau et al. 2020). They are grouped under the domain they belong to. However, some subskills may play a role in activities requiring the other domain as well, depending on the situations they are employed (
Billmayer 2016). They are defined extensively in previous publications (
Wagner and Schönau 2016, pp. 66–79;
Schönau et al. 2020). For eleven subskills, there was enough research evidence to describe three levels of attainment (elementary, intermediate, and competent, cf.
Wagner and Schönau 2016, pp. 80–90). Further research is needed to establish age-based performances that may better serve the goals of curriculum innovation as these studies better support the definition of output requirements for school grades. This study is one of these efforts.
Our investigations of the development of visual skills and abilities are based on the CEFR-VC framework. Our intention is to define age-based development through assessment tools that are authentic both as research instruments and art tasks. Our assessment efforts intend to support educational interventions; therefore, we do not define three levels of attainment, as in the CEFR-VC, but describe the performance of students at different school grades and thus facilitate curriculum development and assessment that is responsive to age-related performance. In this paper, we present the assessment of subskills of the two major domains (skill clusters) of visual communication. Our instruments were designed for digital testing, to ensure easy access by art educators who can administer them in the school computer laboratory without the need of printing test sheets in color. The tasks are organized in tests but may also be used individually for practice or assessment.
3. Results and Discussion
The tests were taken in school settings, in the computer laboratories on laptop computers or in a classroom on tablets. We calculated the reliability indicators for both tests and used Cronbach’s alpha to reveal internal consistency. We employed an independent t-test to measure differences between genders and students using different digital devices. One-way analysis of variance (ANOVA) was used to determine whether there are any statistically significant differences between age groups and whether school results in different disciplines, or if the frequency of playing computer games influences performance in visual communication tests. As the CEFR-VC framework lists major skills only, our hypothesized structure of the visual communication subskills in the “produce” and “respond” domains were verified through confirmatory factor analysis. The results are summarized below.
In the pilots, we tested the relevance of the platform for results and did not find correlations (cf.
Table 4 below). The medium made no difference; paper-based test scores and digital test scores showed similar statistical values. Taking the tests on laptops and tablets and receiving results immediately facilitates the work of art educators who may use our tools for continuous, formative assessment as well as end-of-term, summative testing of skills. This finding is also important because of the increased relevance of digital imaging in private life and in the workplace. Paper-based tests are becoming less and less authentic as more and more programs in visual art education involve digital media. (
Table 4).
3.1. Assessment Results: Subskills of Visual Communication Related to the Produce Domain of CEFR-VC
All tests for both domains underwent validity and reliability analysis. The reliability of the whole test is good (Cronbach’s alpha > 0.8). The high reliability is due to the high variance and the resulting heterogeneous group composition (
Table 5).
To reveal differences in performance of the age (and grade) of students, we performed a one-way analysis of variance (ANOVA). There was a measurable difference between the age groups.; this was confirmed by the high partial Eta-squared effect size value (η
p2 = 0.30). The significance level of the Levene’s test was 0.03 (
p < 0.05); we used the Dunnet T3 test to reveal age-related results. Two groups were separated: 11–12 and 13–14-year-olds (
Table 6).
The test results presented in
Table 7 show no relevant difference between the scores of the boys and girls, in contrast to previous studies performed with paper-based tests (
McGivern et al. 1997;
Abramov et al. 2012;
Siu et al. 2015). While in paper-based tests, manual dexterity is an important and often dominant subskill in digital imaging, a wide variety of other subskills are involved.
Task-level analyses show that there is not much difference between the mean and variance distribution of each task. The average task solution rate ranges between 41% and 61%, which is considered acceptable. The standard deviation is between 26% and 45%, indicating that some tasks are too easy, or their assessment criteria are problematic (
Table 8). These tasks will be corrected in the published assessment tool.
The elimination of the gender performance gap using these digital toolset results is partly due to the absence of freehand drawing among the tasks, as emphasized above. Another factor often mentioned as affecting performance may be the different digital literacy level of the two genders, especially the more intensive online gaming habits of boys. Girls play computer games less frequently, and therefore may experience difficulties in using the functionalities of a complex online test. (
Veltri et al. 2014). In our case, no such difficulties were experienced. The type of digital instrument used produced no significant differences in the results of students, either. One part of the students took the test on personal computers (N = 104) and the other part took it on tablets (N = 182). There is no correlation (r = 0.07) or significant difference between test scores and the device used (
Table 9). However, the small effect size suggests that it would be advisable to repeat the measurement on a larger sample.
In designing the digital test and developing the scoring criteria, we aimed to focus the tasks on the four subskills of visual communication described in
Table 1: 2D composition, abstraction, symbolization, modality shift. We considered isolating and assessing these subskills, focusing only on one of them in each task. However, when the preliminary structure was developed, we realized that the subskills we examined are likely to be activated together. It seemed to be impossible to represent only one subskill within a single task. More than one subskill that we intended to assess was activated during the solution of a visual communication task. Based on the assessment results, it was confirmed that the four creative subskills of visual communication that we focused on are closely interrelated (
Figure 11).
A solution to this problem was to narrow the analysis down to a single skill element, the use of compositional principles and components in
two dimensions. Our attempt to separate the compositional components by item is shown in
Figure 11, in the row labeled “visual communication component”.
According to the new approach, five visual subskills were separated at test-item level (creation of color-tone, position, direction, shape, and size). Our hypothesis was confirmed by a factor analysis reduced to five factors, which showed a 70% match with the components of visual communication skills based on the literature. There was a weak-to-medium correlation between the separated visual communication components. The strongest correlation was between direction and shape (r = 0.57) and direction and position (r = 0.57); the weakest correlation was between shape and position (r = 0.30). However, all the visual communication components were highly correlated with the overall test, i.e., they showed a strong relationship with the overall test score. The co-occurrence of subtest behaviors and the strength of the relationships between the visual communication components support our hypothesis that the test assessed related but not identical visual communication components.
In further development of the test, it would be worthwhile to concentrate on these components. An attempt to assess the use of additional components and principles of composition (balance, emphasis, movement, contrast, pattern, rhythm, uniformity/variety) would be desirable, as these together form the basis of the language of visual communication.
3.2. Assessment Results: Subskills of Visual Communication Related to the “Respond” Domain of CEFR-VC
Our research sample consisted of students of Grade 4, 5, and 6 of the compulsory Hungarian primary school system. No classes with special art instruction were included. A total of 21 classes from 13 schools completed the Grade 4 test, with an average age of 10.6 years. The Grade 5 sample consisted of 338 students from 26 classes in 14 schools, with an average age of 11.4 years. Year 6 was represented by 486 students from 26 classes in 16 schools. The average age of the 6th-graders was 12.5 years. A total of 1256 students participated in the assessment project.
A total of 24 items were developed for the tests of the three grades. The reliability of the test interpreted by the anchor items was good (Cronbach’s α = 0.78).
Table 10 summarizes the psychometric characteristics of the tests for the three grades.
The test was also examined through probability test theory. The EAP/PV reliability index is appropriate for all Grades (
Table 11). The analysis showed that the items cover the average skill level, shifting slightly to higher values. The test has a perceptible ceiling effect, so it separates learners who achieve higher results less than expected.
A confirmatory factor analysis (CFA) was performed a to verify the hypothetical structure of the subskills (
Table 12). The comparative fit index (CFI) indicates the model fit by examining the discrepancy between the data and our hypothesized model; the Tucker–Lewis Index (TLI), also known as the non-normed fit index, was used for linear mean and covariance structure modeling. Root-mean-square error of approximation (RMSEA) was used to measure of the estimated discrepancy between the population and model-implied population covariance matrices per degree of freedom. We realized that our model about major components of the “respond” domain of visual communication skills is weak because the subskills are multidimensional, and there is a significant correlation among the items.
We examined how the differences between the subsamples of grades were reflected in output (
Table 13). In the analysis of variance, no significant differences were found between the grades (Levene’s test = 0.35
p = 0.7; F = 0.11
p = 0.89), but small improvements were detected.
When we examine the sample according to the year of birth (which is a more accurate age definition than school grade), the analysis of variance shows more pronounced differences according to results of Levene’s test, an inferential statistic used to assess the equality of variances for a variable calculated for two or more groups. (3.63 p < 0.05; F = 4.1 p < 0.05). While we detected a steady increase in performance in the “produce” domain, in the “response” domain we found no significant difference between the development of the age groups. We can only detect slight improvement between those born in 2001 and 2002 (t = −3.88 p < 0.001). The reason for this lack of development is related to the Hungarian art education curriculum, emphasizing production over reception and interpretation of images.
As in the “produce” domain, we found no differences between the performance of boys and girls (r = 0.17 for boys and r = 0.41 for girls). However, in the “response” domain, we found correlations of medium strength among learning results and test performance in the three grades (r = 0.48; r = 0.46; r = 0.46). This result calls attention to the findings of
Stavridou and Kakana (
2008), who found correlations between the level of certain visual skills and performance in mathematics: in the “response” domain, we also found that visual communication skills are better developed with successful learners. In the “produce” domain, however, no correlation with learning results was detected.
4. Conclusions
In this paper, we introduced two assessment projects that target the competence structure of visual communication, arguably the most important visual skill cluster of our time. The first project focused on activities related to the perception of images and assessed their development through online, interactive tests administered in the eDia online, interactive testing environment in the Hungarian compulsory primary school system Grades 4–6 (ages 10–12 years). The second experiment was performed through the GeoGebra software and tested major components of the creative domain of visual communication in Grades 5–8 of the Hungarian primary school system (ages 11–14 years). Our results show increasing attainment in subskills through the age groups in the “produce” domain and less significant or no development in the “respond” domain. These findings are in line with the analysis of our art education curriculum, which is creation-focused. Such an approach does not adequately develop perception, analysis and interpretation of art works that constitute the subskills of the "Response” domain.. An important educational-practice-oriented finding of our research is that visual communication, a key life skill of the visual competency cluster, needs more intensive educational interventions in the “response” domain.
We could not identify any other standardized digital assessment tools for students of compulsory primary education (ISCED 2) in visual art education; our most significant result in this area is proving that authentic assessment of visual skills may be performed through digital tests simulating authentic use of predominantly digital visual language of children and youth. Our test items can or should never replace genres and techniques traditionally employed for detecting visual talent or assessing art performance since the foundation of the first art academies in the 1980s but may provide an alternative if skills such as visual communication are assessed that are practiced mainly through digital media.
In comparison to paper-based tasks, where girls often outperform boys from preadolescence, we found no differences between genders. Creating digital images seems to be a skill set that is equally present in both genders of Generation Alpha, (children born in the 2010s), the cohort following the much-researched Generation Z. This finding has a significant relevance for art education: digital imaging is a medium that can be smoothly introduced in art education curricula if the software environment used produces adequate help and the tasks are age-related.
To develop art programs that are individualized and responsive to social, and cultural needs, detailed description of visual skills and subskills are needed, with data on attainable performance in age groups that can be translated to school grades. In a vocational secondary school, different subskills are needed for each group of professions and their acquisition is key for successful actions at the workplace. Our toolsets assist curriculum development as it significantly facilitates the description and mapping of the “produce” and “respond” domains of visual communication. Art education programs using these valid, reliable, and coherent instruments may introduce more focused educational interventions.
Teachers may use our skill descriptions and tasks as an inspiration for the development of teaching programs focusing on relevant subskills and use our tasks in a test or individually for assessing the results of their interventions. In secondary grammar schools, knowledge of visual signs and symbols or methods of scientific visualization may be a valid interdisciplinary skill set. Development of visual communication skills through art education does not seem to be enough: an overarching enhancement introducing visualization genres applicable for different school disciplines may produce development.
We intend to continue developing age-related version of our tests to be used for development and assessment. Our results suggest that the subskills of the “produce” and “respond” domain of visual communication can be revealed through assessment tasks but cannot be separated due to their interrelated employment in any given task. In both domains, several subskills are activated when a single task is completed. Therefore, it is impossible to design a single-subskill task, but we can define the subskills that play a role in a task and indicate their relative importance.
We also want to expand the age range of our study to Kindergarten age groups (3-6 years) to see how frequent encounters with digital images influence visual competence at the age of the genesis of visual language. We want to create more flexible assessment instruments to avoid the ceiling effect through dynamic adaptive testing. Some of our visualizations may be transferred to the smaller screen of the smartphones and may thus offer a play-like activity for students on an easily accessible platform. This way, art educators may include short assessment periods or game-like tasks of edutainment quality in their lessons. A very interesting venue to explore will be research on the effects of visual communication genres that transmit science knowledge (e.g., multimedia infographics, flowcharts, modifiable graphs, and simulations). Visual communication supports learning and may be especially beneficial for poor verbalizers.
Our further research projects will also target the relationship of the skill clusters “produce” and “respond”, as represented in the Common European Framework of Visual Competency (
Wagner and Schönau 2016;
Schönau et al. 2020). Does high performance in the “produce” or “respond” cluster influence performance in the other? In this respect, a more sophisticated approach seems to be promising: which subskill of one cluster corresponds with the other, offering new potentials of development?