Experts and Gamers on Immersion into Reconstructed Strongholds

Abstract

In this study, we have touched upon a problem in evaluating the method of immersion in specific historico-geographical virtual space constructed on the basis of traditional cartographic and graphic materials. We have obtained opinions from two groups of users on the perception of cultural objects reconstructed in a virtual reality previously unknown to them. To achieve our objective and answer the questions, we have adopted four main stages of research: to pinpoint concepts adopted by researchers by discussing two types of approach, to create a virtual reality application according to the scheme based on knowledge from analog sources and digital actions in several workspaces, to prepare and conduct a survey among experts and gamers, and to graphically juxtapose the results of the survey. The evaluation by experts in medieval strongholds and serious story game users of the specific ways of immersion in the VR of reconstructed buildings in the current area provides researchers with an extended view of its effectiveness and attractiveness as well as with suggestions for further design processes.

1. Introduction

In this study, we have touched upon a problem in evaluating the method of immersion in specific historico-geographical virtual space constructed on the basis of traditional cartographic and graphic materials. Appropriate evaluation of a virtual reality application with comments from experts and gamers can help in determining its effectiveness and attractiveness as well as in the process of design. Designing spatial visualizations with reconstructions of historical objects in the virtual reality (VR) system on the basis of analog sources is another issue touched upon in this article [1,2]. The new media allow one to create an immersive virtual reality application for non-existing objects by integrating a virtual environment with traditional specialist graphics, frequently saved only as academic black-and-white sketches [3]. “Presence” in the context of VR is defined as a sense of being in the virtual world [4]. The user of story games is “subjectively” present in the virtual world thanks to the employment of the illusion of 3D movement inside a realistic 2D+ graphic [5]. In turn, “immersion” denotes an objective level of sensory fidelity provided by a VR system [6].

Historical objects, crucial for the country’s history, have been frequently non-existent for centuries, and various sources need to be used to reconstruct them [7,8]. Schematic perspective drawings of buildings, horizontal and vertical maps, as well as situational maps made by scientists to present the hypotheses of the original medieval state constitute significant sources of information. On the other hand, there are digital 3D models of the current state of ruins and traces in relief obtained from TLS (Terrestrial Laser Scanning) and UAV (Unmanned Aerial Vehicle) photos in the form of raw data [9]. The process of integration of all the materials obtained in order to create a VR application can be ordered according to data types and formats, adjusting them to be then processed in suitable programming workspace, which is, again, carried out according to the established order of technological work in several workspaces.

The evaluation of the way space is presented and perceived by users constitutes a significant factor in establishing principles of good design in terms of spatial graphics, interactive maps and other products of multimedia cartography [10]. Creators of geographical space visualizations are searching for suitable ways of evaluation by different user types to be able to determine effectiveness, attractiveness and informativeness of the medium employed [11]. Testing products of multimedia cartography, such as atlases, by users is based on user-centered design principles as well as on principles of map design worked out by theorists and practitioners [12,13].

The so-called “public users” are usually invited to participate in the evaluation of cartographic visualizations [14]. These are frequently students as they constitute a group easily available to scientists and relatively homogenous, especially if they major in the same subject. The large number of students makes it possible to divide them into a few teams so that respondents from several teams can evaluate different versions of mapping techniques that present the same spatial data [15]. Surveys evaluating how specific fragments of urban space are perceived when on a tablet screen, e.g., with cartographic signs applied in the extended reality system, are carried out among random passers-by on the street much more seldomly [16]. The number of respondents in studies devoted to cartographic visualizations ranges between 10 and 30 for the evaluation of a single version, which frequently leads to over 200 respondents for several different versions [17].

The effectiveness of multimedia maps is tested through tasks performed by users on the computer or smartphone screen [18]. The objective effectiveness of visualizations is checked in online surveys by the user having to select the correct answer out of several possible options [16]. The attractiveness of presentations is verified by respondents providing their subjective opinions that not always coincide with objective effectiveness of the spatial information [19]. Surveys rarely include open questions or provide opportunities to comment on questions as that would complicate the entire analysis and make drawing clear-cut conclusions much more difficult.

The answers and marks provided by respondents are then placed in analytical tables, charts and diagrams to demonstrate specific features. To facilitate the process of drawing conclusions and to reveal the relations between users’ evaluation, cartographic research employs forms of graphic enhancement, such as the use of color and other graphic variables, as well as graphically supported evaluation of mapping techniques in the form of advanced graphic forms [16,19,20].

The pragmatic usage of graphic elements resulted in the creation of a simple image that could be read intuitively, with high aesthetic appeal, and this has been described best as “Good design simply ‘looks’ right—it is simple (clear and uncomplicated)” [12]. The knowledge of cartographic design is passed on in the form of more or less precisely described rules, which apply to both simple and complex aspects of design process [21]. When designing immersive visualizations, it is important to consult various perspectives, both with regard to accuracy and artistic presentation of space as well as technical parameters of the application. It seems reasonable to look for a way of formulating and presenting principles and guidelines that can be adopted by designers in their own creative process.

2. Aim and Questions

Evaluating the way of becoming immersed in the VR of reconstructed buildings in present-day state of the area by both experts in medieval strongholds and serious story game users has become the main objective of the research. Apart from the goal set above, we have also raised a series of detailed questions about the evaluation of effectiveness and attractiveness of the designed VR application and the potential that traditional sources have for creating one:

• How to use traditional cartographic and graphic sources to create immersive virtual reality for cultural objects that currently have only their stone remnants and well-preserved traces in relief left?
• How to conduct a process of combined actions on both analog materials and digital spatial data to create a mobile VR application on game engine?
• How to carry out a study on respondents in two different user groups to determine features of the medium in terms of: effectiveness, attractiveness, and informativeness?
• Will graphic elements (i.e., colors, graphic symbols) in tables make it easier to reveal links and will they allow one to capture differences and similarities in comments?
• How to use opinions of two different user groups in designing a VR application?

3. Research Area and Research Objects

In order to conduct a questionnaire among experts and gamers, we have selected the residence of the Piast dynasty’s first rulers on a holm on Lake Lednica (Ostrów Lednicki) with perfectly preserved 10th century ruins of the palatium and the chapel of Duke Mieszko I (ca. 960–992) and his son, the first King of Poland, Bolesław “the Valiant” [22]. What was relevant to our research was the fact that the ramparts of the stronghold were well-marked in the landscape of the island. The stronghold was developing in several phases, however, we focused on the second half of the 10th century [23]. This state of the stronghold included the large palatial-ecclesiastical structure and the small church together, which were accompanied by wooden constructions of varying size. These two stone buildings have become the objects of our virtual reconstruction.

The palatium was built from stones mortared together in the pre-Romanesque style, it served as one of the Piast ruler’s palaces, and retained the court chapel, a baptistry with the two baptismal fonts [24]. This allows us to presume that it was here in 966 that Mieszko I, and symbolically the entire Poland, was baptized. The stronghold’s church was yet another stone building erected in the second half of the 10th century [25]. At that time, the rampart was covered with wooden palisades and was approximately 12 m high [23]. Remnants of the stronghold ramparts, depicted in Figure 1, which are approximately 3 m high, are covered with grass. Current area of the stronghold is covered with grass and has hard-surfaced trails for tourists.

4. Methodology

To achieve our objective and answer the question, we adopted four main stages of research:

• to pinpoint concepts adopted by researchers by discussing two types of approach (Section 4.1.),
• to create a VR application according to the scheme based on knowledge from analog sources (Section 4.2, Figure 2, Figure 3 and Figure 4),
• to prepare and conduct a survey among experts and gamers (Section 4.3, Figure 5),
• to graphically juxtapose the opinions and comments (Section 4.4, Figure 6, Figure 7 and Figure 8).

4.1. Concept

Already, at the stage of concepts, two opposite approaches emerged. The first one was the approach of a cartographer-geographer, a frequenter of Ostrów Lednicki, that has been interested in medieval architecture for years, and the second one was the one of a cartographer-geomatics engineer interested in VR systems made on game engine and a gamer. Having discussed both approaches, we adopted a common concept, including the following initial assumptions:

• specificity of the multimedia presentation: virtual historico-geographical space connecting two distant moments in time: reconstructed palatium and stronghold church from the 10th century on current remnants of the rampart covered with grass,
• viewers of the virtual visualization: a possibly broad group of recipients, including both gamers and experts,

• medium: visual immersion in VR, supported by sound effects (adding artificial sounds: rustle of trees and the footsteps of the user),
• sources: various analog and digital materials, academic literature, LiDAR’s official data, private UAV, and street view images,
• software and equipment: graphic and GIS programs; application worked out on the Unity game engine to fit VR goggles,
• technological process: data transformation and 3D modeling management in the geomatic process in several workspaces,
• respondents: experts on the stronghold in both practice and theory, as well as serious game users via a low-immersive screen-based (2D) presence who have never visited the stronghold in real life,
• geography of the virtual walk around the stronghold: possible exclusively along the designed trail with three viewing points,
• contents of the survey: the same issues/questions for two groups from the same viewing point with an opportunity to select one evaluation out of several versions and to make comments about all the issues,
• the way the survey was conducted: the survey was controlled; issues were read out without repeating; the reply without a time limit; during the survey the respondent was standing and wearing VR goggles; duration: around 20 min,
• presenting the results: the opinions were demonstrated with visual support, using graphical variables at the ordinal level and symbolic signs, and
• the results: formulating conclusions on effectiveness, attractiveness, and tips on designing similar VR applications.

4.2. Creating a VR Application

Adopting all the assumptions listed above, we started creating the application according to the following four main actions placed on scheme in Figure 2:

• obtaining materials and their initial classification,
• the analysis and selection of materials for 3D modeling and texture mapping,
• 2D texture designing, 3D building, and area modeling, and
• creating the VR application.

Obtaining materials and initial classification took place according to three data types: scientific analog stronghold reconstruction, visual documentation of the current state of the stronghold and official digital spatial data. The first types of sources included mainly perspective drawings and reconstruction plans from academic publications as well as a plaster model of the stronghold. Current visual documentation of the stronghold has been obtained by UAV and street view images. The employees of the museum have provided photographs of natural and artificial textures of walls and roofs. Having analyzed them, we have selected crucial materials for our research and considering their format, we have decided upon modeling in four workspaces.

Workspace is an area in the application (i.e., digital work environment dedicated to the specific application) that allows one to perform digital operations (framing, cut, rotation, adding points, georeferencing) for various data types (raster and vector) and formats (.png, .svg, .obj, .shp), operating on both desktop and mobile systems according to the interface and programming scripts. The first “3D Buildings Workspace” in our geomatic process developed manual 3D modeling in the architectural application SketchUp. In this operation, the medieval 10th century buildings of the palatium and the church were reconstructed with the use of the previously mentioned source materials (Figure 2). The reconstructed historical buildings were created as a result of the analysis of historical drawings, descriptive documentation and the plaster model on the one hand, and a digital model of the island on the other hand.

In the second “Raster Graphics Workspace”, photo processing of artificial and natural textures and raster maps took place. Adobe Photoshop was the application used for creating and preparing all textures. In this operation, some textures similar to the original medieval building materials, used for building the palatium with the chapel and the church, were designed. As a result, textures were developed for individual application elements, such as walls, roofs, grasses and pathway. In the third “3D Terrain Workspace”, the 3D modeling of island terrain was the main operation. The point cloud from airborne laser scanning (LiDAR) constituted the basic source material. The triangle grid “Mesh” of Ostrów Lednicki, created with the CloudCompare application, was the final result.

In the last “Virtual Reality Workspace”, spatial data integration and programming script implementation constituted the main operation (Figure 2). The Unity game engine was the application for this workspace. To develop the workspace, a separate specification of visualization parameters, such as the sound system (including the footsteps sound, the wind sound and the rustle of leaves), graphic assets (including trees), programming scripts (including the walking, first-person camera, VR controller, and light) and the Unity scene (Figure 3—layout of the geographical aspect of the virtual walk), was prepared. The EXE program, designed for mobile VR goggles (Figure 4), constitutes the final result of the VR application development process.

We assumed that each research participant will cover the same distance, stopping by the same viewing points, and will answer questions, standing by the last viewing point. Hence, before we prepared the survey, the designing cartographer, who had previously visited the stronghold, determined the starting point and the trails, as well as marked viewing points on the basis of the comparison of the 3D image seen in VR goggles with the real street view from the stronghold (Figure 3). We added rustle of trees and footsteps that are heard by the user during their virtual walk.

4.3. Creating and Conducting a Questionnaire

According to the concept adopted, the survey was addressed to experts and gamers, representing different states of knowledge and skills, which was supposed to help obtain different opinions about the demonstrated way of becoming immersed in historio-geographical space including elements that were historically mutually exclusive.

The first group consisted of experts on the Middle Ages who derived their knowledge from the literature and their visits to Ostrów Lednicki. Nobody from that group was a gamer. The other group consisted of serious story game users via non-immersive screen-based presence, who had general knowledge of history from school and had never visited Ostrów Lednicki. Gamers declared to spend over 10 h per week, playing story games based on extended storylines. It was possible to invite 10 respondents (5 women and 5 men, aged roughly 35–60) to the first group; therefore, the second group also consisted of 10 members (men aged roughly 20–24).

We assumed that each participant would be constantly standing during the survey, basically right from the moment of putting on VR goggles and being handed the controller, then during “walking” down the trail and, finally, while completing the survey at the last viewing point. The difference in the course of the survey was mainly about the initial activity for the gamer, who received the printout of two photographs and was asked to take a look at them: the one with currently preserved remnants of the building’s foundation and the area of the former stronghold in Ostrów Lednicki, and the one of a plaster model of the stronghold from the times of Mieszko I (Figure 1).

Before the participant put the goggles on, he/she was informed that, after putting the goggles on, he would be transferred to the virtual presentation of the stronghold in Ostrów Lednicki, in which the palatium with the chapel and the church from the times of Mieszko I had been reconstructed in the currently existing grass-covered area, along with the remnants of the rampart (Figure 4).

Having put the goggles on, the participant confirmed to see and hear everything correctly and to understand the functioning of the movement buttons placed on the controller that he was holding.

Then, the researcher conducting the survey informed the participant that he/she was located in Ostrów Lednicki, standing on the path before the grass-covered remnants of the rampart (Figure 3 and Figure 4). The researcher asked the participant to stand by the black pale on the rampart and sharing his/her opinion on 5 main topics with 15 detailed issues concerning the entire visualization. With VR goggles on and without any time limits, the respondent answered each question separately and evaluated it. After reading out each question, the researcher would inform the respondent about the opportunity to share his/her own comments (Figure 5).

At the first main topic, the respondents would make the initial evaluation, and at the last, fifth main topic (fifteenth issue) they would provide their final evaluation. Providing evaluation concerning specific questions in three thematic groups was supposed to help obtain separate partial scores for the final evaluation.

Respondents were asked to share remarks (there were four possible evaluation types: very good, correctly, wrong, hard to define) on the following 5 main topics touching upon 15 issues:

• The opportunity to see proportions of the palatium solid figures with the chapel and church in the state from the times of the first ruler of Poland compared to the current state,
• Correctness of the reconstruction of the two pre-Romanesque buildings’ elements: 2A: proportions of the building components, 2B: windows and portals, 2C: roofs and roofing tiles, 2D: plastered walls,
• Functionality of this reconstruction method for two buildings in the current state of the stronghold in the following aspects: 3A: Obtaining knowledge of the features typical of buildings from the Mieszko I’s stronghold by students, 3B: The method of studying historical features of buildings by experts, 3C: The understanding of links between the history of the beginning of the Polish state and the stronghold on the island, 3D: The opportunity to visualize two buildings on the remnants of foundations preserved to this day,
• Parameters of immersion in VR used by researchers: 4A: movement exclusively along the trails and the rampart, 4B: Suggesting the viewing points for the observations of building features, 4C: Observation from the perspective of a pedestrian moving around the stronghold’s area, 4D: Layout of trees with the animation of movement of the leaves, 4E: Hearing the sounds of wind and footsteps, and
• General meaning of the demonstrated immersion method for virtual observation and interpretation of two reconstructed Romanesque buildings (derived from expertise) in the current state of the grass-covered rampart.

4.4. Graphically Juxtapose Opinions and Comments

According to our concept, we have presented the results of the survey, juxtaposed and with visual support. To facilitate the analysis and synthesis visually, three colors, associated with those of traffic lights, were employed for evaluation: green for “very good”, yellow for “correct”, red for “incorrect/wrong” and violet for “hard to define” (Figure 6 and Figure 7). As empty boxes in tables were in colors symbolizing specific grades, symbols in black line were used to present the additional information in order not to conceal the colorful boxes. Simple emoticons for presenting three faces were used: “smiley face”: positive feedback; “straight face”: neutral feedback; and “frowning face”: negative feedback. Additionally, the exclamation mark (“!”) was placed in the table near the emoticon if, in his/her comment on the feature, the participant referred to knowledge from academic publications and/or his/her own experience from the visit to the stronghold. To facilitate the comparative analysis of the evaluation made by experts and gamers according to the group of issues and detailed topics, we prepared a comparative bar diagram in Figure 8, using analogical colors as in Figure 6 and Figure 7.

5. Results

After 20 questionnaires, it turned out that the time gamers needed to complete the experiment ranged between 12 and 19 min, with experts taking between 17 and 26 min, as they would frequently make comments. In the expert group, two women initially reported slight discomfort after putting their VR goggles on.

Figure 6 presents evaluations and comments provided by ten experts. Out of 150 evaluations, “very good” (60%) and “correctly” (27%) prevail. Each expert made a comment on selected answers to 15 questions at least twice. Two experts commented on 11 issues. The experts made 62 comments per 150 evaluated features. In 37 evaluations (60% comments), the experts referred to the knowledge derived from academic publications or their visits to the stronghold. Neutral comments definitely prevailed (47%), and the number of positive comments was the lowest (19%).

Only two experts answered to all the questions: “very good” and “correctly”, and five of them used all four types of evaluation. Interestingly, only one negative answer was not supported with a comment, and, out of all 12 negative answers, 10 were supported with negative comments, and 1 comment was neutral. Surprisingly, in 7 cases, experts delivered negative comments on questions evaluated as “correctly” and, in one case, as “very good”. Generally, experts would provide negative feedback along with negative evaluations, making references to expertise from academic publications or their knowledge of the stronghold from real life. The “hard to define” feedback was usually left without any comments. The majority of cases of positive evaluation were left without comments. Only 4C (Observation from the perspective of a pedestrian moving around the stronghold’s area) was not commented upon by experts, receiving nine “very good” and one “correctly” answers. Other issues were commented on at least thrice, with 4A (Movement exclusively along the trails and the rampart) receiving the most comments. In general, each out of three topics received a similar number of comments.

Although topic 1 (The opportunity to see proportions of the reconstructed buildings) and topic 5, the last one (General meaning of the demonstrated immersion method), received mostly positive comments, the detailed questions from the topics: 2 (Correctness of the reconstruction), 3 (Functionality of this reconstruction method) and 4 (Parameters of the VR immersion) had diverse comments. 3D (The opportunity to visualize two buildings on the remnants of foundations preserved to this day: nine “very good” answers and one “correctly” answer) and 4C (Observation from the perspective of a pedestrian moving around the stronghold’s area) were the exceptions. 4D (Artificial trees with the animation of movement of leaves) received the lowest evaluation. Topic 2 had the highest number of references to expertise and individual experience, which is understandable as it referred to correctness of reconstruction of some elements of two Romanesque buildings. Out of 62 comments made by experts, the following were made repeatedly:

• 1: in the literature, there are different hypotheses of reconstruction,
• 2A: consistency with the plaster model; correctness of reconstruction according to the academic versions remembered; the palatium was a bit taller; the roof cubature should be larger,
• 2B: portal should be rather in a different place; portals should be pre-Romanesque; coherent convention of the pre-Romanesque style should be maintained,
• 2C: shingle roofs rather than lead roofs,
• 2D: there are two parallel hypotheses that walls were either plastered or non-plastered,
• 3A: too simplified; very suggestive visualization to students,
• 3B: too few reconstructed elements for experts to use; no view of the lake; too few details,
• 3C: trees disrupted the view; trees are artificial, trees should be slender as there are tall alders growing there,
• 3D: consistency with other reconstructions,
• 4A: 5 comments expressing the urge to freely walk around the stronghold; 3 comments expressing the urge to walk round the palatium,
• 4B: 4 comments about accurate layout of the viewing points; 3 comments on the very well located last viewing point by the pale,
• 4D: better remove the trees as they were not there in the 10th century,
• 4E: the rustle of trees reflects the real sound; there is no twitter of birds, and
• 5: the structure of plaster on the walls should be changed; other elements of the stronghold can be added; time conflict resulting from the combination of the 10th and the 21st century; the rampart was taller in relation to buildings in the 10th century.

Figure 7 demonstrates the evaluation by gamers with the prevailing green (64%) and yellow (30%) boxes, comprising 94% of boxes in total. In general, all issues received high evaluation, 3B (The method of studying historical features of buildings by experts) and 4D (Artificial trees with the animation of movement of the leaves) being the exception. Three questions, 3A (Obtaining knowledge of the features typical of buildings from the Mieszko I’s stronghold by students), 4E (Hearing the sounds of wind and footsteps) and 5 (General meaning of the demonstrated immersion method), received 10 “very good” evaluations. Four gamers made single comments on the fourth main topic (Parameters of immersion in VR used by researchers) in which they were highly experienced due to their “presence” in the world of computer games. Two negative comments concerned 4D (Layout of trees with the animation of movement of the leaves), as in the opinion of gamers, such artificial movement of leaves on trees disrupted the perception of the 3D view. The other two comments were made to express the willingness of gamers to freely move around the stronghold.

Comparing the way that the opinions were expressed by 10 experts and 10 gamers, one can draw a conclusion that the group of gamers is more homogenous as the same comments are made by most gamers on the same issues. On the other hand, a diverse combination of colors and faces in boxes denoting the same issues indicates individualized personalities of experts. All the respondents used mostly the “very good” or “correctly” evaluation, even though gamers used them slightly more often (94%) than experts (87%). Such high evaluation was generally connected with a positive attitude and interest in new technologies since no respondent had previously had the opportunity to become immersed in VR in goggles.

The “hard to define” answers constitute only 4% of all answers in both groups of respondents, which proves that all of them were highly committed and avoided ambiguous answers. We assumed that gamers would avoid unambiguous evaluation of issues addressed to historians. However, this assumption turned out to be wrong as gamers would willingly comment on issues connected with the knowledge of medieval history, which proves that one is willing to make judgements concerning the topics that one is no expert on, lacking expertise and experience in the field. On the other hand, experts on the history of the stronghold that were not familiar with the world of screen games would express their opinions on parameters of the application that rather required the experience of gamers. Skepticism, expressed more by gamers rather than experts, about the recognition of high functionality of the application in terms of studying historical features of buildings by experts, came as a surprise.

The marks to the third main topic referred to possibly open, informative and didactic functionality of the application to the wide audience. What is striking is that two issues (3A with 100% “very good” and 4C) received similarly positive feedback. The greatest discrepancy in marks occurred for 4A, as experts mostly reported the willingness to freely move around the palatium, whereas gamers approved of the designed way of moving around. It seems that the first-hand experience of experts resulted in their request to be able to move around the most significant building in the stronghold (Figure 8).

Experts were the most critical of the issues about VR parameters, referring to the real state of the area, e.g., criticizing the wrong shape of the artificial trees as they remembered that alders growing there were taller. A few gamers criticized the excessive movement of leaves on trees as they were used to more subtle movements of leaves in games. Interestingly, experts and gamers provided the lowest evaluation to the layout of trees with the animation of the leave movement.

Experts delivered more diverse evaluation with the higher number of comments, in most cases referring to their expertise and first-hand experience in terms of the second main topic. Gamers, in turn, evaluated these issues in a positive way and their lack of criticism, combined with lack of expertise, may be linked to their frequent “presence” in the world of games with buildings only imitating the original medieval architecture.

6. Discussion and Conclusions

Generally, one can conclude that the evaluation by experts in medieval strongholds and serious screen game users of the specific way of immersion in the VR of reconstructed buildings in the current area provided researchers with the extended view on its effectiveness and attractiveness, as well as with suggestions on further design process.

Analog cartographic and graphic materials constitute a solid foundation for creating specific immersive VR for cultural objects that have only remnants and a marked rampart in the relief left (Figure 1) [26]. Reliable reconstructions are saved as perspective drawings in academic publications or in the form of plaster models. Selecting the reconstruction version out of several clashing historical versions, as well as proper analysis and selection of the extensive source material, is what constitutes a problem in the concept of design. Analog and digital data management can be conducted in the orderly way thanks to parallel actions in several workspaces. Unfortunately, despite the assumption to use the lowest possible number of digital workspaces, four workspaces had to be used (Figure 2). A possible simple implementation has become crucial for creating a mobile VR application based on the spatial data game engine, geometry of solid figures and their graphic looks resulting from parallel modeling in three workspaces: 2D raster graphics, 3D buildings, and 3D relief [25].

Working out the same tasks and groups of questions for two different groups of users of the same virtual presentation of the historico-geographical space is essential to make a comparative analysis and draw coherent conclusions. Limiting the choice to only four answers, out of which the right one was supposed to be selected, allowed respondents to focus on delivering the answer and making the comments. It turned out to be relevant as each respondent asked for repeating the issues at least once, which probably resulted from the fact that it was the first time he/she was in the virtual stronghold with VR goggles on and listening to the questions of the researcher at the same time.

The fact that no gamer referred to photographs of the current state of the stronghold and the plastic model watched before putting their goggles on came as a surprise. On the other hand, experts would willingly refer to their expertise, the plastic model, and real-life experience in their comments. It may prove that only deeply engrained knowledge, supported by real-life experience, evokes appropriate reaction, whereas a short presentation of photographs without further explanation will not be remembered and will fail to provoke any comments by gamers, even though all gamers had general knowledge of the beginning of the Polish state and early styles in architecture.

A controversially designed “walking” down currently existing trails and grass-covered remnants of the rampart near the non-existing palatium and church that were 1000 years old was commented upon as a strange clash only by two experts. Thus, it can be thus adopted that users generally do not mind such a way of immersion.

On the basis of the evaluation obtained, one can draw a conclusion that the knowledge of the pre-Romanesque architecture and private first-hand experience had a positive impact on one designer, influencing the way the trail and viewing points were marked. However, experts, who had visited the stronghold more often, demanded that the twitter of birds should be added. It can be linked to synesthesia of the experience, just as it was the case for sounds in the process of cartographic communication [27,28,29]. A frequent use of games by another designer resulted in adding artificial footsteps that received a positive feedback from both user groups.

Graphic elements that enriched evaluation and comments allowed one to visually analyze and synthesize facts and capture links between them (Figure 6, Figure 7 and Figure 8). The adaptation of the ”Chernoff faces” method [12,21], known in traditional cartography, along with the employment of the exclamation mark, enabled one to link evaluation to comments, e.g., despite neutral and critical comments experts indicated positive features, and the majority of cases of negative evaluation were supported by negative comments and references to real-life experience. The issue of whether or not the employment of more types of faces would make the analysis more accurate remains open.

We are left with the issue of how to treat opinions from two different user groups. Both groups consisted of 10 members, and it was impossible to make them more numerous as it was necessary to formulate the criteria for experts with the real-life experience in a highly strict way. Moreover, we wanted to obtain opinions about the VR application that would confront the real world with the virtual one and juxtapose the comments provided by gamers who were transferred to the virtual reality. Hence, we incline toward making a complementary use of opinions provided by two user groups in the process of designing such applications. In our opinion, the choice of suggestions received from experts and gamers should be consistent with the concept of the multimedia presentation designers.

We are aware that there is no simple recipe for designing a good historical-geographic immersive application. On the one hand, applying the principles of designing cartographic signs facilitates the design process, but on the other hand, it limits the graphic creativity of the designer [30]. Therefore, the method of a special compilation of evaluation and comments in several design aspects proposed here is rather aimed at stimulating the design process in the directions preferred by designers of next applications.

Generally speaking, both groups demonstrated great openness to new opportunities to present the stronghold in VR, which proves high attractiveness of such medium. Informativeness in the historical and cultural context of the place, unique in terms of the birth of the Polish state, has also been evaluated very highly. However, defining the effectiveness of this medium in terms of visualizing historico-geographical space for public users requires further research that would also include other groups of respondents.