Multimodal Dictionaries for Traditional Craft Education

Abstract

We address the problem of systematizing the authoring of digital dictionaries for craft education from ethnographic studies and recordings. First, we present guidelines for the collection of ethnographic data using digital audio and video and identify terms that are central in the description of crafting actions, products, tools, and materials. Second, we present a classification scheme for craft terms and a way to semantically annotate them, using a multilingual and hierarchical thesaurus, which provides term definitions and a semantic hierarchy of these terms. Third, we link ethnographic resources and open-access data to the identified terms using an online platform for the representation of traditional crafts, associating their definition with illustrations, examples of use, and 3D models. We validate the efficacy of the approach by creating multimedia vocabularies for an online eLearning platform for introductory courses to nine traditional crafts.

1. Introduction

Traditional crafts are considered part of human societies, embodying cultural heritage and artisanal expertise passed down through generations. Crafts are characterized by a certain type of making in which objects are created by hand through the skilled use of tools [1], objects made or repaired for functional use [2] and not solely of ornamental value. As repositories of intangible heritage, crafts encompass a rich collection of techniques, materials, and traditions that contribute to the shaping of the identities and histories of communities worldwide. Today, in the new digital age, preserving and transmitting such knowledge and practices poses new challenges. Addressing these challenges can support the creation of new opportunities for the crafts sector.

The study of crafts has been traditionally served from the perspective of social anthropology [3]. The usage of ethnographic methods invites experts to explore how handmade artifacts are reinforced with cultural and social significance [4]. Ethnographers collaborate with communities to observe and participate in the creation and use of crafts, uncovering how these practices reinforce social bonds, convey identity, and transmit traditional knowledge across generations.

Our work endeavors to bridge the traditional study of crafts through social anthropology and modern technology for the preservation of ethnographic data. Thus, we foresee the usage of multimodal data as a medium for the preservation and transmission of ethnographic understanding. This is supported through the implementation of novel digital resources compiled in the form of multimodal dictionary entries in an online craft representation system.

Through collaboration with craft practitioners and the support of social anthropologists [5], we employ a diverse array of recording techniques, including photography, multi-view videos, egocentric videos, motion capture, and 3D digitization technologies. These technologies support the capturing of craft processes, materials, tools, and workspaces, to encapsulate the ingredients of traditional craftsmanship in a digitally accessible format. This work also rationalizes the application of these digitization methods to the task at hand through the provision of application guidelines.

Supportive to our methodology is the integration of practitioner insights through in-depth interviews and reflective sessions. Craft practitioners help define the scope of documentation, while their engagement in the review and reflection of recorded materials ensures that the resulting dataset faithfully represents the craft process from the insider’s perspective.

From a technical perspective, the subsequent segmentation and annotation of this multimedia dataset support the online semantic representation system with valuable data. By organizing craft processes into conceptual entities and steps, we construct a semantic framework that documents techniques, materials, and conditions inherent in traditional craftsmanship. At the heart of this framework lies the notion of the “recipe”, called a process schema in our representation, which is a structured abstraction of the crafting process comprising interconnected steps and conditions.

The result of the aforementioned process is multimodal dictionaries of traditional crafts in the form of a dynamic repository of knowledge that preserves the embodied wisdom of traditional artisans for future generations. In this publication, we present the methodology, framework, and initial findings of our ongoing effort to redefine the boundaries of traditional craft documentation.

In this work, we address the development of multimodal digital dictionaries for craft education through the enhancement of ethnographic strategies and the systematization of data analysis and documentation. In this context, the main objective is to address the lack of documentation for specialized educational topics such as traditional crafts. Considering that ethnography is the main discipline focusing on the systematic and in-depth understanding of crafts, we propose an enhancement of ethnographic fieldwork through digital documentation technologies that would result both in the enrichment of ethnographic fieldwork data and the digital documentation of ethnographic outcomes in the form of multimodal digital dictionaries.

The originality of the presented work lies in its comprehensive approach to systematizing the authoring of digital dictionaries for craft education, specifically derived from ethnographic studies and recordings. Unlike traditional methods that often rely on static text descriptions, this work innovatively integrates multimedia elements—digital audio, video, illustrations, examples of use, and 3D models—into the dictionary framework. By presenting detailed guidelines for the ethnographic collection of craft-related data and introducing a classification scheme that uses a multilingual and hierarchical thesaurus for semantic annotation, this research not only enhances the richness and accessibility of craft education resources but also ensures the cultural accuracy and contextual relevance of the terms. Furthermore, the linking of ethnographic resources and open-access data to these annotated terms on an online platform represents a novel way of preserving and disseminating traditional craft knowledge, validated through the creation of multimedia vocabularies for an online platform that covers nine traditional crafts.

The manuscript is organized as follows. In Section 2 we present an overview of the background and related work, with an emphasis on the ethnographic understanding of crafts, multimodal data collection techniques for ethnographic field studies, approaches for integrating practitioner insights in the documentation, and related work on multimodal dictionaries. We conclude this section with an overview of the contributions of this work. Section 3 outlines the proposed methodology, which involves a multi-phase approach to the documentation and analysis of craft practices that results in the formulation of online digital vocabularies. Section 4 focuses on the approach we propose for the collection of multimodal data during ethnographic fieldwork. Section 5 regards methods to analyze and post-process collected data with the objective of their appropriation for usage in online dictionaries. Section 6 regards the process of producing and making available to the public the digital dictionaries. The manuscript closes with a discussion (Section 7) and conclusions (Section 8) on the method and approaches presented by this research work.

2. Background and Related Work

Ethnographic methodologies are considered important for capturing the essence of traditional crafts [6]. Ethnography [7], focusing on fieldwork and participant observation, can support the understanding of craft practices embedded within their socio-cultural contexts. The approach presented in this work utilizes technical means for multimodal data collection, which supports the comprehensive documentation of craft processes by ethnographers, and at the same time makes available the results of this systematic documentation in the form of semantically enriched multimodal online dictionaries.

2.1. Ethnography in Cultural Preservation

Ethnography, as a discipline, is rooted in the systematic study of cultures and societies through firsthand observation and engagement with participants in their natural settings [8,9]. By immersing researchers within the fabric of craft communities, ethnographic studies provide nuanced insights into the cultural significance, social dynamics, and knowledge systems underpinning traditional craftsmanship. Recently, multimodal data collection in the context of ethnographic studies has received increased attention [10] and has been complemented by approaches involving multimodal ethnographic methods that include video recordings, structured interviews, participant observation, etc. [11].

In the craft sector, ethnography [12] identifies and describes the activity of a social unit as “textual construction of reality” [13]. Recently, it has been applied in the workshop, with examples in carpentry [14], glasswork [15], and textile manufacturing [16].

2.2. Multimodal Data Collection Techniques

The richness and complexity of craft practices necessitate the use of diverse multimedia tools to capture the multifaceted nature of the craft process.

2.2.1. Photography

Photography serves as a visual medium for documenting details, techniques, and materials used in craft production [17]. High-resolution images offer a static yet detailed representation of craft artifacts and processes [18], providing valuable visual references for analysis and interpretation.

2.2.2. Video Recording

Video recording techniques, including multi-view videos and egocentric videos, offer dynamic perspectives on craft processes [19]. Multi-view videos capture craft activities from multiple angles, enabling a comprehensive understanding of spatial relationships and gestural movements [20]. Egocentric videos [21] provide a first-person view of the crafting process, offering insights into the artisan’s point of view and decision-making processes [22].

2.2.3. Motion Capture

Motion capture technology facilitates the precise recording of body movements and gestures involved in craft production [23]. By tracking the kinematics of artisanal movements, motion capture data provide quantitative insights into the motor skills and dexterity required for skilled craftsmanship.

2.2.4. 3D Digitization

Three-dimensional digitization technologies enable the digitization of physical objects, tools, and environments associated with craft practices [24,25]. Through techniques such as 3D LiDAR scanning [26] and photogrammetry [27,28], physical artifacts and spatial configurations can be transformed into digital representations, facilitating virtual preservation and analysis.

2.3. Integrating Practitioner Insights

At the heart of our ethnographic approach is the active involvement of craft practitioners in the documentation process [29]. Through in-depth interviews and participant observation, we engage practitioners as co-creators of knowledge, valuing their experiential expertise and cultural perspectives.

2.3.1. Interviews and Participant Observation

Semi-structured interviews provide a platform for dialogue and knowledge exchange between researchers and craft practitioners [30]. By eliciting practitioners’ narratives, experiences, and tacit knowledge, interviews offer valuable insights into the cultural significance, symbolic meanings, and embodied practices associated with craft traditions [31]. Participant observation, complementing interviews, involves immersive engagement in craft activities, allowing researchers to gain firsthand experience and contextual understanding of craft practices within their social and cultural context.

2.3.2. Reflective Practices

In addition to data collection, reflective practices play a crucial role in the documentation process. By involving practitioners in the review and reflection of recorded materials, we create opportunities for collaborative sensemaking and knowledge co-construction. Reflective sessions enable practitioners to critically engage with their craft practices, fostering self-awareness, skill refinement, and cultural preservation.

2.4. Multimodal Dictionaries

Traditional dictionaries primarily rely on textual descriptions to convey meanings and definitions, whereas multimodal dictionaries integrate text with rich media to provide more immersive and contextually rich experiences.

2.4.1. Electronic Dictionaries

Dictionary functions are communication-orientated or cognition-orientated [32]. Communication-oriented definitions generally refer to the design or adaptation of communication to effectively convey information to the receiver. Cognition-oriented definitions focus on the mental processes involved in acquiring knowledge and understanding through thought, experience, and the senses. That is, communication-oriented dictionary definitions aim to enhance practical language use, while cognition-oriented definitions aim to deepen the understanding of how words are processed and structured in the mind. The optimal dictionary contains fully integrated subject-field components that supplement and complement the data included in the articles [32].

Previous research has identified positive effects of picture annotation on vocabulary acquisition [33]. In [34], a framework for multimodal lexicography was proposed, advocating those illustrations of meaning to complement verbal definitions. This paradigm was followed in electronic dictionaries for learning the English language [35,36].

These ideas were extended in [37], where the concept of “multimodal definition” was proposed, integrating verbal definitions with complementary multimodal resources.

2.4.2. Automatic Approaches

Several automatic approaches have been proposed for the organization of information in digital dictionaries. For example, co-clustering has been proposed to associate image descriptors, and textual words that accompany Web images are associated via co-clustering [38]. An alternative approach with the same objective was facilitated through the usage of an unsupervised dictionary learning procedure [39]. In [40], a rule-based machine translation system is proposed for users with no background in linguistics to add new words to a monolingual dictionary.

2.4.3. Domain-Specific Dictionaries

In domain-specific dictionaries, multimedia has a more important role [41]. For example, past work in the domain of architecture and building construction learning work has demonstrated how information extracted from specialized lexicographic resources can be enriched with corpus data and then used to represent specialized concepts [42].

2.4.4. Semantic Technologies for Crafts Representation

Regarding crafts representation, online representation systems building on top of semantic knowledge representation have a strong potential for the documentation and presentation of cultural knowledge. By combining text with rich media, including images, videos, audio recordings, and interactive simulations, these systems offer users a more immersive and interactive learning experience. The Mingei Online Platform (MOP) is an authoring platform for the representation of social and historical context encompassing a focal topic of interest [43,44,45]. The proposed representation is employed in the contextualized presentation of a given topic, through documented narratives that support its presentation to diverse audiences.

The MOP which is considered the foundation for the contribution of this research work is built on top of the Crafts Ontology. The CrO is used to represent all the knowledge about crafts that Mingei is intended to represent and preserve. This knowledge has been collected or co-created within the project. The CrO is an application ontology [46] obtained by integrating several existing ontologies, notably the CIDOC CRM, a top ontology and an ISO standard forming the conceptual backbone of the CrO [47,48]; the Narrative Ontology, a domain ontology focused on the representation of narratives [49,50]; the FRBRoo, a domain ontology for bibliographic records, resulting from the harmonization of FRBR with CRM [51]; OWL Time, a domain ontology recommended by W3C for the representation of time [52]; and Dublin Core for simple resource description [53].

Despite their potential benefits, the development and implementation of multimodal dictionaries pose several challenges [54]. Addressing these challenges requires interdisciplinary collaboration among researchers, practitioners, technologists, and cultural heritage stakeholders to ensure the ethical and sustainable development of multimodal dictionary platforms.

Another example comes from the domain of coastal engineering events, where a process-oriented terminology was proposed together with a domain-specific ontology to link the causing agents, processes, and results [55].

2.5. Contribution

The main contribution of this work regards the systematization of the authoring of digital dictionaries for craft education based on ethnographic studies, fieldwork, and craft recordings. To achieve this, we extend previous work in the MOP towards implementing the Craeft Authoring Platform. We seek to extend the Crafts Ontology to support multilingualism, external dictionaries, and richer representation of digital assets to formulate multimodal craft dictionaries. At the same time, to support such systematic acquisition of data, this work analyzes alternative data collection techniques and provides application guidelines for their appropriation in the context of ethnographic fieldwork. The combination of application guidelines and the proposed classification scheme for craft terms supports the implementation of a multilingual and hierarchical thesaurus supported by an online platform for the representation of traditional crafts.

Regarding the domain of education, this work contributes in several meaningful ways. Traditional crafts are an essential part of cultural heritage, embodying unique techniques, materials, and traditions. By documenting these crafts comprehensively, this work ensures that valuable knowledge is preserved for future generations. This preserved knowledge serves as an educational resource, providing students and researchers with access to cultural practices that might otherwise be lost. The creation of multimodal online craft dictionaries offers a rich, interactive educational tool. These dictionaries compile visual, textual, and semantic data about traditional crafts, making the learning experience more engaging and comprehensive. Students can explore craft techniques through videos, 3D models, and detailed annotations, enhancing their understanding through multiple sensory inputs.

By integrating social anthropology with digital technology, this work fosters an interdisciplinary approach to education. Students from fields such as anthropology, digital humanities, and even engineering can benefit from this integrated perspective. The methodologies and findings encourage learners to think beyond disciplinary boundaries and appreciate the interconnectedness of cultural practices and technological advancements.

The documentation process itself, which involves advanced recording techniques such as photography, multi-view videos, motion capture, and 3D digitization, serves as a practical educational resource. Students can learn about and practice these cutting-edge documentation technologies, gaining valuable skills that are applicable in various professional contexts, including digital archiving, virtual reality development, and cultural preservation.

The project highlights the importance of ethnographic methods in understanding the social and cultural contexts of traditional crafts. By documenting the processes and interviewing practitioners, students learn the significance of participant observation and reflective engagement in ethnographic research. This deeper understanding of ethnographic methods enriches their research capabilities and critical thinking skills.

The guidelines and best practices provided in this manuscript serve as practical tools for educators and students engaged in craft documentation projects. By following these recommendations, students can conduct their projects more effectively. Additionally, the critical analysis of these methods and the reasoning behind each guideline encourage students to think critically about the choices they make in their documentation practices.

The digital nature of the multimodal dictionaries ensures global accessibility. Students and educators from around the world can access these resources, promoting inclusive education. This accessibility is particularly important for those in regions where traditional crafts are prevalent but resources for documentation and preservation are limited.

In terms of providing information for future research developments, the proposed approach has great potential, particularly in the domain of AI. Having annotated datasets is crucial for performing accurate training of AI models, and this is exactly what this research work does for a challenging domain in which much of the knowledge is tacit and deeply embedded within specific cultural contexts; annotated datasets ensure that AI models can capture details that might otherwise be overlooked.

Annotated datasets can enhance the AI’s ability to provide contextually relevant insights, support advanced applications such as semantic search and intelligent tutoring systems, and facilitate the creation of more effective and engaging educational tools. As such, this work not only aids in the preservation of cultural heritage but also empowers AI technologies to support and promote traditional crafts in a manner that is both respectful and innovative.

3. Method

3.1. Formulation of the Methodology

This method was formulated during the past seven years in the domain of the representation and presentation of traditional crafts. In this process, we developed digital tools and guidelines for their application to achieve a digital representation of a craft instance that includes digital representations of knowledge elements and digital assets. In this process, multidisciplinary expertise was required, including that of historians, anthropologists, social scientists, and museum experts, as well as a wide range of technical and scientific expertise in the domain of digital heritage.

The methodology proposed in this work is the product of working with ethnographers, anthropologists of techniques, and historians in the context of ethnographic fieldwork in Germany (silk weaving) and Chios, Greece (mastic cultivation), with CERFAV in France (glass blowing), and in Limoges-France (porcelain); Crete, Greece (pottery, textile production); Yecla, Spain (woodcarving); Ioannina, Greece (silversmithing); and Tinos, Greece (marble carving). In this context, we have applied advanced digitization methods together with the above-mentioned scientists, and we have produced and applied a rich set of digital tools for the analysis and digital documentation of the outcomes.

At the same time, by introducing a classification scheme that uses a multilingual and hierarchical thesaurus for semantic annotation, we enhance the richness and accessibility of craft resources.

The successful application of the above-mentioned approach to craft documentation resulted in the formulation of the methodology provided by this research work. The main contribution of each discipline in the formulation of the proposed methodology includes:

• Ethnographers: Ethnographers conduct fieldwork to immerse themselves in the cultural and social practices of the artisans in each location. They document the traditional techniques, tools, and materials used in the crafts. At the same time, they conduct discussions, interviews, and observations on craft sessions and acquire detailed insights into the daily lives and work environments of the artisans, contributing to an in-depth understanding of the cultural context and the significance of these crafts. The main discussion points here included the identification of the main documentation points that are required for their work, and together with technical disciplines they formulated a set of technical documentation tools to support their work. The main outcomes regarded the appropriation of the traditional photographic and video documentation techniques to their workflow.
• Anthropologists of Techniques: Anthropologists of techniques study the methods and processes involved in various crafts. Their expertise is in the understanding of techniques and how techniques are learned, transmitted, and adapted within each community. Their work requires more advanced capturing tools to preserve and reuse the gestural knowledge of practitioners. The main discussion points regarded the application of recording mediums and digitization tools to acquire workshops, tools, and techniques. In workshops and tools, the need for digitization methods including photographic documentation, 3D reconstruction, and manual 3D modeling were discussed, identifying where each of these is applicable. On the subject of gestural know-how, a combination of scene recording, egocentric video documentation, photographic documentation of key actions, and keyframe extraction was discussed for documentation. Another discussion point for reenactable documentation regarded the integration of motion capture technologies to accurately record individual actions and processes in a way that is directly remappable to virtual humans.
• Historians: Historians provided a historical perspective on the crafts and their evolution. They researched the origins and development of each craft, tracing changes in techniques, materials, and cultural significance over time. Their work contextualized the contemporary practices within a broader historical framework, showing how traditions have been maintained or transformed across generations. Discussion points with historians regarded the type of social and historical information that should be documented to understand the context of the evolution of individual craft instances.
• Technical experts: Discussion with experts on the usage of digital technologies regarded (a) specification of photographic documentation techniques and variations to visually capture the processes, tools, and final products; (b) specifications for recording interviews with artisans, capturing the sounds of their tools and environments, and documenting the processes of creating crafts in real time; (c) MoCap technologies appropriation to understanding the intricacies of each technique, preserving movement information which is essential for the transmission of these skills to future generations; and (d) 3D digitization technologies appropriation to support the creation of high-resolution digital models of the tools, artifacts, and sometimes even the workspace layouts used by artisans. Discussion points with technical experts also regarded the integration of the various forms of data (audio, video, motion capture, 3D models, photographs) into cohesive digital archives and their appropriate post-processing to be usable for documentation purposes.
• Digital Preservation and Accessibility: Ensuring that the collected data are preserved for long-term use and are accessible to researchers, educators, and the public was another crucial contribution. Technical experts contributed to the implementation of the digital repository and online platform where the documented data will be stored and accessed. They also worked on ontologies and metadata standards to make the information easily searchable and usable.

This manuscript presents the methodology rationale and a step-by-step guide for its application in the craft context.

3.2. Presentation of the Methodology

In this section, an overview of the methodology followed in the study is presented. The methodology involves a multi-phase approach to documenting and analyzing craft practices, resulting in the formulation of online digital vocabularies.

In the first phase, ethnographic data collection is conducted within the context of craft practice using several methods. In this context, data collection is foreseen to be conducted in parallel with ethnographic fieldwork that involves working closely with the craft communities and facilitating a plethora of documentation techniques such as structured and unstructured interviews, observation studies, craft practice documentation using fieldnotes, etc. In our methodology, we enrich the aforementioned standard methods with digital ones and enhance the capacity to capture details in digital forms that relate to the craft practice.

Photographic documentation is employed to capture high-resolution images that detail the craft process and materials. Motion capture (MoCap) technology records the movements involved in the craft practice, providing insights into the physical actions and techniques used. Material transformations are observed and documented to understand the changes in materials throughout the process. Video documentation, including egocentric video, captures the dynamic processes and contextual interactions of the craft practice. Additionally, digitization and 3D modeling techniques are used to create digital and three-dimensional representations of the craft objects and tools. Verbal content is recorded to document the narratives and explanations provided by the craft practitioners, while event logging systematically records significant events and actions during the craft practice.

These digital recordings are further enhanced through self-reflection interviews with the practitioners in which they are requested to review recorded data, comment, and recall their actions and techniques. In this stage, it is surprising how much information on the craft practice has been embodied in the gestures of the practitioners, and allowing them to self-reflect can provide details that even the practitioners themselves cannot recall since these have been integrated into their repertoire of automated craft movements.

Following data collection, a pre-processing of the ethnographic data is performed. Audio recordings are segmented into meaningful portions based on content and context, while MoCap data are broken down into distinct actions or sequences. Video footage is segmented to isolate specific events, actions, or phases of the craft practice. Visual annotations are added to the photographic and video documentation to highlight key elements and features. Synchronization of these various data types ensures that corresponding segments from different sources are aligned. Event parsing involves identifying and categorizing significant actions and interactions. All data are then stored for long-term preservation, with keyframes extracted from videos to serve as representative snapshots of the documented activities.

In the final phase, the analyzed data are utilized to produce online vocabularies. These vocabularies relate terms to digital assets stored as media objects and integrate semantic knowledge to represent basic knowledge entities. They are linked to external dictionaries, such as the Getty Art and Architecture Thesaurus (AAT), the Catalog of Art Collections (CONA), the Thesaurus of Geographic Names (TGN), and the Union List of Artist Names (ULAN), to ensure alignment with domain standards. The result is a collection of digital dictionaries specific to each instance of craft practice.

The overview of the proposed process is graphically illustrated in Figure 1.

3.3. Technical Implications for the Application of the Methodology

The application of the provided methodology is bound to some technical specifications of the technologies used in the context of the projects where the ethnographic studies took place. As such, the provision of guidelines, methods, and tools for achieving the objectives of this work may or may not be suitable for another set of technologies. To enhance the clarity and applicability of the method in this section, we provide a set of technical specifications of the technologies used. For photographic documentation, we employed a high-res DSLR camera with a standard 18–55 mm AF lens. The video documentation was performed using DSLR for video recording, GoPro action cameras for a wide field of view recordings, and head-mounted GoPro for egocentric views. Text transcripts were acquired using TurboSquid and YouTube. Digitizations and material transformations were captured using standard photogrammetry-based digitization techniques and the Pix4Dmatic 1.63 software. Images were acquired mainly using the DSLR camera. The 3D modeling was done in Blender. For the acquisition of motion, two motion capture suit alternatives were employed: the first was the NanSense MoCap suit and the second was the Rococo MoCap suit. Both setups included hand-tracking using MoCap gloves. Video and audio segmentation was done in Adobe Premiere, but free alternatives can be also used. Keyframe extraction was done using a VLC media player, and visual annotation was performed manually by editing keyframes. Synchronization of different audiovisual sources was performed using the audio channel of the acquired recording. Long-term storage was acquired by applying all post-processed data as open data repositories in Zenodo. Digital preservation was performed using the Craeft Authoring Platform, and external dictionaries of terms were linked as external linked data.

4. Ethnographic Data Collection

Collection of knowledge is the process of gathering information regarding a topic. The systematic organization and analysis of this data, as referred to by Sack in [56], result in a comprehensive understanding of the researched subject. The objective of the ethnographic knowledge collection on crafts is to align with Sack’s concept of ‘knowledge mining’ and undertake a semantic annotation of all gathered information, ultimately establishing a dictionary on the research topic. This dictionary can then help researchers, individuals, and public or private institutions retrieve information on the topic according to their interests. It is important to mention, though, that those collections of knowledge can differ regarding their content (even when it is about the same topic), because of the vast information that might exist, or their design, depending on the institutions that created them [57].

In this work, we extend the range of ethnographic recording to include novel digital media, and thus the ethnographic study on a given subject results in a diverse array of digital files harnessed to encapsulate various facets of the craft-making process.

The following mediums and modalities are explored in this work:

Audio recordings serve as a capturing mechanism for spoken interactions, conversations, and ambient sounds within the crafting session. This auditory archive not only immerses the researcher in the environment but also facilitates an in-depth analysis, especially when interviews are conducted.

Video recordings emerge as powerful tools for capturing visual information, including body language, facial expressions, and non-verbal cues. This visual medium is particularly valuable for comprehending the scene and delving into the intricacies of social interactions and behaviors within the natural context of the crafting session. Video recordings of interviews, if conducted, stand as indispensable resources for subsequent analysis, offering a visual dimension to the spoken narratives.

Motion capture recordings trace the 3D trajectories of human movements, providing insights into the gestures and motions during craft actions. This medium becomes particularly important for understanding and modeling human motion within the crafting process.

Photographs, as a visual medium, serve to capture details of the environment, artifacts, settings, and the participants themselves. Beyond mere visual documentation, photographs play a crucial role in contributing to the broader ethnographic narrative, offering a tangible visual understanding of the contextual elements at play.

Digitization of tangible entities involved in the crafting process further illuminates and preserves the materials, the tools, the workspaces, and the craft products. The documentation of endurant entities is implemented with digitization technologies.

The appropriate application of these ethnographic methods is important to achieve the most positive results. As such, in the rest of this section, we provide an overview of these methods together with guidelines on their appropriate application in the context of the craft. These guidelines can be considered as an extension of past guidelines we have formulated on the application of the Cognitive Load Theory to e-learning on traditional crafts, where we appropriate the usage of multimedia technologies for craft training [58].

4.1. Documenting Practitioner Motion

4.1.1. Images

When activities are considered, such as crafting actions, carefully selected photographs can serve as “keyframes”. When sequences of keyframes are juxtaposed in temporal order, they trigger visual perception to “interpolate” the motion between them [59], in this way conveying the recorded motion or event. The selection of appropriate keyframes requires insight and judgment, to select the most indicative and educational ones. The selection of the camera viewpoint is also important in producing informative keyframes. Based on the work on craft digitization projects, the following set of guidelines can be employed for image acquisition:

• GIA_1: Gain a comprehensive understanding of the craft action or activity being documented by studying the process, steps, and variations.
• Success criteria: This can be validated by drafting an outline or schema of the craft action to be documented, highlighting critical processes, steps, and variations. This can be cross-examined with the craft practitioner.
• GIA_2: Identify key actions and determine the critical actions or steps within the process that need to be documented.
• Success criteria: This can be validated in the same manner as GIA_1.
• GIA_3: Do not start capturing until you feel that GIA_1 and GIA_2 are fully adhered to.
• Success criteria: You have drafted the schemas and cross-validated them with the craft practitioner.
• GIA_4: Choose keyframes that effectively represent each identified action. Keyframes should be clear, informative, and visually engaging.
• Success criteria: Review the selected keyframes in conjunction with the schema and make sure that each step action is sufficiently captured by the keyframe. Discuss details with the craft practitioner.
• GIA_5: Consider temporal order and arrange the selected keyframes in a logical temporal order that accurately reflects the sequence of actions.
• Success criteria: Reproduce the selected keyframes in temporal order and make sure that the action documented is readable.
• GIA_6: Exercise insight and judgment and choose indicative and educational keyframes, providing viewers with information to perform the task effectively.
• Success criteria: Make sure you have validated GIA_5, then go back and examine each keyframe to make sure that all required information is visible and there is a lack of occlusions or screen cropping that hides important information.
• GIA_7: Pay attention to the camera viewpoint since it is crucial in producing informative keyframes. Select viewpoints that offer the clearest and most comprehensive view of the action, enabling viewers to understand the process easily.
• Success criteria: Make sure you have validated GIA_6. If not, go back and select another set of keyframes. If these are not sufficient, then the camera viewpoint should change, or additional takes from an extra viewpoint should be required
• GIA_8: Maintain consistency in style and presentation throughout the keyframe sequence, making it easier for viewers to follow the instructions.
• Success criteria: Make sure you have validated GIA_5 and GIA_6. If yes, make sure that keyframes from a single viewpoint are used to display the action in temporal order. If this is not possible, validate GIA_7 and acquire additional viewpoints. Use additional viewpoints sparingly and only when necessary to display parts that are hidden from the original viewpoint.
• GIA_9: Consider audience perspective. Anticipate their level of familiarity with the task and adjust the level of detail and explanation accordingly.
• Success criteria: This affects the application of GIA_4 to GIA_8. Low familiarity should result in increasing the number of keyframes to introduce more intermediate steps. GIA_5 to GIA_8 should be adapted accordingly.
• GIA_10: Test the keyframe sequence with individual representatives of the target audience to identify whether the sequence remains readable. Gather feedback and iterate to improve clarity and effectiveness.
• Success criteria: Test with the recorded practitioner for validating accuracy and with end users to validate information quality. If rates are low, the processes from GIA_1 to GIA_9 should be repeated.
• GIA_11: Document and organize the selected keyframes and their sequence systematically. The documentation method can vary based on your style. A simple spreadsheet can be sufficient at this time.
• Success criteria: A digital documentation of the selected keyframes has been done, and basic information (craft, recording location, format, temporal order, etc.) is documented together with the assets.

4.1.2. Video

Video analysis has been widely used, given a few technical requirements. Video documentation of crafting actions and processes is used in craft ethnography, instructions, and documentaries [60,61]. In [61], audiovisual recording tools are proposed both for the practitioner as well as for the ethnographer (notetaking). In some cases, egocentric views [62,63] are employed, using a worn camera to capture the viewpoint of the practitioner. These have been proven to be usable not only for documentation but also for motion analysis and forecasting in Computer Vision research [62]. Video dictionaries of crafting gestures can support studying craft instances, documenting and safeguarding their authenticity [64].

Ethnographic documentaries are designed to provide an overview, rather than a documentation of the craft. Recently, craft documentaries have focused on recording audio and visual stimuli of crafting scenes, to achieve better immersion in and understanding of the crafting workspace [65].

Recording from one or multiple fixed viewpoints has the advantage of minimizing the intrusion of the video recording equipment. Fixed viewpoints are suitable for crafts that are exercised on a tabletop or workbench, where all actions take place there and gestures are not (self)-occluded by the practitioner.

Active viewpoints, in which the ethnographer chooses and changes the camera viewpoint at will, are the most usable because they exhibit the potential to capture the most informative view. On the other hand, they are dependent on the choice of the ethnographer. As such, multiple “takes”, rehearsals, and reviews with the practitioner are proposed to ensure that the information is indeed conveyed in the acquired video. Active viewpoints are valuable for the documentation of crafts that take place in more fine detail than tabletop settings, such as in sculpture and blacksmithing.

Egocentric videos are captured by a wearable camera, usually worn on the head or the chest. They exhibit the advantage of approximating the visual field and attention of the practitioner. They provide a valuable perspective in the understanding of the crafting activity [66]. They can be applied to virtually any type of craft, but on the other hand, they are not very well suited for crafts where an overview of the workshop is required. A set of useful guidelines for video acquisition are provided below:

• GVA_1: Identify the purpose of the video documentation (i.e., craft ethnography, instructional purposes, documentaries, or other specific objectives). Different purposes may impose a different documentation style and the usage of appropriate equipment.
• Success Criteria: Success is evidenced by a well-articulated objective that guides the selection of documentation style and equipment, ensuring alignment with the intended use.
• GVA_2: Assess the technical requirements for video documentation, including equipment needs such as cameras, lenses, and recording tools. Ensure compatibility with the intended use and environment. If needed, perform test recording to evaluate light conditions, noise, etc.
• Success Criteria: Success is measured by the selection of compatible and appropriate equipment and the identification of optimal recording conditions based on test results.
• GVA_3: Choose the appropriate audiovisual recording tools based on the intended users, whether practitioners, ethnographers, or both. Consider tools that facilitate ease of use and capture high-quality footage.
• Success Criteria: Success is evidenced by the effective and user-friendly operation of the equipment, producing high-quality footage suitable for the target audience.
• GVA_4: Use the best possible equipment available to ensure the usability and long-term sustainability of the captured recordings.
• Success Criteria: Success is measured by selecting a viewpoint that provides valuable insight into the practitioner’s perspective without compromising quality or practicality.
• GVA_5: Evaluate the potential benefits of egocentric views, where a wearable camera captures the viewpoint of the practitioner. Consider also alternative setups if needed, such as chest- or shoulder-mounted cameras.
• Success Criteria: Success is measured by selecting a viewpoint that provides valuable insight into the practitioner’s perspective without compromising quality or practicality.
• GVA_6: Consider the advantages of recording from one or multiple fixed viewpoints that minimize intrusion and are suitable for crafts conducted on tabletops or workbenches. Use wide-angle lenses to capture the entire scene.
• Success Criteria: Success is indicated by minimal intrusion into the workspace and comprehensive coverage of the crafting area, capturing all necessary details.
• GVA_7: Assess the suitability of active viewpoints, where the ethnographer chooses and changes the camera viewpoint. Recognize that active viewpoints require careful selection and may necessitate multiple takes and rehearsals. A mobile phone can be sufficient for this style of recording and can minimize intrusion into the scene with recording equipment.
• Success Criteria: Success is shown by producing dynamic, high-quality footage with minimal intrusion, facilitated by rehearsals and multiple takes if necessary.
• GVA_8: Evaluate the use of egocentric videos. Recognize their advantage in approximating the visual field and attention of the practitioner, but also consider their limitations, particularly in crafts requiring an overview of the workspace.
• Success Criteria: Success is demonstrated by selecting the most effective viewpoint for the craft, balancing the need for practitioner perspective with overall workspace visibility.
• GVA_9: Tailor the choice of recording viewpoint to the specific characteristics of the craft being documented. Consider whether fine detail or an overview of the workspace is more critical for conveying the crafting activity effectively.
• Success Criteria: Success is evidenced by the ability to effectively convey critical details or overall workflow, as required, through the chosen recording perspective.
• GVA_10: Minimize the intrusion of video recording equipment and ensure that key actions and gestures are adequately captured. To do so, choose cameras and mounting positions that do not obstruct the practitioner or impede the crafting process. Furthermore, ensure that when cameras are used close to the practitioner, these do not affect the choreography of the craft. This should have been studied earlier when preparing the ethnographic fieldwork with the participation of the practitioners. Finally, for egocentric recording, select the equipment with the lowest possible height and weight and the most comfortable mounting setup to ensure that the practitioner is not negatively affected by the recording equipment.
• Success Criteria: Success is demonstrated by unobstructed, high-quality footage that accurately represents the crafting process without disrupting the practitioner, validated by pre-recording studies and practitioner feedback.
• GVA_11: Collaborate closely with practitioners during the recording process. Engage in multiple takes, rehearsals, and reviews to ensure that the captured video effectively conveys the desired information and insights. Evaluate, and iterate if needed to ensure appropriate coverage.
• Success Criteria: Success is indicated by the production of video footage that meets practitioner and ethnographer standards through multiple takes, rehearsals, and continuous feedback.

4.1.3. Motion Capture

Motion capture, also known as MoCap, is a technology used to capture and record the movement of people or objects and translate it into digital data. Using suitable sensors or tracking devices, fine-grained gestures are recorded, allowing for detailed analysis. To ensure the ethical conduct of these recordings, key considerations include obtaining informed consent from participants and preserving the confidentiality of the captured data.

The process involves creating a sensor protocol that outlines the type of sensors to be used and their specific placement. This protocol also defines the characteristics of the recording, such as the gestures or steps of the routine to be captured. The placement of sensors is determined, considering the required positions to accurately capture the craftsperson’s movements.

In practical terms, the recording phase includes capturing data from a diverse set of users, approximately 15, to ensure variability and a representative sample. Each user undergoes five repetitions of the recorded gestures. The recorded motion data are then segmented into meaningful units, considering temporal boundaries and the duration of each gesture. This segmentation provides a foundation for isolating specific gestures, facilitating in-depth analysis.

The following guidelines are useful when considering the acquisition of human motion:

• GMA_1: Prioritize ethical conduct by obtaining informed consent from participants and ensuring the confidentiality of captured data. Respect participants’ rights and privacy throughout the entire process.
• Success criteria: Please make sure that you have acquired the rights to record human motion and segment and isolate parts of the acquired motion for documentation purposes, consent that a new product can be built that uses the acquired motion, and consent to train AI models using annotated parts of the acquired motion. Optionally, if commercial use is in your objectives, make sure to also include the usage of the acquired data for commercial purposes. Document all the aforementioned information in the consent form, make sure that each participant has read and understood the comments, and keep both the hard copy and a digital copy of the signed document.
• GMA_2: Create a sensor protocol outlining the type of sensors to be used, their placement, and the characteristics of the recording. This protocol ensures consistency and accuracy in capturing movements and gestures.
• Success criteria: Make sure that there is a documented setup of the appropriate sensors and that the protocol has been tested in the lab before its application in the field.
• GMA_3: Determine the placement of sensors based on the required positions to accurately capture the craftsperson’s movements. Ensure that sensors are positioned strategically to capture fine-grained gestures effectively.
• Success criteria: Validate sensor placement by conducting trial recordings to ensure that all necessary movements are captured with sufficient detail. Success is indicated by clear, accurate data from all required positions.
• GMA_4: Capture data from a diverse set of users, approximately 15, to ensure variability and a representative sample. This diversity helps in capturing a wide range of movements and gestures, enhancing the robustness of the data. The number provided has been heuristically validated and provides a good balance between having a good amount of samples to work on and allowing the production of more intelligent applications by training, for example, AI models to understand different variations of gestures. In the case where only documentation is of importance, this can be reduced to a couple of samples per gesture and a limited number of no more than two users.
• Success Criteria: Achieve a dataset that includes recordings from at least 15 different participants for AI training and a couple of participants for documentation purposes. Review the data to ensure that they reflect a diverse range of movements and gestures.
• GMA_5: Have each user undergo five repetitions of the recorded gestures to ensure sufficient data for analysis. Encourage variation within repetitions to capture nuances and variations in movements.
• Success Criteria: Verify that each participant has completed five repetitions of each gesture, with variations captured. Review the recorded data to confirm the presence of these variations.
• GMA_6: Segment recorded motion data into meaningful units, considering temporal boundaries and the duration of each gesture. This segmentation facilitates the isolation of specific gestures for in-depth analysis.
• Success Criteria: Accurately segment the motion data into distinct, meaningful units. Success is demonstrated by the ability to isolate and analyze specific gestures from the segmented data.
• GMA_7: Implement measures for quality control throughout the recording process. Monitor sensor performance, data integrity, and participant compliance to ensure the reliability and accuracy of captured data.
• Success Criteria: Continuously monitor and document sensor performance and data integrity. Success is indicated by consistent, high-quality data and compliance with recording protocols.
• GMA_8: Maintain detailed documentation of the motion capture process, including sensor protocols, participant consent forms, and recording procedures, as a reference for future analysis and to ensure transparency in research practices.
• Success Criteria: Keep comprehensive records of all aspects of the motion capture process. Success is demonstrated by well-organized and accessible documentation that can be referenced for future analysis and research.
• GMA_9: Utilize appropriate techniques for analyzing motion data, such as statistical analysis, pattern recognition, or machine learning algorithms. Explore relationships between different gestures and movements.
• Success Criteria: Success is shown by the ability to identify and explore patterns and relationships within the motion data effectively.
• GMA_10: Continuously assess and refine the motion capture process based on feedback, insights from analysis, and advancements in technology to enhance the effectiveness and efficiency of future recordings.
• Success Criteria: Success is demonstrated by ongoing improvements in the quality and efficiency of motion capture recordings.

4.1.4. Material Transformations

In crafting actions, the counterpart of practitioner movements is their results [67]. Thus, to better assess and understand this “negotiation” between the maker and the material [68], the digitization of material transformations is proposed. In other words, we should capture the dynamic properties of materials under transformation into artifacts.

Digitizing material transformations can provide valuable insights into the mechanics of the crafting process and help improve the quality of the final product. Digitizing material deformation during a crafting process can be challenging, but several techniques can be used, depending on the specific crafting process.

In this work, we propose a simple two-stage measurement process that involves the digitization of structure before and after the action. This approach involves a 3D scanner to capture the structure of the material before and after the crafting action.

When digitizing material transformations, the following guidelines can be of use:

• GMT_1: Identify appropriate techniques for digitizing material transformations based on the specific crafting process. Consider factors such as the type of material being used, the level of detail required, and the complexity of the transformation.
• Success Criteria: Success is indicated by a documented rationale for the chosen techniques, considering material type, detail level, and transformation complexity.
• GMT_2: Capture the structure of the material before and after the crafting action. Use a 3D scanner to generate 3D models of the material in both states.
• Success Criteria: Success is demonstrated by obtaining accurate 3D models of the material both before and after the crafting action, with clear records of each stage.
• GMT_3: Select suitable methods for 3D reconstruction based on the characteristics of the material and the desired accuracy of the models. Options include time-of-flight scanning, photogrammetry, active illumination, or other modalities.
• Success Criteria: Success is shown by the selection and application of a method that accurately captures the material’s characteristics and meets the desired accuracy.
• GMT_4: Develop and use standardized procedures for positioning the material and operating the 3D scanner to minimize variability between measurements.
• Success Criteria: Success is indicated by consistent, reproducible results across multiple measurements.
• GMT_5: Focus on capturing material deformation during the crafting process. Pay attention to changes in shape, volume, and texture that occur as a result of the practitioner’s actions.
• Success Criteria: Success is demonstrated by 3D models that clearly show changes in shape, volume, and texture.
• GMT_6: Compare the 3D models obtained before and after the crafting action to identify the amount and location of material deformation. Use software tools for quantitative analysis and visualization of the differences between the models.
• Success Criteria: Success is indicated by detailed reports and visualizations that highlight and quantify the differences between pre- and post-crafting models.
• GMT_7: Validate the accuracy of the digitization process through calibration and comparison with ground truth measurements where possible. Conduct sensitivity analyses to assess the impact of measurement parameters on the results.
• Success Criteria: Success is demonstrated by validated, accurate digitization results and documented sensitivity analyses.
• GMT_8: Continuously refine the digitization process based on feedback and insights gained from analyzing the 3D models. Experiment with different techniques and parameters to optimize the accuracy and efficiency of the process.
• Success Criteria: Success is indicated by enhanced accuracy and efficiency in the digitization process over time.
• GMT_9: Document the digitization process thoroughly, including details of the techniques used, the parameters selected, and any challenges encountered. Provide clear and concise reporting of the results, including visualizations and interpretations of the material transformations observed.
• Success Criteria: Success is demonstrated by detailed records of techniques, parameters, challenges, and results, including visualizations and interpretations.
• GMT_10: Keep it simple. Prefer the simplest digitization method that can provide the baseline data required for your work. Digitization is a time-consuming process, so consider twice before moving towards more details and thus more time-consuming methods.
• Success Criteria: Success is indicated by efficient data collection that meets baseline requirements without unnecessary complexity.

4.2. Objects, Materials, and Workspaces

This section regards the digitization of endurant entities, that is the tangible entities involved in the crafting process. These are the materials, the tools, the workspaces, and the craft products. The documentation of endurant entities is implemented with digitization technologies.

4.2.1. Photographic Documentation

In the fields of archaeology, art history, and museology, many recordings and examination methods are based on imaging techniques. The oldest and most central approach is photography. Photographic documentation has been conventionally used in the documentation of tangible heritage since the previous century. Photography has been recommended for the documentation of heritage in general, and crafts in particular [69]. Photographs are essential in showing the appearance of craft products, but also the materials and tools of a craft.

Comprehensive guides in the photographic recording of CH artifacts and sites are to be found in [70,71,72]. Spectroscopic imaging methods provide insights into the chemical composition and physical properties of the materials and artifacts [73,74,75,76].

Photographic methods accurately capture the appearance of artifacts. Today, there are multiple types of photographic modalities used in a variety of domains from industrial inspection to forensics. To simplify the recording task, it is important to classify the imaging target as to its shape and material. We follow the Digitization Standards for the Canadian Museum of Civilization Corporation [77], which is an excellent reference for photographic documentation protocols, classified per the material and artifact type.

When performing photographic documentation of tangible heritage, the following guidelines can be used:

• GPD_1: Consult comprehensive guides for photographic recording of cultural heritage artifacts and sites.
• Success Criteria: Show proficiency in selecting and utilizing authoritative guides on photographic recording. Success can be measured by the correct application of techniques and best practices outlined in these guides during documentation efforts.
• GPD_2: Familiarize yourself with spectroscopic imaging methods, which provide insights into the chemical composition and physical properties of materials and artifacts.
• Success Criteria: Demonstrate an understanding of spectroscopic imaging methods and their applications. Success is indicated by the ability to explain how these methods enhance the documentation process and identify situations where their use is appropriate.
• GPD_3: Recognize the variety of photographic modalities available today, used in diverse domains ranging from industrial inspection to forensics. Understand the different types of modalities and their suitability for various imaging targets.
• Success Criteria: Successfully classify and select appropriate photographic modalities for different documentation scenarios. Evidence includes correctly matching imaging targets with suitable modalities and justifying these choices.
• GPD_4: Classify imaging targets based on their shape and material to simplify the recording task. Understand the importance of categorizing artifacts to determine the most appropriate imaging technique.
• Success Criteria: Accurately categorize artifacts by shape and material and select appropriate imaging techniques based on these classifications. Success is measured by the efficiency and effectiveness of the documentation process.
• GPD_5: Adhere to established standards and protocols for photographic documentation. These standards provide guidelines for documenting artifacts based on material and type.
• Success Criteria: Consistently follow established standards and protocols. Success is evidenced by the production of high-quality, standardized documentation that meets professional guidelines.
• GPD_6: Strive for accuracy and precision in photographic documentation. Use high-quality equipment and techniques to capture detailed and accurate images of artifacts and sites.
• Success Criteria: Produce detailed and precise photographic documentation. Success can be measured by the clarity, resolution, and accuracy of the images, as well as positive feedback from peers or experts.
• GPD_7: Collaboration with professionals on the subject should be documented to enhance cultural documentation and leverage expertise from different fields.
• Success Criteria: Actively engage in interdisciplinary collaboration and use this consultancy in the documentation process.
• GPD_8: Stay updated on advancements in imaging technology and techniques. Continuously learn and improve photographic skills to enhance the quality and effectiveness of documentation efforts.
• Success Criteria: Regularly engage in professional development and stay informed about technological advancements. Success is indicated by the incorporation of new techniques and technologies into practice, resulting in improved documentation quality.

4.2.2. 3D Documentation

The digitization of objects and workspaces refers to conventional methods for the photographic and 3D documentation of tangible heritage. The 3D digitization of endurant entities regards structures of a wide variance of spatial scales, in indoor and outdoor environments. The choice of 3D scanning modality depends on the size, material, and environment type. For environments, this digitization may employ multiple scanning modalities, each operational on a specific scale. For example, rooms and outdoor areas require the combination of laser scanning and aerial photogrammetry to systematically cover. For smaller artifacts, photogrammetric reconstruction and active illumination sensors are nowadays simple and widely accessible. Comprehensive reviews of 3D digitization technologies can be found in [78,79,80,81,82].

The most adopted and robust principles for the digitization of tangible CH are time-of-flight or laser scanning, e.g., [83], structured light, e.g., [84], and photogrammetry, e.g., [85]. A range of products employ these principles in variations including digitization over time [86]. In photogrammetry, terrestrial and aerial photogrammetry often differ, with the latter using the global positioning system (GPS) coordinates of the drone sensor to assist reconstruction. Combinations of these principles are found in off-the-shelf devices, such as the family of handheld scanners that combine trinocular photogrammetry with active illumination.

The 3D ICONS Guidelines [87] offer a comprehensive review of 3D scanning and processing guidelines. Important resources for 3D digitization are available from Cultural Heritage Imaging (https://culturalheritageimaging.org/, accessed on 12 July 2024), including tools, technology, and training, for several digitization methods used in the conservation and preservation of tangible CH. Not all types of materials can be digitized using these methods. Artifacts made from challenging materials exist and require specific treatment. Challenging materials are those that are specular, shiny, transparent, and translucent. For their digitization, novel techniques are required (e.g., [88,89]).

When documenting in 3D, the following guidelines can be of use. Most of these guidelines produce results whose validation is qualitative. In the following list, we provide qualitative documentation of the success criteria. Some general principles can be applied. The need for post-processing is reduced as the quality of the reconstruction is enhanced. Empty spaces, holes, and distortions are evidence of poor reconstructions.

• G3D_1: Recognize conventional methods for photographic and 3D documentation of tangible heritage. Understand the importance of accurately capturing objects and workspaces in both 2D and 3D formats for preservation and research purposes.
• Success Criteria: Success is evidenced by the ability to accurately apply these methods to capture high-quality 2D and 3D representations of heritage objects and workspaces, with clear justification for their use in preservation and research contexts.
• G3D_2: Choose 3D scanning modalities based on the size, material, and environment of the objects and workspaces being digitized. Consider employing multiple scanning modalities for environments with varying spatial scales, such as combining laser scanning and aerial photogrammetry for indoor and outdoor areas.
• Success Criteria: Evidence includes the effective use of multiple scanning techniques to achieve comprehensive digitization of diverse environments, with a clear rationale for each chosen modality.
• G3D_3: Embrace robust principles for 3D digitization, including time-of-flight or laser scanning, structured light scanning, and photogrammetry. Understand the strengths and limitations of each principle and select the most suitable technique for the specific requirements of the digitization project.
• Success Criteria: Success is shown by the appropriate use of techniques based on project needs, with an ability to articulate the strengths and limitations of each method used.
• G3D_4: Refer to comprehensive reviews of 3D digitization technologies to gain insights into the latest advancements and best practices. Explore resources such as the 3D ICONS Guidelines and Cultural Heritage Imaging for guidelines, tools, technology, and training related to 3D digitization in cultural heritage conservation and preservation.
• Success Criteria: Success is measured by staying current with advancements and best practices and applying them effectively to enhance project outcomes.
• G3D_5: Acknowledge that not all types of materials can be digitized using conventional methods, especially those exhibiting specular, shiny, transparent, or translucent properties. Develop or utilize novel techniques specifically tailored for digitizing challenging materials, such as glass or shiny metals.
• Success Criteria: Evidence includes clear documentation of the methods used and the ability to produce accurate 3D models of materials that are difficult to capture with conventional methods.
• G3D_6: Determine the number of photographs required to reconstruct a target based on its size, complexity, and desired level of detail. Aim to capture at least 50 images for medium-sized targets, ensuring ample overlap between consecutive images to facilitate photogrammetric reconstruction.
• Success Criteria: Success is indicated by the ability to produce detailed and accurate 3D models with sufficient image overlap, verified by successful photogrammetric reconstruction.
• G3D_7: When acquiring photographs, ensure comprehensive coverage of the target from multiple angles. Capture images showing the entire object and background to facilitate camera pose estimation for photogrammetry. Overlap images by at least 60% to ensure accurate alignment and reconstruction.
• Success Criteria: Success is shown by capturing images from multiple angles with the required overlap, resulting in accurate and complete 3D models.
• G3D_8: Adjust imaging strategies for indoor and outdoor environments. Take more images indoors to compensate for the lack of features and distance variability. Employ different methods, such as translating the camera or moving around the subject, based on the type of target and desired reconstruction outcome.
• Success Criteria: Evidence includes successfully capturing detailed 3D models in both indoor and outdoor settings by adapting the number and type of images taken to the specific conditions and challenges of each environment.
• G3D_9: When using aerial imaging, plan flight paths carefully to ensure comprehensive coverage of the target area. Use grid-wise flight paths with high overlap (at least 80%) between images to facilitate robust reconstruction of the Earth’s surface or map-like representations.
• Success Criteria: Success is demonstrated by producing accurate and detailed 3D reconstructions or map-like representations from aerial imagery, achieved through well-planned flight paths and high overlap.

4.2.3. Manual Modeling

Manual modeling refers to the creation of 3D models in a 3D modeling software, such as Blender. This is useful for tools and artifacts when it is difficult to record them using conventional methods. The 3D modeling of objects can be sufficient for many purposes pertinent to uses where the photorealistic reconstruction of an object is not of primary importance, such as in the modeling of tools to simulate and understand their role in the physical phenomena governing the transformations that materials undergo in crafting processes.

When performing manual 3D modeling, the following guidelines are of use:

• GMD_1: Recognize manual modeling as the process of creating 3D models of tools and artifacts using 3D modeling software, particularly when conventional methods are inadequate. Understand its relevance in situations where photorealistic reconstruction is not essential, such as simulating and understanding the role of tools in crafting processes.
• Success Criteria: Clearly articulate the rationale for choosing manual modeling over conventional methods. Success is demonstrated by documented scenarios or case studies where manual modeling has been applied effectively, highlighting its advantages in non-photorealistic contexts.
• GMD_2: Determine the specific purpose of the 3D model and its intended use. Ensure that the level of detail and accuracy aligns with the requirements of the application, whether for simulation, analysis, or visualization.
• Success Criteria: Define the intended use and requirements of the 3D model. Success is indicated by documented specifications that outline the level of detail and accuracy needed for the model’s purpose, ensuring alignment with application needs.
• GMD_3: Choose appropriate 3D modeling software for manual modeling tasks. Popular options include Blender, which offers a wide range of modeling tools and capabilities suited for creating complex shapes.
• Success Criteria: Select and justify the choice of 3D modeling software. Success is demonstrated by a rationale that matches the software’s capabilities with the project’s requirements, including documentation of software features used.
• GMD_4: Conduct accurate spatial measurements of the objects being modeled to ensure that their dimensions and shape are captured correctly. Use reference materials, dimensions, and any available documentation to guide the modeling process.
• Success Criteria: Accurately capture and document spatial measurements of the objects. Success is shown by detailed reference materials, including dimensions and documentation that guide the modeling process, ensuring dimensional accuracy.
• GMD_5: Utilize a variety of tools within the modeling software to create basic shapes such as cubes, spheres, and cylinders. Familiarize yourself with tools for extrusion, scaling, and manipulation to sculpt these primitive shapes into more complex forms.
• Success Criteria: Demonstrate proficiency with modeling tools. Success is indicated by the creation of initial shapes and subsequent manipulation into complex forms, documented through screenshots or video recordings of the process.
• GMD_6: Combine basic shapes to construct more intricate and detailed models. Employ techniques such as Boolean operations, vertex manipulation, and edge loops to merge and refine individual components into cohesive structures.
• Success Criteria: Successfully combine and refine basic shapes into detailed models. Success is shown by a completed model that accurately represents the object, with documented use of advanced techniques and visual proof of intermediate stages.
• GMD_7: Pay close attention to detail during the modeling process to accurately capture the features and characteristics of the object. Take into account surface textures, edges, and contours to achieve a faithful representation.
• Success Criteria: Ensure high fidelity to the original object’s features. Success is indicated by a detailed model that faithfully represents the surface textures, edges, and contours, verified through visual inspection and comparison with reference materials.
• GMD_8: Iterate and refine the model as needed to achieve the desired level of accuracy and fidelity. Make adjustments based on feedback, reference materials, or additional information obtained during the modeling process.
• Success Criteria: Iteratively improve the model based on feedback and reference checks. Success is demonstrated by documented iterations and refinements, showing the model’s progression towards greater accuracy and fidelity.
• GMD_9: Perform quality checks throughout the modeling process to ensure that the model meets the required standards. Verify dimensions, proportions, and overall fidelity to the original object through visual inspection and comparison.
• Success Criteria: Conduct regular quality checks to ensure standards are met. Success is indicated by documented verification steps, including visual inspections and dimensional comparisons, ensuring the model meets specified standards.
• GMD_10: Document the modeling process, including steps taken, techniques used, and any challenges encountered. Maintain version control to track changes and revisions made to the model over time.
• Success Criteria: Thoroughly document the modeling process and maintain version control. Success is shown by comprehensive records of the steps, techniques, challenges, and changes made, supported by version control logs.

4.2.4. Semantic Properties

Semantic properties with regard to the naming of materials are important. These properties primarily describe the material name and, secondarily, other information such as material provenance, quality, as well as the method of production. The naming of materials has multiple dimensions. The Getty Arts and Architecture Thesaurus [90] is adopted as a generic naming reference, and also offers translation in multiple languages. However, generic thesauri are not sufficient to entail more detailed naming of materials within the context of specific crafts, that represent different material qualities and compositions.

In many cases, legislation exists that regulates the naming of materials according to their composition. For example, in the European Union, the material composition of threads determines the naming of the fabric type produced from them [91,92].

The method of production of the material reveals qualitative characteristics that may determine its price, but also its compliance with ecological (green) and sustainability requirements. Like the naming of materials, such attributes may be subject to legislation and certificates [93].

Other semantic properties refer to the traits of materials that are relevant to the human senses, such as color or tactile and acoustic properties. Naturally, these are determined by the physical properties of the material, but they refer to felt properties. Some indicative examples are the color of amber, the feel of satin, or the timbre of the oboe. Felt properties are referred to as “qualia” [94].

When collecting semantic properties of material, the following guidelines can be considered:

• GSE_1: Recognize the importance of semantic properties in describing materials, including their name, provenance, quality, and production method. Understand that these properties provide valuable information about materials that can influence their use, value, and regulatory compliance.
• Success Criteria: Consistently describe materials with accurate names, provenance, quality, and production methods. Documentation should reflect how these properties influence material use, value, and regulatory compliance, and stakeholders should demonstrate an understanding of the impact of semantic properties on material selection and application.
• GSE_2: Use standardized naming conventions and references such as the Getty Arts and Architecture Thesaurus for generic naming of materials. Understand that while generic thesauri provide broad coverage, they may not encompass the specific naming requirements of certain crafts or materials.
• Success Criteria: Consistent use of standardized naming conventions such as the Getty Arts and Architecture Thesaurus. Regular audits should verify the use of standardized names, and any gaps in coverage by generic thesauri for specific crafts or materials should be identified and documented.
• GSE_3: Recognize the need for context-specific naming of materials within the context of specific crafts, which may have unique material qualities and compositions. Develop or adopt specialized naming standards tailored to the needs of specific industries or crafts.
• Success Criteria: Developing or adopting specialized naming standards for specific industries or crafts. Documentation should include examples of unique material qualities and compositions, and stakeholders should be able to accurately name and describe materials within their specific contexts.
• GSE_4: Be aware of legislation that regulates the naming of materials based on their composition, particularly in regions like the European Union where regulations may dictate the naming of materials such as fabrics based on their thread composition.
• Success Criteria: Materials are named in compliance with relevant legislation, particularly in the EU. Regulatory requirements for material naming should be documented and accessible, and instances of non-compliance should be minimal and promptly addressed.
• GSE_5: Consider the method of production of materials, as it can reveal qualitative characteristics, pricing factors, and compliance with ecological and sustainability requirements. Ensure that naming conventions reflect these attributes and comply with relevant legislation and certification standards.
• Success Criteria: Naming conventions that reflect production methods, qualitative characteristics, pricing factors, and compliance with ecological standards. Documentation should include production methods and related attributes, and materials should comply with relevant legislation and certification standards.
• GSE_6: Take into account sensory properties of materials such as color, texture, and acoustic properties, which are relevant to human perception and experience. Understand that these properties contribute to the overall perceived quality and suitability of materials for specific applications.
• Success Criteria: Documenting and considering sensory properties such as color, texture, and acoustic properties in material descriptions. User feedback should indicate satisfaction with the perceived quality and suitability of materials, and material selection processes should incorporate sensory property assessments.
• GSE_7: Assess qualitative properties of materials, including tactile qualities and acoustic properties, which contribute to their perceived value and utility. Consider how these properties are experienced by users and how they influence material selection and usage.
• Success Criteria: Documentation and assessment of qualitative properties like tactile qualities and acoustic properties. Material selection should reflect consideration of user experiences and preferences, and materials should be chosen based on their perceived value and utility.
• GSE_8: Understand that felt properties, known as “qualia,” refer to subjective sensory experiences such as color perception, texture, and sound. Recognize the importance of qualia in describing and evaluating materials, particularly in subjective or aesthetic contexts.
• Success Criteria: Including subjective sensory experiences, or qualia, in material descriptions. Stakeholders should recognize and document the importance of qualia in material evaluation, and qualitative assessments of materials should include these subjective sensory experiences.
• GSE_9: Document naming conventions, semantic properties, and qualitative assessments of materials to ensure consistency and transparency. Standardize terminology and definitions to facilitate communication and understanding across different stakeholders and industries.
• Success Criteria: Thoroughly documenting naming conventions, semantic properties, and qualitative assessments. Terminology and definitions should be standardized and consistently used, facilitating clear communication across stakeholders.
• GSE_10: Continuously review and update naming conventions and semantic properties based on evolving industry standards, technological advancements, and regulatory changes. Stay informed about developments in material science, legislation, and best practices to ensure relevance and accuracy.
• Success Criteria: Conducting regular reviews and updates of naming conventions and semantic properties. Documentation should reflect current industry standards, technological advancements, and regulatory changes, and stakeholders should stay informed about developments in material science, legislation, and best practices.

4.3. Verbal Content

To enhance craft understanding, it is important to collect verbal descriptions of technical acts together with video recordings and non-directive interviews. This process involves confronting the practitioners with the video recordings of their gestural activity in progress. Using the video elicitation interview allows us to collect verbal data on non-verbal actions and their personal and emotional dimensions. Based on a triad structure of action, recording, and comment, practitioners are invited to describe their actions, their gestures, and their intentionality. The self-confrontation interview method [95,96] re-immerses the practitioners in their activity by confronting them with the recorded gesture [97], triggering comments on intentions, goals, and decision-making processes. This approach considers the uniqueness of each gesture and the degree of gesture variability. Video elicitation interviews prompt discussion of subject–object interaction in detail, because participants re-experience or re-live the activity while watching themselves working. Video elicitation interviews are about verbalization and wording of activities and associate a verbal description with the recall of thoughts, beliefs, and emotions experienced during the activity.

In this process, the following guidelines can be of use:

• GVB_1: Clearly define the purpose of the protocol, which focuses on collecting verbal descriptions of professional gestures. Understand that the aim is to capture technical acts through video recordings and non-directive interviews to gain insights into the practitioners’ actions, gestures, and intentions.
• Success Criteria: Clearly define the purpose of the protocol, which focuses on collecting verbal descriptions of professional gestures. The aim should be to capture technical acts through video recordings and non-directive interviews to gain insights into the practitioners’ actions, gestures, and intentions.
• GVB_2: Select participants who are experienced practitioners in the field relevant to the protocol. Ensure that they are willing to engage in video elicitation interviews and are comfortable discussing their actions and intentions.
• Success Criteria: Selecting participants who are experienced practitioners in the relevant field is crucial. Ensure that these participants are willing to engage in video elicitation interviews and are comfortable discussing their actions and intentions.
• GVB_3: Adopt a non-directive approach during the interviews, allowing practitioners to freely describe their actions, gestures, and intentions without external influence. Encourage open-ended responses to elicit rich verbal data.
• Success Criteria: Adopting a non-directive approach during the interviews is essential. Allow practitioners to freely describe their actions, gestures, and intentions without external influence, and encourage open-ended responses to elicit rich verbal data.
• GVB_4: Utilize the video elicitation method, which involves confronting practitioners with video recordings of their gestural activity in progress. Understand that this approach allows for the collection of verbal data on non-verbal actions and their personal and emotional dimensions.
• Success Criteria: Utilizing the video elicitation method is key. This method involves confronting practitioners with video recordings of their gestural activity in progress, which allows for the collection of verbal data on non-verbal actions and their personal and emotional dimensions.
• GVB_5: Organize the interview structure around the triad of action, recording, and comment. Invite practitioners to describe their actions, gestures, and intentionality while watching the recorded gestures.
• Success Criteria: Organizing the interview structure around the triad of action, recording, and comment is important. Invite practitioners to describe their actions, gestures, and intentionality while watching the recorded gestures.
• GVB_6: Employ the self-confrontation interview method, which re-immerses practitioners in their activity by confronting them with recorded gestures. Encourage practitioners to comment on their intentions, goals, and decision-making processes during the activity.
• Success Criteria: Employing the self-confrontation interview method is necessary. This method re-immerses practitioners in their activity by confronting them with recorded gestures and encourages them to comment on their intentions, goals, and decision-making processes during the activity.
• GVB_7: Acknowledge the uniqueness of each gesture and the degree of gesture variability among practitioners. Allow for variability in responses and interpretations during the interviews to capture the diversity of perspectives.
• Success Criteria: Acknowledging the uniqueness of each gesture and the degree of gesture variability among practitioners is crucial. Allow for variability in responses and interpretations during the interviews to capture the diversity of perspectives.
• GVB_8: Prompt discussion of subject–object interaction in detail during the interviews. Understand that participants re-experience or re-live the activity while watching themselves work, leading to rich verbalizations of their experiences.
• Success Criteria: Prompt detailed discussion of subject–object interaction during the interviews is important. Understand that participants re-experience or re-live the activity while watching themselves work, which leads to rich verbalizations of their experiences.
• GVB_9: Focus on verbalizing and wording activities during the interviews. Encourage practitioners to associate verbal descriptions with the recall of thoughts, beliefs, and emotions experienced during the activity for deeper insights.
• Success Criteria: A focus on verbalizing and wording activities during the interviews is essential. Encourage practitioners to associate verbal descriptions with the recall of thoughts, beliefs, and emotions experienced during the activity for deeper insights.
• GVB_10: Ensure ethical conduct throughout the interview process, respecting participants’ privacy, confidentiality, and autonomy. Obtain informed consent from participants and adhere to ethical guidelines for research involving human subjects.
• Success Criteria: Respect participants’ privacy, confidentiality, and autonomy, obtain informed consent, and adhere to ethical guidelines for research involving human subjects.

4.4. Event Logging

Event logging is used to systematically record and document events, activities, interactions, and observations. It involves keeping a detailed chronological record of events as they unfold during fieldwork. Event logs are verbal annotations that describe the contents of an ethnographic video, temporally corresponding to segments of said video [61]. Event logs promote immediate reflection and a summary of actions that assist in later assessment and comprehension. The content of these logs summarizes the activities taking place in the corresponding video segment. An added-value benefit is that when terminology is followed, event logs can be used as searchable digital records [98], using a keyword search.

These short-term observations provide a narrative of the proceedings but not a complete record. The purpose is to counter the decay of human memory, because details are already forgotten when the video is later reviewed. The utility of event logs is to obtain an immediate review of the session to facilitate the subsequent review of the material. We recommend the authoring of event logs either during observation or as soon as possible after it.

Alongside the recorded events, researchers often include their reflections, impressions, and insights related to the events. This adds a layer of interpretation and analysis to the raw data. The detailed records generated through event logging become valuable data for analysis. Researchers can identify patterns, trends, and themes by reviewing the logged events retrospectively.

When considering event logging in ethnographic fieldwork, the following guidelines can be considered:

• GEV_1: Recommend authoring event logs either during observation or as soon as possible after it. Recognize the importance of reducing the time interval between observations and authoring to counteract the decay of human memory.
• Success Criteria: Consistently author event logs during or immediately after observations. The documentation timelines should be monitored and minimized to ensure prompt log creation, with logs showing minimal evidence of memory decay or loss of detail.
• GEV_2: Embrace advances in speech-to-text transcription technology to enable rapid notetaking during fieldwork. Understand that speech-to-text technology allows ethnographers to commentate while recording, reducing the effect of human memory decay and enabling hands-free operation.
• Success Criteria: Effective use of speech-to-text technology, allowing for rapid and accurate notetaking during fieldwork. Ethnographers should be able to commentate while recording, significantly reducing the effects of human memory decay and enabling hands-free operation.
• GEV_3: Acknowledge the importance of reviewing transcribed event logs for accuracy, as speech-to-text systems may not be infallible. Ensure that the review process is integrated seamlessly with word-processing applications to facilitate corrections and revisions.
• Success Criteria: Thorough review of transcribed event logs for accuracy, acknowledging that speech-to-text systems can have errors. The review process should be seamlessly integrated with word-processing applications, facilitating easy corrections and revisions.
• GEV_4: Document observations of practitioners at work, including techniques, materials used, interactions with others, and the ambience of the work environment. Recognize that event logging focuses on the craft-making process itself, capturing each step from material selection to artifact creation.
• Success Criteria: Event logging should comprehensively capture the craft-making process from material selection to artifact creation.
• GEV_5: Encourage researchers to include their reflections, impressions, and insights related to the events in the event logs. Understand that this adds a layer of interpretation and analysis to the raw data, enhancing their value for subsequent analysis.
• Success Criteria: Researchers consistently add their reflections, impressions, and insights to event logs. This practice adds valuable layers of interpretation and analysis to the raw data, enhancing their value for subsequent analysis.
• GEV_6: Recognize event logs as a form of verification and validation when shared with participating practitioners.
• Success Criteria: Sharing event logs with participating practitioners. Involving practitioners in the review process ensures the accuracy and shared understanding of the recorded events.
• GEV_7: Design a structured template for event logging that includes fields for date, time, description of the event, participants involved, location, and any additional relevant details. Emphasize the importance of maintaining consistency in recording events to capture important details accurately.
• Success Criteria: Design and use of a structured template for event logging. This template should include fields for date, time, event description, participants, location, and additional relevant details. Maintaining consistency in recording events is crucial for capturing important details accurately.
• GEV_8: Encourage the involvement of additional practitioners and community members in the review process to capture multiple perspectives and insights. Understand that diverse viewpoints enrich the analysis and interpretation of the recorded events.
• Success Criteria: Capture multiple perspectives and insights to enrich the analysis and interpretation of the recorded events.

4.5. Legal Implications of Applying the Methodology in Europe

In this section, we discuss ethical aspects when applying the proposed methodology in Europe, concerning the General Data Protection Regulation (GDPR-EU 679/2016) [99]. They are related to the data collected, mainly regarding the participation of humans in research, the approval of research activities by relevant ethics committees, and the provisions regarding the collection of personal data. Several works suggest processes on achieving and maintaining GDPR compliance [100,101,102]. This work does not focus on providing a framework for GDPR compliance. To assist such efforts when considering informed consent procedures under the GDPR regulation, the following guidelines can be considered:

• GETH_01: Adhere to informed consent procedures. Follow an informed consent procedure when conducting ethnographic fieldwork, ensuring ethics approval for all research activities involving human participation.
• Success Criteria: Submit a comprehensive application for ethics approval, detailing all research activities involving human participation. Provide specific information, including informed consent forms, information sheet templates, a description of human participation, and detailed descriptions of all research activities. Ensure consent forms include information on personal data use, protection, and participant rights under the GDPR (EU 679/2016), as well as rights regarding post-processing, annotation, and online publication of data.
• GETH_02: Respect participant autonomy and withdrawal rights. Ensure that participants can freely refuse to participate or withdraw their consent at any time without facing adverse consequences or needing to justify their decision.
• Success Criteria: Clearly state participants’ right to withdraw consent at any time and assure them that withdrawal will not affect the lawfulness of data processing based on consent before withdrawal. Provide participants with the right to request data erasure after withdrawing consent.
• GETH_03: Clarify limitations on data erasure. Inform participants that the deletion of personal data related to video recordings is not feasible under GDPR Article 17 par. 3(d).
• Success Criteria: Ensure that participants are fully informed about the limitations on data erasure in advance of their participation in video recordings. Confirm that all participants understand and consent to the conditions regarding video recording and data usage before their involvement in the study.
• GETH_04: Perform risk assessment regarding the use of wearable equipment and intrusive sensors, controllers, and measuring devices. Make sure that the health and safety manual for each device used is carefully reviewed and that all the risks involved are appropriately documented in a single health and safety sheet that gathers input from all devices. Append this sheet in the informed consent procedure.
• Success Criteria: Clearly inform participants about potential health and safety issues related to the use of equipment and to the equipment materials (for example, there is Lycra, Spandex, and memory foam used in wearable devices that can cause allergies or skin-related symptoms). Include health and safety warnings in participant information and consent forms.
• GETH_05: To ensure compliance with the General Data Protection Regulation (GDPR), it is essential for the parties involved in owning and processing data to establish a comprehensive data processor agreement. This agreement should clearly delineate the roles and responsibilities of each party, including the specific data protection obligations that the processor must adhere to. Key elements of the agreement should include the scope and purpose of data processing, the types of personal data involved, the duration of processing, and the measures taken to ensure data security and confidentiality. By formalizing these aspects in a written contract, both parties can ensure that they meet GDPR requirements and protect the rights and privacy of data subjects.
• Success Criteria: Use a separate data processor agreement or amend the consent form with the data processor agreement and ensure that data owners are informed about the data processing measures and data protection obligations before signing the agreement.
• GETH_06: When possible, consider using semi-automated tools to assist in the process of data anonymization. In particular, for images and videos, several off-the-self solutions support person tracking and personal identity anonymization.
• Success Criteria: Simplification of data anonymization process, reduction of post-processing time, and fewer resources allocated for this task.

5. Data Pre-Processing

This section deals with the pre-processing of this extensive multimedia corpus to serve as the data foundation for the online semantic representation system.

5.1. Data Segmentation

5.1.1. Audiovisual Craft Documentation

The segmentation of audiovisual data involves the following activities. Keyframe extraction can be used to document “key craft poses” that function as “keyframes”. At the same time, the extraction of sequences of such keyframes and then their juxtaposition in temporal order can be used to trigger the visual perception to “interpolate” the motion between them [103], in this way conveying the recorded motion or event.

Video segmentation can be used to identify and isolate individual actions that, when arranged in a temporal order, represent a crafting process, while individual segments represent crafting actions. In the same context, segmentation of egocentric video documentation can identify individual actions that, when arranged in a temporal order, represent a crafting process from the visual perspective of the practitioners. Video segmentation can be combined with audio segmentation to identify the auditory information correlated with crafting actions.

When multiple recording mediums are employed, concurrently we can take advantage of synchronized multi-view audiovisual segmentation that can document actions from multiple perspectives and multiple audiovisual channels.

5.1.2. MoCap Segmentation

An integral part of the MoCap analysis process is the development of a gestural vocabulary that serves as a tool for interpreting and analyzing the captured motion data. Source digital assets are 4D motion recordings, which we call “animations”. In some cases, the acquired MoCap recordings require pre-processing to remove noisy or invalid data. After this pre-processing phase, motion segmentation takes place. Segments created contain synchronous recordings from MoCap sessions on the same processes. In the case where synchronized video recording exists, AnimIO [20] can be employed to segment multiple synchronous recordings from video streams and MoCap. This encoding refers to the annotation of motion data per activity executed. AnimIO is also designed to facilitate the temporal delimitation of motion segments by individual human limbs, while respecting constraints stemming from their hierarchical configuration. Using this feature, individual segments can be isolated from the skeletal data of the original BVH input.

5.2. Visual Annotations

Annotation of endurant digital assets involves the annotation of images and 3D models. By annotating the visual material, we identify objects and symbols in images and video, which can later inform our analysis and understanding of them. Visual annotations enhance the understanding, analysis, and communication of visual materials, such as photographs, sketches, and artifacts.

A wide range of image annotation editors exist, e.g., [104,105,106,107,108]. All of them offer annotations upon the overall file at point locations in images. Standard features include the annotation of arbitrary (freehand) image regions and rectangular bounding boxes within the image.

The case is similar for 3D model annotators, e.g., [109,110,111,112,113,114]. Point location annotations on the surface of the 3D model are offered by all editors. In objects, the user may wish to indicate which piece of a carafe is the handle or indicate a motif or symbol upon a piece of pottery. The annotation of free-hand regions on the 3D model is a feature that is not found in all editors, due to the somewhat tedious user task of marking the region of interest. We recommend using an annotation editor that enables the specification of geodesic regions of interest upon 3D models, a feature not available in most editors that offer only point-based annotation, to facilitate region annotations.

5.3. Event Parsing

Crafting actions are part of the production process and may be specific to a technique or the use of a tool. Video and MoCap recordings of the crafting process are used to demonstrate crafting actions. However, in this way, no measurement of motion can serve documentation purposes and accurate reenactment of the recorded actions.

We call event parsing the task of separating parts of a motion recording that correspond to individual craft actions into individual motion animation clips. Parsing is key in the comprehension of human activities because it enables the association of movements with entries in gesture and action vocabularies. By articulating motion into actions, a more insightful understanding of crafting activities is achieved because individual gestures can be isolated, studied, and practiced. Moreover, using these clips as references, crafting processes can be understood as action sequences, rather than continuous streams of human movement.

In this context, training datasets have been developed [115,116] that offer recordings of individual gestures, where an actor performs the gestures in the environment of a motion capture laboratory. The segmentation of the examples is provided “by construction”, as training examples contain a gesture in each one.

Ethnographic recordings are pre-processed for synchronization, and the AnimIO software v1.0 [20] is used to create motion segments. AnimIO is thus used to annotate motion data per the action performed. AnimIO is used to temporally delimit motion recordings into segments.

In this way, animation libraries are created, which are collections of animation files. Animation files are motion segments, containing the recordings of individual actions by one or multiple recording modalities. The motion data are encoded in a human skeletal model configuration. This is achieved either directly from the MoCap recording or through the analysis of video recordings.

5.4. Data Storage

Data storage for long-term preservation is a critical issue for the sustainability of the developed multimodal craft dictionaries. In collaboration and coordination with representatives of Europeana [117] and Zenodo [118], the digital assets presented in this research work are stored in the Zenodo repository, which is funded by the European Commission. As such, only metadata are stored in the local file system of the MOP/CAP.

Storing assets on Zenodo is important because the online storage space for digital assets of large storage capacity can be costly. It is, therefore, more important for small artisanal communities that may have limited resources. Moreover, storing digital assets on an online repository (Zenodo) instead of a dedicated web server offers several additional advantages. Zenodo is hosted as a cloud service, providing high availability and accessibility from anywhere with an internet connection. Zenodo implements backup and redundancy measures, ensuring the integrity and availability of digital assets. Zenodo has invested in security measures, including encryption, access controls, and regular security updates. Zenodo provides DOIs for the submitted assets and also offers analytics of access, easing the distribution of assets and enabling the tracking of their access. Moreover, Zenodo offers compliance certifications with the Creative Commons licenses [119] and adheres to industry-specific regulations, ensuring that data storage practices align with legal requirements and industry standards.

6. Production of Dictionaries

6.1. Introduction to the Craeft Online Platform

The Craeft Online Platform (CAP) [120] facilitates the representation of the socio-historic context to (1) document, represent, and preserve intangible dimensions along with objects and sites; (2) contextualize the presentation of tangible heritage; (3) systematize and facilitate the presentation of socio-historical context; and (4) explore and promote world heritage, stimulating interest through educational and fascinating content. Furthermore, CAP supports the representation of human processes in the context of traditional crafts and their linking to the social and historical context of the communities that practice them.

In this framework, artifact creation is considered a process that is based on an “archetypal” plan, or a schema. This process schema is conceptual and ostensive, in that it can be demonstrated and verbally described, i.e., as instructions. Audiovisual recordings phenomenologically capture the activity, while semantic descriptions enable the interpretation and abstraction of this activity as a stepwise process. The undetermined nature of handcrafting individual craft articles, or “the workmanship of risk”, is reflected by process schemas that support branching points to represent practitioner decisions taken during practice, or “on the fly”. The proposed representation analytically associates the segments of these recordings with process steps, to support the abstraction of process schemas from the study of process recordings.

6.2. External Dictionaries

To provide the widest interoperability of the knowledge, base classes and properties for linking to prominent semantic dictionaries are facilitated. The considered dictionaries are de facto standards in CH; they are the Getty Art and Architecture Thesaurus (AAT) [121], the Catalog of Art Collections (CONA) [122], the Thesaurus of Geographic Names (TGN) [123], the Union List of Artist Names (ULAN) [124] and UNESCO Thesaurus [125]. In addition to widening the interoperability of the CAP, linking to these dictionaries increases the system’s capacity for precise and comprehensive documentation and eases data entry, by avoiding repeating the entry of information that already exists online.

6.2.1. AAT

The AAT includes generic terms and associated dates, relationships, and other information about concepts related to or required to catalog, discover, and retrieve information about art, architecture, and other visual cultural heritage, including related disciplines dealing with visual works, such as archaeology and conservation, of the type collected by art museums and repositories for visual cultural heritage.

The inclusion of the Getty AAT enhances the system’s vocabulary related to art, architecture, and material culture. The AAT dictionary is the basic dictionary used by the CAP, illustrated below. The class including the Getty terms is GettyTerm. The property for linking to the AAT is hasGettyTerm. Every knowledge entity in the system has an annotation from the AAT, thus the domain of hasGettyTerm is left unspecified for generality. An overview of integrating the AAT is presented in

Table 1. Incorporating the Getty AAT.

GettyTerm is a class

GettyTerm is a subclass of ecrm:E55_Type

hasGettyTerm is an object property

hasGettyTerm is a sub-property of ecrm:P2_has_type

The range of hasGettyTerm is class GettyTerm

.

The specification of the Getty term is ensured and automated by adding the pertinent annotation by default to each instantiated knowledge entity, at the data input form. Naturally, this default semantic annotation is generic and it is left to the user to specialize it, if needed and if known. The default annotations are provided in the table below. We recall that in the craft ontology, all digital assets are media objects, while they may differ in type (image, video, 3D). An overview of default annotations is presented in

Table 2. Automated annotations from the Getty AAT.

Media object (image)	http://semantics.gr/authorities/digital-item-types/1332557240, (accessed on 12 July 2024)—Born-digital photograph
Media object (video)	http://semantics.gr/authorities/digital-item-types/346883033, (accessed on 12 July 2024)—Born-digital video
Media object (3D model)	http://vocab.getty.edu/aat/300411661, (accessed on 12 July 2024)—Virtual models
Person	http://vocab.getty.edu/aat/300024979, (accessed on 12 July 2024)—People (agents)
Social groups	http://vocab.getty.edu/aat/300025948, (accessed on 12 July 2024)—Organizations (groups)
Location	http://vocab.getty.edu/aat/300248479, (accessed on 12 July 2024)—Location (physical position)
Tool	http://vocab.getty.edu/aat/300122241, (accessed on 12 July 2024)—Tools
Product	http://vocab.getty.edu/aat/300387427, (accessed on 12 July 2024)—Products
Event	http://vocab.getty.edu/aat/300069084, (accessed on 12 July 2024)—Events (activities) (Events (hierarchy name))
Process	http://vocab.getty.edu/aat/300138076, (accessed on 12 July 2024)—Processes

.

When the user documents any of these entities, the appropriate annotation is by default entered. The user may specialize these annotations according to the available knowledge about the knowledge entity.

It ought to be noted that although the AAT dictionary is more often used to document tangible heritage, we found it very useful for the documentation of intangible heritage as well. Verbs are equally well represented in the AAT, and their hierarchical structure is quite useful in the semantic characterization of actions and processes.

6.2.2. CONA

The incorporation of CONA extends the system’s capability to include detailed information on art collections, helping users connect and explore diverse art collections worldwide.

CONA compiles titles, attributions, depicted subjects, and other metadata about works of art, architecture, and cultural heritage, both extant and historical, physical and conceptual. Metadata are gathered and linked from museum collections, special collections, archives, libraries, scholarly research, and other sources.

CONA is focused on “unique” artworks or, better, artworks with individual names that have each been cataloged by museums, libraries, or regional authorities (i.e., for monuments, buildings, etc). For this reason, the domain of the property is class ecrm:E22_Human-Made_Object. An overview of integrating the CONA is presented in

Table 3. Incorporating the CONA vocabulary.

CONATerm is a class

CONATerm is a subclass of ecrm:E55_Type

hasCONATerm is an object property

hasCONATerm is a sub-property of ecrm:P2_has_type

The domain of hasCONATerm is class ecrm:E22_Human-Made_Object

The range of hasCONATerm is class CONATerm

.

CONA would seem rather irrelevant for craft artifacts. The reason is that although each craft item is unique, it rarely has a specific name, as opposed to a painting or statue. However, we found it very useful and we are using it for the following two cases: first, when a crafted artifact references another known artifact, and second, when a crafted artifact references a known event. We provide an example below, coming from a collection of handcrafted garments whose design was inspired by antiquities.

The CONA dictionary provides information on the particular archaeological artifact, thus relieving the user from documenting that as well.

6.2.3. TGN and Geonames

Integration of the TGN and Geonames dictionaries provides geospatial context, enriching the documentation system with standardized geographic names for locations relevant to cultural heritage. Thus, the domain of both properties hasTGNTerm and hasGeonamesTerm is class ecrm:E53_Place. An overview of integrating the TGN and Geonames is presented in

Table 4. Incorporating TGN and Geonames.

TGNTerm is a class

TGNTerm is a subclass of ecrm:E55_Type

hasTGNTerm is an object property

hasTGNTerm is a sub-property of ecrm:P2_has_type

The domain of hasTGNTerm is class ecrm:E53_Place

The range of hasTGNTerm is class TGNTerm

GeonamesTerm is a class

GeonamesTerm is a subclass of ecrm:E55_Type

hasGeonamesTerm is an object property

hasGeonamesTerm is a sub-property of ecrm:P2_has_type

The domain of hasGeonamesTerm is class ecrm:E53_Place

The range of hasGeonamesTerm is class GeonamesTerm

.

The Geonames dictionary is more comprehensive and detailed than TGN. We have been using it since early versions of the Mingei Online Platform, in the Mingei project. In this version, we have automated the extraction of location coordinates (GPS) from the Geonames service, relieving the user from the task of entering these data, while at the same time protecting system integrity from human errors.

Although we retrieved location coordinates from the Geonames dictionary, we added the TGN dictionary as well. The reason is that TGN offers descriptions of the locations that provide historical information.

6.2.4. ULAN

Linking to ULAN facilitates the identification and documentation of artists associated with cultural artifacts, ensuring a comprehensive understanding of the individuals behind the creations.

The integration with ULAN serves two purposes. The first is the comprehensive documentation of the represented knowledge pertinent to artisan names, used either in the documentation of artifacts or in the representation of narratives. The second is the automation of the completion of biographical information about the referenced persons. Specifically, we automated the retrieval of the following attributes from the ULAB service: (a) birth/death dates, (b) alternative names, (c) nationality, and (d) role(s). An overview of ULAN integration is presented in

Table 5. Incorporating ULAN.

ULANTerm is a class

ULANTerm is a subclass of ecrm:E55_Type

hasULANTerm is an object property

hasULANTerm is a sub-property of ecrm:P2_has_type

The domain of hasULANTerm is class ecrm:E21_Person

The range of hasULANTerm is class ULANTerm

.

6.2.5. Additional Dictionaries

Support is also provided for two more dictionaries, namely the UNESCO thesaurus and the dictionary of the Greek National Aggregator. No automatic retrieval of information is foreseen for these dictionaries, as the dictionaries mentioned earlier cover our semantic annotation needs. However, the UNESCO dictionary is required for ingestion by all National Aggregators and the SearchCulture.gr dictionary is required by the Greek Aggregator. An overview of integrating the UNESCO thesaurus and the dictionary of the Greek National Aggregator is presented in

Table 6. Incorporating UNESCO thesaurus and the dictionary of the Greek National Aggregator.

UnescoThesaurusTerm is a class

UnescoThesaurusTerm is a subclass of ecrm:E55_Type

hasUnescoThesaurusTerm is an object property

hasUnescoThesaurusTerm is a sub-property of ecrm:P2_has_type

The range of hasUnescoThesaurusTerm is class UnescoThesaurusTerm

UnescoThesaurusTerm is a class

GNATerm is a subclass of ecrm:E55_Type

hasGNATerm is an object property

hasGNATerm is a sub-property of ecrm:P2_has_type

The range of hasGNATerm is class GNATerm

.

6.3. Media Objects

Media objects provide a first-level representation of the multimodal collection of ethnographic documentation. Through the documentation of media objects, the basic metadata are provided linking to their unique URI, as retrieved following the procedures presented in Section 4.4 on data storage. In CAP, media objects can be of the following types:

• Images include the photographic material collected throughout the ethnographic process.
• Videos include both original and segmented recordings. Since media objects follow a hierarchical structure, segmented videos can be represented as children of the parent source video files using the concept of media object segments.
• Audio includes audio files from craft practice, interviews, workshop background noise, etc.
• Motion captures includes the human motion captured using a full-body motion capture suit or any other type of sensor and is available in .bvh format.
• 3D reconstruction contains entities that are digitally reconstructed from original scenes, after documenting and processing them.
• 3D objects are items that were created using some form of 3D modeling software.
• Motion Vocabularies are sets of motion captures that represent the motions recorded for a single step of a process.

6.4. Basic Knowledge Entities

The formulation of basic knowledge entries is systematized through the CAP.

6.4.1. Locations

Locations represent places on Earth and, sometimes, imaginary ones. They are used in the representation of events, intended as spatiotemporal occurrences. These events may be historical, as those in contextualization narratives, or contemporary, as in the recording of crafting processes. Locations are defined as points having two coordinates, latitude and longitude. Known locations are semantically annotated using the Geonames and Thesaurus of Geographic Names databases. The representation in the CAP of a dictionary entry of type Location is presented in Figure 2. On the top left part of the figure, the metadata stored for the specific location are provided. Furthermore, in case this location is associated with historic events, these are also presented. Semantic annotations are offered by linking the location with existing dictionaries namely, Geonames and the Getty AAT. The location is also presented on a map by using the longitude and latitude values and with a collection of media objects. In the case of Figure 2, one media object of type Image is associated. This is a separate entity of type Media Object, as presented in the top right part of the figure. As a separate entity, it has its metadata and is associated with a license from the set of Creative Commons licenses to ensure that users accessing it are aware of the appropriate usage and restrictions that apply to this image.

6.4.2. Materials

Materials are documented by scientific and colloquial names. It ought to be noted that several materials are subject to naming legislation. In Europe, there are regulations on the protection of designation of origin and geographical indication for materials and products (regulation 1151/2012 of the EU). More specific regulations exist for specific material classes, such as the regulation of fiber naming depending on composition (e.g., regulation 1007/2011 of the EU). In addition, collected knowledge on materials concerns their physical material properties. The representation in the CAP of a dictionary entry of type Material is presented in Figure 3. The entry regards the material Glass which is linked with a media object that presents a glass-made object. The material is hierarchically annotated using the AAT thesaurus and associated with glass products. The textual description of the material as an amorphous inorganic substance is also provided in the definition.

6.4.3. Tools

Tools represent the crafting tools and the machines used in the RCIs. For example, in wood carving, the tools are the chisels, the hammers, and the files. Examples of machines are furnaces in glassmaking, looms in textile manufacturing, and vices in woodworking.

Notably, we regard human hands as tools as well, such as in pottery; in this case, we take handedness into account and distinguish hands as dominant and non-dominant (for the majority of the population, the dominant hand is the right one).

In general, in traditional crafts, tools and machines are hand-powered. With the advent of modern technology and electricity, several tools and machines are now electrically powered. This does not introduce significant implications in the collection of knowledge, except that some actions that used to be parts of crafting processes are now no more. For example, in ceramics, the potter’s wheel used to be powered by the practitioner’s foot. Thus, when we are representing pottery on a traditional potter’s wheel, we include the foot action that turns the wheel, whereas nowadays most potters’ wheels are electrically powered and we do not use this action.

The representation in the CAP of a dictionary entry of type Tool is presented in Figure 4. In this example, a diamond shear that is used in glassmaking is presented. A description of the dictionary entry is provided that is semantically annotated using both the AAT and semantics.gr. The tool is also linked to several media objects that are images for its documentation, a 3D model to reveal its structure, video renderings of the 3D model, and recordings regarding how it is handled (affordances) and used.

6.4.4. Persons

Typical knowledge elements related to the life of a person include long-term events such as occupation, education, and places of habitation, as well as key events such as collaborations with other persons, production of masterpieces, and so on. Known artists are, in addition to the AAT, semantically annotated using the Union List of Artist Names (ULAN). The representation in the CAP of a dictionary entry of type Person is presented in Figure 5. The documented person is a sculptor named Yannoulis Chalepas, linked to a media object which is a portrait of the artist. Furthermore, the artist is linked with several events tied to the creation of sculptures, which are in turn considered products that have their semantic representation as discussed in the following subsection.

6.4.5. Products

Products refer to unique craft artifacts. Moreover, as crafting processes have steps, the outcome of a step is a product and, at the same time, a material for the subsequent step. For known artifacts, an additional annotation is provided using the Cultural Objects Name Authority (CONA). In contrast to “fine arts”, in traditional crafts, not many craft products have been recognized as artworks. Thus, not many craft products are included in this index. On the other hand, several craft artifacts are inspired by artworks. Examples are found in textiles, where dress designs are inspired by Greek antiquities. In these cases, the dresses are annotated with the specific archaeological finding that inspired the dress. The representation in the CAP of a dictionary entry of type Product is presented in Figure 6. The entry is a glass carafe that is documented through its dictionary entry and a plethora of media elements of type Image. Semantically, it is annotated using the AAT and the semantics.gr. The creator of the carafe is a “Person” linked with the product. The creation of the carafe is an “Event” linked with both the product and the creator. The documentation of the creator and the event of the creation are provided in Figure 7.

6.4.6. Social Groups

This category represents groups of persons related by ethnicity, region, business interest (e.g., a company), etc. As in the case of persons, key events are associated with social groups, such as the passing of legislation relating to craft products, or an art festival or exhibition. In addition, social groups are associated with events that mark the initiation and termination of participation of individuals in that group. The representation in the CAP of a dictionary entry of type Social Group is presented in Figure 8. In this example, two social groups are presented; the first is the Bulgari family, which is the founder and owner of the Bulgari firm.

6.4.7. Processes

Processes are represented as artifact creation activities based on an “archetypal” plan, or a schema. This process schema is conceptual and ostensive, in that it can be demonstrated and verbally described, i.e., as instructions [126,127]. The schemas are inferred from observing or interviewing participants, as shown in Figure 9.

7. Discussion

The main contribution of this work is the provision of a new way of conducting ethnographic fieldwork that integrates multiple digital technologies in standard ethnographic practices. This new methodology has been defined, applied, and validated in the EU-funded projects Mingei and Craeft. In these projects, eleven Representative Craft Instances were studied (silk weaving, glass blowing, scientific glass blowing, mastic cultivation, pottery, porcelain making, wool textiles, Aubusson tapestry, marble carving, silversmithing, and wood carving). During ethnographic fieldwork, researchers, computer scientists, ethnographers, social anthropologists, and the craft masters worked together to apply new media technologies to the standard ethnographic methods. The outcome of this collaboration is the methodology that we propose in this work. Many of the findings during fieldwork are compiled in the form of application guidelines. The methodology and the guidelines can be applied together to support scientists in applying this work to their practices.

In addition to this new ethnographic approach, we investigated the combination of traditional ethnographic methods with semantic web technologies. The objective is to effectively preserve and transmit the acquired digital assets in the form of semantically interoperable and heritage-standards-compliant knowledge. To this end, scientists engaged in knowledge representation and the semantic web contributed towards implementing the semantic model and an authoring platform to support the representation.

The interdisciplinary approach, combining insights from social anthropology, advanced digital recording techniques, and semantic web technologies, has demonstrated significant potential in creating comprehensive and accessible records of traditional craftsmanship. While formulating the methodology, we understood that direct collaboration with craft practitioners and communities was essential. This collaboration allows the documentation and capturing of craft secrets, i.e., phenomena that craft practitioners can identify and help ethnographers witness. In-depth interviews and reflective sessions with craft practitioners have been invaluable in grounding the documentation process in practical realities and ensuring its authenticity.

The proposed approach addresses a significant gap in the field by introducing a semantic web-based classification scheme and a multilingual, hierarchical thesaurus for the semantic annotation of craft terms. The approach introduces an internal classification that builds on existing knowledge classifications and makes the represented knowledge findable, accessible, interoperable, and reusable. At the same time, our approach allows for richer, more contextualized digital dictionaries that integrate multimedia elements, such as audio, video, illustrations, and 3D models.

From a practical perspective, this approach has demonstrated significant potential in creating comprehensive and accessible records of traditional craftsmanship as part of the documentation of craft instances, and in particular, more than 1500 knowledge entities were documented.

Today, with the advent of AI, many may argue that the methods, technologies, and approaches seem outdated. This is not the case, since synthetic methods are not appropriate for crafts. In our case, working with and recording the actual craft experts is the scientifically valid way to proceed. Indeed, AI-based approaches may be found usable to cope with the laborious nature which is the main limitation of this work. In this context, AI approaches could be used for the post-processing of data, with the main objective of reducing the amount of data that should be manually processed. For example, detecting with AI where a specific action occurs in a video stream can allow for faster indexing and retrieval of information. Furthermore, reducing the size of the recording by removing parts where no action takes place (e.g., preparation, testing sessions) could also make data more usable. We argue that AI-aided approaches can be useful in the context of this methodology, with the condition that human judgment is used to perform decision-making with respect to the acquired data.

Another direction where AI could affect our approach could be in the digitization phase. Modern techniques may reduce the cost of digitization by building on advances in view synthesis methods such as Neural Radiance Fields. Today such approaches do not deliver good results in terms of 3D reconstruction but succeed in rendering a scene in video using multiple photographs. In the future, this could and will change, making several data acquisition tasks easier. We argue that the essence of this research work will not be affected, and adapting to new technologies would require replacing an acquisition method used today with a new method introduced in the future.

All the tasks involved in this methodology require significant effort even if facilitated through some form of digital technology. As such, the majority of future research work will be towards automatizing parts of the methodology but keeping a moderated overview of the results by the scientific operator of each step. AI can be of use to reduce the required laborious effort, and thus further simplify the creation of dictionaries. At the same time, reducing the effort will increase user acceptance and applicability of our approach in various contexts.

Another issue that should be noted when discussing the proposed outcome is the potential for generalizing these outcomes in other contexts. As such, a plethora of systematically documented human activities can enhance education and training. This involves all laborious tasks in workspaces that incorporate dexterous interactions with tools and materials. Following the same approach, we can generalize the outcomes and applicability to production in various working environments and task documentation.

8. Conclusions

The main contribution of this work, in conjunction with previous efforts, is that it provides a methodology that systematizes the process by integrating different disciplines. By combining ethnographic, digital media, and 3D reconstruction expertise, this method captures the same process from multiple viewpoints, with multiple technologies. Through multimodal ethnography involving self-reflection and structured interviews, it collects multiple perspectives from the practitioners. The combination of the above creates a huge corpus of data that needs to be rationalized. Core ethnographic methods describe the processes, subprocesses, and tasks. Through self-reflection interviews, our method dives even deeper into the realities of each craft action. Additionally, structured interviews bind crafts with the social and historical context of the communities that practice individual craft instances. Computer science functions as a medium for the segmentation, post-processing, annotation, and storage of this huge corpus, and through the semantic web the formal documentation and presentation of this corpus in the form of multimodal craft dictionaries.

From the practical side, the use of diverse recording techniques, such as photography, multi-view videos, egocentric videos, motion capture, and 3D digitization, enriches this documentation. The development of multimodal online craft dictionaries presents a promising solution for preserving and transmitting ethnographic data. These dynamic repositories serve as valuable resources that encapsulate the embodied wisdom of traditional crafts, making this knowledge accessible to researchers, academics, and broader audiences.

Using the defined methodology, in this work multimodal dictionaries have been created, as presented in

Table 7. Overview of dictionaries.

	Glass	Porcelain	Clay	Marble	Wood	Silver	Tapestry	Textiles
Locations	34	18	13	16	25	34	23	193
Materials	8	11	12	5	14	8	7	44
Narratives	4	1	3	2	2	3	2	18
Persons	29	8	3	15	17	7	35	154
Products	8	1	1	7	15	1	0	567
Social Groups	18	9	10	4	14	20	2	21
Tools	27	12	21	35	25	17	12	34

. All knowledge is semantically represented following the proposed approach and is available online [120]. Essential parts of this corpus are also made available through content aggregators [128].

At the same time, we argue that the proposed approach has great potential to support future research endeavors and in particular in the domain of AI. Having annotated datasets is crucial for training AI models. These annotations provide structured, meaningful context to the raw data, enabling AI models to accurately learn and interpret. More particularly in the domain of traditional crafts, where much of the knowledge is tacit and deeply embedded within specific cultural contexts, annotated datasets ensure that AI models can capture details that might otherwise be overlooked. This is essential for preserving the authenticity and richness of craft heritage. Moreover, annotated datasets enhance the AI’s ability to provide contextually relevant insights and support advanced applications such as semantic search and intelligent tutoring systems. This can support the creation of more effective and engaging educational tools. By systematically documenting and semantically annotating craft knowledge, this work aids in the preservation of cultural heritage and empowers AI technologies to support and promote traditional crafts in a manner that is both respectful and innovative.

In conclusion, this research highlights the necessity of interdisciplinary collaboration, technological innovation, and active practitioner engagement in the preservation of traditional crafts. The findings suggest that combining traditional ethnographic methods with modern digital technologies can redefine the boundaries of craft documentation, opening new dimensions for preserving intangible cultural heritage through online semantically rich digital vocabularies. At the same time, the outcomes can support further research both for studying other human activities and for supporting AI-based innovation in the craft sector.