Semantic Enrichment of BIM: The Role of Machine Learning-Based Image Recognition

Abstract

Building Information Modelling (BIM) revolutionizes the construction industry by digitally simulating real-world entities through a defined and shared semantic structure. However, graphical information included in BIM models often contains more detailed data compared to the corresponding semantic or computable data. This inconsistency creates an asymmetry, where valuable details present in the graphical renderings are absent from the semantic description of the model. Such an issue limits the accuracy and comprehensiveness of BIM models, constraining their full utilization for efficient decision-making and collaboration in the construction process. To tackle this challenge, this paper presents a novel approach that utilizes Machine Learning (ML) to mediate the disparity between graphical and semantic information. The proposed methodology operates by automatically extracting relevant details from graphical information and transforming them into semantically meaningful and computable data. A comprehensive empirical evaluation shows that the presented approach effectively bridges the gap between graphical and computable information with an accuracy of over 80% on average, unlocking the potential for a more accurate representation of information within BIM models and enhancing decision-making and collaboration/utility in construction processes.

1. Introduction

In the rapidly evolving digital era, the construction industry is experiencing a significant shift towards model-centric and data-driven workflows [1,2]. The digitalization of the construction industry offers remarkable benefits, including informed decision-making, enhanced safety and accuracy, improved integration and communication, cost reduction, increased transparency, and better prediction and control throughout the building lifecycle [3,4]. In this transformation, Building Information Modeling (BIM) emerges as a key component in supporting essential functions for construction project teams and in building advanced digital twins and data analytics pipelines [5,6,7]. The concept of BIM, as defined by researchers over time, has advanced with the expansion of its applications. It can be defined as ‘an advanced evolution of computer-aided design to provide a comprehensive, multi-dimensional view of building data across its lifecycle’. However, developing a singular, universally acknowledged definition of BIM is challenging due to its evolving nature and diverse applications in the field [8]. Furthermore, the semantic correctness of BIM data is essential, highlighting both research initiatives and practical applications by ensuring that the digital representation faithfully mirrors the physical aspects of construction projects. This reliability is essential for the effective utilization of BIM, enabling more informed decision-making and innovation in construction methodologies [9].

BIM embeds 2D or 3D graphical representations of the construction and exploits semantic information to facilitate all the stages and “dimensions” of the construction process including time (4D), cost (5D), facility management (6D), sustainability (7D), and safety (8D) [7,10,11]. However, despite its potential, the adoption of BIM in the European Union (EU) remains moderate. According to the European Construction Sector Observatory (ECSO) just under a third (29%) of construction firms make use of 3D BIM and just 6% implement BIM 4D, calling for an acceleration of BIM market adoption [12]. To address this, several EU countries have mandated BIM requirements for public procurement projects, with Denmark, Sweden, Finland, Italy, Lithuania, and Germany leading the way. Open BIM standards have been established in Italy, the Netherlands, and Austria. Italy has implemented a progressive adoption strategy, making BIM mandatory for complex projects of increasing value over time. This strategy culminates in 2025, when BIM will be expanded to encompass all projects, regardless of complexity [12].

BIM uses a shared semantic structure to simulate real objects digitally. It represents components geometrically and includes computable data for automated construction processing and simulation [13]. The completeness, accuracy, and consistency of semantic parameters are crucial for obtaining effective simulation of the construction model and meaningful data analytics [14]. In terms of completeness, BIM authoring tools often allow objects to be modelled without the definition of attributes related to the geometrical representation (e.g., length, height, number of openings in a window). Graphical details can be created in components without specifying semantic parameters like door openings or beam shapes, leading to information asymmetry between graphics and semantics. This lack of semantic data impedes computational analysis of BIM models, as machines struggle to interpret the complex structures behind graphical representations [15]. Moreover, challenges regarding precision and uniformity are also noteworthy. BIM authoring tools currently offer a basic and limited range of semantic parameters. This limitation becomes evident when one considers the complexity and detail present in graphical representations, particularly for historic buildings, which often surpass the simple categories of parameters that BIM authoring tools can integrate [16]. Consequently, many objects in a BIM model are inaccurately labeled or placed under a broad semantic category (e.g., “mass” or “generic object”). While accurate object categorization at the project level may be achieved through the analysis of graphical representations, ensuring clarity for users, this approach falls short when information must be directly utilized by machines, such as in analytics or simulations [17].

However, BIM models sometimes contain incomplete, inaccurate, and insufficient data that impact project outcomes. Instead, the industry needs BIM tools that can more efficiently adapt to continuously evolving digital workflows, suggesting a gap in the current toolset’s flexibility and adaptability [6,18,19,20]. Yet, the interpretation of images and unstructured data to identify and categorize architectural components and their spatial configurations remains a significant challenge, while the construction sector’s advancement emphasizes the need for intelligent management and sustainable decision-making in complex projects [21]. Despite the transformative potential of BIM, as reported by [22], today’s BIM practices often have a gap between graphical information and computable information that highlights the need for semantic enrichment in BIM, to ensure a more comprehensive and accurate representation of building data. Moreover, they noted that the automation level and accuracy of existing methods need to be further improved. The gap between the detailed graphical representations and the comparatively semantic data challenges the effectiveness of BIM, limiting its utility in complex decision-making processes and multidisciplinary collaborations [23].

The integration of ML, and specifically image recognition technologies, presents a promising chance to overcome the aforementioned limitations. Leveraging ML and computer vision techniques, these images can be employed to autonomously generate 3D as-built models, track construction progress on-site, and evaluate damages for maintenance needs [24,25]. Specifically, ML significantly boosts precision and automation in enriching BIM semantics, thereby streamlining decision-making processes and enhancing collaborative efficiency [26]. Accordingly, Ref. [27] highlighted the potential of methods based on ML and provided a systematic review and meta-analyses of related. They pointed out several research gaps in this field of research including the limited focus on extracting semantic information about building components using drawings and other textual and graphical representations.

To address these gaps, this research work provides a methodological approach to enhance the semantic richness of BIM systems using image recognition [28]. The approach proposes the use of ML techniques to translate graphical information into computable information that can be used in data analytics processes. The image recognition task is based on deep Convolutional Neural Networks (ConvNets) because of their high performance in handling image inputs [29]. Our approach works with BIM models exported as Industry Foundation Classes (IFC) files. A database complements these IFC models to store images that correspond to the elements defined in the IFC file. A trained ConvNet is utilized to analyze these images and extract relevant semantic information that was not originally included in the IFC model. Such semantic parameters are then used to enrich the original IFC model which can be exploited for more accurate simulations or data analytics processes.

In this direction, this research aims to mediate the discrepancy between the detailed graphical content and the limited semantic information in BIM models. This strategy not only extends the semantic depth of BIM systems but also significantly supports their analytical and simulation capabilities. By effectively converting visual details into computable semantic information, this methodology promises to elevate the precision and utility of BIM in the construction sector, driving more informed decision-making and enhancing collaborative efforts.

This study drives the field of digital construction management forward by coupling the power of ML to improve BIM. It facilitates a digital transformation in managing and executing construction projects, thereby addressing key challenges in the field. By embedding advanced algorithms capable of interpreting complex graphical data and enriching semantic content, this approach drives digital construction management toward data-driven insights and machine-enhanced decision-making. Consequently, this leads to exceptional levels of project efficiency, sustainability, and stakeholder engagement.

This work is structured as in Section 2 the background highlights the current challenges and benefits of the BIM model’s semantic representation and the motivations of this research. The proposed framework and the methodology are described in Section 3. Section 4 presents the prototype system based on a convolutional neural network developed using Python. Finally, Section 5 and Section 6 contain the implications, limitations, possible future research, and conclusions.

2. Literature Review

2.1. Open Challenges in the Semantic Representation of Building Information Models

Semantic enrichment aims to decode and integrate implicit building semantics into models, enabling their versatile application across various tasks with reduced need for modifications [30]. Semantic representation in BIM captures data meaning for intelligent processing. This enables computers to understand and analyze data, facilitating advanced decision-making [31,32]. Technical accuracy in BIM tools is crucial for effective semantic representation. Addressing issues in semantic accuracy also requires considering its functional, informational aspects and potential future applications.

The semantic representation of building information models lies in the identification of objects’ categories and the definition of the relationship among those objects [33]. To tackle this problem, some recent work highlight the importance of ontology and semantic web technology in the context of BIM, addressing the need for a more robust system of shared conceptualization that allows for interoperability, linking BIM objects, and logical inference [5,34]. Semantic representation in BIM using ontologies like ifcOWL, enables efficient data assimilation, information updates, and meaningful connections with other domains by adding explicit meaning to building information [35]. However, although semantic web technologies allow for a rigorous definition of BIM object semantics and for the automated understanding of building properties and characteristics [36], they strongly rely on the correctness and completeness of the information stored in the ontology [37].

Both the technical capabilities of BIM authoring tools and the expertise of the users managing these tools play a critical role in determining the semantic quality of BIM object representations [38]. Objects not on the embedded list are assigned to a generic category and personalized for semantic identification, but can’t have custom semantic relations. In contrast, flexible BIM platforms like FreeCAD, highlighted for their open-source nature, allow free modification and customization, enabling collaborative workflows and diverse architectural and engineering tasks [39]. Open-source platforms enhance semantic enrichment in BIM through customizable and extensible data structures, allowing users to define, modify, and customize the attributes and relationships of BIM objects to suit specific project needs [40]. Comparatively, commercial BIM which is robust but less flexible, open-source solutions support specific project needs better [41]. Accordingly, this research work aligns with this paradigm shift, as it explores methods to further enhance the semantic capabilities of BIM models, contributing to the broader goal of advancing BIM technology for more efficient and collaborative construction processes.

Challenges in capturing the semantics of structures are particularly pronounced with existing buildings, especially for the historical ones. There has been a growing interest in the application of BIM for the maintenance of such historical buildings [42,43,44,45]. In this context, notable experimentation on historical buildings includes the “Albergo dei Poveri” in Geneva [46], the “S. Maria di Collemmagio” Basilica in L’Aquila [47], Sondrio’s “Masegra” Castle [48] and the Milan Dome [49]. These studies bring to light the complexities of semantically representing historical elements within a BIM framework. For instance, the intricate features of the spire of Milan’s Dome necessitated the incorporation of a 3D model with an external database and a photographic catalog [50].

However, the geometric model itself lacks inherent object classifications and defined rule-based relationships among objects, with all related information being coordinated through external means. Further investigations reveal that commercial BIM applications fall short when it comes to modeling existing structures, particularly those historical [51]. This shortfall is linked to the intricate geometry of certain elements and the challenge—or at times, the outright impossibility—of accurately semantically representing objects within a BIM context. Additionally, the scenario with historical buildings calls for tailor-made solutions, prompting the advancement of specialized Level of Development (LOD) standards [52] and dedicated methods for managing aggregated informational sources [53]. Still, addressing the constraints presented by proprietary BIM authoring tools is a pressing challenge.

2.2. Automated Image-Based Semantic Enrichment of BIM Objects

To manage the lack of complete, consistent, and accurate semantic descriptions of BIM objects, multiple approaches in the literature highlight the potential of extracting important information from images [35,54,55]. Images often contain more detail than BIM’s semantic attributes, creating an information asymmetry. Due to machine unreadability, this image-embedded data can’t be used for simulations or analytics. Researchers enhanced the information models through the use of images, employing a variety of techniques ranging from rule-based and model-based solutions to data-driven methodologies. Recently, ML and neural networks have emerged as leading strategies for classifying objects within images and extracting pertinent information. Although such ML-based approaches may initially require more extensive training and may yield average accuracy results when compared to other methodologies, they offer distinctive advantages [15]. For instance, they can effectively manage complex, unstructured data, potentially identifying nuanced details that could be missed by rule-based systems [25,36]. Further, they can analyze visual data to detect features and relationships not explicitly delineated in the model, but implied within the visual representation, leading to a significant increase in the semantic richness of the extracted data [56].

In this direction, Ref. [57] Proposed a method for enhancing BIM models using semantic data from unknown-pose images, effectively increasing semantic coverage through accurate 3D integration. Similarly, Ref. [58] employed deep learning for the classification of building types using images extracted from BIM models. Their classification strategy yielded promising accuracy results, especially when using pre-trained neural networks on real-world images, such as ResNet.

Moreover, Ref. [42] automated the recognition and classification of building anomalies within BIM models using a deep learning-based prediction model. They reported accuracy ranging between 50% and 75%, with the potential for improvement as the dataset is further populated with images, Ref. [59] introduced an innovative workflow for the automatic detailing process in BIM using parametric design, achieving a 47.4% error reduction in classification and an 18.8% reduction in regression tasks.

The choice of ConvNets for our study is based on their proven ability to efficiently process and classify large sets of images through deep learning. This approach allows for the automatic extraction of increasingly abstract features from raw data, making it particularly effective for complex image recognition tasks in construction contexts. It specializes in complex object recognition and attribute identification, enriching BIM semantics. They can identify similar objects within one image or different objects with similar features [29]. Hence, ConvNets can be used in the classification of objects but also the identification of peculiar parameters such as the number of openings in windows or the type of a specific door. Figure 1 graphically shows a representation of a ConvNet.

2.3. Aim and Motivations of the Research

To better understand the challenges and benefits of ML-based image recognition for semantic richness in building information models,

Table 1. Main Challenges/Benefits of Semantic Richness in BIM Using ML-Based Image Recognition.

Open Challenges	Technical challenges in BIM’s semantic data interpretation [25,31]. Dependence on ontology correctness and completeness [5]. Quality reliance on current BIM software and operator judgment [38]. Semantic representation difficulties in existing buildings [27,42].
Main Benefits	Valuable details in images beyond BIM semantic attributes [35,54,55]. ML’s ability to handle complex data and detect implicit features [22]. Automatic detailing and error reduction in BIM [59].

summarizes key findings from previous studies and provides a comprehensive overview of the current state of research in this area.

Semantic information in BIM is largely confined to standard authoring tools, with limited research showing effective extraction of valuable, non-standard semantic data through object classification. To address this gap, leveraging ML and image recognition to extract finer granular semantic information and enrich BIM models at a more detailed level is used in this work. Thus, an approach using ML image recognition algorithms is proposed to extract non-geometrical, computable information from the real world and other types of images and embed it into the corresponding BIM objects defined in building information models.

The proposed methodology unlocks the potential for a more accurate and comprehensive representation of both geometrical and associated non-geometrical information within BIM models. Thereby, it enhances decision-making, collaboration, and utility in construction workflows. This integrated and semantically rich approach advances the scope and application of BIM in the construction industry.

3. Methodology

In this section, the framework and methodology of the proposed system, and an image recognition checking system designed to augment and refine the BIM model are described (i.e., this work is retrieved from the methodological approach of a Ph.D. thesis [28]).

The four main methods for semantic enrichment include Semantic Web Technology, Rule-Based Reasoning, ML, and Database-based semantic integration as mentioned in [22]. Among these, ML was chosen due to its unparalleled ability to automatically identify patterns and insights from datasets. ML’s adaptive learning capability makes it particularly suited for the semantic enrichment of BIM, as it can evolve with new data and continuously improve the accuracy and relevance of the semantic annotations. This work uses ML applications to bridge the gap between graphical and semantic representations in BIM, enhancing data quality for analytics applications and addressing the challenges posed by the evolving nature of linked building data and the limitations of current BIM authoring tools. Unlike other technologies, ML can emulate human cognitive abilities to understand context-based graphical information, making it exceptionally suited for enhancing the semantic richness of BIM [25].

The proposed framework assumes the BIM model to be exported in IFC format. IFC was selected due to its widely acknowledged status as a comprehensive data model used in BIM. It provides an open and interoperable data format, ensuring our system can be universally applicable across various software platforms [61].

The logical workload orchestrates the flow of operations and ensures an interplay among the different involved components. It operates based on multiple user-defined parameters to be extracted from images, enabling a systematic approach to identify and address potential discrepancies within BIM.

The core component of the system is an image recognition engine, based on Convolutional Neural Networks (ConvNets). The ConvNet is trained only once with real-world images, and it is then used to predict semantic parameters for new, unseen images extracted from an input IFC model. The presented methodology efficiently manipulates the IFC model to (i) extract relevant images, and (ii) enrich the model with the generated semantic parameters. Thus, the outputs of our image recognition system are not only used for checking the integrity and accuracy of the BIM but also for enriching it by filling in possible gaps or rectifying inconsistencies between geometric and non-geometric data. By integrating these components, our system presents a holistic solution, offering substantial improvements in the construction and refinement of BIM models.

Figure 2 shows the logical process of the proposed system. The process is logically divided into two main parts, namely project, and analytics. These two sections are designed to focus on distinct tasks and requirements inherent to their respective application domains. In the former, the primary emphasis lies in identifying potential discrepancies between geometric and non-geometric data. The purpose of this phase is to identify any conflicts or inconsistencies and resolve them, ensuring a reliable and accurate project execution. In the latter, the scope extends to incorporate both the previous objectives and additional tasks. In addition to resolving inconsistencies between geometric and non-geometric information, this phase strives to convert specific geometric attributes of objects into a format that can be easily processed computationally. The transformed geometric data serves as a means for conducting in-depth data analysis, enabling the extraction of meaningful insights and fostering informed decision-making processes.

The logical process of the proposed system unfolds across four defined stages: (1) parameter definition and training of the image recognition engine, (2) image extraction from a building information model (BIM), (3) model verification, and (4) model enrichment.

3.1. Semantic Parameters and Model Training

Initially, a set of user-defined semantic parameters are defined. This process is predominantly manual and demands domain-specific knowledge [62]. These parameters can vary according to the requirements of the project and the data analytics task. For example, the Level of Detail (LOD) of the objects, the context of the project, and the specific software utilized to create the original model can all influence the selection of these parameters.

These user-defined parameters serve a pivotal role in training ConvNet. Concerning this, we anticipate two likely scenarios. The first scenario involves the availability of a pre-existing set of images of the building or objects relevant to the project. The second scenario foresees the necessity to gather a new set of images explicitly for the development of the approach. Regardless of the scenario, the defined parameters are instrumental. They are either used to sift through and select the appropriate images from the available set or to steer the process of new image collection. The nature of the selected parameters can sometimes necessitate a manual labelling process for the images. For instance, while it might be straightforward to find an image dataset with labelled doors and windows, it becomes considerably more challenging to find one where windows with a varying number of openings are specifically labelled. Despite this, the manual labour involved in the labelling process only needs to be endured once for each unique set of parameters. This is because the resultant image dataset, once defined and labelled, can be reused in future applications, thus adding a valuable resource to the image recognition workflow. At the end of the first stage, the ConvNet is trained using the selected images along with their labels. The model is then deployed in a production environment (e.g., in the Cloud) to serve requests from users.

3.2. Image and Data Extraction from the Information Model

In the second stage, the approach reads the building information model in IFC format. Each object in an IFC model carries a Globally Unique Identifier (GUID), a unique string assigned to each component that allows for clear and precise identification within the model. The framework of IFC permits the examination of every component within an IFC model via the use of GUIDs. It encompasses relationships between objects, enabling the retrieval of visuals that represent not just individual items but also objects situated within their environment (for instance, a window set within its surrounding wall). The quantity of related objects can be defined in the extraction procedure imposing the depth of the query on relations. For example, a depth will retrieve a selected object X and the objects Ys related to it, while a depth equal to two will retrieve the selected object X, the objects Ys related to it, and the objects Zs that are related to at least one in Ys. This research is centered on queries that do not exceed a single level of depth. In this stage, the approach uses the previously defined parameters to select pertinent objects in the IFC model, extract their GUIDs, extract their renders and export them as images (e.g., in .jpg format), and store these data in an external database. In the database, the images are paired with the GUID of their originating objects. This GUID-image pairing enables a direct link between each object in the IFC model and its corresponding image stored in the database.

The criteria for image selection in the experiment were based on the specific requirement of identifying the number of openings in windows, leading to the collection and labeling of images showcasing windows with two and three openings. The training dataset comprised 90 images for each category, while the testing set included 25 images per category, ensuring no overlap between the training and testing sets to maintain the integrity of the evaluation process. The process involved utilizing a Python script specifically tailored to autonomously identify and extract images of “IfcWindow” objects from an IFC model. This script was instrumental in assembling the initial dataset, which incorporated a variety of characteristics to assess potential impacts on the image recognition engine’s efficacy. It should be noted that variations in color and orientation did not significantly impact the recognition process and performance [28].

3.3. Semantic Information Validation

Once the images are extracted, they are passed to the image recognition engine, and the ConvNet trained in the first stage is used to recognize and label these images. This stage aids in verifying the original IFC model by spotting potential discrepancies between geometric and non-geometric information. The output of this stage is a textual report file where such inconsistencies or missing values are reported. It must be highlighted that the definition of the requirements and consequently of the parameters to check and/or to introduce is still a manual activity. In fact, the same information can be defined with different names and/or different meanings in the model due to the variety of standards and requirements that can be employed in each project [63]. Hence, the definition of the well-defined definition procedure remains a critical point that needs to be performed manually by expert users with domain-based and project-based knowledge.

Inconsistencies between geometric and non-geometric information, such as the difference between names and dimensions in an object, emphasize the challenges in maintaining model accuracy and integrity [64]. It must be highlighted that enriching the model can provide structured information usable for cross-checking activities. Respectively, by understanding the number of openings in a window, the supply chain management processes could be enhanced, including ordering and control, among other potential applications. This highlights the importance of detailed and accurate modeling in improving operational efficiency and application.

3.4. Semantic Enrichment of BIM Objects

The fourth and final stage of the process is centered around model enrichment. Upon successful image recognition, the semantic descriptions extracted from images can be used to enhance the existing IFC model. This process of data enrichment is not only about appending the newly discovered information but also includes the verification and possible correction of the existing data. This enrichment process often reveals inconsistencies between the graphical representation and the non-geometric attributes of a given object. For example, ConvNet might identify a certain object in an image as a double-paned window, while the corresponding object in the IFC model is labeled as a single-paned window. Such an inconsistency indicates an error in the IFC model, which is then corrected using the information obtained from the image recognition. The enriched IFC model thus stands as a more complete and accurate representation of the actual building. It not only captures the geometric features but also carries a richer set of non-geometric attributes, providing a more detailed and precise informational resource for any downstream applications [65]. This enriched model also paves the way for more sophisticated data analysis and fosters an enhanced understanding of the building, its components, and their interrelations.

4. Results

4.1. Development of Prototype

A prototype was developed using Python programming language to test the proposed methodology. Python was selected for several compelling reasons. Firstly, it simplifies the development and testing process through its ability to run code directly without compiling it into an executable. It also boasts specialized libraries for crafting ConvNets (e.g., Keras Packages [66]) and offers clear syntax which facilitates the future sharing and extension of the work. The training images used in the prototype have been labelled and selected manually from available web image repositories.

While the initial parameter definition and selection of images for the dataset involve manual processes, steps have been taken to ensure diversity and minimize bias. The manual selection was guided by detailed parameters such as LOD and project context, ensuring a varied image range. This initial dataset, once created, supports future automated applications, thereby minimizing bias over time [67].

4.2. Descriptive Case

An illustrative example was developed to explore the effective applicability of the proposed process. It aimed to identify a particular characteristic within window objects in BIM; specifically to discern between windows with two openings versus those with three. The experimental process unfolded as detailed below:

(i)

Before initiating the training of the image recognition system, it was essential to define and organize image collections based on pre-determined criteria. The experiment focused on the number of window openings. Thus, sets of images representing windows with two and three openings were compiled and labeled accordingly, collecting 90 images for each type for the training set. Moreover, a separate set of 25 images per type was gathered for testing, ensuring no overlap with the training set.

(ii)

Next, the assembled images were employed in the training set and the testing set of the ConvNet, imposing it to the set requirements. This step was crucial for assessing the ConvNet’s efficiency and its capacity to adapt to specific parameters. Therefore, multiple tests were conducted to ascertain the parameter mix that would ensure higher system performance.

(iii)

Once the training and validation phases were complete, the refined image recognition system was ready for application, tasked with identifying images derived from building information models. These images, which could be automatically generated and saved in an external database, might depict a lone object or that object in its situational context. For this study, both single windows and windows within their walls were used for validation. Moreover, the images could capture various perspectives and be in black and white or color. The experiment thus involved testing different image combinations to evaluate how these variations influenced recognition accuracy.

4.2.1. ConvNet Training and Validation, Results Evaluation, and Definition of Parameters

The ConvNet’s algorithm relies on precise calibration of various parameters to achieve an appropriate learning trajectory and optimize image recognition performance. Specifically, three critical parameters are involved: the number of complete passes through the provided dataset (i.e., termed ‘number of epochs’), the quantity of operations executed before an iteration is deemed complete (i.e., ‘steps per epoch’), and the count of validation checks conducted before termination (i.e., ‘validation steps’). The standard-setting values have been used for the algorithm according to the literature [22].

The ‘steps per epoch’ and ‘validation steps’ are determined by the size of the training or validation dataset and the batch size, which is the number of examples utilized in a single iteration. With a batch size set to 30, we defined ‘steps per epoch’ as 4 and ‘validation steps’ as 1. Given the modest size of the datasets, the ConvNet began an early overfit, meaning its learning stopped and performance declined when the number of epochs exceeded 55. Various parameter combinations and found optimal performance between 50 to 55 epochs with the previously mentioned ‘steps per epoch’ and ‘validation steps’ were experimented. The accuracy and loss functions that are used to measure the error in the recognition process, on the training dataset are shown in Figure 3.

In the assessment of the validation set, Figure 4 indicates a progressive advancement in the learning curve of the ConvNet. Notably, the accuracy ascends and appears to level off at approximately 84% towards the end of the learning and validation phases. This trend is observable in the validation loss as well, which is documented alongside the accuracy in these figures, reflecting a consistency in ConvNet’s performance throughout validation.

The graphical performance data relates to the identification of actual images, whereas the experiment aimed at recognizing simulated objects retrieved from building information models. Consequently, a secondary validation level was determined, wherein the trained ConvNet was employed to identify images extracted directly from the building information models.

4.2.2. Validation and Evaluation of Building Information Model Object Images

The ConvNet’s precision in identifying images from building information model objects was first validated using a dataset of 50 images. As previously explained, these images varied according to different parameters including angle, context inclusion, or/and color inclusion. This initial dataset was therefore assembled with diverse characteristics to identify any impact on the image recognition system’s efficacy. Figure 5 illustrates various images of a window from the building information model; particularly, image 1 depicts a two-opening window complete with material textures set within the surrounding wall, image 2 presents the same window in isolation, captured frontally in grayscale, and image 3 captures the same window at a different angle, also in grayscale and situated within its hosting wall.

The initial dataset used to evaluate ConvNet’s recognition capabilities revealed a 70% accuracy rate in distinguishing between windows with two openings versus those with three. A closer examination of the outcomes indicated a correlation between image attributes and algorithmic success. Standalone window images, devoid of contextual elements like hosting walls, yielded higher recognition accuracy. This aligns with ConvNet’s architecture, which deconstructs images, rendering them indifferent to color or object orientation. However, when parts of the windows were obscured by walls, there was a notable drop in recognition performance.

To validate these findings, a refined dataset exclusively consisting of isolated window images was assembled. Subsequent assessments conducted 100 times over this refined set, demonstrated an improved average accuracy of 82%, occasionally achieving highs of 90%. This confirmed that ConvNet’s efficacy was significantly influenced by the visual isolation of the window features in the images.

5. Discussion

Researchers are increasingly leveraging these cutting-edge technologies to enhance traditional methods, yielding remarkable results. The application of image recognition in the construction industry for geometry reconstruction from models of existing structures has demonstrated promising accuracy levels in various studies [68].

Reflecting on the advancements in this field, Ref. [42] utilized ML to automate anomaly detection in BIM models, achieving an accuracy between 50% and 75%. Similarly, Ref. [59] introduced a nouvel workflow for automatic detailing in BIM, resulting in an average error reduction of 47.4% in classification tasks and 18.8% in regression tasks. Several research works suggest applying image recognition techniques for classifying different object types within existing structures [27,32].

The accuracy achieved in the experiment under discussion states the viability of image recognition techniques in construction-related tasks. The exploration into ConvNet’s applicability within the construction domain, particularly for BIM model enrichment, unveils nuanced insights into the interdependencies between image characteristics and ML efficacy. A comprehensive evaluation of the proposed approach shows that the presented methodology can enrich BIM models with an accuracy that is equal to 84% with realistic photographs and 82% with digital rendering.

Nevertheless, it is important to note that the current prototype was trained using a relatively small image dataset. Increasing the dataset size could potentially enhance the model’s recognition capabilities, as evidenced by successful outcomes in other research domains. This suggests that further improvements in the image recognition process could be realized with the expansion of the training dataset.

The recognition process in existing studies is limited to standard components such as walls, doors, and windows, excluding elements that are not definable within BIM authoring software. Moreover, these studies do not consider the non-geometric attributes linked to the geometric data in their recognition mechanisms. While, this study introduces an innovative approach that facilitates the unrestricted classification of objects beyond the constraints of standard BIM authoring solutions, and the identification of features grounded in the geometric depiction of virtual entities. Additionally, it demonstrates the proficiency of ConvNet in discerning particular attributes of virtual objects by employing real-world images sourced from the internet as a basis for training and validating the system.

Furthermore, the evolution from CAD to BIM, incorporating artificial intelligence and ML, marks a significant development in construction technology, enhancing model accuracy and fostering a data-rich, collaborative environment [69,70]. This integration is most effective in technologically advanced settings, promoting efficiency and innovation in digital construction management. However, The effective fusion of ML and BIM mainly occurs in technologically equipped settings: in a dynamic environment where diverse areas are associated and multiple interpretations on different levels can cooperate [25,28]. Enriching BIM models, particularly ML-based image recognition, has extended BIM’s capabilities, enhancing its semantic richness and facilitating a deeper understanding of construction elements [26,71]. This progression points towards the potential of BIM across the construction lifecycle, driving industry-wide adoption and innovation [3,6].

The practical implications of image recognition innovation into BIM systems extend significantly beyond theoretical advancements for industry professionals to be considered as future work. Moreover, the ability to automatically update and enrich BIM models with real-world images aids in maintaining the relevance and accuracy of the model throughout the building’s lifecycle, offering benefits in facility management and renovation planning [3]. The adoption of such technologies by industry professionals can lead to improved collaboration, where decision-making is supported by data-rich, accurate BIM models.

Incorporating image recognition innovation into BIM systems offers tangible benefits for a diverse range of stakeholders in the construction industry. Architects and engineers, as primary users of BIM models, stand to gain from enhanced model precision and detail, facilitating more informed design decisions. Construction managers can leverage improved model accuracy for efficient planning and execution, potentially reducing project timelines and costs. Facility managers benefit from enriched BIM models for more effective maintenance and renovation planning, ensuring that buildings continue to meet occupant needs over time [72]. Additionally, property developers can utilize these enriched models to communicate more effectively with potential buyers or tenants, showcasing the building’s features with greater accuracy.

6. Conclusions

This work navigates the complexities of modern construction practices, and the integration of precise and comprehensive digital models stands as a foundation for advancing the industry. The core aim of this research was to address the inconsistency between the graphical details and the computable semantic information within BIM models. Through a method to transform these visual details into computable semantic information, the proposed framework significantly enhances the semantic depth and utility of BIM systems. This advancement supports more robust analytical and simulation capabilities within the construction sector, promoting informed decision-making and improved collaboration.

This research work proposed a framework designed to validate the consistency between geometrical and non-geometrical information in building information models, and to verify the accurate semantic representation of model-contained objects, thus laying the groundwork for enhanced future data utilization. The framework employs a ML image recognition engine, which has exhibited substantial efficacy in discerning features of virtual objects using a training set derived from real-world images.

An image recognition engine built upon Convolutional Neural Network (ConvNet) technology, was trained in accordance with set parameters and conditions. After its training phase, the engine processed images of BIM objects archived within a separate database and each linked to the object’s unique Global Unique Identifier (GUID). The engine was then applied to identify specific features in these images, leveraging this insight to verify and potentially enrich the BIM data. This adept interpretation of image data by the ML engine helps bridge the gap between graphically transmitted information and computable data, surpassing the semantic limitations inherent in existing BIM authoring tools.

The research included the development of a prototype utilizing Python, demonstrating a significant reduction in the time required for extracting images from the Industry Foundation Classes (IFC) file format, thereby transitioning the task from a manual to an automated process. With a ConvNet optimized for a computing environment equipped with an Intel i7 processor and 16 GB RAM, the ML training process was approximately one hour. It is noteworthy that the training was not multi-core optimized, suggesting further time reductions could be achieved through algorithmic refinement.

Several aspects have been identified for potential future developments:

I.

The methodology relies on identifying individual components within models by their classes or other details discernible from their graphical depiction. Yet, distinguishing some elements proves challenging due to their visual resemblance and the complex assessment of scale in the images sourced. In alignment with the cognitive processes of humans and ongoing research in automatically generating BIM from point cloud data, future research could focus on integrating algorithms that understand the context, providing a dual verification process that assesses both individual object recognition and its placement within the model’s environment.

II.

Defining the recognition and verification rules is both time-intensive and knowledge-extensive. Future studies could investigate the potential for reusing such rules to enhance the practical applicability of the system.

III.

The current requirement for manual collection and labeling of images represents another bottleneck. Investigating automated solutions for image collection and labeling, could significantly enhance the efficiency and quality of the training set compilation.

IV.

While ConvNet was selected for its proven capabilities in image recognition tasks, the exploration of alternative algorithms could provide valuable comparative performance data, potentially revealing more efficient or accurate options for image recognition within the construction industry domain.