A Peer Review System for BIM Learning

Abstract

This research proposed a peer review system for the instruction of building information modeling (BIM). With the rapid growth of BIM technology, 3D modeling has become an essential skill for civil engineers. However, it is difficult for an engineer to gain 3D modeling skills through conventional lecture-based learning. The peer review approach has been recently introduced as an educational method and is widely utilized to promote learners’ active learning that is hard to teach using conventional approaches; however, no peer review system has been developed thus far for teaching 3D modeling. Therefore, this research developed a peer review system for 3D modeling courses and proposed guidelines for developing a peer review-based BIM course. Additionally, this study developed a web-based reviewing system, which allows the learner to review others’ works online. The system was implemented in an undergrad-level course for validation. The results revealed that using the peer review system for 3D modeling instruction can help learners to gain more detailed skills; it also allows learners to learn from others’ works. The system can also help the instructor to identify potential mistakes that the students may make while constructing the BIM model and to make continuous improvements for future courses.

1. Introduction

1.1. BIM Education

The concept and tools of building information modeling (BIM) are now increasingly utilized in construction projects in the Architecture, Engineering, and Construction (AEC) industry. All information relating to the entire life cycle of a construction project (i.e., buildings and infrastructures) can be integrated using BIM technologies [1]. BIM has been proven to have several benefits for a construction project, such as improving the commutations between sectors [2], elevating process efficiency [3], increasing data integrity for the operation phase [4], and reducing project costs [5]. Recently, utilizing BIM has become one of the basic requirements of construction projects. Therefore, how to train a BIM-ready engineer for the industry is now receiving increasing attention and is thus worthy of further research.

The introduction and development of BIM have had an evolutional impact on the AEC industry. The demand for education on the use of BIM tools has grown rapidly, making this topic one of the main focuses of research in engineering education. However, while most of the previous studies have focused on the implementation process [6], the benefits of BIM [7], and the challenges involved in solving strategies [8,9], few studies have emphasized the use of BIM tools [10]. BIM tools, such as Autodesk Revit and Tekla Structure, are hard to learn via conventional teaching methods because the BIM model usually contains thousands of components with many different properties. Thus, it is difficult for the instructor to cover every detail within a limited time. Additionally, as an engineer may encounter various kinds of problems when dealing with different projects, it is difficult for the instructor to illustrate every possible scenario during the lectures. Suwal and Singh [10] found that the teaching of BIM tools relies on the students having a positive learning sentiment. Magana et al. [11] stated that active learning should be implemented in engineering education to motivate students’ learning to achieve better learning effectiveness.

Therefore, this research aims to develop a BIM tool learning system using the peer review mechanism to enable active learning for students. Besides the peer review system itself, this research will also propose guidelines of how to develop a peer review-based BIM course. Additionally, the developed system will then be implemented into an undergraduate level course for validation.

1.2. Active Learning in Engineering Education

Although traditional lectures in classrooms are still the mainstream, their efficiency in engineering education has been challenged for years. Felder and Silverman [12] pointed out that most engineering classes are passive, auditory, and abstract, thus leading to poor student performance and the loss of potentially talented engineers. The concept of active learning has, therefore, gained increasing attention over recent years [13]. Compared with passive learning, active learning requires students to participate in visible actions instead of merely sitting, listening, and reading [11]. Active learning has also been shown to facilitate the long-term retention of information, develop problem-solving abilities, and motivate students’ interests [14].

New educational approaches with the concept of active learning have become a trend in the training of engineers [15]. For instance, Gillet et al. [16] developed a collaborative web-based experimentation environment for engineering education in the Ecole Polytechnique Fédérale de Lausanne, and Carlson and Sullivan [17] built the Integrated Learning and Teaching laboratory to create an experimental learning environment that enabled students to learn through hands-on experience. Tsai et al. [18,19] developed a serious game-based package for disaster-related engineering education, and Clark et al. [20] explored the feasibility and effectiveness of utilizing blended and flipped instructions for numerical methods courses. Project-based teaching methods have also been implemented in engineering education [21,22,23].

Apart from experimental, game-based, and project-based learning, peer review is another popular direction for engineering education. The peer review mechanism has been widely used for academic purposes, and several studies have been conducted to bring such mechanisms into the classroom [24]. The features of the peer review mechanism can benefit engineering education, especially for subjects that contain many details and complex contents (i.e., writing courses, programming courses, or 3D modeling courses). However, utilizing the peer review mechanism to teach about BIM tools, which include many details and complicated contents, has rarely been discussed. The following section discusses the benefits and potentials of implementing the peer review mechanism in the education of BIM tools.

1.3. Peer Review Systems

The peer review mechanism allows students to learn from their peers’ and their own work through the process of reviewing, commenting, and self-reflection. The process of reviewing a peer cultivates the positivity of active learning because it enables students to participate in the learning process actively [25]. Learning through peer reviewing can also help students to set their learning goals and evaluate their performance while reviewing others’ work [26]. The peer review mechanism allows students to not only review and give feedback to their peers but also receive detailed and timely feedback, which is important for students but often difficult for the instructor to provide [27,28].

The peer review mechanism has been extensively discussed in the engineering education literature. For instance, Gehringer [24] developed a peer review system named Peer Grader for students to review and grade one another’s assignments in computer science courses. Liu et al. [29] developed a web-based peer review system for computer science students in an Operating Systems class. Additionally, EduPCR, developed by Wang et al. [30], used the peer assessment approach for teaching a computer programming language. Carlson et al. [31] and Member [32] both attempted to implement peer review learning in engineering design courses. The results of their studies implied that peer review could transform the student from being a passive learner to being an active learner. Although previous research has illustrated that peer review can benefit the training of engineers, little research has discussed the application of peer review in BIM tool training.

A peer review system for BIM tool training should contain two major parts: guidelines for the instructor to develop the assessment rubrics and a platform for the learner to conduct the peer assessments. The guidelines should help the instructor to develop appropriate assessment rubrics for not only evaluating the learning performance of students but also achieving the ultimate learning goal of the course. Since the learner is required to build 3D models when learning about BIM tools, it is also important for the system to have a reviewing platform that allows the learner to review others’ models easily. The platform should be well-designed with an easy-to-use user interface so that operating the system will not create any resistance to such learning activities.

2. Research Objectives

This research aims to discuss the feasibility and effectiveness of implementing a peer review system for teaching about BIM tools. As BIM has gradually become the trend in the AEC industry, how to cultivate a BIM-ready engineer is an urgent topic for educational institutes. However, unlike other knowledge-based courses, BIM tool learning is a skill-based course, making it hard for the student to learn from a conventional lecture-based approach. Therefore, this research develops a peer review system for the BIM tool learning course. The system contains guidelines for the instructor to develop the course and a web-based platform for the learner to review and grade others’ work.

3. Peer Review System for BIM Tool Learning

This research proposes a peer review system to discuss the feasibility and effectiveness of peer review in the teaching of BIM tools. The developed system included two major modules: (1) design guidelines for instructors to design the peer review-based course and (2) a web-based platform for learners to review others’ work. The following two subsections describe the proposed guidelines and the developed platform, respectively.

3.1. Guidelines for the Peer Review-Based BIM Course

The methods used to develop a peer review-based course are different from those used to develop a conventional lecture-based course. To help instructors develop an effective peer review-based course, this research proposes guidelines for developing a peer review course for BIM tool learning. The guidelines contain the process of designing a peer review-based BIM tool course, which includes the assessment tasks and the related assessment rubrics.

Several studies have discussed how to develop an effective peer review system. For example, Chism [33] summarized that an effective peer review process should:

• provide both formative feedback and summative decision-making,
• have a developed process and instrumentation with attention to thoroughness and fairness,
• allow the peer reviewers to understand their tasks and be prepared to accomplish them,
• foster trust and confidence in the process among all parties,
• invest ongoing efforts in improving the peer review process itself,
• develop assignments in ways that are likely to result in helpful collaborations,
• be a valued process within the academic unit, and
• let all parties cooperate in accomplishing peer review tasks in a timely way.

This research develops a process for designing a peer review-based BIM course by following the characteristics summarized by Chism. Figure 1 illustrates the proposed process, which includes the following eight major steps:

• Design a real case project: The instructor should use a real case, such as a real building or a real bridge, as the learning project. To take into account the variety of real construction projects, the instructor needs to convey a considerable amount of information to cover many different kinds of scenarios. As this is difficult for the instructor to do, it is important for the learner to study real cases so that the learner can then gain active experiences after finishing the course project.
• Set teaching/learning milestones: After finding a real case project, the instructor should set the teaching/learning milestones by following the course schedule. The milestones should follow the actual process in the industry. For example, the construction of a building should start with the foundation work and build up to the basement, the main structure components, and the exterior and interior design. By doing so, the learner can become aware of how the industry uses the tool in practice. This method can also help the learner to connect what they learn with real industry work.
• Break down the project into sub-tasks: After setting the learning milestones, the instructor should break the entire project work down into several sub-tasks. Each sub-task should be designed by following the milestone that he/she has just set. When breaking down the sub-task, it is important to set simpler goals for the learner in the early stage and gradually add complexities to the learning goals in stages. For instance, in the early stage, the sub-task may be “Build a model of a wall” or “Build a model of different windows.” After that, more complexities can be added to the following sub-task, such as “Construct a wall with two windows.”
• Develop assessment rubrics: The instructor should develop rubrics for each sub-task for the learner to use as the assessment guidelines when reviewing others’ work. These rubrics should follow the set milestones for each sub-task. In BIM tool learning, it is important to make the learner aware of potential mistakes that they may make. Therefore, this research suggests that the rubrics can be developed using a binary checklist. The instructor should list the critical parts of each milestone and let the learner take a close look at others’ models. Additionally, unlike in architectural design or the composition of an article, evaluating the BIM tool learning performance of peers contains few subjective elements. A binary checklist could also make the peer review rubrics more objective, thus facilitating the learner’s implementation of the assessment. For instance, instead of the question, “Do you think the model of the wall is well built with the approximate information attached?” the question should be phrased as “Is the wall built to the right scale and in the right position?” or “Is the window installed in the right place according to the shop drawing?”
• Conduct the peer review: Before conducting the peer review process, it is necessary for the instructor to provide detailed instruction of the peer review mechanism. With the instruction of the entire mechanism, the learner can have a better understanding of how it works and thus have trust and confidence in the mechanism. The instructor also needs to provide a platform for reviewing the works. Such a platform should (a) be simple to use, to avoid adding extra load to the workload of the learner, which may create resistance to using the platform; (b) allow all reviewing to be conducted anonymously to ensure the fairness of the peer review process; and (c) make the learner randomly review more than one work, so that every learner can obtain more than one set of feedback from their peers.
• Review the result per sub-task: The instructor should review the results of the peer review after learners have conducted every sub-task to determine whether the designed model acquired the expected results. By reviewing the results of each sub-task, the instructor could make ongoing improvements to the tasks.
• Collect feedback after the project: After finishing the project/course, the instructor should collect feedback from the learners. A subjective questionnaire could be utilized at this stage to acquire learners’ feedback about the peer review mechanism and establish what they have learned from reviewing others’ work.
• Review the feedback: At the end of the course/project, the instructor should review all feedback received from the learners and use it to make improvements to the entire system, including the process, the milestones, the sub-tasks, and even the reviewing platform, for the next course.

3.2. Web-Based Reviewing Platform

The second module of the proposed system is a web-based reviewing platform. Figure 2 illustrates the structure of the designed platform. The instructor and students have major roles in the system. First, the instructor needs to input the students’ basic information and designed rubrics into the platform. The system automatically sends email notifications to the students and asks them to upload their project files for each milestone. After the students have uploaded their files to the platform, the system automatically assigns the peer review tasks and rubrics to students anonymously. The students receive their final grades when they have finished the entire course.

4. Implementation

This research developed a five-week BIM learning program aligned with the proposed peer review system and created a prototype of the web-based reviewing platform. The following section describes the development of the BIM learning program and the prototype of the web-based platform.

4.1. Development of the BIM Learning Program

This research used Autodesk Revit, which is currently the most popular BIM tool in the industry, to develop a learning program. The real case project was a nine-floor building with seven rooms on the ground floor and two basements; this building is now used by the National Architectural Graphics Certification Exam. The project was separated into five milestones based on actual construction sequences (

Table 1. Developed BIM learning program with five milestones.

Milestone	Learning Items
Milestone 1: Foundation and basement
Learning target: Learn how to lay out the construction site in BIM software
	Introduction of the Revit environment
	Grid and floor line functions
	Foundation beams and columns
	Raft foundation slab and basement slab
	Basement columns and exterior walls
	Basement interior walls
	Basement openings (windows and doors)
	Basement exterior stairs
	Basement interior stairs and section lines
	View range functions in Revit
Milestone 2: First floor and property lines
Learning target: Learn how to use the basic 3D modeling operations in BIM software
	1st floor beams and slabs
	1st floor columns and exterior walls
	1st floor stairs and interior walls
	Create and edit family type – window example
	1st floor windows and doors
	Cornered windows
	1st floor to 2nd floor stairs
	Site and property lines
Milestone 3: Second floor and driveway
Learning target: Learn how to use the advanced operation skills in BIM software (e.g., parametric modeling for openings and furnishing)
	2nd floor beams and slabs
	2nd floor walls and openings
	2nd floor balcony and ramps
	2nd floor furnishing – bathroom and kitchen
	Import CAD file function in Revit
	Driveway model
	Edit driveway terrain
	2nd floor to 3rd floor stairs
Milestone 4: Third floor to seventh floor and roof
Learning target: Learn how to combine basic and advanced operation skills for detailed modeling
	3rd floor beams and slabs
	3rd floor columns, walls, and openings
	3rd floor to 4th floor stairs and elevator openings
	4th floor to 7th floor models
	Roof beams and slabs
	Roof GFRC board
	Roof P2 and P3 floors
	Water tanks and roof plates
	Roof shafts and balcony handrails
	Textures and materials
Milestone 5: Finalization—rendering, quantity takeoff, and shop drawings
Learning target: Learn how to use the advanced functions of BIM software including floor plan drawing, 3D model rendering, quantity takeoff, and section view
	Model review
	Camera setting and rendering
	Basic animations
	Model and site terrain integration
	Generate quantity takeoffs
	Shop drawing template setup
	Floor plan drawings export
	Section view drawings export

). The learning targets of the first milestone are laying out the construction site in the BIM tool and building the model of the foundation and the basement, which contain ten learning items for the foundation and basement model build. The learning target of the second milestone is to learn the basic operational skills for the structural 3D modeling. This is comprised of eight learning items for the first-floor model building and for defining the property line. This milestone contains detailed instructions of model building, including the beams, columns, walls, openings, and stairs. The third milestone focuses on the advanced operation skills for openings and furnishings. In this milestone, the second floor and the driveway model of the building will be built. For the fourth milestone, students are expected to perform detailed modeling by combining the basic and advanced skills learned from the previous milestones. In this milestone, students will learn how to build subsequent floors and the roof. Lastly, in the fifth milestone, students learn how to use the built-in functions for rendering, generating quantity takeoffs, and exporting shop drawings.

For the teaching approach, this research developed video-based tutorials with detailed step-by-step instructions for every learning item in each milestone. Students are supposed to follow the video instructions and finish the BIM model of the building project. Besides utilizing the video tutorials, the instructor may also need to review students’ works and illustrate the common mistakes within their works for the students to learn from each other.

For the peer reviewing rubrics, this research developed five peer review sections for each milestone based on the proposed guidelines (

Table 2. Developed BIM learning program with five milestones.

Milestone	Review Rubrics	Points
Milestone 1	The raft foundation is completed.	5
	The basement floor (beams, columns, slabs, and walls) are completed.	5
	The beam under the exterior stairs of the basement is corrected.	10
Milestone 2	The building elements (beams, columns, slabs, and walls) of the 1st floor are completed.	5
	Site and property lines are drawn.	5
	All the cornered windows are finished appropriately.	5
	The elevator openings are finished.	5
Milestone 3	The overall model of the 2nd floor is completed (structure, driveway, and terrain).	5
	The furnishings of both the bath and the kitchen are completed.	5
	The balconies of the 1st floor are completed.	5
	The ramps of the balcony are completed.	5
Milestone 4	The overall model from the 3rd floor to the roof level is completed.	5
	The water tank and the roof plates are completed.	5
	The GFRC boards are completed.	5
	Two roof shafts and the balcony handrails are completed.	5
Milestone 5	The overall model of the entire building is rendered.	5
	The model file contains at least two section views.	5
	The model of the water tank with 20 cm plate is modified.	5
	The model of the stair platforms of each floor is modified.	5

). For milestones 1 and 2, students are asked to review the main structure (columns, beams, and walls) of the basement and the first floor in detail. However, for milestones 3 and 4, students need to focus on the parts that require more skills to build the model, such as the balcony ramps, roof glass fiber reinforcement concrete (GFRC) board, and roof plates. In the last section, students are asked to review the completed model, rendered views, section views, and exported drawings. Since complete BIM models usually contain many details, students should be asked to use their best judgment to evaluate the quality of others’ work and grade it between zero and the maximum preset point value.

4.2. Prototype of the Reviewing Platform

This research developed a prototype of the web-based platform. The instructor uploads the students’ information and the peer review rubrics onto the platform (Figure 3 and Figure 4), and the students upload their models onto the platform for review. Then, after the students have submitted their assignments, the instructor assigns the reviewing tasks to the students via email (Figure 5). After receiving the email, the students review and grade their peers’ work based on the rubrics (Figure 6). Last, the system automatically calculates and generates the final results of the reviews for the instructor.

5. Validation

For validation, the developed BIM learning program was implemented in a BIM fundamental course. The students were asked to peer review others’ Revit model assignments. After finishing the semester, the feedback was collected from the students for further analysis.

5.1. Actual Course Implementation

This research utilized an undergraduate-level course of 32 students for validation. The course was an 18-week course targeting the learning of BIM tools, such as SketchUp, Revit, Dynamo, Navisworks, and Tekla. The developed learning program was implemented in the course for five weeks. During the class, students were asked to use Revit to build a model of the seven-floor building. During the peer review process, each student was assigned to review three of their peers’ works. The instructor used the average of the three grades as the students’ final review grades for the assignment portion.

5.2. Evaluation of Effectiveness

At the end of the semester, results from the 32 students were filtered for analysis; only 20 remained after removal of the students who either failed to complete the peer review or quit the course during the semester. For effective evaluations, this research compared the assignment scores of the peer review results with the instructor’s grades (Figure 7). Figure 7 illustrates the grades of the peer reviews (bars) with those of the instructor (line) for the 20 students.

The results show that the distributions of the peer review results matched the instructor’s grades. Kendall’s correlation coefficient was also conducted to measure the correlation between the two grading results. Kendull’s tau was 0.4450, and a p-value of 0.0608 was obtained, indicating that a positive correlation existed between the two sets of grades, with a significance of 0.05. However, the peer review results had a higher average of grades than that of the instructor. This outcome is because when grading the assignments, the instructor had to consider and penalize any late submissions. However, for student 20, the instructor’s grade is higher than that of the peer review results since the student failed to complete milestone 5 due to some personal issues, and the instructor allowed the student to complete it after the submission deadline.

5.3. Qualitative Feedback Analysis

Besides comparing the peer review results with the instructor’s grades, this research also collected qualitative feedback from 26 of the students after they finished the semester. Their feedback can be divided into five positive and five negative themes.

The five positive feedback categories are as follows:

• The course allows students to learn from their peers’ works (9 of the 26 provided feedback): The majority of the positive feedback mentioned that it was a good experience to learn from others’ work. Some found the creativity of their peers valuable in addition to what they learned from the instructor. They also observed others’ mistakes and considered these to reflect on their own work. For example, one of the students mentioned, “It is a very good system. We can see other students’ work and learn the advantages of other students, thus correcting our own shortcomings.”
• The course allows students to learn from the rating process (7 of the 26 provided feedback): Students placed a high value on the feedback process because it offered a fresh experience that helped them understand the difficulty that the instructor and teaching assistants have when grading their work. For example, one of the students stated, “I think this gives us a great opportunity to learn how to check mistakes and where to pay attention when modeling; it also lets us know how the teaching assistant and professors give marks and what the criteria are.”
• The course is an interesting way of learning (9 of the 26 provided feedback): As the course is very different from the old ways of teaching/learning, the students could learn from a different perspective, which was interesting and motivating for them. For instance, one of the students replied that “Using the peer-to-peer evaluation method, I think it is quite new, and it is also a time for us to learn extra. After all, if only the traditional assignments are assigned by the assistants or teachers, only the mistakes made by themselves can be found and modified, but that is it. If you use peer-to-peer evaluation methods, you can not only find out that you are different from others but also learn how others do well. Therefore, I think the [peer review method] is unique and meaningful.”
• The course helps the students understand the goal of the learning subject more clearly (7 of the 26 provided feedback): By rating others’ works, the students could better understand the learning goal through the grading criteria and reflect on their own work to see if they achieved the goal. For instance, one of the students mentioned, “Through peer review, we can see the work of other students and find out where they can do better. We can also review what we need to improve by grading our classmates.”
• The platform for conducting the review work is easy to operate (5 of the 26 provided feedback): Some said that the platform was easy to operate, stating that the platform design was clear and required little effort to understand. For example, one of the students stated that “I personally feel that the peer review system is very useful, the user interface clear at a glance, and you can enter the system without logging in. The operation is simple and user-friendly. If you want me to score, I will give full marks.”

The following are the five negative feedback categories:

• The course scores may have low reliability (10 of the 26 provided feedback): The majority expressed concern about the reliability of the scores. Some worried that their peers might put little effort into the ratings, and others said that the scores that they received from their peers varied considerably. For instance, one of the students mentioned, “I think some of the students did not spend sufficient effort reviewing others’ work and might randomly mark it, which would lose the original meaning of the reviewing mechanism.”
• The system operation requires improvement (9 of the 26 provided feedback): Some students said that they received the link but were unable to open the file. One said that he did not receive the link when it was time to rate, another said that it was time-consuming to upload the file onto the system, and another said that he did not click the confirm button and accidentally gave someone a score of zero.
• The course requires improvements regarding the anonymity of the mechanism (4 of the 26 provided feedback): The reviewing mechanism was designed to be anonymous. However, some students put their names or student ID numbers on the file name, which revealed their identities. Some students noticed this and were worried that it might have affected the reliability of the scores (e.g., one might have given a higher score to a friend).
• The rating criteria need to be more specific (1 of the 26 provided feedback): One student suggested that the rating criteria should be more detailed because she received three very varied ratings and thought that it was because the standards were not uniform.
• The review should include comments as well as scores (4 of the 26 provided feedback): The students suggested adding comments to the rating system so that they could write comments alongside the rating. By reading any qualitative comments, they would also be able to understand why they did not receive full marks. For example, one student mentioned, “I ended up with one 0 points. However, I could not see the comments from others. In addition, I hope that the system can [add] the function of inputting short comments, thus we can receive the most sincere suggestions from colleagues.”

6. Benefits

The findings of this research indicate that several benefits lie in the implementation of the peer review system for the teaching of BIM tools:

• The learning goals are clearer. By taking the position of a rater, the students can have a clearer picture of the scope and goals of what they should learn from this course. Additionally, the instructors have a chance to polish their rubric and make it more effective.
• The system can expand the method of learning. The students can learn from each other rather than only from the instructors. Especially for complicated subjects to learn, such as BIM, it is not easy for the instructor to cover every detail. Students can, therefore, learn from others’ good and bad work during the process of rating.
• The system can serve as a way to validate scores. With the instructor as the sole rater of students’ work, the scores awarded could be biased. By considering the students’ scores from the peer review system, the instructor has an additional reference by which to assign scores.
• The feedback can help to improve future courses. The instructor can understand the students’ learning outcomes from the peer review system and adjust future courses based on the students’ performance. In addition, the instructor can ask the students to write anonymous feedback about the system to obtain a better understanding of the students’ needs.

7. Conclusions

This research provides a brand-new style of teaching to enhance BIM tool learning. The peer review system developed in this study contains guidelines on designing a peer review-based course and a web-based platform to conduct peer reviews. The results indicated that this system has fairly good feasibility and effectiveness for evaluations. The scores given by the students were consistent with those of the instructor. In addition, the students had an overall positive experience of the peer review system, which favors their learning outcomes. However, this study has some limitations that need to be improved. For example, the platform needs to be made completely anonymous by unifying the uploaded files and sending the files to assigned students with unified file names. Additionally, the validation section can have a comparison of students’ performance between the semester with the peer review system implemented and the semester without the system implemented for understanding the effectiveness of using the proposed system. Therefore, future studies can develop a more complete peer review platform based on this research and compare the course performance with other semesters or similar courses that use conventional approaches.