**2. Materials and Methods**

#### *2.1. Short History of Interlaboratory Comparisons Organized by Ceprocim*

In 2007, the Romanian laboratory Ceprocim (notified body number 1830), authorized to test in the scope of EN 12004, initiated a project of ILC of the initial adhesion strength of cementitious CTAs. Nine laboratories, mainly Romanian, participated in the first edition of ILC, while twenty-seven laboratories of research institutes and manufacturers of CTAs from the following nine countries participated in the fifth edition: Austria, Bulgaria, Croatia, the Czech Republic, Germany, Poland, Portugal, Romania, and Slovenia [30]. Proficiency tests/interlaboratory comparisons organized by Ceprocim were carried out according to uniform rules and the requirements of EN ISO/IEC 17043 [39]. All laboratories used identical concrete slabs for the tests and the same ceramic tiles provided by the test organizer. According to the study's authors, more than 90% of the test results obtained by the participating laboratories can be described as satisfactory (|z| ≤ 2) according to EN ISO/IEC 17043, and the remaining were questionable or unsatisfactory [30].

In 2014, Ceprocim extended the research to the second characteristic—adhesion strength after water immersion. In 2018, during the tenth jubilee edition of the study, three characteristics were measured: initial adhesion strength, adhesion strength after immersion in water, and open time [34]. The last extension of the scheme took place in 2020, when two more tests were introduced: tensile adhesion strength after heat aging and tensile adhesion strength after the freeze-thaw cycle [40]. Randomly assigning a code number to each laboratory in each edition ensures confidence in the entire study. Reference to each laboratory in all reports is made by code number.

Concrete slabs of various origins were used for the twelve editions of the ILC. Each of the laboratories participating in the eleventh and twelfth editions used their own concrete slabs.

It is also important to note that the laboratories participating in ILCs organized by Ceprocim represent both accredited laboratories according to EN ISO/IEC 17025 [41] and non-accredited laboratories. Twenty-nine laboratories participated in the eleventh edition of the ILC (2019–2020) and twenty-seven in the twelfth edition a year later. Nineteen laboratories participated in both the eleventh and twelfth editions, and the results obtained by these laboratories are discussed later in this paper. The 19 laboratories are from the following countries: Austria—1, Germany—3, Greece—2, Italy—1, Mauritius—1, Poland— 1, Portugal—1, Republic of Moldova—1, Romania—5, Slovenia—1, Spain—1, and United Arab Emirates—1.

#### *2.2. Ceramic Tile Adhesives (CTAs)*

CTAs are an important group of construction products intended to install ceramic cladding for internal and external purposes [42]. Ceramic tiles are commonly used on all continents. In 2020, their production reached 16.093 billion m2, while consumption was slightly lower and amounted to 16.035 billion m2 [43]. Assuming an average consumption of 4 kg CTA per 1 m2 of ceramic cladding, this means a global production of about 65 million tons of CTAs.

Requirements for all CTAs (cementitious, dispersion, and reaction resin) applying to all member states of the EU, three of the EFTA members (Iceland, Norway, and Switzerland), and other states (United Kingdom, North Macedonia, Serbia, and Turkey) specify EN 12004. EN 12004 was first published in 2001, and the last version of the standard published in the list of European harmonized standards [44] is EN 12004:2007+A1:2012 [38]. The next version of the standard published by CEN in 2017, i.e., EN 12004-1:2017 [45], has not yet been included in the list of harmonized standards published in the Official Journal of the European Union and, therefore, cannot be the basis for the assessment and verification of constancy of performance. Based on the EN 12004, the global standard ISO 13007-1 was implemented in 2004 [46]. Establishing the standard specifying requirements, terminology, working methods, and application properties for CTAs for internal and external tile installations on walls and floors by the ISO organization has resulted in their harmonization worldwide. The current ISO 13007-1 standard comes from 2014 [47].

The fundamental issue in ILC is that the tested product (CTA), and other materials used for the tests, i.e., concrete slabs and ceramic tiles, are identical. The organizer provided the CTA and ceramic tiles in the analyzed studies, and each of the participating laboratories provided concrete slabs. Residual CTA determinations were repeatedly performed on a 250 μm sieve to ensure that the CTA was homogeneous. Checking the adhesive homogeneity was performed with the same equipment, by the same operator, during a short period. The sample was considered homogeneous when all the results had been placed in the range: average residue value on the 250 μm sieve ± 2 s (%). The value of s represents the standard deviation of repeatability.

The CTA used in the research was classified as C2E under the requirements of EN 12004 [38]. The initial adhesion strength and adhesion strength after water immersion were determined following the test methods and using auxiliary materials (concrete slab, ceramic tiles) specified in EN 12004 [38]. Finally, it is essential to note that each ILC participant received also written guidelines, in addition to the CTA, for the study and ceramic tiles.

#### *2.3. Evaluation of the Results Using the z-Score*

For the statistical calculation, algorithm A in Annex C from the standard ISO 13528 [48] was used. It implies, for initial adhesion strength, for tensile adhesion strength after heat aging, for tensile adhesion strength after water immersion, for tensile adhesion strength after freeze-thaw cycle, and for open-time after not less than 30 min, calculation of the robust values for average and for standard deviation, from the results obtained of each participant.

Based on an iterative calculation, the calculus of robust average (*x*\*) and the robust standard deviation (*s*\*) were made. The calculation was carried until there was no change from one iteration to the next in the third significant figure in the robust standard deviation and the equivalent figure in the robust average. The value obtained for the robust average after the last iteration represents the assigned value (*xpt*), chosen to be the consensus value.

The standard uncertainty *u*(*xpt*) of the assigned value is given by Equation (1):

$$
\mu(\mathbf{x}\_{pt}) = 1.25 \times \sigma\_{pt} / p \tag{1}
$$

where:

*σpt*—standard deviation for proficiency assessment, *p*—the number of participant laboratories that carried on the test on concrete slab.

The z-score is calculated with Equation (2):

$$\mathbf{x}\_{\text{Zi}} = \mathbf{x}\_{i} - \mathbf{x}\_{\text{pt}} / \sigma\_{\text{pt}} \tag{2}$$

where:

*xi*—the value obtained by each participant for each test,

*xpt*—the assigned value on total participants for each test.

The evaluation of the results was made according to EN ISO/IEC 17043:2010, as follows:


In the z-score calculation program, the assigned value and the robust standard deviation value obtained after the last iteration have been used as they result from calculation without being round.

The interpretation mentioned above of z-score is conventional (see ISO/IEC 17043: 2010 [30], B.4.1.1.). A result that gives 2.0 < |z| < 3.0 is considered to give a warning signal. Participants of the ILC/PT should be advised to check their measurement procedures following warning signals if they indicate an emerging or recurrent problem. The justification for using the limits of 2.0 and 3.0 for z-score is as follows. Measurements that are carried out correctly are assumed to generate results that can be described (after transformation, if necessary) by a normal distribution with mean xpt and standard deviation σpt. z-score, which will then be normally distributed with a mean of zero and a standard deviation of 1.0. Under these circumstances, only about 0.3% of scores would be expected to fall outside the range −3.0 ≤ z ≤ 3.0, and only about 5% would be expected to fall outside the range −2.0 ≤ z ≤ 2.0.

#### **3. Results**

The initial tensile adhesion strength, tensile adhesion strength after water immersion of CTA, and the predominant mode of failure obtained in the eleventh edition of the ILC (2019–2020) and the twelfth edition (2020–2021) are presented in Tables 1 and 2.


**Table 1.** The initial tensile adhesion strength and tensile adhesion strength after water immersion of CTA with the predominant mode of failure obtained by 19 laboratories in the eleventh ed. (2019–2020).

CF-A—cohesive failure within the adhesive, AF-T—adhesion failure between adhesive and tile.

**Table 2.** The initial tensile adhesion strength and tensile adhesion strength after water immersion of CTA with the predominant mode of failure obtained by 19 laboratories in the twelfth ed. (2020–2021).


CF-A—cohesive failure within the adhesive, AF-T—adhesion failure between adhesive and tile, AF-S—adhesion failure between adhesive and substrate.

It is worth adding that two other failure patterns not listed in Tables 1 and 2 are possible, namely, CF-S - cohesive failure in the substrate or CF-T - cohesive failure in the tile.

#### **4. Discussion**

#### *4.1. ILC Results in the Light of ISO 13528 Guidelines*

Table 3 shows the lowest and highest values of the initial tensile adhesion strength and tensile adhesion strength after water immersion of CTA reported by the laboratories participating in the ILC, out of 19 laboratories participating in both the eleventh and twelfth editions, 29 laboratories participating in the eleventh edition (2019–2020), and 27 laboratories participating in the twelfth edition (2020–2021).

**Table 3.** The lowest and highest values of the initial tensile adhesion strength and tensile adhesion strength after water immersion of CTA obtained by 19 laboratories and by all participating laboratories in the eleventh and twelfth editions.


\* Eighteen laboratories reported results for the measurements of the tensile adhesion strength after water immersion.

The guidelines specified in ISO 13528 [48], including the recommendations on the interpretation of proficiency testing data, were applied to analyze the results obtained in the ILC. Table 4 shows the results of the calculations made following ISO 13258 [48].

**Table 4.** The value of statistical parameters calculated following ISO 13258 [48] for measurements of CTA initial tensile adhesion strength and tensile adhesion strength after water immersion during the eleventh and twelfth editions of the ILC.


*x\**—robust average of the results reported by all participating laboratories; *s\**—robust standard deviation of the results reported by all laboratories; *xpt*—assigned value—consensus value; σ*pt*—standard deviation for proficiency assessment; *u*(*xpt*)—standard uncertainty of the assigned value; *V*—coefficient of variation. \* Eighteen laboratories reported results for the measurements of the tensile adhesion strength after water immersion.

The z-score values calculated following the Equation (2) for each laboratory for the initial tensile adhesion strength and tensile adhesion strength after water immersion measurements are presented in Figure 1 and Figure 2, respectively.

**Figure 1.** The z-score value for the initial tensile adhesion strength for each of 19 laboratories participating in the eleventh and twelfth editions of the ILC.

**Figure 2.** The z-score value for the measurement of tensile adhesion strength after water immersion for each of 19 laboratories participating in the eleventh and twelfth editions of the ILC.

The analysis of the z-score shows that in the eleventh ed., 2 out of 19 laboratories in the scope of the initial tensile adhesion strength measurement obtained results classified as "questionable" (2 < |z| < 3), and the remaining 17 laboratories obtained the "satisfactory" status (|z| ≤ 2). A year later, in the twelfth ed. of the ILC, only 1 laboratory was labeled "questionable" based on the results obtained, while the remaining 18 laboratories were "satisfactory". In the case of the tensile adhesion strength after water immersion measurement in the eleventh ed. of the ILC, the results obtained by all laboratories allowed

them to receive the status of "satisfactory", and in the following twelfth ed. of the ILC, 1 of the 19 laboratories received the result referred to as "questionable". The z-score analysis also allows for the indication among of the 19 laboratories participating in both editions, those whose obtained measurement results are the closest assigned value (consensus value), namely laboratories marked with numbers: 1, 7, 9, 10, and 13 and, in further order, 19, 6, 16, and 18. When analyzing the obtained results, it is worth noting that among the results for which the z-score is |z| ≤ 2, these are the results classified as "satisfactory" from the perspective of the measurement laboratory and the analysis of ILC results under the requirements of ISO 13528 [48]. Another dimension of this result is from the product evaluation perspective. Although the z-score meets the condition of |z|≤ 2, for example, in the case of the initial tensile adhesion strength measurement in the eleventh ed., it means both 1.2 N/mm2 or 1.3 N/mm2 (participant code 4, 12, and 15), next to 2.0 N/mm<sup>2</sup> or 2.1 N/mm<sup>2</sup> (laboratories marked as 16 or 17). There is a significant difference between the value of 1.3 and 2.1, which may cause the product to be assessed as failing the acceptance criterion when it is not. The situation can become even more difficult for the product and its manufacturer when the product is reassessed by a market surveillance authority external to the manufacturer. The difficulty level may increase when market surveillance authorities apply the simple acceptance rule that does not consider the variability resulting from MU. Although the obtained results are classified as "satisfactory" in the ILC/PT evaluation categories, differences between the results are significant from the product evaluation perspective, and the product manufacturer cannot ignore this fact in their product evaluation. Producers after the so-called "safe side" must also consider the measurement variability in the value of the acceptance criterion. As shown by the results of ILC studies, this value is significant for CTAs tested under EN 12004 requirements. The analysis of the results of the predominant mode of failure showed more significant differentiation between laboratories. The obtained results are summarized in Table 5.

**Table 5.** The number of the predominant mode of failure for the initial tensile adhesion strength and tensile adhesion strength after water immersion measurements in the eleventh and twelfth ed. of the ILC.


\* Eighteen laboratories reported results for the measurements of the tensile adhesion strength after water immersion in the eleventh edition (2019–2020).

Figure 3 shows a schematic summary of the results of the CTA studies in two subsequent editions of the ILCs. The data presented in Figure 3 combines the data previously included in Figures 1 and 2, plus Table 5. Figure 3 shows no correlation between the laboratory classification using the z-score and the observed failure pattern of tensile adhesion strength, regardless of whether the subject of the measurement was the initial or after immersion in water tensile adhesion strength. The lack of this correlation, and a different distribution of the observed mode of predominant failure, is an additional argument that the interpretation of the initial tensile adhesion strength and tensile adhesion strength after immersion in water results should be approached with caution.

**Figure 3.** Summary of the results of the CTA measurements in two editions of the ILC.

One of the goals of the Romanian Ceprocim project was to show that constant participation in the laboratory proficiency testing programs improves the quality of laboratories work. In this respect, the organizers of the study achieved the intended goal. Analyzing the results of both editions of the ILC/PT with the participation of the same 19 laboratories, it can be concluded that the differences between the individual editions are minor, but they do exist. It leads to the conclusion that laboratories maintain a constant level of the quality of the performed measurements. In addition, this is one dimension of the ILC/PT being conducted.

#### *4.2. Factors Influencing the Measurement of Tensile Adhesion Strength*

From a practical point of view (in risk analysis), the reproducibility of the results, i.e., the degree of agreement between the results obtained by different analysts in different laboratories using a given measurement procedure, is essential for the producers of CTAs. In this regard, while discussing the results of the PT described in this paper, one should also pay attention to several other aspects.

Felixberger [49] described the results of initial adhesion tests of 7 cementitious CTAs, performed in 10 laboratories using 2 different concrete slabs. As the first concrete slab, each of the laboratories participating in the study used a standard concrete slab meeting the requirements of EN 1323:2007 [50]. The second concrete slab was purchased by the research organizer and delivered to all participating laboratories. The standard deviation of the measurement described by Felixberger ranged from 15% to 20%, showing also the influence of concrete slab on the value of the determined adhesion. Felixberger found that for cementitious CTAs with a lower adhesion value, more significant differences between individual measurements exist than in the case of adhesives with higher adhesion. Felixberger formulating the conclusions, stating that the priming of the concrete slab surface for testing would result in the uniformity of the slab surface in terms of its absorption properties. It also creates a situation closer to the actual use where manufacturers of cementitious CTAs recommend using a primer before laying ceramic tiles [49].

It is worth noting here that the subject of the analysis in this study was C2E classified CTA, i.e., with higher adhesion strength values. In the case of the research results analyzed in this article, the concrete slab effect was also present.

Another factor worth noting is the water used to season the samples. The EN 12004 standard does not contain any requirements in this respect. When testing the effect of seasoning water on the adhesion of CTAs, samples were stored in three types of water: in distilled water (pH = 7.09, specific conductivity = 0.040 mS/cm), in tap water (pH = 8.25, specific conductivity equal 0.805 mS/cm), and in softened water (pH = 8.63, specific conductivity equal 1.228 mS/cm) [51]. The tests showed that the origin and type of water used to season the samples significantly affects the adhesion of CTAs. Samples stored in distilled water represent a higher adhesion value than in the two other cases. The differences between the test results were so significant that they decided the fulfillment of the evaluation criteria.

The adhesion of CTAs is determined in the following system: concrete substrate (slab)— CTA—ceramic tile. Some properties of ceramic tiles approved for use during adhesion tests are specified in the standards referred to for a given test method in EN 12004. Niziurska assessed the influence of ceramic tiles' chemical composition and surface structure on the adhesion of CTAs to tiles [52]. The results obtained in the tests confirmed the impact of the quality of auxiliary materials (ceramic tiles) used in the tests on the compliance of CTAs with evaluation criteria.

It is also worth noting that the result of the adhesion measurement (regardless of whether the initial tensile adhesion strength or the tensile adhesion strength after water immersion) is the result of two other measures. It means the maximum load of the sample during the tensile strength measurement and the sample surface (adhesion surface). Therefore, the accuracy of the destructive force determination and the accuracy of the sample cross-sectional area determination influence the accuracy of the final result (adhesion).

Considering the obtained results and additional conditions related to the tensile adhesion strength of CTAs measurements described above, the manufacturer designing the product during the risk analysis determining the value of the acceptance criteria has to consider the simultaneous occurrence of all possible variations that may accompany the measurement.
