Constructs for Assessing Integrated Reports—Testing the Predictive Validity of a Taxonomy for Organization Size, Industry, and Performance

Abstract

The lack of an agreed valid measurement of integrated reporting (<IR>) among organizations poses a constant problem to empirical researchers. Lueg and Lueg proposed an <IR> score that rates and categorizes reports according to their compliance with the principles of <IR>. This study tested and validated their proposed coding catalogue, constructs, and taxonomy using capital market data and multivariate statistics. These findings suggested that this <IR> score and the corresponding taxonomy for classifying reports has a high predictive validity and can be used by future researchers to measure <IR>. In particular, this <IR> score confirmed that integrated thinking reports tend to be published by large organizations in controversial industries with an above-average performance (Adj. R² = 42.2%). The findings also suggested that the construct form of the integrated report contributes more explanatory power than the construct content. In this regard, the results indicated that only full implementers of <IR> show associations with performance, unlike organizations that partially comply with the <IR> principles.

1. Introduction

Researchers are increasingly interest in understanding the antecedents and consequences of integrated reporting (<IR>) and integrated reports [1,2,3]. <IR> is “a process founded on integrated thinking that results in a periodic integrated report by an organization about value creation over time and related communications regarding aspects of value creation”, and an integrated report constitutes “a concise communication about how an organization’s strategy, governance, performance and prospects, in the context of its external environment, lead to the creation of value in the short, medium and long term” [4]. <IR> can be employed in research only if it is assessed by a measurable construct. The development of an <IR> construct is difficult, however, as <IR> has been designed as a principle-based framework and not as an easily measurable set of indicators [5,6]. Recently, some advancements have taken place in this field. For instance, Pistoni et al. [7] developed a construct for measuring the quality of <IR>. Lueg and Lueg [8] provided an entire taxonomy on <IR> with an associated coding catalogue. Their taxonomy enables researchers to measure the form and content of <IR>, as well as an overall <IR> score, and to form distinct groups of reports (traditional reports; enhanced content reports; integrated thinking reports).

However, these recently developed constructs need to demonstrate their predictive validity. Suggested constructs need to be able to replicate well-known theoretical relationships in statistical models with actual data from organizations [9]. Only after passing these tests for predictive validity can they be further used in research to test or explore theoretical relationships that have not been tested before [10]. In this study, I surveyed the construct of Lueg and Lueg [8]. I posed the research question:

Does the <IR> score of Lueg and Lueg [8] have predictive validity?

In order to test for predictive validity, I used a measure of the <IR> score from Lueg and Lueg [8]. Subsequently, I performed statistical analyses such as an OLS regression to test for well-known relationships. The findings supported the predictive validity of the <IR> score of Lueg and Lueg [8]. I replicated well-established relationships between organizational characteristics and pre-<IR> sustainability reports: integrated thinking reports tend to be published by large organizations in controversial industries with an above-average accounting/market-based performance. This implied that both organizations and future research in this field may use this <IR> score to assess the status of integration in organizational reporting.

2. Theoretical Background and Hypotheses

2.1. Integrated Reporting, Integrated Reports and Integrated Thinking

<IR> is directly and indirectly being developed by international organizations. The International Integrated Reporting Committee (IIRC) is the original and lead developer of <IR>. It cooperates with and receives support from other organizations, such as the Financial Accounting Standards Board (FASB), Global Reporting Initiative (GRI), International Accounting Standards Board (IASB), International Federation of Accountants (IFAC), Sustainability Accounting Standards Board (SASB), Prince of Wales’ Accounting for Sustainability Project (A4S), and United Nations Global Compact (UNGC). For instance, the A4S and GRI supported the foundation of the IIRC in 2010. Many of the guidelines of the named organizations have become part of the basis of or an inspiration for <IR>. In 2011, the IIRC published the discussion paper “Towards Integrated Reporting—Communicating Value in the 21st Century” to receive feedback on the theoretical foundation of <IR>, as well as on the feasibility of transferring the concept into practice. In total, 140 organizations from 26 countries took part in <IR> pilot projects. In 2013, these collaborations enabled the IIRC to present its <IR> framework that suggested content and form for integrated reports [11].

Despite the name framework, <IR> is a principles-based approach [5,6]. It neither prescribes steps for implementation, nor does it define a list of easily measurable outputs or processes. The <IR> framework intends to evaluate neither the quality of organizational strategy nor the sustainable outputs [12,13]. Instead, it aims to strike a balance between comparable reporting across organizations and the flexibility of the individual organization to report on its specific business model in one or more reports [14,15,16]. However, certain requirements for both the content and form of published reports must be met in order for them to be considered integrated. For instance, it is well-established that simply combining the annual report and the CSR report, or adding some sustainability intentions to the annual report, is not considered integrated reporting [17].

In particular, an integrated report has to demonstrate integrated thinking [18]. Integrated thinking exists if a report “takes into account the connectivity and interdependencies between the range of factors that affect an organization’s ability to create value over time” [19] (p. 2), [20]. Organizations may use more than one report for integrated reporting, provided these reports meticulously refer to each other and the external stakeholders [21] (p. 11f), [22]. Likewise, they may include further sources to demonstrate their integrated thinking, such as investor/stakeholder relationships, social media, and press releases [20], [21] (p. 11f). Eventually, <IR> should support organizations in attracting funding from equity and debtholders.

The understanding of what <IR> should look like in practice is not unanimous [12,13]. Thus, several approaches exist to measuring <IR> for research purposes. While there are some seminal examples for measuring <IR> quality [7,23,24], only one study took a broad view of both the content and form of integrated reports and applies a theory-based taxonomy to develop reporting types [25]. Thus, I used Lueg and Lueg’s [8] recently established taxonomy of integrated reporting to classify reports (hereafter referred to as ‘the taxonomy’). The taxonomy is more than simply assigning scores (such as a rating or framework) to annual reports; rather, it classifies reporters and their reports into distinct groups. To do so, the coding catalogue mainly draws on the two constructs of content (what the reports contain) and form (how the information is conveyed). Lueg and Lueg [26] considered content a necessary condition that sets enhanced content reports apart from traditional reports. They also suggested that form—and in particular the subconstruct of external assurance of reports—is a sufficient condition that distinguishes integrated thinking reports from enhanced content reports. Thereby, form explains most of the subsequent classification into report groups. Figure 1 reproduces the framework of Lueg and Lueg [8].

The coding catalogue of the taxonomy attributes between 0 and 10 points of <IR> sophistication to annual reports. Up to two points can be assigned based on the content of the report, and up to eight points can be assigned based on the form. Lueg and Lueg [8] classified the reports into three groups: traditional reports (less than three points) elaborate mainly on financial information; enhanced content reports (minimum three points but less than seven points) disclose more content but largely lack integrated thinking (form); and, integrated thinking reports (minimum seven points) score highly for both content and form and, can be published as a single report or as separate reports that reference each other. Researchers can assign values to the reports through content analysis, which is a widely-recognized method in research on <IR> [17,27]. Lueg and Lueg [8] suggested that the signifiers (i.e., the bases for assigning scores) should be generously coded and without detailed coding rules because organizations may use various or even ambivalent terms for the same phenomenon. For instance, organizations might report on resources or focus areas instead of using the official <IR> term capital.

2.2. Hypotheses

I tested the predictive validity of Lueg and Lueg’s [8] taxonomy. I checked whether the well-established relationships between the organizational variables and sustainability reporting (often based on pre-<IR> studies) would still exist if I used this new taxonomy [25,28]. This revealed whether the taxonomy has an acceptable predictive validity for future research.

2.2.1. Size: Organizations Issuing Integrated Thinking Reports Tend to Be Relatively Large

Previous studies have shown a positive relationship between the organizational size and the adoption/sophistication of sustainability reporting [27,29]. Both CSR implementation [30] and CSR reporting [31] tend to be higher among large organizations. Studies have tended to apply two standard arguments for this phenomenon. First, large organizations tend to be more visible and have larger impacts on society, so stakeholders pressure them toward more comprehensive reporting [32,33]. Thereby, the costs of voluntary sustainability disclosure might exceed the benefits for smaller organizations—due to the generally smaller group of stakeholders they serve [34]. On another note, small capital organizations might understand that simply ‘filing’ voluntary disclosures might be critically received [35,36]. Second, larger organizations have more resources in absolute terms, which decreases the marginal cost of CSR reporting [31]. However, Läger et al. [37] questioned these two arguments, pointing out that visibility and size are not synonyms and identifying that large organizations do not always have a consistently higher slack they can use for sustainability reporting. Instead, Läger et al. [37] mobilized a third argument: the mutual relationship of organizational complexity and environmental complexity [38,39]. Building on Blau [40], Läger et al. [37] theorized how different forms of complexity would affect corporate sustainability. They demonstrated that size, as a measure of complexity, is often related to the degree of corporate sustainability; see also [41]. There are only a few exceptions, such as the international presence of an organization (spatial differentiation): if organizations intend to circumvent the high standards for corporate sustainability in their country of listing, they might choose to internationalize offshore operations. In this case, complexity might be negatively related to corporate sustainability. Yet overall, these findings led me to propose:

H1. 

Large organizations espouse more integrated thinking (i.e., sophisticated integrated reports).

2.2.2. Industry: Integrated Thinking Reports Often Appear in Controversial Industries

Organizations operating in controversial industries, such as manufacturing, supposedly owe transparency and accountability to the public, for instance due to generally higher greenhouse gas emissions or production practices in non-EU offshore locations [42,43]. They are pressured by shareholders toward sustainability reporting [44,45]. Similarly, healthcare organizations often have a wide range of stakeholders in addition to their shareholders (patients, payers, and providers). They are often subject to political debate and public scrutiny in terms of quality of care and monetary efficiency [46,47]. Conversely, the service and finance industries tend to belong to the ecologically cleaner industries—provided their debtors are not included as part of their business models [48,49,50], while the business model of financial organizations can often be expressed in financial terms, which lowers the need for voluntary disclosure of ecological and social issues [51,52,53]. I proposed:

H2. 

Organizations in controversial industries espouse more integrated thinking (i.e., sophisticated integrated reports).

2.2.3. Financial Performance: Integrated Thinking Reports Are Associated with Better Performance

Different theoretical perspectives purport that voluntary disclosure improves performance. Institutional theory suggests that a higher legitimacy of an organization (e.g., through <IR>) increases the number of customers and their willingness to pay for products and services [35,54]. This should increase accounting-based performance [55,56]. Higher profitability may then translate to higher stock market valuation [44]. Beyond this, signaling and principal–agent theory suggest that high transparency (e.g., through <IR>) eliminates the uncertainty discounts and improves the market performance [57,58,59].

Other theoretical perspectives suggest that performance precedes voluntary disclosure. Organizations with strong (expected) market performance use voluntary disclosure to distinguish themselves [60,61]. They might also adopt reporting fashions by chance, either as an experiment to find out if they contribute to performance, or simply to appear legitimate in case stakeholders start to care at some point in the future [62]. Financially sound organizations can also more easily bear the cost of such voluntary disclosure and associated activities [63,64,65].

Therefore, there is no unique causal explanation of how voluntary disclosure relates to financial performance. A few studies have not found any significant relationship [27] (p. 28), [55], but, on closer inspection, most of said studies did not theorize why the relationship was not significant. However, the coefficients are mostly positive [27,32,42,66]. So, there is at least a substantial consensus in the literature that sustainability activities/reporting and (mostly market) performance appear to have a positive relationship [20].

To test the predictive validity of the taxonomy, my goal was to demonstrate some statistically significant concurrence (viz. without inferring causality) of performance and the taxonomy. I tested for KPIs that were promoted in the Highlights sections of most of the annual reports to follow the organization’s definition of performance. These are return on assets (ROA), return on equity (ROE), return on capital employed (ROCE), and the market-to-book (MtB) ratio. I chose KPIs for all reporting years from 2013 to 2015 to accommodate both schools of disclosure-performance theories. I proposed:

H3a. 

Integrated thinking (i.e., sophisticated integrated reports) is associated with a higher return on assets (ROA).

H3b. 

Integrated thinking (i.e., sophisticated integrated reports) is associated with a higher return on equity (ROE).

H3c. 

Integrated thinking (i.e., sophisticated integrated reports) is associated with a higher return on capital employed (ROCE).

H3d. 

Integrated thinking (i.e., sophisticated integrated reports) is associated with a higher market-to-book (MtB) ratio.

3. Methodology

To ensure alignment between the original development of the taxonomy and the subsequent tests of this study, I used Lueg and Lueg’s [8] original data. Their original sample consisted of 148 listed organizations from Nasdaq OMX Copenhagen [67]. They removed eight organizations that had foreign International Securities Identification Numbers, which indicated that these organizations were not necessarily subject to Danish regulation. They also removed 9 organizations that intended to delist soon and 3 organizations with incomplete data due to recent listing. The final sample consisted of 128 listed organizations. The sample is of interest because the Danish Financial Statement Act (section 99a) mandates organizations listed in Denmark to provide sustainability reporting on a comply-or-explain basis [68].

Their annual reports were from the year 2014, where some reporting periods included smaller parts of the years 2013 or 2015. The evaluated, consolidated reports were pdfs from the organizations’ websites. Interactive web reporting provided insights for corroboration purposes only. All reports were independently analyzed by two coders of the author team. Their inter-rater reliability ranged between 93% and 100%. Ratings were corroborated through the analysis of further sources such as the investor relationship section on the organizations’ websites [8]. The rest of the reported data on the organizations (e.g., size and performance metrics) stemmed from the rating agency, Morningstar. The statistical methods applied in this study were mainly descriptive frequencies, cross-tables, T-tests, and univariate OLS regression.

4. Findings

4.1. Descriptive Statistics

Table 1. Descriptive statistics of <IR>-types.

<IR>-Scores	Integrated Thinking Reports	Enhanced Content Reports	Traditional Reports	ALL
Number of organizations	21	73	34	128
in %	16.4%	57.0%	26.6%	100.0%
Actual range	7 to 10	3 to 6.5	0.5 to 2.5	0.5 to 10
Average	8.190	4.411	1.868	4.4
Median	8.0	4.0	2.0	4.0
Standard deviation	1.220	1.111	0.657	2.272
Standard error	0.266	0.130	0.113	0.201

shows the descriptive statistics of the reporter types. Of the 128 organizations investigated, 21 (16.4%) were traditional reporters, 73 (57.0%) were enhanced content reporters, and 34 (26.6%) were integrated thinking reporters.

4.2. Confirmatory Analyses

H1: My analyses supported the predictive validity of the <IR>-taxonomy because integrated thinking reports were associated with the size of organizations (Figure 2). Using the NasdaqOMXNordic classification of large, middle, and small market capitalization, 67% of all integrated thinking reports belonged to large-cap organizations. As expected, based on previous studies, traditional reports tended to be small cap organizations. Thereby, H1 was confirmed.

H2: My analyses supported the predictive validity of the <IR>-taxonomy because integrated thinking reports were associated with controversial industries (Figure 3). Organizations in manufacturing and health care often published integrated thinking reports (43% and 19%, respectively). Traditional reporters were associated with the finance industry, as well as with consumer services (38% and 24%, respectively). Thereby, H2 was confirmed.

H3: My analyses supported the predictive validity of the <IR>-taxonomy because integrated thinking reports were associated with better performance (Figure 4). The <IR>-score correlated with the MtB ratios across all years. The <IR>-score did not correlate with any of the accounting-based performance measures, except for ROA in 2013, which is similar to a previously reported result [58]. This nonrelationship could be rooted in the endogenous nature of accounting numbers. Moreover, Clarkson et al. [42] argued that there might be no measurable relationship because organizations also report performance through timelier channels than annual reports. Additionally, weak performance is not improved by disclosing more information about it [69].

I also investigated the two criteria, form and content, separately. The criterion content did not exhibit any correlations with performance. However, the criterion form positively correlated with all MtB ratios over the years. In addition, form, on its own, showed more variance in the accounting measures than the <IR>-score. This suggested that the predictive validity of the tangible criterion content was somewhat lower, viz. what is being reported by the metrics of capital, organization, strategy, and outlook. This was expected because almost all organizations scored very high in the content criterion. Conversely, the more intangible criterion, form, had a strong predictive validity, viz. how organizations report on value creation and the basis for disclosure, the connectivity of the information, and whether the additional information was subject to external assurance [17] (p. 106), [70,71]. In the literature, it was argued that poor environmental performers would try to mimic the reporting practices of strong performers but only to a degree that is achievable without much effort [27]. Given that the only related KPIs are not accounting- but market-based, a provisional explanation might be that the market participants show an ability to tell the integrated thinking form apart from content mimicry. This implies that the criterion form especially carries value for future research. Thereby, H3 was confirmed.

Altogether, the findings in this subsection favored of the predictive validity of the taxonomy. The summarizing regression analysis in

Table 2. Regression of <IR>-score.

Independent Variable: <IR>-Score
	Std. Beta Coefficients		VIF
Size	0.583	***	1.078
	(0.202)
Industry	0.178	**	1.087
	(0.178)
TSR2014	−0.123		1.009
	0.003
Constant	1.030	**
	(0.427)
Adjusted R2	0.422	***
D.f.	124

** p < 0.01; *** p < 0.001 (two-tailed). VIFs, variance inflation factors. Values < 5 signify the absence of multicollinearity. Dependent variable: <IR>-score is the combined score of the criteria content (min: 0; max: 2) and form (min: 0; max: 8), and describes the sophistication of a report according to these <IR>-criteria. Independent variables: Size is an ordinal variable following the NasdaqOMXNordic classification of large (coded 3), middle (coded 2) ,and small (coded 1) market capitalization. Industry is an ordinal variable that is coded 3 for controversial industries (manufacturing and health care), 1 for noncontroversial industries (services and finance), and 2 otherwise. TSR2014 is a scale measuring the total return to shareholders in 2014.

shows that size and controversial industries explained 42.2% of the variance of the <IR>-score (p < 0.000). Because I argue that the performance indicators discussed above are outcomes, the model cannot use performance as a predictor of the <IR> score. Therefore, I added only one other performance indicator (total shareholder return from 2014 (TSR)) as a control variable. The regression model showed, as expected, that performance was not a significant predictor. Because it was not significant, the direction of the sign of the coefficient is irrelevant for interpretation. The insignificant coefficient may also be explained by previous findings e.g., [71], that, for some organizations, the cost of reporting exceeds the benefits.

4.3. Exploratory Analyses

Lastly, I tested for differences in the means across the three identified report types.

Table 3. Differences in performance by report type.

	PANEL A		PANEL B
	Traditional vs. Enhanced Content Reports		Integrated Thinking vs. All Other Reports
	Difference in Mean	p-Value	Difference in Mean	p-Value
ROA2013	−0.001	0.984	0.085	0.015	*
ROA2014	0.089	0.127	0.036	0.432
ROA2015	−0.006	0.857	0.070	0.044	*
ROE2013	0.025	0.729	0.189	0.010	**
ROE2014	0.081	0.233	0.148	0.040	*
ROE2015	0.136	0.167	0.133	0.199
ROCE2013	0.024	0.626	0.124	0.018	*
ROCE2014	0.139	0.154	0.067	0.408
ROCE2015	−0.002	0.966	0.131	0.028	*
MtB2013	−0.007	0.300	0.023	0.031	*
MtB2014	−0.004	0.487	0.025	0.034	*
MtB2015	−0.005	0.657	0.034	0.058	(*)

**, *, (*) Correlations are significant at the 0.01, 0.05, and 0.1 levels, respectively (2-tailed). ROA: Return on assets; ROE: return on equity; ROCE: return on capital employed; MtB: market-to-book ratio. Numbers after the ratios signal years. For robustness, I also conducted two individual sets of t-tests between integrated thinking reports and the other two groups. These results conveyed a consistently similar picture. Among the 21 integrated thinking reports, I performed a T-test between the 10 reports that showed full integrated thinking, and the 11 reports with selected integrated thinking. This test did not show any differences that were significant at p < 0.05.

(Panel A) shows that traditional reports did not differ statistically from enhanced content reports, which often have extensive sustainability reports. Contrary to this, Panel B shows that integrated thinking reports were associated with significantly higher MtB ratios, ROA, ROE, and ROCE than the other reports. The means that integrated thinking reports had accounting returns that were between (statistically significant) 7.0% and 18.9% higher than the rest of the reports. This was accompanied by the fact that the mean integrated thinking reports had an MtB valuation of 2.3% to 3.4% higher than the other reports. For future research, this implies that integrated thinking reports are quite different from the rather similar traditional reports and enhanced content reports. Altogether, the findings in this subsection support the predictive validity of the taxonomy.

5. Concluding Discussion

5.1. Conceptual Contributions

The study provided a useful tool for future research by demonstrating the predictive validity of the <IR>-taxonomy [25]. I was able to replicate previous findings on general sustainability reporting. I showed that Lueg and Lueg’s [8] <IR> score (and especially the criterion form) has a statistical association with an organization’s size, industry, and performance. Moreover, my results from the sustainability-friendly Danish capital market serve as a benchmark for comparative macro-level studies [72]. For European settings such as this, the EU Sustainability Reporting Standards (ESRS) will likely add further aspects to both the criteria, content and form, such as sector-specific standards for sustainability reporting. The ESRS support the Corporate Sustainability Reporting Directive (CSRD) of the EU, and its developer, the European Financial Reporting Advisory Group (EFRAG), has put a draft of the ESRS up for public consultation.

5.2. Implications for Practice

My findings offer assurance to external users of annual reports (especially shareholders, policy makers, and students of sustainable accounting) that Lueg and Lueg [8] already offered a valid taxonomy of <IR> reports. In particular, the measurements of content and form offer a nuanced angle on what constitutes integrated thinking, rather than just checking whether the integrated report is published as one single document [21,71,73].

Organizations can profit from this taxonomy as it outlines the external expectations of <IR>. Attempting to comply with the taxonomy might spark prolific discussions among managers on the direction of strategy and governance, and how they can execute them with management control systems [2,38]. The taxonomy also prompts what to emphasize in the report, such as o improving analyst coverage, attracting long-term investors, or optimizing risk [74,75,76].

5.3. Limitations and Future Research

I set out to test the constructs proposed by Lueg and Lueg [8]. To achieve this goal, I had to test limited, already established relationships with only basic statistics [25,28]. I used the same data for the tests as the original construct development to ensure alignment of the original development of the taxonomy. Future research should additionally attempt to replicate these constructs using out-of-sample tests and alternative statistical methods, preferably taken from settings other than Nordic countries. In this regard, future studies might also employ computer-aided text analysis (CATA) instead of human coders [10,77]. In the interest of identifying the most suitable constructs for <IR> in research, studies should compare validity and reliability criteria of this taxonomy with alternatives, such as those of Pistoni et al. [7] or Landau et al. [71], and explain which construct should be used under which circumstances.

Beyond these initial tests, future research may extend the constructs for this taxonomy. For instance, the findings suggested a low importance of content; however, extant research generally covered content through the highly popular ratings on environmental, social, and governance aspects (ESG ratings), while form has been given little consideration. Current research also suggests that organizations have particular problems in measuring and managing (and thus reporting on) social aspects [78,79]. A further refinement of the content construct might shed light on this issue.

Lastly, future research may use the <IR> taxonomy to test whether known relationships from the general corporate sustainability research also hold for <IR> in particular. This includes tests for performance, especially for nonlinearity [73,80]; bivariate effects [69,81]; the role of top managers in <IR> implementations [82]; and the linkage to compensation [83]. The taxonomy may also be used in event studies to investigate gaps between reporting and actual practices, such as when previously praised reporters are associated with cases of misconduct [84,85]. In summary, the <IR> score and Lueg and Lueg’s [8] corresponding taxonomy for classifying reports has a high predictive validity and can be used by future researchers to measure <IR>.