**4. Discussion**

As noted in the Introduction, this paper has built upon the work of the Carnegie Initiative on the Doctorate to describe a general curricular framework applicable across disciplines and throughout higher education to promote the explicit formation of stewardship. This Mastery Rubric for Stewardship (MR-S) describes how scholars, professionals, and practitioners, whether inside and outside the academy, can develop and document the characteristics (KSAs) of a steward of the discipline. The three case analyses show considerable—but varying—alignment of the KSAs for stewardship and the professional/ethical practice guidelines and standards for these three disciplines/professions. The di fferent features of stewardship identified as KSAs can thus be seen to be achievable by *all* practitioners in these diverse fields, and are not limited to those who have completed doctoral training, thus accomplishing the goal of expanding the construct of stewardship beyond the scholar first and foremost. Stewardship can therefore be introduced earlier and more widely—to a far wider audience—than was originally envisioned, and the MR-S describes how performance at each of four stages can be concretely observed—and elicited—in the developing steward. The alignment of each of the KSAs with at least some of the professional practice guideline principles across diverse disciplines suggests that stewardship can be learned and exercised by the full range of professionals across fields as diverse as those we analyzed here. Moreover, focus on the stewardship KSAs during professional/pre-professional training is aligned with these professional practice guidelines.

The purpose of the paper was to broaden the definition of "stewardship" so that it could apply to professionals and practitioners as well as scholars, and to describe a developmental trajectory that can be initiated earlier than doctoral education and at any point in a career so that the broadening could be concretely described. The MR-S was iteratively developed by articulating the KSAs and drafting/revising the PLDs once the stages were identified. During this process, as can be seen in Table 1 (the MR-S):


## *4.1. Development of the Stewardly KSAs*

The CID's construct of stewardship focused exclusively on doctoral education, and we have agreed that stewardship is most fully and explicitly formed at this level. This is particularly true for the traditional academic disciplines, as well as those where highly specialized knowledge is required to create and critique new ideas. Nevertheless, the MR-S shows that, and how the characteristics and commitments of professional and disciplinary stewardship can be fostered earlier than the doctoral level. Since many professionals engage in disciplinary practice without pursuing the terminal degree, instilling stewardly attitudes at the undergraduate and masters' level would benefit the discipline overall. For example, it has been argued [36] that undergraduate statistics and data science majors can be oriented to the importance of stewardship generally, even if their engagemen<sup>t</sup> with the discipline or profession entails neither producing nor critically consuming scientific argumentation. Fostering similar attitudes in other disciplines would benefit not only the disciplines individually but also society more broadly. Importantly, although the first KSA, "requisite knowledge", encourages the

steward-in-training to explore "professional practice standards", these may not exist, may be out of date, or may simply be too specific to the particular profession (e.g., historian, statistician, or neuroscientist) for a modern professional (who may do historical analysis one day, statistical analysis another day). This KSA does not require the professional to rely, or rely solely, on one set of standards but rather, to be aware of those that exist and their relevance in practice.

Since the performance level descriptors of the MR-S are based on Bloom's taxonomy, the Rubric *can* be used with students and professionals across levels. Indeed, since the highest levels of Bloom's taxonomy include "evaluation" and "synthesis", two key characteristics of stewards, all levels of higher education that develop these higher-order cognitive abilities would be able to promote stewardship without dramatically altering coursework or assessment. However, without attention to the growth and development of the characteristic KSAs that define the steward, it is unlikely that higher education can prepare all practitioners to be those to whom the integrity of their respective fields can be confidently entrusted. The MR-S can be used in continuing education and other standard training initiatives across workplaces, if that responsibility is taken up by employers; it can also be used by the self-directed learner to demonstrate their intention and commitment to be stewardly.

In addition to its support of the plausibility of integrating stewardship earlier in education than doctoral level training, the MR-S also represents a curricular structure that is general and flexible enough to be applied across disciplines and institutions. The results of the degrees of freedom analyses showed considerable alignment between the KSAs of stewardship and the professional practice standards from three diverse fields. This alignment suggests that the framework is applicable across many areas of study. In particular, curricula that incorporate disciplinary guidelines can use stewardship to underpin e fforts to ensure a developing engagemen<sup>t</sup> with the discipline and document the achievement of these pre-professional behaviors. If stewardship were adopted as part of "general training" or general education across a college or department, all students would gain experience in their roles as future stewards of their profession or field. Implementation research can be easily envisioned, with "percent documenting journeyman-level performance" using a portfolio approach, as an outcome to be compared across cohorts from di fferent disciplines or training programs.

As such, the MR-S has the potential to enhance the goals of professional and academic societies and to promote the explicit integration of stewardship at any level or career stage. Many other disciplines (but not all) have published codes of professional conduct. These codes represent the profession and describe how professionals in the discipline can and should engage in their craft (see, e.g., [36–38]). While these disciplinary guidelines and codes all serve to promote the integrity of their respective fields, they are essentially focused on the individual practitioner. The concept of a disciplinary steward may be implied in these codes, but it is never made explicit. A formal consideration of how to promote stewardship in a discipline through the discipline's own model of professional behavior would therefore support the explicit incorporation of both stewardship and its development into the process of training the next generations of scholars *and* professionals. The incorporation of the developmental path to journeyman or Master level stewardship can also facilitate and promote engagemen<sup>t</sup> with disciplinary guidelines from early in training. Professional associations have an interest in attracting new members, but the professions themselves have a vital interest in inculcating new members of the profession with the habits of mind that are necessary to promote successful engagemen<sup>t</sup> in the domain and between domains (in multidisciplinary work) or between their domain and the public and other stakeholders. Any individual, regardless of where they are in their career, can begin to curate evidence that they possess the KSAs at each stage of their development (e.g., [15]). While we hope the stewardship construct will be taught and practiced more universally in doctoral education, the MR-S can also serve disciplines and professions as they train practitioners prior to and within doctoral programs.

## *4.2. Documenting Teaching and Learning*

The MR-S's potential to serve as a general curricular framework lies both in its usefulness in developing scholars, professionals, and practitioners, and its utility for documenting their development. In this way, the MR-S, as with other Mastery Rubrics, is consistent with recent calls to better document and more e ffectively communicate learning outcomes in higher education. In 2016, the National Institute for Learning Outcomes Assessment (NILOA) published guidelines for learning outcomes targeting undergraduate education [39]. In 2017, the Council of Graduate Schools (CGS) published the findings of a study of the use of learning outcomes in doctoral education [40]. The 2017 report noted widespread use of learning outcomes among CGS members, as well as growing interest of accrediting bodies in documenting and assessing these outcomes [40] (p. 2). CGS defined a doctoral degree framework as "as set of reference points that defines general skills and competencies of all doctoral recipients" [40] (p. 9). The MR-S (like all Mastery Rubrics) begins with KSAs, and not skills or competencies, but also provides guidance about how "general skills and competencies", if they exist, can be developed in learners [41]. Thus, the MR-S, like other Mastery Rubrics (see [41]) fits and possibly expands on this CGS description. New research to demonstrate how the MR-S meets, and supports meeting, these guidelines, and particularly how implementation of the MR-S in training/education across disciplines can meet the new recommendations, are possible directions for studies of the MR-S and its utility.

Since a key attribute of any Mastery Rubric is flexibility with respect to the source of evidence used to support a claim of achieving a given KSA, the MR-S is well suited for the formal documentation and evaluation of professional development in a variety of settings. Just as a teaching portfolio holds evidence curated to document an individual's attitudes and growth relating to the specific features of education, a "stewardship portfolio" could be curated to document one's attitudes and growth related to their discipline. Those who document their performance of a given (target) KSA, with evidence that is appropriate to their field, would be recognized as achieving that stage. Those who compile this evidence at the journeyman level are designated a "Steward" of their discipline. Those who further meet the Master-level descriptors have demonstrated their expertise and experience in actively and successfully forming earlier-stage stewards. A portfolio approach to the documentation of achievements in stewardship would require the identification and qualification of cohorts of Master level stewards, and a portfolio-based training and assessment program based on the MR-S would support the development and recognition of these cohorts. The potential for faculty development and the strengthening of teaching portfolios could also support faculty buy-in to efforts to integrate the MR-S throughout a curriculum [42]. The stewardship KSAs are opportunities, or suggestions for how to find or create opportunities, to demonstrate accomplishments as well as learning plans. Independent (preferably Master-level) evaluators should agree that evidence supports claims about performance stage achievements (of any KSA) in stewardship. The flexibility in the MR-S arises from the types or sources of evidence that an individual can use to justify claims of achievement, and *portfolios* support this flexibility.

## *4.3. Limitations of this Project*

Weaknesses must be acknowledged in the development of the MR-S and particularly in our validation analyses. The primary consideration is that this project was inherently multidisciplinary but does not capture a consensus from any of the three case study disciplines on the alignment between the disciplinary standards and the KSAs of stewardship. This is particularly true for history, since the guidelines themselves were interpreted (by CMR) before they were aligned with the KSAs. We hope and encourage further exploration of the applicability of the MR-S KSAs and their developmental trajectories within these and other disciplines, so that consensus on alignment, as well as dissemination and endorsement of an emphasis on cultivating stewardship, can be developed. In addition, while the MR-S meets or addresses the NILOA and some of the CGS criteria, it is silent on two specific areas of concern regarding the doctoral degree: assessing the need for and quality of doctoral degrees; and improving public understanding of the value of the doctorate. However, by design, the MR-S would specifically increase transparency and understanding for students of the implicit expectations of degree programs, albeit with respect solely to standards of practice. The MR-S is intended to be used to create a "contract" between the instructor and the learner—making program requirements more student-centered by engaging the student in the identification of opportunities to learn, grow, or demonstrate each KSA at the desired level. It can also support better alignment of training with career paths, particularly with respect to standards of practice along any selected path.

A second consideration is that we did not observe perfect alignment with Guidelines/Standards and the KSAs of stewardship. However, since stewardship was proposed wholly independently of the three sets of practice standards we analyzed, finding any alignment at all tends to support our assertion that stewardship KSAs can support professional practice whether or not the individual will be a "scholar first and foremost", or has/will obtain a PhD. Since not all doctorate holders will be "scholars first and foremost" (many go into industry and government, where scholarship is not a principal priority), at least some divergence from the MR-S KSAs and the professional guidelines is expected. Moreover, not every practitioner or professional can identify with a single set of professional guidelines, and some professions do not have practice standards. The MR-S can help *all* to meet the definition of the steward: "one to whom the integrity and vigor of the discipline can be entrusted". Our alignment results sugges<sup>t</sup> that the stewardship KSAs can support the development of professionalism across a wide range of disciplines, suggesting opportunities for both training and research to test this hypothesis.

The DoFA matrix results do di ffer by discipline, and each shows some gaps. Some attributes of stewardship are more consistent with some disciplinary guidelines than others. For example, the guidelines for Neuroscience (2010) [35] are highly focused on publication and the creation of new knowledge (i.e., "scholarship"), while the guidelines for Statistics (2018) [32] are mainly focused on data and decision-making. The gaps in the alignment of stewardship with disciplinary guideline principles suggests that disciplinary stewardship is *not redundant with,* and is actually supportive of, the professional ideals for conduct for these disciplines. From this observation, we conclude that stewardship is consistent with the core ideals of these professions. Critically, none of these three disciplines systematically inculcates their professional guidelines into doctoral or pre-doctoral training. Because membership in each disciplinary organization is *optional*, their practice guidelines, while consensus-based and reflective of ethical and professional conduct by all practitioners, cannot be mandated. If educational programs integrate stewardship into their completion requirements, then even superficial engagemen<sup>t</sup> with some professional guidelines, and with stewardship, *can* be required by institutions that accomplish this integration.
