1. Introduction
Information technologies are reshaping the competition in enterprises and are leading to a global knowledge society [
1]. In order to achieve a competitive advantage, organizations need to be prepared for and regularly updated on future trends and applications with the help of business intelligence (BI) tools [
2]. In this vein, the use of machine-learning techniques is becoming irreplaceable in knowledge management [
3,
4] from Industry 4.0 in general [
5] to healthcare [
6], spanning a wide range from predicting diseases [
7,
8,
9] to high-level knowledge extraction [
10,
11].
Enterprises can be considered as capital-intensive enterprises or knowledge-intensive enterprises [
12,
13], and knowledge-intensive enterprises attract intense attention in knowledge-management research [
14]. Studying the dimensions of knowledge management orientation and analyzing the survey data from managers of large firms, Lin showed that the major factor of internal process performance is knowledge sharing, and that financial performance is affected by non-financial performance measures such as learning and growth [
15]. Through a comprehensive literature analysis bridging the professional complex systems and knowledge network structure, Rangachari found that, contrary to complex systems where the effective knowledge-sharing networks may be richer in density, it may be richer in brokerage and hierarchy in the professional complex networks [
16]. Based on empirical data, Gerbin and Drnovsek studied knowledge management in academia–-industry interactions and found negative effects of collaborations with industry on knowledge sharing of researchers [
17].
Via integration of the open innovation and knowledge-transfer literature, Simeone et al. developed a framework for effective knowledge translation mechanisms, in particular for the R&D projects [
18]. Due to the dynamic and complex nature of knowledge systems, managing knowledge which might not be predictable or controllable requires an intense effort [
19,
20], advanced theories and framework development [
21]. Leadership in knowledge-intensive firms is also a center of attraction in knowledge-management research. Very recently, focusing on the thought leadership in knowledge-based organizations, Harvey et al. addressed the problem of providing a comprehensive definition of the thought leadership, found specific tensions as a consequence of creating thought leadership and developed a framework for the engagement of thought leadership at various levels [
22].
Extracting knowledge from medical or clinical documents through developing frameworks based on NLP has been a key direction in the field, and transformer models are attracting increased attention. For example, the Bidirectional Encoder Representations from Transformers (BERT) [
23] model was used by Arnaud et al. to learn contextual embeddings from free-text triage notes of a French hospital’s emergency department [
10]. BERT models were also used by Harnoune et al. to extract knowledge from biomedical clinical notes [
11].
While extracting and managing knowledge in the healthcare industry has already been taking advantage of developing frameworks based on various natural language processing (NLP) techniques [
24], there is still a great deal of room for research and development. Based on a two-decade experience of NLP practices at the Mayo Clinic, Wen et al. recently pointed out room for improvement by developing a framework for stressing the potentials of NLP in healthcare [
25]. As social media platforms have already become a major information source, NLP techniques are recently being employed to extract knowledge from the communications among patients and healthcare professionals on social healthcare networks [
26].
Aiming to contribute to knowledge management in the healthcare industry, the present work is based on the following steps. First, in order to understand the need for BI systems and the capabilities of business intelligence software, we interviewed 32 experts. Second, based on the interview data, we translated the data into meaningful information by the framework developed as the core contribution of the present work in alignment with the recent literature exploring the potentials of NLP in knowledge management in healthcare. To achieve the framework goals, we developed a software to generate log files. Third, for the validation of the defined framework, we conducted a case study of a healthcare group (Hospital 1), a leading actor in the Turkish medical industry, with 17 hospitals, 13 medical centers, and around 13,000 staff. Please note that, in compliance with regulations, the name of Hospital 1 is undisclosed. We analyzed the log files generated for Hospital 1 that contained data for 455 days. In order to extract features from the data, we applied the Bag of Words (BoW) method, and then clustered the feature vectors using the K-Means clustering algorithm [
27]. Monitoring the types of data created by the organizations, we analyzed the log files that contained demand-specific reports and their reviews by different managers. After discussions with medical and technical professionals, the framework was tested with queries for confirmation. The reliability of the presented framework can be defined as correctly identifying people in accordance with the management pyramid through their backgrounds, and this reliability was confirmed through feedback from various professionals, including the Chief Technology Officer, nurses and doctors. It is found that the present study of investigating the reports of various personnel of a hospital and mining the text to determine their place in the applying knowledge management pyramid (strategic, tactical and operational level) as a real-life case is an embodiment of the decision-making levels described in the knowledge-management literature.
This paper is structured as follows. Following a more detailed literature review below, first, the concepts used in the paper are defined. Then, the case in consideration is introduced in the methodology, and the case study and text-mining algorithms are discussed. Finally, the results of the study are set out and potential research directions are discussed.
Hospitals are complex organizations and diseases are complex problems that are difficult to solve [
28,
29]. A solution is usually produced by gathering and evaluating data from many sources. While creating a solution, it is imperative that people with different goals and expertise work together in harmony.
Being a knowledge-intense industry, healthcare is a cradle of research in knowledge management and business intelligence, where advanced techniques are used. For example, through structural equation modeling (SME) on survey data, Kim et al. found evidence that the institutional structure is a key influencer in knowledge-sharing practices, improving patient safety considerably [
30]. Performing SME on survey data, Wang and Byrd found that units in healthcare industry can substantially benefit by obtaining valuable knowledge via business analytics tools through knowledge absorption effectively, and improve the decision-making process [
31]. Through purposive and snowball sampling and thematic analysis, Pahani et al. discovered major themes and sub-themes in the tacit knowledge flow amongst physicians on social networks [
32]. As healthcare virtual communities emerge and grow, the natural question arises: what are the influencing factors in the growth of these communities?
Addressing this question, Antonacci et al. conducted a seven-year research study and collected communications data from 16 healthcare virtual communities of practice. Applying multilevel regression models, they found that the growth of a community can be predicted through structural and content-based variables, and that dynamic leaders play a key role in the growth of the community [
33]. Analyzing the data collected from healthcare end users, Sánchez-Polo et al. showed how continuous learning practices can help overcome the barriers created by IT assimilation in healthcare industry through lack of appropriate training, hierarchical or bureaucratic procedures and cultural gaps [
34]. Besides worldwide issues in knowledge management in healthcare industry, due to cultural differences, research based on specific regions make a substantial contribution to knowledge-management science. An interesting recent research study in this vein was conducted by Maheshwari et al., focusing on the healthcare industry in India. Performing partial least square SME on survey data collected from healthcare professionals, they explored how the indigenous cultural factors such as face, reputation and reciprocity affect the knowledge-sharing practices in Web 2.0 [
35]. Collecting data from the frontline community health workers fighting breast cancer in the Gujarat region of India, Fletcher-Brown et al. found that mobile technology generates knowledge as a resource for improving healthcare [
36]. Focusing on the Lebanon case and analyzing the data collected from hospital workers, Epaminonda et al. found no significant relationship between independent and dependent variables such as information and communications technologies, knowledge management, job satisfaction and customer satisfaction [
37].
Framework development is at the heart of knowledge management in healthcare. Focusing on the practices of information-communication technologies in healthcare delivery, Tripathi et al. developed a framework to demonstrate the creation of knowledge at organizational boundaries [
38]. Collecting data from a Brazilian hospital system and adding absorptive capacity to the organizational knowledge cycle, Souza et al. designed a framework to develop a novel knowledge management for healthcare [
39].
Knowledge management (KM) is based on data and therefore evidence. It is often used in the healthcare industry as evidence-based medicine, and in the literature it is called “Evidence-Based Knowledge Management” [
40]. Knowledge management should define a continuous process as factors such as technology change rapidly. In order for organizations to achieve their goals, relevant and valuable data must be gathered, shared and knowledge management rituals should be applied as a continuous process [
41].
A good knowledge-management system should measure an organization’s achievement of its goals at strategic, tactical and operational levels, which are often called decision-making levels [
42]. At the strategic level, decisions are made by answering the questions: “At which state must the organization reside?” and “Which area of business must be chosen”. At the tactical level, strategic ideas are identified by answering questions such as “How do we compete?”, “How do we outclass other organizations?” and “How do we succeed in this competition?”. At the operational level, functional and supportive actions are carried out to achieve the determined strategies. Through these actions, tangible decisions are made [
43,
44]. We identified that it is carried out at different levels, i.e., strategic, tactical, and operational levels, as illustrated in
Figure 1; this is also called a management pyramid [
43,
44].
BI is a part of knowledge management, covering the methods and processes of converting data into information and information into knowledge [
45,
46], and it is defined as an analytic process that uses acquired data to formulate decisions [
47].
The business value of BI in healthcare organizations is measured through the effective use of resources [
48], and measures of cost avoidance [
49] and improved patient care [
50]. Critical success factors for BI have been investigated and BI maturity, information content quality, information access quality, the analytical decision-making culture, and the use of information in business processes are defined as BI success dimensions [
51].
4. Discussion
Developing NLP-based frameworks for extracting knowledge and actionable insights from medical and clinical data is at the heart of knowledge management in the healthcare industry [
4,
7,
8,
9,
10,
11,
25,
28]. Aligned with these research efforts, the main purpose of this work is to provide a framework to extract knowledge management from business intelligence for the scholars and professionals in the healthcare knowledge-management field. Knowledge management can be defined as gathering, sharing and applying knowledge to create business value. Using business intelligence fully supports data collection and the sharing element of knowledge management, but applying knowledge to create business value may differ according to position.
In previous research, Türkeli and Erçek defined 3 strategies (strategic level), 21 strategic targets (business management, tactical level) and 223 criteria at the operational level in a teaching and research hospital [
69]. The current research results are in good agreement with the model (Management Pyramid) shown in
Figure 1 through the analyses of the total number of strategic, tactical and operational level reports. However, as seen in
Table 5, the data form two clusters: a cluster showing agreement with the model, and a cluster that does not show agreement with the model. Managers from a technical background concentrate on an operational (143 reports) level primarily and on a tactical (74 reports) level secondarily. This result can be understood from the perspective of project-management science, as follows. Functional managers who use the analytic approach are specialists in the relevant field and they know the details of each operation in their remit. Project managers, on the other hand, who use the systems approach and are expected be skilled at synthesis, should experience a “metamorphose from a technical caterpillar into a generalist butterfly” to be able to oversee many functional areas and gain the ability to synthesize the pieces to form a coherent whole [
70]. Hence, an important contribution of this work to the field is to evaluate the managerial competency of healthcare organizations’ managers by extracting the knowledge corresponding to their report viewing data.
At every decision level (management pyramid), the determination of correct strategies and decisions is related to a reliable decision-making process. Fountas et al.separated the decision-making process into three parts: gathering data; correlating data with information; and making further connections with this information, leading to a decision [
71]. March and Hevner identified the first phase of this process as intelligence, in which information is gathered to be processed for management decisions [
72]. Business intelligence (BI) is defined as the collection of the remaining levels, where a decision is achieved after performing analytical operations. In a rapidly changing environment, i.e., one with growing amounts of data, it is vital to perform analytical operations using BI tools and BI dashboards [
47,
73]. Sahay and Ranjan approached BI from two perspectives: process and technological. They clarified BI as an analytical process that shapes acquired data into decisions [
47]. In this way, strategic goals need to be adapted to enable the business to survive in a competitive environment. As their second approach, they claimed that BI was a cluster of technological concepts, such as text mining and online analytical processing (OLAP), and that BI tools store data about analytical operations. Türkeli identified BI as applications that are capable of providing the desired information to the required site within an appropriate time [
74].
The significance of the decision-making process is apparent, as every organization encounters a dynamic environment shaped by the nature of business competition [
47]. As this technology-dependent process determines the survival of organizations, various kinds of research have discussed thriving BI systems and the benefits of BI tools. Türkeli determined the significance of BI by using a survey of information technology (IT) managers by the Gartner Company, and pointed out that BI investment is acknowledged as the most important and privileged IT investment [
74]. Murphy and Simon discussed the complexity of benefit evaluation in BI tools, as BI tools provide mainly intangible benefits that are not directly visible through cost analysis. They summarized the intangible benefits under five topics: operational, managerial, strategic, IT infrastructure and organizational [
75].
The terms of strategic, tactical and operational decision-making have been set out here, and it has been determined that these levels are relevant to BI tools. In this study, using log file mining, the number of reports at strategic, tactical and operational levels were identified.
The concept of mining knowledge from a log file is not a new idea [
54,
76], but mining knowledge elements from a log file is a new idea, presented herein. The interviews demonstrated that the data needed for this study were included within log files. Consequently, the BI tool log file was used as the data source. When the study had progressed to that point, the necessity of a research method for understanding the existence of changes in log files was raised, as in
Figure 3. Among the text-mining methods, we chose the model-based method with K-means analysis, a type of cluster method [
55], as detailed in the Methods section.
Limitations of the present work can be considered as follows. Firstly, although the BoW method was found to be successful in designing and testing the present framework, other approaches have attracted more attention recently. Transformer-based machine-learning techniques for NLP are especially worth considering within the same context. For example, BERT models [
23] are considered promising for future improvements and possible generalizations of the present framework, as well as for comparing the performance of such methods. Though considered to have a minor impact on the results of testing the framework, another limitation is the number of interviewed experts. Thus, having the opportunity to access more experts could potentially improve the results.
Choosing the number of clusters,
k is a challenge in general for evaluation of the quality of clusters, requiring metrics such Within-Cluster-Sum of Squared Errors. In the present work, however, the number is set to
by definition, associated with the clusters of
tactical,
operational, and
strategic level as imposed by the theory of Management Pyramid [
43], and following the expert opinions, each word in the BoW is pre-associated with one of the clusters. Hence, although another choice with
could potentially lead to slightly better or worse results, it would not be in accordance with the theory.
The final validation of the framework was conducted through expert reviews which were consulted in design process of the framework, as listed in
Table 2. The experts reviewed the results carefully and stated that our results are in a good alignment with their knowledge and experience, meeting their expectations from the developed framework. In more detail, an important aspect of the results is that, as presented in
Table 5, the agreement of the report examining process with the Management Pyramid (in
Figure 1) depends on the level of control. That is, while a good agreement is found for the top level, this is not the case for the lower levels, being one of the major findings of our framework, which is confirmed by the experts.
5. Conclusions
In this work, a novel framework for distinguishing knowledge application (decision-making) and management levels by analyzing log files is presented. The reliability of this framework was confirmed by professionals from various medical organizations. After analyzing log-file data for the selected case using text mining, we concluded that most of the reports reviewed by various managers were not at a strategic level (only 6.21%). The majority of the reports created by the BI tool were operational (73.98%) or tactical (19.81%). Our results clearly indicate that this real-life case is an embodiment of the well-known management pyramid.
The Chief Executive Officer, Director of Business Management (Doctor), and General Manager showed agreement with the model, whereas the Director of Business (Finance), Director of Business Management, Outpatient Center Manager, Managers of Nursing Services, Medical Center Manager, Hospital Service Managers, and General Manager Accounting did not show agreement with the model. Managers from a technical background concentrate on the operational and tactical levels. The gap between a business intelligence system and applying knowledge (decision-making levels) is determined by the background and behavior of the users, which are very important at the decision-making level. For example, nurses, accountants and financial officers have technical backgrounds and solve routine business problems daily. The Managers of Nursing Services are experienced nurses who are specialized in patient care. When a nurse becomes a manager, his/her background and behavior is aimed at solving daily or weekly problems, so their decision-making level is operational and tactical. The chief executive officer, general manager and director of business management have a functional background and behavior. Typically, they have a Bachelor’s degree in the health sciences or business administration and all of them have a Master’s of business administration. Their background and behavior are aimed at solving daily, weekly, monthly and yearly problems, so their decision-making level is operational, tactical and strategic. The business intelligence system should be redesigned after looking at the decision-making levels based on user background and behaviors.
From an academic research perspective, this study extracts knowledge management from business intelligence levels. In practice, this study suggests a framework for analyzing the performance of business intelligence systems. By analyzing the performance of business intelligence systems organizations can maximize the benefits of their business intelligence systems and applying knowledge to create business value.
In future research, it would be useful to examine text mining in business areas other than the medical sphere to validate the reliability of the management pyramid. This study contributes to the significance of information systems research in the medical domain by verifying the reliability of text mining and clustering methods.
Finally, it should be remembered that business intelligence is sufficient at the point of gathering and sharing elements of knowledge management. It is important to make personalized designs for the values expected from people and their demands at the point of applying knowledge. The proposed framework will be beneficial not only for the health sector, but also for banking, insurance and all other businesses using business intelligence.