Pioneering a Framework for Robust Telemedicine Technology Assessment (Telemechron Study)

Abstract

The field of technology assessment in telemedicine is garnering increasing attention due to the widespread adoption of this discipline and its complex and heterogeneous system characteristics, making its application complex. As part of a national telemedicine project, the National Center for Innovative Technologies in Public Health at the Italian National Institute of Health played the role of promoting and utilizing technology assessment tools within partnership projects. This study aims to outline the design, development, and application of assessment methodologies within the telemedicine project proposed by the ISS team, utilizing a specific framework developed within the project. The sub-objectives include evaluating the proposed methodology’s effectiveness and feasibility, gathering feedback for improvement, and assessing its impact on various project components. The study emphasizes the multifaceted nature of action domains and underscores the crucial role of technology assessments in telemedicine, highlighting its impact across diverse realms through iterative interaction cycles with project partners. Both the impact and the acceptance of the methodology have been assessed by means of specific computer-aided web interviewing (CAWI) tools. The proposed methodology received significant acceptance, providing valuable insights for refining future frameworks. The impact assessment revealed a consistent quality improvement trend in the project’s products, evident in methodological consolidations. The overall message encourages similar initiatives in this domain, shedding light on the intricacies of technology assessment implementation. In conclusion, the study serves as a comprehensive outcome of the national telemedicine project, witnessing the success and adaptability of the technology assessment methodology and advocating for further exploration and implementation in analogous contexts.

1. Introduction

1.1. The Importance of the Telemedicine Technology Assessment

Telemedicine systems are inherently complex and interoperable structures, integrating diverse and heterogeneous components that interact with various stakeholders in the health domain. Their significant impact and potential come with substantial regulatory, normative, social, economic, and ethical implications. The intricate nature of these systems involves the convergence of technology, healthcare practices, and regulatory frameworks, creating a dynamic environment where seamless interaction between diverse components is essential.

The transformative power of telemedicine not only introduces new possibilities for healthcare delivery but also necessitates careful consideration of the multifaceted challenges and responsibilities inherent in its adoption. Addressing these complexities is crucial for unlocking the full potential of telemedicine while ensuring that ethical, legal, and social considerations are seamlessly integrated into the design, implementation, and utilization of these innovative healthcare solutions.

In this context, technology assessment plays a prominent role, operating across multiple domains. Exploring trends in scientific publications within the telemedicine domain, using the keywords in Box 1, reveals valuable insights.

Box 1. The proposed composite key.

  “(Telemedicine[Title/Abstract]) OR (TeleHealth[Title/Abstract]) OR (Digital health[Title/Abstract]) OR (Digital healthcare[Title/Abstract])

  ((Telemedicine[Title/Abstract]) OR (TeleHealth[Title/Abstract]) OR (Digital health[Title/Abstract]) OR (Digital healthcare[Title/Abstract])) AND (Technology assessment[Title/Abstract])

The cumulative number of publications in telemedicine since 1974 has now reached 44,607. However, within this extensive landscape, studies specifically addressing technology assessment in telemedicine amount to 153 since 1996. Remarkably, these studies make up only 0.34% of the overall research efforts in the field.

1.2. From One Health to Key Performance Indicators: An Examination of Telemedicine Technology Assessment Trends

The recent analysis of studies in the realm of technology assessment (TA) [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21] uncovers a diverse array of focal points within the field. From nuanced explorations of the One Health paradigm [1] to considerations of sustainability [3] and the intricate landscape of personalized medicine [4], these studies consistently highlight key dimensions.

Stakeholder Emphasis (I): A persistent emphasis on the pivotal role played by stakeholders and the establishment of robust frameworks is evident [6,8,13]. These frameworks serve as the scaffolding for effective technological assessments, ensuring a holistic understanding of the complex interplay between technology and various stakeholders.

Global Collaboration (II): There is a growing recognition of the importance of adopting a collaborative, global approach [10,14]. In an era where technology advancements transcend geographical boundaries, international collaborative efforts become paramount, fostering the exchange of the best practices and accelerating the standardization of assessment methodologies.

Metric Definition (III): The studies consistently stress the critical need for defining and implementing appropriate metrics, such as key performance indicators (KPIs) [18]. These metrics serve as the compass for reliably measuring and evaluating the success and impact of digital health solutions.

Tailored Assessment Tools (IV): A strategic focus on the development of tailored assessment tools for navigating the intricacies of digital technologies is noted [2,5,11,16]. These tools are indispensable for ensuring accurate, efficient, and contextually relevant evaluations in the rapidly evolving landscape of digital health.

Economic Impact Analyses (V): There is a noteworthy emphasis on economic impact analyses [9,10]. Recognizing the profound implications of digital health technologies on healthcare economics, these studies delve into fiscal aspects, aiming to unravel the economic dimensions that underpin the adoption and sustainability of such technologies.

Holistic Exploration (VI): The studies delve into the multifaceted impacts of digital health technologies on social dynamics, behavioral patterns, and remote rehabilitation [7,19,21]. This holistic exploration encompasses societal and behavioral considerations, providing valuable insights into the broader implications of integrating digital interventions into healthcare practices.

Role of Medical Devices (VII): The pivotal role of medical devices within telemedicine systems is underscored [20]. Recognized as integral components, these devices shape the efficacy and scope of telemedicine initiatives, playing a key role in influencing the trajectory of remote healthcare delivery.

This emphasizes the critical role of technology assessment in shaping the future of healthcare.

1.3. The Vital Role and the Need of a Framework in Telemedicne Technology Assessment

The need for a framework for technology assessment in telemedicine arises from several key factors outlined above:

Stakeholder Emphasis (I): Robust frameworks are essential to understanding the intricate relationships between technology and stakeholders. They provide a structured approach to assess the impact of technology on various parties involved in telemedicine [6,8,13].

Global Collaboration (II): The global nature of technological advancements requires a collaborative approach on an international scale. Frameworks facilitate the exchange of the best practices, accelerate the standardization of assessment methodologies, and promote a unified response to global healthcare challenges [10,14].

Metric Definition (III): Establishing appropriate metrics, such as key performance indicators (KPIs), is crucial for reliably measuring and evaluating the success and impact of digital health solutions. Frameworks play a key role in defining and implementing these metrics [18].

Tailored Assessment Tools (IV): Frameworks support the development of tailored assessment tools that navigate the complexities of digital technologies in telemedicine. These tools are recognized as indispensable for accurate, efficient, and contextually relevant evaluations in the rapidly evolving landscape of digital health [2,5,11,16].

Economic Impact Analyses (V): The studies highlight the need for economic impact analyses in assessing digital health technologies. Frameworks contribute to a strategic focus on unraveling the economic dimensions that underpin the adoption and sustainability of such technologies [9,10].

Holistic Exploration (VI): Frameworks enable a holistic exploration of the multifaceted impacts of digital health technologies on social dynamics, behavioral patterns, and remote rehabilitation. They provide valuable insights into the broader implications of integrating digital interventions into healthcare practices [7,19,21].

Role of Medical Devices (VII): A framework for medical devices in telemedicine is crucial for ensuring seamless integration, standardization, and compliance with regulations. It facilitates the scalability, security, and quality assurance of telemedicine systems while fostering innovation and supporting interconnected workflows among diverse medical devices.

In summary, a comprehensive framework for technology assessment in telemedicine is essential to address the diverse challenges and considerations associated with the integration of digital health technologies into healthcare practices. It provides a structured and collaborative approach that considers stakeholders, metrics, tailored assessment tools, economic impacts, and holistic explorations of societal and behavioral implications.

1.4. The Telemechron Study and the Role of the Istituto Superiore di Sanità

Telemechron (a telemedicine for the home-based management of patients with comorbidities) is a project under the Network-Oriented Finalized Research, funded by the Ministry of Health and the Italian regions (agreement date: 09/07/2020, research code: NET-2018-12367206, start date: 1 October 2020, duration: 36 months).

Participating in the Telemechron project are healthcare entities from different regions: USL Tuscany Northwest in the Tuscany region, IRCCS Maugeri of Lumezzane (Brescia) in the Lombardy region, and the Provincial Company for Social and Health Services in the Autonomous Province of Trento, along with the Italian National Institute of Health (ISS). The 36-month network program is organized into four projects, with three of them involving each participating healthcare entity in experimenting and studying the use of one or more telemedicine services in homecare for different types of chronic patients. The fourth project, led by ISS, aims to support the other projects during service implementation.

Specifically:

In WP1, USL Tuscany Northwest is the coordinator of the Telemechron project and focuses on patients with chronic renal failure.

–

In WP3, IRCCS Maugeri of Lumezzane concentrates on patients with chronic heart failure and comorbidities.

–

In WP4, the Trento Company directs its attention to patients with type 2 diabetes mellitus.

–

WP2, led by ISS, serves as a bridge between diverse telemedicine experiences. Its primary focus is on facing technological assessment issues, including: (a) Examining the effectiveness, clinical safety, organizational, technological, and financial aspects of telemedicine services designed and implemented by the three healthcare entities. (b) Elaborating and validating a set of multidimensional indicators for clinical governance.

This structure ensures a comprehensive approach to telemedicine implementation, covering different chronic conditions and involving diverse regions, while the ISS-led WP2 plays a crucial role in harmonizing and assessing technological aspects across the projects.

1.5. Purpose of the Study

The primary objective of this project study is to delineate the design, development, and application of assessment methodologies within the telemedicine project proposed by the ISS team. These methodologies have been crafted based on a specific framework proposal. The sub-objectives include the following:

Evaluation of the Proposed Methodology:

Analyzing the effectiveness and feasibility of the proposed methodology.

Gathering feedback to identify strengths and areas for improvement.

Assessment of Methodology Impact:

Examining how the methodology has strengthened various project components.

Investigating the influence of the proposed framework on different aspects of the project.

In essence, this study aims to offer a clear overview of the assessment methodologies employed in the telemedicine project. It seeks to highlight their development and assess their impact on various project components, contributing to a comprehensive understanding of the project’s assessment framework.

2. Methods

The methodology employed in this project, particularly in the domain of technology assessment and its corresponding evaluation, is outlined in detail in this project report. Central to the methodology is the iterative development of a framework, regularly presented to project partners for input, fostering continuous improvement cycles. This dynamic framework requires partners to contribute content aligning with a comprehensive grid spanning various facets of technology assessment across different domains.

After each iteration, a thorough audit is conducted, providing partners with detailed suggestions for modifications based on the predefined grid. The process unfolds through two complete rounds (Round 1 and Round 2), concluding in a final consolidation round (Round 3). In this phase, a discussion meeting ensures additional refinements through collaborative interactions, ensuring a robust and refined methodology.

To affirm the stability and effectiveness of this methodology, it underwent an evaluation using computer-aided web interviewing (CAWI) tools. A dedicated CAWI assessment was conducted to gauge the tangible impact of the refined methodology on project components. This dual-pronged approach—iterative improvement cycles and robust evaluation mechanisms—allows the assessment of the project methodology as a dynamic and adaptive framework capable of effectively addressing the evolving landscape of technology assessment.

The study delves into an in-depth exploration of the framework and its underlying rationale in Section 2.1, while Section 2.2 offers a comprehensive examination of the two CAWI tools, shedding light on their individual roles and contributions within the overarching assessment methodology.

2.1. The Structure of the Framework for the Telemedicine Technology Assessment

The proposed framework for the Telemechron project is structured according to Figure 1 and Figure 2, involving the creation of “fact sheets” for completion by different teams. The framework comprises several sheets, each serving a specific purpose:

Sheet no. 1—Personal Data of the Telemedicine Service:

–

Captures key information about the telemedicine initiative, including the official name, responsible healthcare company, reference region, operational unit, manager, work team, and types of telemedicine services offered;

–

Provides an overview of the organizational structure, key personnel, and the nature of services offered.

Sheet no. 2—Strategic Framework of the Telemedicine Service:

–

Conducts a SWOT analysis of healthcare before the introduction of the telemedicine service;

–

Identifies healthcare needs that the telemedicine service aims to satisfy;

–

Describes the functional aspects of the telemedicine service and its integration into patient care;

–

Specifies the target population and socio-economic indications;

–

Incorporates evidence from the scientific literature supporting the rationale for the telemedicine services;

–

Offers a global perspective on the strategic framework of the telemedicine initiative.

Sheet no. 3—Design and Implementation of the Telemedicine Service:

–

Includes components related to organizational–management and technical–technological requirements;

–

Covers economic–financial assessments and procurement strategies;

–

Describes the adopted methodology and actors involved in design and implementation;

–

Addresses normative and regulatory aspects;

–

Focuses on risk management and best practices;

–

Outlines crucial elements involved in designing and implementing the telemedicine service.

Sheet no. 4—Adoption of the Telemedicine Service:

–

Defines criteria and procedures for service activation and management;

–

Specifies methodologies and indicators for monitoring service quality;

–

Addresses service tariffs and cost-sharing rules;

–

Discusses change management and the activation date of the telemedicine service.

–

Quantifies the volumes of provided services;

–

Provides crucial details related to the adoption phase of the telemedicine service.

Sheet no. 5—Evaluation of the Telemedicine Service:

–

Identifies dimensions and indicators for service evaluation, including expectations and KPIs for assessing the effectiveness of the telemedicine application;

–

Describes methods for data collection and analysis during service evaluation;

–

Presents results of service evaluation;

–

Gathers lessons learned, critical success factors, and recommendations for large-scale adoption or transferability of experience.

The framework involves an iterative process through audit reports, where partners contribute content aligned with a comprehensive grid spanning various facets of technology assessment. The entire process unfolds through two complete rounds, culminating in a final consolidation round, ensuring a robust and refined methodology. The study provides an in-depth exploration of the framework and its underlying rationale, along with an examination of the computer-aided web interviewing (CAWI) tools used for evaluation. This comprehensive approach allows for the assessment of the project methodology as a dynamic and adaptive framework capable of addressing the evolving landscape of technology assessment.

Figure 1. The structure of the proposed framework (first page).

Figure 1. The structure of the proposed framework (first page).

Figure 2. The structure of the proposed framework (second page).

Figure 2. The structure of the proposed framework (second page).

2.2. The CAWI Tools for the Evaluation of the Proposed Methodology and for the Assessment of the Methodology Impact

In our study, we proposed two electronic tools using computer-aided web interviewing technology. Microsoft Forms was chosen, which is available in the Office 365 suite provided to the staff of the ISS, certified compliant by Microsoft with current IT security regulations from a systems perspective. The following modules were used in the CAWI:

–

Single-choice questions;

–

Multiple-choice questions;

–

Evaluation (graded) questions (with a 6-level psychometric scale);

–

Likert questions with a 6-level psychometric scale;

–

Open-ended questions (in a few cases).

The first tool, the “Feedback Form”, allows the collection of feedback on the proposed framework. It has two similar versions. The first version is dedicated to the project staff. The second version is dedicated to external observes, i.e., experts from scientific societies.

The link and the QR code (Figure 3) for the external observers are the following ones.

https://forms.office.com/e/7R8zfmeXG8 accessed on 28 February 2024.

The second tool is the tool “Evaluation process summary form”. It is dedicated to the ISS team involved in the telemedicine technology assessment.

The link and the QR code (Figure 4) is for the evaluation process summary form.

https://forms.office.com/e/2ht3Ce28WL accessed on 28 February 2024.

3. Results

3.1. Evaluation of the Proposed Methodology

The evaluation of the proposed methodology through targeted feedback is a pivotal strategic element, serving dual purposes: firstly, to secure an analytical quantification of the tool’s efficacy, and secondly, to gather crucial improvement suggestions vital for its solidification in anticipation of its integration into the national health service. At the core of this evaluative process is a computer-aided web interviewing (CAWI) tool developed explicitly for this purpose, deployed among both the internal project team and a select group of external observers.

Careful consideration was given to the selection of external observers, chosen based on their concurrent experience and affiliation with esteemed scientific societies in the field. Their expertise specifically focused on digital health, the administration of heterogeneous network systems, and interconnected issues. This approach ensures that the feedback received is not only insightful but also derived from a well-versed perspective. In the assessment, individual graded and Likert responses were employed, utilizing a scale ranging from a maximum score of 6 to a minimum of 1. An average score surpassing 3.5 =$\frac{1 + 6}{2}$ signified a positive evaluation, with a higher score approaching 6 indicating a more favorable response. On the flip side, a score falling below 3.5 signified a negative evaluation, with a lower score approaching 1 indicating a more critical stance. Insights gleaned from the internal project team played a crucial role in the consolidation process, unearthing valuable suggestions such as the need for “simpler terminology for those not familiar with technology assessment” from the “telemedicine service designers”, and post-analysis considerations like “the possibility of translating it into a CAWI tool” from the framework proponents.

Despite inherent biases within this group—stemming from the contrasting perspectives of the “telemedicine service designers”, who directly “experienced” the tool, and the proponents, who originally “conceived” the tool—the internally collected data reflected an overall positive trend. It is noteworthy that all questions with graded and Likert evaluations (N. 4, N. 6–10; N. 12) received an average rating above 3.5, indicating positive acceptance.

For transparency, the analytical results made by 16 external observers, deliberately chosen to mitigate the impact of internal biases, are reported. This external group, unaffected by antithetical biases, offers a more impartial and objective perspective. To ensure a thorough and scientifically relevant assessment of the proposed framework, a rigorous selection process was undertaken to engage experts from scientific societies focused on telemedicine. The criteria encompassed diverse expertise, including hands-on experience in telemedicine implementation, technological assessment proficiency, regulatory acumen, and strategic insight. The resulting panel of experts not only brings a wealth of knowledge but also reflects a comprehensive representation of the telemedicine landscape, ensuring a robust evaluation of the framework’s efficacy and applicability.

The familiarity level with evaluation tools in this field garnered an average rating of 5.06. The overall evaluation of the tool received an average rating of 5.63. The structure of the various sheets earned the following respective average scores: Sheet 1: 5.56, Sheet 2: 5.59, Sheet 3: 5.56, Sheet 4: 5.63, and Sheet 5: 5.75.

In response to the multiple-choice question, two respondents expressed, “I believe that the proposed tool allows for a comprehensive evaluation of a heterogeneous telemedicine system”, while fourteen respondents affirmed, “The tool has proven to be suitable and has the potential for transfer to the Italian National Health System for similar project experiences”.

The butterfly diagram representation indicates excellent acceptance, with no tail below zero (see Figure 5).

The integration of open-ended responses has proven invaluable, extending beyond quantitative metrics to elicit a nuanced qualitative assessment through the computer-aided web interviewing (CAWI) system. This qualitative evaluation, akin to a focus group with external observers, highlights the project’s substantial utility. It strongly suggests the potential evolution of the project into a sophisticated web-based platform. This envisioned transition could be further fortified by incorporating user-friendly help features and comprehensive information support, catering particularly to those less versed in the subject matter.

As the project approaches its conclusion, a resounding recommendation is put forth for a formal dissemination strategy targeted at scientific societies. This strategic outreach aims to effectively communicate the project’s outcomes, encouraging a broader adoption and meaningful engagement within the professional community. The resonance of this recommendation lies in its potential to amplify the impact of the project and foster continued advancements in the field.

3.2. Assessment of Methodology Impact

The active participation of three out of four experts currently engaged in the project (with one member now retired) in completing the survey through the proposed CAWI tool marks a significant stride toward in-depth analyses of improvement trends. This process primarily focuses on consolidating documentation and indirectly evaluates the tool’s ability to steer the project toward optimization, both from a project and documentation analytical standpoint. The responses provided by the experts serve as valuable indicators of the project’s internal dynamics, outlining a path toward strengthening documentation and implicitly suggesting the crucial role of the tool in guiding the project toward overall optimization. This analysis not only provides an overview of current project and documentation practices but also offers an opportunity to identify areas of potential improvement and strategic optimization.

In the assessment, individual graded and Likert responses were employed, using a scale ranging from a maximum score of 6 to a minimum of 1. An average score surpassing 3.5 indicated a positive evaluation, with a higher score approaching 6 indicating a more favorable response. Conversely, a score falling below 3.5 signaled a negative evaluation, with a lower score approaching 1 indicating a more critical stance.

An analysis of the results reveals a distinct transformation in the project dynamics. In Round 1, both Sheet 1 and Sheet 2 exhibited an overall negative evaluation for all project components, while the remaining sheets received negative ratings for each project aspect. However, by the conclusion of the third round, there was an unequivocally positive assessment. All butterfly diagrams displayed no tail below 0 (indicating that no component received a rating below 4), with an average rating surpassing 4.9.

As an illustrative example (see Figure 6), let us focus on one of the three butterfly diagrams from Sheet 5, representing a specific component of the project. It is worth noting that the other components exhibit similar trends, and the intention here is not to identify the best among them.

This specific diagram highlights a significant improvement in documentation, evident in the assigned analytical evaluations. This notable enhancement underscores the commendable progress achieved across all facets of the project.

4. Discussion

4.1. In Focus: Evaluating the Project Report’s Value and Contributions

The study delves into the pivotal role of dedicated tools for technology assessment within the health domain, particularly in the intricate landscape of telemedicine, as previously showcased in groundbreaking experiences [22,23]. Given the inherent complexity and heterogeneity of the system, addressing a diverse array of domains simultaneously becomes imperative. In a broader sense, the study has illuminated how an expansive framework within this realm can emerge as a valuable and indispensable tool. Expanding on the distinct values, the study brings forth the following:

Framework and Rationale: The initial added value lies in the meticulous design of the framework itself, accompanied by a step-by-step elucidation of its rationale. This not only provides a structured approach but also offers a transparent understanding of the methodology’s underpinnings.

High Acceptance Level: A secondary yet significant value is the study’s revelation of a notably high level of acceptance garnered by the proposed framework. This attests to its resonance and alignment with the needs and expectations of the stakeholders involved.

Impact Evaluation: The third added value emerges from the comprehensive evaluation of the methodology’s impact. By employing dedicated tools, the study gauges the tangible effects and improvements catalyzed by the methodology across various dimensions of the telemedicine project.

CAWI Tools: Noteworthy is the introduction of two CAWI tools as a substantial last value addition. These tools have proven not only to be useful but also easily exportable, contributing to the practicality and versatility of the methodology.

In essence, the study’s multifaceted contributions underscore its significance in shaping effective technology assessment practices within the dynamic landscape of telemedicine.

The incorporation of a well-justified framework, widespread acceptance, meticulous impact evaluation, and practical tools positions the study as a valuable resource for future endeavors in similar domains. Additionally, interpreting these results in the context of historical studies focused on the technology assessment process in telemedicine is beneficial.

A search on PubMed using the composite key ((telemedicine[Title/Abstract]) AND (process[Title/Abstract])) AND (“technology assessment”[Title/Abstract]) leads to the identification of 13 studies. Among these, six [22,23,24,25,26,27] particularly concentrate on the process and encompass broad applications/approaches, not limited to a single domain or a unique telemedicine application.

The collective narrative emerging from the cited studies, as well as the one proposed here, underscores a growing recognition and exploration of potential within the field, emphasizing the strategic importance of technology assessment. Diverse in their contexts and methodologies, these studies collectively convey an increasing interest in harnessing telemedicine technologies for clinical decision support and patient-centered care, highlighting the crucial role of technology assessment across multiple domains.

Mackintosh et al.’s systematic review [24] emphasizes the integration of telemedicine with clinical decision support, indicating a broader acknowledgment of technology’s role in enhancing decision-making processes in healthcare. The study advocates for further research accompanied by process evaluations to ascertain the effectiveness, implementation, and associated costs.

Doupi’s work [25] contributes to the evolving landscape of health information technology systems evaluation, aligning with the recognized convergence of health informatics and health technology assessment. The author underscores the interconnectedness of these domains, emphasizing the need for comprehensive evaluation approaches.

Ekeland and Grøttland’s proposed technology assessment [26] centers on patient-centered telemedicine pilots in Europe, offering insights based on the MAST approach. This study emphasizes the importance of tailoring technology to meet the specific needs of individuals, providing a patient-centric perspective.

An Italian study by Giansanti et al. [27] introduces a practical dimension, showcasing a web-based health technology assessment. This experience highlights the application of assessment methodologies in real-world telemedicine projects and emphasizes the evaluation of technology’s impact on specific healthcare domains, leveraging web technologies. The study proposed here, in conjunction with these studies, collectively illuminates the ongoing evolution and diverse applications of technology assessment in telemedicine. Together, they underscore the increasing significance of systematically evaluating the impact of telemedicine technologies, not only in critical care scenarios but also in broader patient-centered contexts, contributing to the overall advancement of healthcare practices across multifaceted domains.

4.2. Work in Progress

The study is currently focusing on the initial perspective of the technology assessment evaluation. The ongoing phase involves a meticulous and detailed dissemination of results, a process explicitly characterized by the consolidation of contents within the established framework. This critical final phase, spanning a duration of six months, encompasses indispensable internal steps within the project, including administrative procedures, and external steps that involve interactions with the project’s funding bodies. The envisioned dissemination aims to provide a complementary contribution to the study, reaching a wider audience through international journals and ISTISAN Reports—dedicated tools with a proven track record in previous technology assessment projects within this field [28].

As the project advances, the team has actively incorporated valuable suggestions obtained through a structured methodological approach. This endeavor is aimed at not only fortifying the completeness of the methodology but also initiating a systematic integration process. Notably, significant emphasis has been placed on the iterative development process, which not only sheds light on the challenges encountered but also articulates the strategies employed. This nuanced understanding is expected to be of great value to scholars and practitioners within the technology assessment domain.

Furthermore, the documentation for transfer to the national health service is set to include concrete examples and case studies. This strategic inclusion serves a dual purpose—it not only enriches the overall documentation but also provides practical illustrations of the framework’s application in real-world scenarios. By doing so, the study aims to enhance the efficiency and effectiveness of the methodological approach followed.

4.3. Suggestions for Future Investigation

Looking ahead, the study lays the groundwork for future investigations that can propel the discourse around telemedicine assessment forward. Emphasizing the long-term impact and sustainability of telemedicine initiatives post implementation opens a pathway for researchers to explore the enduring implications of these interventions. Additionally, the proposed exploration of potential barriers to implementation and strategies for overcoming them anticipates the challenges that may arise, providing a strategic roadmap for the successful adoption of the methodology. The proposed framework directly faces ethical issues (e.g., checks for considerations of ethical committees and ethical justification). However, a call for the continuous examination of the ethical considerations associated with technology assessment in telemedicine is not just a suggestion but an imperative invitation. Regulations and standardizations are fields upon which a country depends. Therefore, establishing stronger connections between the methodology and established national healthcare assessment standards and regulations is a strategic suggestion.

4.4. Limitations

The study also identifies specific areas for improvement. In particular, the CAWI assessment section could benefit from a more detailed exploration in future iterations during the transfer to the national health service, overseen by professionals, to make it a stable tool. While the study hints at potential risks associated with telemedicine initiatives, it emphasizes the need for the continuous calibration and monitoring of these risks, especially in the light of national initiatives. Ethical considerations within the context of the technology assessment process in the telemedicine project are recognized as a critical dimension requiring ongoing and targeted investigations in each national context.

4.5. Takeway Message

The study emphasizes the pivotal role of dedicated tools in technology assessment for telemedicine, building on historical experiences. Key contributions include a meticulously designed framework, high acceptance levels, comprehensive impact evaluation, and the introduction of CAWI tools. The collective narrative, alongside historical studies, highlights a growing recognition of telemedicine’s potential, stressing the strategic importance of technology assessment. Overall, the study and related research underscore the need to systematically evaluate telemedicine technologies’ impact, fostering advancements in healthcare practices across various domains.

5. Conclusions

This study elucidates the outcomes stemming from (a) the conceptualization, development, and implementation of a technology assessment initiative based on a framework proposed by the National Centre for Innovative Technologies in Public Health at the ISS within a national telemedicine project; (b) the comprehensive assessment of the proposed methodology through CAWI; and (c) the CAWI evaluation of the methodology’s impact. The study underscores the multidimensionality of action domains and emphasizes the pivotal role of technology assessment in telemedicine across diverse realms through multiple interaction cycles with project partners. The assessment of the proposed methodology revealed a remarkably high degree of acceptance and facilitated the collection of valuable insights for refining the framework in future initiatives. Delving into the impact assessment of the methodology allowed the identification of a consistently marked trend of improvement in the project’s proposed products, as evident in the methodological consolidations. The overarching message emanating from this report is one of encouragement for similar initiatives in this domain. The study not only sheds light on the intricacies of technology assessment implementation but also highlights the crucial importance of iterative cycles of interaction with project partners. The evaluative processes conducted, particularly through CAWI, serve not only to affirm the efficacy of the methodology but also to underscore its adaptability for future endeavours. The marked improvement observed in project deliverables reaffirms the methodology’s relevance and positions it as a robust framework for guiding telemedicine initiatives.

In conclusion, the study serves as a comprehensive outcome of the national telemedicine project, witnessing the success and adaptability of the technology assessment methodology and advocating for further explorations and implementations in analogous contexts.