Hospital Web Quality Multicriteria Analysis Model (HWQ): Development and Application Test in Spanish Hospitals

Abstract

The Hospital Web Quality Multicriteria Analysis Model (HWQ) is constructed, designed, and validated in this research. For this purpose, we examined the web quality analysis models specialized in hospitals and health centers through a literature review and the most current taxonomies to analyze digital media. Based on the benchmarking and walkthrough methods, the analysis model was built and validated by a panel of experts (X = 3.54, CVI = 0.88, Score Σ = 45.58). To test its applicability and reliability, the model was pilot-tested on the websites of the ten public and private hospitals with the best reputation in Spain in 2022, according to the Merco Sanitario ranking. The results showed very similar web structures divided by specific proposals or sections of some centers. In this regard, this study identifies a general communication proposal in hospitals that does not adapt to the guidelines of screen-mediated communication, as well as a lack of personalization and disruptive storytelling ideation. In addition, the work concludes that Spanish hospitals, for the moment, have not opted for formats and technological developments derived from the possibilities of gamified content, 360° immersion, Virtual Reality (V.R), or Augmented Reality (A.R).

1. Introduction

For over two decades, websites have become essential for all types of entities, as they are the “presentation cards“ of organizations to their stakeholders [1,2]. In the case of hospitals, the sites facilitate access to medical information and can provide up-to-date details on the center’s medical staff, diseases, treatments, and procedures [3].

In addition, some hospital websites allow access to electronic medical records [4] and appointment scheduling, improving the efficiency and accessibility of medical care, reducing waiting and patient care times, and increasing patient satisfaction [5]. These features contribute to transparency and confidence generation, helping patients make more informed decisions when choosing a hospital or physician, as well as reducing unnecessary visits to the health center [6].

In this regard, hospital websites present a complex structure to ensure effective information quality and interactivity, and to achieve the purposes described above, there must be an important effort of information, updating, and programming, without neglecting key aspects such as accessibility, lightness and simplicity, responsive design, loading speed, and ease of navigation.

Some health institutions, such as the Mayo Clinic in the U.S., have developed multilingual health libraries on their website, which serve as a relevant resource for promoting information on diseases and conditions, symptomatologies, tests and procedures, medications, and supplements [7], as well as resources for health professionals and educational content. On the one hand, this makes it an international reference for health information; on the other hand, it substantially improves its positioning in search engines such as Google or Bing (SEO) by including more metadata and keywords that link to its corporate website.

Regarding Spanish hospital websites, in 2008, Díaz Cuenca and Chain Navarro noted that, although gradual progress had been made, the quality of the content of the websites analyzed was insufficient and that it was necessary to unify content criteria and include and agree on a common information policy for citizens on hospital websites [8]. Similarly, in 2014, Calvo-Calvo concluded that the websites of large Spanish hospitals were simply informative spaces that did not facilitate the interaction, participation, and collaboration between users and patients [9]. Observations were also reached by García-Lacalle, Pina, and Royo in 2011 [10]. In fact, by 2005, in a more global study of hospital websites in Spanish-speaking countries, Cardelle and Rodriguez warned that Hispanic patients were not only at risk of residing on the wrong side of the “technology gap,” but of being on the wrong side of the “quality gap,” recommending the development of a mechanism to standardize the quality of information on Spanish-speaking hospital websites [11].

The objective of this research is to design and validate, based on theoretical constructs, benchmarking by walkthrough method and expert panel, a multicriteria model of quantitative–qualitative analysis of hospital websites (HWQ). Subsequently, the study aims to evaluate its applicability in Spanish hospitals in order to develop a mechanism to standardize dimensions and indicators that should be present in these informative spaces.

Web Quality Analysis Dimensions and Indicators

The scientific community specializing in health communication has made many efforts to create models and taxonomies for analyzing hospital web quality. However, many researchers have suggested that the best content for hospital websites will depend on the specific needs of their patients [12,13], the accuracy and completeness of their information [14], their ease of use and accessibility [15,16,17], their design and layout [18,19,20], their security protocols [21], and their overall user experience (UX) [22,23,24].

The main analysis models for evaluating hospital web quality were as follows [25]: MiLE, MINERVA, Web-Med-Qual, Fuzzy PROMETHEE, Web-QEM, and MCDM. Although there are many more experimental, benchmarking, and conceptual models, these are the most widely used in previous studies (

Table 1. Main analysis models for evaluating the web quality of hospitals.

Name of the Model or Taxonomy of Analysis	Authors	Year	Criteria
MiLE (Milano-Lugano)	Triaca [26]	2005	Content, services, navigation, cognitive features of the interface, aesthetics, and technology.
MINERVA (Ministerial Network for Valorising Activities in Digitalization)	Signore [27]	2005	Transparent, effective, maintained, accessible, user-centered, responsive, multilingual, interoperable, managed and preserved.
Web-Med-Qual	Provost, Koompalum, Dong, and Martin [28]	2006	Content, authority of source, design, accessibility and availability, links, user support, confidentiality and privacy, e-commerce.
Fuzzy Preference- Ranking Model (PROMETHEE)	Bilsel, Büyüközkan, and Ruan [29]	2006	Tangibles, reliability, responsiveness, assurance, empathy, quality of information, integration of communication
Web-QEM (Web Quality Evaluation Model)	Cimino and Micali [30]	2008	Usability, reliability, efficiency, and functionality (based on ISO-9126 *).
MCDM (Multicriteria Decision Making)	Perçin [31]	2019	Visual content, technical service, assurance, responsiveness, reliability, and customer service.

Source: Author’s elaboration. Note: The criteria for each model were extracted textually from them, regardless of whether they were dimensions, variables, or indicators. * International Organization for Standarization (2001). ISO/IEC 9126-1:2001. Available at: https://www.iso.org/standard/22749.html.

).

Although there are several models for analyzing hospital websites, research on the quality of hospital websites is not as robust [32], so the application of the models in Table 1 shows inconsistent quality and evidence for their use is weak [33]. Therefore, it is necessary to complement and update the evaluation criteria used by the main models (Table 1) to adapt them, through a benchmarking exercise, to the theoretical construction of the model proposed in this research.

Based on a literature review of 305 publications on web quality, Morales-Vargas, Pedraza-Jiménez, and Codina found 13 dimensions and more than 120 factors of web quality that are most frequently used in the 50 most cited publications [34] (

Table 2. The most frequently mentioned website quality parameters (dimensions and indicators).

Dimension	Strategic	Focus Functional	Experiential
Usability and accessibility.	Effectiveness.	Accessibility.	User control.
	Efficiency.	Reversal of actions.	Learnability.
	Customization.	System status visibility.	Ease of comprehension.
	Error prevention.		Usability.
			Flexibility.
			Intuitive.
			Recognition before memory.
			Reduced memory load.
			Ease of use.
Content and services.	Persuasive discourse.	Up-to-date.	Authority, guarantee, and support.
	Equity and inclusion.	Attribution of authorship.	Clarity.
	Expertise.	Written for the Web.	Completeness and exhaustiveness.
	Contact information.	Writing and spelling.	Conciseness.
	Multilingualism.	Linking of content.	Reliability.
	Objectivity.	Visualization of information.	Readability.
	Social responsibility.		Precision.
	Uniqueness and value added		Relevance and pertinence.
	Truthfulness and rigor.
Information architecture.	Discovery or predictive recommendation	Internal site search.	Mobile adaptability.
	Home page or main page.	Preventing orphan pages.	Localization and state.
		Site map.	Labeling.
		Main menu.	Navigability
		Organization.
User experience.	Trust.	Utility	Convenience.
	Satisfaction.		Credibility.
	Perceived value.		Expectations.
			Empathy.
			Relevance.
Graphic design.	Corporate identity.	Adequate use of color.	Visual appeal.
	Creativity and innovation.	Consistency and coherence.	Expressiveness of icons.
		Appropriate choice of font.	Familiarity.
		Clear links.	Simplicity and clarity.
		Aesthetic integrity.
Technology and security.	Domain and URL.	Adherence to standards.	Speed and response time.
		Compatibility and interoperability.	Security.
		Availability and stability.
		Link performance.
		Technological integration.
		Validation of HTML code CSS.
Interactivity			Dialogue.
			Responsiveness.
			Feedback.
			Gamification.
Performance and effectiveness.	Conversion.
	Reputation.
	Traffic.
	Online transactions.
	Page views.
	Return on investment.
	Visibility.
Legal aspects.	Transparency.		Sensitive content.
	Declaration of sponsorship.		Privacy and confidentiality.
	Copyright and intellectual property.		Protection of minors.
	Honesty in advertising policy.		Data protection.
Assistance and support.		Help and documentation.	Customer service.
		Frequently Asked Questions.
Advertising and marketing.	Marketing.
	Brand image.		Popularity.
	Sales orientation.		SEO for website positioning.
	Special offers.
	Advertising and advertisements.
Multimedia.		Animation.
		Audio and sound.
		Images, graphics, and photographs.
		Video and audiovisual.
Participation and sociability.	Social assessment.		Community.
	Comment forum.		Opinions.
			Participation.
			Social networks.

Source: Author’s elaboration based on Morales-Vargas, Pedraza-Jiménez and Codina (2023, pp. 105–106) [34].

).

Additionally, Tejedor, Romero-Rodríguez, and Codina presented and validated a taxonomy of dimensions and integrated indicators for digital native media, mainly aimed at evaluating the quality of media web information, divided into five areas and 16 dimensions [35] (

Table 3. Taxonomy of digital media analysis.

Field	Dimension
Narrative	Form of content and narrative
	Journalistic genres and styles
	Headlines
Quality	Metadata of the news stories
	Transparency
	Truthfulness and validity of content
	Related content
	Content curation
Usability	Uninterrupted heading
	Interface layout
	Legibility of content
	Content customization
Search for information	Web browser
	SEO and visibility
Redissemination of content and socialization	Interaction with new stories
	Redissemination

Source: Author’s elaboration based on Tejedor, Romero-Rodríguez, and Codina [35].

):

2. Materials and Methods

This research aims to design and validate, based on theoretical constructs, the walkthrough method and expert panel benchmarking techniques, a multicriteria model for quantitative–qualitative analysis of hospital websites. It subsequently evaluates its applicability in Spanish hospital websites in order to develop a solid mechanism to standardize dimensions and indicators that should be present in these digital spaces (Figure 1).

2.1. Instrument

First, theoretical sampling of the main web analysis models was carried out, including those taxonomies that, although out-of-date, can provide certain analysis criteria. Considering the dimensions and indicators of previous research, a benchmarking process was carried out, defined as a comparative evaluation of indicators to apply the best practices for continuous quality improvement [36].

This benchmarking process was performed in two ways. First, among all the instruments reviewed in the previous section, once we had a consolidated matrix, the websites of the five best hospitals in the United States were reviewed according to the U.S. News 2022–2023 Best Hospitals Honor Roll ranking (

Table 4. Top 5 U.S. News 2022–2023 Best Hospitals Honor Roll.

Pos.	Hospital	Location	Website
1	Mayo Clinic	Rochester, Minnesota	https://www.mayoclinic.org/
2	Cedars-Sinai Medical Center	Los Angeles, California	https://www.cedars-sinai.org/
3	NYU Langone Hospitals	New York, New York	https://www.cedars-sinai.org/
4	Cleveland Clinic	Cleveland, Ohio	https://my.clevelandclinic.org/
5 (tie)	Johns Hopkins Hospital	Baltimore, Maryland	https://www.hopkinsmedicine.org/
5 (tie)	UCLA Medical Center	Los Angeles, California	https://www.uclahealth.org/

Source: U.S. News. Note: accessed on 21 September 2023.

). This second process was carried out following the steps of the walkthrough method because, as a relatively closed technical system, websites pose new methodological challenges for sociocultural research on digital media. Benchmarking of hospitals in the United States was performed because most of the models and theoretical constructs referenced above were applied in this context.

The walkthrough method consists of establishing the intended use environment of an application or website and identifying and describing its vision, operating model, and governance modes. It then deploys a walkthrough technique to systematically and forensically navigate the different phases of registration and entry into the web, daily use, and abandonment of use [37].

Therefore, the proposed model would initially be comprised of dimensions and indicators from the scientific literature, but also those that emerge from the benchmarking process by the walkthrough method, analyzing the websites of the hospitals reviewed in Table 4. The content extracted from other theoretical models was created based on an analysis sheet that made it possible to organize them into families of content codes. This initial model (

Table 5. Initial hospital web analysis model.

Dimension	Indicator	Content	Validation
Content	Hospital basic information	Hospital name in the header	Checklist
		Address	Checklist
		Phone numbers	Checklist
		E-mail address or contact form	Checklist
		Description of the activity performed (mission, vision, values, achievements).	Checklist
		List of services and specialties offered by the hospital (emergency, surgical, external consultations, imaging, laboratory, rehabilitation, etc.).	Checklist
		Opening hours	Checklist
		Information on insurance coverage and treatment or consultation coverage or prices/fees for consultations or interventions.	Checklist
	Transparency	Management type identification, sponsors or promoters	Checklist
		Hospital history	Checklist
		Corporate report for the previous year	Checklist
		Institutional accreditations and/or awards	Checklist
		EFQM, ISO, or AENOR quality assessment	Checklist
		Patient quality assurance claims	Checklist
		Indication of copyright and intellectual property of the website and its content	Checklist
		Hospital data (patients attended, healthcare personnel, number of beds, number of operating rooms, scientific publications, etc.).	Checklist
		Statistics (successful operations, average waiting time for consultations).	Checklist
		Results of previous years’ satisfaction surveys	Checklist
		Site policies, data protection, terms and conditions and cookie policy	Checklist
		Privacy and confidentiality statements.	Checklist
		Ethics committee: members, protocols, guidelines, and procedures.	Checklist
	Corporate information	Organizational chart and hospital structure	Checklist
		Hospital authorities and governance	Checklist
		Department directory: names, description, training, experience, responsibilities and contact information	Checklist
		Link to press or communication department	Checklist
		Hospital news section	Checklist
		Institutional agenda or calendar	Checklist
		Identification of collaborating entities or strategic alliances	Checklist
		Research groups or educational units	Checklist
		Corporate social responsibility activities	Checklist
		Employment portal	Checklist
		Testimonials and comments on successful patient experiences
	e-Health resources	Health library (symptoms, conditions, treatments, diagnoses)	Checklist
		Materials on preventive medicine and prevention programs	Checklist
		Interactive materials (videos, infographics) on diseases, symptoms, treatments, maternity.	Checklist
		Clinical research in which the hospital or its staff participates.	Checklist
Navigability	Basic aspects	Site Map	Checklist
		Top menus	Checklist
		Language selection (minimum Spanish and English)	Checklist
		Internal search menu on the page	Checklist
	Readability and UX	Legibility and ease of user navigation on the website	Likert scale.
		Help and documentation.	Checklist
		FAQ section	Checklist
		Suggestions and claims mailbox	Checklist
	Medical staff	Directory of healthcare services and professionals: names, responsibilities, and contact details	Checklist
	Appointments for external consultation	Portal for users/patients access (intranet)	Checklist
		Platform for appointment management with external consultants	Checklist
	Web loading speed	−10 s to load the home page	Checklist
	Link performance.	No broken links or links with errors	Checklist
	Socialization	Links to hospital social networks	Checklist
Aesthetics	Responsive design	Mobile and tablet adaptability	Likert scale.
		Ease of navigation in mobile format	Likert scale.
	Corporate Visual Identity	Use of corporate logo, font, and colors	Checklist
		Visual and iconographic coherence	Likert scale.
		Aesthetic integrity	Likert scale.
	Menus and links	Menus and links are clear and organized. Information is easy to find.	Likert scale.

Source: Author’s elaboration.

) was sent to a panel of experts to assess each structured dimension’s clarity, concreteness, centrality, and relevance. The emerging content variables were organized by the researchers into a set of indicators, which in turn belonged to a dimension. This pre-categorization was also assessed by an expert panel.

The expert panel trial was conducted between June and July, 2023. The panel comprises 15 digital communication experts, including seven university professors/researchers, four hospital communication directors, and four digital advertising and web design experts. The inclusion criteria were as follows: (1) those with more than ten years of experience in the field of digital communication; (2) those who considered themselves qualified to validate an instrument for evaluating hospital websites; and (3) those who completed all the obligatory questions of expert judgment. The experts were asked to evaluate the quality, concreteness, centrality, and importance of each dimension and its content using a 4-point Likert scale (e.g., 1 = not clear, 4 = very clear).

In addition to using the average of the responses as an indicator of validity, the Content Validity Index (CVI) was also used to measure the level of inter-judge agreement, which was calculated by adding the percentage agreement scores of all items that received a rating of “3” or “4” from the experts [38]. Microsoft Excel 365 was used for statistical analysis with the XLSTAT statistical package.

Equation (1) Content Validity Index (CVI)

$$CVI={\displaystyle \frac{N of raters giving a rating of 3 or 4}{N total of raters}}$$

(1)

Items are considered adequate if the concordance is above 79%, questionable if the concordance is 70–79%, and unacceptable if the concordance is below 69% [38] (

Table 6. Results of the expert panel.

Dimension	Indicator	X Value	CVI	Score *
Content	Basic information about the hospital	3.93	1.00	3.93
	Transparency	3.60	0.86	3.09
	Institutional information	4.00	1.00	4.00
	e-Health resources	3.20	0.80	2.56
Navigability	Basic aspects	4.00	1.00	4.00
	Readability and UX	3.53	0.86	3.03
	Medical staff	3.66	0.93	4.59
	Appointments for external consultation	2.93	0.73	2.13
	Website loading speed	3.00	0.73	2.19
	Link performance	4.00	1.00	4.00
	Socialization	2.86	0.66	1.88
Aesthetics	Responsive design	3.60	1.00	3.60
	Corporate visual identity	3.40	0.80	2.72
	Menus and links	3.86	1.00	3.86
		3.54	0.88	45.58

Note: Green (very good), yellow (good), red (regular). X = Average score. * Score = X Value × CVI.

).

Among the main observations of the judges who gave lower values to certain items was that the model should consider differences between ownership (public and private hospitals), size (small, medium, and large), as well as the existence of certain specialized hospitals (oncology, maternal, and child hospitals, etc.). The main objection to the model is that in many public hospitals, the service of appointments with specialists is managed by family physicians, with patients unable to make appointments themselves. The same happens with social networks in certain public hospitals, some of which in certain communities are forbidden to have their own profiles, concentrated, and managed in the same account of the Department of Health of the Regional Government.

Another of the experts’ comments concerned the section that values research groups or teaching units in the institutional information indicator, as this would only be available to university hospitals.

About these observations, the following is explained:

• Do not consider the appointment scheduling service for public hospitals, but have a portal that allows consultation, modification, or cancellation of appointments with specialists.
• In hospitals that are not university centers, the criterion research groups or teaching units should not be considered, but if they are, the professionals who teach at the university and the research groups to which they belong should be indicated.
• The links to the social networks of the public hospital’s website lead, if they do not have their own profiles, to the social networks of the Department of Health of the Regional Government.
• Neither the size nor the specialty of the hospital modifies the instrument because it does not examine in-depth issues such as the amount of information, healthcare personnel, or e-Health resources, but rather their existence.

2.2. Sample

According to data from the Spanish Ministry of Health, there are 449 public and 438 private hospitals, totaling 887 hospitals in the national territory. However, it is important to note that most public hospitals do not have their own websites but rather depend on the state (autonomous community) in which they are located, while private hospitals, in their vast majority, belong to business groups and, as a general rule, maintain the same web format and content.

For the application test, we chose to analyze the websites of the ten hospitals with the best reputation in Spain, according to the Merco Healthcare (Merco Sanitario) 2022 ranking. In this sense, the five best-rated public and five private hospitals were chosen (

Table 7. Merco Healthcare Ranking, 2022: Top 5 Spanish hospitals by ownership.

Pos.	Hospital	Location	Website
Public Hospitals
1	Hospital Universitario La Paz	Madrid	https://www.comunidad.madrid/hospital/lapaz/
2	Hospital Clinic de Barcelona	Barcelona	https://www.clinicbarcelona.org/
3	Hospital Universitario Gregorio Marañón	Madrid	https://www.comunidad.madrid/hospital/gregoriomaranon/
4	Hospital Universitario 12 de Octubre	Madrid	https://www.comunidad.madrid/hospital/12octubre/
5	Hospital Universitari Vall D’Hebron	Barcelona	https://www.vallhebron.com/
Private Hospitals
1	Clínica Universidad de Navarra	Pamplona	https://www.cun.es/
2	Hospital Universitario Quirón Salud Madrid	Madrid	https://www.quironsalud.es/hospital-madrid
3	Hospital Universitario HM Sanchinarro	Madrid	https://www.hmsanchinarro.com/
4	Hospital Universitario Sanitas La Zarzuela	Madrid	https://www.hospitallazarzuela.es/
5	Centro Médico Teknon Quirón Salud	Barcelona	https://www.teknon.es/

Source: Merco Healthcare Ranking, 2022. Note: accessed on 21 September 2023.

). As explained above, most public hospitals standardize their websites according to the state or autonomous community in which they are located, while private hospitals dependent on business conglomerates also standardize their websites and content. In this sense, the selection of this non-random sample also allows for a variety of interfaces, as if a larger number of hospitals were selected, the results would coincide with the first five. It is also important to note that this sample selection is exploratory since what is sought with this is to validate the usefulness of the proposed model.

To ensure objectivity in the application of the instrument, two coders independently evaluated the hospital websites and shared the evaluation results afterward. In the event of differences in evaluations, a third coder settled the dispute. All coders were familiar with the instrument and participated in the model design.

The Merco healthcare reputation ranking was used as a reference because it is the only specialized annual multi-stakeholder corporate reputation monitor in Spain with an open data methodology audited by KPMG.

3. Results

The work carried out has allowed, from the literature review, the construction and validation of the Hospital Web Quality Multicriteria Analysis Model (HWQ), the elaboration of an analysis table based on the literature review of web quality analysis models specialized in hospitals and health centers, as well as the most current taxonomies to analyze digital media. This research has led to the creation of a table consisting of three thematic dimensions, 14 indicators, and 56 dimensions of analysis (

Table 8. Hospital Web Quality Multicriteria Analysis Model (HWQ).

Dimension	Indicator	Content	Validation
Content	Basic information about the hospital	Hospital name in the header	Checklist
		Address	Checklist
		Telephone numbers	Checklist
		E-mail address or contact form	Checklist
		Description of the hospital activities (mission, vision, values, corporative objectives, achievements)	Checklist
		List of services and specialties offered by the hospital (emergency, surgical, outpatient, diagnostic imaging, laboratory, rehabilitation, etc.)	Checklist
		Hours of operation	Checklist
		Information on health insurance coverage and coverage for treatment or consultations or prices/fees for medical consultations or interventions	Checklist
	Transparency	Identification of the type of management, sponsors or promoters	Checklist
		Hospital history	Checklist
		Corporate annual report for the previous year	Checklist
		Institutional accreditations and/or awards	Checklist
		EFQM, ISO, or AENOR quality assessment	Checklist
		Patient quality assurance statements	Checklist
		Indication of copyright and intellectual property of the website and its content	Checklist
		Hospital data (patients attended, health personnel, number of beds, number of operating rooms, scientific publications, etc.)	Checklist
		Statistics (successful operations, average waiting time for consultations)	Checklist
		Results of previous years’ satisfaction surveys	Checklist
		Website privacy policy, data protection, terms and conditions, and cookies policy	Checklist
		Privacy and confidentiality of data statement.	Checklist
		Ethics committee: members, protocols, guidelines, and procedures	Checklist
	Institutional information	Organizational chart and hospital structure	Checklist
		Hospital authorities and governance	Checklist
		Departmental directory: names, description, education, experience, responsibilities, and contact details	Checklist
		Link to press or communication department	Checklist
		Hospital news section	Checklist
		Institutional agenda or calendar	Checklist
		Identification of collaborating entities or strategic alliances	Checklist
		Research groups or teaching units	Checklist
		Corporate social responsibility activities	Checklist
		Job portal	Checklist
		Testimonials and comments on successful patient experiences
	e-Health resources	Health library (symptoms, conditions, treatments, diagnoses)	Checklist
		Materials on preventive medicine and prevention programs	Checklist
		Interactive materials (videos, infographics) on diseases, symptoms, treatments, maternity.	Checklist
		Clinical research in which the hospital or its staff participates	Checklist
Navigability	Basic aspects	Site map	Checklist
		Superior menus	Checklist
		Language change (minimum Spanish and English)	Checklist
		Internal page search menu	Checklist
	Readability and UX	Legibility and ease of user navigation on the website	Likert scale.
		Help and documentation	Checklist
		FAQ section	Checklist
		Suggestions and claims mailbox	Checklist
	Medical staff	Directory of healthcare services and professionals: names, responsibilities, and contact details	Checklist
	Appointments for external consultation	User/patient access portal (intranet)	Checklist
		Platform for managing outpatient appointments	Checklist
	Website loading speed	−10 s to load the home page	Checklist
	Link performance	No broken links or links with errors	Checklist
	Socialization	Links to hospital social networks	Checklist
Aesthetics	Responsive design	Mobile and tablet adaptability	Likert scale.
		Ease of navigation in mobile format	Likert scale.
	Corporate visual identity	Use of corporate logo, font, and colors	Checklist
		Visual and iconographic coherence	Checklist
		Aesthetic integrity	Checklist
	Menus and links	Menus and links are clear and organized. Information is easy to find.	Likert scale.

Source: Author’s elaboration.

), which offers a valuable proposal for future research using the walkthrough method or similar techniques to examine the websites of hospitals and health centers.

Based on the above and the benchmarking and walkthrough method, the research has applied the proposed table to check its applicability and reliability to the websites of the ten public and private hospitals with the best reputations in Spain.

3.1. Content Dimension

Regarding the basic information provided on hospital websites (see Figure 2), all the centers analyzed presented key data on the name, address, contact telephone numbers, and e-mail address. These data, which are located in different sections or headings, nevertheless acquire different prominences in each case. The main shortcomings are, in general, and on the websites as a whole, information on operating hours, medical insurance coverage and treatment coverage, or prices/rates for medical consultations or interventions, which are practically non-existent. Although the Hospital Universitario Quirón Salud of Madrid does not offer prices, it has a section on offers and promotions.

Regarding “Transparency” (Figure 3), the research reveals the non-existence of content related to identifying the type of management, sponsors, promoters, or the corporate report of the previous year. Concerning institutional accreditations and/or awards, there are allusions to these recognitions and accreditations in different ways, both through autonomous sections and in spaces dedicated to news. However, it is noted that the meaning, characteristics, and scope of a non-specialized public have not been clarified.

On the other hand, it is essential to point out that almost no hospital refers on its website to statistical information, such as successful procedures or average waiting time for consultations. At the same time, there are no reports on the aspects of patient quality assurance statements or the results of previous years’ satisfaction surveys.

Half of the websites studied presented clear information regarding the indication of copyright and intellectual property of the website and its content. However, news stories are not signed in any of the cases, except for one hospital, which, as credit, indicates the name of the center itself. There is content on more than half of the websites related to data, such as patients attended, health personnel, number of beds, number of operatory rooms, or scientific publications. At the University Hospital Gregorio Marañon of Madrid, they list the number of beds or operating rooms but do not take advantage of infographics’ informative capacity or more visual formats.

Finally, in the “Transparency” category, the study highlights that almost all the websites of the hospitals analyzed present information on the website privacy policy, data protection, terms and conditions, and cookies policy. Along the same lines, the aspects linked to the privacy and confidentiality of data statements were detailed for all platforms. To a lesser extent but with a notable presence, are aspects related to the ethics committee (covering aspects such as members, protocols, guidelines, and procedures).

Sanitas La Zarzuela University Hospital and University Hospital 12 de Octubre in Madrid are among the few hospitals that mention environmental management.

As a responsible part of society, it is aware that its healthcare, teaching, and research activities must be carried out within a framework of protection and respect for the environment, which is why it has established an environmental management system certified following the UNE-EN ISO 14001 Standard and is also part of the exclusive European Register of Organizations Adhering to the European Union Eco-Management and Audit Scheme, EMAS.

For its part, the Hospital Vall D’Ebron in Barcelona sets out its principles with references to “people”, “innovation”, “excellence,” and “knowledge”, but makes no mention of ethics.

The analysis of institutional information offers interesting data that reveals some of the main communication weaknesses of the websites of the hospitals analyzed (see Figure 4). On the one hand, practically half of the centers offer information on the organizational chart, hospital structure, hospital authorities, and governance. However, at a similar percentage, there are aspects related to a departmental directory, which usually include names, descriptions, education, experience, responsibilities, and contact details. In the case of the Hospital Universitario Quirón Salud Madrid, a great job of content curation can be seen in the presentation of its staff. Along with a photograph of each staff member, it summarizes their CV in prose.

Regarding news and similar services, less than half of the hospital websites studied had a link to the press or communication department. However, it is important to note that all of them had a news section. It should be noted that in all cases, the content is unsigned and uncredited. In addition, although the news posts present the date, they forget to credit photographs or other resources. In the case of Hospital Clínic de Barcelona, a press office includes a guide for journalists and specific contact details for those responsible for the hospital’s communications office.

The institutional agenda or calendar, corporate social responsibility, activity groups or teaching units, and job portals exist in less than half of the centers analyzed. Similarly, the identification of collaborating entities or strategic alliances is scarce. Only two centers presented testimonials and comments on successful patient experiences. The presence of testimonials was only detected in two of the hospitals analyzed and was supported by videos hosted on YouTube.

The e-Health resources section (see Figure 5) presents the results of interest in improving the information available on hospital websites. Practically, all the centers studied have a health library that offers information, primarily textual, on symptoms, conditions, treatments, diagnoses, and materials on preventive medicine and prevention programs. Only videos and some infographics appear regarding interactive materials (videos, infographics) related to diseases, symptoms, treatments, and maternity.

The inclusion of more innovative formats is practically non-existent. The Hospital Universitario Sanitas La Zarzuela and Hospital Universitario Quirón Salud in Madrid offer virtual visits that do not operate. In the case of Hospital Clínic de Barcelona, there is a section of “health podcasts”, an aspect that connects with a type of format that is currently widespread among the public and which allows a very autonomous, non-sequential, and flexible consumption of information (Figure 6):

Hospital Universitario Quirón Salud Madrid, Hospital Universitario Sanitas La Zarzuela, Hospital Vall d’Ebron, or Centro Médico Teknon Quirón Salud present maps of their facilities but do not take advantage of the interactive potential (see Figure 7). These are flat images that lack the depth made possible by hypertext.

3.2. Navigability Dimension

Regarding navigability, the absence of sections for frequently asked questions (FAQs) and the lack of suggestions and claims mailboxes stand out. The first element is crucial, particularly in solving the most common doubts and recurring aspects among users of this type of site. On the other hand, hospitals should analyze and improve the absence of spaces for submitting suggestions or complaints, beyond generic e-mail forms, to encourage horizontal dialogue and to open diversified channels to improve the services offered. All hospitals present links to their respective social networks in their websites’ upper and lower frames (Figure 8).

The Hospital Universitario La Paz in Madrid highlights the existence, on its website, of a “library”, conceived as an “attractive space where both patients and their companions can find publications appropriate to their age, as well as an environment that encourages reading”. The center offers a book-lending service for children and adults, with over 5000 titles in all literary genres.

The Gregorio Marañón University Hospital also has a library “specialized in Health Sciences, whose aim is to provide the center’s staff with the scientific information they need to carry out their healthcare, teaching, and research activities”. Finally, it highlights the multilingual nature of most of the websites, which, in some cases, offer content in up to five languages. Conversely, the presence of numerous broken links in about half of the sites analyzed is an aspect that points to the need for permanent editing of the content.

3.3. Aesthetics Dimension

The “Aesthetics” section shows that the hospitals take care of corporate colors and logos that give the platforms a cohesive use of fonts and colors. Thus, visual and iconographic coherence were present on the platforms. Specifically, in this section, there is a need to improve the adaptability of the content to mobile and tablet screens, an aspect that occasionally generates problems in accessing the content.

4. Conclusions and Discussion

This study concludes that the hospital’s information, communication, and services are concentrated, with a few exceptions, on a series of recurring aspects in all the centers analyzed. This uniformity is also observed in content curation, which hardly adapts to the guidelines and editorial guidelines for small screens. In addition, there is a notable presence of photographs that are not very personalized, are derived from photographic banks, and are, therefore, far removed from the everyday lives of health centers. This aspect is also connected with a predominance of gray texts that hardly exploit the use of bold, internal, and external links, which are critical aspects of organic positioning in search engines or SEO.

In addition, in almost all cases, texts are very long and do not invite reading on the screen or “small” screens. Visual or multimedia resources such as infographics or interactives are practically non-existent, sacrificing one of the main potentialities of Internet content. Added to this is the absence of taxonomy or folksonomy work, which is crucial for labeling and subsequent content recovery through keywords. Capitalization is another element that should be reviewed in this section regarding adaptation to screen-mediated communication. This set of reflections supports the postulates of Díaz Cuenca and Chain Navarro (2008) [8], who warned of the lack of cohesion, rigor, and informative quality in the criteria for generating hospital web content.

On the other hand, it is important to note that the study, except in the case of one of the hospitals studied, did not find sections that played a prominent role in the coronavirus. Likewise, a structural and physical design aimed more at patients and not at accompanying persons and family members, aspects that centers such as the Clínica Universidad de Navarra or the Hospital Universitario La Zarzuela include in their digital offer of content and services. Aspects linked to accessibility are only directly addressed on some hospital websites, while there is a growing use of documents in PDF format, making browsing difficult on mobile devices. These are aspects that reinforce the preeminence of informative proposals over communicative ones, as a result of works such as those of Cardelle and Rodriguez (2005) [11], García-Lacalle, Pina, and Royo (2011) [10], Calvo-Calvo (2014) [9], and Tejedor et al. (2020) [35]. However, improvements have been focused on intranet space or services based on contact forms.

Except for specific proposals, such as the use of WhatsApp, there is little or no commitment to disruptive information formats or dynamics. This aspect allows us to conclude by stressing the importance of betting on new content, such as gamification, Virtual Reality (VR), or Augmented Reality (AR), or simply by taking advantage of interactivity and multimedia. In this sense, the study stresses the need to generate new stories or thematic approaches that, on the one hand, bring the centers closer to their daily life through more personalized storytelling, and on the other hand, humanize them with testimonials and real cases.

In addition to the quantitative analysis, this study carried out a qualitative selection of specific cases or examples directly related to some of the more specific and unique aspects identified in the sample. This selection was carried out by the researchers in search of concrete cases that allowed them to visualize examples and, therefore, respond to a discretionary criterion or quality; that is, one that is not subject to a rule but to the researcher’s criteria.

Regarding the model’s applicability, based on its theoretical and practical implications, this study verifies its feasibility and adequacy to the crucial aspects of the fundamentals of digital communication. However, it is essential to note that the proposal focuses on websites and not on social platforms, an area that would require an analysis table with other approaches and thematic dimensions. In addition, there is an interest in projecting this study to other contexts, both globally and regionally, that is, from a global study to works that focus on Latin America, Central America, and the Caribbean, for example, where the differentiation between public and private hospitals could also connect with the communicative proposals of their respective websites.

This study opens up the possibility of generalizing the HWQ model to hospitals in other countries. In this sense, the study allows for the feasibility and applicability of the proposed model in different socio-cultural contexts. This aspect is directly linked to the general and transversal nature of the analysis categories. However, in line with Ford et al. [23] and Tejedor, Romero-Rodríguez, and Codina [35], the design of taxonomies of analysis for web interfaces must also consider aspects specific to the idiosyncrasies of the contexts being addressed.

Based on the above, the results of this study allow us to outline future research based on the findings of this study. On the one hand, the relevance of investigating possible adaptations of the HWQ model to other types of healthcare institutions can be observed. In the framework of a collaborative scenario open to permanent interaction with users, it would be of interest to analyze the integration of patients’ opinions in the evaluation criteria. Finally, it is important to promote studies that analyze the communicative work carried out specifically on social networks, on the one hand, and the synergies established with the main website, on the other. In short, this work inaugurates several lines of research from a communicative perspective and, specifically, from questions linked to concepts such as storytelling, formats, Artificial Intelligence (AI), or developments linked to Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) in a scenario marked by the leading role of Big Data.