Health Technology Assessment for Medical Devices

Summary

Health Technology Assessment (HTA) methods have become a standard part of decision-making processes in healthcare service. Although it is routinely applied in drugs and surgery, HTA in medical devices is still quite challenging. The reason is that the main objective of HTA studies for devices is not optimization of the cost-effectiveness ratio, but rather decisions about procurement and/or incorporation of the device. The clinical benefit is not expressed in terms of quality of life, but in the rate of diagnostic yield, and in the extent to which the technology makes the therapy shorter and/or more patient-friendly. Utilization of multiple-criteria decision-making methods for evaluation of the aggregated clinical, technical and user´s effect (outcome) is recommended as the input to cost-effectiveness analyses. Different methods are derived for strategic and/or operational assessment of new technology. Other studied problems are identification of requirements for medical device selection and purchase, composition of expert panels, and assessment of medical device maintenance demandingness.

Keywords:
health technology assessment, medical device, cost-effectiveness analysis, multiple-criteria decision-making, analytic hierarchy process

Introduction

Limited resources and continuously growing costs in health care have led to the necessity to assess effectiveness, appropriateness and costs of health technologies, i.e. drugs, biologics, devices, equipment and supplies, medical and surgical procedures, support systems, and organizational and managerial systems in health care [20, 22].

Health Technology Assessment (HTA) comparing clinical outcomes and costs of interventions has established as a standard method in drug reimbursement process worldwide since the 1990s [22, 56]. However, its potential is much wider. From the beginning, it was stressed that this method can be used to assess all kinds of health technologies, which has been attempted with varying success. HTA utilization to medical devices has been a challenge until today [14, 39, 46].

This paper summarizes the results reached in this field by the CzechHTA group within last five years, when they concentrated on HTA methods applied to devices under a grant of the Ministry of Health of the Czech Republic.

Principal methods of HTA

The core methods of HTA are different cost analyses [8, 22, 56]. The frequently used ones are listed in Table 1.

The most widely used method is the cost-effectiveness analysis (CEA). It compares costs, C, expressed in monetary units, with outcomes (also called effects), E, expressed in natural units in the cost-effectiveness ratio, C/E:

If two interventions are to be compared (e.g. a new drug with the standard drug, a surgery with a conservative intervention, or two different imaging modalities), the incremental cost-effectiveness ratio, ICER, is often calculated instead of two full C/E ratios:

Thus, ICER is the ratio of the change in costs to incremental benefits of the intervention. The cost-utility analysis (CUA) and the cost-benefit analysis (CBA) are special kinds of CEA. In CUA, the outcome (called the utility in this case) is expressed in QALYs, quality-adjusted life years, which is a parameter combining the length of life gained (in years) with its quality. One QALY is equal to a one-year life in full quality, or to a 2-year life in 50% quality, or for example, to a 15-month life in 80% quality. The utility is measured by patient self-reporting; the best-known surveys are EQ-5D by EuroQoL and SF-36 by the Medical Outcomes Trust. CUA is considered a standard for assessments of new pharmaceuticals. In other technologies, however, other analyses may be more frequent.

The cost-benefit analysis (CBA) is based on expressing the outcome in money units. Its roots are in the welfare economics, and it is frequently used by governments to appraise their policies and/or investments (for typical applications, see e.g. [19, 24, 57, 64]). In HTA, however, it has been used much less frequently, as it requires expressing the intervention outcomes in monetary units. This, in fact, means finding an equivalent for the quality of life (and consequently for the human life) in a monetary form. Many criticize this approach from the ethical point of view [9, 25, 63].

The challenges of cost analyses are sometimes avoided by using the cost-consequence analysis (CCA). This approach resigns from the ambition of calculating one ratio; instead, separate lists of (various) outcomes and costs are compiled. Hence, it is a more descriptive method.

Specificities of medical devices

Medical devices can be divided according to their purpose to therapeutic and diagnostic. While the former group allows assessment of clinical outcomes similarly to drugs, the situation is much more complex in the latter group. These are usually relatively expensive technologies requiring the perspective of a hospital or a region as a whole. Thus, the main goal of HTA studies is not to optimize the cost-effectiveness ratio, but to support decisions about procurement and/or incorporation of the device. The clinical benefit is not expressed in terms of quality of life, but rather in the rate of diagnostic yield. Another important issue is also the moral lifetime of the device; rapid upgrades pressurize researchers into assessing devices immediately, without much experience, and in a shorter time, which does not allow for sufficient clinical experience [14, 39, 46].

Recently, Santos, Tavares et al. published a series of papers describing the specificities of medical devices from the point of view of the product development process [53, 54]. They point out above all that medical devices create a regulated industry, similarly to the automotive or nuclear industries; this is a very dynamic branch with permanent changes and a short lifetime of products. Manufacturers have many obligations even after selling the device, namely post-market surveil-lance and adverse event reporting. In case of adverse events, they should take action. Moreover, the field is very sensitive to ethical and economic issues. Similar problems appear if we decide to submit medical devices to HTA methods.

In 2009, Value in Health published two papers summarizing arguments for equal or different character of drugs and medical devices in economic evaluations [14, 62]. While Taylor and Iglesias [62] stood up for an equal approach, Drummond et al. [14] listed the main problems that a researcher encounters when assessing medical devices. The following list is a slight modification of their reasons, why assessments of medical devices differ from assessments of pharmaceuticals:

• many medical devices are diagnostic, hence the outcome cannot be separated from the treatment and, moreover, most such devices have multiple applications;
• due to short life cycles in medical devices, their frequent modifications, and the existence of “learning curves”, there is unlikely to be a substantial steady-state period, during which the device could be evaluated in a randomized controlled trial;
• in addition, it is generally impossible to undertake blinded studies with medical devices;
• the efficacy of a device depends not only on the device itself, but also how it is used (e.g. the skill and experience of the surgeon);
• implementation of a new therapy involving a device can have wider economic implications;
• equivalent clinical evidence may not be available for all products, making comparisons difficult;
• prices are likely to change over time, because new better products enter the market, or because of the ways, in which procurement takes place in many health care systems.

Especially the problems with carrying out larger randomized controlled trials can be hardly overcome, and the problems with determining QALYs due to impossibility of patient self-evaluation disqualify the CUA. The situation calls for a special methodology for HTA studies in medical devices, quite different from that for drugs [46].

Example 1. CEA/CUA for an electronic (C-Leg) versus mechanic prostheses [36]. A socio-economic comparison of the electronic prosthetic joint C-Leg and a mechanical prosthetic joint in patients with a transfemoral amputation can serve as an example of a typical HTA study. As the artificial knee can be classified as a therapeutic device, the cost analysis causes no significant difficulties. The study involved 42 patients with a high grade of activity in six prosthetic centres in the Czech Republic. The time horizon was four years (the lifetime of special femoral prostheses) and the effect was assessed using the PEQ questionnaire that has nine dimensions [6]. The dimension of walk was used as the effect in the CEA. Next to that, QALY was assessed using the EQ-5D questionnaire [16], and used in the CUA. Costs were calculated from the perspective of a health insurance company. The results of the cost-effectiveness analysis and the cost-utility analysis are shown in Table 2.

The cost-effectiveness analysis speaks in favour of C-Leg, while the cost-utility analysis slightly prioritizes the mechanical prosthesis. However, the ICER value calculated from the cost-utility data is 176 628 CZK/QALY, which is far below the recommended threshold of three times GDP per capita for each DALY¹ averted (i.e. QALY gained) [68]. From this point of view, C-Leg is a cost-effective option.

---

¹DALY, disability-adjusted life years, can be practically understood as inverse values to QALY; however, these terms originate from different disciplines, and hence they can give slightly different values [55].

---

Multiple-criteria decision analysis as the main method in medical device assessment

The principal difficulty in any evaluation of technology consists in solving the problem of evaluating the effective component of instrumentation. Instrumentation may not directly affect the parameters associated with the quality of life, nevertheless, it affects the quality of therapeutic and diagnostic processes, the attending physician’s way of working and, last but not least, the patient’s comfort [36]. When evaluating the quality of medical devices, data such as technical specification of individual devices, information about their control, failure rate, quality of results, etc. are largely analysed. No standard procedure has been decided for such evaluations yet.

In this paper, evaluating the effect of medical devices using multiple-criteria decision analysis (MCDA) [23, 60] is suggested. The multiple-criteria decision-making (MCDM) techniques have been used in both evaluation and selection of equipment [30, 35]. Balestra et al. [1] applied the analytic hierarchy process (AHP) to support the acquisition of pacemakers and implantable defibrillators. Chatburn and Primiano [10] used the AHP to select and purchase ventilators for an intensive care unit. Pecchia et al. [43] designed a CT scanner selection approach based on 12 specifications. Montevechi et al. [40] used AHP for ultrasonic scanning system selection in private hospitals in Brazil. Sloane et al. [59] demonstrated how to build a neonatal ventilator evaluation model. In addition, Santos and Garcia [52] used AHP, multi-attribute failure mode analysis (MAFMA) and elimination and choice expressing reality (ELECTRE) to demonstrate a decision model for incorporating indicators into the acquisition of hospital medical equipment.

These and other studies indicate the possibility to use MCDA in HTA. A frequently discussed option is a total replacement of CEA with MCDA [2, 66]. However, this approach challenges serious consideration of cost [11]. Instead, we recommend maintaining CEA as the main tool, and utilizing MCDA only for evaluation of the effects (outcomes) [45]. Thus, the value used for decisions will be the ratio of outcomes evaluated by means of MCDA, and costs in their natural expression. Next to MCDA, value-engineering methods can also be used. This approach allows us to assess medical devices based on their technical data, which is impossible in the standard pharmacoeconomic methodology.

Two different procedures are recommended. A full AHP process combined with the Delphi method is time and capacity consuming [30], and can be used above all for strategic decision-making [35]; this will be described in the next section. A simplified procedure suitable for operative management at the hospital/clinic level utilizes simpler MCDA methods [32, 45]; this approach will be the topic of the next but one section.

Medical device evaluation for strategic purposes

In order to be able to evaluate medical devices, the problem must be described in a satisfactory detail. The knowledge of the values of all selected criteria (specifications) is essential for all individual device models to be covered by the evaluation. The availability of the assessed models in the respective market is usually also assumed. A satisfactorily detailed description must also be provided if the analysis is calculated in any field different from medical devices.

To determine the weights of the main specifications of the medical device assessed, we propose, within the framework of MCDA, to use a web-oriented means of polling, using Saaty´s analytic hierarchy process (AHP) [49,50,51] and the Delphi multi-round polling procedure [12]. In the first round, the experts compare the key parameters for the selection of the medical device and then compare the available choices relative to each parameter. Data received from the experts are processed to determine the mean and extreme values of the evaluations. The first-round results are reported to the experts. If an expert’s evaluation deviates significantly from the weighted mean value, the expert is asked to either provide reasons for the opinion or to correct it. Consecutive rounds follow the same scheme. According to [42], after three to four rounds, the responses become stable and do not require any more changes usually. During the survey, the experts’ responses remain anonymous, in order to avoid the influence of authorities.

MCDA methods are designed for a mutual comparison of several (n > 2) variants (e.g. devices). Each variant is described by a set of parameters – hence, we obtain a matrix, the rows of which are formed by individual variants, and the columns by values of parameters. Several requirements are established for the parameter values, namely the optimising criteria. These criteria can be maximising (the higher the value, the better), or minimising (the lower the value, the better). Our task is to order the variants based on these criteria from the best to the worst (it is often sufficient to find the best variant) [23].

The Delphi method does not lead to a full agreement in experts’ opinions. Here, we take into account the opinion of each expert individually. We compare individual pairs of criteria in a square matrix, where we determine the preference measure of the row criterion over the column criterion. Then we complete the matrix symmetrically according to the main diagonal with inverse values, and place the value one to the main diagonal (no criterion can dominate itself); what we create is the so-called Saaty´s matrix (see Table 3) [49]. The resulting weights of criteria are obtained as geometric averages of the lines that are subsequently normalized.

After selecting and evaluating the criteria, the group of experts evaluates each alternative (i.e. each device included in the assessment process) according to each selected criterion. The result consists of n Saaty´s matrices, where n is the number of criteria. It is necessary to check the consistency of each obtained Saaty´s matrix. We assess this using the consistency index CI:

where λ_max  is the largest eigenvalue of the Saaty´s matrix and n is the number of alternatives. The matrix is considered consistent if CI/RI < 0.1, where RI is the consistency index in the case of randomly generated results. Otherwise, the expert responsible for the inconsistent value is asked to correct his/her evaluations [29].

Eventually, the results for individual criteria are combined using the previously determined weights of the criteria. We can also determine the weights of individual experts in the expert group. In this case, the weights of experts are also used in this step.

Example 2. Selection of a mammography device [36]. Let us illustrate this procedure using a model selection of mammography equipment. The list of parameters important for the choice was determined taking into account expert opinions, requirements of medical facilities in the past and the literature review (especially [15]). The following parameters were chosen: (i) the X-ray focal spot size, (ii) the detector size, (iii) the modulation transfer function (MTF), and (iv) the detective quantum efficiency (DQE). Other parameters considered important (space necessary for device installation, device mass) were omitted, as they are important only in the case of technical limitations. If necessary, the evaluation of technical parameters can be extended adding further parameters.

The assessment was done for five devices that are currently available in the Czech market and that are destined for screening centres. Their producers and exact commercial names are not important for methodology illustration. Their parameters are given in Table 4.

The next step is the proper evaluation of the selected parameters importance. Individual qualitative and quantitative characteristics are represented by colour bands; the evaluators (five experts were consulted; their choices will be discussed below) can move them using the cursor, which expresses their preferences between each compared pair of parameters. Subsequently, the evaluators compare the assessed devices for each parameter in the same way. The results are presented in the form of Saaty´s matrices (see Table 5 and Table 6).

The normalized weights of parameters of individual devices are included in one matrix (see Table 7), and this is multiplied by the vector of normalized parameter weights (see Table 5). If we are given the weights of expert competences (in this case they are given in Table 8), we can use them in the last step of the analysis (otherwise, this step is skipped). The final order of devices is given in Table 9.

Medical device evaluation for operational purposes

Although the method described in the previous section seems to be quite sophisticated, it is also time consuming. However, the selection must often be carried out in a short time. In this section, we present an alternative procedure that represents a compromise between the precision and time requirements. The basic procedure and the links between individual methods used are displayed in Figure 1. Its individual blocks are described further below.

The procedure consists in a combination of a value engineering and a MCDM methods. Different methods were tested [32, 45]; the best combination proved to be Saaty´s matrix [3, 23] used to determine the parameters’ normalized weights, combined with the TOPSI² method [3, 23] used for evaluation of the devices themselves.

---

²TOPSIS is an abbreviation of the Technique for Order of Preference by Similarity to Ideal Solution, originally suggested by Hwang and Yoon [27].

---

The method is intended for hospitals making decisions about purchasing a new device. The group taking the decision is usually composed of a senior consultant (or more clinicians), a biomedical engineer, the financial manager, and a charge nurse.

The Saaty´s matrix method is based on an evaluation of relative significances; decisions on how many times the i-th indicator is more significant than the j-th one are entered in the respective fields of a square matrix by integers v_i. The other value of the compared pair is set as the inverse value 1/v_i. Such a matrix is shown in Table 10.

The evaluation itself is performed by means of eigenvalues of Saaty´s matrix. Individual steps of the calculation are as follows:

1. The sums of all n elements s_ik, of each k-th column of Saaty´s matrix are calculated,
   

   

2. Individual elements in each column are divided by these sums. We obtain elements t_ij of a new matrix T,
   

   

3. Now we calculate line sums of the matrix T, i.e. the sums of all elements t_ij in the j-th line,
   

   

   
   and the sum of all t_ij in the matrix T:
   

   

4. Quantified values of relative significances of indicators v_j are subsequently calculated by the normalisation of the sums in individual lines,
   

   

The TOPSIS method is based on selecting the alternative that is closest to the ideal alternative represented by the vector (H₁, H₂,…, H_k), and farthest from the basal alternative represented by the vector (D₁, D₂,…, D_k). First, the normalized criterion matrix R=(r_ij) must be created using the formula

where i = 1,2,…,p and j = 1,2,...,k. After this transformation, the columns of the matrix R form unit vectors. Subsequently, the weighted criterion matrix W is calculated so that each j-th column of the normalized criterion matrix R is multiplied by the corresponding weight v_j, that is

The next step is a determination of the ideal (H) and the basal (D) alternatives with respect to the weighted criterion matrix W. The formula for calculation of the distance from the ideal alternative is

and the formula for calculation of the distance from the basal alternative is

The distances from the basal and the ideal alternative are used for computation of the relative indicator of the distance from the ideal alternative c_i using the formula

Now, individual variants are ranked according to c_i’s from the most suitable variant to the least suitable one.

Example 3. Lung ventilator assessment [45]. The data for lung ventilator assessment were collected from two university hospitals in the Czech Republic and from companies present on the Czech market, based on a detailed market research. Similarly to Example 2, the producers and exact commercial names are not important for methodology illustration. The source data for the lung ventilator assessment are given in Table 11, and the prices of the devices are shown in Table 17.

In the cost-effectiveness ratio, costs are shown in the numerator, while effects are represented in the denominator. However, in this example, some criteria (effects in the cost-effectiveness language) are also prices. These are the parameters that can significantly affect the economy of the apparatus operation. In some devices, the purchase price is low, but the consumables and other operational costs may be much higher. In the effect part, not absolute values, but only relations between individual devices in each criterion separately are taken into consideration. Hence, each criterion can be related to other units (e.g. 24 hours vs. one year).

The respective Saaty´s matrix is given in Table 12, and the weights for individual parameters are shown in Table 13. All price effects must be assigned the same weight. The reason is that a monetary unit spent presents the same burden regardless of what we spend it on. Subsequently, these weights are utilized as the input to the TOPSIS method to find the most suitable device. The criterion matrices R and W are given in Table 14 and Table 15. The resulting effects for individual devices are given in Table 16.

CEA is used, and the decision is made based on the effect-cost ratio. The actual prices are substituted for the costs, while the results of the MCDA are considered the values of the effects. The result is shown in Table 17. If we compare outcome for money, Device #4 shows the best results; it exhibits a well-balanced effect and price, while the cheapest, Device #1 only ranks third.

Identification of requirements for medical device purchase

The methods presented in the previous sections can be used for a medical device assessment in any phase of its life. Mostly, they are applied at the moment of a new device introduction to the health care practice at the national regulator level. However, the same methods can be employed at the level of an individual clinic, hospital, or a group of hospitals when deciding about investments, above all about purchasing new technology. Then they can be utilized to identify necessary parameters of the new device, selection of the supplier, or the particular brand.

Rational decisions in medical device selection are based on a proper identification, analysis and evaluation of requirements of various involved actors. The types of requirements are described in [4]. Commercial reasons describe why the project is initiated, the objectives to be reached, and the methods to be applied to measure the satisfaction.

Commercial conditions describe the needs of the organization as a whole. They are developed and defined by means of a company analysis. Requirements of involved parties describe needs of individual parties, and how the parties will collaborate in the solution. Functional requirements describe the system behaviour and the required functions it will be able to satisfy. Non-functional requirements are those that are not directly related with the system behaviour or do not affect its functions (e.g. requirements for capacity, speed, safety, availability, or information architecture). Transitional requirements describe conditions that are necessary for the transition of the current state to the final one, but will not be needed once the transition has been completed.

The requirements management process consists of four basic stages, which are as follows [21]: (i) identification; (ii) analysis; (iii) prioritization; (iv) validation. The whole process is shown in Figure 2. The process can be repeated several times, as new knowledge, new interested actors, and new requirements can repeat during the application.

Identification. It is important to understand the requirements from the point of view of different types of knowledge and functions. A mnemonic tool called CATWOE can be used [50]. This is a combination of intuition and real-world experience [4]. The goal of the CATWOE method is to obtain the so-called rich picture, where the problem is described from all angles.

The ambition is to identify all requirements which might be significant in the solution of the problem.

Analysis. The goal of the analysis is to disclose all objectives, system functions, structures and relations to the neighbourhood, and to enhance system phenomenon algorithmization.

Prioritization. The next step is requirement prioritization. There are several ways to do this that differ in time requirements and precision. Each particular case requires that a suitable method is selected. One possibility is the cost-value approach [33] using the analytic hierarchy process [51] (see above). Another prioritization method is MoSCoW [7] selecting the requirements to four classes (must, should, could, won’t).

Validation. Requirement validation is the last step. It checks whether the equipment chosen fulfils all requirements. The outcome of the process should be a coherent model of requirements responding as much as possible to the expectations and possibilities of the uses. This creates a compromise between the requirements and what can be carried out [26].

Expert group composition

The methods described above depend mostly on a consensual decision of a panel of experts. Natural questions appear whether and how the composition of the panel affects the final decision. Although these questions have been studied for quite a long time (see e.g. [65]), the appropriate panel composition remains a controversial issue requiring critical reflection [17]. The principal problems are (i) the size of the panel, (ii) the expertise of the panellists, (iii) the consistency of the panel and (iv) whether the selection of panellists should be internal or external [29].

The size of the panel is important for a reliable and quality assessment of all criteria and alternatives [29]. A small panel can rapidly come to a consensus, but may not take all details into consideration. In the case of an excessively large panel, there is a risk that the assessment time will be extended, or the panel may not come to a consensus within the given number of rounds. Of course, large panels also increase the cost of the process. Kendall’s W assessing agreement among raters [34] can be used to determine the optimum expert panel size [29]. The W value ranges from zero (no agreement) to one (full agreement). We determine H in such a way that P(W>H)=α, where α is the chosen significance level of the statistics W. Practical recommendations for calculation of W can be found in [58].

The expertise of the panellists has been discussed since the 1980s without clear results. It is obvious that the panellists’ expertise affects the final decision, but the advice derived is rather weak [5, 18, 42, 65]. Due to a lack of quantitative methods, this paper’s authors based their selection on Russian sources [41, 42] stressing the formal side of expertise. While the methods used [36] seem to be appropriate, the details of the evaluation are affected by a different economic and cultural realty, and should be remodelled for utilization in Europe. Moreover, social psychology has demonstrated that expertise itself may not be sufficient to obtain accurate judgment, but should be combined with diversity [37, 69]. Diversity in a panel proves to be beneficial for obtaining useful results in several ways: the advice comes from multiple independent sources, and the panellists have different backgrounds, skills and points of view. The definition of diversity, or inconsistency, usually involves demographic characteristics as well as aspects related to the individuals’ professional experience [18, 38, 69]. Group size plays a critical role in determining the optimal group: in small groups, the most accurate type should be in the majority, whereas in large groups, heterogeneity may dominate [29, 37].

The least discussed question is whether the experts should be chosen from inside the company, or whether they should rather be external. Recently, this was studied experimentally by Förster and von der Gracht [18]. According to their results, internal Delphi surveys are applicable when the time frame is limited or resources are restricted. Internal topics can be discussed, and information is kept confidential; moreover, less time is needed to recruit participants and their response time is shorter. External Delphi panels should be consulted when numerous perspectives on different subjects are desired. Conducting Delphi surveys with both internal and external panels offers the most diverse and plentiful opportunities for decision makers. Differing expectations can be analyzed to find group patterns and match strategies more aptly.

Demanding nature of medical equipment maintenance

When assessing the economic effectiveness of medical devices, next to the purchase price, cost of service and consumables have an important role. Medical device maintenance planning is crucial, while legislation, producer’s instructions and the healthcare facility’s own experience should all be considered. Healthcare facilities should pay particular attention not only to the device’s safety, but also to maintenance cost effectiveness. Many hospitals have begun to use more cost-effective methods of maintenance planning, different from producers´ instructions.

Medical equipment maintenance demandingness assessment is a multiple-criteria decision-making problem [61]. The quality of results highly depends on the availability of input data; thus, implementation of these methods in practice depends on the device operation data collection system. The input data are:

• healthcare facility registration information on medical devices;
• statistics concerning breakdowns and servicing, including their cost; and
• statistics of operation for each device.

The proper procedure is again based on AHP [49, 51]. The implementation of the medical devices maintenance demand assessment consists of the following steps:

1. Identification of all sufficient and independent criteria. The first step in AHP application is designing the hierarchic structure of the criteria. All assessment criteria should be identifiable and placed at the proper level in the hierarchy. Basic criteria are located on the second level; each criterion can be divided into several sub-criteria of a lower level.
   There are a few approaches to criterion selection. Fennigkoh and Smith proposed three basic criteria: function, physical risk and maintenance requirements [17]. Wang and Levenson introduced two new criteria, namely incident history and utilization rate, and reinterpreted the function criterion [67]. A new study recommends the following six basic criteria: function, critical mission, age, risk, recalls and alerts and maintenance requirements [61].
2. Criterion weight setting. The method of paired comparisons is used for setting the weights of individual criteria and sub-criteria on all levels, and for individual criterion scale quantification (item 4) [51].
3. Criterion scale identification. Within an individual criterion, the scale contains a priori states (categories or classes); for example ‘new’, ‘quite old’ and ‘old’ can be three states of device age. In addition, rules for how to attach a particular state to the device should be given.
4. Criterion scale quantification. Using paired comparisons (see item 2), a number value expressing importance within the particular criterion is matched with any state.
5. Individual device assessment in each criterion. Each medical device should be assessed in relation to all criteria.
6. The total assessment of the i-th device, H_i, is the sum of products of any criterion weight and the criterion scale value assigned to the device, i.e.
   

   

   
   where N_t is the number medical devices, h_ij is the value coefficient of evaluation of the i-th device according to the j-th criterion, w_j is the weight of the j-th criterion on the basic level and N_k the number of criteria on the basic level. The total assessment can be standardized so that the resulting value H´_i is between 0 and 1,
   

   

   
   The value H´_i is a metric enabling decisions concerning which category the device belongs to (in comparison with the pre-defined limits); the authors’ suggestion for the categories that a healthcare facility can adopt is given in Table 18.

Conclusions

In 2012, the International Federation for Medical and Biological Engineering (IFMBE) identified HTA to be of prior importance for ‘Medical and Biological Engineering’ and decided to reactivate a special division focussed on HTA issues [28]. The objectives of this special division are:

• a) to stimulate research and application of new developments in HTA;
• b) to reveal the importance of the role of Biomedical Engineers in many aspects of HTA and stimulate co-operation and professional development and growth;
• c) to improve co-operation between Biomedical Engineers working in the field of HTA in different countries and promote collaboration with other specialists including those belonging to other scientific societies and, in particular, to medical societies;
• d) to facilitate sharing of information, technical and professional guidelines for the practices within the Healthcare Technology Assessment field and promote capacity building;
• e) to promote improvement in the decision-making process concerning healthcare technology planning and acquisition in healthcare delivery systems.

This initiative is very topical, as there has been little progress in the development of HTA methods for medical devices since the situation five years ago, when Value in Health initiated a discussion on the similarities and/or differences of drugs and medical devices from the point of view of HTA studies [14, 62]. Drummond’s paper [14] listing the main problems that a researcher encounters when assessing medical devices has been frequently cited. Above all, the effectiveness assessment of diagnostic devices requires a novel approach. There are incomparably few HTA studies focussed on devices compared to those focussed on drugs or surgery, and most of them deal with therapeutic devices. The recent champions are stents and da Vinci systems [44].

Utilization of multiple-criteria decision methods within health economy has been investigated independently [60]. At present, the question is not whether, but how to employ them [2]. Medical devices seem to be a top candidate for MCDM utilization, as they require quite complex assessment. While clinical aspects (usually in the form of patient-reported quality of life) and cost calculations are assessed in HTA studies focussed on drugs and clinical interventions, technical and operational aspects should also be assessed in devices [13, 29].

In this paper, we recommend utilization of MCDM for medical device assessment in several different ways. We do not expect a full replacement of CEA (or other cost analyses) by MCDM, but utilizing MCDM; rather, MCDM should be utilized within CEA for effect (outcome) evaluation. Thus, cost considerations are the same as in traditional pharmacoeconomic studies, but the clinical effect is replaced by a combined measure of important technical parameters (possibly in combination with clinical outcomes). This enables parameters that are important from the engineering point of view to be taken into consideration. Hence, problems posed by the IFMBE HTA Division can be addressed. Usually, the main goal of HTA studies is to make a decision about procurement and/or incorporation of the device, often on a healthcare facility (hospital) level.

The methods introduced in this paper were applied to various devices including MRIs [29, 30], X-ray devices [31], vital signs monitors [32], lung ventilators [45], mammography devices [36], electronically and mechanically controlled knee systems [36], or remote navigation systems of angiography catheters [47].

Acknowledgement

The authors acknowledge financial support for the development of HTA methods for medical devices by the Ministry of Health Services of the Czech Republic under the Grant No. NT11532 ‘Medical Technology Assessment’.

The authors would like to gratefully appreciate contributions of Juraj Borovský in the early stage of this research, and express their hearty thanks for his constructive criticism and support.

prof. MUDr. Jozef Rosina, Ph.D.

Faculty of Biomedical Engineering

Czech Technical University in Prague

nám. Sítná 3105, CZ-272 01 Kladno

E-mail: rosina@fbmi.cvut.cz

Phone: +420 224 359 939