Introduction

Economic analysis is one of the three key components of health technology assessment (HTA) report and cost-utility analysis (CUA) is probably the most common type of economic analysis. In CUA, costs are measured in monetary units and benefits are expressed in quality adjusted life years (QALYs). QALYs are calculated by multiplying the number of life years gained by a quality-of-life weight of a given health state. The methods, which determine quality-of-life weights, are divided into: direct, such as the time-trade off (TTO) method, standard gamble (SG) and visual analogue scale (VAS), or indirect, employing utility instruments, such as EQ-5D, Short Form 6D (SF-6D), Health Utilities Index Mark 2 or Mark 3 (HUI-2 and HUI-3). In order to use a questionnaire as a generic preference tool, somebody has to previously value health states, described by the questionnaire, using one of the above-mentioned direct methods, TTO being the most common in this context. See [1,2] for a detailed description of TTO and the valuation procedure [1,2].

At first, in EQ-5D valuation studies, based on TTO method – in United Kingdom [3], Spain [4], Germany [5] and United States [2] - respondents from the general population valued 13 health states. Some further studies used lower – 7 (Zimbabwe [6]) or extended number of states per respondent - 16 (Denmark [7]) or 17 (Japan [8] and the Netherlands [9]). In a Polish TTO valuation study, 23 health states were presented to each respondent, and this has been the highest number used so far in a general population preference study [10].

Theoretically, a higher number of health states per respondent means more available valuations, what may decrease estimation error and increase estimation model accuracy or allow for fewer respondents in the study; the latter advantage is favorable with regards to obvious budgetary limitations. However, a possible problem with TTO method extension may simply be physical fatigue of respondents to answer the last TTO questions with satisfactory credibility level.

There are different ways to verify if TTO exercise extension results in bias or not. The results of testing the stability of means and variances of consecutive TTO valuations were described in detail elsewhere [10]. Simply, a comparison of health state values, regardless whether assigned in the middle or at the end of experiment, showed no statistically significant differences, neither in means or in variances.

The aim of the present study was to evaluate a possible bias, resulting from TTO experiment expansion to 23 states per respondent in a Polish valuation study. Stability of regression coefficients was assessed in models, based on health state valuations from different stages of TTO experiment. 

Materials and methods

Polish valuation study

The data, employed in the reported study, originated from a Polish EQ-5D valuation study, performed in 2008 [10]. That study was based on the modified Measurement and Valuation of Health (MVH) protocol. Each respondent ranked 10 health states, valued four health states, using the VAS methodology and 23, using the TTO method. A total of 7,351 TTO valuations from 321 respondents were available before exclusions and 6,769 from 305 respondents after exclusions (see Table 1).

Table 1. The number of available health state valuations from the Polish EQ-5D TTO-based valuation study after exclusions

Study sample, study design, pilot tests, interview procedure, exclusion criteria, modeling and stability of means and variances within TTO experiment tests were described in detail elsewhere [10].

Stability of regression coefficients within TTO experiment

In order to verify the stability of regression coefficients, while using an increasing number of TTO experiments per respondent, the Chow test was employed [11]. The Chow test was performed on the whole sample, divided into two or three subgroups. In the first case, the whole sample was divided into subgroups, with experiments 1-17 (n=5,009) and 18-23 (n=1,760). The second version was designed in such a way as to account for possible instability during the warm-up period in the first TTO experiments. Thus the whole sample was divided into three “periods”: 1-5 (n=1,461), 6-17 (n=3,548) and 18-23 (n=1,760) experiments. In both cases, the basic model with no interaction terms was applied. Accordingly, in the former case, the equality of 11 parameters was tested (constant term and 10 domain specific parameters) in two subperiods and, in the latter one, the equality between the second and the third subperiod was additionally verified (the equivalence of the first and the third subperiod is implied automatically, hence 11 and 22 restrictions, respectively). The null hypothesis was that the parameters are equal in two or three subgroups, as appropriate.

Value sets, based on above-mentioned two or three “period” models, were graphically compared, as well as contrasted with a Polish EQ-5D TTO value set, calculating the following values: (1) the mean absolute difference between health states values, (2) the number of health states (out of 243) with values different by more than 0.01, 0.02, 0.03, 0.05 or 0.10 from the Polish value set and (3) the correlation coefficient between value sets, using simple linear regression.

Results

Regression coefficients of the two parsimonious models, built on valuations from 1-17 or 18-23 experiment, did not differ significantly (p=0.5521; see Table 2).

Table 2. Regression coefficients (SD) of two parsimonious models, built on valuations from 1^st-17^th or 18^th-23^rd experiment

Similarly, regression coefficients of the three parsimonious models, built on valuations from 1-5, 6-17 or 18-23 experiments, did not differ significantly, either (p=0.4334; see Table 3).

Table 3. Regression coefficients (SD) of three parsimonious models, built on valuations from 1^st-5^th, 6^th-17^th 18^th-23^rd experiment

A graphical comparison of the two value sets, based on 1-17 or 18-23 experiments, shows that although individual states differ, both sets are similar (see Figure 1). 

Figure 1. Graphical comparison of two value sets: (1) built on valuations from 1^st to 17^th experiment and (2) built on valuations from 18^th to 23^rd experiment

A graphical comparison of three value sets shows that, in a set built on valuations from experiments 1-5 , the health states closest to death are valued somewhat higher than in the two other sets (see Figure 2).

Figure 2. Graphical comparison of three value sets: (1) built on valuations from 1^st to 5^th experiment, (2) built on valuations from 6^th to 17^th experiment and (3) built on valuations from 18^th to 23^rd experiment

 

 

 

 

 

 

Table 4 presents a statistical summary of cross-model comparisons.

Table 4. Comparison of four different experimental value sets with the Polish EQ-5D TTO value set

The mean absolute differences between health states values were relatively low (from 0.009 to 0.031) and health states values correlated significantly (R² from 0.990 to 0.999). The most outlying value set was built on valuations from experiments 1-5.

Discussion

No systematic changes were identified in model parameters after TTO experiment extension. The stability of regression coefficients within TTO experiment was verified using the Chow test and failed to show that parameters were not equal. Value sets, built on experiments 1-5, 6-17, 1-17 or 18-23, were similar, both in cross-comparisons and in a comparison to the Polish EQ-5D value set.

The most outlying value included the valuations from experiments 1-5, what seems fairly normal, as the first TTO valuations are sort of a warm-up task. In valuation of the first health states, respondents learn the rules of and get familiar with TTO exercise. Moreover, the first states differed from the states valued later on, as interviewers were asked not to reveal states worse than death at the beginning of the TTO exercise. The fact that respondents require this warm up period may prompt using more experiments per respondent, so as to outweigh the somewhat atypical initial valuations in subsequent analysis.

The obtained results should be approached together with the earlier presented analysis [10]. Regardless whether the comparison of health state values was assigned in the middle (position 6 to 17) or at the end (position 18 to 23) of the experiment, no statistically significant differences were observed, either in mean values or in variances, using the Holm-Bonferroni correction. We therefore inferred that additional states were valuable by increasing credibility (with identical means) and precision of the final estimation (did not inflate the total variance).

The combined results of both studies have strong practical implications. In a valuation study, an extension of TTO experiment means that more health state valuations will be obtained in the same population of respondents. It also means that credible valuations can be performed in population samples of moderate size. The results may support the estimation of national value sets in other countries, especially in situations of study budget constraints.

Conclusions

The present study supports the use of more health states per respondent in TTO experiments than it was previously assumed. No systematic changes were found in model parameters after TTO experiment extension. Therefore, there is no risk of bias or efficiency decrease in the estimation. This finding provides evidence for the need to improve the efficiency of valuation protocols and supports the estimation of national value sets in other countries.

Acknowledgements
This study was supported in part by unrestricted grants from GSK Commercial, Pfizer Poland, and Astra Zeneca Pharma Poland.
We are grateful to Anna Jabłońska, Anna Jawoszek, Aneta Dwojak, Ola Możeńska, Anna Gąsiewska, Malwina Hołownia, Krzysztof Orłowski, Szymon Zawodnik, Agnieszka Gaczkowska, Adam Golicki, and Łukasz Kołtowski from Student Pharmacoeconomics Chapter, Medical University of Warsaw for assistance in data collection.

Corresponding author

Maciej Niewada, PhD

Department of Experimental and Clinical Pharmacology, Medical University of Warsaw, Poland

Krakowskie Przedmieście 26/28; 00-927 Warsaw, tel. + 22 826 21 16

mail: maciej.niewada@wum.edu.pl or maciej.niewada@gmail.com fax: + 22 8262116