Article - JHPOR

Approach to uncertainty in health technology assessment in a Central and Eastern European country: appraisal of cancer drugs by a Polish HTA agency in presence of high crossover rates in clinical trials.

jhpor, 2016, 2
DOI:10.7365 / JHPOR.2016.2.2

Copyright
© 2016 PRO MEDICINA Foundation, Published by PRO MEDICINA Foundation
User License
The journal provides published content under the terms of the Creative Commons 4.0 Attribution-International Non-Commercial Use (CC BY-NC 4.0) license.

Authors

Name	Affiliation
Anna Panasiuk	Aestimo s.c., Kraków, Poland
Monika Homa	Aestimo s.c., Kraków, Poland
Dariusz Pawlik	Aestimo s.c., Kraków, Poland
Patrycja Prząda-Machno	Pfizer Poland Sp. z o.o., Warsaw, Poland
Marcin Kaczor	Aestimo s.c., Kraków, Poland;Jagiellonian University Medical College, Kraków, Poland

contributed: 2017-05-18

final review: 2017-05-18

published: 2016-12-31

Corresponding author: Marcin Kaczor marcin.kaczor@aestimo.eu

Abstract

Background: Health technology assessment (HTA) plays an important role in reimbursement decision making in Poland and its principles are similar to those used in other countries. However, specific inter-country differences, such as substantial divergence in budgetary resources, may lead to variation in actual HTA practices, e.g. in the approach to uncertainty. Cancer drug reimbursement is a decision-taking area associated with substantial uncertainty. One of its important sources is the presence of crossover (treatment switching) in clinical trials. Objectives: To review the appraisal processes completed for cancer drugs by the Polish HTA agency (AOTMiT) and to compare AOTMiT to the British, Australian and Canadian HTA bodies with respect to strategies of addressing crossover-related uncertainty.

Methods: Cancer drug assessment processes in AOTMiT, where a substantial crossover took place were reviewed and subsequently matched with the assessments conducted by NICE, PBAC and pCODR. Ways to approach the crossover-related uncertainty, the influence of uncertainty on the recommendation and uncertainty management strategies were examined.

Results: 29 HTA processes related to 6 drugs were included. The crossover rate ranged from 51% to 85% and ITT analyses did not show statistically significant survival benefit. AOTMiT more often yielded negative recommendation, showed less consistent approach to crossover-related uncertainty and a narrower scope of adopted uncertainty management strategies.

Conclusions: Crossover constitutes a vital source of uncertainty in the assessments of new cancer therapies. The lack of consistent standards decreases the transparency of assessment processes and can contribute to undertaking suboptimal reimbursement decisions.

Keywords: treatment switching, uncertainty, Central and Eastern Europe, crossover, oncology

Introduction

The change of the political system which took place in the 1990s in Central and Eastern European (CEE) countries resulted in many vital changes in the healthcare policies, including recognition of the need for rational management of healthcare budgets. This contributed to the support for the Health Technology Assessment (HTA) given by the authorities of particular countries [1, 3, 4, 5]. In Poland, HTA Agency was established in 2005 (since 2015: the Agency for Health Technology Assessment and Tariff System, AOTMiT) and 2007 saw the publication of the first HTA guidelines [1]. Moreover, since 2012, key submission requirements have been legally regulated [6]. The submission of a systematic review of clinical studies, cost-effectiveness analysis and budget impact analysis is required and a cost-effectiveness threshold is set by a legal act [1, 7]. The previous version of the Polish HTA guidelines (effective 2009-08.2016) [8] was methodologically similar to “Western” guidelines [1], and the aim of the recent (08.2016) update was to decrease the existing differences even more by implementing the recommendations of the European Network for HTA (EUnetHTA) [9, 10].

Apparently HTA methodology in Poland does not diverge from those adopted in other countries with well-established HTA system. However, it’s not clear whether the actual HTA practice can converge between countries that have different income and healthcare budgets. OECD data indicate that in Poland, Hungary and in the Czech Republic, 2015 healthcare expenditures amounted to 4.5–6.3% of GDP, and in Great Britain, France and Germany – 7.7–9.4% of GDP [11]. Moreover, significant differences in GDP between “old” and “new” European countries still exist [12]. At the same time, it was demonstrated that the differences in decisions taken by individual HTA bodies often reflected not the differences in scientific evidence interpretation but different approaches to risk and uncertainty [13, 14]. The reimbursement of innovative oncological products seems to be the area particularly burdened with a coincidence of numerous sources of uncertainty, risky decisions and unmet needs, where HTA agencies often struggle to evaluate expensive palliative therapies, not resulting - by nature - in either long-term life prolongation or a noticeable decrease in productivity loss [15, 16]. Consequently, cancer drugs appraisals may constitute an adequate sample for examination of uncertainty management strategies in the HTA practice.

One of recognised sources of uncertainty in cancer treatments appraisal is the permitted switch (crossover) from a randomly assigned control therapy to the investigational treatment, usually at the point of disease progression. The switching opportunity is considered ethically justified and it speeds up clinical study enrolments. However, high cross-over rate causes problems in interpretation of the survival outcomes by leading to the dilution of relative treatment effect, and resulting increase in the incremental cost-effectiveness ratio (ICER) [17, 18].

Attempts are taken to demonstrate the close relationship between OS and progression free survival (PFS) or response rate, which would justify decisions on reimbursing a given drug despite no reliable OS estimation [19-21]. Often, however, such approach is considered inadequate within the HTA process, failing to meet the expectations of the substantial improvement in the “final endpoint” which in advanced cancer patients is considered to be the OS gain [22, 23].

In response to such requirements, efforts are being made to adjust survival estimates with statistical methods [24, 17, 18]. The vast variety of switching adjustment methods that are applied in HTA submissions, resulted in the need to develop the principles of good practice for HTA institutions, in particular in those regions where appraisal outcomes depend on the ICER estimation. National Institute for Health and Care Excellence Decision Support Unit (NICE DSU) developed a review of switching adjustment methods and provided an analytic framework for identification the method suitable to the case under evaluation, indicating at the same time the lack of consistent approach to the analysed issue in the current practice of NICE [18]. The NICE DSU document encouraged other authors to analyse the practices of other HTA agencies, by developing reviews of HTA appraisals [25–29] or detailed case studies [30–32]. However only few such studies were peer-reviewed and majority of the reviews covered HTA processes undertaken in highest-income countries [25-32].

We examined the approach of a Polish HTA agency to uncertainties associated with the assessment of clinical efficacy and cost-effectiveness, using the crossover-related uncertainty in cancer drug studies as an example. Practice of HTA agencies in countries with higher GDPs - Great Britain, Australia and Canada served as the reference.

Methods

Data search and selection. At the first stage the AOTMiT website was searched for the documentation of processes covering the full HTA assessment between 2012 and 2016. The HTA assessment processes completed with the issue of a recommendation and concerning a cancer drug were subject of a detailed review. Then, the collected documents were examined for information on crossover in pivotal trials and where such information was found – original study publications were retrieved. Cases of crossover rate exceeding 50% of control group were eligible for inclusion in the review.

At the second stage, completed HTA processes were searched for, on the websites of The National Institute for Health and Care Excellence (NICE), The pan-Canadian Oncology Drug Review (pCODR) and the Pharmaceutical Benefits Advisory Committee (PBAC), with respect to the health technologies selected at the first search stage. In order to ensure comparability, the review covered only the assessments based on the evidence common with the AOTMiT processes (i.e. the processes based on new studies, not considered in the AOTMiT appraisals, were excluded). In the case of an process concerning drug use in multiple indications, assessment for each indication were considered a separate process.

The data set is based solely on publicly available documents, which included non-concealed results of clinical efficacy or cost-effectiveness assessment. In Poland, a full HTA process, including cost-effectiveness assessment, is currently required only in the cases of reimbursement applications initiated by the MAH, for the authorised products which does not contain the active substance already reimbursed for the given clinical condition. The processes in which the full HTA assessment is not required (i.e. reimbursement withdrawal, off-label indications) were not eligible.

The following document categories were retrieved: (1) recommendations issued at the completion of the HTA process– made available by each of the analysed HTA agencies; (2) report or summary of assessment report of the company submission – made available by AOTMiT, NICE and pCODR; (3) company submission, in parts concerning the clinical efficacy and cost-effectiveness – made available by AOTMiT and NICE; (4) clinical trials – publications and conference abstracts identified based on reference lists of the HTA documents. The basic sources of data were recommendations and assessment reports, and where discrepancies occurred or data were incomplete, the remaining sources were used.

Where multiple HTA processes for the same health technology were identified, each of them was separately considered for eligibility. The possibility to include multiple processes (resubmissions) in the review depended on the given HTA agency’s practice regarding the leaving or removal of expired recommendations from website. It should be noted that for the scope of the review it was important that the HTA processes conducted for the same health technologies in particular agencies to be based, to a large extent, on the results of the same pivotal trial, in which a substantial crossover took place. As a result, the most up to date recommendations for some health technologies could have been excluded from the review.

Data extraction and analysis. For each included HTA process, the following data have been extracted: active substance and drug name, indication, comparator and comparison type (direct vs indirect), type of economic analysis, type of OS estimation in the economic model with regard to crossover adjustment (unadjusted, adjusted or both; if adjusted – adjustment methods were extracted), comments on the crossover included in the assessment reports, type of recommendation (positive, conditional/restricted, deferred or negative), key reasons for recommendation, conclusion of the recommendation on clinical efficacy and, separately, cost-effectiveness, comments on crossover included in the recommendation. Moreover, the crossover rate in the study control group and unadjusted (intention-to-treat, ITT) and crossover-adjusted hazard ratios (HR) of death were extracted from the trial publications and HTA documents (if available).

Based on the extracted data, own assessment was made of the relative influence of crossover in pivotal trial on the result of each of the HTA processes analysed, by classifying it to one of the following influence categories: considerable (+++), moderate (++), weak (+), uncertain or difficult to assess due to concealing of an important part of evaluation results (+/-) or lack of noticeable influence (-). Reasons were given for each assessment of the crossover influence, together with a list of strategies for handling the crossover problem identified in the given HTA process based on the documents included.

Finally, differences and similarities between the practice of the Polish HTA agency and that of the remaining agencies were analysed in terms of the approach to the crossover issue at the assessment and appraisal stages, the severity of the impact of this problem on the recommendation and types of strategies adopted in order to manage the crossover-generated uncertainty.

Results

By 22 June 2016, AOTMiT published on its website HTA assessment documentation of 251 processes concerning cancer drugs which were conducted from 2012 to 2016 (Supplementary Figure 1.). 6 HTA processes, concerning regorafenib [33–37], sorafenib [38–41], crizotinib [42–45], everolimus [46–50], sunitinib [51–55], pazopanib [56–59] met the inclusion criteria. The AOTMiT-conducted processes were matched with 23 assessments made by the reference agencies [60–98], so a total number of 29 HTA processes were analysed (Table 1.).

Table 1. HTA assessment processes which met the review inclusion criteria (n=29), in the order of AOTMiT’s recommendation issue date.

Drug	Indication	Pivotal trial ID	Cross-over rate^a	Recommendation issue date (YYYY.MM)
Drug	Indication	Pivotal trial ID	Cross-over rate^a	AOTMiT (n=6)	NICE^b (n=2)	PBAC^c (n=15)	pCODR (n=6)
Regorafenib (n=3)	GIST, unresectable/ metastatic	GRID [99]	85%	2015.03 [33-37]	n/a	2015.03 [60]	2014.05 [61-63]
Sorafenib (n=5)	RAI-R DTC, advanced/ metastatic	DECISION [100]	75%	2015.02 [38-41]	n/a	2015.11 [64] 2015.03 [65] 2014.07 [66]	2015.07 [67-69]
Crizotinib (n=6)	NSCLC ALK-positive, advanced/ metastatic, 2nd/2+ line	PROFILE 1007 [101]	64%	2013.09 [42-45]	2013.09 [70-72]	2014.11 [73] 2014.03 [74] 2013.11 [75]	2013.05 [76-78]
Everolimus (n=4)	pNET, unresectable/ metastatic	RADIANT-3 [102,103]	73%	2013.08 [46-50]	n/a	2014.03 [79] 2012.11 [80]	2012.08 [81-83]
Sunitinib (n=6)	pNET, unresectable/ metastatic	A6181111 [104]	69%	2013.07 [51-55]	n/a	2013.08 [84] 2012.07 [85] 2012.03 [86] 2011.07 [87]	2012.05 [88-90]
Pazopanib (n=5)	RCC, advanced/ metastatic, 1st line	VEG105192 [105,106]	51%	2012.10 [56-59]	2013.08 [91-93]	2012.03 [94] 2010.07 [95]	2012.01 [96-98]

Abbreviations: AOTMiT, The Agency for Health Technology Assessment and Tariff System (Agencja Oceny Technologii Medycznych i Taryfikacji); GIST, gastrointestinal stromal tumours; n, number of HTA processes included in the review; n/a, not applicable; NICE, The National Institute for Health and Care Excellence; NSCLC ALK-positive, non-small-cell lung cancer associated with an anaplastic lymphoma kinase fusion gene; PBAC, Pharmaceutical Benefits Advisory Committee; pCODR, the pan-Canadian Oncology Drug Review; pNET, pancreatic neuroendocrine tumours; RAI-R DTC, differentiated thyroid carcinoma refractory to radioactive iodine; RCC, renal cell carcinoma

Notes: ^aPercentage of patients randomised to the control group who crossed over to investigational treatment. ^b Due to unavailability of documentation on appraisal processes which have been superseded with the updated ones, the review included only the most current NICE assessment process for a given health technology. ^cEach PBAC recommendation was considered as a separate HTA process.

The crossover rate [99–106] was in range of 51 to 85% and unadjusted hazard ratios of death were not statistically significant, apart from HR for sunitinib at the first cut-off date (Figure 1.).

Figure 1. Hazard ratios (with 95% confidence intervals) for overall survival (data from most recent cut-off) with oncological drugs compared with placebo in patients with different cancers using different statistical methods to account for crossover.

Legend: ^a adjusted for baseline characteristics; ^b unadjusted for baseline characteristics; IPCTW, inverse probability of treatment and censoring weighted; IPCW, inverse probability of censoring weighting; IPE, iterative parameter estimation algorithm, ITT, intent-to-treat; RPSFT, rank-preserving structural failure time; TDTA, time-dependent treatment analysis (treatment as a time-dependent covariate in the Cox model)

Notes: In the case of sunitinib at the moment of trial termination (15 April, 2009) there was statistical difference in OS, HR = 0.41 (95% CI: 0.19-0,89). The plot shows updated HR (June 2010).

A total of 36 comparisons were identified (Table 2). For detailed characteristics of each of the processes in terms of comparators, comparison types and types of OS estimation see the supplementary appendix (Supplementary Table 1).

Table 2. Basic characteristics of the HTA processes included in the review.

Characteristics^a	AOTMiT	NICE	PBAC – all assessments	PBAC – last assessment only	pCODR	Total	Total – last assessment only
Number of processes	6	2	15	6	6	29	20
Direct comparison(s) only	3	0	11^b	4	5	19 (56.5)	12 (60.0)
Indirect comparison(s) only	1	0	4	2	1	6 (20.7)	4 (20.0)
Direct + indirect comparison(s)	2	2	0	0	0	4 (13.8)	4 (20.0)
Single comparator	3	0	14	6	6	23 (79.3)	15 (75.0)
Multiple comparators	3	2	1	0	0	6 (20.7)	5 (25.0)
Number of comparisons	9	5	16	6	6	36	26
Comparison with BSC	4	2	8	3	4	18 (50.0)	13 (50.0)
Comparison with active treatment	5	3	8	3	2	18 (50.0)	13 (50.0)

Abbreviations: AOTMiT, The Agency for Health Technology Assessment and Tariff System (Agencja Oceny Technologii Medycznych i Taryfikacji); NICE, The National Institute for Health and Care Excellence; PBAC, Pharmaceutical Benefits Advisory Committee; pCODR, the pan-Canadian Oncology Drug Review

Notes:^a Data presented as the number and percentage (in brackets) of HTA processes or of the number of comparisons. ^b Including one process based on two direct comparisons – the control group in the pivotal trial was divided into two subgroups using different drugs.

All HTA agencies noted that crossover likely affected the reliability of OS and/or ICER estimates (sixth column of Supplementary Table 1). However, for a majority of the analysed technologies, AOTMiT analysis seemed to be less thorough. In three processes (sunitinib, everolimus, crizotinib), AOTMiT comments were limited to an indication that crossover is a study limitation, whereas the analyses of the other agencies dealt also with other issues, such as the meeting of assumptions critical for the use of specific crossover-adjustment methods (PBAC – regorafenib, NICE – crizotinib, crizotinib – PBAC) and choice of the best estimate (PBAC2 – sunitinib). The analyses of reference agencies more often acknowledged the direction of crossover-related bias– i.e. the underestimation of the OS gain (pCODR – sunitinib, pCODR – pazopanib), and used external data to assess the possible extent of OS overestimation in the control group (pCODR – pazopanib). Differences across the reference agencies are also noticeable, however, the NICE, PBAC and pCODR assessments seemed more comprehensive than the ones conducted by AOTMiT. In the case of regorafenib, AOTMiT apparently mirrored the comments presented in the previously published pCODR reports, whereas the pazopanib assessment directly referenced to the NICE recommendation (issued in 2011, currently unavailable), justifying the selection of crossover adjustment method recalling reasons given by NICE. The most thorough AOTMiT analysis of the crossover-related bias was found in a sorafenib assessment report, though a clear influence of the prior PBAC recommendation was noted in this process.

A total of 29 recommendations were included (Table 3, Supplementary Figure 2.). AOTMiT and NICE had the highest percentage of negative recommendations (50%). pCODR recommended reimbursement of the assessed drug most often, however, in each case, the positive recommendation included additional conditions to be met. Crossover-related uncertainty was directly pointed out among the reasons for 7 non-positive recommendations. However, the true influence of crossover on the final recommendation was not limited to those cases, since high crossover rate often contributed to a conclusion of “uncertain efficacy,” “uncertain clinical superiority”, “uncertain risk-to-benefit ratio” or “uncertain cost-effectiveness” – stated among direct recommendation reasons (Supplementary Table 2). Moreover, lack of evaluator’s approval for the crossover handling approach initially proposed in the submission, in some cases resulting in an adjustment of economic model assumptions during the HTA assessment towards more conservative ones, contributed to the fact that submission estimates were considered uncertain or not sufficiently robust, the clinical effect estimates - overestimated and the ICER value - high and/or uncertain, also forming grounds for non-positive recommendations.

Table 3. The direction of recommendation and the result of assessment of the influence of crossover on the HTA process.

Drug	Recommendation^a (rating of the influence of crossover-related uncertainty)^b
Drug	AOTMiT	NICE	PBAC	pCODR
Regorafenib	NEG (+)	n/a	NEG (+++)	CON/RES (+)
Sorafenib	NEG (++)	n/a	NEG (++) → DEF (+++) → DEF (+++)	NEG (++)
Crizotinib	NEG (++)	NEG (++)	DEF (+++) → DEF (+++) → CON/RES (++)	CON/RES (+)
Everolimus	POS (-)	n/a	NEG (+) → POS (+/-)	CON/RES (+)
Sunitinib	POS^c(+/-)	n/a	NEG (++) → DEF (++) → NEG (+++) → POS (++)	CON/RES (++)
Pazopanib	CON/RES (+/-)	POS (+)	NEG (-) → POS (-)	CON/RES (++)

Notes:

^aRecommendations were classified to one of four categories: POS – positive (reimbursement recommended, without further conditions), CON/RES – conditional or restricted (reimbursement recommended under certain conditions, mostly related to a decrease in the costs to the cost-effective level or in restricting the target population); DEF – deferred (the issue of recommendation deferred); NEG – negative (reimbursement not recommended)

^bInfluence of crossover on recommendation (own judgement): +++ considerable; ++ moderate; + weak; -/+ uncertain or difficult to assess due to concealing of an important part of assessment results; - lack of noticeable influence

^cRecommendation text partially concealed; conditional/restricted recommendation cannot be excluded

Crossover-related uncertainty influence on the HTA outcome were the least frequently identified in the Polish HTA documentation – however, at least weak influence was judged to be present in 50% of the analysed processes. In the cases of NICE, pCODR and PBAC assessments, it amounted to 100%, 100% and 80% (67% – with multiple PBAC recommendations being accounted for as a single comprehensive process), respectively. Accordingly, the share of HTA processes, in which the influence of crossover was rated as moderate or considerable, was the lowest for AOTMiT (33%), whereas the remaining agencies’ score was at least 50%. Detailed reasons for each of the influence judgements are given in the Supplementary appendix (fifth column in Supplementary Table 2). For positive recommendations, the influence of crossover was generally considered weak or uncertain, with the exception of cases where crossover-related uncertainty caused a prolongation of the whole HTA process (PBAC – sunitinib). In the cases in which crossover-related uncertainty contributed to AOTMiT issuing negative recommendations, in the other agencies corresponding processes more often accounted for the conditional reimbursement (pCODR – regorafenib, crizotinib; PBAC3 – crizotinib) or the decision being deferred (PBAC3 – sorafenib).

Various strategies implemented by individual HTA agencies were identified for the crossover-related uncertainty management in the analysed sample. Among the identified approaches, “calculation” strategies – mainly aimed at identifying the best estimates of HR of death and/or ICER – as well as approaches basically independent from OS estimation can be distinguished (Table 4; for further details see sixth column of Supplementary Table 2).

Table 4. Actions and/or strategies to manage the uncertainty caused by high crossover ratio, identified in the analysed sample of the HTA processes.

Identified categories of actions/strategies^a		HTA processes (total)
Identified categories of actions/strategies^a		AOTMiT	NICE	PBAC	pCODR
Directed at the identification of the best HR of death and/or ICER estimates	Use of adjusted estimations from the submission (statistical methods of HR adjustment)^b	√	√	√	√
	Analysis of validity of using particular methods of statistical adjustment^b		√	√
	Recommendation to use statistical adjustment (in case of no relevant adjustment in the submission)^b			√
	Attempt at indicating a reliable adjustment method based on external guidelines/recommendations^b	√			√
	Adjustment applied to survival modelling^c (use of estimations from the submission)	√	√	√	√
	Change of assumptions on the crossover influence on OS/HR of death in the submission model		√	√	√
	Conservative approach, worst-case scenario^e	√	√	√	√
	Use of published analyses (external to the submission)^b			√
Independent from HR of death estimation	Stronger focus on the magnitude of the observed effect (regardless of the presence of statistical significance or its lack)				√
	Attempt at assessing the drug value based on the endpoints not confounded by crossover (mainly PFS)	√	√	√	√
	Strive to obtain data from clinical practice in order to explain uncertainties in the future			√
	Reduction of the risk of lack of cost-effectiveness with risk sharing instruments^d			√
	Reduction of the risk of lack of cost-effectiveness through reducing therapy costs^d			√
	Accounting for other decision-making factors as a counterbalance for uncertain influence on OS^f			√

Abbreviations: AOTMiT, The Agency for Health Technology Assessment and Tariff System (Agencja Oceny Technologii Medycznych i Taryfikacji); HR, hazard ratio; HTA, health technology assessment; ICER, incremental cost-effectiveness ratio (including incremental cost-utility ratio); NICE, The National Institute for Health and Care Excellence; OS, overall survival; PBAC, Pharmaceutical Benefits Advisory Committee; pCODR, the pan-Canadian Oncology Drug Review; PFS, progression-free survival

Notes: ^a One action could have been included in more than one category, e.g. further calculations, unaccounted for in the submission, based on an ITT result, would be categorised as a “change of assumptions on the crossover influence onf OS/HR of death in the applicant submission model” and “conservative approach, worst-case scenario”, therefore, this list should not be considered in a purely quantitative way; ^b The statistical adjustment in a given process could be used to assess the clinical efficacy and cost-effectiveness, or in just one of these domains; ^c OS modelling based on the assumptions of a relationship between PFS and OS and between the risk of death prior and post progression or based on external data; ^d Classified as a strategy to deal with crossover-related uncertainty only if the recommendation accounted for such a relationship (not all recommendation cases on risk sharing or cost reduction have been considered here); ^e E.g. taking into account an ITT result (unadjusted), the assumption of no OS gain, assumption of equivalent survival after progression with compared interventions; ^f E.g. unmet clinical need, no alternative treatments, low budget impact

Within “calculation” approach category, the following strategies were mainly identified in the AOTMiT processes: the use of adjusted assessments provided in submissions, accounting for highly conservative assessments (mainly – based on the ITT analysis) and one case of referring to the recommendation of another HTA agency (NICE) in determination of the most reliable adjustment method. Documentation of the remaining HTA agencies indicated the use of a more varied scope of actions. NICE and PBAC examined validity of using particular methods of statistical adjustment, considering the direction and extent of a possible crossover-related bias, and the possibility of meeting the critical assumptions for the proper performance of the given adjustment technique. During the assessments conducted for NICE, PBAC and pCODR, evaluators made changes in the applicant model. Furthermore, in PBAC documents, cases of recommendations for use of crossover statistical adjustment were identified, and use of an external, published analysis which addressed the crossover problem.

The Polish HTA agency seemed to have been reluctant to use methods of drug value assessment other than the calculation ones. Only one AOTMiT recommendation (everolimus) directly indicated other benefits than OS gain among the arguments for drug reimbursement. Australian and Canadian agencies, apart from quite consistent examination of PFS predictive value for OS and the clinical analysis of PFS improvement significance as such, also resorted to such decision-making tools as: analysis of the magnitude of the observed hazard ratios, regardless of statistical significance (pCODR), reduction of the risk of financing a cost-ineffective therapy by the use of a risk-sharing instrument addressing the identified uncertainty or reduction of therapy cost (PBAC) and accounting for, in the comprehensive cost-effectiveness assessment, the importance of such decision-making factors as unmet clinical need, lack of alternative treatment or a small number of patients resulting in a minor budget impact (PBAC).

In order to accurately illustrate the AOTMiT approach to the crossover-generated uncertainty, the case of crizotinib was selected for a more detailed description.

Case study: crizotinib in advanced or metastatic non-small-cell lung cancer associated with an anaplastic lymphoma kinase fusion gene (ALK-positive NSCLC).

Crizotinib efficacy in previously treated patients with advanced or metastatic ALK-positive NSCLC was studied in the randomised phase 3 PROFILE 1007 study (n=347) versus chemotherapy (docetaxel or pemetrexed). The study protocol allowed for one-way crossover from the control group in the case of disease progression and additional OS analysis with an crossover adjustment with the RPSFT method was planned. At data cut-off, 64.4% patients in the control group crossed to crizotinib [101]. Unadjusted HR of death in the interim analysis did not show crizotinib effect (HR=1.02; 95% CI: 0.68–1.54). Upon the crossover adjustment, the point estimates indicated 17% death risk reduction, but the confidence interval was broad and still included unity (RPSFT HR=0.83; 95% CI: 0.36–1.35) [71, 108]. The ITT analysis showed significant superiority of crizotinib over chemotherapy in PFS (HR=0.49; 95% CI: 0.37–0.64; p<0.001), being the primary study endpoint [101]. AOTMiT’s report [43] pointed out that the crossover allowed by the study protocol was the limitation which hampered the evaluation of the drug’s impact on OS and safety assessment and indicated PFS as the most reliable efficacy endpoint. The cost-effectiveness assessment results were concealed to a considerable degree. However, available data indicate that some crossover adjustment method was applied in the submission model, which was not accepted by AOTMiT, due to use of unpublished HR estimate and objections to the model description. The ICER estimate, indicated as the most reliable in the assessment report, exceeded the cost-effectiveness threshold established in Poland [43].

When comparing the scope of strategies of managing the crossover-related uncertainty, used by AOTMiT in the case of crizotinib, with the course of assessments of the other drugs, significant differences in the evaluation course can be observed. The assessment report for crizotinib [43], in the clinical efficacy part, showed only unadjusted OS estimates, despite the fact that adjusted results were provided in the submission [44, 45]. In the assessment reports for sorafenib [39], sunitinib [52] and pazopanib [57] adjusted HR estimates were shown without, however, any particular reservations as to their reliability. Among the key reasons of the negative recommendation for crizotinib, “restrictions of the study hampering the unequivocal data interpretation” and the lack of OS gain were indicated [42]. The same limitations were not pointed out as obstacles to everolimus reimbursement, where for the conclusion on proven efficacy, PFS improvement, an increase in the probability of stable disease and tumour mass reduction were sufficient [46], to pazopanib reimbursement – for which only the comparability to another active therapy (indirect comparison) was analysed [56], without examining whether the assessed therapy improves OS, or to that of sunitinib – where the recommendation did not include any reference to the lack of statistical significance of HR of death in the updated analysis [51].

When comparing the strategies for managing the crossover-related uncertainty in the crizotinib assessment conducted by AOTMiT, with the approach of the other HTA agencies, it seems that, although the approaches of NICE, PBAC and pCODR varied, each of them dealt with the problem in a more comprehensive manner. Although the NICE recommendation, similarly to that of AOTMiT, was negative, it was preceded with a detailed analysis of adjusted HR estimates, taking into account the sensitivity of particular methods to such factors as sample size or the time-dependency of the treatment effect [71]. Finally, despite clearly stated limitations of OS gain estimation, NICE recognised the therapy as associated with the clinical benefit as based on the “noteworthy extension to PFS” and “very high response rate.” It was also recognised that treatment with crizotinib would result in an overall survival gain compared with docetaxel, with the uncertainty related to the exact magnitude of the gain [70]. In this case, negative recommendation was issued due to the high ICER and objections related to the indirect comparison with BSC, which AOTMiT did not consider at all. The two remaining agencies decided to apply compromise solutions by recommending crizotinib reimbursement under additional conditions. The PBAC’s crizotinib assessment seems particularly interesting for analysing crossover-related uncertainty approaches. In total, it included three recommendations (first submission [75] and re-submission [74] resulting with a deferred recommendation and the second re-submission that yielded a conditional recommendation [73]), and it seems that here the uncertainty related with OS estimation constituted the central problem. The OS modelling method was the subject of discussion between the company and the evaluator and after all, a change of assumptions in the base-case analysis was made by PBAC. The PBAC’s scenario was more conservative than the manufacturer's submission base-case, but it still assumed the non-zero incremental OS gain as a result of crizotinib use. The resulting ICER indicated the lack of crizotinib cost-effectiveness, but PBAC proposed a complex Managed Entry Scheme in which future therapy financing conditions depended on the outcomes to be observed in the real clinical practice [73]. Thus, by recognising that the available scientific evidence does not provide sufficient data for an accurate clinical benefit assessment (mainly due to crossover-caused interference), it was made possible to gather data for the purpose of further drug evaluation with the uncertainty-related risk shared by the MAH and the public payer. The recommendation also indicated an unmet clinical need and a small number of patients [73]. The pCODR process included into the review was the re-submission analysis, since at the time of the first submission [112], the RCT PROFILE 1007 results were not yet available. A substantial uncertainty around the OS benefit and related difficulties in the interpretation of the result were noted [77], whereas the economic evaluation report (partly confidential) focused on uncertainty sources other than crossover, such as the selection of utility weights. The changes made in the model resulted in an increase in the ICER beyond the acceptable level [78]. The final recommendation acknowledged that the crizotinib treatment was associated with a net clinical benefit based on the demonstrated improvement in PFS and quality of life. The favourable conclusion for clinical benefit resulted in the drug being recommended for reimbursement provided that the MAH agrees to the additional pricing and/or cost arrangements that would improve the cost-effectiveness [76].

Regardless of the differences in the HTA recommendations, as of the time of finalising this review (June 2016), crizotinib used in the 2nd line treatment of ALK-positive NSCLC had been reimbursed in all the considered countries except for Poland – in England and Wales as part of the National Cancer Drugs Fund List [113], in Australia – within the Managed Entry Scheme [114] and in Canada – in insurance plans of all provinces [115–124]. Eventually, crizotinib was granted reimbursement in Poland in November 2016, which was over 3 years after the completion of the HTA process.

Discussion

The review demonstrated that the Polish HTA agency, similarly to more experienced agencies, identifies crossover as an issue in the assessment of the survival benefit. However, at the same time AOTMiT uses a narrower scope of strategies to manage the crossover-related uncertainty, which results in a less consistent influence of the crossover issue on final recommendations. Lack of regular approach to the crossover issue across assessments is also noticeable. More comprehensive analyses of the crossover-related uncertainty in the AOTMiT processes were conducted in cases where there was the possibility to use assessment of another HTA agency as reference. Such a trend in the AOTMiT practice was already noticed [125]. Making references to the recommendations of other HTA agencies, especially well-established ones, is not controversial as such. In some cases, however, such practice may lead to unequal treatment of particular health technologies – a technology the assessment of which cannot account for, e.g., NICE or PBAC recommendation, may be assessed in a less comprehensive manner and due to the lack of the basis to indicate the appropriate adjustment method in the final assessment, the possibility of such adjustment may be ignored.

The distinctive difference between the AOTMiT processes and these of pCODR and PBAC is the fact that the two latter agencies use a broad array of “non-calculative” methods to reduce uncertainty, showing clear willingness to work out, in cooperation with the applicant, the conditions under which the drug could be made available to patients and the risk of financing a cost-ineffective therapy would be adequately reduced. NICE, PBAC and pCODR recommendations include content indicating that substantial attention is paid to identifying unmet clinical needs and relevant patients’ values. These decision-making aspects seem to be rather scarcely present in the analysed AOTMiT documents what may lead to the recommendation process being more freely finalised as negative, without sufficient attempts at dealing with the uncertainty. This situation can partially explain the limited array of strategies used when compared with the remaining agencies.

There are obvious limitations to the method of comparing the assessment process with respect to the same HTA technologies by different agencies, as adopted in our study. The considered HTA bodies work under different formal and legal conditions. For example, at NICE, a health technology assessment starts with a scoping phase, during which the HTA agency position on the key submission elements such as comparators are presented to the applicant, before the final recommendation is preceded by the publication of a draft version. In Poland, there is no formal procedure for any initial arrangements between the evaluator and the MAH preceding the submission and the applicant has very limited options of making adjustments in the submission during the assessment process. Such cross-agency differences certainly have an impact on the final HTA outcome. Therefore, comparison of the recommendations’ direction in our review was of supportive significance, whereas our focus was on the details of assessment processes. To ensure the true presence of the crossover issue, only cases affected with very high (>50%) crossover rate were eligible. However, it was acknowledged that the final HTA recommendation is the result of a multifaceted evaluation in which different weights are attached to specific factors . The simultaneous presence of another source of substantial uncertainty, such as the lack of a relevant head-to-head study or early study termination, overlapped with the crossover-related uncertainty - the extraction of a direct influence of crossover on a given HTA process in such cases was particularly challenging.

In our study, the crossover issue served as an example of the uncertainty source in the HTA process, whereas a detailed analysis of the justification to use certain strategies, e.g. specific statistical models, was beyond the scope. It is, however, an important issue, with far reaching consequences for reimbursement recommendations and decisions [24, 30, 31]. Another important limitation of our study was the need to rely solely on the data disclosed in the public domain. We are aware that disclosed content may not fully reflect the actual contribution of particular factors to the final recommendation. Transparency is, however, acknowledged as one of a key features of a properly conducted HTA process. Thus, it should be assumed that the publicly disclosed HTA content may enable development of an unbiased picture of the appraisal conduct, in spite of a confidential exclusions.

Conclusions

According to our knowledge, this review is the first study analysing the methods used by an HTA agency in a CEE country to manage the crossover-related uncertainty. Our data indicate that the Polish HTA agency, in comparison with its counterparts in Great Britain, Australia and Canada, uses a narrower array of strategies to manage uncertainty and its approach to the crossover issue demonstrates little consistency. Considering the dynamic development of the HTA methodology in Poland and a strong influence of HTA assessment results on the drug reimbursement decisions, it can be expected that in the years to come, the decision-taking practice of the Polish HTA agency will, also in terms of the issue analysed herein, be approaching the standards developed by more experienced institutions.

Acknowledgements

This study was supported by a grant from Pfizer Poland Sp. z o.o., Warsaw, Poland.

Supplementary Materials

http://www.jhpor.com/images/suplementary/approach_to_uncertainty/supplementary%20material.pdf

References

1. Gulácsi L, Orlewska E, Péntek M. Health economics and health technology assessment in Central and Eastern Europe: a dose of reality. Eur J Health Econ 2012;13(5):525-531.

2. Nizankowski R, Wilk N. From idealistic rookies to a regional leader: the history of health technology assessment in Poland. Int J Technol Assess Health Care 2009;25(Suppl 1):156-162.

3. Antoun J, Phillips F, Johnson T. Post-Soviet transition: improving health services delivery and management. Mt Sinai J Med 2011;78(3):436-448.

4. Gulácsi L, Rotar AM, Niewada M, et al. Health technology assessment in Poland, the Czech Republic, Hungary, Romania and Bulgaria. Eur J Health Econ 2014;15(Suppl 1):S13-S25.

5. Rogalewicz V, Borovský J, Juřičková I. Health Technology Assessment in the Czech and the Slovak Republics. in: Jobbágy Á eds, 5th European Conference of the International Federation for Medical and Biological Engineering, Budapest, Hungary: Springer Berlin Heidelberg, 2012.

6. Regulation of the Minister of Health of 2 April 2012 on the minimum requirements to be satisfied by the analyses accounted for in the applications for reimbursement and settings the official sales price and for increasing the official sales price of drug, a special purpose dietary supplement, a medical device, which do not have a reimbursement counterpart in a given indication. Available from: http://www.aotm.gov.pl/www/wp-content/uploads/wytyczne_hta/2012/Regulation_MoH_minimum_requirements_03042012_eng.pdf [Accessed December 20, 2016].

7. Matusewicz W, Godman B, Pedersen HB, et al. Improving the managed introduction of new medicines: sharing experiences to aid authorities across Europe. Expert Rev Pharmacoecon Outcomes Res 2015;15(5):755-758.

8. Agency for Health Technology Assessment. Guidelines for conducting Health Technology Assessment (HTA). Version 2.1. Available from: http://www.aotm.gov.pl/www/wp-content/uploads/wytyczne_hta/2009/Guidelines_HTA_eng_MS_29062009.pdf [Accessed July 1, 2016].

9. Agency for Health Technology Assessment and Tariff System. Health Technology Assessment Guidelines. Version 3.0. Issued: August 2016. Available from: http://www.aotm.gov.pl/www/hta/wytyczne-hta/ [Accessed December 20, 2016].

10. HTA Network reflection paper on “REUSE OF JOINT WORK IN NATIONAL HTA ACTIVITIES”, adopted by the HTA Network, April 2015, Ref. Ares(2015)1982600 - 11/05/2015. Available from: http://ec.europa.eu/health/technology_assessment/docs/reuse_jointwork_national_hta_activities_en.pdf [Accessed July 11, 2016].

11. Organisation for Economic Co-Operation and Development. Statistical Resources. Available from: http://stats.oecd.org/# [Accessed July 1, 2016].

12. Eurostat Database. Main tables, GDP per capita in PPS, data from 1st of June 2016. Available from: http://ec.europa.eu/eurostat/tgm/graph.do?tab=graph&plugin=1&pcode=tec00114&language=en [Accessed July 11, 2016].

13. Clement FM, Harris A, Li JJ, et al. Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. JAMA 2009;302(13):1437-43.

14. Chabot I, Rocchi A. Oncology drug health technology assessment recommendations: Canadian versus UK experiences. Clinicoecon Outcomes Res 2014;6:357-67.

15. Neumann PJ, Bliss SK, Chambers JD. Therapies for advanced cancers pose a special challenge for health technology assessment organizations in many countries. Health Aff (Millwood) 2012;31(4):700-8.

16. Kudrin A. Reimbursement challenges with cancer immunotherapeutics. Hum Vaccin Immunother 2012;8(9):1326-1334.

17. Ishak KJ, Proskorovsky I, Korytowsky B, et al. Methods for adjusting for bias due to crossover in oncology trials. Pharmacoeconomics 2014;32(6):533-46.

18. Latimer NR, Abrams KR. NICE DSU Technical Support Document 16: Adjusting survival time estimates in the presence of treatment switching. Issued: July 2014. Available from: http://www.nicedsu.org.uk [Accessed July 11, 2016].

19. Broglio KR, Berry DA. Detecting an Overall Survival Benefit that Is Derived From Progression-Free Survival. J Natl Cancer Inst 2009;101(23):1642-1649.

20. Blumenthal GM, Karuri SW, Zhang H, et al. Overall response rate, progression-free survival, and overall survival with targeted and standard therapies in advanced non-small-cell lung cancer: US Food and Drug Administration trial-level and patient-level analyses. J Clin Oncol 2015;33(9):1008-14.

21. Shi Q, de Gramont A, Grothey A, et al. Individual Patient Data Analysis of Progression-Free Survival Versus Overall Survival As a First-Line End Point for Metastatic Colorectal Cancer in Modern Randomized Trials: Findings From the Analysis and Research in Cancers of the Digestive System Database. J Clin Oncol 2015;33(1):22-28.

22. Velasco Garrido M, Mangiapane S. Surrogate outcomes in health technology assessment: an international comparison. Int J Technol Assess Health Care 2009 Jul;25(3):315-22.

23. Kleijnen S, Lipska I, Leonardo Alves T, et al. Relative effectiveness assessments of oncology medicines for pricing and reimbursement decisions in European countries. Ann Oncol 2016;27(9):1768-75.

24. Morden JP, Lambert PC, Latimer N, Abrams KR, Wailoo AJ. Assessing methods for dealing with treatment switching in randomised controlled trials: a simulation study. BMC Med Res Methodol 2011;11:4.

25. Heyland K, Samjoo IA, Grima DT. Reimbursement recommendations for cancer products without statistically significant overall survival data: a review of Canadian pCODR decisions. Value Health 2014;17(7):A100.

26. Maervoet J, Skaltsa K, Ivanescu C, Van Engen A. Do health technology agencies accept methods for dealing with treatment switching? Value Health 2014;17(7):A330.

27. Ruof J, Staab TR, Walter M. Dilemma of cross-over trials and their impact on benefit assessment in oncology. Value Health 2014;17(7):A658.

28. Isbary G, Staab TR, Amelung VE, Dintsios C, Ruof J. G-BA does not adjust evidence requirements in early benefit assessment in cases of pre-defined, efficacy-based cross-over decisions in oncology trials. Value Health 2015;18(7):A486.

29. O’Leary BA, Gordois AL, McElroy H. Adjusting for cross-over in oncology trials: approaches taken to support drug reimbursement in Australia. Value Health 2015;18(7):A439.

30. Chabot I, LeLorier J, Blackstein ME. The challenge of conducting pharmacoeconomic evaluations in oncology using crossover trials: the example of sunitinib for gastrointestinal stromal tumour. Eur J Cancer 2008;44(7):972-7.

31. Jönsson L, Sandin R, Ekman M, et al. Analyzing overall survival in randomized controlled trials with crossover and implications for economic evaluation. Value Health 2014 Sep;17(6):707-13.

32. Thivolet M, Kornfeld A, Toumi M. Health technology assessments in oncology: crizotinib case study. Value Health 2014;17(7):A661.

33. Agency for Health Technology Assessment and Tariff System. Recommendation No 19/2015 of 23 March 2015 of the President of the Agency for Health Technology Assessment and Tariff System (AOTMiT) on the reimbursement of the medicinal product Stivarga, regorafenib, film-coated tablets, 40 mg, 84 tablets, as part of the drug program “Treatment of gastrointestinal stromal tumours (GIST) (ICD-10 C 15, C 16, C 17, C 18, C 20, C 48).” Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2015/829-materialy-2015/3822-004-2015-zlc [Accessed July 13, 2016].

34. Agency for Health Technology Assessment and Tariff System, Department for Health Technologies Assessment Application for the medicinal product Stivarga (regorafenib) to be put on the reimbursement list as part of the drug program: “Treatment of gastrointestinal stromal tumours (GIST).” Verification analysis. No: AOTMiT-OT-4351-1/2015. Completion date: 13 March 2015. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2015/829-materialy-2015/3822-004-2015-zlc [Accessed July 13, 2016].

35. Regorafenib (Stivarga®) in gastrointestinal stromal tumours (GIST). Clinical analysis. HealthQuest. Warsaw 2014. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2015/829-materialy-2015/3822-004-2015-zlc [Accessed July 13, 2016].

36. Regorafenib (Stivarga®) in gastrointestinal stromal tumours (GIST). Economic analysis. HealthQuest. Warsaw 2014. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2015/829-materialy-2015/3822-004-2015-zlc [Accessed July 13, 2016].

37. Regorafenib (Stivarga®) in gastrointestinal stromal tumours (GIST). Response to letter No MZ-PLR-4610-943(2)/BR/14. HealthQuest. Warsaw 2014. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2015/829-materialy-2015/3822-004-2015-zlc [Accessed July 13, 2016].

38. Agency for Health Technology Assessment and Tariff System. Recommendation No 9/2015 of 9 February 2015 of the President of the Agency for Health Technology Assessment and Tariff System (AOTMiT) on the reimbursement of the medicinal product Nexavar (sorafenib), 200 mg, film-coated tablets, 112 tablets, in the indication: “Treatment of advanced, radioactive iodine-refractory thyroid cancer (ICD-10: C 73)” as part of drug program. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2012-2015?id=3718 [Accessed July 11, 2016].

39. Agency for Health Technology Assessment and Tariff System, Department for Health Technologies Assessment Application for the medicinal product Nexavar (sorafenib) in the indication: “Treatment of advanced, radioactive iodine-refractory thyroid cancer (ICD-10: C73)” as part of drug program. Verification analysis. No: AOTM-OT-4351-41/2014. Completion date: January 30, 2015. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2012-2015?id=3718 [Accessed July 11, 2016].

40. Nexavar® (sorafenib) in the treatment of locally advanced or metastatic, differentiated, radioactive iodine-refractory thyroid cancer. Clinical analysis, version 1.0. Cracow 2014. Aestimo s.c. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2012-2015?id=3718 [Accessed July 11, 2016].

41. Nexavar® (sorafenib) in the treatment of locally advanced or metastatic, differentiated, radioactive iodine-refractory thyroid cancer. Economic analysis, version 1.0. Cracow 2014. Aestimo s.c. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-mz-2012-2015?id=3718 [Accessed July 11, 2016].

42. Agency for Health Technology Assessment (AOTM). Recommendation No 114/2013 of September 9, 2013 r. of the President of the Agency for Health Technology Assessment (AOTM) on the reimbursement of the medicinal product Xalkori, crizotinib, hard capsules, 250 mg, 60 capsules; Xalkori, crizotinib, hard capsules, 200 mg, 60 capsules, in the indication: treatment of ALK-positive non-small-cell lung cancer (ICD-10 C 34), as part of drug program. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-151-2013/151-2013-zlc [Accessed July 11, 2016].

43. Agency for Health Technology Assessment, Department for Health Technologies Assessment. Application for putting on the reimbursement list and setting official price of the medicinal product: Xalkori (crizotinib) as part of drug program: “Treatment of ALK-positive non-small-cell lung cancer (ICD-10 C 34)”. Verification analysis. No: AOTM-OT-4351-13/2013. Completion date: August 29, 2013. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-151-2013/151-2013-zlc [Accessed July 11, 2016].

44. Xalkori® (crizotinib) in patients with previously treated, anaplastic lymphoma kinase-positive (ALK), advanced non-small cell lung cancer. Clinical analysis, version 1.1. Warsaw, July 19, 2013, MAHTA Sp. z o.o. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-151-2013/151-2013-zlc [Accessed July 11, 2016].

45. Xalkori® (crizotinib) in patients with previously treated, anaplastic lymphoma kinase-positive (ALK), advanced non-small cell lung cancer. Economic analysis, version 1.1. Warszawa, July 16, 2013, MAHTA Sp. z o.o. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-151-2013/151-2013-zlc [Accessed July 11, 2016].

46. Agency for Health Technology Assessment (AOTM). Recommendation No 106/2013 of August 19, 2013 of the President of the Agency for Health Technology Assessment on the reimbursement of the medicinal product Afinitor (everolimus), tablets, 5 mg, 30 tablets, EAN code 5909990711567 and Afinitor (everolimus), tablets, 10 mg, 30 tablets, EAN code 5909990711598, as part of the drug program “Treatment of unresectable or metastatic highly differentiated progressive pancreatic neuroendocrine tumours in adult patients (ICD-10: C 25.4).” Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-149-2013/149-2013-zlc [Accessed July 11, 2016].

47. Agency for Health Technology Assessment, Department for Health Technologies Assessment. Application for the medicinal product Afinitor (everolimus) to be put on the reimbursement list as part of the drug program: treatment of unresectable or metastatic highly or moderately differentiated progressive pancreatic neuroendocrine tumours in adult patients. Verification analysis. No: AOTM-OT-4351-12/2013. Completion date: August 8, 2013. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-149-2013/149-2013-zlc [Accessed July 13, 2016].

48. The clinical analysis of the use of medicinal product Afinitor (everolimus) versus placebo and sunitinib in the treatment of unresectable or metastatic highly or moderately differentiated progressive pancreatic neuroendocrine tumours (pNET) in adult patients. Study systematic review. Cracow, January 2013. Centrum HTA Sp. z o.o. Sp. k. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-149-2013/149-2013-zlc [Accessed July 13, 2016].

49. The economic justification assessment from the public payer’s and the payer’s of medical benefits point of view to use the medicinal product Afinitor® (everolimus) in the treatment of unresectable or metastatic highly or moderately differentiated progressive pancreatic neuroendocrine tumours in adult patients as part of the drug program in Polish conditions. Economic analysis. Cracow, January 2013. Centrum HTA Sp. z o.o. Sp. k. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-149-2013/149-2013-zlc [Accessed July 13, 2016].

50. The supplement of the application for putting on the reimbursement list and setting official price for the medicinal product Afinitor (everolimus), hard capsule, 5 mg, 30 tablets, EAN: 590999071567, as part of the drug program: “Treatment of unresectable or metastatic highly differentiated progressive pancreatic neuroendocrine tumours in adult patients (ICD 10: C25.4)” in terms of the comments of the President of Agency for Health Technology Assessment (letter No: AOTM-OT-4351-12(5)/SZ/2013) to the analyses attached to the above mentioned application. Warsaw, July 11, 2013, Novartis Oncology. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-149-2013/149-2013-zlc [Accessed July 13, 2016].

51. Agency for Health Technology Assessment (AOTM). Recommendation No 80/2013 of 1 July 2013 r. of the President of the Agency for Health Technology Assessment (AOTM) on the reimbursement of the medicinal product Sutent (sunitinibum), hard capsules, 12.5 mg, 28 capsules (4 blisters x 28 capsules) and Sutent (sunitinibum), hard capsules, 25 mg, 28 capsules (4 blisters x 28 capsules) as part of the drug program: treatment of highly differentiated pancreatic neuroendocrine tumour (ICD 10: C25.4). Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-059-2013/059-2013-zlc [Accessed July 11, 2016].

52. Agency for Health Technology Assessment, Department for Health Technologies Assessment. Application for drug reimbursement: 1) Sutent (sunitinib), hard capsules, 12.5 mg, 28 capsules, EAN code 5909990079377 2) Sutent (sunitinib), hard capsules, 25 mg, 28 capsules, EAN code 5909990079384, as part of the drug program: treatment of highly differentiated pancreatic neuroendocrine tumour (ICD 10: C25.4). Verification analysis. No: AOTM-OT-4351-5/2013. Completion date: June 20, 2013 Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-059-2013/059-2013-zlc [Accessed July 11, 2016].

53. The clinical efficacy analysis of sunitinib (Sutent®) in the treatment of unresectable highly differentiated progressive pancreatic neuroendocrine tumours (locally advanced or metastatic) in adult patients. Cracow 2012. Instytut Arcana Sp. z o.o. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-059-2013/059-2013-zlc [Accessed July 11, 2016].

54. Sunitinib in the treatment of unresectable highly differentiated progressive pancreatic neuroendocrine tumours in adult patients – economic analysis and the influence analysis on the healthcare system. Cracow 2012. Instytut Arcana Sp. z o.o. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-059-2013/059-2013-zlc [Accessed July 11, 2016].

55. The supplement of the Arcana Institute analysts to the HTA report for the Sutent® formulation according to the AOTM comments included in the letter of the Minister of Health MZ-PLR-460-17945-8/MG/13 (ref. No: R12121659). Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2013/zlc-059-2013/059-2013-zlc [Accessed July 11, 2016].

56. Agency for Health Technology Assessment (AOTM). Recommendation No 95/2012 of October 30, 2012 of the President of the Agency for Health Technology Assessment (AOTM) on the reimbursement of the medicinal product Votrient (pazopanib), film-coated tablets, 200 mg, 30 capsules; EAN code 5909990764877, as part of the drug program: “Treatment of renal cancer using pazopanib.” Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2012/844-materialy-2012/195-072-2012-zlc [Accessed July 11, 2016].

57. Agency for Health Technology Assessment (AOTM), Department for the Healthcare System. An application for the medicinal product Votrient (pazopanib) to be put on the reimbursement list in the indication: 1st line treatment of patients with renal cell carcinoma (RCC); treatment of patients previously treated with cytokines due to advanced renal cell carcinoma (RCC). Verification analysis. No: AOTM-DS-433-10/2012. Completion date: October 15, 2012. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2012/844-materialy-2012/195-072-2012-zlc [Accessed July 11, 2016].

58. Votrient® (pazopanib) in the 1st line treatment of the advanced renal cell carcinoma. Clinical analysis. Version 2.2. Aestimo s.c. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2012/844-materialy-2012/195-072-2012-zlc [Accessed July 11, 2016].

59. Votrient® (pazopanib) in the 1st line treatment of the advanced renal cell carcinoma. Economic analysis. Version 2.2. Aestimo s.c. Available from: http://bipold.aotm.gov.pl/index.php/zlecenia-2012/844-materialy-2012/195-072-2012-zlc [Accessed July 11, 2016].

60. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. 5.20 REGORAFENIB 40 mg tablet, 84; Stivarga®; Bayer Australia Ltd. March 2015 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2015-03/regorafenib-stivarga-psd-03-2015 [Accessed July 13, 2016].

61. Pan-Canadian Oncology Drug Review (pCODR). PCODR Expert Review Committee (pERC) Final Recommendation. Final Recommendation for Regorafenib (Stivarga) for GIST pERC Meeting: February 20, 2014; Reconsideration Meeting April 17, 2014. Final recommendation: May 2, 2014. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-stivarga-gist-fn-rec.pdf [Accessed July 13, 2016].

62. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Clinical Guidance Report: Regorafenib (Stivarga) for Gastrointestinal Stromal Tumors. May 2, 2014. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-stivarga-gist-fn-cgr.pdf [Accessed July 13, 2016].

63. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Economic Guidance Report: Regorafenib (Stivarga) for Gastrointestinal Stromal Tumours. May 2, 2014. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-stivarga-gist-fn-egr.pdf [Accessed July 13, 2016].

64. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. 4.05 SORAFENIB tablet, 200 mg Nexavar®, Bayer Australia Ltd. November 2015 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2015-11/sorafenib-nexavar-psd-11-2015 [Accessed July 11, 2016].

65. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. 7.05 SORAFENIB 200 mg, tablet; Nexavar®; Bayer Australia Ltd. March 2015 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2015-03/sorafenib-nexavar-psd-03-2015 [Accessed July 13, 2016].

66. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. 6.2 SORAFENIB, 200MG TABLET, NEXAVAR®, BAYER AUSTRALIA LTD. July 2014 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2014-07/sorafenib-psd-07-2014 [Accessed July 13, 2016].

67. Pan-Canadian Oncology Drug Review (pCODR). PCODR Expert Review Committee (pERC) Final Recommendation. Final Recommendation for Sorafenib (Nexavar) for Differentiated Thyroid Cancer pERC Meeting: April 16, 2015; pERC Reconsideration Meeting: July 2, 2015. Final recommendation: July 16, 2015. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr_sorafenib_nexavar_dtc_fn_rec.pdf [Accessed July 13, 2016].

68. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Clinical Guidance Report: Sorafenib (Nexavar) for Differentiated Thyroid Cancer. July 16, 2015. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr_sorafenib_nexavar_dtc_fn_cgr.pdf [Accessed July 13, 2016].

69. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Economic Guidance Report: Sorafenib (Nexavar) for Differentiated Thyroid Cancer. July 16, 2015. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr_sorafenib_nexavar_dtc_fn_egr.pdf [Accessed July 13, 2016].

70. National Institute for Health and Care Excellence (NICE). Crizotinib for previously treated nonsmall-cell lung cancer associated with an anaplastic lymphoma kinase fusion gene. NICE technology appraisal guidance 296, issued: September 2013. Available from: guidance.nice.org.uk/ta296 [Accessed July 11, 2016].

71. Duarte A, Burch J, Smith A, et al. Crizotinib for the treatment of previously treated non-small-cell lung cancer associated with an anaplastic lymphoma kinase (ALK) fusion gene: A Single Technology Appraisal. Centre for Reviews and Dissemination/Centre for Health Economics, 2013. Available from: https://www.nice.org.uk/guidance/ta296/evidence [Accessed July 13, 2016].

72. Crizotinib for the second line treatment of ALK positive non-small cell lung cancer. STA Submission. November 2012. Available from: https://www.nice.org.uk/guidance/ta296/evidence [Accessed July 13, 2016].

73. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. 4.1 CRIZOTINIB 200 mg capsule, 60 and 250 mg capsule, 60; Xalkori®; Pfizer Australia Pty Ltd. November 2014 PBAC Meeting. http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2014-11/crizotinib-psd-11-2014 [Accessed July 11, 2016].

74. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. 7.11 CRIZOTINIB, 200 mg capsule, 60 and 250 mg capsule, 60, Xalkori®, Pfizer Australia Pty Ltd. March 2014 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2014-03/crizotinib [Accessed July 11, 2016].

75. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. Product: Crizotinib, 200 mg and 250 mg, capsule, Xalkori®. November 2013 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2013-11/crizotinib [Accessed July 11, 2016].

76. Pan-Canadian Oncology Drug Review (pCODR). PCODR Expert Review Committee (pERC) Final Recommendation. Final Recommendation for Crizotinib (Xalkori) Resubmission for Advanced NSCLC pERC Meeting: February 21, 2013; pERC Reconsideration Meeting: April 18, 2013. Final recommendation: May 2, 2013. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-xalkoriresub-fn-rec.pdf [Accessed July 11, 2016].

77. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Clinical Guidance Report: Crizotinib (Xalkori) Resubmission for Advanced Non-Small Cell Lung Cancer. September 19, 2014. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-xalkoriresub-fn-cgr.pdf [Accessed July 11, 2016].

78. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Economic Guidance Report: Crizotinib (Xalkori) Resubmission for Advanced Non-Small Cell Lung Cancer. May 2, 2013. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-xalkoriresub-fn-egr.pdf [Accessed July 11, 2016].

79. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. 7.12 EVEROLIMUS, tablets, 5 mg and 10 mg, Afinitor®, Novartis Pharmaceuticals Australia Pty Ltd. March 2014 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2014-03/everolimus-pnet [Accessed July 13, 2016].

80. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. Product: Everolimus, tablets, 5 mg and 10 mg, Afinitor®. November 2012 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2012-11/everolimus-pnet [Accessed July 13, 2016].

81. Pan-Canadian Oncology Drug Review (pCODR). PCODR Expert Review Committee (pERC) Final Recommendation. Final Recommendation for Everolimus (Afinitor) for pNETs pERC Meeting: June 21, 2012; pERC Reconsideration Meeting: August 16, 2012. Final recommendation: August 30, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-afinitor-fn-rec.pdf [Accessed July 13, 2016].

82. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Clinical Guidance Report: Everolimus (Afinitor) for Pancreatic Neuroendocrine Tumours. August 30, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-afinitor-pnets-fn-cgr.pdf [Accessed July 13, 2016].

83. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Economic Guidance Report: Everolimus (Afinitor) for Pancreatic Neuroendocrine Tumours. August 30, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-afinitor-pnets-fn-egr.pdf [Accessed July 13, 2016].

84. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. ADDENDUM- AUGUST 2013. Product: SUNITINIB, capsule, 12.5 mg, 25 mg, 50 mg (as malate), Sutent®. August 2013 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2013-08/sunitinib [Accessed July 11, 2016].

85. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. ADDENDUM. Product: Sunitinib, capsule, 12.5 mg, 25 mg, 50 mg (as malate), Sutent®. July 2012 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2012-07/sunitinib [Accessed July 11, 2016].

86. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. Product: Sunitinib malate, capsule, 12.5 mg, 25 mg and 50 mg (base), Sutent®. March 2012 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2012-03/sunitinib-malate [Accessed July 11, 2016].

87. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. Product: Sunitinib malate, capsules, 12.5 mg, 25 mg and 50 mg (base), Sutent®. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2011-07/pbac-psd-sunitinib-malate-july11 [Accessed July 11, 2016].

88. Pan-Canadian Oncology Drug Review (pCODR). PCODR Expert Review Committee (pERC) Final Recommendation. Final Recommendation for sunitinib malate (Sutent) for pNETs pERC Meeting: February 16, 2012; pERC Reconsideration: April 19, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-sutent-pnet-fn-rec.pdf [Accessed July 11, 2016].

89. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Clinical Guidance Report: Sunitinib malate (Sutent) for pancreatic neuroendocrine tumours. May 3, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-sutent-pnet-fn-cgr.pdf [Accessed July 11, 2016].

90. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Economic Guidance Report: Sunitinib malate (Sutent) for pancreatic neuroendocrine tumours. May 3, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-sutent-pnet-fn-egr.pdf [Accessed July 11, 2016].

91. National Institute for Health and Care Excellence (NICE). Pazopanib for the first-line treatment of advanced renal cell carcinoma. NICE technology appraisal guidance 215. Issued: February 2011 last modified: August 2013. Available from: https://www.nice.org.uk/guidance/ta215 [Accessed July 11, 2016].

92. Kilonzo M, Hislop J, Elders A, et al. Pazopanib for the first line treatment of patients with advanced and/or metastatic renal cell carcinoma: A Single Technology Appraisal. Aberdeen HTA Group, Institute of Applied Health Sciences, University of Aberdeen, 2010. https://www.nice.org.uk/guidance/TA215/documents/renal-cell-carcinoma-first-line-metastatic-pazopanib-evidence-review-group-report2 [Accessed July 11, 2016].

93. Pazopanib (Votrient®) for the first-line treatment of patients with advanced renal cell carcinoma (RCC). Single technology appraisal (STA). 16 April 2010. GlaxoSmithKline UK. Available from: https://www.nice.org.uk/guidance/TA215/documents/renal-cell-carcinoma-first-line-metastatic-pazopanib-manufacturer-submission-submission2 [Accessed July 11, 2016].

94. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. Product: PAZOPANIB, tablet, 200 mg and 400 mg (as hydrochloride), Votrient®. March 2012 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2012-03/pazopanib [Accessed July 11, 2016].

95. Pharmaceutical Benefits Advisory Committee (PBAC). Public summary document. Product: PAZOPANIB, tablets, 200 mg and 400 mg (as hydrochloride), Votrient®. July 2010 PBAC Meeting. Available from: http://www.pbs.gov.au/info/industry/listing/elements/pbac-meetings/psd/2010-07/pbac-psd-Pazopanib-july10 [Accessed July 11, 2016].

96. Pan-Canadian Oncology Drug Review (pCODR). PCODR Expert Review Committee (pERC) Final Recommendation. Final Recommendation for Pazopanib Hydrochloride (Votrient) for metastatic renal cell carcinoma pERC Meeting: October 20, 2011; pERC Reconsideration Meeting: December 15, 2011. Final recommendation: January 5, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-votrientmrcc-fn-rec.pdf [Accessed July 11, 2016].

97. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Clinical Guidance Report: Pazopanib hydrochloride (Votrient) for metastatic renal cell carcinoma. January 5, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-votrientmrcc-fn-cgr.pdf [Accessed July 11, 2016].

98. Pan-Canadian Oncology Drug Review (pCODR). Pan-Canadian Oncology Drug Review Final Economic Guidance Report: Pazopanib hydrochloride (Votrient) for metastatic renal cell carcinoma. January 5, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-votrientmrcc-fn-egr.pdf [Accessed July 11, 2016].

99. Demetri GD, Reichardt P, Kang YK, et al.; GRID study investigators. Efficacy and safety of regorafenib for advanced gastrointestinal stromal tumours after failure of imatinib and sunitinib (GRID): an international, multicentre, randomised, placebo-controlled, phase 3 trial. Lancet 2013;381(9863):295-302.

100. Brose MS, Nutting CM, Jarzab B, et al.; DECISION investigators. Sorafenib in radioactive iodine-refractory, locally advanced or metastatic differentiated thyroid cancer: a randomised, double-blind, phase 3 trial. Lancet 2014;384(9940):319-28.

101. Shaw AT, Kim DW, Nakagawa K, et al. Crizotinib versus chemotherapy in advanced ALK-positive lung cancer. N Engl J Med 2013;368(25):2385-94.

102. Yao JC, Shah MH, Ito T, et al.; RAD001 in Advanced Neuroendocrine Tumors, Third Trial (RADIANT-3) Study Group. Everolimus for advanced pancreatic neuroendocrine tumors. N Engl J Med 2011;364(6):514-23.

103. Ito T, Okusaka T, Ikeda M, et al. Everolimus for advanced pancreatic neuroendocrine tumours: a subgroup analysis evaluating Japanese patients in the RADIANT-3 trial. Jpn J Clin Oncol 2012;42(10):903-11.

104. Raymond E, Dahan L, Raoul JL, et al. Sunitinib malate for the treatment of pancreatic neuroendocrine tumors. N Engl J Med 2011;364(6):501-13.

105. Sternberg CN, Davis ID, Mardiak J, et al. Pazopanib in locally advanced or metastatic renal cell carcinoma: results of a randomized phase III trial. J Clin Oncol 2010;28(6):1061-8.

106. Sternberg CN, Hawkins RE, Wagstaff J, et al. A randomised, double-blind phase III study of pazopanib in patients with advanced and/or metastatic renal cell carcinoma: final overall survival results and safety update. Eur J Cancer 2013;49(6):1287-96.

107. Brose MS, Jarzab B, Elisei R, et al. Updated overall survival analysis of patients with locally advanced or metastatic radioactive iodine-refractory differentiated thyroid cancer (RAI-rDTC) treated with sorafenib on the phase 3 DECISION trial. J Clin Oncol. 2014;32(Suppl):abstr 6060.

108. Shaw AT, Kim DW, Nakagawa K, et al. Phase III study of crizotinib vs pemetrexed od docetaxel chemotherapy in patients with advanced ALK-positive NSCLC (PROFILE 1007). Abstract 2862 and presentation, presented at the 37th ESMO Congress, Vienna, Austria, 2012. Available from: http://oncologypro.esmo.org/Meeting-Resources/ESMO-2012/Phase-III-study-of-crizotinib-versus-pemetrexed-or-docetaxel-chemotherapy-in-patients-with-advanced-ALK-positive-non-small-cell-lung-cancer-NSCLC-PROFILE-1007 [Accessed July 14, 2016]

109. Lombard-Bohas C, Van Cutsem E, Capdevila J, et al. Updated Survival and Safety Data From RADIANT-3 − a Randomized, Double-blind, Placebo-controlled, Multicenter, Phase III Trial of Everolimus in Patients With Advanced Pancreatic Neuroendocrine Tumours (pNET). Eur J Cancer 2011;47(Suppl 1):S459.

110. Valle J, Nicooli P, Raoul JL, et al. Updated Overall Survival Data From a Phase III Study of Sunitinib Vs. Placebo in Patients With Advanced, Unresectable Pancreatic Neuroendocrine Tumour (NET). Eur J Cancer 2011;47(Suppl 1):S462.

111. Raymond E, Niccoli P, Raoul J, et al. Updated overall survival (OS) and progression-free survival (PFS) by blinded independent central review (BICR) of sunitinib (SU) versus placebo (PBO) for patients (Pts) with advanced unresectable pancreatic neuroendocrine tumors (NET). J Clin Oncol 2011;29(Suppl):abstr 4008.

112. Pan-Canadian Oncology Drug Review (pCODR). PCODR Expert Review Committee (pERC) Final Recommendation. Final Recommendation for Crizotinib (Xalkori) for Advanced NSCLC pERC Meeting: July 19, 2012; pERC Reconsideration Meeting: September 20, 2012. Final recommendation: October 4, 2012. Available from: https://www.cadth.ca/sites/default/files/pcodr/pcodr-xalkorinsclc-fn-rec.pdf [Accessed July 11, 2016].

113. National Health Fund (NHS) England. National Cancer Drugs Fund List Ver6.1, 1st February 2016. Available from: https://www.england.nhs.uk/wp-content/uploads/2016/02/ncdf-list-01-02-16.pdf [Accessed July 7, 2016].

114. Australian Government Department of Health. The Pharmaceutical Benefits Scheme (PBS). Schedule of Pharmaceutical Benefits, effective 1 July 2016. Available from: http://www.pbs.gov.au/publication/schedule/2016/07/2016-07-01-general-schedule.pdf [Accessed July 7, 2016].

115. British Columbia Pharmacare Benefits Lookup. Application Version: 1.1.119 Coverage information was last updated on June 28, 2016. Available from: https://pcbl.hlth.gov.bc.ca/pharmacare/benefitslookup/ [Accessed July 7, 2016].

116. Alberta Health Services Outpatient Cancer Drug Benefit Program. Ministerial Approval: February 3, 2016; Revised: May 12, 2016. Available from: http://www.albertahealthservices.ca/assets/programs/ps-1025651-drug-benefit-list.pdf [Accessed July 7, 2016].

117. Régie de l’assurance maladie. List of medications covered by the basic prescription drug insurance plan. List of Medications. Last Updated on 15 June 2016. Available from: http://www.ramq.gouv.qc.ca/en/regie/legal-publications/Pages/list-medications.aspx [Accessed July 11, 2016].

118. Nova Scotia Department of Health. Formulary, July 2016. Available from: http://novascotia.ca/dhw/pharmacare/formulary.asp [Accessed July 11, 2016].

119. New Brunswick Drug Plans Formulary, July 2016. Available from: http://www2.gnb.ca/content/gnb/en/departments/health/MedicarePrescriptionDrugPlan/NBDrugPlan/ForHealthCareProfessionals/NewBrunswickDrugPlansFormulary.html [Accessed July 11, 2016].

120. Manitoba Drug Benefits and Interchangeability Formulary Amendments. Bulletin #74, effective October 17, 2013. Available from: http://www.gov.mb.ca/health/mdbif/bulletin74.pdf [Accessed July 14, 2016].

121. The Prince Edward Island Pharmacare Formulary. Updated: June 2016. Available from: http://healthpei.ca/formulary [Accessed July 7, 2016].

122. The Ontario Ministry of Health and Long-Term Care. Ministry of Health and Long-term Care Exceptional Access Program (EAP). EAP Reimbursement Criteria for Frequently Requested Drugs. Updated: August 1, 2015 Available from: https://www.formulary.health.gov.on.ca/formulary/ [Accessed July 7, 2016].

123. The Saskatchewan Cancer Agency Drug Formulary, Apr 8, 2016. Available from: www.saskcancer.ca/Formulary%2008-04-2016 [Accessed July 7, 2016].

124. The Newfoundland and Labrador Prescription Drug Program. Criteria for the coverage of special authorization drugs. Available from: http://www.health.gov.nl.ca/health/prescription/covered_specialauthdrugs.html [Accessed July 7, 2016].

125. Kranc R, Czech M. Decision-making processes related to drug pricing and reimbursement. Is Poland far away from global standards? JHPOR 2012;1:29-38.

126. Jönsson L, Sandin R, Ekman M, Ramsberg J, Charbonneau C, et al. Analyzing overall survival in randomized controlled trials with crossover and implications for economic evaluation. Value Health 2014;17(6):707-13.
127. Chabot I, LeLorier J, Blackstein ME. The challenge of conducting pharmacoeconomic evaluations in oncology using crossover trials: the example of sunitinib for gastrointestinal stromal tumour. Eur J Cancer 2008;44(7):972-7.