AHTAPol appraisals and reimbursement decisions made by the Polish Minister of Health in oncological indications – surrogate endpoints as an argument for negative positions of the Transparency Council and recommendations of the AHTAPol's President.


Name Affiliation
Anna Kordecka
HTA Registry
Joanna Łapa
HTA Registry
Ewa Walkiewicz-Żarek
HTA Registry
Mariusz Kordecki
HTA Registry
contributed: 2019-03-21
final review: 2019-05-19
published: 2019-07-18

Background: Reimbursement decisions are a part of the decision-making process specified in applicable legal provisions. In the Polish system, the key stakeholders are the Agency for Health Technology Assessment and Tariff System (AHTAPol) (President of the Agency and the Transparency Council), and the Minister of Health (MoH).

Methods: The analysis included orders regarding oncological drugs, directed to the AHTAPol's assessment in the years 2009-2018 and a review of announcements of the MoH published between 01.2012-09.2018. Reasons for negative positions/recommendations (nCP/nRP), especially lack of results or lack of a statistically significant improvement in overall survival (OS), were analysed.

Results: For the analysed orders, 16% of the Council’s positions and 18% of the President`s recommendations were positive. 40% of positions and recommendations were negative. Limitations of OS results were pointed out in 51% of nCP and 66% of nPR (QoL: 28% of nCP; 41% of nPR). In 2018, a fewer number of nCP/nPR with that justification was noted. Over half of the analysed orders ended with a reimbursement decision, but it is usually significantly postponed in time (for drug programmes: an average of 11 months after a PR is issued, not including time needed for AHTAPol's appraisal).

Conclusions:  Limitated results regarding clinically significant endpoints and drawing conclusions based on surrogate endpoints were the frequent reasons for nCP/nPR. Differences in the approach to surrogate endpoints between regulatory agencies and the AHTAPol were identified. The decision to include a technology in the reimbursement announcement is significantly delayed in relation to the President`s recommendation. 

Keywords: AHTAPol, Ministry of Health, Health Technology Assessment, surrogate endpoints, overall survival, oncology, haematology, reimbursement, Council position, President recommendation


The goal of conducting a reimbursement policy is to allocate limited financial resources while maintaining the widest possible access to medicinal products characterised by proven efficacy and safety. [1] Reimbursement decisions are a part of the decision-making process specified in applicable legal provisions. In the Polish reimbursement system, the key stakeholders involved in shaping the above-mentioned policy are the Agency for Health Technology Assessment and Tariff System (AHTAPol, pol. AOTMiT) (as part of its structures, the President of the Agency and the Transparency Council – supporting the President in the assessment process), as well as the Minister of Health (MoH). Their competences have been characterised in the Act on reimbursement of medicines, foodstuffs intended for particular nutritional uses and medical devices ( “Reimbursement Act”) [2] and the Act on healthcare services financed from public funds [3].

For several years, changing trends in the assessment of health technologies by regulators (European Medicines Agency (EMA), Food and Drug Administration (FDA)) could be observed, in particular in therapies applying for early access. In such cases, more and more often, decisions about the granting of marketing authorisation are made based on surrogate endpoints (surrogates) and/or lower-quality evidence (e.g. single-arm studies). HTA agencies adopt various policies regarding the use of surrogates in health technology assessment. [4]

This analysis takes into account the practice of the AHTAPol in the assessment of technologies applying for reimbursement in oncological indications, with particular reference to the reasons for negative decisions being issued. Furthermore, announcements of the MoH were analysed to show the relationship between positions presented by bodies of the AHTAPol and the listing of substances on the list of reimbursed medicines.


The analysis included orders directed to the AHTAPol's assessment in the years 2009-2018. The analysis includes decisions issued by the Transparency Council/Consultative Council (since January 2007 the Consultative Council, in February 2012 replaced by the Transparency Council) and the President of the AHTAPol for oncological drugs. For the identified orders, announcements of the MoH on reimbursed medicines, foodstuffs intended for particular nutritional uses and medical devices published between 01.2012 and 09.2018 have been reviewed.

Decisions on orders directed to the AHTAPol

The orders were identified using the Public Information Bulletin (BIP) of the AHTAPol [5] filtered for the above-mentioned period of time. The orders made out to the AHTAPol under Article 35.1 of the Reimbursement Act, concerning the reimbursement of new health technologies (for directed to the Agency after January 2012) and orders regarding the preparation of the recommendation of the Agency’s President pursuant to Article 31 of the Act on health care services financed from public funds (for orders assessed by the Agency before the Reimbursement act came into force) in oncological indications. Article 35.1 of the Reimbursement Act includes the mode of the AHTAPol‘s assessment regarding medicines, foodstuffs intended for particular nutritional uses and medical devices for which there is no reimbursed equivalent in a given indication. The analysis did not include supportive therapies and radiopharmaceuticals.

Qualification of the order was carried out on the basis of tabs dedicated to individual orders, taking into account in particular the "indication" area. The identified orders were aggregated in two areas of indications (haematological and other oncological indications [solid tumours]) and divided into 25 groups of indications selected with regard to the tumour’s location.

The publication constitutes an analysis of orders regarding the reimbursement coverage. The following categories were distinguished in the analysis: A – general list, B – drug programme, C – chemotherapy catalogue.

On the basis of issued Council positions (CP) and recommendations of the President (PR), orders were qualified to the following groups:

  • positive: the position (pCP)/recommendation (pPR) is positive and was expressed directly in a decision of the authority in the entire scope of the reimbursement application,
  • negative: the position (nCP)/recommendation (nPR) is negative and was expressed directly in a decision of the authority in the entire scope of the reimbursement application,
  • conditional: the position (cCP)/recommendation (cPR) is positive or partially positive, provided that a certain condition is met.

Based on the justification presented in the CP and the PR, reasons for issuing negative decisions were classified to at least one of 9 groups:

  1. OS – lack of results or no statistically significant improvement in overall survival (OS) has been proven, little benefit was demonstrated,
  2. QoL – lack of results or no statistically significant improvement in quality of life (QoL) has been proven,
  3. Efficacy – the results (not referring directly to OS or QoL) included in the clinical analysis were subject to high uncertainty,
  4. Safety profile – the therapy’s uncertain or unfavourable safety profile,
  5. Cost-effectiveness – lack of demonstrated cost-effectiveness of the therapy,
  6. Economic analysis – incorrectly conducted economic analysis or economic analysis associated with high uncertainty,
  7. Population – the evidence did not match the population indicated in the reimbursement application,
  8. Methodology – the studies included in the analysis were characterised by poor quality or the wrong methodology of clinical analysis was used,
  9. Other.

Based on the justification presented in the CP and the PR, reasons for issuing conditional decisions were classified to at least one of the following 4 groups:

  1. Price – regarding the condition of reducing the price per se or "reducing the cost of therapy to the level of cost-effectiveness",
  2. RSS (risk-sharing agreements– when the condition consists in the development of a risk-sharing instrument or when the RSS proposed by the MAH has not been accepted,
  3. Change of stipulations – in particular when those changes concern stipulations of a drug programme (e.g. population reduction),
  4. Other – e.g. in the case a decision limited in time and subject to conditions is issued, after the laps of which a reassessment will take place or in the case of which reimbursement will be granted under the condition of joining the existing drug programme.

Decisions taken by the Minister of Health

Only orders directed to the AHTAPol until the end of 2017 were the subject of the reimbursement availability analysis. The analysis did not include orders for which it was not possible to identify the cause and effect chain related to a positive/negative reimbursement decision being taken (in particular orders directed to the AHTAPol before 2012).

Information on reimbursement of individual substances was taken from the published announcement of the Minister of Health on reimbursed medicines, foodstuffs intended for particular nutritional uses and medical devices, which are published from January 2012 for medicines on the open list and from May 2012 for medicines in the B and C catalogue.

In order to estimate the time needed to obtain reimbursement for the technologies analysed herein, the date of the settlement understood as the date of the President's recommendations was taken into account. Average and the median waiting time for reimbursement coverage were taken into account.  Reimbursement success was defined as the date of the first inclusion of the substance in the announcement of the MoH in accordance with the indication as applied. Additionally, in the case of orders not concluded with reimbursement, further actions of the entity applying for reimbursement of the  technology, and the waiting time has been included for the entire analysed pathway (sequential approach).


Decisions on orders directed to the AHTAPol – Analysis of the Council's positions and the President's recommendations in the years 2009-2018

In the 2009-2018, 2090 orders were created on the AHTAPol websites, of which 23% (491) were orders regarding oncological indications. Of the oncology orders, 30% (152) concerned the coverage in accordance with the accepted inclusion criteria for analysis. Chart 1 Presents the distribution of orders sent to the AHTAPol broken down by individual years.

Chart 1. Distribution of orders for reimbursement coverage directed to the AHTAPol in individual years – breakdown by the reimbursement area as applied, (N)

In the analysed period, a steady increase in the number of orders for reimbursement directed to the AHTAPol assessment can be noted.

Of the analysed orders, 16% (25 out of 152) CP and 18% (27 out of 152) RP were positive. Conditional reimbursement was indicated in 38% cCP and cPR. 40% of positions and recommendations were negative (nCP and nPR). 5% of the orders  directed to the Agency for assessment did not result in a position/recommendation being issued due to withdrawal, cancellation or termination of the order. One of the orders included in this group is still being assessed by the AHTAPol at the time of analysis. [6] In the case of one order, no CP has been identified. [7]

The decisions of the two AHTAPol bodies were consistent in 76% (104 out of 144 cases).

Consistent decisions concerned 19 positive, 50 negative and 40 conditional positions/recommendations. The same decisions concerned 76% of orders in haematological indications and 75% of orders in oncological indications.

At the same time, it is worth to emphasise the differences in the number of orders which formed the basis for overlapping between the President’s recommendations with positions of the Council. The overlapping rate for the most frequently considered indication (lung – 22 orders) was 73%. Results for indications in which individual orders were recorded should be treated with a high degree of uncertainty. (Chart 2)

Chart 2. Percentage of overlapping between the Council’s positions and the President’s recommendations in relation to individual indications, (%)


Negative decisions

Chart 3 presents the number of orders completed with individual types of positions/recommendations in particular years and the number of nCP/nPR , in which limitations in the scope of OS were indicated as one of the reasons. In the analysed period of time (2009-2018), almost 50% (72 out of 152) of orders directed to the AHTAPol ended with the issuance of a negative decision by at least one of the Agency’s bodies. Taking into account only the period after the entry into force of the Reimbursement Act (after 2012), this percentage is 45.5% ( 2012 – 10% (2 out of 20), 2013 – 47% (9 out of 19), 2014 – 30% (6 out of 20), 2015 – 63% (10 out of 16), 2016 – 70% (14 out of 20), 2017 – 79% (19 out of 24), and 29% (5 out of 17) in 2018).

Chart 3. Decisions of the AOTMiT bodies in 2009-2018, (N)

Among the identified negative positions/recommendations (respectively: 48/61, 46/61) the most frequent cause was failure to demonstrate cost-effectiveness of the technology. The above accounted for 79% of nCP and 75% of nPR.

In 52% nCP and 66% of nPR, the justification for the negative decision contained a reference to the poor quality of the studies or the wrong methodology of clinical analysis. The reasons listed included i.a. a small number of RCTs or their low quality, lack of long-term follow-up confirming the effectiveness of interventions [8] or the lack of direct comparisons of efficacy and "the scale of adverse effects" due to the small, diverse group of patients included in the trial [9]. In 34% of both the nCP/nPR, results in the field of efficacy raised doubts (e.g. no/insufficient evidence confirming the efficacy of intervention, limitation of outcomes for other endpoints than clinical relevant, limitations of clinical analysis). The reservations did not directly concern the measures of treatment benefit, including OS and QoL.

Limitations of OS results (lack of results/statistically insignificant results/no median achieved/slight benefit demonstrated) were pointed out in 31 (51%) of nCP and 40 (66%) of nPR. Cases in which, despite showing the superiority of the proposed technology in the scope of OS, its insufficient level was emphasised in 4 nCP and 6 nPR. Table S1 (Supplement) presents the arguments of the AHTAPol bodies referring to limitations of results in the scope of OS.

In the Agency's opinion, the QoL constitutes an important measure of clinical benefit, especially when assessing the efficacy of health technologies used in advanced stages of cancer, when obtaining a complete cure it usually not possible. Objections regarding the absence/limitation of QoL were raised in 17 (28%) of nCP and 25 (41%) of nPR. In the case of 14 nCP and 19 nPR, the objection of lack of results or lack of demonstrated statistically significant improvement in the scope of OS was correlated with a similar objection in the area of QoL.

The largest number of discrepancies in the justifications of negative positions/recommendations concerned the economic analysis (12 (20%) of nCP vs 36 (59%) of nPR, respectively) and evidence not corresponding in full to the population of patients specified in the reimbursement application (18 (30%) nCP vs 5 (8%) nPR).

Differences in the frequency of the most common reasons for nCP and nPR issued since the Reimbursement Act has been in force were analysed. In the last two years, an increased agreement between CP and PR have been observed. The analysis has shown the argument of lack of cost-effectiveness has been used less frequently last year. At the same time, it should be emphasised that the year 2018 is characterised by a small number of nCP and nPR in total. What is more only in 3 positions/recommendations, limitations of the OS analysis were identified as the cause of nCP/nPR. (Chart S1, Supplement)

The justifications included in negative positions/recommendations for oncological indications (solid tumours) and haematological indications were also compared. The biggest differences between the reasons for nCP/nPR can be observed in nPR referring to the safety profile of the therapy. This argumentation was indicated in 40% of nPR (32% of nCP) in oncological indications vs 71% of nPR (65% of nCP) in haematological indications.

Low quality of the studies included in the application or methodological errors were pointed out in 42% of nCP and 58% of nPR in oncological indications and 50% of nCP and 71% of nPR for haematological indications. It is noteworthy that the majority of haematological indications are rare (or ultra-rare) diseases, and thus drawing conclusions on the efficacy of drugs is often conducted on the basis of lower-quality studies (smaller sample, single-arm studies).

In the analysed time period, a similar percentage of nCP/nPR may be observed in relation to orders in haematological and oncological indications (haematological indications – 38% vs oncological 41%, in both nCP and nPR). However, the difference between the number of orders directed to the AHTAPol for assessment in oncological indications as compared to the haematological indications (109 vs 42, i.e. more than double) should be emphasised. (Chart 4)

Chart 4. Reasons for negative positions of the Council and negative recommendations of the President in oncological indications and haematological indications, (%)

In the period included in the analysis, 74 orders directed to the AHTAPol resulted in a condition being imposed by at least one of the Agency’s bodies (58 cCP and 57 cPR). It should be emphasised that in the comparison to past years, 2018 is characterised by increased number of conditional decisions (59% CP and 53% PR vs 50% CP, 25% PR in 2017, 35% CP, 30% PR in 2016 and 31% CP, 44% PR in 2015). (Chart 3)

Among the oncological indications, 35 (32%) cCP and 37 (34%) cPR were reported, while there were 18 (42%) cCP and 15 (35%) cPR in haematological indications.

The conditions for reimbursement coverage were classified into four groups: price, RSS, change of stipulations and other. (Table 1)  

Table 1. Type of condition indicated in positions and recommendations

The most common type of condition was to propose an adequate risk sharing instrument in 69% of positions and 61% of recommendations. Conditional positions and recommendations often also indicate the need to reduce the price of the health technology, allowing for obtaining cost-effectiveness of treatment (59% of cCP and 47% of cPR). Changes to the stipulations of the drug programme were a condition for reimbursement coverage in 31% of cCP and 19% of cPR.


Decisions taken by the Minister of Health - an analysis of reimbursement announcements, 2012-2018

MoH decides to reimburse a product by listing it in the reimbursement announcement. Reimbursement announcements are published regularly, every two months, starting from 2012 (01. 2012 for the open list, 05.2012 for drug programmes and the chemotherapy catalogue). 122 orders were included in the analysis of reimbursement decisions.

Basic approach

Over half of the analysed orders (57%, 70 out of 122) ended with a positive reimbursement decision in line with the applied indication.

An overwhelming majority of orders concerned applications for funding under a drug programme (103 orders). 61 health technologies (59%) were granted positive reimbursement decision. Technologies included in the reimbursement announcement within the drug programme were listed on average 11 months (333 days) after the decision was issued by the AHTAPol. Due to the occurrence of outliers for individual orders, the median differed from the average and amounted to 213 days. It should be noticed that average time from a submission new order made by MoH to AHTAPol decision takes 89 days (85 days for drug programmes only).

Those orders which resulted in a positive decision, for which the waiting time for reimbursement  amounted up to 1000 days (2.7 years) are particularly noteworthy. Among the analysed orders, the longest reported waiting time was the 1,254-day period for temsirolimus used under a drug programme in treatment of advanced renal cell carcinoma in the group of patients with an unfavourable prognosis (Table 2).

Table 2. Orders for drug technologies characterised by the longest waiting times for reimbursement from among technologies included in reimbursement announcements