Presentation
A Risk-Based Approach to Determining Non-Inferiority Margin (d-value) for Comparative Use Human Factors Studies for Generics
DescriptionUnder Section 505(j), an Abbreviated New Drug Application (ANDA) applicant must demonstrate that their proposed generic drug-device combination product (“generic product”) can be substituted for the Reference Listed Drug (RLD) without the intervention of a healthcare provider or without additional training prior to use of the generic product. To fulfil this requirement, it is strongly recommended that the user interface of the device constituent part of the proposed generic product is designed to be as similar as possible to the RLD. However, the two products do not have to be identical, in which case the manufacturers may need to provide additional data to support substitutability.
The US Food and Drug Administration (FDA) published its Draft Guidance in 2017 – “Comparative Analyses and Related Comparative Use Human Factors Studies for a Drug-Device Combination Product Submitted in an ANDA: Draft Guidance for Industry” – which is aimed at supporting manufacturers of generic products in fulfilling the ANDA’s Human Factors (HF) requirements. The Draft Guidance primarily recommends three things: (1) design the user interface of your proposed generic product to be as close as possible to the RLD, (2) conduct comparative Threshold Analyses to understand any design differences, and (3) if there are “other” than minor design differences, conduct a Comparative Use Human Factors (CUHF) study to demonstrate that your proposed generic product is “non-inferior” to the RLD. In other words, a generic product can be different in the user interface from the RLD but additional data from a CUHF study is required to demonstrate that the use error rates of the proposed generic product is not statistically greater than that of the RLD by more than an acceptable margin, called a non-inferiority (NI) margin or d-value. The NI margin is a key parameter in determining the statistical sample size for the study, as well as performing the statistical hypothetical testing. But the biggest question is how do we determine the acceptable NI margin?
The Draft Guidance says: “the best choice of d enables creating a statistical test through which one can demonstrate that the error rate using the proposed generic combination product will not be unacceptably greater than that of the RLD while acknowledging and allowing for the inherent variability in use error rates”. It provides some basic principles of choosing a d-value but falls short of providing details on how one should determine the d-value for their proposed generic product. There are other open questions arising from the Draft Guidance, however, the determination of the d-value has baffled many HF practitioners. This topic has become a challenge and has been discussed in many forums, such as the HFES Healthcare Symposium and the Center for Research on Complex Generics (CRCG) workshops, without much fruition. Many HF practitioners and manufacturers of generic products have resorted to using a randomly selected “small value” or a d-value of 10% which has been provided only as an example in the Draft Guidance. These arbitrary selections could falsely demonstrate that the proposed generic product is inferior (or not non-inferior to be precise) compared to the RLD, or in the worst case, they may falsely demonstrate that the proposed generic is non-inferior to the RLD, causing harm to the patients. We at Cambridge Consultants have researched and developed a methodology to determine an appropriate d-value for individual products, reducing the risks of such false positive or false negative outcomes.
The concept of a non-inferiority test and NI margin (d-value) has essentially originated from clinical studies. As HF practitioners, we have to first understand what it actually means. In simple terms, this NI margin allows the proposed generic product to be slightly inferior, in the worst case, by this “small” but “unimportant” margin without compromising the clinical effect and safety profile of the generic product compared to the RLD. In other words, there should be no “important” loss of efficacy and safety if the generic product is used instead of the RLD. In clinical studies, this margin is determined based on different approaches, e.g. taking a proportion of the “superiority margin” of the RLD compared to a placebo from prior studies; or assessing the efficacy of other similar interchangeable products on the market and comparing it with the RLD for the difference. However, these approaches are either not feasible or not appropriate for a CUHF study. This proposed Oral Presentation will highlight these shortfalls and present a methodology for calculating the NI-margin.
Cambridge Consultants has taken the essence of the non-inferiority clinical study designs, i.e. the basic principles of how one should consider determining the NI margin, and formulated a risk-based approach of determining a sensible NI margin, appropriate for CUHF studies. This presentation will aim to take the audience, especially the HF practitioners and manufacturers of generic products, through a step by step approach. This approach is based on the principles of safety-and-effectiveness (which is similar to clinical safety and efficacy) but focuses on the use process, potential use errors and how they affect the safety and effectiveness of the proposed generic compared to the RLD.
To provide a brief synopsis of the approach for the reviewers of this proposed Oral Presentation, it has the following steps – (1) understanding the baseline, i.e. the use error rates for the RLD through literature review and database searches; (2) conducting thorough Threshold Analyses and determining the critical tasks impacted by the “other” design differences; (3) conducting a thorough use-related risk assessment for the proposed generic product including both safety and effectiveness (e.g. medication errors beyond safety considerations) aspects of the product and estimating the use error rates associated with each critical task impacted by the design differences; (4) determining the maximum acceptable limit of the use error rates that would not change the safety or effectiveness profile of the generic product to an unacceptable level based on the Hazard Analysis, (5) determining the difference in the maximum acceptable limit of the use error rates for the generic product and the baseline use error rates of the RLD for each critical task, and finally (6) selecting the worst case, i.e. the maximum difference of all the relevant critical tasks. This provides a NI margin (d-value) that can be used for the sample size calculation as well as the statistical testing for a CUHF study data.
The FDA’s Draft Guidance proposes statistical test based on the use error rates, but the FDA’s current thinking has shifted towards the statistical test of the overall use success rates. As the success is the flip side of the error/failure, both test methods result in the same conclusion. Therefore, the NI margin (d-value) calculated using the proposed methodology is applicable to both approaches. This methodology, for both approaches, has been successfully applied to a number of CUHF studies which have been reviewed and approved by the FDA. We intend to disseminate this methodology to the wider HF community so that everyone can benefit from it.
The US Food and Drug Administration (FDA) published its Draft Guidance in 2017 – “Comparative Analyses and Related Comparative Use Human Factors Studies for a Drug-Device Combination Product Submitted in an ANDA: Draft Guidance for Industry” – which is aimed at supporting manufacturers of generic products in fulfilling the ANDA’s Human Factors (HF) requirements. The Draft Guidance primarily recommends three things: (1) design the user interface of your proposed generic product to be as close as possible to the RLD, (2) conduct comparative Threshold Analyses to understand any design differences, and (3) if there are “other” than minor design differences, conduct a Comparative Use Human Factors (CUHF) study to demonstrate that your proposed generic product is “non-inferior” to the RLD. In other words, a generic product can be different in the user interface from the RLD but additional data from a CUHF study is required to demonstrate that the use error rates of the proposed generic product is not statistically greater than that of the RLD by more than an acceptable margin, called a non-inferiority (NI) margin or d-value. The NI margin is a key parameter in determining the statistical sample size for the study, as well as performing the statistical hypothetical testing. But the biggest question is how do we determine the acceptable NI margin?
The Draft Guidance says: “the best choice of d enables creating a statistical test through which one can demonstrate that the error rate using the proposed generic combination product will not be unacceptably greater than that of the RLD while acknowledging and allowing for the inherent variability in use error rates”. It provides some basic principles of choosing a d-value but falls short of providing details on how one should determine the d-value for their proposed generic product. There are other open questions arising from the Draft Guidance, however, the determination of the d-value has baffled many HF practitioners. This topic has become a challenge and has been discussed in many forums, such as the HFES Healthcare Symposium and the Center for Research on Complex Generics (CRCG) workshops, without much fruition. Many HF practitioners and manufacturers of generic products have resorted to using a randomly selected “small value” or a d-value of 10% which has been provided only as an example in the Draft Guidance. These arbitrary selections could falsely demonstrate that the proposed generic product is inferior (or not non-inferior to be precise) compared to the RLD, or in the worst case, they may falsely demonstrate that the proposed generic is non-inferior to the RLD, causing harm to the patients. We at Cambridge Consultants have researched and developed a methodology to determine an appropriate d-value for individual products, reducing the risks of such false positive or false negative outcomes.
The concept of a non-inferiority test and NI margin (d-value) has essentially originated from clinical studies. As HF practitioners, we have to first understand what it actually means. In simple terms, this NI margin allows the proposed generic product to be slightly inferior, in the worst case, by this “small” but “unimportant” margin without compromising the clinical effect and safety profile of the generic product compared to the RLD. In other words, there should be no “important” loss of efficacy and safety if the generic product is used instead of the RLD. In clinical studies, this margin is determined based on different approaches, e.g. taking a proportion of the “superiority margin” of the RLD compared to a placebo from prior studies; or assessing the efficacy of other similar interchangeable products on the market and comparing it with the RLD for the difference. However, these approaches are either not feasible or not appropriate for a CUHF study. This proposed Oral Presentation will highlight these shortfalls and present a methodology for calculating the NI-margin.
Cambridge Consultants has taken the essence of the non-inferiority clinical study designs, i.e. the basic principles of how one should consider determining the NI margin, and formulated a risk-based approach of determining a sensible NI margin, appropriate for CUHF studies. This presentation will aim to take the audience, especially the HF practitioners and manufacturers of generic products, through a step by step approach. This approach is based on the principles of safety-and-effectiveness (which is similar to clinical safety and efficacy) but focuses on the use process, potential use errors and how they affect the safety and effectiveness of the proposed generic compared to the RLD.
To provide a brief synopsis of the approach for the reviewers of this proposed Oral Presentation, it has the following steps – (1) understanding the baseline, i.e. the use error rates for the RLD through literature review and database searches; (2) conducting thorough Threshold Analyses and determining the critical tasks impacted by the “other” design differences; (3) conducting a thorough use-related risk assessment for the proposed generic product including both safety and effectiveness (e.g. medication errors beyond safety considerations) aspects of the product and estimating the use error rates associated with each critical task impacted by the design differences; (4) determining the maximum acceptable limit of the use error rates that would not change the safety or effectiveness profile of the generic product to an unacceptable level based on the Hazard Analysis, (5) determining the difference in the maximum acceptable limit of the use error rates for the generic product and the baseline use error rates of the RLD for each critical task, and finally (6) selecting the worst case, i.e. the maximum difference of all the relevant critical tasks. This provides a NI margin (d-value) that can be used for the sample size calculation as well as the statistical testing for a CUHF study data.
The FDA’s Draft Guidance proposes statistical test based on the use error rates, but the FDA’s current thinking has shifted towards the statistical test of the overall use success rates. As the success is the flip side of the error/failure, both test methods result in the same conclusion. Therefore, the NI margin (d-value) calculated using the proposed methodology is applicable to both approaches. This methodology, for both approaches, has been successfully applied to a number of CUHF studies which have been reviewed and approved by the FDA. We intend to disseminate this methodology to the wider HF community so that everyone can benefit from it.
Event Type
Oral Presentations
TimeMonday, March 231:52pm - 2:15pm EDT
LocationGramercy
Medical and Drug Delivery Devices