Adopting new diagnostics: Patient safety delivered within existing resources

Tony Cambridge, Lead Biomedical Scientist

Pathology Management

Adopting new diagnostics: Patient safety delivered within existing resources

14 April 2021 | 12min

Quick Takes

  • Rapid diagnostic innovation requires scrutiny to assure patient risk is minimized

  • In vitro diagnostics undergo pre-market assessment and adopters should review their approach to verification of performance claims

  • Adoption of diagnostic tools can be quicker, cheaper and with improved standardization

Reconsidering how we determine “safe for adoption” in diagnostics

The medical laboratory diagnostics sector is now at the forefront of the world’s consciousness with an increased scrutiny on the manufacture and supply of diagnostic tools to support care delivery across multiple healthcare settings. For the last year, the commercial healthcare industry has been challenged by the SARS-CoV-2 pandemic in ways never experienced before. The same is true for care deliverers, wholly dependent on supply of equipment and consumables, new innovations and the resource pressures that come with adoption of novel solutions.

The scientific community has delivered an impressive amount of innovation in a short period of time, equalled by the diagnostic and manufacturing sectors increasing their operational activities to meet the demands of fast moving healthcare delivery. This progress has been met with equal measures of championing and scepticism. We live in a world where adopters of new and existing technologies are risk averse. Decision makers are conscious of the impact adopting novel solutions can have on patient safety, creating layers of governance throughout the process to ensure the most appropriate decisions are made, minimizing risk to care delivery.

This poses a real and significant question. What level of assurance is required to safely introduce diagnostic tools that could have a positive impact on patient health outcomes? What considerations should be made and do they change depending on resource availability, level of risk to patient wellbeing, and how rapid the solution is required? Indeed, the underlying discussion here is what constitutes ‘safe’.

In the context of existing resources and the rate at which new solutions need to be implemented, can adopters afford to be risk averse? Is there an opportunity to improve the awareness of premarket quality processes so that end user resources are not wasted duplicating effort? Must we evaluate performance characteristics that have already been confirmed?

Assurance is key to adoption

My observations have led me to believe that in many cases the level of assurance sought goes well beyond what is required to assure safe use of many diagnostic solutions, including laboratory assays and testing provided in decentralized locations. All manufacturers of in-vitro diagnostics (IVDs) are required to state the performance characteristics of the marketed test in the ‘information for users (IFU)’. This alerts the adopter to the potential limitations of the test, how rigorous the product has been evaluated before release to market, and what additional work the end user feels is required to validate or verify the performance claims. This, in essence, is a risk assessment for adoption. The level of additional scrutiny will depend on the level of risk, the knowledge and expertise of the adopter, and the severity of harm that could become the patient if inaccurate results are produced and acted upon.

All IVDs approved for market in the European Union (EU) are required to gain a Conformitè Europëenne (CE) mark and every medical device sold into Europe needs to have a current Clinical Evaluation Report (CER). Many IVDs require a notified body to assess the conformity of the product to the standards required for its safe intended use. Europe is set to move away from the IVD Directive, which allowed IVDs which were then categorized as low risk to be self-certified by the manufacturer. The Directive will soon be replaced by the EU-IVD Regulation (IVDR) and EU-Medical Device Regulation (MDR) which have re-categorized a number of IVDs based on risk to the patient, require additional evaluation data, and wider assessment by notified bodies. Would this additional scrutiny change the way an adopter approaches verification? The landscape is further complicated by Brexit and UK compliance, whilst CE marked IVDs can continue to be marketed until the middle of 2023.

The process of verification

Most recently, as the global marketplace became flooded by SARS-CoV-2 tests, national verifications were rapidly mobilized to identify recommended tests and devices. National bodies such as NHS England performed verifications on a small selection of diagnostic products to confirm the performance characteristics. These included medical laboratory SARS-CoV-2 tests such as polymerase chain reaction (PCR) methods, immunoassay antibody tests and lateral flow devices used in decentralized locations. Decisions would have been made with regard to which products to verify first.

Considerations would have included:
  • the manufacturer’s performance claims
  • speed and volume of production
  • logistical assurances around supply chain
  • a proven history of quality assurance of manufacturing
  • resources available for product verification
  • risk to patient outcomes
  • Rapid need for a solution to a public health crisis

Many tests reaching the market did so on the basis of an Emergency Use Authorization (EUA). National regulatory bodies such as the Medicines and Healthcare product Regulatory Agency (MHRA) in the United Kingdom, would have applied regulatory flexibility due to the exceptional circumstances of the pandemic, to grant marketing approval for products submitted with limited but acceptable data on performance. Regulatory bodies in other global territories have similar mechanisms. The EUA is awarded for a limited time only and the manufacturer will be required to submit further performance information for full clearance of the product.

Having reviewed the overall picture it is clear that the early evaluation of these diagnostic solutions stopped short of what could be termed a full validation, no doubt due to the speed the solutions were required in response to the pandemic. The key word is assurance. The verification of these tests at a national level would have given adopters a level of assurance that implementing the tests in the intended settings would be safe, as long as accepted local procedures for adoption of new IVDs would also be followed. Special scrutiny should be applied to the patient samples used in manufacturer’s product testing, the care delivery setting and the numbers of samples used across the concentration range.

Adopters are duty bound to perform their own internal verifications, the aim being to reproduce the manufacturer’s stated performance characteristics and mirror any known verification data produced by other organizations. This is where variability becomes a real risk to standardized patient care. When verifying performance, evaluators must use widely recognized criteria for acceptance or service users risk misunderstanding the utility of the results leading to suboptimal care. There are a number of international sources such as The Clinical & Laboratory Standards Institute (CLSI), for example. There may be a lack of standardization possibly driven by varying knowledge and experience within the diagnostic sector’s many care delivery points.

Identifying resource requirements

A key component of successfully introducing a new product is to understand the resources required for implementation. Expertise from a number of key clinical and scientific roles is required in order to assess the risk around a change process.

The following roles should be considered, and their input planned as soon as the requirement to implement change is known:
  • Clinical oversight, leadership and sign off
  • Scientific – performing evaluations, writing reports, stating initial conclusions
  • Quality and Governance Team – organizational or service specific compliance with standards
  • Information Technology – data management, connectivity of devices, information governance
  • Healthcare workers – patient testing, training and competency, maintenance, quality assurance, consumable management

Where these resources are not available the adopter should consider ways in which they can be accessed. This may include working closely with the manufacturer/distributor’s operatives in designing verifications, implementing and installing IT solutions, designing quality processes, providing procedural documentation, and training resources and materials. Where verification time cannot be covered by scientific staff the adopter should consider accessing resources from outside of their immediate service; this may include outsourcing, using other competent laboratory or research staff, or supervising the equipment provider in generating the data. The last point can be contentious for some as data should, where possible, be independently generated.

Where clinical oversight is not available, provision should be considered from other organizations or through consultancy, likewise for quality management functions provided by companies specializing in these activities. For activities performed by individuals outside of the parent organization, a formal agreement must be in place to address accountability, ownership and responsibilities, whilst credentials of the individuals involved must be assessed and confirmed. All expectations must be clear to all stakeholders.

Committing resources to established processes

A new approach is required within the constraints of resources and funding available to the adopter. Smarter, streamlined processes, which still deliver the required outcomes are needed to ensure that patient safety is not compromised. Knowing the minimum requirements from the outset is paramount to conclude a process efficiently. A number of characteristics must be evaluated based on the type of test being assessed, all of which are resource driven. The extent of the verification will be fully dependent on whether the test is quantitative, and the verification design will change significantly as a consequence.

The following must be considered in most cases:
  • Limit of detection
  • Sensitivity
  • Specificity
  • Accuracy
  • Precision
  • Bias (when compared to an existing diagnostic test)
  • Negative and positive predictive value (NPV/PPV)
  • Clinical decision limits

It is essential that the acceptance criteria for any validation or verification, is appropriately set by clinical leads responsible for the safe delivery of the service utilizing the test. This may, in some cases, require clinical teams to discuss the requirements across disciplines i.e. medical laboratory and the patient facing clinical settings. Those requirements may differ and the thresholds of the test utility will need to be agreed so that the results produced are interpreted effectively.

For instance, where a test is being used in a decentralized location to screen or triage patients, does that initial test require the accuracy and precision of the laboratory test? Clinicians in patient facing roles will use that test as one indicator of the clinical decision they will make, in the context of symptoms, patient history and clinical experience. If a test is being used to trend a patient’s response to treatment over time, does it matter that the test used in that setting has a constant bias to the test used in the laboratory? The answer may well be no, depending on the test. A test with a high level of imprecision has more clinical risk in terms of correct interpretation and appropriate action taken to deliver optimized patient care. The verification design is essential in delivering the statistical outputs that give assurance that a test is safe for its intended clinical use.

Meeting international standards of proficiency

There are a number of international standards that offer guidance for the selection, verification and validation of IVDs. Those standards published by the International Standards Organization (ISO) and by the College of American Pathologists (CAP) aim to drive a medical laboratory and outreaching services to attain and maintain high quality patient supporting clinical services. Those individuals with knowledge of the standard and the clauses will know that the accrediting body is not prescriptive in how the standard should be met, nor will the clauses within the standard describe
how to achieve compliance. This gives the service provider seeking accreditation for a diagnostic service a degree of flexibility in the approach they can take to comply with the standard.

The applicant must ensure that the design of a particular verification can be justified, from a clinical risk perspective, and as long as this is agreed by individuals responsible for this element of the service then the requirement of the standard can be met with sufficient data.

Improving collaborative working

During the SARS-CoV-2 pandemic NHS England led the way on the selection and distribution of test kits and devices to NHS organizations, based on clinical demand in different territories. Some areas of England received solutions early in the process and others remained low on the priority list primarily based on regional infection rates. This led to a staggered adoption of the nationally selected solutions, which could have been avoided or minimized through improving collaborative working across NHS pathology networks.

The impact of this has seen a number of those solutions fail local verifications meaning they have not been adopted in the clinical setting. Ultimately this has slowed the diagnostic response to the pandemic, with further solutions being procured to fill the void at great financial cost to the health service. NHS Improvement have identified and encouraged the formation of 29 pathology networks across England. Each of these networks will comprise a varying number of individual organizations delivering patient care. Some networks are fully integrated and work closely together but many are not. This is where opportunities to share resources, knowledge and experience are evident and must be exploited to make widespread regional adoption of nationally verified solutions more streamlined and economical. Why replicate the work to varying levels of quality, if one site can represent the wider group?

There remains an excessive duplication of work across the healthcare landscape. Better communication between national bodies, service providers and the commercial diagnostic sector is required to bridge the resource gap and share the economic and intellectual burden.

Accepting a new approach to adoption

It is time to re-evaluate the approach to adopting innovative diagnostic solutions, especially those that have been rapidly produced and marketed to expand the response to global public health crises. This approach should take into account the changes in IVDR/MDR requirements and what this means for manufacturers and distributors of diagnostic tools that are now subject to increased scrutiny, more extensive pre-market evaluations and assessment of clinical risk. As previously seen with the IVD Directive, there was a gap in efficacy data that caused concern and trepidation amongst adopters. The IVD Regulation goes some way to allaying those concerns and IVD users should reassess how they approach the verification, adoption and implementation of new innovations.

Similarly the adopter can recalibrate their acceptance criteria and establish new thresholds at which they consider diagnostics fit for their intended use. This may include a reduction in the level of verification but an increase in post adoption monitoring of the clinical impact of the diagnostics solution. This in itself may shift the resource commitment from areas under pressure to areas where there is more flexibility.

Where organizations currently lack the expertise or resource to deliver verifications in a timely way, one solution may be to consider outsourcing the process to a third party or improving the awareness of what is required to evaluate and implement new solutions. This may form the basis of an in-house training programme delivered through collaboration across networks.

It is admirable that the adoption of diagnostics drives the pursuit of excellence and the safest of care for our patients. The challenge now is in striking a balance between the resource deficit and accepting new technologies into use. Continually striving for excellence can slow innovation, whereas aiming for good can rapidly transform healthcare with positive outcomes.

Tony Cambridge, Lead Biomedical Scientist is the Managing Director of Thornhill Healthcare Events and Consultancy, and Lead Biomedical Scientist in the Pathology Management team of a busy acute care hospital in England. He frequently speaks at national and international healthcare events and is a key opinion leader for point of care testing. He recently cowrote the British Society of Haematology’s point of care testing guideline for general hematology and remains active across healthcare platforms offering advice and guidance. He is also a member of a global diagnostics company’s scientific advisory committee.