The complexity of the work
The process of identifying what is warranted vs unwarranted variation, right through to actually changing clinical practice and unlocking savings is an inherently complex and resource intensive process. The scale of the change management and clinical input required should not be underestimated.
Trusts tell us that although they are actively wishing to engage with the GIRFT programme, they are working in an increasingly pressured and fragile environment, with widespread operational and workforce challenges. They are therefore finding it difficult to consistently enable the frequency, quality, and depth of discussions and follow up work needed, given the time and complexity involved. Furthermore, participating in GIRFT requires a significant investment of time from clinicians, senior medical leaders and other board-level staff, such as financial and operational directors. Trusts tell us that they sometimes struggle to provide sufficient time for senior management to support the programme’s implementation given the competing pressures currently faced.
The GIRFT programme must also take into account the reality that not all unwarranted variation can be tackled due to locally specific circumstances. Variation exists across the provider sector in terms of centrally controlled funding and payment activity, which has a knock-on impact on the quality and productivity of a trust’s services:
"The hospital in which I work has a lower-level funding per unit of activity than nearly all other NHS hospitals… which largely reflects factors without any real-world justification, and which creates a postcode lottery which is centrally determined."
Medical director, district general hospital trust
Changing clinical practice
Trusts widely commented that embedding clinical practice will take time, and there are a number of very legitimate reasons why it might be an iterative process.
For example, implementing changes to clinical practice will require substantial re-training in many cases, and in the interim might lead to short term error-rates increase as clinicians embed new techniques. There might also be a legitimate argument, in the current climate, to focus on stabilising performance before attempting to change clinical practice which, in the short time, might undermine performance further.
Unlocking the savings identified through headline benchmarking metrics is an inherently complex process. There will be a whole host of factors which mean that trusts might take a substantial amount of time to unlock these, perhaps because clinical reconfiguration might be required, or that trusts might not be able to realise the full scale of savings earmarked.
The extrapolation of cash savings from top level benchmark data should be treated with caution, as trust feedback on GIRFT recommendations suggests that in many cases upfront investment will be required to deliver changes where the GIRFT data indicates they are warranted – for example to staff rostering, diagnostics and facilities.
"We had already delivered the main savings they [the GIRFT team] identified on length of stay. They also made some very expensive recommendations for capital upgrades, which we’ve not been able to progress."
Chief operating officer, district general hospital trust
Validity of the GIRFT programme data
Trusts explained that there can be significant issues with the quality of data being produced by the central team to inform conversations with trusts. For example, by using old and historical data as the starting point for conversations with local clinicians, it is difficult for the trust to challenge the GIRFT team’s view that significant service improvements have occurred in the time since. This can mean that, unless the GIRFT team acknowledges this shortcoming explicitly in their engagement, initial conversations with clinicians may be less productive.
"In many cases the data was two years old and therefore resulted in conversation regarding historical practice. In many cases the problems had been resolved. The historical nature would also allow obfuscation, as claims to a much improved current position could not be challenged."
Medical director, large regional hospital trust
Data quality challenges can also inhibit the triangulation of financial and clinical datasets needed to interrogate sources of variation locally.
"Perhaps the biggest variation was within the data, making clinical and financial comparisons impossible. This was driven by a variety of counting and contractual variations with different units counting clinical activity in different ways. The conversation was spent on explaining the apparent statistical variation rather than on clinical variation."
Medical director, large regional hospital trust
Where substantive concerns over the data existed, clinical engagement was more challenging to secure and the outputs from the GIRFT programme were less useful in informing trusts’ operational plans. Clearly, the appropriate selection, use and maintenance of data in the GIRFT programme are important for establishing the programme’s credibility with clinicians at the outset.
"We have had five or six visits. They have mostly been either helpful or neutral. The ones which were neutral were because the lead clinicians or the team were not sufficiently aware of the detail underpinning the data, and so couldn’t answer questions about what it incorporated. Having said that, it has been a good start in most areas and we have found it helpful and welcome the process continuing to mature and more reliable data coming from it."
Chief Operating Officer, teaching hospital
Some trusts had concerns over how valid the GIRFT programme’s analysis could be in those clinical specialties which have limited robust datasets to work with. In those specialties a primary data capture may be required in the future; this would place a potentially significant burden on involved trusts.
"The interpretation of the data is influenced by the views of the attending expert. On occasion this has been contrary to the views of the majority of the clinical team. The programme should concentrate on procedures and interventions that have a clear evidence base and a national consensus view."
Medical director, teaching hospital
However, it is important to recognise that too much data could obscure clarity of insight into performance and inhibit change on the ground. Some trusts felt that there were almost too many GIRFT datasets, and were concerned about retaining focus on the most impactful metrics alone. Given the developmental stage of the GIRFT datasets, it is also essential for the programme’s ongoing credibility with trusts that GIRFT data is used in a non-judgemental way.
The data packs generated by GIRFT must be seen as the starting point in a conversation with trusts rather than the authoritative position, until such time that confidence across the trusts is established in the datasets and the timeliness of the data included. It is also important that GIRFT provides value for trusts in terms of the diagnostic tools to help trusts deliver clinical change off the back of the analysis, so they are ‘closing the loop’ – without this assistance, for many organisations GIRFT will continue to feel like ‘feeding the beast’ with insufficient return on the effort it takes to do it.