Validation of the Clinical Frailty Scale in vascular surgery: a protocol

Introduction

As life expectancy increases, so does the prevalence of older patients and age-related health conditions. Frailty is one such condition which has become increasingly recognised as a clinical syndrome, distinct from chronological age, disability and comorbidity.^1–3 A complex, dynamic and multifactorial syndrome, frailty may be theoretically defined as ‘a state of increased vulnerability, resulting from age-associated declines in reserve and function across multiple physiological systems, such that the ability to cope with every day or acute stressors is compromised’.^3-5 In the UK, increasing frailty in the community is associated with higher rates of adverse events such as falls, hospitalisation and institutionalisation, and death.^6-9 Since frailty may be considered a measure of reduced physiological reserve,¹⁰ its severity has implications for patient response to medical and surgical therapy¹¹ and risk associated with therapeutic interventions.¹¹ Therefore, at the individual level, frailty can impact shared decision making and development of patient-centred care plans,^7,8 while the effect of frailty on health economics and resource distribution^8,12 may be felt by society.

Although the above definition is broadly accepted in theory, a single unified approach to assessing and quantifying frailty in a clinical setting remains elusive.^13,14 The following operational models are some of the most commonly used and studied: frailty as an accumulation of deficits, such as the frailty index² (FI); frailty as a phenotype of low energetics, for example, Fried’s frailty phenotype;³ frailty as derived from medical, nutritional, functional and psychological assessments, for example using Comprehensive Geriatric Assessment (CGA);¹⁵ frailty as a clinical judgement of function, such as using the Rockwood Clinical Frailty Scale (CFS).¹⁰

The plethora of available scales and scoring systems likely reflects uncertainty in the underlying components and pathophysiology that comprise frailty,¹⁰ leaving clinicians and policymakers alike without a clear directive of which assessment tool to use. Moreover, not all frailty measurements have been robustly validated, and many are used in a modified form rather than the original validated version,¹³ or used in populations other than those in which the tools were originally validated.¹⁶ For many specialties, the clinical value of frailty assessment depends on its validity in prognostication: is frailty an independent predictor of adverse outcomes?^16-18 If frailty simply correlates with related factors such as chronological age¹⁹ or comorbidity burden,²⁰ then time spent performing a designated frailty assessment could be better spent elsewhere.

Most patients who require major vascular surgery are chronologically and physiologically older: the majority are aged 65 or over^21-23 and have significant comorbidities such as diabetes, hypertension, cardiac disease, respiratory disease and a history of smoking.^21,22 Moreover, frailty in these patients is becoming increasingly recognised as a preoperative risk factor which can impact postoperative recovery and is thus collected in the UK National Vascular Registry (NVR) quality improvement audit.²² There is potential for the use of frailty scores in prognostication for patients undergoing major vascular surgery,^11,16 but the tools used need to be validated for use in a vascular cohort.²⁴

This study aims to explore whether the CFS, mapped onto the NVR four levels of frailty, is a valid method of quantifying frailty in patients who are undergoing major vascular surgery. The CFS is a rapid method of frailty assessment, making it ideal for use in acute and busy settings.¹¹ Clinicians consider comorbidity, cognitive impairment and disability to form a judgement of a patient’s frailty status, based on pictures and descriptions of each level of frailty.¹⁰ This scoring system has been validated for identifying frailty in adults aged 65 or older in the UK,^25-27 but has yet to be convincingly validated as a tool for identifying frailty itself within the cohort of inpatients with vascular disease. Previous studies have demonstrated that the CFS has a high specificity for identifying frailty in vascular outpatients and therefore could be useful in assessing frailty.^28,29 These findings could be extrapolated to imply that the CFS is a valid measure of frailty in vascular inpatients; however, the studies done so far outside the clinic setting appear to consider the CFS as a prognostic factor only,^30-32 rather than seeking to evaluate the CFS against another measure of frailty. This study seeks to evaluate the evidence to determine whether the CFS measures and assesses frailty in the population of vascular inpatients.

Since 2019 the NVR has recommended that commonly used formal frailty assessments, including the CFS, may allow patients to be categorised as:

1. Not frail: well or managing well, routinely walking

2. Mild frailty: evident slowing such as difficulty walking outside

3. Moderate frailty: need help with some personal care or keeping house

4. Severe frailty: completely dependent for personal care.³³

The CFS allows clinicians to stratify patients by frailty level: if the CFS provides a consistently accurate estimate of the extent of frailty in patients with vascular disease, we aim to then explore whether frailty has an independent prognostic value for adverse outcomes in patients undergoing major vascular surgery.

 

Objectives

To validate the Clinical Frailty Scale (CFS) and NVR four levels of frailty in patients undergoing major vascular surgery and explore the prognostic value of CFS in predicting adverse events.

 

Outcomes

1. Sensitivity and specificity of CFS (and subsequently the NVR four levels of frailty) in diagnosing frailty in patients undergoing major vascular surgery. The electronic frailty index (eFI) will be the reference standard. This is a scoring system that uses the cumulative model of frailty: the patient’s score is the fraction of deficits they are recorded as having from a list of 36 pre-specified diagnoses, deficits and disabilities. Conversely, the CFS is based on the phenotype model of frailty and describes differing degrees of physical performance^10,34 based on standardised pictures and descriptions.
2. Risk associated with different CFS degrees of frailty in developing adverse events after major vascular surgery.
3. Assess the differences in frailty between groups of patients undergoing different vascular surgical procedures.

 

Methods

The study will be reported with reference to the Standards for Reporting Diagnostic accuracy studies (STARD) 2015 reporting guidance.³⁵

Study design

This is a single-centre validation study of the CFS frailty assessment tool in patients undergoing major vascular surgery.

Patient population

Consecutive patients admitted under the vascular services at a tertiary care centre from 3 August 2022 to 31 December 2023 will be included. This period has been calculated to ensure sufficient patients may be included for each subgroup, based on the volume of each type of surgery performed at the reference centre.

Only patients who undergo a major vascular surgery, as reportable to the NVR,²¹ will be included. This includes any patients undergoing major lower limb amputation (MLLA), for example, for chronic limb threatening ischaemia (CLTI) or diabetes-related foot complications; patients with abdominal aortic aneurysms (AAA) who undergo repair (open/endovascular); those presenting with CLTI or acute limb ischaemia (ALI) undergoing lower limb revascularisation (open/endovascular including angioplasty/hybrid); and patients with carotid disease undergoing revascularisation (open/endovascular).

Index test

We will compare the performance of a phenotype model of frailty assessment with the cumulative model of frailty assessment. The CFS will be validated in patients undergoing major vascular surgery. This is the preferred method of assessing frailty in the Centre for Perioperative Care guidelines,³⁶ and has previously been validated in hospital inpatients aged >65 years^25,27,37 and in the outpatient setting for vascular patients;²⁸ however, there have been questions raised over the tool’s applicability in those with lower limb ischaemia.³⁸

Reference standard

The eFI will be the reference standard. This tool is applied to all people over 65 years old to identify those at risk of being moderately or severely frail in the community setting.^6,39 The eFI score is calculated as a fraction of 36 deficits determined from around 2000 GP read codes. This tool is used as a method of screening the community population, with the aim of identifying people who may benefit most from additional interventions^6,40,41 to enable them to live well with frailty. As this tool is applied to everyone aged >65 years in England, all included patients aged >65 years should have an eFI in their records, and we will use this tool as the reference standard. For those patients aged <65 years, the eFI will be manually calculated from the GP read codes.

Data collection

Baseline data collection will include patient demographics, biochemical tests (haemoglobin, estimated glomerular filtration rate), comorbidities, American Society of Anaesthesiologists (ASA) grade and indication for the operation. The CFS is assigned prospectively, ahead of any surgical intervention, and all other data will be collected retrospectively.

Operative data will include the type of operation undertaken and the type of anaesthetic used.

Postoperative data will include admission to the intensive care unit (ICU), length of ICU stay, length of inpatient stay, input from dieticians and therapists (occupational, physical and speech and language); return to theatre, Clavien–Dindo classification of inpatient complications;^42,43 discharge equipment and discharge destination.

Outcome data will be assessed at 30 days and 1 year, including re-admission, return to theatre, major adverse limb events (MALE, defined as amputation of the index limb, or major re-intervention such as new bypass graft; graft revision; angioplasty, thrombectomy)^44-46 and major adverse cardiovascular events (MACE) defined as myocardial infarction, stroke and death (any cause).^44-46

Reference standard

The eFI score will be collected for all consecutive patients admitted under the vascular services and will be extracted from GP records or calculated using the eFI guidance note (Table 1) when the GP score is not available. This is calculated as follows: eFI = number of deficits/36 (total number of deficits).

Index test

The CFS will be extracted from the hospital medical records. The CFS is recorded contemporaneously on admission to the vascular ward by the clerking doctor: either a foundation trainee, vascular registrar, consultant vascular surgeon or vascular physician. Consistency is achieved by training all clerking doctors in how to calculate the CFS, highlighting that the score should reflect a patient’s pre-morbid state or baseline function.⁴⁷

It is recommended that a comprehensive history is taken about the patient’s usual function at least two weeks prior to acute illness onset, such that the CFS recorded reflects their baseline function and not their status whilst acutely unwell. Assessment of frailty should consider direct patient history, observation of the patient plus collateral history from the patient’s relatives. However, if the patient is dying, they will always be classified as CFS 9, and not by their baseline function.⁴⁸

Reliability

To examine inter-rater reliability, CFS scores assigned by resident doctors during initial assessment will be compared to a blinded assessment made by a consultant vascular physician (geriatrician). To examine intra-rater reliability, clinicians who assess the CFS will be asked to repeat the frailty assessment at least 24 hours after initial assessment. All reliability data will be collected as a separate prospective sub-study, with new patients.

Analysis

Data analysis

Descriptive statistics reporting the mean, median, standard deviation and interquartile range will be reported for continuous data where appropriate. Categorical data will be reported as counts, frequencies and percentages. Data will be tested for normality. Data that are not normally distributed will be analysed using non-parametric tests. A p value of <0.05 will be interpreted as statistically significant.

Diagnostic accuracy

The frailty scores will be interpreted as in Table 2.

The following cut-off points will be explored:

• Not frail (NVR 1 and 2) versus Frail (NVR 3 and 4)

• Not frail (NVR 1) versus Frail (NVR 2, 3, 4)³³

Convergent validity will be used to assess validity of the CFS. For the CFS to be valid in patients undergoing major vascular surgery, it should agree with the eFI in more than 75% of cases and there should be a correlation of >0.5 between CFS and eFI. This will be tested using Spearman rank correlation. Analysis will be completed for all included patients and then explored across each of the four major patient cohorts – namely, those undergoing MLLA; repair of AAA; lower limb revascularisation; and carotid endarterectomy (CEA).

The diagnostic accuracy of CFS in identifying frailty compared with eFI will be explored by varying the test positivity cut-off point and constructing a receiver operator curve for each test positivity cut-off.

The sensitivity and specificity of CFS will be compared with eFI in a contingency table.

Reliability will be determined by test–retest correlation analysis from initial assessment score and second assessment score. This process will be repeated for inter-rater reliability with test–retest correlation analysis between blinded CFS scores allocated to the same patient.

Prognostication

If the CFS is shown to be an accurate way of diagnosing frailty in patients undergoing major vascular surgery, the role of CFS in predicting important patient outcomes will be explored. If CFS is not an accurate tool to diagnose frailty, eFI will be analysed for prognostication.

A multivariate logistic regression model will be used to explore whether each of eFI, CFS and NVR categories are independently correlated with adverse outcomes. Patient outcomes will include mortality, ICU admission, MALE, MACE, length of stay and discharge destination.

Subgroup analysis

For AAA and lower limb revascularisation, a subgroup analysis will be performed to examine the validity and prognostication of CFS/eFI. In patients undergoing AAA repair, the subgroups will be rupture versus not rupture; infrarenal versus juxta-renal/suprarenal repair; and open versus endovascular repair. In patients undergoing lower limb revascularisation the subgroups will be indication for revascularisation (ischaemia, aneurysm, trauma, etc) and type of revascularisation. Ischaemia will be further subcategorised into acute limb ischaemia, intermittent claudication and CLTI.

Sample size

The local prevalence of frailty is taken to be 43%, based on a recent trust frailty audit.⁴⁹ An expected sensitivity of 90% and specificity of 85% have been estimated from other patient groups in the literature,^50,51 since no studies have clearly established the sensitivity and specificity of CFS for identifying frailty in vascular patients. A 10% dropout rate (missing data) has been included in the sample size calculations as it is possible that not all patients will be assessed for frailty preoperatively. From the values shown in Table 3a,^52,53 97 patients will be required (Table 3b). For subgroup validity, this means 97 patients will be required per type of procedure (MLLA, AAA repair, lower limb revascularisation and CEA), giving a total sample size of 388 to determine the diagnostic validity of CFS compared with eFI across each vascular patient subgroup.

Ethics

The project was submitted as an audit to Hull University Teaching Hospital NHS Trust and has received local approval: 2022.111 Re-audit of the Prevalence of Frailty and the Impact on Surgical Management and Resource Use, for Vascular Inpatients Using the National Vascular Registry (NVR) Frailty Classifications. Based on the UKRI decision tool, this study does not need independent NHS Research Ethics Committee (REC) approval. Data collected will be handled according to Good Clinical Practice guidelines, General Data Protection Regulation 2018 and information governance policies. Only study team members will have access to the data.   All data will be anonymised. The study will not involve a change to routine patient care.

Dissemination of results

The results will be presented at local and national meetings and submitted to a peer review journal. A lay summary will be produced for patients and the public.

 

Discussion

Although there have been many studies of frailty in patients undergoing major vascular surgery,¹¹ there is no standardised approach to frailty assessment in these patients. The CFS is a rapid method of frailty screening which may have prognostication value but is yet to be convincingly validated in the population of vascular patients. This study will contribute to the growing field of frailty literature by validating the CFS and NVR four levels of frailty in vascular patients. It will also improve on existing data by validating these methods of frailty assessment in each subpopulation of the vascular cohort: patients undergoing MLLA, AAA repair, lower limb revascularisation and CEA. If the CFS is valid, this study may also offer insights into the prognostic value of frailty assessment, identifying whether baseline preoperative frailty is an independent risk factor for adverse outcomes in patients undergoing major vascular surgery.

The study is based in a large tertiary vascular centre in the North of England which has a catchment population of over 1.25 million people.⁵⁴ This will result in a diverse cohort of patients in terms of socioeconomic status, ethnic diversity and health status, so is likely to be generalisable to other settings in the UK. The method of validating frailty will consider different operations, indications and use an existing validating tool to assess frailty in the diverse vascular population. The study will also assess the reliability of CFS using a range of assessors with different experience of assessing frailty using CFS.

Potential caveats are: the results will only be applicable to patients who actually undergo surgery; external validity of the results may be limited as the data will be collected from a single centre; and potential biases from missing data due to the retrospective validation methods.

This protocol outlines a comprehensive validation study of CFS in patients undergoing vascular procedures reportable to the NVR. The results will provide key insights into the performance of some of the commonest frailty tools used in different populations of vascular patients. This information is vital when integrating frailty assessments into treatment decision making in future practice.