Skip to content
ALL Metrics
-
Views
84
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

Early detection of colorectal cancer using symptoms and the ColonFlag: case-control and cohort studies

[version 1; peer review: 2 approved with reservations]
PUBLISHED 24 Jan 2023
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

Abstract

Background

Early detection of colorectal cancer confers substantial prognostic benefit. Most symptoms are non-specific and easily missed. The ColonFlag algorithm identifies risk of undiagnosed colorectal cancer using age, sex and changes in full blood count (FBC) indices. The aim of this study was to investigate whether the ColonFlag detects undiagnosed colorectal cancer prior to the recording of symptoms in general practice.

Methods

We conducted case-control and cohort studies by linking primary care data from the Clinical Practice Research Datalink with colorectal cancer diagnoses from the National Cancer Registry. A ColonFlag score was derived for each FBC. We assessed the prevalence of symptoms at six-monthly intervals prior to index date (diagnosis date for cases, randomly selected date for controls). We then derived odds ratios (ORs) and area under the receiver operating characteristic (AUROC) curve for the ColonFlag, and for symptoms using logistic regression at each interval (primary outcome 18–24 months).

Results

We included 1,893,641 patients, 10,875,556 FBCs and 8,918,037 ColonFlag scores. ColonFlag scores began to increase in cases compared with controls around 3–4 years before diagnosis. The AUROC for a diagnosis 18–24 months following the ColonFlag score was 0.736 (95% CI 0.715-0.759), falling to 0.536 (95% CI 0.523-0.548) with adjustment for age. ORs for individual symptoms became non-significant prior to 12 months before index date, except for abdominal pain (females OR=1.29, p<0.0001 at 12–18 months) and rectal bleeding (females OR=2.09, males OR=1.92, p<0.0001 at 18–24 months).

Conclusions

Symptoms appear relatively late in the colorectal cancer process and are limited for supporting early stage detection. The ColonFlag can discriminate usefully at 18–24 months before diagnosis, suggesting a role for this algorithm in primary care, although some of its discriminatory ability comes from the age variable.

Plain Language Summary

Plain English summary

Colorectal cancer remains one of the commonest causes of cancer death in the UK. Early detection is very important. The sooner treatment is started, the better the chances of survival. An Israeli company developed a tool (the ‘ColonFlag’) for detecting subtle changes in the full blood count, a common blood test in general practice, using up to 20 different markers from it, as well as the person’s age and sex. As the diagnosis approaches, changes become more abnormal and the ColonFlag can spot an affected person more easily. In this project, we investigated whether the ColonFlag could identify people with undiagnosed colorectal cancer in the early stage prior to the onset of symptoms.

First of all, we needed to find out exactly what the timescale is for this ‘pre-symptomatic’ phase. We looked at the (anonymous) health records of 1.9 million people to identify when the reporting of possible colorectal cancer symptoms becomes more frequent in those later diagnosed compared to those who were not. We found that prior to 18 months before diagnosis, there was very little difference, but within 18 months, symptoms become increasingly recorded in colorectal cancer cases.

We then assessed whether the ColonFlag could usefully identify colorectal cancer risk at this early stage before symptoms. We found that the ColonFlag scores of people destined to be diagnosed started to increase around 3–4 years before the diagnosis. However, the differences are initially small. In the phase 18–24 months before diagnosis, the ColonFlag could effectively discriminate patients with cancer from patients without, although at this interval a lot of its ability to do so was relying on the person's age. Our results are encouraging, that detecting changes in full blood count markers could add usefully to current approaches to promote earlier diagnosis and treatment of this common cancer.

Keywords

Colorectal cancer, primary care, early detection, symptoms, ColonFlag, blood test trends

Abbreviations

AUROC      Area Under the Receiver Operating Characteristic

CI               Confidence interval

CPRD         Clinical Practice Research Datalink

CRC           Colorectal Cancer

FBC           Full Blood Count

FIT            Faecal Immunochemical Test

NCR          National Cancer Registry

OR            Odds Ratio

Introduction

Colorectal cancer (CRC) (bowel cancer) is the 4th commonest cancer in the UK1. Five-year survival is around 93% if diagnosed at stage 1, where it is confined to the bowel, but only 10% at stage 4, where it has metastasised2. This type of cancer develops slowly from pre-cancerous polyps, which may grow for years before becoming malignant, making it amenable to early detection. Endoscopic excision of these lesions may either remove early cancers or prevent cancer developing in the first place3.

Until recently, the UK bowel cancer screening programme in England offered the faecal occult blood test to adults aged 60–74 years every two years to screen for CRC in England, and this has shown to be successful in reducing mortality from CRC4. However, its impact was limited, with a 59% response rate5 and a negative result in around half of cases (sensitivity 49%)6. This test has now been replaced with Faecal Immunochemical Testing (FIT), using a threshold of ≥120µg Hb/g stool7. In addition to its role in screening, the high sensitivity and negative predictive value of FIT at the lower threshold of ≥10µg Hb/g stool has led to national recommendations for FIT to be used as a triage test in primary care for people with symptoms8. A high proportion of symptomatic patients present at a stage beyond surgical cure. Some patients present with specific CRC symptoms, such as rectal bleeding or change in bowel habit, but symptoms are often non-specific with insidious onset, such as abdominal pain and weight loss. Consequently, symptomatic patients report delays in referral for investigation despite presenting to their GP9. Risk algorithms have been developed based on symptoms to improve patient selection for investigation by identifying high-risk groups10. These algorithms may assist in case finding, but rely on symptoms that appear relatively late in the disease process, and have limited uptake in primary care11.

In 2016, a risk algorithm for CRC was developed using machine learning techniques (Gradient Boosting Model and random forests) applied to full blood count (FBC) indices by an Israeli company, Medial EarlySign12. For each FBC in a patient’s record, up to 20 indices are available and a 'ColonFlag' score ranging from 0 to 100 can be derived, with higher scores indicating a higher risk of undiagnosed CRC. Previous FBC results may contribute to the current score, by indicating trends in the indices, but a score can be produced even if only one FBC is available. The ColonFlag combines these indices with age and sex, but not with symptoms, to stratify CRC risk but does not estimate an absolute risk of CRC within a fixed time. In 2017, we published a report of an independent validation study of the ColonFlag using UK primary care data obtained from the Clinical Practice Research Datalink (CPRD)13. Similar performance was found in the UK as in Israel. The ColonFlag could discriminate high-risk from lower-risk patients 18–24 months prior to CRC diagnosis. However, an important assumption remained to be investigated, that the ColonFlag detects CRC cases prior to the reporting of symptoms in primary care. If confirmed, the ColonFlag could add significantly to current approaches to early cancer detection and may have a role in support of the national bowel cancer screening programme.

The aim of this study was to investigate whether the ColonFlag detects undiagnosed CRC prior to the recording of CRC symptoms in general practice.

Methods

Setting

We conducted our analyses using the CPRD GOLD dataset used for our previous initial external validation project, including patients with at least one FBC between 01/01/2000 and 28/04/201513. The CPRD is linked to over 600 UK general practices and contains information on patients' demographic data, registration status, medical history (including cancer diagnoses), and laboratory test values (including FBC indices)14.

Study design

Case-control and cohort analyses were conducted to compare when the ColonFlag algorithm identified increased CRC risk with the timing of symptom recording in the primary care record. Patient data was confined between study entry (latest date of: registration with a contributing practice, the patient’s 40th birthday, or 01/01/2000) and study exit (earliest date of: death, leaving the practice, or the most recent update of the CPRD dataset, which may differ between contributing practices).

Study population

We included people over the age of 40 years with at least one FBC in their primary care CPRD record between 01/01/2000 and 28/04/2015 and an associated ColonFlag score. The ColonFlag score was derived by Medial EarlySign, who applied the ColonFlag algorithm to each FBC considered valid according to their criteria during the previous project. We excluded people with less than two years of follow up after study entry, less than 12 months registered at the practice, or a haemoglobin gene defect, as this may produce abnormalities in red cell indices that mimic iron deficiency, such as reduction in haemoglobin and mean cell volume.

Outcome

We identified cases of CRC up until 14/01/2014 using codelists developed for the first project13, for practices linked to the National Cancer Registry (NCR) (clinical codes have previously been reported15). An initial inspection of the capture of CRC diagnoses in CPRD indicated that linkage to the NCR would be important for high quality outcome assessment (analysis available on request). Our sample was therefore confined to those practices linked to the NCR.

Predictors

Age and sex were available for all individuals. FBC indices were extracted using CPRD Medcodes (clinical codes have previously been reported15). We extracted symptom codes after searching the literature to identify symptoms associated with CRC1619. A final list was produced through discussion and consensus between two clinicians (TAH and BDN) and used to derive a longitudinal dataset for each patient reporting at least one symptom. The eight symptom groups were: abdominal pain, appetite loss, diarrhoea, weight loss, constipation, rectal bleeding, change in bowel habit and 'other'. More than one symptom could be recorded on a single day.

Case-control analysis

Symptoms: the index date for cases was the date of CRC diagnosis. The index date for controls was a randomly selected date between study entry and exit. There were 15 controls matched to each case based on age at index date. For the primary analysis, we ascertained whether a patient had any symptom between 18 and 24 months before the index date. Logistic regression was used to derive odds ratios (ORs) and the AUROC including a binary variable for each symptom group as the independent variables in a multivariable model with age at index date. Females and males were analysed separately throughout.

ColonFlag: a ColonFlag score was identified 18 to 24 months before the index date. If there was no score available, the patient was excluded. If a patient had more than one score in this interval, one score was randomly selected. There were 100 controls matched to each case based on age at ColonFlag score date, sex, and year of score. Logistic regression was used to derive ORs and the AUROC for the independent variable, the ColonFlag score.

We then compared the discrimination performance measures of the ColonFlag score with that of symptoms. The primary outcome was the difference in the AUROC scores in the two analyses for the 18–24 month interval. The analysis was repeated for additional time intervals, 0–6, 6–12, 12–18, and 24–30 months before the index date.

Cohort analysis

Next, to investigate the result of applying the ColonFlag to a defined cohort of patients, all patients with a ColonFlag score in 2011 not previously diagnosed with CRC were followed for up to 24 months from the ColonFlag date (baseline). The year 2011 was selected as this was the most recent year for which 24 months of follow up were available in the NCR. If a patient had more than one ColonFlag score in 2011, then the earliest was selected. All diagnoses confined between 18–24 months following baseline were included, with patients diagnosed before 18 months excluded. Patients without a diagnosis within 24 months were censored at 24 months following baseline, with those with less than 24 months of follow up excluded. Logistic regression was used with CRC diagnosis within 18–24 months of the ColonFlag score as the outcome and the ColonFlag score, age at baseline, and sex, as predictors to derive ORs and the AUROC. In addition, diagnostic performance characteristics for the ColonFlag score were calculated, corresponding to 99.5% specificity and 50% sensitivity.

Subsequently, for the same set of patients, we identified all symptoms reported between 3 months before and 3 months after the date of the ColonFlag score (baseline). We used logistic regression with a diagnosis of CRC within 18–24 months as the outcome and presence of any of the types of symptom, age at baseline, and sex as predictors.

The analyses were repeated for a diagnosis of CRC within 24 months following the ColonFlag date, which, unlike the 18–24 month outcome, included all patients diagnosed within the 24 month follow up period.

Trajectories of ColonFlag scores

Following our case-control analysis design for the ColonFlag score analysis, we modelled the trajectories of the ColonFlag score (which increases spontaneously over time simply due to increasing age) from index date backwards using linear mixed models and compared cases and controls. All ColonFlag scores were included that preceded the patient’s index date. We investigated whether there was an accelerated increase (a 'change-point') in ColonFlag scores for cases compared with controls at an identifiable time interval before diagnosis. This method has been used to identify people at risk of ovarian cancer based on serial CA125 levels20, and for that condition improves detection compared with a single value cut-off21.

Ability of ColonFlag to detect increased CRC risk when the FBC is ‘normal’

We then investigated whether the ColonFlag might identify undiagnosed CRC when all FBC indices are within their normal reference ranges by detecting adverse trends in previous indices even if the current FBC is ‘normal’. This is important because ‘normal’ FBCs are not examined in any detail by busy clinicians filing results and in some situations might be ‘batch filed’ without even being opened. Batch filing reduces to zero the already small chance that a human clinician will notice ‘within range’ changes of concern, so the help of an algorithm to do so would be very useful. We examined the proportion of FBCs in our dataset where all the indices were within range; of these, the proportion with a high ColonFlag score above our two thresholds (corresponding to 99.5% specificity and 50% sensitivity); and of these, the proportion diagnosed with CRC within 3 years. Three years was chosen to allow enough time to determine whether CRC was present at the time of the normal FBC. Longer follow up intervals (e.g. 4 years) might risk identifying people with a later cancer diagnosis that was not initially present, but developed subsequently to the normal FBC.

All analyses were performed in Stata SE version 17 (RRID: SCR_012763). Alternative, open-access software, such as R (RRID: SCR_001905), can perform the equivalent analyses.

Results

The initial dataset contained 16,537,017 FBC observations belonging to 2,856,020 patients. Each patient had between 1 and 383 FBCs in their record (median 4). Of these, 13,381,427 (80.6%) had an associated ColonFlag score, as one fifth did not pass the quality/eligibility checks conducted by Medial EarlySign.

We were confident that our study population had good quality recording of the main outcome, a CRC diagnosis, based upon a high proportion (95%) of the diagnoses occurring within 6 months of each other in the two data sources (Figure 1). There were 386 linked practices, providing patient numbers ranging between 431 and 21,102 per practice. Of the 1,893,641 patients, 24,557 (1.3%) had a diagnosis of CRC. We compared the age and sex distribution of our sample (aged at least 40 years) with that of England in two years (2007 and 2012). There was a satisfactory match, apart from in the younger age bands, 40-60, where there was a shortfall of males, presumably due to less frequent FBC testing in this age group (Figure 2). We then compared the incidence of CRC with that of England, and found a close match, based on cancer registrations in the NCR, which were part of our dataset (Figure 3).

ba35bdc4-019d-4361-b46a-0e31ab301f59_figure1.gif

Figure 1. Date of colorectal cancer diagnosis in the CPRD compared to the NCR.

Legend: Clinical Practice Research Datalink (CPRD). National Cancer Registry (NCR).

ba35bdc4-019d-4361-b46a-0e31ab301f59_figure2.gif

Figure 2. The age and sex distribution of our CPRD sample.

Legend: 2007 (a) and 2012 (b).

ba35bdc4-019d-4361-b46a-0e31ab301f59_figure3.gif

Figure 3. Incidence of colorectal cancer in our sample and in England.

Legend: 2007 (a) and 2012 (b).

Restriction to practices linked to the NCR resulted in 386 practices, as described above. This resulted in 1,893,641 patients, 10,875,556 FBCs, and 8,918,037 associated ColonFlag scores.

Case-control analysis

Symptoms: the proportion of cases and controls reporting symptoms is displayed for each 6-monthly interval before index date for any symptom (Figure 4) and each individual symptom group (Figure 5), with proportions reported in Table 1 for both. The prevalence of symptoms fell from an initially high level from 0–6 months in cases to a level comparable with controls by 12–18 months. Prior to this (at 18-24 months), the only symptom significantly more common in cases was rectal bleeding (Table 2). These results confirmed our chosen interval of 18–24 months for the pre-symptomatic phase, which we had also used as the primary outcome interval in our original validation study13.

Table 1. Number and proportion of patients reporting symptoms per 6-month time interval before index date.

Legend: Case-control analysis matched 1:15 on age at index date. Time is increasing backwards from index date (-6 is 0–6 months before index, -12 is 6–12 months before index, etc.).

Females
Time interval
(months)
0–6 6–12 12–18 18–24 24–30
Cases
n=8344
Controls
n=125160
Cases
n=8344
Controls
n=125160
Cases
n=8344
Controls
n=125160
Cases
n=8344
Controls
n=125160
Cases
n=8344
Controls
n=125160
Symptom
Constipation613 (7.4%)2279 (1.8%)224 (2.7%)2115 (1.7%)17 (2.0%)1981 (1.6%)115 (1.4%)1979 (1.6%)104 (1.2%)1804 (1.4%)
Abdominal pain1514 (18.1%)3447 (2.8%)381 (4.6%)3462 (2.8%)286 (3.4%)3329 (2.7%)225 (2.7%)3316 (2.6%)210 (2.5%)3286 (2.6%)
Loss of appetite59 (0.7%)157 (0.1%)19 (0.2%)148 (0.1%)10 (0.1%)166 (0.1%)9 (0.1%)134 (0.1%)9 (0.1%)141 (0.1%)
Diarrhoea656 (7.9%)2265 (1.8%)184 (2.2%)2165 (1.7%)150 (1.8%)2095 (1.7%)141 (1.7%)2098 (1.7%)144 (1.7%)2026 (1.6%)
Weight loss181 (2.2%)553 (0.4%)51 (0.6%)466 (0.4%)27 (0.3%)440 (0.4%)31 (0.4%)406 (0.3%)26 (0.3%)388 (0.3%)
Rectal bleeding1222 (14.6%)690 (0.6%)176 (2.1%)628 (0.5%)109 (1.3%)636 (0.5%)84 (1.0%)615 (0.5%)55 (0.7%)644 (0.5%)
Change of bowel habit489 (5.9%)359 (0.3%)63 (0.8%)368 (0.3%)26 (0.3%)351 (0.3%)28 (0.3%)350 (0.3%)16 (0.2%)342 (0.3%)
Other symptom114 (1.4%)155 (0.1%)12 (0.1%)126 (0.1%)8 (0.1%)156 (0.1%)6 (0.07%)131 (0.1%)10 (0.1%)127 (0.1%)
Any symptom3886 (46.6%)8939 (7.1%)970 (11.6%)8574 (6.8%)718 (8.6%)8329 (6.6%)585 (7.0%)8185 (6.5%)530 (6.4%)7959 (6.4%)
Males
Time interval
(months)
0–66–1212–1818–2424–30
Cases
n=9786
Controls
n=145590
Cases
n=9786
Controls
n=145590
Cases
n=9786
Controls
n=145590
Cases
n=9786
Controls
n=145590
Cases
n=9786
Controls
n=145590
Symptom
Constipation658 (6.7%)2636 1.8%)203 (2.1%)2336 1.6%)131 (1.3%)2122 (1.5%)122 (1.2%)2021 (1.4%)138 (1.4%)1903 (1.3%)
Abdominal pain1419 (14.5%)3044 (2.1%)331 (3.4%)2856 (2.0%)246 (2.5%)2844 (2.0%)185 (1.9%)2794 (1.9%)194 (2.0%)2871 (2.0%)
Loss of appetite56 (0.6%)151 (0.1%)14 (0.1%)123 (0.1%)12 (0.1%)130 (0.1%)8 (0.1%)96 (0.1%)4 (0.05%)96 (0.1%)
Diarrhoea842 (8.6%)1975 (1.4%)202 (2.1%)1927 (1.3%)134 (1.4%)1732 (1.2%)111 (1.1%)1802 (1.2%)103 (1.1%)1698 (1.2%)
Weight loss255 (2.6%)526 (0.4%)45 (0.5%)487 (0.3%)28 (0.3%)481 (0.3%)34 (0.4%)419 (0.3%)27 (0.3%)415 (0.3%)
Rectal bleeding1522 (15.6%)819 (0.6%)186 (1.9%)759 (0.5%)105 (1.1%)767 (0.5%)101 (1.0%)776 (0.5%)92 (0.9%)798 (0.6%)
Change of bowel habit941 (906%)429 (0.3%)66 (0.7%)342 (0.2%)39 (0.4%)364 (0.2%)19 (0.2%)351 (0.2%)19 (0.2%)330 (0.2%)
Other symptom84 (0.9%)109 (0.1%)7 (0.1%)80 (0.1%)4 (0.04%)100 (0.1%)5 (0.05%)102 (0.07%)6 (0.06%)78 (0.05%)
Any symptom4737 (48.4%)8802 (6.1%)927 (9.5%)8143 (5.6%)641 (6.6%)7819 (5.4%)522 (5.3%)7682 (5.3%)531 (5.4%)7524 (5.2%)

Table 2. Multivariable logistic regression including types of symptoms per 6-month time interval.

Legend: Case-control analysis matched 1:15 on age at index date. Females (n=133,504) and males (n=155,376). Each symptom is modelled as a binary variable, =1 if the patient experienced that symptom in that time interval.

Females – outcome diagnosis of colorectal cancer
Time before index date0–6 months6–12 months12–18 months18–24 months24–30 months
SymptomORp-valueORp-valueORp-valueORp-valueORp-value
Constipation2.62 <0.00011.43<0.00011.220.020.840.090.850.14
Abdominal pain6.77<0.00011.59<0.00011.29<0.00011.010.920.960.63
Loss of appetite2.20<0.00011.920.0080.990.981.010.971.110.76
Diarrhoea3.12<0.00011.110.211.000.980.980.841.070.45
Weight loss4.01<0.00011.430.020.890.591.030.871.010.97
Rectal bleeding28.47<0.00014.04<0.00012.50<0.00012.09<0.00011.290.09
Change of bowel habit18.72<0.00012.25<0.00010.990.981.100.660.790.37
Other symptom8.24<0.00011.140.690.710.380.770.531.210.58
Age at index date (years)1.000.111.000.0081.000.021.000.031.000.03
Males – outcome diagnosis of colorectal cancer
Time before index date0–6 months6–12 months12–18 months18–24 months24–30 months
SymptomORp-valueORp-valueORp-valueORp-valueORp-value
Constipation2.44<0.00011.170.0520.840.090.870.190.990.93
Abdominal pain7.31<0.00011.69<0.00011.29<0.00010.950.560.980.81
Loss of appetite1.760.011.310.400.900.791.270.540.750.57
Diarrhoea4.75<0.00011.47<0.00011.100.340.850.140.850.15
Weight loss5.31<0.00011.180.350.830.381.220.300.940.76
Rectal bleeding30.12<0.00013.18<0.00012.04<0.00011.92<0.00011.64<0.0001
Change of bowel habit31.06<0.00012.46<0.00011.740.0010.740.260.910.72
Other symptom6.41<0.00011.100.820.510.250.760.760.870.79
Age at index date (years)0.99<0.00010.99<0.00010.99<0.00010.99<0.00010.99<0.0001

ColonFlag score: We then assessed the predictive ability of the ColonFlag score. Descriptive statistics for cases and controls in the primary (18–24 month) analysis and a summary of the ColonFlag scores are given in Table 3. We derived AUROC curve scores for the ColonFlag score at each time interval, showing that predictive performance decreases the earlier the ColonFlag score is from diagnosis (Table 4). Each interval had its own sub-population (descriptive statistics available on request). The AUROC for ColonFlag without including age in the calculation, due to matching cases to controls using age, was 0.536 (95% CI 0.523, 0.548), which although small was significantly different from chance. The ColonFlag AUROC scores reported in this analysis are substantially lower than those reported in the cohort study below, due to the effective removal of the age component of the ColonFlag through matching for age.

Table 3. Descriptive statistics for age and sex by colorectal cancer (CRC) diagnosis.

Legend: Case-control analysis matched 1:100 on age at ColonFlag score, sex, and year of score. ColonFlag scores are between 18–24 months before diagnosis for cases and with at least 18 months of follow up after the ColonFlag score for controls. Age is at time of ColonFlag score. Cases are according to the National Cancer Registry.

No diagnosis of CRCDiagnosis of CRC
NumberAge (SD)
Range (years)
ColonFlag score
(SD) Range
NumberAge (SD)
Range (years)
ColonFlag score
(SD) Range
Female22230074.5 (11.3)
40–98
74.4 (20.3)
0–100
222374.5 (11.3)
40–98
76.2 (21.0)
0–100
Male23560072.9 (10.0)
40–99
81.9 (17.0)
0–100
235672.8 (10.0)
40–99
83.3 (16.9)
0–100
Total45790073.6 (10.7)
40–99
78.3 (19.1)
0–100
457973.6 (10.7)
40–99
80.0 (19.2)
0–100

Table 4. AUROC scores for a diagnosis of CRC at different time intervals before index date.

Legend: Case-control analysis matched 1:100 on age at ColonFlag score, sex, and year of score. The ColonFlag score as the only predictor.

AUROC score (95% CI)
MalesFemales
Time interval
(months)
0–66–1212–1818–240–66–1212–1818–24
ColonFlag 0.624
(0.618,
0.6360
0.605
(0.593,
0.616)
0.557
(0.545,
0.568)
0.536
(0.523,
0.548)
0.623
(0.616,
0.630)
0.624
(0.612,
0.636)
0.567
(0.554,
0.579)
0.536
(0.523,
0.549)
ba35bdc4-019d-4361-b46a-0e31ab301f59_figure4.gif

Figure 4. Percentage of cases and controls reporting any symptom in 6-month time bands.

Legend: Case-control analysis matched 1:15 on age at index date. Time is increasing backwards from index date (-6 is 0–6 months before index, -12 is 6–12 months before index, etc.).

ba35bdc4-019d-4361-b46a-0e31ab301f59_figure5.gif

Figure 5. Prevalence of individual symptoms in 6-month time bands.

Legend: Case-control analysis matched 1:15 on age at index date. Time is increasing backwards from index date (-6 is 0–6 months before index, -12 is 6–12 months before index, etc.). Males (a) and females (b).

Cohort analysis

In this analysis, patients diagnosed within 18 months following baseline were omitted (N=1,722). Additionally, patients without a diagnosis who were lost to follow up (N=90,599) or died (24,443) within two years baseline were omitted.

There were 434 patients diagnosed with CRC within 18–24 months following baseline (Table 5). Figure 6 shows the distribution of ColonFlag scores in people at baseline in 2011, comparing those who would and would not be diagnosed with CRC within 18–24 months, showing an excess of high ColonFlag scores in the future cases before adjustment for age. Logistic regression for CRC diagnosed within 18–24 months including the ColonFlag score, age, and sex gave an AUROC of 0.736 (95% CI 0.715, 0.759) (Table 6). Adding any symptom instead of the ColonFlag score gave an AUROC of 0.725 (0.704, 0.747) (Table 7). When breaking down to each symptom group individually in a multivariable model, the only symptom with a significant OR at 18–24 months was rectal bleeding, as expected from the case-control analysis (Table 8). It is clear that much of the discriminatory ability of either approach is due to the age and sex variables, which themselves give an AUROC of 0.725 (0.703, 0.747) (Table 9). This analysis produces the primary outcome for this project, a comparison between the AUROC for the ColonFlag (0.736) and the AUROC for any symptom plus age (0.725) for a diagnosis of CRC 18–24 months into the future (Table 10). It was not possible to remove the age and sex component of the ColonFlag score here. This was conducted in the case-control analysis reported earlier, producing an AUROC at 18–24 months of 0.536, which, although small, is significantly different from chance. ORs for the ColonFlag score are provided in the subgroup of patients with symptoms (Table 11) and without symptoms (Table 12).

Table 5. Characteristics of patients with a ColonFlag score in 2011 with and without a diagnosis in 18–24 months.

Legend: Cohort analysis. Those diagnosed have a diagnosis of colorectal cancer between 18–24 months after the score date in 2011 (baseline) and those with no diagnosis have at least 24 months of follow up after the score date (baseline) with no diagnosis.

NumberMean age (range)Mean ColonFlag score (range)% Females
With diagnosis43473.1 (41–98)83.8 (6–100)46.5
No diagnosis40010062.8 (40–106)56.3 (0–100)57.1
ba35bdc4-019d-4361-b46a-0e31ab301f59_figure6.gif

Figure 6. Distribution of ColonFlag scores in 2011 for colorectal cancer in 18–24 months.

Legend: Cohort analysis. Patient who would not (0) and those who would (1) be diagnosed with colorectal cancer.

Table 6. Logistic regression of ColonFlag score for diagnosis within 18–24 months later.

Legend: Cohort analysis. Age at ColonFlag score. Those diagnosed have a diagnosis of colorectal cancer between 18–24 months after the score date in 2011 (baseline) and those with no diagnosis have at least 24 months of follow up after the score date (baseline) with no diagnosis.

Diagnosis at 18–24 monthsOR95% CIp-value
ColonFlag/unit increase1.0251.017, 1.033<0.0001
Age/year increase1.0251.011, 1.039<0.0001
Female0.7920.646, 0.9710.025
AUROC = 0.736 (95% CI 0.715, 0.759) N=400,534

Table 7. Logistic regression for any symptom for diagnosis within 18–24 months later.

Legend: Cohort analysis. Any symptom is a binary variable, =1 if the patient reported any symptom at time of ColonFlag score (baseline) ± 3 months. Age at ColonFlag score. Those diagnosed have a diagnosis of colorectal cancer between 18–24 months after the score date in 2011 (baseline) and those with no diagnosis have at least 24 months of follow up after the score date (baseline) with no diagnosis.

Diagnosis at 18–24 mOR95% CIp-value
Any symptom1.1150.834, 1.4910.46
Female0.6220.514, 0.751<0.0001
Age/year increase1.0621.054, 1.070<0.0001
AUROC =0.725 (95% CI 0.704, 0.747) (n=400,534)

Table 8. Multivariable logistic regression including individual symptoms for diagnosis within 18–24 months of the ColonFlag score.

Legend: Cohort analysis. Each symptom is a binary variable, =1 if the patient reports that symptom at ColonFlag score (baseline) ± 3 months. Age at ColonFlag score. Those diagnosed have a diagnosis of colorectal cancer between 18–24 months after the score date in 2011 (baseline) and those with no diagnosis have at least 24 months of follow up after the score date (baseline) with no diagnosis.

Diagnosis at 18–24 mOR95% CIp-value
Constipation0.2650.085, 0.8270.022
Abdominal pain0.7850.468, 1.3170.36
Appetite loss1.2230.170, 8.7710.84
Diarrhoea1.2590.722, 2.1960.42
Weight loss1.1510.429, 3.0920.78
Rectal bleeding2.9701.707, 5.170<0.0001
Change in bowel habit1.9040.846, 4.2850.12
Other1.5080.211, 10.7740.682
Female0.6220.514,0.752<0.0001
Age/year increase1.0621.054,1.070<0.0001
AUROC = 0.732 (95% CI 0.710,0.753) n=400534

Table 9. Logistic regression with only age and sex for diagnosis within 18–24 months of the ColonFlag score.

Legend: Cohort analysis. Age at ColonFlag score. Those diagnosed have a diagnosis of colorectal cancer between 18–24 months after the score date in 2011 (baseline) and those with no diagnosis have at least 24 months of follow up after the score date (baseline) with no diagnosis.

Diagnosis at 18–24 mOR95% CIp-value
Age/year increase1.0611.053, 1.069<0.0001
Female0.6210.514, 0.751<0.0001
AUROC = 0.725 (95% CI 0.703, 0.747) (N=400,534)

Table 10. Comparison of AUROCs for a diagnosis of colorectal cancer 18–24 months from ColonFlag score.

Legend: Cohort analysis. Age at ColonFlag score. Those diagnosed have a diagnosis of colorectal cancer between 18–24 months after the score date in 2011 (baseline) and those with no diagnosis have at least 24 months of follow up after the score date (baseline) with no diagnosis.

Diagnosis at 18–24 mAUROC95% CI
Age and sex0.7250.703, 0.747
Any symptom, age and sex0.7250.704, 0.747
ColonFlag, age and sex0.7360.715, 0.759

Table 11. Diagnostic accuracy of ColonFlag to predict diagnosis within 18–24 months in patients with symptoms.

Legend: Cohort analysis. Those diagnosed have a diagnosis of colorectal cancer between 18–24 months after the score date in 2011 (baseline) and those with no diagnosis have at least 24 months of follow up after the score date (baseline) with no diagnosis, and all reported any symptom at the time of the ColonFlag score ± 3 months.

Diagnosis in 18–24
months
OR95% CIp-value
ColonFlag1.0381.017, 1.059<0.0001
Age1.0000.965, 1.0350.98
Female1.0890.613, 1.9350.77
AUROC = 0.744 (95% CI 0.673, 0.815) n=44626
CRC diagnosis
PositiveNegativeTotal
Risk≥0.381%6223229
Risk<0.381%464435144397
524457944626
At 99.5% specificity: sensitivity=11.5%, PPV=2.6%, NPV=99.9%:
CRC diagnosis
PositiveNegativeTotal
Risk≥0.228%2672127238
Risk<0.2284%263736237388
524457444626
At 50.0% sensitivity: specificity=83.8%, PPV=0.4%, NPV=99.9%:

Table 12. Diagnostic accuracy of ColonFlag to predict diagnosis within 18–24 months in patients without symptoms.

Legend: Cohort analysis. Those diagnosed have a diagnosis of colorectal cancer between 18–24 months after the score date in 2011 (baseline) and those with no diagnosis have at least 24 months of follow up after the score date (baseline) with no diagnosis, and all reported no symptom at the time of the ColonFlag score ± 3 months.

Diagnosis at 18–24 mOR95% CIp-value
ColonFlag1.0221.014, 1.031<0.0001
Age1.0291.014, 1.044<0.0001
Female0.7510.604, 0.9350.01
AUROC = 0.736 (95% CI 0.712, 0.759) n=355908
CRC diagnosis
PositiveNegativetotal
Risk ≥0.419%317771780
Risk<0.419%379353749354128
382355526355908
At 99.5% specificity: sensitivity=0.8%, PPV=0.2%, NPV=99.9%
CRC diagnosis
PositiveNegativetotal
Risk≥0.194%1916618266373
Risk<0.194%191289344289535
382355526355908
At 50.0% sensitivity: specificity=81.4%, PPV=0.3%, NPV=99.9%

There were 2,150 patients diagnosed with CRC within 24 months following baseline (Table 13). An increase in the ColonFlag score was associated with a higher likelihood of CRC diagnosis within 24 months (OR=1.050, 95% CI 1.047, 1.053), with an AUROC of 0.783 (95% CI 0.773, 0.792) (Table 14). Additionally, at 99.5% specificity, the ColonFlag score has a sensitivity of 10%. When investigating symptoms, each symptom group was associated with an increased likelihood of diagnosis within 24 months, except appetite loss (OR=1.667, 95% CI 0.989, 3.093) (Table 15). Table 16 reports the numbers of patients at baseline with and without any symptoms reported within 3 months of the ColonFlag score and Table 17 demonstrates the changes in predictive values of the ColonFlag in the presence of symptoms.

Table 13. Characteristics of patients with a ColonFlag score in 2011 with and without a diagnosis within 24 months.

Legend: Cohort study. Those diagnosed have a diagnosis of colorectal between 0–24 months after the ColonFlag score date (baseline) and those with no diagnosis have at least 24 months of follow up.

NumberMean Age
(range)
Mean
ColonFlag
score
(range)
%
Females
With
diagnosis
215073.0 (41–98)82.1 (2–100)46.5
No
diagnosis
40009262.8 (40–106)56.3 (0–100)57.1

Table 14. Diagnostic accuracy of ColonFlag to predict diagnosis within 24 months.

Legend: Cohort study. Those diagnosed have a diagnosis of colorectal between 0–24 months after the ColonFlag score date (baseline) and those with no diagnosis have at least 24 months of follow up. Predictive values for ColonFlag score cut-offs >99.8 (associated with 99.5% specificity) and >88.1 (associated with 50% sensitivity).

Diagnosis within 24
months after FBC
OR95% CIp-value
ColonFlag/unit increase1.0501.047, 1.053<0.0001
AUROC = 0.783 (95% CI 0.773, 0.792) (N=402242)
CRC diagnosis
ColonFlagPositiveNegativeTotal
Score ≥99.823721422379
Score<99.81913397950399863
2150400092402242
At 99.5% specificity: sensitivity=10.0%, PPV=10.0%, NPV=99.5%
CRC diagnosis
ColonFlagPositiveNegativeTotal
Score ≥88.110755489555970
Score<88.11075345197346272
2150400092402242
At 50.0% sensitivity: specificity=86.3%, PPV=1.9%, NPV=99.7%

Table 15. Logistic regression predicting diagnosis within 24 months following the ColonFlag score (baseline).

Legend: Cohort study. Each symptom is included as a binary variable, =1 if the patient reports that symptom at baseline ± 3 months. Those diagnosed have a diagnosis of colorectal between 0–24 months after the ColonFlag score date (baseline) and those with no diagnosis have at least 24 months of follow up.

Diagnosis within 24 months
after FBC
OR95% CIp-value
Constipation1.4981.219, 1.840<0.0001
Abdominal pain2.6122.282, 2.990<0.0001
Appetite loss1.6670.989, 3.0930.11
Diarrhoea2.0831.737, 2.498<0.0001
Weight loss2.4311.809, 3.267<0.0001
Rectal bleeding6.9055.838, 8.168<0.0001
Change in bowel habit4.9974.028, 6.198<0.0001
Other symptom4.0352.453, 6.639<0.0001
Female0.6020.552, 0.656<0.0001
Age/year increase1.0621.058, 1.066<0.0001
AUROC = 0.767 (95% CI 0.758, 0.776) n=402242

Table 16. Patient characteristics by prevalence of symptoms at time of ColonFlag score (baseline).

Legend: Cohort study. Those diagnosed have a diagnosis of colorectal within 18–24 months after the ColonFlag score date (baseline) and those with no diagnosis have at least 24 months of follow up.

NumberMean Age(range)Mean ColonFlag
score(range)
% Females
With symptomsWith diagnosis5272.3 (42–89)78.6 (6–100)55.6
With symptomsNo diagnosis4457462.4 (40–104)55.5 (0–100)61.9
No symptomsWith diagnosis38273.1 (41–98)77.5 (6–100)45.3
No symptomsNo diagnosis35552662.8 (40–106)56.4 (0–100)56.5

Table 17. Predictive values for the ColonFlag in patients with and without any symptom.

Legend: Cohort study. Symptoms are ± 3 months of ColonFlag score (baseline). Predictive values are at a threshold of specificity = 99.5% for the two outcomes.

Outcome CRC between
18 and 24 months
Outcome CRC up
to 24 months
PPV (%)NPV (%)PPV (%)NPV (%)
No symptoms0.299.95.099.6
Symptoms2.699.912.598.6

Trajectories of the ColonFlag scores

Figure 7 shows the trajectories of the ColonFlag score moving back in time from the diagnosis, in cases compared to controls, using age 70 years at index as an example. Similar patterns were seen for other ages. Changes in ColonFlag scores (and therefore changes in FBC indices) appear to begin to occur up to 3–4 years before the diagnosis. The ColonFlag represents a function of risk whose form is unknown and very high values are rounded down to 100, and this effect is evident in Figure 8 and Figure 9, which display the LOWESS lines derived from our data, and may underestimate the gradient as the diagnosis approaches. An abrupt ‘change point’ in the score trajectory was not observed.

ba35bdc4-019d-4361-b46a-0e31ab301f59_figure7.gif

Figure 7. Predicted trends for the ColonFlag score for cases and controls aged 70 years at index.

Legend: Case-control analysis matched 1:100 on age at ColonFlag score, sex, and year of score. “Predicted” refers to trends from mixed models. Time=0 represents index date.

ba35bdc4-019d-4361-b46a-0e31ab301f59_figure8.gif

Figure 8. Trend in the ColonFlag for male cases aged 65–75 years at diagnosis.

ba35bdc4-019d-4361-b46a-0e31ab301f59_figure9.gif

Figure 9. Trend in ColonFlag for female cases aged 65–75 years at diagnosis.

Ability of ColonFlag to detect risk when the FBC is ‘normal’

Among all FBCs, 18.6% were within ‘normal range’ for all indices (Table 18). Among these patients, there were high ColonFlag scores. At the threshold associated with 99.5% specificity, a small proportion (0.10%) had a positive score, of whom 3.17% were diagnosed with CRC within 3 years. At the 50% specificity threshold, 10.25% had a positive score, of whom 1.46% were diagnosed.

Table 18. Proportions of FBCs that are ‘normal’, with ColonFlag scores above two thresholds, and CRC outcomes.

FBCs available
with an associated
ColonFlag score
Number (%) with all
indices within their
reference ranges
Number (%) of these with
a ColonFlag score above
99.8 (99.5% specificity)
Number (%) with a
ColonFlag score above
88.1 (50% specificity)
8,832,859 1,642,474 (18.60%) 1,671/1,642,474 (0.10%) 168,298/1,642,474 (10.25%)
CRC within 3 years of FBC:53/1,671 (3.17%)2,456/168,298 (1.46%)

Discussion

Summary of overall findings

This project was designed to determine whether the ColonFlag, an algorithm derived in Israel through machine learning techniques that draws on FBC data in electronic healthcare records, can identify patients with undiagnosed CRC prior to the reporting of symptoms. In our case-control analysis, trajectories of the ColonFlag score began to diverge in cases compared with controls at around 3–4 years before diagnosis, a time when symptoms are only minimally different between cases and controls. This was further evidenced by non-significant ORs for symptoms for diagnosis in 18–24 months, except rectal bleeding. In our cohort analysis, the addition of either a ColonFlag score or symptoms to age and sex did not significantly improve predictive ability for diagnosis in 18–24 months than age and sex alone. This is consistent with our case-control analysis, showing that much of the discriminatory performance of the ColonFlag at this timescale draws on the age variable rather than the changes in FBC indices, with reduction in AUROC from 0.736 to 0.536 when age is eliminated through the age matched case-control design.

Strengths and limitations

We were confident that our study population had good quality recording of the main outcome, a CRC diagnosis, based upon a high proportion (95%) of the diagnoses occurring within 6 months of each other in the two data sources. Additionally, when we compared the incidence of CRC with that of England, we found a close match, based on cancer registrations in the NCR. Our analysis also confirmed our chosen interval of 18–24 months for the pre-symptomatic phase, which we had also used as the primary outcome interval in our original validation study13.

The company Medial EarlySign had, as part of a previous project, derived ColonFlag scores for almost 9 million FBC reports in 1.9 million patients. The study is limited by the need to exclude a proportion of patients to ensure high quality outcome capture from the NCR and that a further one fifth of FBCs were excluded during quality checks by Medial EarlySign. The ColonFlag also represents a function of risk whose form is unknown, with very high values rounded down to 100 and so the score may underestimate risk overall and the increasing ColonFlag gradient as the diagnosis approaches. There is potential bias in UK data (unlike in Israel), in that a FBC is, in the majority of instances, requested for a reason, to answer a clinical question rather than as a purely routine check. A further limitation in the cohort study was the need to exclude people censored prior to 24 months from baseline.

Comparison to the existing literature

A recently published systematic review has identified that the FBC can play a significant role in detection of CRC22. The FBC is a commonly used test both in general and hospital practice. It measures a number of cellular indices in the blood, and determines whether the person has anaemia (low haemoglobin). Hamilton et al reported not only that anaemia recorded in general practice was an independent risk factor for CRC, but also that iron deficiency, even in the absence of anaemia was also predictive23. Iron deficiency can develop through slow loss of blood from a colonic polyp or cancer, and may become evident subtly through changes in the mean cell volume (MCV), mean cell haemoglobin and mean cell haemoglobin concentration. In addition, a raised platelet count (thrombocytosis) has been identified as a risk marker, particularly for colorectal and lung cancer24. In one study, a third of cases with thrombocytosis and cancer had no recorded cancer symptoms25.

A recent longitudinal study highlighted the potential for early detection of CRC through measuring changes in a number of FBC indices, potentially predating the onset of symptoms15. This study highlighted that FBC trends in cases diverge from controls around 4 years before a CRC diagnosis, which is consistent with this study, as we report trajectories in ColonFlag scores diverge between cases and controls 3–4 years before diagnosis.

Implications for research and practice

Our case-control analysis suggested a significant AUROC for the ColonFlag at 18–24 months prior to diagnosis, a time interval at which symptoms are no more evident in those with undiagnosed cancer than in controls, with the exception of rectal bleeding. This incidentally reinforces the need to investigate rectal bleeding, which we had expected to be a late development leading to rapid referral and diagnosis. We have also demonstrated an upward trend in the ColonFlag score that appears to diverge in cases compared with controls at a time interval of around 3–4 years prior to diagnosis, which clearly sits in the pre-symptomatic phase based on the symptom data we have presented.

It is evident through this study that much of the discriminatory performance of the ColonFlag draws on the age variable rather than the changes in FBC indices, with reduction in AUROC from 0.736 to 0.536 when age is eliminated through the age matched case-control design. Nevertheless, it is clearly appropriate to include age in such an algorithm, and our cohort analysis demonstrated at a ColonFlag score threshold of >99.8 a sensitivity of 10% and a positive predictive value (PPV) of 10% for CRC. In the presence of symptoms, this PPV increases slightly to 12.5%. These are easily sufficient to justify further investigation in patients with scores above this threshold based on current guidelines that recommend referral for cancer investigation ≥3% cancer risk26. Such patients could be offered the non- invasive FIT test, prior to consideration of a colonoscopy.

The ability of the ColonFlag to detect risk in some cases in the presence of a ‘normal’ FBC emphasises the potential for machine-assisted pattern recognition, supporting clinicians either at the point of care or through off-line population searches. This is important, because ‘normal’ FBCs are not examined in any detail by busy clinicians filing results, and in some situations might be ‘batch filed’ without even being opened. Batch filing reduces to zero the already small chance that a human clinician will notice ‘within range’ changes of concern, so the help of an algorithm to do so would be very useful.

Conclusion

Given the widespread availability of coded FBC data in the NHS, and the relative paucity of symptom recording (much of which is free text and therefore invisible to a symptom based algorithm), this project suggests a role for the ColonFlag applied to large volume health data to identify individuals likely to benefit from further investigation to exclude CRC.

Consent

CPRD has ethical approval from the Health Research Authority to hold anonymised patient data and to support research using that data. CPRD’s approval of data access for individual research projects includes ethics approval and consent for those projects. Ethical approval was therefore covered for this study by the CPRD (protocol 14_195RMn2A2R).

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 24 Jan 2023
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
VIEWS
675
 
downloads
84
Citations
CITE
how to cite this article
Holt TA, Virdee PS, Bankhead C et al. Early detection of colorectal cancer using symptoms and the ColonFlag: case-control and cohort studies [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2023, 3:6 (https://doi.org/10.3310/nihropenres.13360.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 24 Jan 2023
Views
6
Cite
Reviewer Report 09 Jul 2024
Jasmin Zessner-Spitzenberg, Medical University of Vienna, Vienna, Austria 
Approved with Reservations
VIEWS 6
This study by Holt et al. aimed to assess the diagnostic  performance of a prediction model that has been built in a large population cohort and validated in primary care patients. The utility of this model in asymptomatic patients was not ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Zessner-Spitzenberg J. Reviewer Report For: Early detection of colorectal cancer using symptoms and the ColonFlag: case-control and cohort studies [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2023, 3:6 (https://doi.org/10.3310/nihropenres.14491.r32225)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
15
Cite
Reviewer Report 07 Sep 2023
Gianluca Rompianesi, Division of Hepato-Bilio-Pancreatic, Minimally Invasive, Robotic and Transplant Surgery, Department of Clinical Medicine and Surgery, Federico II University Hospital, Naples, Italy 
Approved with Reservations
VIEWS 15
In the manuscript titled “Early detection of colorectal cancer using symptoms and the ColonFlag: case-control and cohort studies” Holt and colleagues aim at investigating the ability of a tool with the ability of detecting FBC changes (ColonFlag) in detecting colorectal ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Rompianesi G. Reviewer Report For: Early detection of colorectal cancer using symptoms and the ColonFlag: case-control and cohort studies [version 1; peer review: 2 approved with reservations]. NIHR Open Res 2023, 3:6 (https://doi.org/10.3310/nihropenres.14491.r29977)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 24 Jan 2023
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Are you an NIHR-funded researcher?

If you are a previous or current NIHR award holder, sign up for information about developments, publishing and publications from NIHR Open Research.

You must provide your first name
You must provide your last name
You must provide a valid email address
You must provide an institution.

Thank you!

We'll keep you updated on any major new updates to NIHR Open Research

Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.