Update 14/09/2015: I've added a bit at the bottom.
Abstract
The Work Capability Assessment (WCA) of Employment and
Support Allowance (ESA) is the legal test by which eligibility for ESA, the UK’s
social security benefit for those unable to work due to illness or disability,
is determined. This is an update of my previous analysis, including an extra
200,000 results, of Incapacity Benefit claimants being migrated over to ESA.
All the data is from freely-available government datasets. I show that there is significant bias towards the poorest and
most disabled in the WCA and this must be urgently addressed.
Introduction
In 2013 and 2014, I conducted a statistical analysis of
the outcomes of the Work Capability Assessment for Incapacity Benefit Claimants
being migrated to Employment and Support Allowance on my old blog. I showed
significant relationships between WCA judgements and variables like life
expectancy and local rates of poverty, and interpreted this to mean that the
WCA is biased against people in poorer or less healthy areas.
Since that analysis, I’ve learned that the United Nations
are investigating the UK for “grave and systematic” abuses of the human rights
of disabled people; a group in which I am included. What better time to update
that analysis?
There are numerous variables which ESA should logically
be related to. The most immediately
obvious is local rates of disability and health. There should be a higher
proportion of people being found fit for work in healthier areas, with lower
disability. Conversely, there should be a higher proportion of people going
into the Support Group in areas with lower life expectancy and higher rates of
disability.
There have been accusations that the WCA is not a valid
assessment of disability. If this is the case, there will be relationships with
variables other than disability. In the UKs current political climate, cutting government
expenditure has been touted as one of the key goals of both governments since
2010, when ESA began being rolled out on a wide scale. As the majority of
people getting ESA are poor (and get income-based ESA), I have chosen to test
relationships with financial deprivation and, at the suggestion of an ESA whistleblower,
I have chosen to test the relationship between WCA outcome with educational
attainment; their suggestion was that people with better educations are more
capable of getting into the WRAG and Support Groups.
Method
I collated data from various government sources (full
details are in the data provided) including Public Health England and the Department
for Work and Pensions. I designed the dataset to show relationships at the
Local Authority level as this was the most consistent level of abstraction used
amongst the various sources of information. I chose to combine the deprivation
variables into a composite measure, averaging out the Local Authority residents
in the 20% richest areas of England with the 20% poorest areas of England. In
this composite measure, a lower score means lower rates of deprivation.
I then conducted statistical analysis using SPSS 20 and
Microsoft Excel.
Descriptive
Statistics
There is a great deal of variance between areas in the
proportion of claimants that are put into each WCA category as shown in Table
1. In some Local Authorities, almost 50% of people are being judged fit for
work compared to 25% in other areas.
|
% Fit for Work
|
% WRAG
|
% Support Group
|
Mean
|
18.95
|
37.17
|
43.88
|
Standard Deviation
|
3.57
|
4.177
|
5.12
|
Minimum
|
10.87
|
24.72
|
33.24
|
Maximum
|
46.63
|
46.63
|
58.25
|
Table 1: Descriptive statistics for WCA Outcome rates
Correlations
I first chose to test whether there were relationships
between WCA outcomes and the variables of interest to determine if it was worth
conducting a deeper analysis, and this is shown in the correlation matrix (table
2) below. All results are statistically significant (p=<.05) and N=324. All tests
are two-tailed.
|
Life Expectancy
|
% with moderate or
severe disability
|
Deprivation
|
% with 5 A*-C GCSEs
|
% Fit for Work
|
-.559
|
.315
|
.501
|
-.233
|
% WRAG
|
ns
|
.314
|
ns
|
-.239
|
%Support
|
.437
|
-.475
|
-.340
|
.358
|
Table 2. Pearson’s r correlation matrix of WCA outcomes
with important variables. NS = not
significant.
These correlations are illustrated below in Images 1, 2
and 3.
Image 1. WCA Outcome vs local life expectancy.
Image 2. WCA outcome vs local educational attainment
Image 3. WCA outcome vs local deprivation
Interim conclusion
It is clear that there is a complex relationship between
the variables, given that the relationship between the test variables and WCA
outcomes changes. However, it is also clear that the outcome of the WCA is not
purely dictated by the levels of disability in an area; if this was the case,
the proportion of claimants going into the WRAG and Support Groups would be
positively correlated with the proportion of people with moderate to severe
disabilities, and the proportion of people being found fit for work would show a
negative correlation. This is not the case, and as the results show, the more
disabled people there are, the fewer people are going into the Support Group and
more are being found capable of work and having their benefit eligibility withdrawn.
Given the complex interrelationship between the
variables, I have chosen to conduct three stepwise linear regressions, one for
each WCA outcome. This will statistically control for the effect that, for
example, poverty has on health, and will give us a clearer picture of what the
true relationship between the variables is.
Regression
Analysis
Fit For Work judgements
In this regression, both deprivation and the rates of
severe disability contributed to a statistically significant model (R=.511,
F(2,321)=56.595, p<.001). As
deprivation increased, the rate of fit for work judgements also increased. As
the rate of severe disabilities increased, the rate of fit for work judgements
increased also.
Work-Related Activity Group judgements
In this regression, the proportion of people with
moderate disabilities, the proportion of those with 5 A*-C GCSEs and local
deprivation levels all contributed to a significant model (R=.408, F(3,320)=21.341,
p <.001).
As deprivation and the rate of GCSEs decreased, the
proportion going into the Work-Related Activity Group also decreased. However,
as rates of moderate disability increased, so did the rate of people going into
the WRAG.
Support Group
In this regression, both the proportion of people with
severe disabilities and the proportion of those with 5 or more A*-C GCSEs were
contributors to a significant model (R=.515, F(2,321)=57.911, p<.001).
As the rate of severe disabilities increased, the
proportion of claimants in the Support Group decreased. However, as educational
attainment increased, the proportion going into the Support Group also
increased.
The regression analyses are summarised in Table 3, below.
WCA Outcome
|
Variable
|
B
|
Std. Error
|
beta
|
t
|
Sig.
|
Fit for Work
|
Constant
|
13.610
|
.754
|
|
|
|
|
Deprivation
|
1.497
|
.226
|
.419
|
6.611
|
<.001
|
|
Severe Disability %
|
.234
|
.117
|
.127
|
2.00
|
.046
|
|
|
|
|
|
|
|
WRAG
|
Constant
|
36.198
|
3.096
|
|
|
|
|
Moderate Disability %
|
.981
|
1.63
|
.331
|
6.030
|
<.001
|
|
GCSE %
|
-.114
|
.035
|
-.190
|
-3.238
|
.001
|
|
Deprivation
|
-.704
|
.235
|
-.169
|
-2.996
|
.003
|
|
|
|
|
|
|
|
Support Group
|
Constant
|
44.411
|
2.960
|
|
|
|
|
Severe Disability %
|
-1.077
|
.139
|
-.406
|
-7.735
|
<.001
|
|
GCSE %
|
.140
|
.039
|
.190
|
3.615
|
<.001
|
Table 3. Final regression models and statistics for each
WCA outcome
Discussion
It is clear that there is a complex and counterintuitive
relationship between WCA outcomes and variables which should logically be related
and others that should not be related. If the WCA were a fair and accurate
assessment of disability, the proportion of people going into the WRAG and
Support Groups would be positively correlated with the rates of moderate and
severe disability in the local area. This is not the case as shown in Images 1,
2 and 3.
Image 4, below, further illustrates the relationship
between economic deprivation and WCA outcomes. It is apparently that fit for
work judgements occur more frequently in areas a higher proportion of their
population living in the 20% poorest LSOAs of England, while the inverse is
true of areas with fewer fit for work judgements than average.
Image 4. Proportion of people in poverty in areas with
above or below average proportions of fit for work judgements.
Fit For Work Judgements
That fitness for work judgements decrease when the rate
of severe disabilities in an area increases is a strong indictment of the WCA.
If the test was valid, it would be expected that this relationship would be
reversed. It is logical that, in areas with higher rates of disability, there would
be fewer people found able to work. That this is not the case raises concerns
about the validity of the WCA.
Furthermore, this decision is influenced by the affluence
of the claimant’s area. It would be expected that ESA claimants are poorer than
the general population because of the means-testing component but also because
disabled people are generally poorer. However, this should have no effect on
the outcome of the WCA; all these people are recognised as disabled and pass
the financial or contributory criteria required to even undergo the WCA. It is
a great concern that people in poorer areas – where social security benefits contribute
more to the local economy – are facing more stringent or less fair WCAs than
people in richer areas.
The ESA Groups
Once a claimant has had their WCA and hasn't been found
fit for work, they are placed into either the WRAG or Support Group. What is
most concerning here is the pattern of significant findings; it would be
expected that the WRAG, for claimants “capable of work-related activity” but
incapable of work, would be correlated with the proportion of moderate
disabilities. However, the Support Group, for claimants “incapable of
work-related activity” should be positively related to the proportion of severe
disabilities, but is not. Instead, areas with higher rates of severe disability
have fewer Support Group judgements.
Furthermore, educational attainment has a significant
effect on both WRAG and Support Group judgements. Higher educational attainment
in a local area correlates with more Support Group judgements, but the opposite
is true of the WRAG. This suggests that the WCA outcome is somewhat influenced
by the claimant’s experience or skill at navigating the bureaucratic structures
of the Department for Work and Pensions, such as fully understanding all the
questions of the ESA50 medical questionnaire. This is a variable that should
have no effect on an objective and valid assessment of disability, and that it
does is a concern, particularly considering how many disabilities start in
childhood and the direct and indirect effects on education.
Limitations
As with all investigations, this one has limitations.
Firstly, the data I have used is local averages, and this limits the
conclusions that can be drawn. Were a similar investigation completed with
access to full details of ESA claimants and WCA (points, claimants age and
education, etc) it would be easier to see particular biases. Local Authority averages
were all that were publicly available.
Secondly, this study has looked at some nebulous concepts
(e.g. “health”). The use of surrogate variables is common, but is something to
bear in mind when interpreting the data. For example, a drug to prevent heart
attack might be tested on its ability to lower blood pressure, and from that it
would be inferred it could prevent heart attacks. The rate of heart attacks is
inferred from a third variable. This is what I have done with, for example,
education. I am inferring the level of education in an area by the rate of high
GCSE performances.
Conclusions
The Work Capability Assessment is simply not a valid
assessment of disability and is unduly affected by variables which should, in
theory, have no impact on the results. This is a source of great concern for
disabled people. However, improvements to the method by which successful
claimants are assigned to the WRAG or Support group has become more important
since the Budget, when it was announced that new claimants placed in the WRAG
would receive 30% less money, further impoverishing this already disadvantaged
group.
As for the WCA itself; there are three key points raised
in this analysis:
1. The
poorest are the least likely to get onto ESA at all, as areas with higher
socio-economic deprivation have a higher rate of fit-for-work judgements.
2. The
assignment of claimants to the ESA groups has no relationship to the severity
of the disability the claimants experience, as severe disabilities reduce the
probably of a Support Group finding but also increase the probability of a fit-for-work
judgement
3. Educational
attainment should play no role in the WCA, but higher educational attainment in
a local area increases the number of Support Group judgements, which are
preferred by Claimants for their lower conditionality and larger payments.
I believe I have shown
that the Work Capability Assessment is biased, particularly against the poorest
claimants who are more likely to ‘fall at the first hurdle’ and be found fit
for work. However, the WCA also shows bias against the most severely disabled
who it should be helping the most. That it can have such a relationship with
educational attainment is alarming, and again this suggests a further bias towards those
with better educations.
As others have said, the WCA needs an urgent overhaul or
to be scrapped in its entirety. This analysis shows why: by favouring richer,
healthier areas with better access to education, the WCA is reinforcing the
systemic disadvantages faced by disabled people.
Extras
It's hard to get across how important this is using just numbers; a lot of people don't quite grasp that, behind each number, there's a person. That there's a correlation of r=.55 between two variables, with a p value of less than 0.000000000001 (literally!) means a lot to me, but not much to others, so here I'll be adding a few more graphs and such as I add new variables or come up with ideas that might help illustrate what's going on above.
My data is available
here, and is colour-coded to show sources.
Here are two graphs; one shows the rate of early deaths in areas that most or least frequently give fit for work judgements.
The second shows the proportion of people living in either the 20% poorest or richest areas depending on whether that area gives amongst the least or most Fit for Work and Support Group judgements: In areas where there's the highest rate of fit-for-work judgements, over 35% of people are in the poorest areas, vs only 5% in areas with the lowest rate.
Deprivation
I've added a new variable to make the relationships with socioeconomic deprivation a bit easier to understand; in the future, I'll be talking about the proportion of local households with two or more indices of deprivation (such as low income, low access to good healthcare, and so on).
Targets
@BendyGirl, over on Twitter, raised the possibility that the relationships could be accounted for by targets used to control how many judgements WCA assessors can make. The short answer is: I'm not sure! It's not really possible to answer with the data available, but some of it is consistent with the idea of targets so they may well be a source of the bias I've found.
This graph, using the new definition of deprivation, is a good example.
What we see is WRAG judgements staying approximately the same, while Support Group judgements go down as poverty increases and fit for work judgements go up. This is consistent with the 'targets' hypothesis.
Assume an HCP is only allowed to make 1 Support Group judgement out of every 10 they make. Now imagine they're in an area where there are going to be a lot of claimants. They have no idea which conditions are going to come through the door next; they're pressured by the system to save that one Support Group allocation for someone who "really needs it".
They're therefore more inclined to place claimants into the WRAG regardless of how severe their disability is, in case someone worse comes next, but there's a target for the WRAG too! The WRAG fills up with people that should be in the Support Group, and there's nowhere for people who should be in the WRAG to go; they're found Fit for Work.
This scenario is consistent with the graph above, and the fact that the proportion of Support Group judgements decreases (and FFW increases) as the number of WCAs increases significantly, albeit weakly and with poor fit (Support Group r2=.032). If targets were having a strong role in the bias I've found, I would expect the r2 to be much higher. This is an example of a statistically significant result but one of dubious real-world relevance.
However, it's important to note that this data only includes the 'highest-level' decision. If the claimant went to appeal, then the tribunal's decision is the one recorded. Likewise, if the claimant hasn't had a formal decision by the DWP, then the HCPs decision is the one recorded. It's possible that targets are playing a minimal role, as tribunals are (to my knowledge) not subject to them; any bias introduced by targets at the HCP and DWP level would be would be undone by the tribunal decisions assuming the majority of people affected appeal the decision (as these were IB claimants being migrated to ESA, I assume most would appeal!)
It's also worth noting that the correlation between the proportion of Support Group judgements and the number of WCAs done is non-existent when controlling for deprivation (r=.047, p=.402) - if targets were having a large effect, I'd expect there to be a correlation; more WCAs done mean the assessors are more likely to reach the targets and have to start putting people into the WRAG instead of Support Group, so the proportion in the Support Group would go down. This isn't the case, as the relationship between the number of WCAs done and the proportion of judgements can be explained by local poverty - areas with more poverty do more WCAs as they have more ESA claimants.
The relationship between poverty and WCA outcome isn't accounted for by the total number of WCAs done, however. Together, this is evidence against the hypothesis that the observed bias is caused by targets or "norms".