Should The Washington Group Questions Determine If Difficulties Are Long-term Or Short-term?

Some data producers and analysts have expressed concern over how the Washington Group (WG) questions deal with people who have temporary functional limitations. Do we want people with a temporary condition – for example, a person with a broken leg that is expected to heal completely — to be identified by the WG questions? And if we don’t, shouldn’t the questions have a clause that identifies difficulties that have “lasted or are expected to last at least 6 months” or even refer to “permanent” difficulties?

Most countries want to identify people with long term limitations, because long term limitations will have a greater effect on long-term participation in society and well-being, or because there is an interest in estimating the number of people who would qualify for social protection benefits, eligibility for which is based on having a long-term condition. The presumption is that short term limitations are more easily overcome and not in need of the same level of government interventions.

On the other hand, if the interest is in a snapshot of who, at the time of a census or survey, is facing these difficulties, and would at that time benefit from accommodations or universal design, or be in need of assistive devices, medical care, or some other intervention, then identifying people with short term difficulties is not problematic. For example, if the WG questions were being used as part of an intake procedure at a refugee camp (a use of the questions that is currently being studied) then maybe identifying people with short term difficulties – like the example of the broken leg – is important for determining who should be referred to services.

If those using the WG questions are mostly interested in long-term difficulties,  additional questions could be asked of people who identify as having difficulties using the WG questions, such as, “How long do you expect these difficulties to last?  Less than 6 months? Between 6 months and a year? Over a year?”  Of course, this would add more time to a survey –  would be too much for a census – and such questions would have to be tested before they were used.

So, should a clause aimed at distinguishing permanent from temporary conditions be included in the WG questions?  We think not for a few reasons.

1. Clauses that refer to length of time or permanence/temporary are complicated and often misinterpreted. In fact, earlier versions of the WG questions had such clauses and cognitive testing found them to be problematic. Often people ignored the clause altogether, focused on only a part of the clause, or misinterpreted it. They may think of “permanent” as the fact that they always had these difficulties, not that they will always have them from now on. They, at times, respond to “at least 6 months” as if it were “only 6 months,” so temporary conditions are included and longer-term ones are excluded – the exact opposite of what is intended. Cognitive testing of these questions shows that although on paper we may think the question is clear and will capture what we’re interested in, that it often does not. Worse, we have reason to expect that less educated people will be more likely to misinterpret the question, so we not only get measurement error, but biased measurement error.
2. People tend to report their usual situation, not a temporary one. In our cognitive testing, we found that in most cases people actually did not report temporary conditions. They understood – for the most part – that the questions were aimed at long term conditions.
3. Including people with temporary conditions in the population identified as having a disability is not large enough to significantly influence the results. The purpose of the WG questions is to get disability prevalence and to compare outcomes for people with and without disabilities. The small number of people with temporary disabilities who identify themselves as having a lot of difficulty would add only a tiny fraction to prevalence, and would not be big enough to qualitatively influence the observed correlation between disability status and employment, education, etc.

So, there is a choice.  Add a clause to get at the long -term nature of a person’s difficulty and introduce measurement error that could very well be biased, or leave such a clause out and misclassify some people as having a limitation in functioning when the limitation is temporary. Based on our testing, we have concluded the second option generates less error, and will not affect the types of analyses for which the WG questions were designed.

Remember, as explained in a previous blog, the WG are not designed to determine eligibility for particular individuals for various programs. In those instances, knowing the temporary/permanent nature of a functional limitation may be very important. But for population statistics, we will introduce more error in our prevalence estimates and the comparisons of various outcomes via disaggregating by disability if we try to identify whether a condition is long-term within the WG questions themselves. The complications introduced by adding phrases such as, “lasting or expecting to last 6 months or more”, will introduce more error, and not exclude that many people who would have otherwise responded they had serious difficulties.  Having a few people with temporary disabilities identified by the WG questions as having a disability will not significantly affect population estimates of prevalence or the impact of disability on various outcomes. If it is very important to data users, then additional questions should be developed that would follow the WG questions and address that issue of duration of difficulty.