Translating The Washington Group Short Set Questions

The Washington Group Short and Extended Sets of questions were developed through a long process that involved multiple rounds of cognitive and field testing. The results from multiple empirical investigations lend support that the questions are effective in identifying people whose impairments pose difficulties undertaking basic activities that put them at risk of exclusion from participating in society because of barriers in the environment. By participation we mean things like going to school, getting a job, and fully participating in civic events and family life. By barriers, we mean not only physical barriers but cultural and attitudinal barriers, institutional barriers (such as laws restricting jobs open to people with disabilities), and inaccessible ICT. These are complicated concepts which is why it has taken so long to develop, test, and approve the Washington Group Questions.

However, these questions were developed in English, and most people in the world do not speak English. The Washington Group Short Set (WGSS) has been translated into a range of languages, but many of these have not undergone cognitive testing, leaving doubt to their validity. Many upcoming censuses and surveys will require new translations, and available translations may not be suitable for use. Two countries that speak the same language, might have differences in the connotation of certain words and phrases. Arabic in Morocco and Saudi Arabia is not the same, nor is Portuguese in Portugal and Brazil, or Spanish in Spain and Central America, French in France or the countries of West Africa. Even English in the United Kingdom is not always the same as in the United States.

Responses to questions are very sensitive to wording, and even seemingly small differences in the words used can affect the results. Therefore, it is very important that when the Washington Group questions are used in different countries that both the questions and the response options are translated properly, and preferably, cognitively tested in the language to be used.

Simple back translation is not the best practice. This technique can leave many inconsistencies or misinterpretations in place.  For example, in one survey in Mumbai, a question up for consideration in a disability survey was “Does your child have difficulty walking around the house?” In English, “around the house” generally means within in the house, for example walking from room to room, or across the room. When this was interpreted into Marathi, it meant literally walking around the circumference of the house, which was very confusing to respondents. Why would their child circle the outside of their house? Back translation did not catch this error. Translating back from Marathi to English yielded the exact same English words, but it did not catch the fact that in the two languages they were different questions. After testing the translation, it was determined that the words for “across the room” in Marathi captured the intended concept without causing confusion among respondents.

The Washington Group recommends a different protocol for translation, which can be found on the website. This conceptual or team translation protocol recommends that two individuals or teams first review the concepts in the WGSS and then, working separately, translate the questions and response options into the target language. The translators/teams and at least one reviewer then meet to review the translations and make comments on issues they find or changes they recommend by consensus. An adjudicator (who may or may not be at the review meeting) will ultimately decide whether to adopt the changes or recommendations, or make other changes based on the reviewer(s)’ findings. Then the reviewed translated document is cognitively pretested. Throughout the process, decisions made at every step are documented to inform designers and analysts about how the final translation was reached.

This process involves more effort than a simple forward/back translation, but it is vital. A lot of effort and expense goes into administering a survey, and the data can be used by many stakeholders for many years to help design and evaluate public policies. Making sure the questions and response options are of high quality is therefore very important.

In fact, the best practice would be to undertake a cognitive test of the translation to determine if potential respondents are interpreting the new translation as intended. Guidelines on cognitive testing of translations are forthcoming on the WG website

There will be no way after the fact to determine if the questions were being interpreted as intended, or to what extent measurement error due to a bad translation is affecting the results. Following this procedure will increase the validity and usefulness of the data collected.