Sources, accessibility and use of disability data in Uganda

Government surveys and censuses

Between 2009/10 and 2019/20, Uganda Bureau of Statistics (UBOS) collected disability data through one census and eight unique surveys. Some surveys were conducted multiple times.^[1] The National Panel Survey was conducted five times,^[2] the National Household Survey four times, and the Demographic and Health Survey (DHS) twice –bringing the total number of UBOS sources that included disability data over the period to sixteen (Table 1).^[3]

Administrative data systems

Administrative data is data derived from the functions of public administration, for example relating to registration, transaction and record keeping. There are five administrative data systems operated by ministries, departments and agencies (MDAs) in Uganda which capture data on disability (Table 2). Despite intentions, the government does not implement a disability management information system (MIS) or a disability-focused database.

Content of government disability data in Uganda

Types of questions used to collect data

In some of the sources, listed in Table 1, UBOS generated disability data through direct questions on disability. For example, in the Functional Difficulties Survey (FDS) (2017) and DHS (2016) data was collected on “visual, hearing, mobility, communicative, and cognitive” functional difficulties. The FDS also collected data on “psychological/intellectual” difficulties. The 2014 census collected data on “visual, hearing and mobility” disabilities, as well as on “memory”. Examples of disability-related questions that UBOS asked in the census are: “do you have difficulty seeing, even if wearing glasses?” and “do you have difficulty hearing, even if using a hearing aid?”. The responses UBOS provided for each question were “no – no difficulty, yes – some difficulty, yes – a lot of difficulty, cannot do at all”.

In other sources, UBOS generated disability data by asking indirect questions on disability. For example, in the National Household Survey (2016/17) and National Panel Survey (2015/16) UBOS provided “disability” as one of four multiple choice answers to questions such as “what was the main reason that you were absent from your job last week?” (National Household Survey), and “what was the main reason why you did not seek work or try to start a business in the last four weeks?” (National Panel Survey).

Level of detail and disaggregation

Disability data from UBOS sources is often disaggregated by age, gender and geography (e.g. rural/urban). More detailed disaggregation of UBOS disability data, such as employment/economic activity, household income, savings and assets, access to the internet, health status, and literacy, are less common, but are present in some data sources, as shown in Table 3.

The aspects of data collected by UBOS relevant to disability are often limited in scope and can lack detail, whereas the socio-economic, health and education aspects of the same data are typically detailed. Usually the levels of disaggregation available are reduced to the binary of ‘disability, yes or no’ or more traditional categories of disability (e.g. seeing, hearing, cognitive). Very few sources allow data to be disaggregated by the severity of disability or disabilities.

Questionnaire frameworks

UBOS has not adopted a standardised framework for its questionnaires. The disability questions used in its surveys have been based on three different types of questionnaire frameworks: Washington Groups Questions (WGQs), modified WGQs and national frameworks. These different frameworks have been used interchangeably and intermittently. For example, between 2013 and 2016, UBOS collected disability data in six sources, using three rounds of National Panel Survey, one DHS, a National Service Delivery Survey and the national census. Nationally defined questions were used in the National Panel Survey and the National Service Delivery Survey, whereas WGQs were used in the DHS and modified WGQs were used in the census. A unique set of questions has been used each time UBOS has used nationally defined questions.^[20]

There are a range of challenges constraining the disability data landscape in Uganda. For government data, challenges include the following.

Limited resources and capacities hinder data capture by ministries, departments and agencies (MDAs)

Data collection by MDAs in Uganda is hindered by their lack of financial resources: the MoGLSD is considered by interviewees to be “pitifully underfunded”; UBOS relies on financial support from development partners, which influences its focus; and the NCD’s two-person research department conducts mostly qualitative desk research rather than in-depth data-led studies. Limited resources also constrain the ability of many local government offices and facilities to capture data, particularly on disability. Interviewees reported that, “data capture [using the Health Management Information System (HMIS)] at health centers is burdensome due to low manpower in clinics, so nurses both treat patients and enter HMIS data” and “the HMIS questionnaire is already big and time-consuming, the small staff at a health unit may not capture everyone”. They also commented that teachers often struggle to fulfil their responsibilities with respect to the Education Management Information System (EMIS) due to overload from other responsibilities.

Data from surveys and censuses is not disaggregated to a useful level for those working on disability inclusion

Data disaggregation refers to the level of detail in which data can be divided into sub-groups such as disability type, gender, age, geographic location, socio-economic group, etc.^[26]To be useful for policy design, budget allocation, programme planning and monitoring progress, data needs to be highly disaggregated. Almost all interviewees emphasised inadequate disaggregation as a key challenge with disability data produced by UBOS surveys and censuses. One interviewee explained, “we do not know how many persons with disabilities are in each age category, region, or socioeconomic category”.

Broadly speaking, issues around disaggregation for surveys can be categorised into two inter-related groups: those concerning sample design (e.g. disaggregation by geographic location) and those concerning questionnaire design (e.g. disaggregation by categories of disability).

Interviewees highlighted that a lack of disaggregation by geographic region is a major concern. Much of the data cannot be disaggregated beyond the regional level to districts, counties, and municipalities as it becomes unrepresentative. Typically, a given geographic area can be represented by a sample size of 800 to 1,000 people. Therefore, to generate data disaggregated to the level of Uganda’s 134 districts a minimum sample size of 107,200 is needed. Currently, the average sample size of UBOS household surveys that have generated disability data is 7,195 (see Annex, Table A2). This number exceeds that required for regional disaggregation but falls far short of the levels needed to enable district-level disaggregation.

Issues around categories of disability in government sources are a product of questionnaire design. Interviewees stressed that some categories of disability are missing from the data. For example, psychosocial disabilities are entirely omitted and UBOS has only recently started collecting data related to albinism and dwarfism. Interviewees also noted that, “there is no sufficient data on women with disabilities”. Adjusting current questionnaire frameworks would solve this issue.

The current levels of disaggregation mean that government surveys and censuses often do not contain relevant information to meet the needs and interests of different persons with disabilities and OPDs. The lack of coverage and disaggregation in UBOS disability data therefore undermines its potential utility.

To cater for the disaggregation needs of all persons with disabilities and OPDs is not feasible, as UBOS would have to overcome practical constrains inherent in surveys and censuses, such as the limited number of questions that can be asked in such exercises and the difficulties of financing large surveys. Ideally, stakeholders would reach consensus on which levels of disaggregation would benefit the largest proportion of users, while being conscious of what is feasible for data collection systems to deliver. Other data needs and interests could then potentially be met using other systems, such as administrative data systems.

The disability data produced by surveys and censuses lacks timeliness

Between 2009 and 2017, UBOS published a total of sixteen surveys and censuses that contained disability data. At the time of writing (in September 2020), UBOS most recently published disability data in 2017.^[27] Given the relative abundance of data produced through the previous seven years, the current three-year gap is comparatively large. The next round of disability data UBOS is scheduled to collect is the Functional Difficulties Survey in 2022. By that point there will have been a five-year gap.^[28]

Concerns about the reliability of disability data from surveys and censuses reduces trust in the data

Many interviewees held the perception that UBOS disability data is unreliable. For example, one interviewee commented that none of UBOS’s disability data has been “tested for quality by an auditing team”. Another interviewee suggested that, “many people do not declare disability during surveys”, and there was a general agreement that underreporting of disability is more pervasive than overreporting. One interviewee from an OPD explained that because of the inaccuracies “when you are writing a report and you are going to present a paper, you cannot quote any figures with confidence”.

Conversely, some interviewees argued that the data is accurate. A government respondent explained that UBOS surveys are prepared and conducted by professional, trained staff using representative samples and following both national and international standards and guidelines. Some interviewees also felt that the 12.4% disability rate reported by the 2014 census was roughly accurate.

Different models of categorising disability in surveys and censuses have led to inconsistencies in the data

As shown in Figure 1, surveys and censuses in Uganda have adopted a number of different systems for categorising and measuring disability. While the use of different frameworks has caused inconsistency and reduced comparability between surveys, even surveys which have consistently used the same framework have produced different prevalence numbers, as shown in Table 3.

Stakeholders have reservations about the use of Washington Group Questions (WGQs)

The reputation of WGQs as the leading global standard for collecting accurate and comparable disability statistics is growing. However, despite this, and UBOS’s role in the development of the Washington Group,^[29] there were strong concerns among some research participants about the use of WGQs. Multiple interviewees felt that WGQs “are not appropriate in the African context”, with one interviewee going so far to claim that “they are culturally insensitive”.^[30] The limitations of the short set of questions were recognised by respondents, with one noting that, “WGQs do not capture psychosocial disabilities”. The distrust of WGQs went much deeper than this though. One interviewee argued, “we have to be aware of the limitations of the WGQs and their impact on the accuracy of disability reporting”. Another comment that, “WGQs takes a person’s opinion, which is not enough”.

Perceived difficulties in accessibility of disability data from surveys and censuses have limited the use of disability data

Some OPDs interviewed held misinformed beliefs about the availability and accessibility of UBOS data. For example, one interviewee claimed, “it is hard to come to UBOS if I need data, because it would take me the whole day to visit UBOS to request for data”. In fact, there are multiple ways to request UBOS data without physically visiting the institution, or even without using a computer or telephone. Clearer communication about this by UBOS, MDAs and large OPDs could keep the disability rights movement informed of all the mechanisms through which UBOS data can be requested and accessed, as well as the different forms which it can come in (for example, in reports, in raw data, etc.).

The disability data captured by government administrative data systems is very limited

There has been a lack of tangible efforts to better integrate the capturing of disability data into the administrative data systems which exist, or to create a system that solely focuses on disability, such as a registry of persons with disabilities, in Uganda. Therefore, sections of existing administrative systems which collect disability data are underdeveloped and produce negligible amounts of disability data. For example, the EMIS produces data only on limited indicators such as the completion and retention rates of students with disabilities. And, where it does exist, data is often not useful, as the use of non-standardised questions limits the potential for cross-system analysis. For example, the Gender-based Violence Management Information System asks, “do you have any disabilities, yes or no?”, whereas the birth registration form asks, “disability if any?”.^[31]

Administrative data systems are not extensively deployed, and persons with disabilities are disproportionately omitted

Existing administrative data systems in Uganda are deployed in a limited number of settings. For example, one interviewee explained, “there is a lot of focus on primary education data but […] there is no EMIS for secondary schools or tertiary/technical education institutions”. Another interviewee noted that more public health facilities report into DHIS2 (District Health Information Software 2) than private health facilities. Filling these gaps would improve the coverage of disability data.

In general, administrative data systems only collect data from persons who engage with or use an administered service. Those people who do not engage with government services are omitted from the corresponding data systems. This particularly affects the records associated with persons with disabilities, as they may not engage with services as often as other groups. The MoGLSD reported, for example, that in 2015 it assisted 1,828 people out of a total of over three million persons with disabilities in the country. One interviewee pointed out that, in Uganda, “there are persons with disabilities that are not in school, a vast majority – who collects data on them?”.

There are problems with collection and storage of information in administrative systems

Most of the administrative systems can be described as ‘hybrid systems’, in that they are partially paper-based and partially digital. For example, within the MoGLSD, data is manually collected at sub-county level and submitted on paper to district-based community development officers; these officers then aggregate the records and submit, in paper-format, to the MoGLSD, where it is then entered into a digital information management system or database. Similarly, in the Ministry of Health’s HMIS local health clinics submit paper-based aggregated data to district statisticians who key the data into DHIS2 on a monthly basis. The HMIS is the most sophisticated of all the administrative data systems as data is input at the district level, and by a biostatistician.

Using hybrid systems means that some of the benefits gained by completely digitising data capture – such as a reduction in human errors, reduced labour, and instant outputs – are lost. The systems are also siloed and are not interoperable with one another, which represents a missed opportunity to combine the systems to generate data that is of better quality.

In some instances, perverse incentives inhibit accurate data collection. For example, one interviewee claimed that due to budget allocation processes “teachers sometimes have incentives to overreport, and other times have incentives to underreport”.

Data from administrative data systems is not accessible

The primary reason disability data from government administrative systems is not widely used, is that access to it is blocked for institutions which do not implement the systems.^[32] This disconnect feeds into the false belief held by some interviewees that Uganda’s administrative systems contain no disability data: “there is no disability data in the HMIS/DHIS2”, “MDAs all collect data but have nothing on disability, the GBV [gender-based violence] database has no information concerning persons with disabilities”.

Non-government studies and surveys

In addition to government entities, a number of OPDs and other types of organisations collect disability data in Uganda. This includes development partners (e.g. the UK’s Department for International Development^[33] and the Netherland Development Cooperation), multilateral organisations (e.g. Unicef, World Health Organization and International Labour Organization), universities, medical groups and NGOs (domestic and international). These groups usually conduct surveys or qualitative studies themselves, but sometimes commission consultants to carry out the work on their behalf. These efforts are often motivated by shortcomings in government disability data.

Development partners (bilateral and multilateral organisations), directly and indirectly, dominate the production of disability data outside of government. Their own sources (e.g. scoping papers or databanks) are themselves significant features of Uganda’s disability data landscape. And, because they provide funding, the needs and interest of these groups also tend to dictate which data is collected by the other types of actors in the country (e.g. medical groups, universities or Ugandan OPDs) and when data is collected. According to one interviewee, “donor interests affect what work OPDs do, and where the focus should be. Most of them are only accountable to their donors rather than to their members or the country in general”.

OPDs that have collected disability data include the National Union of Disabled Persons of Uganda (NUDIPU),^[34] National Union of Women with Disabilities of Uganda, Uganda National Association of the Blind, Uganda Parents of People with Intellectual Disabilities, Epilepsy Support Association of Uganda (ESAU), and Triumph Uganda. Examples of data collected by universities and medical groups are the works led by the current State Minister for Health and current President of the Uganda Medical Association.^[35][36]

Non-government administrative systems

Many OPDs and other organisations operate their own administrative data systems. The majority of these systems consist of records of membership and records of beneficiaries as and when they access services. Most OPDs operate paper-based systems that are very modest. According to one interviewee, “many OPDs do not even have a typewriter, let alone a computer, therefore most still use simple exercise books to keep bits of information on their members”. However, a few organisations implement more sophisticated systems. For example, ESAU registers an individual when they become a member and issues them with a personalised membership card. This is the case for all 10,500 members. ESAU also records when members collect medicines from their services in a dedicated registry.

There is at least one employment-based administrative system in operation, which is owned by the Uganda Nation Association the Blind. However, an informant explained that, it is “very small, it currently holds around 120 persons with disabilities and does not include their skills sets”.

There are two separate plans to create systems that collect data on persons with disabilities and their employment. NUDIPU committed to building a skill-set focused data bank of persons with disabilities in its strategic plan for 2020–2024, and Cheshire Services Uganda committed to creating an “accessible online database bringing persons with disabilities job seekers and employers together” as one of the outputs for their program Increasing Access to Waged Employment for Persons with Disabilities in Kampala 2018–2020.^[37] Neither system is operational yet.

The Uganda Nation Association the Blind and Cheshire Services Uganda are not well placed to implement data systems for persons with disabilities and their employment indefinitely.^[38] Such systems are expensive to run and require large-scale deployment to generate significant amounts of data. Therefore, a more sustainable option would be for the state to collect economic and employment disability data and to construct and maintain a database. The MoGLSD, UBOS, the NCD, or NUDIPU possess the structural foundation needed. Moreover, the remit of the MoGLSD’s ESD and UBOS includes responsibility for this.

There are a range of challenges with non-government data which are also constraining the disability data landscape in Uganda. These include the following.

Resource and capacity constraints hinder the ability of many organisations of persons with disabilities (OPDs) to collect disability data

Development partners, international OPDs and other NGOs operating in Uganda typically have funds to generate disability data. However, the majority of Ugandan OPDs face significant resource constraints, which usually translate into a lack of knowledge when it comes to data collection. According to one interviewee, many OPDs “do not have computers and operate from people’s homes” and “the vast majority do not have monitoring and evaluation staff or officers with research or data skills”. A national ODP explained that: “We have 15 staff, 11 of whom are junior support staff and 4 of whom are managerial staff. We have no data, research or monitoring and evaluation department right now. The Executive Director, as per the Human Resources manual, performs those functions”.

Countering the claims that resource constraints limit data collection, one interviewee from a government entity argued that OPDs do not value data enough to produce it. Likewise, an interviewee from a national ODP explained: “Saying that few resources is the cause is not enough. Even with few resources you can collect data. It is interest, more than budget, that causes lack of data among OPDs”.

Non-government disability data often has limited re-use value as it is project focused

Disability data collected by or on behalf of development partners is usually collected in an ad hoc manner for the purposes of a given programme of activities. Moreover, it is nearly always a secondary objective or output which complements a primary intervention. One interviewee explained, “collecting data is usually tied to an individual project; project-driven work means donor-driven work, hence donor priorities are the focus”. Another noted that, “data collection is not proactive; it is haphazard and dictated by donors”. This means that, beyond the context of the project it was collected to inform, the data’s potential utility is minimal.

The quality of non-government disability data is not trusted by users

The impression among some research participants was that sources of non-government disability data are normally unreliable. One participant argued that, “there is no reliable data developed independently by OPDs”. Another explained that, “we carried out a baseline survey in 2012 but could not establish accurate data in terms of numbers for persons with intellectual disabilities in Uganda”.

Higher quality data sources are difficult to achieve because resource constraints preclude groups from hiring teams of skilled data experts (e.g. statisticians, data scientists etc.) and from obtaining large sample sizes in surveys. Additionally, several OPDs, universities and medical groups (some of whom complete data collection on behalf of or in collaboration with development partners) feel they have not received consistent support from the government to enable them to develop more robust institutional capacity for data collection.

Despite concerns about quality, the organisations that capture their own data still tend to use it to inform decisions about the project it was collected as a part of. They also typically recycle it again in the future if possible. For example, one interviewee said that their organisation conducted a survey in 2012 and despite the data’s inaccuracies they still refer to it. However, the interviewee also emphasised that their organisation has mostly come to depend on UBOS data, because eight years had passed since the organisation had collected its own data. In many cases even the organisations which collect disability data in surveys cannot produce it regularly enough to meet their own needs.

Limited sharing of disability data ensures lower use

Organisations with their own disability data usually store it internally, either on closed-access computer systems or in paper archives, and do not pro-actively share it with their peers. One interviewee explained that, “data sharing among OPDs is very limited”, while another that, “data sharing is not systematic; it happens haphazardly depending on who is requesting for it, most OPDs keep the data to themselves; it is not even shared online; this is a big problem”. As a result, organisations that do not collect their own data are almost entirely precluded from accessing these sources. In turn, the potential for them to make of use them is significantly reduced.

Similarly, the data collected in non-government administrative systems sits in siloes and are closed to external use. The extent to which OPDs and other NGOs make use of their own records, or the records of a peer, is not entirely clear. However, that no interviewees reported using this type of data is a strong indication of the low value attached to it.

A culture of data use has been slowly developing in Uganda. However, there is a long way to go before disability data is widely used. Where disability data is used, it is mainly to inform or strengthen advocacy campaigns, to design strategies, plan projects, undertake monitoring and evaluation, and to support funding applications and reports. There is limited evidence that disability data is used widely in the design, implementation and evaluation of programmes, policies and services aimed at improving disability inclusion and inclusive employment.

Use of data has been significantly constrained by challenges in data quality, accessibility and relevance, as outlined previously, but there are other factors that have impacted on data use.

Firstly, demand for both government and non-government data on disability is growing, but this growth in demand is emerging from a low starting point. There has been a relatively short history of data use by MDAs, OPDs and civil society, with the value of disability data only being realised lately in some pockets. Interviews confirmed that data from UBOS was in highest demand, and that UBOS provides the most commonly used sources of disability data for MDAs and OPDs. However, it was also reported that “demand for disability data is not as robust as demand for gender data”. Interviewees explained that the majority of grassroots OPDs in Uganda are not invested in data. For example, one interviewee outlined, “we have not seriously tried to access data”. Some interviewees considered some aspect of data use, such as monitoring and evaluation, as being imposed by donors rather than something they could genuinely benefit from.

Secondly, limited capacities and confidence to use disability data are widespread. This includes the technology (computers, software, internet and electricity) to access and process quantitative data, as well as the technical skills to analyse and apply it. One government interviewee emphasised though that non-government actors are diverse in their abilities and interests, and that “while most do not have the capacity to use data, some do”. The general lack of skills led one interviewee to state, “quantitative data means nothing to some of us; we can’t make sense of them”. Both MDAs and OPDs also highlighted that quantitative data was only of limited use to them, as they felt it omits the day-to-day realities of persons with disabilities, or “the stories behind the numbers”.