The final aspect of the functioning of the Universal Credit system it is important to understand is the information DWP publishes about how the system is performing. As noted earlier, there is limited documentation about the systems the department operates. This is also true of the performance data about Universal Credit, which can be characterised as inconsistent and incomplete.

The data that is released by DWP about the performance of Universal Credit tends to be published periodically, rather than in real-time. It is published as a mix of formats, is sometimes non-machine readable, and the datasets published change over time, while some data, such as those on advance payments, are published on an ad hoc basis. The department publishes this data on the GOV.UK website as well as DWP’s StatXplore website (Annex 4 contains a list of statistics published by DWP). This is despite the evidence that the Universal Credit team have built up a modern analytics capability where real-time data would be the expectation.

The gaps in performance data exist at both the macro and micro-level. At the macro-level, the United Nations Special Rapporteur on extreme poverty and human rights criticised DWP for the poor quality of the data published about use of sanctions.¹ When it comes to understanding if Universal Credit is saving money for the government, researcher Anna Powell-Smith has concluded that:²

Government either isn’t collecting meaningful data on the cost savings of Universal Credit. Or, it is collecting this data but is inexplicably failing to publish it.

When considering if Universal Credit has resulted in more people in work, the NAO concluded that, not only does the data not exist but that DWP will:

never be able to measure whether Universal Credit actually leads to 200,000 more people in work, because it cannot isolate the effect of Universal Credit from other economic factors in increasing employment ³

At the micro-level, datasets about how the digital service is used, such as the number and types of activities required of claimants, are almost entirely absent from the public domain. Where detailed data is published, it appears to be in response to a particular point that the department is trying to make, or in relation to a particular research project.

There is also limited information published about the systems used to deliver Universal Credit. As the UN Special Rapporteur noted, there is no definitive list of all the component systems that make up the Universal Credit service:⁴

The existence, purpose and basic functioning of these automated government systems remains a mystery in many cases, fueling misconceptions and anxiety about them.

The Government Service Standard, which, in theory, all government services must meet, requires departments to release the source code for their services. The department does maintain a public code repository at https://github.com/dwp⁵, but it appears that no part of the Universal Credit codebase has been published online, not even the code used to calculate a Universal Credit payment. The GDS service assessment of the alpha version of the service from 2014 notes that:

Despite having identified some reusable components, the lack of a plan to open up any source code at all is disappointing. At this stage the services code should already be opening up (or otherwise be able to explain clearly the specific and compelling reasons why this can not be done for a particular subset of the code). A working assumption that security prohibits the opening up [of] any code isn’t reasonable. ⁶

Further, DWP has stated that they do not maintain an archive of user interface changes. ⁷ Given the importance of prototyping, user research and testing in the development process, this seems unlikely.⁸ They also do not appear to maintain a definitive list of experiments being run in different regions. In short, an adequate description of the systems used to deliver Universal Credit and the data used to monitor it are absent from the public domain.

The multiple reviews conducted by the NAO and the Public Accounts Committee are invaluable in developing an understanding of Universal Credit. (The NAO’s focus on the agile development process that DWP have implemented is particularly useful).⁹ However, the focus of these reports has generally been on the progress of the rollout, costs and the need to address particular issues with the policy.¹⁰ Mostly absent from these reports are questions about some of the core political and administrative assumptions that remain from Universal Credit’s origins in 2010 - where have these concepts succeeded, and where have they failed? The other thing that is absent is an ethical assessment of the technical and design choices made in the digitisation of working-age welfare services. For example, what are the implications of deploying digital technology and service design the way that DWP has with Universal Credit? Ten years on from the formulation of Universal Credit, 2020 is a good time to start asking these questions.