There's general distrust and dissatisfaction with the way that data is collected and used, both by the private and the public sectors. Dr Jeni Tennison explains what’s going wrong and why, what we need to do instead, and how we might practically get there.
I recently had to set up a new email account, which meant having to decide which of Google’s “smart features” to activate. I knew these features would involve Google processing data from my email, calendar and elsewhere. As I worked through them, I was left trying to figure out not only how it would affect me personally (would it be useful, would they use it to target me somehow, would it affect how I use email) but also larger questions such as how my choices would affect other people, and whether it would further entrench Google’s market position.
Even as a well-educated, technically savvy, data-aware person, these choices were difficult to make. I have no idea if I made the “right” ones for either my own protection or that of the people I correspond with. I don’t know how my small choices here might (in combination with everyone else’s similar choices) have economic, societal, democratic or environmental impacts.
So I was left feeling uneasy and concerned that I might have made the wrong choice, and guilty that I didn’t spend more time working out what the right choice would have been.
This feeling of disquiet is not uncommon. Many people feel dissatisfaction with the way that data is collected and used, and distrust of the private and public sector organisations that do it. A Washington Post / Schar School poll of US households found that 73% of people think how companies collect information to target ads is an unjustified use of people’s private information. An ODI / YouGov poll found only 30% of those surveyed trust the government to use personal data about them ethically.
In this post I’m going to look at what’s going wrong and why, what we need to do instead, and how we might practically get there.
What’s going wrong
I believe the root of the problems we have with data is the way we push decisions about it down to individuals. There are three reasons this is inherently problematic:
- We’re not good at making informed decisions about data
- The way that data is processed in aggregate means our decisions made in isolation affect other people (and vice versa)
- Data has wider societal and collective impacts that we cannot hope to factor into our decision-making
We’re not good at making decisions about data
The excellent paper “The Myth of Individual Control: Mapping the Limitations of Privacy Self-management” by Kröger, Lutz and Ullrich summarises many of the issues with individualising choices about data in this diagram.
As the diagram shows, the informedness and rationality of the privacy choices we make (e.g. agreeing to T&Cs, giving consent, setting privacy options) are impaired by our time constraints, knowledge gaps and cognitive biases. They are also challenged by the overwhelming complexity, lack of transparency and practical obstacles to presenting privacy information in modern data collection and processing.
What’s more, our privacy choices aren’t entirely voluntary: data controllers nudge and coerce us through dark patterns in user interfaces; they provide financial incentives; and they only offer non-negotiable agreements for services we are dependent on.
Just as many economists have come to realise that we are not rational actors, we in the data governance community have to realise the same of data subjects.
Data is processed in aggregate
A second challenge to the individualised notion of data governance is the way in which data about each of us is now processed based on its relation to data about other people like us.
It used to be that organisations would primarily look at data about one individual when making choices about them, such as whether to provide a bank loan. Now, organisations use data about groups and whole populations when making those decisions. They examine what people who are similar to you have done in the past, and use that to make predictions about what you might do in the future.
Salomé Viljoen’s paper “A relational theory of data governance” spells out the implications of this shift, and the need to develop data governance approaches that factor in the impacts on groups and communities, and on people who are not data providers, but are affected by the use of data.
Data has wider collective impacts
The way data is used has broader implications as well: effects on our democracy, our economy, and our environment.
There has been much discussion of the degree to which big political decisions, such as President Trump’s election and the Brexit referendum, were influenced by micro-targeted adverts that disrupted the democratic process.
Our economies are now dominated by big tech companies, whose power is supported by the amount of data they have and how they use it. Amazon has used data to get into an advantageous position in markets for physical goods; the use of data at an individual level in the insurance market affects our ability to pool risk; personalised pricing introduces biases into who can afford goods and services.
Data collection and processing also has a wider environmental effect, with about 3.7% of carbon emissions being the result of digital technology.
These are broad externalities from the use of data that stem from an uncoordinated aggregation of our individual choices.
What we need
The inherent features of the way data is used, and the way we function as humans, mean individualised approaches aren’t going to fix problems like excessive surveillance, unfair data-driven decision-making, or environmental externalities. Instead, we need to look at more collective approaches to govern how data is collected, used and shared.
This doesn’t mean getting rid of individual choices altogether – there will always be places where we have different risk appetites, preferences, and moral compunctions. Rather, it means adopting participative models to decide the bounds of those choices, and how they are framed and presented to us, so that the choice architecture protects us rather than nudging us in harmful directions.
Collective data governance approaches look different in different places:
- We already have decisions that are made democratically about data: for example, data protection laws say data should be shared in life-or-death situations (vital interests), and that we should participate in the census every 10 years. However imperfectly, these decisions reflect societal norms through democratic participation.
- We also already see, particularly in health, independent research and ethics boards being used to determine who can get access to data and what they can do with it. These boards should have representatives that provide input on behalf of the communities affected by the use of data.
- Some organisations are also experimenting with participatory processes such as citizen juries and assemblies to gather information about how the public feel about different kinds of uses of data.
- Finally, there are new forms of institution being set up such as data trusts, unions or cooperatives which aim to provide a collective locus for decisions about data.
All of these work in different ways and have different advantages and disadvantages: citizen juries are out of reach for many small organisations, for example, and not all decisions can be made at a parliamentary level. There is no one-size-fits-all answer, but a range of approaches to explore.
One particular challenge, though, is the need to give these mechanisms teeth.
One reason we like being given privacy controls, despite not always using them, is that we don’t trust decisions made by other people or organisations are going to be made in our best interest. Any collective approach will need to establish its trustworthiness and legitimacy.
That will need to include transparency about the decision-making process and the resulting decisions from that process; accountability through third-party scrutiny, whether that’s by civil society or by regulators; routes of appeal that enable those not intimately involved in the process to question it; and routes for redress when the decision-making process goes wrong and causes harm.
How we get there
I’m working through an initiative called Connected by data on three areas where work is needed to change how data governance works:
- Changes to the narrative and the words we use when talking about data, particularly to challenge the notion of individual data ownership.
- Changes to practice so that organisations large and small can adopt more collective and participatory data governance approaches.
- Changes to public policy, so that regulation enables and encourages the adoption of these approaches.
Internationally, changes to public policy around data governance are likely to originate from national governments and from the European Union (EU), which has particular influence in this area through GDPR. Some might be achieved simply by data protection authorities, such as the Information Commissioner’s Office in the UK, encouraging consultation and participatory approaches through the guidance they issue, and indicating that they will look kindly on organisations that can demonstrate active engagement with their customers, clients or citizens.
But other changes are more fundamental. Data protection laws have historically largely focused on the individual interests and rights of data subjects – those about whom data is collected rather than those who are impacted by the use of that data. New measures around data governance more broadly, particularly in the context of artificial intelligence, such as the EU’s Data Governance Act, are starting to introduce consideration of wider interests, particularly for non-personal data, but have limited scope to change those data protection fundamentals.
With that context in mind, there is a particular opportunity to encourage and enable collective data governance through revisions to the UK’s data protection laws this year, outlined in the Data: a new direction consultation, led from the Department for Digital, Culture, Media and Sport.
One opportunity is around the legitimate interests lawful basis. This lawful basis allows organisations to process data if the balance of their interests against individual privacy interests goes in their favour. As currently defined and implemented, legitimate interests suffers both from illegitimacy – a perception that it’s used by organisations to obscure what they’re doing with data – and a narrow focus on only those interests relevant to the organisation and individuals. Changes could:
- Broaden its scope to enable organisations to consider community, societal and other wider interests.
- Require organisations to have community representation in decision making when performing the balancing test, so that it isn’t just the organisation making the decision on its own behalf.
- Require transparency around both the decision making process and the results of the balancing test, to build trust with those affected by it.
In general, though, data is not the first domain to need to make the transition from one focused on individual interests to one that recognises the interests of groups, communities and societies. We can see a similar transition in food safety, for example, where requirements for honest and transparent labelling to inform consumer choice have been gradually supplemented with laws and institutions that provide fundamental protections and quality control, and even towards those that take away some of our preferred choices from us, such as setting levies to reduce the amount of sugar in soft drinks.
Through my Affiliated Researcher position at the Bennett Institute, I will be exploring what we can learn and apply from other sectors like food safety in which collective and individual interests need to be balanced and managed. What might we learn from the various layers of decision-making around local planning, for example, or the introduction of regulation to limit air pollution.
While the direction of travel towards more collective ways of making decisions about data is clear, where we will end up, and how quickly we can move along that path, is much less so. But these are questions that will determine the future relationship between us, as individuals and as a society, and the digital technologies that have already revolutionised our lives.
View Dr Jeni Tennison’s presentation on Collective Data Governance.
The views and opinions expressed in this post are those of the author(s) and not necessarily those of the Bennett Institute for Public Policy.
