Data is always embedded in a context, and in understanding that context we can begin to see its limits and its biases…

The challenge is to move from big data to data with depth, and also as our OR community would say, it is to move from data to decisions. Here’s a nice talk by Kate Crawford, and some key points that I liked from the talk:

“1. Bias – we’re trying to make impressions about the world with data that has gaps in it, and may not be completely representative of the complete picture. We need to know where the data is coming from, and to think about who it represents, need to account publicly for the weaknesses in our data…”

“2. Signal – We’re making choices every time we decide to represent our data. For example, if we start using GPS data to find out what parts of the cities we want to repair, we might get data that’s biased towards areas with high income populations (more smartphone users in these areas, than the low-income areas). This can have very material consequences for social inequity. As we move into an era where are our devices are passively collecting data from our surroundings and becoming proxies for public needs, we run the risk of entrenching particular kinds of social inequity… With every dataset, we need to ask who’s excluded, who’s visible, who lives in the shadow of a big datasets. Neither of these problems go away as more and more people use smartphones.. Technologies will always be differentially be adopted…”

3. Scale – How do we think about orders of magnitude?

“Always the same mistake: to believe that to see the logical pattern of social facts, you must extract yourself from the details, go upward until you embrace vast landscapes panoramically.” – Gabriel Tarde, 1898.

“… We need to go to people to get their individual stories, ask the why and the how questions and not just the how many. Numbers don’t speak for themselves. We are the ones that give them voice, we are the one who lend them interpretations and read into the data — blindly following “big data” represents a possible epistemological blindness — can create problems if you are making business decisions or public spending decisions…”

“Solution – data insights exist at all levels, but need to bring together a range of tools and keep a critical eye on what we can’t see in our datasets…”

Challenge – to move from big data to data with depth.

Also our OR community would say, the challenge is to move from data –> decisions, instead of data –> model –> decisions.