Working with data in complexity

James Gleick, the author of the classic book “Chaos: Making a New Science” has written a terrific review of Jill Lepore’s new book “If Then: How the Simulmatics Corporation Invented the Future.”The book covers the origin of data science as applied to democracy, and comes as conversations about social media, algorithms, and electoral manipulation are in full swing due to the US election and the release of The Social Dilemma.

Gleick’s review is worth a read. He covers some basic complexity theory when working with data. He provides a good history of the discovery of how the principles of “work at fine granularity” helps to see patterns that aren’t otherwise there. He also shows how the data companies – Facebook, Google, Amazon – has mastered the principle of “data precedes the framework” that lies at the heart of good sensemaking. For me, both of these principles learned from anthro-complexity, are essential in defining my complexity practice.

Working at fine granularity means that, if you are looking for patterns, you need lots of data points before seeing what those patterns are. You cannot simply stake the temperature in one location and make a general conclusion about what the weather is. You need not only many sites, but many kinds of data, including air pressure, wind speed and direction, humidity and so on – in order to draw a weather map that can then be used to predict what MIGHT happen. The more data you have, the more models you can run, and the closer you can come to a probable prediction of the future state. The data companies are able to work at such a fine level of granularity that they can not only reliably predict the behaviour of individuals, but they can also serve information in a way that results in probable changes to behaviour. AS a result, social media is destroying democracy, as it segments and divides people for the purpose of marketing, but also dividing them into camps that are so disconnected from one another that Facebook has already been responsible for one genocide, in Myanmar.

Data preceding the framework means that you don’t start with a framework and try to fit data to that matrix, but rather, you let the data reveal patterns that can then be used to generate activity. Once you have a ton of data, and you start querying it, you will see stable patterns. If you turn these into a framework for action, you can sometimes catalyze new behaviours or actions. This is useful if you are trying to shift dynamics in a toxic culture. But in the dystopian use of this principle, Facebook for example notices the kinds of behaviours that you demonstrate and then serves you information to get you to buy things in a pattern that is similar to others who share a particular set of connections and experiences and behaviours. Cambridge Analytica used this power in many elections, including the 2016 US election and the Brexit referendum as well as elections in Trinidad and Tobago and other places to create divisions that resulted in a particular result being achieved. You can see that story in The Great Hack. Algorithms that were designed to sell products was quickly repurposed to sell ideas, and the result has been the most perilous threat to democracy since the system was invented.

Complex systems are fundamentally unpredictable but using data you can learn about probabilities. If you have a lot of data you gain an advantage over your competitors. If you have all the data you gain an advantage over your customers, turning them from the customer to the product. “If you’re not paying, you are the product” is the adage that signals that customers are now more valuable products to companies that the stuff they are trying to sell to them.

Putting these principles to use for good.

I work with complexity, and that means that I also work with these same principles in helping organizations and communities confront the complex nature of their work. Unlike Facebook though )he says polemically) I try to operate from a moral and ethical standpoint. At any rate, the data we are able to work within our complexity work is pretty fine-grained but not fine-grained enough to provide accurate pictures of what can be manipulated. We work with small pieces of narrative data, collecting them using a variety of methods and using different tools to look for patterns. Tools include NarraFirma, Sensmaker and Spryng, all of which do this work. We work with our clients and their people to look for patterns in these stories and then generate what are called “actionable insights” using methods of complex facilitation and dialogic practice. These insights give us the inspiration to try things and see what happens. When things work, we do more and when they don’t we stop and try something else.

It’s a simple approach derived from a variety of approaches and toolsets. It allows us to sift through hundreds of stories and use them to generate new ideas and actions. It is getting to the point that all my strategic work now is actually just about making sense of data, but doing it in a human way. We don’t use algorithms to generate actions. We use the natural tools of human sensemaking to do it. But instead of starting with a blank slate and a vision statement that is disconnected from reality, we start with a picture of the stories that matter and we ask ourselves, what can we start, stop, stabilize or create to take us where we want to go.

In a world that is becoming increasingly dystopian and where our human facilities are being used against us, it’s immensely satisfying to use the ancient human capacities of telling stories and listening for patterns to create action together. I think in some ways doing work this way is an essential antidote to the way the machines are beginning to determine our next moves. You can use complexity tools like this to look at things like your own patterns of social media use and try to make some small changes to see what happens. Delete the apps from your phone, visit sites incognito, actively seek out warm connections with real humans in your community and look for people that get served very different ads and YouTube videos and recommended search results. Talk to them. They are being made to be very different from you, but away from the digital world, in the slower, warmer world of actual unmediated human interaction, they are not so different.

Postscript

Over the past few years, my work has taken shape from the following bodies of work:

• Dave Snowden’s theories of anthro-complexity, which forms the basis of my understanding of complexity theory and some of the tools for addressing it, including facilitation tools and Sensemaker.
• Cynthia Kurtz’s Participatory Narrative Inquiry, which is a developmental evaluation approach that uses stories and methods of sensemaking that she partly developed with Dave and then subsequently. I use her software, NarraFirma, for most of our narrative work now.
• Glenda Eoyang’s Human System Dynamics is a set of tools and methods for working with complex adaptive systems.
• The facilitation and leadership practices from the Art of Hosting which help us to develop the personal capacity to work dialogically with complexity.