Why model?

I came across this little paper on the Introduction to Dynamical Systems and Chaos online course from Santa Fe. It was provided as a supplementary reading in the ‘Modelling’ section. The paper lays out some of the most enduring misconceptions about building models.

“The modeling enterprise extends as far back as Archimedes; and so does its misunderstanding.” Epstein (2008)

So, why model? What are models? And who are modellers?

Prior to reading this paper, my short answers to these questions would have been in accordance with the widely held misconceptions that:

We model to explain and or predict. Models are formal representations (often mathematical) of phenomenon or processes. And a modeller is someone who builds these explicit formal mathematical models. However, Epstein explains:

“Anyone who ventures a projection, or imagines how a social dynamic—an epidemic, war, or migration—would unfold is running some model.”

I like the idea that we all run some implicit models all the time. In the social and political sphere, where it is extremely difficult to operationalize and specify variables, this perspective gives implicit modelling such as drawing dynamical analogies, its due importance.

The paper lays out 16 reasons other than prediction for building models. And the idea that prediction and explanation aren’t the only modelling goals was revelation to me given that I’ve had a love hate relationship with modelling in the past. I am attracted to models, specially those with dynamical systems inclination but the overall tendency towards prediction as a goal often frustrates me. Just to clarify, prediction is a fine goal but my objection arise when 1) we’re deluded into thinking that models give us the tools to predict specific individual behaviours and 2) we can model a phenomenon, especially human behaviour, without first understanding it.

xkcd: Machine Learning

Let me elaborate further in the context of automated predictive system that are currently trending (at least, within my academic circle) and often preoccupy my thinking. Claims to predict “criminal” and “risky” behaviour are examples from last week’s headlines: UK police wants Artificial Intelligence (AI) to predict criminal behaviour before it happens and Predictim, a commercial data analytics firm, claims its AI can flag “risky” babysitters. Unfortunately, these are not the outrageous exceptions but the general direction where things in the digital surveillance sphere seem to be heading.

Behaviours such as “criminal” or “risky” are very complex adaptive behaviours which are a result of infinite ongoing factors, which we can never fully specify in the first place. This makes it impossible to predict criminal behaviour with certainty. Juarrero reminds us why it is impossible to predict human behaviour with precision:

“When we are dealing with complex adaptive systems, surprises are unavoidable. Because of their sensitivity to initial conditions – due, in turn, to their contextual and temporal embeddedness – complex adaptive systems are characterized by unusual twists and novel turns. Since we will never be able to specify any dynamical system’s initial conditions to the requisite (infinite) degree, a fortiori we will never be able to capture all the details and circumstances of anyone’s life and background. Given this limitation, we must always keep in mind that reconstructing specific instances of behavior will always be, at best, an interpretation and not a deduction – a much more fallible type of explanation than we had previously hoped was available. Interpretations of human action are always tentative. Absolute certainty about either what the agent just did, or what he or she will do – specifically – a year from now, is therefore impossible.” (Juarrero 1999, p. 225)

These claims to predict “criminal” or “risky” behaviour are more than a mere misunderstanding of human nature or simple illusions about what AI tools are capable of doing. As these tools are being implemented into the social world, they have grave consequences on people’s lives. When claiming to predict someone’s potential criminality, errors are inevitable. The stakes are high when we get things wrong. Unsurprisingly, it is often society’s most vulnerable, those who are disfranchised, that pay a high price. Indeed, such models are used to further punish and disfranchise those that fall prey to these models.

A slightly different but interrelated issue with modelling to predict is that the strive to predict and explain often ignores the value of describing and/or observing to gain deep understanding. Sure, describing to understand, and explaining and predicting aren’t mutually exclusive. However, in reality, we seem to have blindly adopted prediction and generalization as primary goals of science. Studying to describe and understand, as a result, are undervalued. What is the point of describing? you might ask. I think it is fundamental to understand any phenomena or process as deeply and comprehensibly as possible before we can attempt to explain or predict it, and description is key to gaining such understanding.

I’ll leave you with an insightful Geertz (1973) passage from The Interpretation of Cultures:

“… I have never been impressed with claims that structural linguistics, computer engineering or some other advanced form of thought is going to enable us to understand men without knowing them.”

Advertisement

Situating China’s Social Credit System in history and context

If you have been following the developments in the digital humanities, it is very likely that you’ve come across the news that China is implementing a Social Credit System, officially known as Social Credit Score (SCS). Although the SCS is portrayed as a single integrated system that quantifies all behaviour into credit scores, it is in fact an ecology of fragmented initiatives with many different stakeholders. Broadly speaking, it consists of scoring systems developed by private sectors and by governmental bodies. As far as the governmental perspective is concerned, the SCS is an attempt to promote “trustworthiness” and transparency in the economy which is expected to combat perceived lack of trust in the marketplace, and more generally to harmonize social conduct.

Citizens “trustworthiness” is rated based an individual’s social behaviour such as their crime records, what they say on social media, what they buy, the scores of their friends, and so on. This has possible positive or negative implications on individual’s job, visa, loan applications. As a commitment towards radical transparency is a central driving force behind the SCS, information on subjects’ trustworthiness is made publicly available, and in some circumstances even being actively broadcast. Individual citizens and businesses alike are publicly ranked where the records are publicly open.

Roncheng’s “civilized families” are displayed on public noticeboards like these. (Simina Mistreanu)

The SCS is to become mandatory by 2020 and is currently being implemented in some form or another across parts of China. Areas that are socioeconomically deprived seem prior targets. Rongcheng in the eastern province of Shandong, where the SCS has been rolled out for some time now, is, according to government officials, one of the best examples of the system working as intended, according to government officials.

From a general systems science perspective, the SCS is a self-organizing system that operates through incentive and punishment mechanisms. People with low ratings will, for example, have slower internet speeds, restricted access to restaurants, and the right to travel invoked.

“Higher scores have already become a status symbol, with almost 100,000 people bragging about their scores on Weibo (the Chinese equivalent of Twitter) within months of launch. A citizen’s score can even affect their odds of getting a date, or a marriage partner, because the higher their Sesame rating, the more prominent their dating profile is on Baihe.” (Creemers, 2018)

The SCS has been described as an insidious digital panopticon and a dystopian nightmare where individuals’ every move are monitored and ranked through data generated from all sorts of activity and interactions, online or otherwise through digital technologies (facial recognition tools and biometric information). Many draw parallels between the SCS and the dystopian science fiction Black Mirror episode “Nosedive” where people rate each other based on their interactions.

Many ethical and human rights issues as well as the complete eradication of the idea of privacy have been raised and the negative consequences of such a dystopian nightmare system is indisputable.

With the realization that ‘digital reputations’ could limit opportunities comes the tendency to self-censor and the tendency to be risk-averse. We are unlikely to hit “like” on a Facebook post that protests some government policy knowing that it could impact our ‘digital reputations’. Consequently, people gradually change their behaviour to align with what the system requires, to get better scores. In the process those behaviours and norms defined as “acceptable” by the government are reinforced.

Nonetheless, among the misconceptions surrounding the SCS, there seems to be some consensus that using individual’s digital traces to directly or indirectly influence individual’s behaviour is something that only happens in non-Western totalitarian states. In fact, credit scoring practices are not unfamiliar in Western societies. Facebook, for instance, seems it is developing its own system of rating users trustworthiness.

Facebook’s socioeconomic classifier

It is also worth mentioning Facebook’s emotion tracking patent (where the aim is to monitor individuals’ typing speed in order to predict emotions and adapt messages in response), which was granted in May 2017 and the currently filed Socioeconomic classifier (which might enable Facebook to rank its users according to different social classes), among its series of patents. These developments in combination with others, such as Facebook’s ability to flag individuals through its facial recognition technology without the consent of the user, in some sense constitute a surveillance society. Facebook’s ability to rank and categorize people into a variety of socioeconomic categories has possible impacts on individuals’ opportunities depending on their class, gender, race and sexual orientation. Whether its the type of job ads one is excluded from viewing (due to their gender, class or age) or the exclusion from certain housing ads, Facebook’s ranking and categorizing systems often impact the under-privileged and those who fail to conform to the status quo.

Marshall Allen, July 2018, ProPublica

Along social media platforms, health insurers, and schools, can also be mentioned as examples that share features of the SCS. Like the SCS, these Western industries and institutes, track and surveil people through digital technologies including face recognition tools and biometric information.

We are rated, ranked and categorized using data extracted from us. Similar to the SCS, such ranking and rating often has possible “real” life consequences whether in the form of how much we pay for our insurance, what ads are pushed on us, or how we behave in school yards. The difference between the Chinese SCS and Western tech industry is, while the former is clear and upfront about it, the latter is much more invisible. In fact, such tech giants go out of their way to hide what they are doing.

Rating systems, those by the SCS or deployed through Western tech industry, create unwanted incentives and increase pressure on individuals to conform to the status quo. This creates and contributes to a society that is risk averse.

SocialCooling

“When doctors in New York were given scores this had unexpected results. Doctors that tried to help advanced cancer patients had a higher mortality rate, which translated into a lower score. Doctors that didn’t try to help were rewarded with high scores, even though their patients died prematurely.” Tijmen Schep

Situating the SCS in history and context

The history and context which are crucial to the development of the current SCS are often missing from how the SCS is framed, at least within in Western media .

“[social systems] must be viewed whole cloth as open dynamical systems embedded in a physical, historical, and social fabric” (Juarrero, 1999, p. 201)

As far as China’s political tradition goes, morality and authority are inextricably linked. Enforcing moral standards, monitoring and disciplining the conduct of local officials and individual citizens is seen as the role of the state. “Governing the country by virtue” equals to “governing the country by the law”. Unlike the Western legal system where rights, responsibilities and entitlement of private actors and public sectors are relatively easily categorized, such categories are much more blurred within the Chinese legal system. Individual citizens, government officials, communities and business are all expected to contribute to the whole social and economic harmony and development.

“Chinese political tradition has, for centuries, conceived of society as an organic whole, where harmony can be achieved if all its members conduct themselves as appropriate to their position in public and civil structures. … Critical in this process were ideas about systems theory, derived from natural science and applied in the social context. Influenced by Western scholarship on cybernetics and systems theory, scholars such as Qian Xuesen and Song Jian worked closely with government to develop a conceptual framework for the adoption of systems engineering techniques in governance. Particular regard was given to the role of information flows, not just towards and within government, but also as part of cybernetic feedback loops to create self-correcting responses in society.” (Creemers, 2018, p. 7)

Historically the Chinese government has experimented with some forms of social control and controlling social order through self-policing and social controlling mechanisms go all the way back to the Song Dynasty.

“An 11th-century emperor instituted a grid system where groups of five to 25 households kept tabs on each other and were empowered to arrest delinquents” Mistreanu, 2018. The current SCS then is an extension of such historical traditions. The difference now is the addition of digital technologies.

From the Chinese authorities perspective the SCS epitomizes a self-correcting feedback loop where “trustworthiness” and social morality are fostered through incentives and punishments.

This by no means is to argue that the SCS is any less of a digital panopticon. However, by highlighting history and context, often missing from the SCS narrative, we can paint a somewhat complex and nuanced image of the system (as opposed to the often alarming pieces which are stripped of context and history). Furthermore, while we are preoccupied by the stories of how China is becoming one giant surveillance prison, we miss the indirect and evasive practices that are happening within our own “civilized” Western system.

 

Bibliography

Creemers, R. (2018). China’s Social Credit System: An Evolving Practice of Control.

Juarrero, A. (1999). Dynamics in action: Intentional behavior as a complex system (p. 127143). Cambridge, MA: MIT press.

 

 

The Scope of Existential Anthropology – Jackson

Such a beautifully written passage which compels you to reflect, wonder, and think …

Like other human sciences, anthropology has drawn inspiration from many disciplines and sought to build its identity through association with them. But the positivism that anthropology hoped to derive from the natural sciences proved to be as elusive as the authenticity it sought from the humanities. Moreover, though lip service was paid to the models and methods of biology, ecology, psychology, fluid mechanics, structural linguistics, topology, quantum mechanics, mathematics, economics, and general systems theory, anthropologists seldom deployed these analytically or systematically. Rather, they were adopted as images and metaphors. Thus, society was said to function like a living organism, regulate energy like a machine, to be structured like language, organized like a corporation, comparable to a person, or open to interpretation like a text.

Jackson. M (2013) Lifeworlds: Essays in Existential Anthropology. (Chapter 1, The Scope of Existential Anthropology, p.3)