The Context as Resource: Contextual Adaptations and Explanations in the Field

From a sociological perspective and on the basis of ethnographic research and semi-structured expert interviews, our paper presents and discusses contextual factors involved in the explanation of a ML prediction by a Predictive Policing research unit called POLAR (a pseudonym). Every Monday, POLAR’s Random Forest model generates predictions about the probability of residential burglaries for individual residential quarters. These predictions are then shared with regional police agencies through a mutually accessible platform. In each agency designated analysts receive these predictions and ’evaluate’ them by assessing, editing, and forwarding them to several operative forces. Finally, the predictions reach frontline officers who are expected to follow their indications, e.g. by patrolling the risk areas more intensely. Difficulties in understanding the model predictions can occur at each of these steps. We will focus on one case in which an explanation request was sent to POLAR from an individual police department. The retelling of this explanatory communication points towards several factors that contextualize the text of the question and the explanation in the specific setting of an organization: dissemination media, formality/informality, hierarchical relations, and others.  

This analytical contextualization of an explanation presupposes that every text – in this case an explanation – is produced in a context. Then, several questions have to be answered by the analyst. Should the interpretation of the ongoing communication be restricted only to the contextual factors that the text itself refers to indexically  or should it refer to additional contextual factors that are known to the observer – deliberately breaking with the participant’s perspective?  And which of the many possible contextual factors is relevant for the task of interpreting the text at hand?  

For the purpose of design, however, contextualization can be understood as the need to modify an alleged acontextual explanation to make it ’fit’ in the context of a specific domain. The questions would be: ’What are the relevant context factors to be taken into account in adapting an explanation to specific domains, users, or situations, and how can one model these contextual factors in an XAI system?’. Our paper discusses the observed empirical case following a different approach. The explanation we describe is produced in response to the explainee’s request, who is always inevitably contextual: bringing their own interests, motivations and skills, and institutional positions, which lead them to formulate their request in a given way at a given place and time - using the appropriate media and formality modes. These contextual factors are not additional constraints to be taken into account to modify the abstract explanation but are the starting elements from which to produce a specific explanation. The context, in our understanding, is not an additional ’disturbance,’ but the starting resource that should be looked for and exploited in the genesis of the explanation. 

The empirical case we focus on happened on one of those Monday mornings, shortly after POLAR shared their predictions for the upcoming week. When we were conducting an interview with POLAR’s leading data scientist, we were interrupted by one of their colleagues who just received a phone call from an individual police department and forwarded their message. The department mentioned that during the weekend there were 23 burglaries in residential areas that were not predicted as forecast areas. They wanted to know in writing how that could be the case and what criteria were used to calculate the prediction. 

Our paper describes in detail the ensuing explanatory communication, starting from the data scientist’s first attempt at looking at the data that was cut short by a software program that didn’t start, the approaching lunch break and a group meeting right after. In the group meeting, the POLAR members discussed how to proceed by first considering ’who asked the question?’, whether it was asked by telephone or by e-mail, and what might be the ’question behind the question’. In the following days POLAR produced a Power-Point presentation that had to be formally run by their Head of Department to ensure formal correctness. Finally, the analyst who asked the question received the explanation and reported it to the director who had commissioned the request in the first place.