Hazard communication by volcanologists: part 2 - quality standards for volcanic hazard assessments

Abstract

Contextualisation is the critical process of interactions between volcanologists and risk governance decision-makers and specifically the tailoring of hazard assessments to ensure they are driven by the needs of decision-makers. Quality assurance standards for the contextualisation of the analysis and communication of volcanic hazards do not formally exist. For volcanologists this governance lacuna creates a foreseeable and avoidable managerial hazard.

This is the second of two papers that together investigate the interface between the scientific treatment of volcanic hazards and the governance of volcanic risks. Both papers are principally concerned with issues of risk governance and their focus is hazard communication by volcanologists at this hazard-risk interface (the interface) during periods of volcanic unrest. In our first paper “Hazard communication by volcanologists: Part 1 - Framing the case for contextualisation and related quality standards in volcanic hazard assessments”, (Bretton et al, J Appl. Volcanol. DOI 10.1186/s13617-018-0077-x, 2018) we investigated the perceived qualities of more ‘socially robust’ hazard assessments and argued that the working practices of contextualisation must be more methodical.

In this paper, we focus on the actual process of contextualisation and argue that quality assurance standards need to be devised and adopted by volcanologists undertaking hazard assessments that are robust enough to bear legal scrutiny. Such standards are necessary: (1) to facilitate a more structured approach to contextualisation; (2) to preserve the core values of traditional scientific methodologies; (3) to address a working assumption that, in the absence of effective regulation, the ‘equilibrium of contextualisation’ may be unduly influenced by the demands of decision-makers; and (4) to mitigate the managerial risks related to volcanologists assuming responsibility for contextualisation.

Our initial literature review reveals the realities of contextualisation in the absence of effective regulation and identifies many of the foreseeable practical challenges historically faced by volcanic hazard assessors. We investigate and characterise these challenges in order to develop quality standards tailored for practical use. In addition, we present new empirical data acquired from a survey of 33 experienced volcanologists, 18 of whom were selected for one-to-one semi-structured interviews. The survey captures a snapshot of working practices and related sentiments that might indicate whether a more structured approach to contextualisation would find favour amongst volcanologists and, if so, what principles of contextualisation would be most acceptable. This evaluation of published evidence and new data permits us to identify seven aspects of volcanic hazard assessments relevant to quality assurance - the methods and status of hazard analyses, and the delivery, content, status, perception and advice content of hazard communications.

Our proposed quality standards and related proto-type code of practice together address issues of materiality, comprehensibility, proximity and integrity. They offer the possibility of hazard assessments having greater validity and utility in that they will be framed by reference to the sentiments and actions of their users.

Introduction

This is the second of two related papers¹ that together investigate the interface between the scientific treatment of volcanic hazards and the governance of volcanic risks. Both papers are principally concerned with issues of risk governance and their focus is hazard communication by volcanologists at this hazard-risk interface (the interface) during periods of volcanic unrest.

Two extremes of knowledge production can be described and contrasted. Mode-1 products have been described as conventional scholarly reflections that are driven by the motivations of their producers “to get the science right” (US/NRC 1996). Detached from both social context, and political and other values, these products seek to characterise and predict the natural world ‘as it is’, for a separate, sequential, downstream political management response. By contrast, Mode-2 products are driven by the requirements of their users and strive “to get the right science”. Their production and application take place through co-evolutionary processes of open dialogue between producers and users and hence these processes can be said to be processes through which the science and ultimately the actions resulting from that science are ensured to be appropriate for the social context in which they take place. This contextualisation involves methodical changes of not only scientific-knowledge outputs and outcomes (i.e. processes of communication), but also scientific-knowledge production itself.

The trial of six scientists and a government decision-maker for involuntary manslaughter following the L’Aquila earthquake highlights the practical realities of contextualisation, the influence of multiple cultural, political, social and scientific factors before, during and after the disaster, and the complexities of risk governance regimes involving government entities at national, regional and local levels (Alexander 2010, 2013, 2014a, 2014b; Gabrielli and Di Brucci 2014). We hypothesise that this trial identified a failure of contextualisation, as we have defined above and in our first paper, and we argue that, to improve hazard communication at the interface, volcanologists need to adopt an approach that openly embraces the benefits and confronts the challenges of contextualisation.

In our first paper, we argued that the working practices of contextualisation must be more methodical and should strive to be open, transparent and fully articulated. Contextualisation that meets proposed quality assurance standards will enhance hazard assessments and, thereby, the utility of their outputs and outcomes. That utility should be framed and reviewed by reference to the sentiments and informed actions of decision-makers.

In this paper, the focus is directed away from the perceived qualities of more ‘socially robust’ hazard assessments towards the actual process of contextualisation itself. The four quality assurance standards that were advocated in our first paper are summarised in Table 1. In this paper, we investigate: (1) whether these four standards reflect the sentiments of practitioners at the interface: and (2) whether they can be deployed to address practical challenges. Although our focus is on volcanic risks, this approach could be applied equally to communications about other natural hazards.

Table 1 Four quality assurance standards for the processes of contextualising volcanic hazard assessments

Methods

Assumptions and terminology

This investigation principally concerns volcanologists in their role as volcanic hazard assessors and some of its findings are based upon data derived from them. We accept that we could have approached the same research issues by analysing data from decision makers acting in their capacity as stakeholders using scientific assessments. The decision to acquire data only from volcanologists was based upon two working hypotheses.

Firstly, while accepting evidence that a degree of informal contextualisation of hazard assessments is becoming more commonplace (e.g. Mayberry 2016; Potter et al. 2014; Jolly and Cronin 2014) we adopt as a working assumption that, in many regions with volcanic hazards, it is not common practice for volcanic hazard assessments to be ‘contextualised’ by any formal, methodical, documented or auditable process. Secondly, we hypothesise that the pre-communication expectations of decision-makers (i.e. in terms of the timing, scope, depth and format of information provision) are dynamic, neither unique nor universal and therefore, with limited research resources, less would be gained by investigating and attempting to characterise them at a single moment in time.

The risk governance terminology used within this paper is defined in Table 1 of our first paper and, we have deliberately avoided use of the expression “end-users” to describe decision-makers receiving science-based services. Given that we argue in favour of the planned and integrated contextualisation of services throughout the processes of their production and delivery, we see the term “end-users” as inadequate. Alternative expressions, such as “collaborators”, “co-producers” and “partners”, have other weaknesses. We also assume that, in most countries, the usual role of volcanologists is not to lead risk governance efforts, but instead to undertake hazard assessments, to contribute hazard knowledge and, if requested, related advice.

Literature review and survey design/content

A literature review was first undertaken to find historical published evidence of interface challenges.

Secondly, a survey (Additional file 1) targeted a cohort of 33 volcanologists (Additional file 2) with experience of hazard assessments derived from involvement with volcano observatories and/or relevant academic research. A minimum degree of anonymity was guaranteed by ensuring that specific responses could not be linked to identifiable individuals. Each participant has been given an identifying number and further gender, age and nationality details are summarised in Additional file 2.

The aim of the survey was: (1) to capture a post-L’Aquila snapshot of working practices for hazard assessments and related sentiments; and (2) to reflect a range of ages, experience, hazard settings and cultures. We argue that the current practices and sentiments of the interviewees, who included many of the world’s most influential leaders in the science of volcanoes, may indicate whether a more structured approach to contextualisation would find favour amongst volcanologists and, if so, what principles of contextualisation would be most acceptable. It is not suggested that the cohort was either big enough or sufficiently representative to enable assumptions to be made about results that might have been elicited from a bigger, different or more diverse cohort. It is accepted that the data may have an unintended Anglo-Saxon, Anglo-USA and/or Northern Hemisphere bias. In the absence of a larger or different cohort of participants, it is not possible to assess the nature, extent and implications of this possible bias. A future survey of volcanologists practising in, inter alia, Iceland, South-East Asia, Latin and South America, New Zealand, Japan and Africa would provide helpful complementary data.

The questions in the survey were drafted after, and based upon, our literature review, which revealed evidence that the interface is “treacherous” because “strong tensions arise”, and “a neat demarcation between scientific evidence, societal values and beliefs, economic considerations and policy-decisions is not always easy” (OECD 2015, 13). This led in turn to the identification of the main components of hazard assessment and several commonly used value statements. It is not possible to attribute the genesis of each value statement to a definitive source. The sources referred to in Additional file 3 are acknowledged in respect to certain key values.

The main substantive questions considered in this study adopted the same general format, which was:

You will find below a list of possible characteristics of the [methods] of volcanic hazard assessment [processes]. Please indicate the extent to which you consider them to be important for the effective fulfilment of your usual hazard/assessment/risk management role (Additional file 1, questions 24, 27, 30, 36, 39, 46).

The words in square brackets changed, question-by-question, to redirect the focus from say, methods of analysis, to the status of analysis, to the delivery/content/status/perception of communications or to the advice content of communications.

Since one of our goals was to measure the relative importance that hazard assessors attached to several related value statements about a common research dimension (for example values related to hazard communications) the survey used Likert scale questions (Bryman 2012).

For the substantive value statements in Questions 24, 27, 30, 33, 36 and 39 (considered in Tables 4 and 5), a 6-point range was used, the points being Irrelevant (scored − 1), Unsure (scored 0), Relevant – Slightly important (scored 1), Relevant – Important (scored 2), Relevant- Highly important (scored 3) and Critical (scored 4). For the statements involving the forms of advice/guidance that may accompany hazard communications in Question 46 (considered in Table 7), a 5-point range was used, the points being Never provide (scored − 1), Unsure (scored 0), Sometimes provide (scored 1), Routinely provide (scored 2) and Always provide (scored 3). It was not assumed that all the participants perceived the interval between each point in the same way.

A quantitative analysis of the scores was undertaken to produce an approximate ranking of all related value statements. In Tables 4 and 5, the values have been ranked from highest (most important) to lowest (least important) based on the percentage of the maximum score (i.e. number of participants for each question times a maximum score of 4). The working assumption that is derived from the answers is that they give an indication of the relative importance attached to each value statement by the participating cohort of volcanologists. In Table 7, the forms of advice/guidance have been ranked from most common (always) to least common (sometimes) based on the percentage of the maximum score (i.e. number of participants for each question times a maximum score of 3).

For each value/form of advice, a median narrative is also provided. Although cardinal standard deviations were calculated, they are not set out here because it is accepted they are not of any statistical significance given the small number of survey participants.

Interviews

Eighteen one-to-one semi-structured interviews with experienced volcanologists were conducted by the first author over a period of 8 months beginning in July 2014.² Although the principal purpose of each interview was to complete the survey, which was used as an interview agenda to provide structure, the interviewees were encouraged, by both open and closed questions, to talk freely and in greater detail around the issues raised in the survey (Bryman 2012). The interviewees, all of whom spoke fluent English, were handed an Ethical Agreement (Additional file 4) that was signed by the first author. No interview requests were declined, and all interviews were recorded by express consent and later transcribed for detailed qualitative analysis. The interviews averaged 1 h 17 min in duration (shortest 38 min; longest 2 h 6 min).

The data derived from the interviews, other than the scored statements referred to above, were subjected to two forms of qualitative analysis – ‘grounded theory’ (Glaser and Strauss 1967, Strauss and Corbin 1998; Bryman 2012) and ‘thematic analysis’ (Bryman 2012).

An approach based upon grounded theory was adopted because we neither had, nor wished to use, preconceived, ‘fixed’ (i.e. standardised) codes “to label, separate, compile and organise data” (Charmaz 1983, 186). It is accepted that the layout and content of the survey provided a considerable degree of initial coding. The data collection (by means of the survey and interviews) and analysis proceeded in parallel. Evolving conceptual ideas and substantive theoretical arguments were developed out of the data, through an iterative approach. Within the thematic analysis, examples of initial/open coding and selective/focussed coding including axial coding were identified (Strauss and Corbin 1990; Charmaz 2006).

Background – Historical evidence of interface challenges

It is necessary to identify and characterise carefully the foreseeable practical realities faced by scientists engaged at the interface in order to help develop the more structured contextualisation that we advocate as being necessary. A literature review of published work was undertaken to find evidence of challenges faced at the interface and this revealed a rich discourse that includes some general warnings and identifies many difficulties that require further detailed consideration. Commentators, however, have thus far failed to identify common themes, and possible underlying causal trends, that might form the basis for the development of practical solutions that can be developed and tested for utility by further research.

The interface is treacherous (OECD 2015) having been described as the “science-management” interface (Jolly and Cronin 2014) delineating the demarcation between the ‘factual’ world of knowledge and understanding, and the ‘normative’ world of value. The separation of these two worlds, which clearly and inevitably interact within risk governance, is increasingly problematic, a naïve oversimplification, murky³ and somewhat contrived (Horlick-Jones 1998; Stirling 2007; Renn 2008; Donovan and Oppenheimer 2012⁴; OECD 2015). The interface can create stress and friction, and become “critical and contentious” (Aspinall and Cooke 1998; Newhall et al. 1999, 2000; Metzger et al. 1999; Solana et al. 2008; Renn 2008⁵; Macrae 2011; Potter et al. 2014; Pierson et al. 2014; Neuberg 2015).

The role of volcanologists at the interface appears to be poorly defined (being divisive, blurred, and porous and confused) provoking debate involving a wide range of academics and commentators from both general scientific and geo-scientific backgrounds (Peterson 1996; Nowotny 2003; Renn 2008; Ronan et al. 2000; Stirling 2010; Donovan and Oppenheimer 2012, Marzocchi et al. 2012; Donovan and Oppenheimer 2014⁶; Bretton et al. 2015; OECD 2015⁷).

Our review highlights several themes involving roles, uncertainties, changes and difficulties, which we summarise in Table 2.

Table 2 Evidence of the practical realities faced by volcanic hazard assessors at the hazard-risk interface

Demands and pressures upon volcanologists at the interface not only create anxieties and emotional responses, but also influence their behaviour and decisions. On closer investigation of the evidence presented, and the sources listed in Additional file 3, the practical realities can be identified from, and are reflected in, the types of behaviour summarised in Table 3.

Table 3 Behaviour reflecting assessments influenced by factors not strictly confined to, or derived from, evidence-based scientific analysis of volcanic hazards, and detrimental to quality standards of integrity, materiality, comprehensibility and proximity

Although it is not suggested that Table 3 contains a comprehensive list of interface issues, we argue that: (1) it presents a realistic characterisation of the realities of hazard assessments; and (2) it identifies many of the foreseeable practical challenges faced by volcanic hazard assessors. The list can therefore be used to begin the identification and characterisation of some of the complex realities that quality standards for contextualisation may need to consider and attempt to mitigate.

We refer to the ‘equilibrium of contextualisation’ introduced in our first paper and the effect that ‘stakeholder pressures’ have upon it. Adopting Rothstein’s model (Rothstein et al. 2006), we argued that contextualisation is in ‘equilibrium’ when there is ‘balance’ between the influences of quality standards (regulation), natural and operational constraints (reality) and stakeholder pressures (deregulation). Balance is achieved when regulatory sources are effective, quality standards are unequivocal and enforced, within natural and operational constraints stakeholder pressures are addressed, and structured contextualisation preserves scientific methodologies and probity.

The nine items listed in Table 3 represent evidence of actual and potential behaviour consistent with ‘stakeholder’ influences and/or some degree of informal and unstructured contextualisation. Careful characterisation of these items is important, if quality standards to safeguard values attributed to traditional scientific methodologies are going to be proposed. The first eight items in Table 3 may infringe the advocated standard of integrity. Item 9 (communications that do not respond to the needs and expectations of risk-mitigation decisions-makers) may breach Fischhoff’s standards of materiality, comprehensibility and proximity (Fischhoff 2013).

This paper adopts the figure we used in our first paper which is reproduced here as Fig. 1. As illustrated in Fig. 1, the behaviour in Table 3 may create disequilibrium and imbalance, and thereby increase the possibility of a diminution in traditional standards of scientific probity.

Fig. 1

Contextualisation in Equilibrium and Disequilibrium

Key: Examples of possible influences, constraints and pressures are given; Hazard Assessments (HA); Decision-makers (DM); Contextualisation is in ‘equilibrium’ when there is ‘balance’ between the influences of quality standards (regulation), natural and operational constraints (reality) and stakeholder pressures (deregulation). Balance is achieved when regulatory sources are effective, quality standards are unequivocal and enforced, within natural and operational constraints stakeholder pressures are addressed, and structured contextualisation preserves scientific methodologies and probity. Disequilibrium and imbalance may result when regulatory sources are ineffective, quality standards are equivocal or unenforced, within natural and operational constraints stakeholder pressures are not addressed, and ad hoc contextualisation fails to preserve traditional standards of scientific probity (Reproduced from Bretton et al. 2018)

Survey of current practices and sentiments at the interface

For the sake of clarity, the functions of ‘scientific analysis’, ‘scientific communication’ and ‘scope of scientific advice’ are addressed separately under sub-headings and, for the sake of brevity, lengthy verbatim quotes from interviewees are contained within end-notes.

Scientific analysis

Two questions focussed upon the analytical function undertaken by volcanologists when assessing volcanic hazards. One question (question 27) addressed the ‘status’ or ‘nature’ of some of the measurable characteristics of that function; the other (question 24) the ‘methods’. Table 4 sets out the wording of the values used within the relevant questions and the words in bold are adopted hereinafter for ease of reference. This general format is also adopted in Tables 5 and 7.

Table 4 Value statements relating to the analysis of volcanic hazards

Table 5 Values related to the nature, perception, delivery and content of communications

Within this section, more detailed reference will only be made to those values that: (1) are of particular relevance to our consideration of quality standards for contextualisation; or (2) may provide evidence that analytical processes are undergoing changes that may lead to better hazard assessments in the future and additional scope for contextualisation.

The critical role of monitoring data was recorded, and interviewees stated the importance of acquiring historical and benchmarking data over lengthy periods and fully and carefully characterising past events. Several interviewees highlighted the values of multi-disciplinary contributions, systematic analysis and deliberation. In the context of more systematic approaches to analysis, six interviewees (Ints. 3, 4 12, 15, 19, 24) described an emerging debate about: (a) the nature of and the role for subjective judgements/contributions; (b) the utility, if any, of ‘doppelganger’ and ‘analogue’ studies; (c) the transferability of experience between hazards with different magma compositions; (d) the benefits, if any, of gravity, geo-magnetic and ‘real-time’ geo-chemical and petrological data; and (e) the use of bespoke research, computer models, community/local sources of knowledge and global data resources.

A link was established by two interviewees (Ints. 7, 19) between ‘open and transparent’ analytical processes and ‘balanced’, ‘written’ communications capable of building ‘confidence/trust’. To support this view, the rankings for the scores for these four values were very similar and relatively high, being respectively 78, 78, 82 and 81.

Scientific communication

Four questions (questions 30, 33, 36 and 39) focussed upon the nature, perception, delivery and content of communications made by volcanologists to decision-makers. These questions were relevant to all four of our proposed quality standards.

Several interviewees said that the survey’s status values relating to issues of integrity were in reality a list of ‘ideals’. In the words of one (Int. 4), they are “very hard to achieve in practice and [equally] difficult to demonstrate…These are all very high standards to really achieve. I mean these are aspirations.” One interviewee (Int. 19) prefaced their commentary on these characteristics by implying that these were matters upon which the recipients of hazard communications, and not the providers thereof, should comment. All these matters were relevant because each one had to be ‘considered’ by the provider scientists and thereafter discussed with recipient stakeholders. From their own personal experience, “There is very little discussion with the consumer about what they want”. Another (Int. 3) said, “It all depends on the user”. Another respondent (Int. 11) stated that their scores reflected their belief that “We need to keep it simple and clear. Too much detail on some aspects of the information can be a bad thing, as it might introduce confusion.” Table 6 contains a summary of comments about the nature, delivery and perception of communications made by volcanologists to stakeholders insofar as these are relevant to quality standards for contextualisation.

Table 6 A summary of comments about the nature, perception, and delivery of communications made by volcanologists to end-users relevant to quality standards for contextualisation

Scientific communication (advice scope)

One survey question (question 46) investigated the range of advice and guidance provided by volcanologists during periods of emerging volcanic unrest. This question was relevant to, and complements, the historical evidence of a treacherous interface and the possibility that the distinction between advisory and decision-making functions might become blurred. All the values in this question, which are detailed in Table 7, related to our proposed materiality quality standard.

Table 7 The range of advice/guidance provided by volcanologists during periods of emerging volcanic unrest

Interviewee 27 prefaced all answers by stating that the nature and extent of the advice that accompanies hazard communications was “culture dependent”, particularly in relation to risk alert levels. Another (Int. 28) wished to emphasise the importance of long-term monitoring, even if it is only through a basic skeletal system. In addition to providing “absolutely vital” threshold data, such ongoing monitoring may also help to familiarise at-risk communities regarding hazards in a routine way and thus enable them to understand the significance of minor earthquakes and other evidence of possible unrest.

Interviewee 1 did not like ‘status of volcano level’ systems, believing them to be dangerous because spatial, temporal and intensity parameters cannot be pigeon-holed. They did not like knowing that risk levels and risk mitigation strategies might be linked automatically to hazard levels. “I really don’t like [volcano status level] systems because they are implicitly risk level [systems]. The other problem is [that] with volcanoes if you go through their status level, volcanoes often go very quiet just before they explode. They lock up. Seismicity drops, gas drops….” (Int. 1).

Issues of experience and training were raised. One interviewee thought that the provision of hazard mitigation advice depended upon the relative experience levels of the hazard and risk assessors. “I think advice should be offered…even when those who have to implement it may not have thought there is a possibility [of mitigation].” The same interviewee made the same point in relation to risk mitigation advice. “If you have the experience, you should provide it if you can, but the point is, if you don’t do it, I don’t think you should be found wanting…I would steer clear of [the giving of risk mitigation advice] most of the time… because [if you give it] you take responsibility for it.” Interviewee 27 expressed a complementary view saying that all advice must be sound and within the bounds of the adviser’s “mandate and expertise”.

Interviewee 11 volunteered “[m]y scores reflect my training…We work on hazards, not risk. Risk is the domain of other organisations we work with, so we should leave that to the experts…until that becomes something we work on ourselves.” This response directly raised the issue of the existence and nature of the interface and the way in which this interface determines roles and vice versa.

Discussion of current practices and sentiments at the interface

The results presented in the last section are now discussed with reference to our proposed quality standards.

Quality standard - integrity

Many of the results are directly relevant to this proposed standard which goes to the very heart of the way in which the analysis of volcanic hazards is undertaken. Although current methodologies and processes of analysis are often neither structured nor recorded in a formal way (i.e. they are not ‘protocolised’ and remain largely undocumented), there is clearly a high degree of consensus as to how analysis should be undertaken. None of the values carefully selected for inclusion in the survey was ‘irrelevant’ or less than ‘important’, and no alternative or additional values emerged during the survey and interviews despite active prompting (see survey questions 26, 29, 32, 35, 38 and 41).

Survey participants favoured analysis based upon: (1) the monitoring of data for the volcano being assessed; (2) a wide range of scientific disciplines; (3) a collective inter-disciplinary consideration of several expert views; and (4) expertise derived from knowledge of many volcanoes. In the context of an apparently favoured transition to a less ad hoc and more methodical approach to contextualisation, we noted that, unsurprisingly, participants rated as either ‘critical’ or ‘very important’ a systematic and rational, defensible, documented, open and transparent process.

Current practices and sentiments may indicate whether a more structured approach to analysis would find favour amongst volcanologists and, if yes, what principles of contextualisation would be most acceptable. In this regard, survey participants clearly expressed a continuing need to separate hazard analysis (the physical dimension of risk) from risk-related influences (the societal and managerial dimensions). This was evidenced by values that emphasised the need for analysis to be non-political and free from commercial, ideological and religious influences. The demarcation is also illustrated by the apparent reluctance of hazard analysts to offer mitigation advice routinely in respect of risk and hazard mitigation actions, and risk alert levels.

Consistent with this demarcation, the survey also reaffirmed the significance of the ‘traditional’ qualities of scientific independence,⁸ neutrality, freedom from value, balance and objectivity, whilst also rating highly the ‘new’ influences of purpose (i.e. recipient/outcome focus) and objective quality (i.e. good practice). It is noted here that these values were considered by some of the interviewees to be no more than ideals incapable of either demonstration or evaluation. Future research should consider these understandable reservations and investigate whether qualitative norms of ‘good science’ have any utility.

Some values that are relevant to integrity attracted relatively low rankings. There is less support for ‘planned and auditable’, ‘reproducible’, ‘peer-review’ and ‘quality assured/audited’ processes which all had lower quartile rankings. The merits and practicalities of ‘peer review’ generated considerable debate and this may be unsurprising given the almost total absence of academic commentary on its role in the context of hazard assessments.⁹

The ‘traditional’ and ‘new’ values, referred to above, may represent a basis for: (1) the quality assurance steps scientists must take “to ensure the reproducibility of their analysis and the quality of their advice” (OECD 2015, 21); and (2) the rules of engagement that provide the “foundation on which scientists and risk decision-makers should base their operations and interactions”, as advocated by the UK’s Government Office for Science (UK/GOS 2011, 32); and the “rigorous, replicable methods” of analysis referred to in the 1996 US/NRC report (US/NRC 1996).¹⁰

Quality standard - materiality

There are new data to support the conclusion that we reached in our first paper. We concluded that there is a slow-moving paradigm shift away from a traditional linear one-way educational approach to communication towards a more iterative two-way ‘outcome-focussed’ dialogue with user decision-makers. The rankings that support this are the lowest possible ranking for ‘provider centric’ and upper quartile rankings for ‘recipient centric’ and ‘outcome/output centric’.

There was a clear rejection of the ‘provider centric’ traits that are inherent features of hazard assessments that initiate traditional linear models of risk management. Whilst describing the need for hazard assessments to be ‘recipient-centric’, several of the interviewees described the hallmarks of contextualisation without actually using that word; i.e. they acknowledged the existence of the managerial dimension of risk governance and a need to consider the requirements of decision-makers.

Another theme, which is related to and flows from a shift towards some degree of contextualisation, is the evolving acceptance of the value of, and the need for, ‘deliberation’ during hazard assessments. This is evidenced by:

• the upper quartile rankings for ‘deliberation’, ‘recipient-centric’ and ‘outcome-centric’ communications;

• the middle quartile rankings of values expressly or implicitly supporting or encouraging a two-way dialogue with decision-makers such as openness and transparency;

• the ranking of ‘clarity’ above ‘precision’ for statements of likelihood (Score 82%, Ranking of 15/42 against 48%, 41/42) and for statements of scientific confidence (72%, 29/42 against 53%, 38/42); and

• the importance attached by some interviewees to understanding commercial, religious and ideological interests whilst simultaneously maintaining their objective independence from them.

Quality standard – Comprehensibility

The choice of narrative and the use of numerical and graphical content in order to make communications understandable remains a significant challenge but there seems to be agreement about the prioritisation of issues for further discussion and research. Providing analytical ‘clarity’ to users is more important than analytical ‘precision’ as evidenced by the respective scores for likelihood clarity (82%) and likelihood precision (48%). Comparable results were obtained for expressions of confidence (variability due to epistemic/aleatory uncertainties) for which the comparable scores were clarity (72%) and precision (53%).

There remains an unresolved debate about the use of narratives to support numerical expressions of likelihood and confidence as reflected in the lowest quartile rankings for the use of qualitative as well as quantitative terms for expressions of hazard ‘likelihood’ and scientific ‘confidence’. These two values produced some of the largest spreads of opinion and this is consistent with an ongoing discourse that has been joined by commentators such as Cooke (2015) who note the near-impossibility of devising meaningful qualitative narratives to describe numerical probabilities.

Quality standard - proximity

Practitioners at the interface know, and already attach significant importance to, the many factors that contribute to effective provider-centric communications. When the value statements for ‘scientific analysis’ and ‘communication’ are ranked together, 11 out of the top 15 related to communication.

The participants ranked highly communications “delivered from one source to avoid mixed scientific messages” and thereby difficulties such as those experienced in Guadeloupe in 1976 (Fiske 1984). This result is consistent with the generally accepted view that authoritative scientific information and advice should be sourced through a coordinating person or body (Fiske 1984; Newhall and Punongbayan 1996; Lipshitz et al. 2001; Driedger et al. 2008; NZ/MCDEM 2008; Barclay et al. 2008; Frenzen and Matarrese 2008; OECD 2015). Doyle et al. (2015) note that this approach also facilitates the integration of the wide range of expert opinions required to manage uncertainty during decision-making and helps combat issues arising from any conflicts between scientists.

Recommendations

Clearer goals for hazard assessments at the interface

The interface role of volcanologists is either changing or, at least, is likely to change. Our literature review revealed that an increasingly wide range of services is sought from volcanologists. Consistent with this, and based on new data presented in Table 6, we have concluded that bare characterisations of the temporal, spatial and physical parameters of volcanic hazard scenarios are ‘often’ accompanied by advice, and that advice ‘sometimes’ includes advice about hazard and risk mitigation issues.

These changes are potentially significant and have foreseeable consequences (Weinberg 1972, 1992; Nowotny 2003; Van Nuffelen 2004). Volcanologists are exposed to new managerial hazards and are vulnerable to managerial risks, when they stray from their traditional roles and practices (e.g. providing lahar hazard zone maps) into an extended role (e.g. providing risk-related advice such as risk alert levels based upon lahar hazard zones). The ‘exposure’ comes from volcanologists extending their scientific advisory roles; the ‘vulnerability’ is derived from any lack of competence, time or resources to fulfil those extended and more complex roles (OECD 2015; Bretton et al. 2015; Bretton and Aspinall 2017).

With a view to assessing management risk exposures and vulnerabilities, a starting point is to define ‘scientific analysis’ by reference to the relevant qualities of the reasonably competent volcanologist engaged to undertake monitoring and analysis. These qualities include education, training, expertise, experience, skills and knowledge. This definition is adopted to characterise the usual role and thereby the usual exposures and vulnerabilities of volcanologists. It is argued here that caution must be exercised when scientists either volunteer or are required to adopt an extended role that includes making judgements that are not based upon the ‘scientific analysis’ of monitoring data, without acquiring relevant extended competencies in advance. The dangers of role extensions and poor communication were well illustrated by Simoncini (2014) in the context of the L’Aquila trial.¹¹

In our first paper, we investigated the discourse that argues that contextualisation undertaken by scientists must be constrained within certain boundaries. To address an absence of guidance for managing these boundaries, we advocated a more structured approach based upon open, transparent and fully articulated quality standards of materiality, proximity, comprehensibility and integrity. To convey the purpose of a more structured approach, we here recommend the setting of three overriding interface goals.

Building upon the terminology of the US/NRC (1996) and Nowotny (2003), these interface goals might be:

• To identify, articulate and preserve the essential values and methodologies of ‘reliable’ or ‘good’ science - Getting assessments right, by preserving ‘good science’.

• To seek out the requirements of decision-makers in a proactive iterative way so as to help ensure hazard assessments are relevant to, and fit for, their needs and expectations – Identifying which assessments will have greatest validity and utility, by means of deliberation.

• To ensure hazard analyses are not only right (i.e. reliable or good) but also socially robust (i.e. valid and relevant) – Getting the right assessments, by means of informed contextualisation.

We argue that there is neither a necessary nor an insurmountable conflict between these three goals if the principles of hazard assessment at the interface are fully articulated with an elevated level of transparency and are continually tested as advocated by Nowotny (2003) and the OECD (2015).

A structured, fully articulated and integrated role for deliberation

Deliberation has a pivotal role as the precursor to, and driver of, contextualisation.¹² For deliberation to be meaningful, we argue that decision-makers must:

• have an enforceable right to seek and receive such information and training as is necessary to enable to them participate fully and effectively, and

• be given every opportunity to identify and make known: (a) the essential characteristics of scientific analysis (i.e. scientific methodology) that they value; and (b) their wider risk-related hazard assessment requirements.

There must also be a “structured requirement that others listen” Grabill and Simmons (1998, 427).

Deliberation will drive not only the ‘creation’ and ‘evolution’ of the negotiated quality standards required to regulate contextualisation and enhance its utility, but also ‘evaluations’ of those standards. In the future, deliberation will be a more integral part of scientific analysis (a tail that wags the analytical dog) and not just a social science afterthought (a wagging flexible tail grafted onto the end of an inflexible linear process of analysis).

Although we fully accept that effective dialogue may be very difficult to achieve in practice, the very process of deliberation itself, which would be described by some commentators as a process of ‘co-production’ (e.g. Scolobig et al. 2014), may contribute to: (1) a shift of emphasis from ‘individual’ to ‘collective’ perspectives (Doyle et al. 2015¹³); (2) the building of trust between scientists and decision-makers; and thereby (3) better risk-mitigation outcomes (Hincks et al. 2014; Fischhoff 2013).

We are not attempting to supplement the established discourse describing the difficulties of undertaking volcanic hazard assessments. Rather, we are advocating new initiatives that encourage the trial of practical solutions for those difficulties based on structured deliberation with decision-makers. Four sub-questions within question 70 of our survey asked the participants to state whether, within the volcanic settings best known to them, they saw any evidence of a trend in recent years towards more public participation in hazard monitoring and assessment and risk assessment, management and mitigation. Whilst participants recorded little public participation in providing knowledge for hazard assessments (only 2 participants out of 24) and making risk assessments (2/24), more participation but still very low participation was noted in respect of hazard monitoring (5/24) and risk management including the selection of mitigation strategies and measures (6/24). Accordingly, deliberation may have to be given a higher priority before it fulfils its potential within volcanic risk governance.

More structured contextualisation

We argue that it is possible to begin to formulate some general principles that might lead to hazard assessments with greater utility. Fine-tuning methodologies of contextualisation is both unnecessary, and inappropriate for three main reasons.

First, at a location-specific level, acceptable contextualisation practices should always be the product of structured deliberation involving relevant decision-makers. Secondly, acceptable contextualisation practices will reflect a very wide range of geological, geographical, technical, historical, governmental (at multiple levels), cultural, spiritual, linguistic, legal, political, economic, time and resource drivers (management drivers) (Bretton et al. 2015). Thirdly, scientists should consider carefully, and have appropriate regard to, the human rights of individuals, and the democratic and legal mandates, if any, of civil protection authorities, to take risk decisions based upon a wide range of factors (UK/GOS 2011). Individuals will have a variety of reasons for living with volcanic risks. Scientists should accept that a more informed understanding of volcanic hazards, to which they will make an essential evidenced-based contribution, will represent only part of the knowledge upon which difficult risk decisions will be made. Those decisions may address risks from multiple natural and human-made hazards, and reflect “structural constraints rooted in difficulty in accessing resources, historical and cultural heritage and political-economy” (Gaillard 2008, 325).¹⁴

It follows that acceptable practices (supported by evidence-based regional guidance) in one volcanic region may differ markedly from the acceptable practices in other regions. We argue, however, that all principles, values and methodologies that are widely recognised as critical hallmarks of ‘good science’, should be accepted as paramount and non-negotiable quality standards (i.e. a constant) in every region.

Thirdly, contextualisation is inherently ‘dynamic’ because volcanic hazards and risks (and related management drivers) emerge and change over a wide array of time-scales ranging from minutes and hours during periods of unrest; to weeks, months or even years during cycles of risk governance.

In Table 8, we present several general principles, firstly for further consideration and development at a location-specific level, and, secondly as a reasoned starting point for future empirical research, particularly in Iceland, South-East Asia, Latin and South America, New Zealand, Japan and Africa. These important volcanic regions will have multiple management drivers that may not be adequately addressed by our theoretical choices and the new, but limited, survey and interview data presented in this paper. Any principles relating to integrity should strive to mitigate the first eight characterisations of behaviour listed in Table 3.

Table 8 Prototype Code of Practice and guidance for volcanic hazard assessments

We argue that a more structured, but carefully nuanced, approach to contextualisation reflecting location-specific management drivers will:

• Enhance the governance of societal risks (i.e. promote “getting better” Bretton et al. 2015, 17–18), by giving much needed clarity to:

  • ◦ the treacherous, increasingly problematic and murky hazard/risk interface; and

  • ◦ the divisive, blurred, porous and confused role of interface volcanologists.

• Enhance the governance of managerial risks (i.e. promote “getting smarter” Bretton et al. 2015, 19), by identifying managerial hazards at the interface, and mitigating avoidable managerial risks:

  • ◦ by structured delineation of the role of interface volcanologists;

  • ◦ by structured consideration of all necessary role competencies (including education, training, expertise, experience, skills and knowledge);

  • ◦ by careful recruitment and selection to meet those competencies

  • ◦ by structured identification of all competency shortfalls; and

  • ◦ by structured mitigation of those shortfalls by the provision of suitable and sufficient: (a) information, education, supervision, training, coaching; and (b) time and other resources.

All stakeholders at the interface (and not just volcanologists) should have clear roles, responsibilities, resources and rights to guide their actions and interactions (Simoncini 2014; UN/ISDR 2015; Bretton et al. 2015; Scolobig 2015), and these should be driven by and reflect the stakeholder-negotiated contextualisation practices to which reference has already been made. Stakeholders should also value and accept the principles they have negotiated so that no stakeholder is tempted or pressurised to bend or ignore them, or the legal mandates upon which they are based.

By arguing the merits of contextualised hazard assessments, we must consider all foreseeable consequences. Volcanologists may be exposed to new managerial hazards and will be vulnerable to managerial risks if they are not competent or lack either the time or suitable resources to fulfil more extensive and complex roles. In the absence of both careful planning and clear agreed governance roles at the interface, changed roles may result in: (1) an unstructured blurring of the boundaries between the physical, societal and managerial dimensions of risk governance; (2) the interface challenges and resulting behaviour detailed in Tables 2 and 3; (3) contextualisation that is in disequilibrium and unbalanced, and fails to preserve traditional standards of good science; and (4) foreseeable legal and other consequences (Scolobig 2015; Bretton et al. 2015).

Quality regulation is intended to support the three interface goals, referred to above, whilst actively mitigating the managerial risks posed to volcanologists by role changes. In our first paper we referred to six possible sources of quality regulation, namely: external, collective or self, employer, personal, expert and negotiated. Our proposed quality standards can be used to discuss and develop stakeholder-negotiated regulation norms and, over longer periods, may influence the other five sources of regulation. By enhancing the clarity of relevant and effective quality standards and reducing the unconstrained influences of stakeholders, they are intended to produce contextualisations that are more likely to be in equilibrium and balanced as illustrated in Fig. 1.

Conclusions

In the absence of relevant quality standards, the contextualisation of hazard assessments is likely to be methodologically diverse and determined by contingent pressures. An unstructured approach may not lead to acceptable standards of societal risk governance, and may expose volcanologists to managerial hazards, leaving them vulnerable to the legal and other consequences of managerial risks.

We do not advocate acceptance of this status quo but present for consideration four reasoned quality standards (standards of materiality, proximity, comprehensibility and integrity as described earlier), and some general principles. They: (1) are framed by the theoretical choices in our first paper; (2) are focussed upon this paper’s characterisations of the existing challenges and resulting behaviour based upon published literature; and (3) reflect current practitioner sentiments and practices evident in new empirical data.

We introduce the concept of the equilibrium of contextualisation in order to illustrate the likely effect of unconstrained stakeholder pressures and risk influences upon the traditional role of volcanologists and the hallmarks of long-established scientific methodologies. Quality standards for contextualisation offer the possibility of hazard assessments having greater validity and utility, when measured by reference to the sentiments and actions of their users. They will also mitigate the managerial risks faced by volcanologists undertaking more complex roles in the future.