Risk Assessment for Subjective Evidence-based Ethnography Applied in High Risk Environment: Improved Protocol

Subjective Evidence-Based Ethnography (SEBE) is a family of methods developed for investigation in social science based on subjective audio-video recordings with a miniature video-camera usually worn at eye-level (eye-tracking techniques are included). Facing a lack of tools for SEBE risk assessment when applied to high risk professional environments (e.g. anesthetists, aircraft pilots, Method Article Fauquet-Alekhine et al.; AIR, 16(3): 1-15, 2018; Article no.AIR.43259 2 nuclear reactor pilots), a protocol (version 1.1) was successfully developed and tested in nuclear industry with N1=59 participants and presented in a previous article. However, further cases were needed to demonstrate the robustness of the risk assessment protocol in other contexts. Further applications were thus undertaken with N2=75 participants from Air Force army, Police, Medicine and Nuclear industry during work activities lasting from 10 minutes to several hours. SEBE equipment was worn and the original risk assessment protocol was applied and/or discussed between participants and researchers for improvement. The protocol was enriched (version 2.3): 37% items were added. This illustrated the context sensitiveness of this sort of risk assessment. Limits of this new series of tests are discussed.


INTRODUCTION
Using video recordings allows the researcher to access to the reality of work activities which is one of the major concerns of work analysts, permitting multiple visualisations retrospectively, very useful in the case of complex situations. Within the paradigm of Cognitive Task Analysis [1][2], using video recording as a tool for postanalysis of activities is referred to as "process tracing". It helps the work analyst involving participants in a reflexive analysis of their activity, learning about themselves in action and thus improving their professional practices if need be. The video is a data source and a support of expression (body, speech), of mediation for the analysis [3].
One could almost say that the use of video is a necessity because the principle of cognitive economy puts participants in a limited attention and consciousness span that makes it difficult afterwards to recall events from memory only [4]. Video recording gives thus an objective reporting of what happened for an exhaustive recall.
Amongst all the possible devices available for process tracing, the first person approach, or subjective approach, uses a recording device (miniature video-camera most of the time worn at eye-level or "subcam" [5]). This kind of process tracing, conceptualised by Lahlou [6][7] under the name of Subjective Evidence-Based Ethnography (SEBE), integrates a confrontation of participants with these subjective recordings in order to undertake a reflexive analysis of the activity. The use of SEBE methods brought an interesting series of improvements on the quality of activity analyses [8][9].
With the recent progress regarding miniaturised cameras and camcorders, researchers have developed SEBE applications. For example, the consumers' behavior analysis through subjective recordings was obtained without the usual disturbance due to heavy and bulky equipment [10][11]. In marketing, Fauquet-Alekhine et al. [12][13] analysed consumers' behavior shopping for wines. Gobbo [14] applied the SEBE approach to shopping for shoes (videos are available on line: ethnoshoes.com). Occupational day life was adjusted after applying SEBE analysis: examples of application are available for nuclear industry [15][16][17] or for students' day at work [18].
If the use of SEBE equipment does not present any special risks for the participants who wear the subcam themselves, conversely, it might induce problems due to the interaction between the SEBE equipment and the work environment for example (e.g. cables may be trapped in the industrial equipment) or due to a disturbance of participants' actions (e.g. SEBE glasses might change the participants' vision). A solution might appear to withdraw cables and use a WiFi system for example; this is just transferring the issue to another domain, this of electromagnetic interference between the WiFi equipment and the control-command system of the industrial process, of the medical environment or of the cockpit. Control-command systems usually require avoiding this kind of interferences; this leads the choice to wire-based SEBE equipment. Despite these potential additional risks induced by SEBE equipment, before our work [34], the literature lacked a protocol for SEBE risk assessment in high-risk environment.
Our previous article [34] presented a SEBE risk assessment protocol version 1.1 developed and evaluated for work activities of participants (N 1 =59) on full-scale simulators and real operating situations in the field of nuclear reactor operations. However, it pointed out two main limits: i) only one industrial field had been explored and ii) no particular biotechnical constraint was met except wearing glasses. To improve the robustness of the protocol, it was estimated necessary to push these limits. The aim of the present article is to present what was undertaken and obtained in this perspective, resulting in the version 2.3 of the protocol.
Seven professions were observed: engineers, operations and maintenance professions (pilot and technician) on nuclear power plant, physicians in hospital, policemen and Air Force pilots.
In order to understand better what is studied here, readers are suggested to read the previous study in [34] beforehand.

Design
In the previous study [34], the elaboration of the SEBE-risk assessment protocol version 1.1 consisted in three phases. Phase 1 was observations of activities of workers equipped with SEBE metrology followed by interviews in order to elaborate a draft for the SEBE-risk assessment protocol. The activities were mainly performed on full-scale simulators (simulated situations or SimS) due to safety concerns. Phase 2 was the elaboration of the protocol based on results from phase 1 and a bibliographic review (version 1.0). Phase 3 was a test-application of the protocol in real operating situations (ROS) before performing the activity followed by a semi-structured interview of workers to adjust the protocol if necessary (version 1.1).
The present study addressed work activities of different professions while the first study focused on nuclear professions on a French nuclear power plant. Three methods were applied.
The first method applied was equivalent to that of phase 3 in the previous study, hereafter called "Method with application": participants were equipped with SEBE metrology and the SEBE-risk assessment protocol version 1.1 was applied. This was undertaken for 4 professions on a French nuclear power plant and 1 medical profession in a French hospital (see Table 1 listing professions, methods used and characteristics of participants). All cases were real operating situations (or ROS) except for physicians: two work situations taking place in the operating theater were not authorised in ROS and were performed in simulated situations (or SimS).
The second method was based on post-analysis of SimS: tests being undertaken in the framework of other research programs, it was not possible to apply the risk assessment protocol before the work activity. Therefore, SimS were followed by a semi-structured interview on the basis of the existing protocol version 1.1. Two other professions were concerned by such limits: policemen involved in arresting a suspect in a public space and Air Force pilots in training flights on Cirrus or Alpha jet or A400M. The method was named "Method with post-analysis only".
The third method was applied to one case only: an engineer with audio disability was met in his office on the nuclear power plant. This case aimed at refining the robustness of the protocol for participants with hearing disabilities. This meeting was necessary as no such case was met during investigations in SimS or ROS. In addition, to avoid a bias due to the type of prostheses he wore, an audiologist was met: possible issues for a large variety of equipment were discussed, especially regarding electromagnetic interferences between the SEBE equipment and the prostheses. This was called "Method with analysed interview only".
During the interviews with participants, several areas were systematically explored resulting from the structure that was elaborated for the SEBE-risk assessment protocol following the previous study:  One additional area was explored, resulting from applications and from discussion with researchers met in different seminars or conferences whilst presenting SEBE methods:  Ergonomics of the SEBE-risk assessment protocol form.
As the aim of the present study was to improve a SEBE risk assessment version 1.1 for any member of a staff, gender, age and experience were not considered as variables to be analysed. However, subjects were chosen so that a large range of age and work experience could be represented by the sample.

Apparatus
Participants were dressed with their o garments, including professional safety equipment if needed. The SEBE equipment fulfilled the requirements of video quality, energy autonomy, data storage, size and industrial environment disturbance.

SEBE equipment used with NPP workers (Fig. 1)
The SEBE equipment was made up of three parts linked with cables: i) a micro audio digital recorder DVR-500-HD2 self-powered by internal batteries, touch screen, dimensions 80x52x22 mm, ii) a 12x12x8 mm camera (subcam mounted on safety glasses, iii) a lavaliere microphone. This SEBE equipment was assembled from components produced at Active Media Concept (website: www.amc The main advantage of this equipment was to be adaptable to any kind of glasses (safety or vision).

SEBE equipment used with Norwegian Policemen (Fig. 2)
The SEBE equipment was made up of two parts linked with one short cable: i) a 7g and 43x14x11 mm miniature wide-angle video camera with a stereo microphone mounted on a pair of glasses, and ii) a digital recorder composed of two press buttons (power and record), self internal batteries, dimensions 65x49x17 mm. 5 One additional area was explored, resulting from from discussion with researchers met in different seminars or conferences whilst presenting SEBE methods: risk assessment As the aim of the present study was to improve a SEBE risk assessment version 1.1 for any er of a staff, gender, age and experience were not considered as variables to be analysed. However, subjects were chosen so that a large range of age and work experience could be Participants were dressed with their own garments, including professional safety equipment if needed. The SEBE equipment fulfilled the requirements of video quality, energy autonomy, data storage, size and industrial SEBE equipment used with NPP SEBE equipment was made up of three parts linked with cables: i) a micro audio digital powered by internal batteries, touch screen, dimensions 80x52x22 mm, ii) a 12x12x8 mm camera (subcam) , iii) a lavaliere microphone. This SEBE equipment was assembled from components produced at Active Media Concept (website: www.amc-tec.com). The main advantage of this equipment was to be adaptable to any kind of glasses (safety or

equipment used with Norwegian
The SEBE equipment was made up of two parts linked with one short cable: i) a 7g and 43x14x11 angle video camera with a stereo microphone mounted on a pair of glasses, corder composed of two press buttons (power and record), self-powered by internal batteries, dimensions 65x49x17 mm. This SEBE equipment was produced by the Lab, Department of Psychological & Behavioural Science, London mics and Political Science (UK). This subcam equipment can be worn at eye level on a pair of glasses or any other apparatus adapted to the activity. The angle of the camera is wide enough to capture both hands interactions and faces of people.

SEBE equipment used with French Air
Force pilots (Fig. 3) The SEBE equipment was eye-tracking system: TOBII glasses 2 with a 160-degree lens for frontal camera (1920 x 1080 pixels), 4 eye camera, integrated microphone, external battery (130 x 85 x 27 mm). The tracking technique corneal reflection, binocular, dark pupil tracking.

Participants
All participants (N=75) were volunteers and signed an informed consent before using SEBE equipment. The distribution of the participants per experiment fields, their mean age and professional experience are given in Table 1.
Nuclear professionals worked at Chinon nuclear power plant (France); they were pilo of operating a nuclear reactor in a control room (Fig. 4a) and technicians (handling equipment in the field, for example in the machine room; 4b). Maintenance professionals on nuclear power plant were technicians or preparers in charge o testing or repairing the equipment. Further information may be found in [17].
Physicians were anesthetists involved in a radial puncture (used to provide a sample of blood from All participants (N=75) were volunteers and informed consent before using SEBE equipment. The distribution of the participants per experiment fields, their mean age and professional experience are given in Table 1.
Nuclear professionals worked at Chinon nuclear power plant (France); they were pilots (in charge of operating a nuclear reactor in a control room (Fig. 4a) and technicians (handling equipment in or example in the machine room; Fig.  4b). Maintenance professionals on nuclear power plant were technicians or preparers in charge of testing or repairing the equipment. Further Physicians were anesthetists involved in a radial puncture (used to provide a sample of blood from the patient's artery for the blood gas measurements (Fig. 5a) or in distres resuscitation in operating theater (Fig. 5b) at the university hospital of Angers, France.
Policemen were met at the Norwegian Police University College (PHS, Norway). They were involved in an outdoor intervention in a public space (Fig. 6). More information may be found in [35][36].
Air Force pilots (French army) were interviewed after training on school planes, fighter jets or transport planes (Fig. 7).
The nuclear engineer with audio disability was met in his office at Chinon nuclear powe (France). His job did not often require him to be in the field and the opportunity to observe him in this situation was not met. the patient's artery for the blood gas measurements (Fig. 5a) or in distressed patient's resuscitation in operating theater (Fig. 5b) at the university hospital of Angers, France.
Policemen were met at the Norwegian Police University College (PHS, Norway). They were involved in an outdoor intervention in a public More information may be found in Air Force pilots (French army) were interviewed after training on school planes, fighter jets or The nuclear engineer with audio disability was met in his office at Chinon nuclear power plant (France). His job did not often require him to be in the field and the opportunity to observe him in

RESULTS
The results of the previous study [34] led to a questionnaire version 1.1 for risk assessment divided in 5 categories as follows:

1-Usual biotechnical constraints
1.1-Do you wear a hearing aid? The results of the previous study [34] led to a questionnaire version 1.1 for risk assessment

Fig. 7. Types of planes used by French Air Force for training: a) Cirrus, b) Alpha jet, c) A400M
If Yes to any of the questions, is this resulting in particular regular manipulations?

Biotechnical constraints of the activity
Do you wear equipment that may interact with the SEBE equipment? (e.g. belt metrology,

5-Induced biotechnical constraints (once SEBE metrology in place)
5.1-Do you feel a particular discomfort for: The field of vision? 5.2-Do you feel a particular discomfort for: Listening? 5.3-Do you feel a particular discomfort for: The weight of the glasses? 5.4-Do you feel a particular discomfort for: The placement of the camcorder? 5.5-Do you feel a particular discomfort for: The placement of cables? 5.6-Do you feel a particular discomfort for: The length of the cables?

Results Regarding the Content of the Questionnaire
The present study led to add the following questions or comments when taking into account the participants' feedback or remarks:

1-Usual biotechnical constraints
1.5-Might there be any possible discomfort due to the camcorder vibrations?

2-Biotechnical constraints of the activity
Added comments in 2.1: "e.g. audio headset, protective visor"

3-Performance constraints
Added comments in 3.3: "need a strap to prevent falling" 3.4-If SEBE metrology must be set up not before but during the activity, can it have an impact on your activity?

4-Equipment safety
Added comments in 4.1: "e.g. mechanical chock, water projection or rain, equipment falling down when getting out of a vehicle -need a strap to prevent from falling"

5-Induced biotechnical constraints (once SEBE metrology in place)
5.4-Do you feel a particular discomfort for: The stems of the glasses?
5.5-Do you feel a particular discomfort for: The external battery (if any)?
5.6-Might you feel any pain after a lapse of time due to SEBE metrology? (e.g. a helmet or headset pressing stems of glasses) 5.7-Is there any risk of being throttled by the cables?

Usual biotechnical constraints (including concerns about individual's safety and comfort)
 Question 1.5 was added to assess the possible discomfort due to the camcorder vibrations: when the camcorder used at Chinon NPP (Fig. 1) has recorded a file up to about 1Go, recording begins on another file and the camcorder is vibrating for a few seconds. (suggested by 1 Air Force pilot)

Biotechnical constraints of the activity
 For question 2.1, the existing comment was complemented after remarks from 2 Air Force pilots.

Performance constraints
 Comment in 3.3 "need a strap to prevent falling" was added because glasses were identified to possibly fall into an unexpected place such as the reactor tank for nuclear workers or to fall down on the ground and being destroyed when a policeman gets quickly out of a car.  Question 3.4 was added because an Air Force pilot reported the impossibility to undertake the capture of an activity due to too long implementation of the metrology; it was implemented during the flight, that is during the activity, not before beginning the activity conversely to others professions.

Equipment safety  For question 4.1, a comment was added to
give examples of how the SEBE equipment might be damaged. (suggested by 5 police officers and 3 workers at the NPP)  This comment for question 4.1 is supposed to lead the analyst to suggest a strap to the participants in order to maintain the glasses in place, useful if participants have to bend over water or to make a sudden physical effort for example.

 Question 5.4 was added because 2 Air
Force pilots reported having been hurt by the stems of the eye-tracking glasses due to the pressure of their headset.  Question 5.5 was added because 2 Air Force pilots reported needing an external battery plugged to their eye-tracker even though it was basically a wireless system.  In the line of question 5.4, it was found relevant to identify a possible forthcoming pain through question 5.6 as an Air Force pilot warned about the cognitive load induced by pain, a possible source of biotechnical constraint resulting in a decrease in performance. This question was numbered 5.6 in order to be asked in case of wireless SEBE metrology: in this case, questions after #5.6 are not asked.  Question 5.7 was added to assess the risk of being throttled by the cables. (suggested by 1 worker at the NPP) Overall, application of methods with users' feedback helped us to improve the 16-question protocol (version 1.1) by adding 6 questions and 3 comments. No question or comment was found to be withdrawn.

Ergonomic Analysis of the of the SEBE-risk Assessment Protocol Form
Applying the protocol version 1.1 showed several areas for improvement for easier use from an ergonomic standpoint.
The introduction sheet presented text boxes to be filled in by the analyst which were too small. These boxes aimed at collecting information regarding the context of work and conclusions of the risk assessment. Applications showed that more space was needed for each box, especially when several people participated in the risk assessment or in the activity performance, or when the activity description could not be written in a few words. Boxes were thus enlarged. One box was also added to write references related to participants and to the experiment or the analysis in case of need.
Just over this table, a reminder in version 1.1 was written for the participants not to forget that the priority was to achieve successfully their activity, not to wear the SEBE equipment. The repetitive use of the protocol showed that this might be forgotten: a box to tick was added at the beginning of the reminder and this was moved after the table for the analyst to remind this to the participants at the end of the protocol rather than at the beginning.
On the introduction sheet, the introduction was adjusted to take into account the results of the present study and in the section "How to use the SEBE risk assessment protocol"; a warning was added for the analyst. Indeed, when putting on the SEBE equipment, participants tend to naturally put cables under the vest or tee-shirt. Then, whilst performing the risk assessment, when asking if there was any problem with cables, participants said "no" and the analyst could be not conscious that "putting the cables under the clothes" was a remedial to the cable disturbance to be taken into account.
For each possible issue investigated through a question of the risk assessment, two perspectives were explored: safety and technical. Two tables were associated with these assessments (Figs. 3 and 4 in [34]), both related to one risk-question of the protocol and each printed on a single page in version 1.1. It was found better to put these two tables on the same page with the related risk-question printed only once at the top of the page. The section "How to use the SEBE risk assessment protocol" was subsequently adapted.
In the section "How to use the SEBE risk assessment protocol", a statement was added: "in case of a participant's hesitation when answering a question, if the answer is 'perhaps' or 'possible', consider it as a 'yes'".

SEBE Risk Assessment and Hearing Disability
The engineer with hearing disability met in his office on the nuclear power plant was presented with the SEBE metrology used at Chinon NPP ( Fig. 1). He was equipped with the subcam stuck on the stem of his glasses, the microphone attached on the collar of his shirt and the camcorder worn in the bag fastened on his trousers belt. The recorder was launched before the risk assessment protocol begun: this was done to know whether the metrology would create any interference with the hearing aid during the interview.
The interview based on the application of the SEBE risk assessment protocol version 1.1 and the following discussion did not yield any additional question or comment.
In addition, the audiologist met was presented with the SEBE metrology used at Chinon NPP ( Fig. 1). He explained that the only component of hearing aids capable of electromagnetic interferences with the SEBE equipment was the solenoids. However, the electric intensity inside the cables of the SEBE metrology was so low that electromagnetic interferences would not occur, whatever the brand or the type of hearing aid considered.

Application of the SEBE Risk Assessment
As for the previous study, applying the SEBE risk assessment protocol was indeed easy and quick. Most of the answers to the questions were negative and the protocol was applied in about five minutes.
No case led to withdrawing the SEBE metrology equipment whilst performing a work activity.
No incident or accident was observed or reported whilst performing the work activities.

Qualitative Aspect
The protocol was perceived by participants as a useful tool. Nevertheless, for all professions investigated, none of the participants had heard about the use of such a tool before the present study. This came to confirm the finding mentioned in section "Introduction" regarding the absence of existing protocol for the assessment of risks induced by SEBE metrology when used in high-risk environment, especially when a wireless solution is not possible due to potential interactions with control-command systems. We actually filled this gap with this study.
Regarding the experimenters' standpoint, a potential bias in the risk assessment was identified to be borne in mind. As for any activity, applying the SEBE-risk assessment protocol is sensitive to repetitiveness and may lead to unconscious misuse. For example, the fact that participants put systematically and naturally the cables under their garments when equipped with the SEBE metrology leads them to answer questions #3.3, #5.5 or #5.6 with a "no", meaning "no issue identified with cables" since they do not perceive any more issue with cables. In this case, experimenters may forget to verify that the answer is actually "yes" despite the fact that participants answered "no". The answer "no" is false when considered in the context of the protocol: indeed, there is no issue with cables because cables are under the vest. It means that "issue with cables" must be considered in the risk assessment and "put cables under vest" must be noted as a remedial and reported in the concluding tables at the end of the questionnaire.
The fact that no problem was encountered whilst using the SEBE equipment with prior risk assessment in real operating situations is encouraging: it suggests that the developed protocol for SEBE risk assessment is a relevant tool.
However, experimenters may tackle another sort of issue: we must here remind a warning resulting from the previous study [34: 10]. One case of discomfort was reported during an interview after performing the activity. The participant was a reactor pilot. However, observations led to the assumption that this person was using any reason to justify his difficulties in achieving the tasks (lack of competencies). Due to ethical concerns, this point could not be discussed neither with his managers nor with his colleagues for confirmation or not. This highlighted a very important point: if an individual may attempt to hide a kind of lack of competencies by invoking the effect of the SEBE equipment, we may assume that, in case of accident occurring in situation, the SEBE equipment might be designated by the participants as a main factor contributing to the accident even though it would not be really the case. This finding gives even greater importance to the necessity of this sort of risk assessment protocol. Indeed, in case of the occurrence of an accident whilst using the SEBE equipment with risk assessment beforehand, it gives arguments to support the absence of contribution of the SEBE equipment to the accident. Obviously, this does not prevent the workers to carry out also their usual risk analysis of their work activity.
This protocol may be applied to any kind of SEBE, including wireless devices or systems for which the camcorder and/or the microphone are integrated inside the glasses: in these cases, the related questions are merely not applicable: the protocol is applied until question #5.6 in case of wireless device. Application might seems heavy at a first glance, especially to experimenters using risk assessment protocol for the first time. To help them, a tutorial video has been upload on http://hayka-kultura.org/larsen.html; the 7-minute video (86Mo) can be downloaded for free. In the video, the time necessary to implement SEBE equipment is 01'10" and the time necessary to undertake the risk assessment is 05'50". These values are in accordance with the experimental data.

Reliability and Validity
The reliability and the validity of the questionnaire are relative. To date, we may consider that, for the work environments for which is was tested, the questionnaire matches the expectations: avoiding any issue due to the SEBE equipment. However, when applied in new environments, it is suggested to test the questionnaire and add new questions if need be. Should this happen, the users are encouraged to contact the corresponding author in order to update the questionnaire and upload online a new version. At this stage of the study, we assume that, due to the heterogeneity of the professions studied, we reached a satisfactory level of confidence.

LIMITATIONS
Despite the fact that results of the present study suggest that the developed protocol for SEBE risk assessment is a relevant tool, the application as well as the exploratory phase preceding the elaboration did not investigate situations with infection or contamination. It would be worth to test the application of the protocol in such contexts that may be met in bio-industry for example.

CONCLUSION
More than 30% questions were added from version 1.1 to version 2.3 showing the necessity of the present study.
A protocol for risk assessment regarding the application of SEBE metrology equipment was validated for work activities in nuclear power plant (previous study) and hospital, police and air force (present study). This protocol was based on the recommendations and applications of the International Atomic Energy Association [37], the Institute of Nuclear Power Operations [38][39] and the National Aeronautics and Space Administration [40] (see [34]).
The protocol gave satisfactory results in terms of risk prevention and time duration for application.
From the previous study, we found important to add a reminder in the protocol document for the participants not to forget that the priority remains the work activity carried out by them. In case of feeling any discomfort due to SEBE equipment, they must request its immediate withdrawal. Furthermore, recommendations of INPO led us to highlight the necessity to perform a systematic risk assessment before each application, even though we had the same participant and/or the same activity. From the present study, we understood the necessity to question the potential source of pain induced by the SEBE metrology because of its possible consequences on the cognitive load of the participants.
The previous study highlighted however a risk of side-effect that is worth reminding here: workers who are not at ease in their job due to lack of skills might declare that the SEBE equipment was disturbing them to justify a problem and not to accept their own responsibilities in case of low performance regarding their work activity; moreover, in case of an accident, SEBE metrology equipment might be accused as disturbing workers even though that was not the case. These findings gave greater importance to the necessity of this sort of risk assessment protocol.
This protocol may be applied to any kind of SEBE, including wireless devices or systems with integrated camcorder and/or the microphone inside the glasses. Yet, the protocol needs to be tested in other biology contexts in order to be improved and/or to confirm its robustness when addressing potential infections or contaminations. Despite that, the SEBE risk assessment protocol we obtained clearly fills a gap with efficiency for researchers and analysts using SEBE techniques.

CONSENT
The authors declare that written informed consent was obtained from subjects for publication of this paper.

ETHICAL APPROVAL
This study received ethical approval of the Ethics Committee of the Dept. of Social Psychology, London School of Economics and Political Science (London, UK) and has therefore been performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki.