Conducting Nuclear Security Culture Self-Assessments in Nuclear Research Facilities Using the IAEA Methodology

The National Nuclear Energy Agency of Indonesia (BATAN) has promoted the culture and assessment of nuclear security in Indonesia since 2010. After conducting a successful trial assessment in 2012, BATAN performed a second self-assessment in 2015 in three nuclear facilities. The assessment's methodology followed the guidelines set by the International Atomic Energy Agency (IAEA). The objective of this paper is to demonstrate the techniques used in applying the IAEA methodology and to discuss the primary outcomes of the assessment. The data collection consisted of surveys, interviews, document reviews, and observations. A total of about 50% of the three facilities’ employees participated in the survey and/or interview process. Three categories of sub-cultures—security personnel, non-structural personnel, and structural personnel—were assessed, covering a range of twelve general characteristics and thirty indicators of security culture. An overall average score of 5.25 on a 7-point scale was obtained from the survey. A consolidated assessment of the results from all data collection techniques showed that most of the security culture characteristics assessed reflected some of the anticipated best practices and also demonstrated strengths necessary in security culture. However, certain aspects of leadership behavior need to be enhanced, especially ones relating to motivation. By examining the survey results from the 1 Antariksawan et al.: Conducting Nuclear Security Culture Self-Assessments in Nuclear Research Facilities Using the IAEA Methodology Published by Trace: Tennessee Research and Creative Exchange, 2018 International Journal of Nuclear Security, Vol. 4, No. 1, 2018 three subcultures, we conclude that further dissemination of nuclear security concepts and policy is needed—particularly to researchers and administrative personnel. The methodology for the IAEA’s nuclear security culture self-assessment has been successfully implemented and has resulted in positive outcomes. The potential improvement of assessment implementation should also be considered, such as in the development of survey questions and the selection of indicators to be elaborated on interviews.


I. Introduction
Nuclear and radioactive materials should always be kept safe and secure in order to protect employees, the public, and the environment from undue exposure to radiation. Potential threats from theft, sabotage, unauthorized access, illegal transfer, or other malicious acts involving nuclear material must be considered when designing an effective security system. One important element of any security system is involvement of human security measures; technical measures alone cannot assure the achievement of the required level of security. The human factor must be considered and built into the calculation of the success of the security system [1]. Individual understanding of and dedication and commitment to security from all individuals engaged in related activities will strengthen the security system. In such a context, it is widely believed that establishing and promoting a strong security culture is essential for building an effective security system. Highlighting its importance and significance, at the Nuclear Security Summit of 2014, The Hague Communiqué listed nuclear security culture as the first of its three pillars of nuclear security (the other two being physical protection and materials accountancy and control) [2].To respond to the need for guidance in security culture, the International Atomic Energy Agency (IAEA) established a model and published an implementation guide for security culture [3].
The concept of nuclear security culture emerged after that of nuclear safety culture, which was triggered in large part by the Chernobyl nuclear power plant accident in 1986. However, both concepts share the common objective of protecting society and human lives. As both goals are important for nuclear facilities and activities, synergy between them must be enhanced [4,5]. Many countries with welldeveloped nuclear programs, such as China and Japan, have made the enhancement of nuclear security culture a top priority [6,7].
Indonesia has three research reactors and several supporting nuclear facilities, including nuclear fuel and radioactive waste treatment facilities, which are operated by the National Nuclear Energy Agency of Indonesia (BATAN). The work at these facilities involves the use of nuclear materials. Therefore, as the operator of the reactors and facilities and in accordance with national policy, BATAN regards safety and security as its top priorities. In Indonesia, all nuclear licensees must include the development of nuclear security culture in their security management system [8]. Along with the effort to promote and enhance its nuclear security culture, BATAN decided to launch a nuclear security culture self-assessment trial in 2012 [9, 10]. The trial was conducted with support from the IAEA and the Center for International Trade and Security (CITS) at the University of Georgia (UGA). The IAEA security culture assessment guidance [11], which was then being drafted, was used as the framework. The objectives included testing the process's draft version and providing feedback to the IAEA regarding the results. The results were also expected to: improve understanding of employees' concerns, needs, aspirations, and motivations; illuminate employee opinions about key management issues; and enhance safety culture assessments [9]. Based on the experience gained from administering self-assessments in nuclear security culture, Indonesia established the Center for Security Culture and Assessment (CSCA) in 2014 in order to administer and promote activity in nuclear security culture and assessment under the guidance of BATAN [12].
While there has been increasing interest in issues regarding nuclear security culture, to date only a few countries or institutions have been conducting nuclear security culture self-assessments. In 2014, Bulgaria conducted a self-assessment on its nuclear security culture in the Kozloduy Nuclear Power Plant by using the IAEA methodology. The self-assessment included a survey, an interview, a document review, and an observation. A survey form was distributed to 600 employees (or about 15% of the total employee population), and an additional twenty (20) employees were interviewed. One of the conclusions from the self-assessment was that the methodology not only reveals the areas which need to be improved and the overall level of security culture, but also improves security [13]. The Republic of Korea has also performed a nuclear security culture survey on the personnel who work at nuclear power plants. The goal was to develop a questionnaire used to evaluate the level of awareness of nuclear security issues. The survey was not based on the IAEA methodology. The questionnaire covers four categories: beliefs and attitudes, operating systems, leadership, and staff behaviors. A total of 858 people were surveyed, and one important finding was that awareness increases with age until an employee reaches his or her fifties [14]. A preliminary nuclear security culture self-assessment has been conducted in a nuclear instrument company in Hungary [15]. The IAEA's methodology was used, but only the survey was conducted.
Though the survey was only preliminary, the response from the personnel was positive. The experience gained from the activity may be considered in future self-assessment efforts.
Considering the results of Indonesia's first nuclear security culture assessment and in order to foster nuclear security culture, BATAN decided to conduct a second nuclear security culture self-assessment in 2015 [16]. The objective was primarily to apply all the assessment tools in the IAEA self-assessment methodology [17,18]. In the first assessment, only two of the four assessment tools were used. This second assessment was also intended to follow up on the results from the first assessment and to assess the performance of security culture in different categories of personnel (subcultures). The subjects of the assessment were the multipurpose research reactor, the fuel technology facility, and the radioactive waste technology facilities located in the Serpong Nuclear Area. These facilities were chosen because all three involve the use of nuclear materials and are located at the same site where the assessment was administered. Twelve out of thirty characteristics from the IAEA list of elements of security culture were selected based on factors which were considered most important for the implementation of nuclear security in BATAN, those which were relatively easy to assess through a questionnaire, and those which illuminated findings from the first assessment's results. Though the basic methodology used was IAEA's, some adjustments were made to adapt to the existing resources and the field circumstances. The selfassessment was performed by a team consisting primarily of personnel from CSCA/BATAN. This paper aims to describe the implementation of and general results from the second self-assessment of Indonesia's three nuclear research facilities, as conducted by CSCA/BATAN. The results will be presented generically to conform to BATAN's confidentiality policy. A brief description of the adopted IAEA assessment methodology and the assessment's implementation is provided in Chapter II. The significant findings of the assessment and the conclusions are presented in Chapters III and IV, respectively.

A. Methodology
Per the IAEA's self-assessment guidelines [17], there are four methods for obtaining data and information regarding the implementation of nuclear security: There are no specific guidelines from the IAEA regarding the sequence of the methods, so the selfassessment team decided to perform the stages in the abovementioned order ( Fig. 1). The last two methods were performed simultaneously. The survey is the main data collection technique for providing quantitative data. It was used as a basis for the interview and document review stages. The data collected from the survey were analyzed to obtain the average scores for each measured characteristic and categorized into positive and negative responses. The negative responses were used by the assessment team as the items to be discussed and elaborated on during the interview stage. Additionally, some of the positive responses were verified through the document review process.
The survey used a 7-point scale system (see Fig. 2) [17]. Based on the findings of the first self-assessment and in order to avoid many responses of "neither agree nor disagree" (a score of 4 on the scale), the respondent was asked to include a comment explaining his/her response. Additionally, respondents were encouraged to include comments regardless of their responses. The comments served as important context for the respondents' numeric answers. To provide a quantitative figure for each indicator, all scores obtained from the survey were collected, tabulated, and represented in the form of a histogram. The data was analyzed according to each indicator and each type of respondent. For each indicator i, the average score, Pav(i), was calculated as follows: where: SD : Strongly Disagree D : Disagree SWD : Somewhat Disagree NAD : Neither Agree nor Disagree SWA : Somewhat Agree A : Agree SA : Strongly Agree NR : Number of Respondents The average score for each characteristic c, Pav(c), and of all the indicators assessed, Pav, are then given by: where NI(c) and NI are the number of indicators in characteristic c and total number of indicators, respectively; Pav(i) in (2)   There is a possibility that some responses indicated conflicted views. Such conflicted views represent divided opinions among the respondents; the majority of the views are favorable to the survey statement, but unfavorable views are also considerable, as shown in Fig. 3(b). Results of this kind need to be explored further in order to discover the cause. Moreover, based on the color zones as specified in the IAEA guidelines (explicated herein), the average score for an indicator below 4 should be in the red zone, between 4 and 5 falls in the yellow zone, and scores higher than 5 should be in the green zone [17]. Data collected from all methods were then consolidated. The analysis of the data provided quantitative findings (from the survey stage) and qualitative findings (from the interview, the document assessment, and the observation stages) concerning the security culture characteristics being assessed. The results of the consolidated assessment data were then plotted in a diagram composed of three zones-i.e., red, yellow, and green-indicating regions of weakness (serious problems), gaps needing to be dealt with, and strengths (areas of good performance), respectively.

Respondents
About 60% of employees from three nuclear research centers were expected to be involved in the selfassessment, particularly for the survey and interview stages. The respondents could be classified into three subcultures: structural personnel, non-structural personnel, and security personnel. Structural personnel are ones in charge in an organizational structure (i.e., management), non-structural personnel are researchers or administrative employees, and security personnel are guards and physical protection employees.

Characteristics and Indicators Assessed
The IAEA model of security culture consists of three layers: belief and attitudes, principles and management systems, and behaviors of leadership and personnel [3]. The management systems and behaviors are identified the observable elements, also called "artifacts," which could be gathered directly based on the self-assessment. Each element comprises several characteristics. In total, there are 30 characteristics. The IAEA guidelines assign cultural indicators to the characteristics of nuclear security culture, which can be used to evaluate the characteristics' performance.
Thirty indicators have been defined which represent twelve major characteristics in three elements of the nuclear security culture model-i.e., the management system, leadership behaviors, and personnel behaviors. Each indicator was converted to a personalized question (or survey statement) for the survey.
The twelve characteristics and the number of associated indicators used for each characteristic are shown in Table 1.

Surveys
Prior to conducting the survey, a validation of its questions was performed to check that they were easy to understand. For this step, 25 respondents were invited to participate. The check concluded that the survey could easily be understood.
The total number of respondents for the survey, or the target response rate, was expected to be about 60% of the three facilities' total employees-i.e., about 315 people. As described above, the respondents were classified as structural, non-structural, and security personnel. There were thirty (30) total statements on the survey. Given that number, a total of 8,310 responses were expected to be collected.
In actuality, the total number of survey respondents was 277 people, or about 52% of the total employees of the three facilities. This deficit resulted from other urgent activities that prevented some employees from participating on the day of the survey. Based on the category (or subculture) of the respondents, the numbers of security personnel, structural respondents, and non-structural respondents were 36 (about 13%), 43 (about 15.5%), and 198 (about 71.5%), respectively. The survey was implemented on one day in three batches. All respondents were given a short introduction on the objective of the self-assessment and instructions on how to respond to the survey prior to its implementation. The time allocated for each batch of the survey was about 60 minutes. This time was considered adequate based on the previous selfassessment.

Interviews
Forty-five employees from the three facilities were invited to participate in the interview stage of the assessment. Half of interviewees participated in the survey. They also represented the three subcultures. There were two interviewers and 60 minutes allotted for each interviewee. None of the interviewers were from the security personnel subculture. The technique used for the interview was a semi-structured session developed from the survey and written comments. The guidelines for the interview were prepared in advance by the self-assessment team. The interview was focused on the elaboration of so-called negative responses obtained from the survey results, as they appeared in the histograms.

Document Reviews
Document reviews were done to determine whether the documents relating to security management had been well-managed or not, and to re-check certain positive responses obtained from the survey. Many documents assessed were classified as confidential, so the assessment was performed at each individual site, and all assessors agreed regarding the assessment's confidentiality. Considering the number of documents and time allocated, the number of assessors at each site was about nine or ten people.

Observations
The assessment's observations were intended to observe and record actual performance in the field in real time, and they would be compared to the results of the survey. The observations were done during the joint training exercise session conducted in the Serpong Nuclear Research Area, where the three centers being assessed are located. They were conducted by an unseen observer team.

III. Results and Discussion
The second nuclear security culture self-assessment was administered somewhat more successfully than the first self-assessment trial of 2012-2013, particularly with respect to the following three areas: (1) the utilization of the four data collection methods, (2) the involvement of the team members with a greater variety of backgrounds (a multidisciplinary approach), and (3) the use of personalized questions/survey statements and validation of survey statements prior to its administration. All four assessment tools were carried out efficiently. However, the quantitative results from the survey was used as the primary data for the overall assessment, while the results from interview, document review, and observation stages were more frequently used as supplementary data.
The collected responses on the survey were nearly 99%. This figure means that not all respondents responded to all 30 survey statements. One hundred and twenty-two (122) written responses were received, a figure markedly higher than the thirty-five (35) written responses from the first selfassessment. Some comments were general in nature, such as those relating to management issues, and these were not precisely related to security matters. The increase in collected comments represents some of the best improvements gained from this second self-assessment. These comments are valuable for the interpretation of the survey results and also for the consideration during the interview process.
After collecting all response data, the histogram of each indicator was drawn up and the average score of each indicator, Pav(i), and each characteristic, Pav(c), was calculated using Eq. (1) and (2). Figure 4 shows examples of histograms for two survey statements. The average score of each characteristic, Pav(c), ranges from 4.60 to 5.67, and the overall average score, Pav, is 5.25.  The average score of each indicator is placed in the green zone if the score is higher than 5, in the yellow zone if the score is between 4 and 5, and in the red zone if the score is lower than 4. As shown in Fig. 4, no characteristic is in red zone; only one characteristic, motivation, is in the yellow zone; and all others are in the green zone. These results indicate that the survey respondents viewed most of the assessed characteristics of security culture favorably. Regarding the characteristic of motivation, three indicators were examined through the following three survey statements: 1. My managers recognize commendable attitudes and behaviors (statement #7) 2. My facility has reward systems recognizing staff members' contributions towards maintaining nuclear security (statement #8) 3. I am aware of the systems of rewards and sanctions relating to nuclear security (statement #9) Based on the collected survey scores, the responses for statements #8 and #9 are categorized as negative responses and conflicted views. Later, during the interview, these responses were elaborated on. Most interviewees thought that there was no clear reward system relating to security issues. In this regard, it is interesting that some interviewees defined the reward as being strictly limited to acknowledgment of the employee's important contribution. Further analysis found that there is a correlation between the characteristic of motivation and other leadership characteristics being assessed, e.g., the use of authority and communication. The responses for these last two characteristics also demonstrated some conflicting views, including survey statement #1, "My superior reserves time to provide direct guidance (on nuclear security)." The survey results on this statement showed conflicting views. About 18% of the responses fell between "strongly disagree" and "somewhat disagree." The interviews revealed that the respondents had expected their superiors/managers to give more time to talk about security, and to guide and direct employees about the security matters. Concerning the motivation of personnel, results from the survey conducted in other studies in nuclear power plants indicate similar findings. It has been found that respondents are generally discontented with their management for not motivating personnel [14]. The role of the managers is also important for enhancing adherence to procedures by, for example, closely monitoring, evaluating, and providing exemplary behavior in implementing good practices in adhering to procedures. Or, the organization could consider two policy strategies regarding the adherence to rules and procedures, which are based in command-and-control models and self-regulatory approaches [19].
The survey also uncovered is that compared to the other elements of security culture, a comparatively large number of indicators of leadership behavior being assessed yielded negative responses and conflicting views in their categories. The responses indicated that respondents have a higher level of concern about these leadership behaviors. This may also be result of Indonesian cultural views of leadership, which are based on paternalistic leadership practices combined with elements of visible leadership as its most dominant factors [20].
From the survey results, it is possible to study any differences in views of the respondents from the three different nuclear research facilities and the three different subcultures. Figure 6 depicts the results of the average scores of the twelve characteristics for the three nuclear research facilities. It shows that, despite some differences in the results from center two, the results for all three facilities are, on average, almost identical. In other words, there is no significant difference in views with regard to the employees' nuclear security culture, regardless of the employees' place of work. Concerning the different subcultures' views on security culture, Fig. 7 shows the average scores of twelve characteristics for the three different subcultures obtained from the survey. It is interesting to note that in Fig. 7, the pattern of responses from all subcultures are quite similar to one another. In general, the nonstructural subcultures scored lower in all indicators than the security personnel and the structural subcultures. It should be noted that the number of survey participants from the non-structural category was higher than the security and the structural categories. When this is cross-checked with the interview results, it is possible that the difference in responses may result from the fact that some non-structural category respondents do not understand the security concepts well. Some thought that security is simply the responsibility of guards. On the other hand, it should be noted that the number of non-structural respondents was higher than the number of respondents from the security personnel and structural personnel, comprising 71.5% of those surveyed. It is also known that many of this 71.5% were never previously exposed to the dissemination of nuclear security information. This finding provides a warning to the management that there should be improvements and enhancements made to the effectiveness of the dissemination of security policy and security culture programs, especially to researchers and administrative personnel.

Lessons Learned
From implementing the self-assessment according to the IAEA methodology, there are several lessons that could be learned by both the self-assessment team and the organization, which include: 1. Though surveys and interviews are considered the primary techniques for collecting data for selfassessments, the document reviews and observations were useful for enriching the data and enhancing the analysis. Observations should be done over a greater period of time, rather than exclusively during the drill session, as was done in the current assessment, and should focus on predetermined aspects of security culture.
2. The validation of survey statements prior to the survey implementation was found to be very helpful for avoiding misunderstandings about the survey statement. Moreover, the use of personalized question types of survey statements helped the respondents to understand the survey statements. The team also realized that some survey statements are quite similar to each other. So, in the future, the survey statements should explore a greater variety of aspects of each determined characteristic.
3. The request for respondents to provide a written comment if their response falls on a 4 on the 7point scale helped to prevent the respondents from responding "Neither Agree nor Disagree." As well, this practice increases the number of comments received.
4. The interviewer's skills are crucial, especially when trying to make the interviewee feel more comfortable and be more open in his/her answers. Interviewees should not feel intimidated. By helping the interviewees feel relaxed, interviewers were able to uncover more information from the interviewees. This practice is important because some interviewees seemed less open in answering specific questions. The team should consider organizing a training session on interviewer skills.
5. The development of consolidated results is a critical step that requires in-depth knowledge and analysis. From this present assessment, the implementation was found to be needing improvement.
6. The involvement of high-level management in the self-assessment was appreciated and increased the motivation of personnel involved in the self-assessment (i.e., the self-assessment team and respondents).

IV. Conclusion
The second self-assessment on nuclear security culture in three nuclear research facilities in Indonesia using the IAEA methodology has been successfully performed. All four data collection techniques recommended in the IAEA methodology were used, though the surveys and interviews were still the most important data sources for the assessment. The implementation of the self-assessment has also provided meaningful information to the organization in terms of illustrating the present status of the nuclear security culture and the security culture characteristics which need to be improved. From the twelve characteristics of the three security culture elements assessed, it was found that, in general, there are already good practices and strong elements in place. However, some indicators need to be improved. The most important one is the motivation characteristics. To that end, the management should also consider enhancing the dissemination of nuclear security concepts and policies to the researchers and administrative personnel.
The implementation of the second self-assessment was a marked improvement over the first assessment. However, some items require further improvement, such as the formulation of survey statements, in determining the indicators to be explored in the interview, and more in-depth observations. The skill of the interviewers in guiding the interviewees should also be improved. The experience gained from this second self-assessment is very important to prepare for self-assessments at other facilities involving radioactive materials.