In professional projects, failure mode and effects analysis (FMEA) have been used for risk analysis and reliability assessment. FMEA results help team members to manage risks under uncertainty, although the conventional risk priority number (RPN) method on FMEA has been criticized extensively  . Fuzzy Expert System is a method that was used to assess the risks. Reference  researched on the mathematical model of the fuzzy expert system to calculate the system risk and decide the acceptable risks through the risk assessment activities. On the acceptable risk, they found many factors beyond the control are the human subjectivity and lack of knowledge of the system. The human error is at most of these risk factors that are the reason why the fuzzy logics have been used to solve the problem of human uncertainty.
In the engineering education, the project-based learning (PBL) is widely used to enable students to gain knowledge, understanding, and skills from experience in the real world. Learning how to assess risk helps students understand the risks of particular courses of action. It also helps them to gain their knowledge about the project itself. Project managers often overlook students’ lack of experience, especially, in risk assessment. They often involve students as if they were professional people although the students do not understand the concept of risk and are therefore unable to assess risks. To answer this question, the fuzzy failure mode and effects analysis method (Fuzzy FMEA) are currently used in project risk management as a way of avoiding the limitations of traditional FMEA. In particular, even in projects involving professionals, there are often problems with the accuracy of the RPN and even experts often assess the RPN subjectively. In the student projects, where team members lack experience, the problem is even greater.
This paper proposes the application of a fuzzy failure mode and effects analysis model for project-based software engineering education. We use a fuzzy rule-based system with the learning agents to carry out risk analysis in an application of project-based learning. The membership function is constructed by the learning agents. It will help students who lack experience in risk assessment to develop their expertise in that skill. On the fuzzy rule-based system, the classification technique will generate to judge the risk based on a fuzzy inference system. In which the fuzzy methodology presented can lead to better results than a traditional FMEA approach.
This paper is organized as follows. In the background, Sections 2, 3, and 4 summarize the principles and basis of this research, including the risks of PBL; the general fuzzy FMEA model; and their shortcomings, respectively. The proposed model is described in Section 5. This section presents a step-by-step explanation based on the concept of fuzzy inference system and FMEA processes. In Section 6, we propose the experimental design. Section 7 discusses the advantages and limitations. Finally, Section 8 evaluates the proposed methods and its contribution.
2. The Risks of PBL
Risk is unavoidable when projects are undertaken, but it can be understood and managed. Risk is defined as the possibility of loss or injury. Risk exposure or risk impact refers to the relation between the probability of an unsatisfactory outcome and the loss to the parties affected if the outcome is unsatisfactory  . In student projects, assessing risks should be a simple process. And the risks are directly impacted to the expected learning outcome of Project-based learning.
Engineering education is a rapidly developing field. Nowadays, it often involves constructing educational plans for learning through real-world experiences. PBL is a form of situated learning in the real-world context that allows students to gain a deeper understanding of the material  . In PBL, students should aim to improve their understanding of their field and master the practical skills involved.
2.2. Problem of Risk Assessment in Student’s Software Projects
Reference  stated that a single instructor cannot effectively coach projects involving more than 25 to 30 students. If an instructor spends more time than what is allocated to the course, then some students maybe unsympathetic to such dedication. As a result, both students and instructors may fail to achieve the learning objectives of the project.
In practice, most assessors evaluate only the results of the project itself, rather than addressing the learners’ lack of understanding and basic skills. Figure 1 is a typical scenario for a student project in the field of software engineering. The scenario is based on the systems development life cycle. It shows that with regard to the risk factors, in practice, most assessors evaluate only the project’s results and typically focus only on successful learners rather than addressing the learners that lack understanding and basic skills.
In this case, the risks are abnormal states for projects that may lead to its overall failure. The outputs or products are directly related to the expected learning outcome. However, the real situation is complicated and requires a high-performance tool to assess the risks. In this study, we chose FMEA as the tool for risk analysis.
3. General Fuzzy FMEA Model
3.1. Traditional FMEA
The FMEA is a systematic failure analysis technique used to identify the effects
Figure 1. A typical scenario of student project in software engineering fields.
of failure on system operations or projects. It is sometimes referred to as failure mode, effects, and criticality analysis (FMECA) and is often the first step of a system and software reliability study  . It involves reviewing as many compo- nents, assemblies, and subsystems as possible to identify failure modes, their causes, and their effects. Figure 2 shows the five iterative stages that make up the typical flow of FMEA  . Noted that it is almost all are evaluated by the expert opinion of the three risk factors, which will be calculated as RPN for each failure mode.
As mentioned above, the RPN is a key quantity calculated by multiplying the three input factors―severity (S), occurrence (O), and detection (D)―as follows:
where S represents the seriousness of failure after it has occurred, O represents the probability of occurrence, and D represents the probability of detecting a defect on system. All three factors are usually estimated by subjective numerical weighting on a scale from 1 to 10. FMEA offers a risk analysis for each stage of the project and can deal with risks in a timely manner during project execution.
3.2. Fuzzy Rule-Based System
The fuzzy rule-based system has been widely used to classify and judge according to the rules of fuzzy logic. Figure 3 is the basic flowchart of the fuzzy rule- based system  . The input variable X is a fuzzy set. The input interface is designed to receive the input fuzzy set. The rule-based system is composed of a set of fuzzy if-then rules that relate input to output variables. The data-based system includes the parameter values of the rule-based model’s scaling factor, including the details of criteria, membership functions, and others.
Fuzzy inference is a process that uses rule-based data-based systems to drive the fuzzy inference and approximate reasoning process. The output interface transmits the results of fuzzy inference into output Y.
Figure 2. Five stages involved in a typical flow of FMEA.
Figure 3. Flowchart of the fuzzy rule-based system.
An if-then rule expresses a certain relation between a fuzzy variable, an input X, and an output Y. The basic rule-based format is
where A and B describe the some pieces of the domain knowledge of the problems. In the case of fuzzy if-then rules, A and B represent the degrees of fuzzy values.
The membership function denotes a classical subset A of X. The function forms part of a data-based system within a fuzzy rule-based system and depends on the combination of fuzzy sets. The notation is a grade of membership of X in A. The rules and membership functions are defined relative to the context and situation of problems.
4. Shortcomings and Research Questions
The nature of software failure often arises from the fail design of human. Although FMEA is a powerful analytical tool to identify risks, they have long been established for hardware systems, they are not very well understood for software systems  . There are weaknesses in the way RPN works with FMEA. When everything is going well in a professional project, it is easy enough to design a process that works well. However, student projects differ from professional projects in that students are not ready to judge the risk of problems in all the aspects of their project because they lack experience. However, risk identification using the fuzzy FMEA model also has problems related to the fuzzy rule-based and membership functions. Advisors or assessors cannot define the rules and membership functions as well as in professional projects. This paper proposes a method of alleviating this problem by improving the traditional fuzzy FMEA approach for risk evaluation on student projects by applying intelligent agents.
In the present study, we examine two research questions. First, how can individual learners be supported to evaluate the risks by themselves or encouraged to gain specific skills and knowledge? Second, how can we apply FMEA to support PBL? This paper proposes a method to apply the traditional fuzzy FMEA model for risk evaluation on student projects by applying intelligent agents.
5. The Proposed Methodology
This study addressed the research questions by developing a methodology that supports the PBL through student projects. The study includes introducing into the fuzzy rule-based system two extra elements―the construction of the membership function and the voting rule-based classification. Figure 4 shows the overview of this proposed methodology. Risks are identified during interaction between the team members. The process is two-fold:
5.1. Agents Construct Membership Functions
Intelligent agents are autonomous agents that perceive the data through sensors and act on their environment  . Intelligent agents may learn or use historical
Figure 4. Overview of the proposed methodology.
knowledge to achieve the desired goals. The basic structure of agents includes three main components: a monitor for perceiving the environment, a set of goals leading to the desired results, and an actuator for responding to the results.
Figure 5 is the proposed model for the agent that constructs the membership function. Hong and Lee described a knowledge acquisition facility―a learning method that automatically derives fuzzy rules and membership functions from a given set of training instances  . Based on this method, we propose the use of learning agents as an expert knowledge system to construct the membership function and to transfer it into the fuzzy rules-based classification in the fuzzy FMEA model. The steps are briefly described below:
・ Step 1: Perceive and prepare the input value from historical data
The input values are perfected by monitors of agents. Import data must be prepared first. It may take the form below:
where V is the input space of the historical membership data of each risk factor in FMEA. The variable v refers to the values of the risk factors and n is the total amount of v in the historical input space V. The input values are sorted to find the relation between each value. The result of this step is
The difference (d) between the values in Equation (4) provide the data about similarity (s) between them. The variable v' refers to the values of the risk factors that sorted. For each pair of values
, the difference is calculated a
, and we convert each
where is the similarity between each pair of values and , CP is the control parameter for shaping the membership functions, and sd is the standard deviation of the values. Note that the larger the CP, the greater the similarity.
・ Step 2: Cluster values into a group by hierarchical clustering and determine the number of groups as goals
This step is adopted from the ∝-cut of similarity. The variable alpha is the threshold for a pair of values to be included in the group, using p and q as itera-
Figure 5. Flowchart for contracting the membership function.
tion variables. The approach followed in this step is shown as a pseudo code as follows:
FOR p = 1 to list size
FOR q = 1 to list size
IF (si < alpha) THEN
Divide a pair value into the difference group
Next q, Next p
Divide a pair value into the same group, Next p
#Set a goal
n = 5 #Define the among of linguistic criteria
m = 0.2 #Define the tuning variable
#Learning to a goal
IF (length (q) == n) THEN
ELSE IF (length (q) < n) THEN
alpha = alpha + m
alpha = alpha − m
#Check amount in each groups
IF (amount in each groups <= 3) THEN
Send to the near group above
Do LOOP again
・ Step 3: Determine the central value bj and calculate the results for the similar value
The value is the peak value for each group. The next step is to divide a pair of values into a group.
・ Step 4: Find a minimum value and a maximum value in each group as follows
The minimum point (a, 0) in each group is defined as
The maximum point (0, c) in each group is defined as
Note that if the maximum point is above the highest value of each group then we assume the highest value is a maximum point.
・ Step 5: Actions to construct the membership function
In the previous step, the membership function was constructed using the three variables to form a triangle for each group, which gives the minimum variable a, the central value b, and the maximum variable c.
5.2. Voting Fuzzy Rule-Based Classification
After constructing all membership functions for each risk factor, the if-then rule classification can be obtained. Ishibuchi and others proposed a way to classify these rules by fuzzy reasoning based on voting both a single winner rules and multiple rules  . Later, Liu and others also presented fuzzy evidential reasoning and a belief rule-based method  . By focusing on the aforementioned rule-based belief and voting by multiple fuzzy if-then rules, we propose the final step in arriving at the fuzzy if-then rule classification system.
Classification of fuzzy if-then rules: In general, the number of rules (R) generated is equal to the number of possible combinations of different grades of assessments of risk factors. In this case, there are three risk factors: S, O, and D. The grade of assessment is calculated, and the fuzzy if-then rules are classified using the average value voting by multiple rules as below:
where is the voting result of compatibility grade of each input value, variable is the average value of each membership function, and T is the number of classes with amount m. Variable CF is the grade of certainty, which can be adjusted by the learning processes  . The calculation results from the above equation can be plotted in the rule table (Table 1). The fuzzy FMEA model and the
Table 1. Format of rule table.
agent-based model are combined to improve the FMEA and support the evaluation of risk in the student’s software projects.
6. Experimental Design
We tried the approach described above for a software engineering project undertaken by student. It allowed us to illustrate the systematic processes followed by the research questions. As shown in Figure 6, the main stages of the experiment are to test the way we construct the membership function and the process used for classifying the fuzzy rule. In this case, risks were identified during the interaction between the users, both students and project advisors. Traditional FMEA cannot evaluate RPN with great accuracy because the calculation of RPN is based on subjective weighting of RPN by team members. Moreover, in our scenario, the participating students are not experts and cannot evaluate RPN as professional projects do.
Firstly, the linguistic variable is defined. Table 2 is the level definition and linguistic variable of three risk factors. The membership functions are constructed as an expert knowledge system. A set of linguistic criteria is Very Low (VL), Low (L), Moderate (M), High (H), and Very High (VH). Then, this experiment was designed to determine the method of recording data from the student’s software projects. Everyone in the team acts as a risk evaluator on their projects. However, the advisor, being the expert, is the main risk evaluator.
In the example reported here, the software project was evaluated in four phases of software development, including modeling and requirement analysis, design, implementation, and testing phases. Table 3 shows the format for weighing the S, O, and D values. It is used to assess a student project by project advisor, many people gathered and classified according to the software development phase. Subsequently, historical data is added in. The accumulation of data on expert risk assessment is recorded in the historical risk assessment data that was generated in the previous project. The historical data will be retrieved for the preparation to construct the next stages. The membership function will be constructed by Equations (4)-(7). Figure 7 is the typical graph that shows the hierarchical clustering by pair distance and similarity values. In practice, the evaluator often evaluates an integer number or first decimal figure of the floating-
Figure 6. Flowchart of experimental design.
Figure 7. Typical results of hierarchical clustering.
Table 2. Level definition and linguistic variable of three risk factors.
Table 3. Format for weighting severity, occurrence, and detection values in software pro- jects involving students.
point. Thus, for realism, we sampled by random evaluations of two experts who have evaluated the different value around 0.5 to 1.0.
As shown the results in Figure 8, the membership functions results of S, O, and D are defined by the five steps in Section 5.1. Then, Table 4 shows typical results of rule table that are computed by the Equation (8) in Section 5.2. The maximum risk needs to be considered in a case of student project because it needs to fix the problem first. The numbers of rules are generated equal to the number of possible combinations of different grades of assessments. In this case that is 125 rules for each phase. The voting result varies from 0 to 10, depending on each input value. The risks are classified into five levels as the linguistic variables. Lastly, all results include membership function and rule table will be used to assess student projects which encourage students to understand their own risk assessment more.
As mentioned above, this present study had two advantages. First, it can be developed the supporting students to check the risks in their projects. Most people in engineering education have been using project-based learning as a tool for teaching knowledge and practical skills but are not aware of the students’ lack of experience and treat them as professional staff. Second, the technical term can be a part of the prototype for generating the risk assessment model for software engineering education. Advisors can track and improve project direction through- out the teaching process. Simultaneously, they can employ the outcomes to design curriculums that are well suited to project-based software engineering education. In addition, it can be used to identify basic admission requirements for software engineering courses.
However, this present study had two issues, which need to continue to develop. Firstly, these proposed classification techniques for fuzzy rule-based agent system still require improvements. It limits itself to the basics of solving too many rules. Reducing the number of rules needs to be developed at this point. The flexible classification method also needs to define a serious looking path. Secondly, this construction technique also required the development of the forms other than the triangular fuzzy number. While this supports the creation of a membership function triangular. Because it is not complicated in the clustering process and finding the center, maximum, and minimum values in each group. However, the trapezoidal fuzzy number is an alternative that should be developed further to see how effective it is with this system.
Figure 8. Membership function results of (a) modeling and requirement analysis; (b) design; (c) implementation; and (d) testing.
Table 4. Typical results of rule table.
This study proposed the application of fuzzy FMEA model for supporting students undertaking PBL. We base our method on two main processes. The first proposed process is the agent-based membership function contraction. It is used as an expert system to simulate the membership function for data-based systems on fuzzy inference systems. The second proposed process is the use of voting fuzzy rule-based classification. Fuzzy rule-based classification is a challenging approach because we had to set up the rule-based system for fuzzy FMEA under extremely difficult conditions. Thus, the fuzzy rule-based classification enhanced by the voting techniques supports students to assess the risks easily. In this study, we also designed an experiment that allowed us to illustrate the systematic processes that addressed the research questions. The main ideas in this experiment are to test the way of constructing the membership function and to test the process by which the fuzzy rule in this study is classified.
We thank the graduate school of Nippon institute of technology for supporting.