NIMSAD Framework to Evaluate the Role-based Goal Modelling

— Requirements analysis is a process of analyzing requirements of various stakeholders that represents a specification of system behaviour that must be stated precisely to proceed to the design phase. It is noted that the current process of requirements analysis is not sufficient to identify and represent the existence of multiple stakeholders which could lead to various conflicts and overlapping requirements. Furthermore, the involvement of various stakeholders normally leads to inconsistencies and misinterpretation of requirements. In this study, the goal modelling representation has been improvised and named as role-based goal modelling. The role-based goal modelling highlights each stakeholder’s role identification in discovering the intentions and requirements of various stakeholders including the integration of data elements in order to determine the dependency of data when dealing with multiple stakeholders. Labour Management System (LMS) owned by Gates IT Solution Sdn Bhd is selected as a case study with participation of different stakeholders. Yet, no standard evaluation method can be used to measure and compare our improved goal modelling with the previous work. Thus, NIMSAD evaluation framework is proposed to compare the effectiveness of our solution with other solutions. In the expected result, this evaluation provides a gap analysis in goal modelling solution. Furthermore, the proposed method could facilitate the process of analyzing and prioritizing requirements from multiple stakeholders in the early stages of the software development process.


I. INTRODUCTION
The process of requirements analysis is carried out not only to define customers' needs, objectives, and functions, but also to optimize performance requirements, and this requires synthesized solutions. The process of analyzing requirement still becomes the root cause of problems in the development of software project [1]. It is a challenging process to deal and achieve with multi-stakeholders' requirements at one single time in requirements analysis [2].
Catering to multi-stakeholders is one major theoretical concern in requirement analysis for many years. In developing a large and complex system, different levels of stakeholders from several departments need to share information and communicate with each other. All stakeholders' needs must be determined clearly from the very first phase of the development process. It is difficult for a requirement engineer to deal with all collections of need [3; 4; 5] that come from different stakeholders with their own desires about the system being developed. Each of the stakeholders plays an important role which determines the success or failure of the system development. Under such circumstances, Christopher [6] holds the view that the increment of stakeholder list will lead to unmanageable requirement since each requirement engages with different level of risk. Besides, the lack of determining multistakeholders' role should be taken seriously in a large and complex system development.
Throughout the requirements analysis process, goal modelling represents the relationships between stakeholders' role where one goal will require one stakeholder's role or depends on the other stakeholders' role to provide the information needed. Goals have been applied by Kenneth et al. [7] who performed a goal sketching technique that emphasizes the presence of assumptions and distinguishes them from the various system elements to be constructed. However, the goal sketching used does not describe the requirement from multiple stakeholders. Vikas and Guillaume [8] stated that needs, requirements should be traced from the various rationale stakeholders with their preferences because multi-stakeholders may cause potential traceability errors and reduce the quality control of the system development.
Other approaches used an obstacle analysis [9] which is crucial as the analysis makes requirements more adequate and complete. On the other hand, Ashila et al. [10] have proposed a new representation of goal modelling that visualizes and assesses multiple stakeholders' commitment.
Although all the above techniques have been applied in the industry nowadays until now requirement analysis process still become the root causes of the software-project delays, overruns, and failure in systems development [1]. Under those circumstances, this paper focus on how to evaluate the effectiveness of improvement of goal modelling that has been made compared to the previous work. Thus, in this paper, the objectives are two folds: i) to demonstrate the implementation and comparison between a goal modelling proposed by Kenneth et al. [7] and role-based goal modelling by Ashila et al. [10] and ii) to evaluate both the above goal modellings using NIMSAD framework. This paper has been organized as follows; Section II explains the implementation of the case studies and the goal modelling representation, Section III briefs on the analysis and discussion of the result findings and finally Section IV summarize this paper.

A. Material-The Case Study of Labour Management System
In LMS application, labour module that involved multistakeholders is chosen since it has been deployed in three different level of operations which are headquarters (HQ), region and branch. Fig 1 demonstrates a segment of business workflow in managing labour requisition. There are five direct stakeholders (those who use the system) identified. From the business workflow, each of the processes is extracted and transformed into goal realization graph. The goal realization graph is constructed using goal modelling solution by Kenneth et al. [7] and Ashila et al. [10]. Both goal modelling solution has been compared in terms of degree of feasibility, adequacy, and risk. Ashila et al. [10] have improved the origin goal modelling by Kenneth et al. [7] by adding a new representation of multiple stakeholders. Figs. 2(a) and 2(b) shows the goal graph representation that has been constructed based on given business workflow in Fig 1. The significant difference between both goal modellings is a notation of ROLE in her goal graph representation. Furthermore, the difference leads to the different goal graph formation as well as different priority and stakeholder's commitment in analyzing the requirements. Therefore, the assessment of the recent goal graph has some slight changes.
In the assessment process, four confidence factors are adopted from [7], [11] which are: (i) assumptions (ASSUME), (ii) achievability (ACHIEVE), (iii) stakeholder's mandate (MANDATE) and (iv) refinement (REFINE). This study requires an expert's rating of the confidence factors in order to obtain the degree feasibility and the adequacy attached to each goal. , the feasibility (FEASIBLE) assessment is based on the weakest link technique that propagates the goal towards the root from the leaves. The FEASIBLE is then determined using both the ASSUME and ACHIEVE ratings. For instance, if the lowest sub goal ASSUME = HIGH, ACHIEVE = MEDIUM, then the upper subgoal is determined as FEASIBLE = MEDIUM. This may be supplemented using the following rules: • If lower subgoal has ASSUME = LOW and ACHIEVE = HIGH then Upper sub goal/root goal has FEASIBLE = LOW.
• If lower subgoal has ASSUME = HIGH and ACHIEVE = MEDIUM then Upper sub goal/root goal has FEASIBLE = MEDIUM.
• If lower subgoal has ASSUME = HIGH and ACHIEVE = HIGH then Upper sub goal/root goal has FEASIBLE = HIGH. Meanwhile, the adequacy (ADEQUATE) assessment is calculated based on parent goal (root) towards their leaves. According to Kenneth et al. [11], ADEQUATE assessment cannot be based on the combination of one single REFINE rating and one single MANDATE and should traverse from the root towards the leaves with the assumption that a possible lack of confidence in the ratings of REFINE and MANDATE. In this study, each of the subgoals has been given the value of REFINE and MANDATE by an expert using the value of NONE, LOW, MEDIUM and HIGH. Subsequently, each subgoal is given a rating based on REFINE and MANDATE profile matrix as tabulated in Table 1  Once the FEASIBLE and ADEQUATE ratings have been propagated through the goal-realization graph, the risk of each subgoal is identified. According to Kenneth et al. [11], the higher the fraction of subgoals in the PROCEED zone, risk in the requirement of the project would be lesser. The CAUTION zone requires great concern due to the subgoal may be insufficient or infeasible in order to be implemented. A higher number of leaf goals in the DO NOT PROCEED ZONE would be a cause for further requirements work by the requirement engineer to get more information about the mandate and to redefine the requirement. Table 2 shows the risk identification as a result of the value of FEASIBLE and ADEQUATE in the goal-realization graph.
In addition , Fig 3(b) has shown another improvement of goal graph representation by adding new notation, named as data element. Ashila et al. [10] claimed that data dependency has an impact in goal formation in which one data that is attributed from several requirements may be contributed from multi-stakeholders and goals as well. Thus in Table 2, the assessment of role-based goal modelling by Ashila et al. [10] has been evaluated as well in terms of priority (PRIOR) and conflict (CONF) in order to estimate the degree of the data complexity of each element. Table 2 tabulated the role-based goal modelling and shows the result of HIGH complexity for goal A.2.1.1. It shows that DATA attached to this subgoal carries important information in order for the other subgoal to be initiated. In this study, the importance of the inflow and outflow of each data element is considered based on the role of the stakeholders. The higher the proportion of the goals that fall under the COMPLEX zone the higher is the chances of risk to the project. It will cause greater concern for the requirement analyst to assess the inadequate requirement in the project.

B. Normative Information Model-based System Analysis and Design Evaluation
Normative Information Model-based Systems Analysis and Design (NIMSAD) framework is a systemic framework for understanding and evaluating problem-solving processes methodologies in general. This framework was developed through action research used to evaluate both soft and hard methods. NIMSAD framework is based on a general model of the essential elements of a problem situation and informal interconnections and relationships [12]. It is an action research evaluation framework, which enables understanding of subjective processes. Fig 4 shows the comparative evaluation framework used in this study [13]. The framework explains the list of criteria needed to measure the effectiveness of role-based goal modelling during requirements analysis process. The list of criteria is divided into three categories, namely: general concepts, modelling, and analysis. In the context criteria, the role-based goal modelling is investigated from the viewpoint of the general concepts; while in the elements criteria, the process is viewed for the essential content itself. As for user criteria, the role-based goal modelling is described in the centre of the users targeted for the process, whereas the validation criteria relate to the validation of the result and the maturity of the model against the target domain problems. Table 3 shows the list of questions to describe the criteria guideline.   Table 3 shows the comparative evaluation description that has been used to evaluate the goal modelling between Kenneth et al. [7] and Ashila et al. [10] based on NIMSAD evaluation criteria.

A. NIMSAD Evaluation
Both studies aim to appraise the confidence level of the stakeholders' engagement that might attach in the analysis demonstrate in the goal graph representation. As a result, the risk of each goal is assessed in order to determine whether the related requirements are essential, feasible and adequate to be implemented. If there is the appearance of greater risk, requirement engineer could re-examine the goal and its requirements before it can be brought to the other stage of development. In addition, role-based goal modelling has attempted to visualize the role of stakeholder for each goal involved for better identification of who needs more attention and precaution while handling the requirements analysis. Instead of notation of assessment that has been introduced in the origin goal modelling [6], role-based goal modelling suggested for more notation to be used which are the role of stakeholders and data elements.
Both goal modellings were firstly motivated from KAOS model [14; 15]. Nevertheless, role-based goal modelling has extended the design by improvising further goal modelling by Kenneth et al. [7]. Role-based goal modelling is the systemic features or component of the system in the abstract level which specifies the role and its risk assessment. Goal graph allows hierarchical decompositions of goals. The identified goals guide the subsequent activities that influence decisions that have to be taken during the requirement refinement.   Table 4 presents the different concepts between goal sketching by Kenneth et al. [7] and role-based goal modelling by Ashila et al. [10] Since requirement must be clearly, consistently, unambiguously and should be welldefined, it is important to analyze multiple stakeholders. Multiple stakeholder role engagements have been seen as a risk factor for project success. Analyzing requirement is important to improve the selection of high feasibility requirement to a level of detail sufficient for system design phase. The role-based goal modelling significantly highlights the representation of stakeholder role oriented identification in each goal in goal graph formation based on requirements gathered from the stakeholders. Besides, rolebased goal modelling is applicable to the large and complex system development requires a reliable process of analysis requirements.

C. Analysis on Modelling Criteria
The construction model of role-based goal modelling was motivated from Kenneth et al. [7]. However, the hierarchical tree of role-based goal modelling was attributed to the process desired from multi-stakeholders. The hierarchy of high-level goals may involve key business processes that must be accomplished by multi-stakeholders. The constructed goal graph was then allowed hierarchical decompositions of goals. The identified goals guide the subsequent activities that influence decisions that have to be taken during the requirement refinement. Furthermore, the role-based goal modelling has extended the model by portraying the risk assessment together with the data complexity assessment (Table 5).

D. Analysis Criteria
In role-based goal modeling, the goal of the assessment is to appraise what confidence the stakeholders might attach in the analysis expressed in the goal-realization graph. The focus is to assess stakeholders' representation based on the stakeholder's role involved in the system-to-be. Furthermore, the risk assessment on work or task is essential during requirements analysis to determine whether it is feasible and adequate to be implemented. If there is an appearance of risk prediction specifically when any data or entity is contributed from multiple stakeholders, requirement engineer could reexamine the requirement before to the next stage of development. Therefore, Table 6 presents that the role-based goal modelling has extended the analysis criteria by looking at any data integration amongst stakeholders.

IV. CONCLUSIONS
In summary, this paper has made a comparative study between the role-based goal modelling by Ashila et al. [10] and the previous work done by Kenneth et al. [7]. The results for both works have been analyzed, and our solution proves able to identify requirements that could have potential risk. Subsequently, implementation has been extended to other case study known as LMS. It is found that the role-based goal modelling with data is able to demonstrate the importance of the stakeholders' role during the development of software project. The complexity being determined could facilitate the degree of the data dependency between the sub-goals.