Project Summary

The goal of the CREWS project is to develop, evaluate, and demonstrate the applicability of methods and tools for cooperative scenario-based requirements elicitation and validation.

Requirements specifications abstractly describe a future real world, usually in a combination of formal, semi-formal and informal representations, which stakeholders and requirements engineers have agreed upon. Involving users and customers during the development of a requirements specification is a generally accepted goal. CREWS perceives the Requirements Engineering (RE) process as a cooperative learning process in which stakeholders and requirements engineers have to understand each other when eliciting and understanding requirements, detecting gaps and inconsistencies by validating the requirements specification, and reconciling differences at technical and social levels.

Scenario-based approaches suppose that mutual learning must be grounded in instance-level use cases or scenarios. A scenario is defined as a sequence of actions or events for a specific case of some generic task that a system is meant to accomplish. It consists of a behaviour and a context describing the agents, the objects, and environmental setting of the system under development. Scenarios promote creativity (e.g. SystemCreativity by Intel, Scenario Engineering by Xerox Parc). However these approaches exist in separate communities of practitioners and researchers and have not found their ways into RE methods.

The CREWS project has developed four related methods and tools for cooperative scenario-based requirements elicitation and validation. Multiple stakeholders are supported in eliciting and negotiating requirements under different perspectives from real world scenes captured in multimedia. Natural language understanding is used to allow stakeholders to elicit requirements from scenarios described by short natural language statements. Validation is facilitated by cooperative requirements animation, and by systematic comparison of the specification with usage test scenarios interactively generated using domain knowledge.


CREWS Research at  Paris 1 - Sorbonne : Objectives, Concepts, and Results

The following presents some of the key objectives, concepts and results of the research developped by Paris 1 - Sorbonne in the CREWS project. The overall approach called CREWS-L'Ecritoire is a requirements elicitation approach based on the exploitation of textual scenarios. The topics covered are the following :

[Objectives of the CREWS-L'Ecritoire Approach]
[Key Concepts and Terminology of the CREWS-L'Ecritoire Approach]
[Results : The CREWS-L'Ecritoire Approach]
[The CREWS-L'Ecritoire Prototype]
[Evaluating the CREWS-L'Ecritoire Approach]
[Engineering the CREWS Method]
Requirements Engineering (RE) is concerned with the elicitation and definition of system requirements. Whereas initial research efforts focused on the requirements definition facet [Spi92, Jon90, Gut93, Rum91, Jac95] and address what questions only, recent attempts have been made to develop approaches that support the requirements elicitation facet. Within these approaches why questions can be addressed in addition to the usual what questions. It then becomes expected to elicit system requirements better meeting the organisation’s goals and needs. Goal-oriented RE and scenario-based RE are two distinct trends aiming at eliciting requirements from an analysis of the wider context in which the system will operate.

The argument of goal-driven approaches is that the rationale for developing a system is to be found outside the system itself, in the enterprise in which the system shall function [Lou94]. RE is therefore concerned with the elicitation of high-level goals to be achieved by the envisioned system [Ant96, Bub94, Dar93], the refinement of these goals [Dar93, Yu94, Rol98] and their operationalisation into system requirements specifying how goals should be accomplished by the proposed system [Ant96]. However, practical experience shows that (a) goals are not given and thus, goal discovery is not an easy task [Rol98, Ant96], (b) application of goal reduction methods [Dar93] to discover component goals of a goal is not as straight-forward as the literature suggests [Bub94, ELE97, Ant96] and (c) eliminating uninteresting and spurious goals is necessary and difficult [Pot97].

Independently of goal modelling, an alternative approach to RE, the scenario-based approach, has been developed. By capturing examples, scenes, narrative descriptions of contexts, use cases and illustrations of agent behaviours, scenarios have proved useful in requirements elicitation in a number of ways : to elicit requirements in envisioned situations [Pot94], to help in the discovery of exceptional cases [Pot94, Rol98, Sut98], to derive conceptual object-oriented models [Dan97, Rum91, Jac95, Rub92], to understand needs through scenario prototyping [Hsi94], to reason about design decisions [Car95, You87] and so on. The argument is that typical scenarios are easier to get in the first place than goals. Goals can be made explicit only after deeper understanding of the system has been gained. The industrial practice survey conducted by the CREWS consortium confirms that scenarios are useful in particular when abstract modelling fails [Wei98]. In addition, since scenarios describe concrete behaviours, they capture real requirements. However, because they deal with examples and illustrations, scenarios are inherently partial and only provide restricted requirements descriptions which need to be generalised to obtain complete requirements.

In order to overcome some of the deficiencies and limitations of goal-driven and scenario-based approaches used in isolation, some proposals have been made recently to couple goals and scenarios together. In [Dan97, Jac95, Lei97, Poh97] goals are considered as contextual properties of use cases whereas in [Coc95] they are used as a means to structure use cases. The goal scenario combination has been used to operationalise goals [Ant96, Hol90, Pot94, Rol98], to check whether or not the current system usage captured through multimedia scenarios fulfils its expected goals [Hau98], to infer goals specifications from operational scenarios [Vla95] and to discover new goals through scenario analysis [Rol98].

The CREWS-L’Ecritoire approach [Rol97, Rol98] developed within the CREWS ESPRIT project uses a bi-directional coupling allowing movement from goals to scenarios and vice-versa. The complete solution is in two parts : when a goal is discovered, a scenario can be authored for it and once a scenario has been authored, it is analysed to yield goals. By exploiting the goal-scenario relationship in the reverse direction, i.e. from scenario to goals, the approach pro-actively guides the requirements elicitation process. In this process, goal discovery and scenario authoring are complementary steps and goals are incrementally discovered by repeating the goal-discovery, scenario-authoring cycle.

[Top]

A Requirement Chunk (RC) is a pair <G, Sc> where G is a goal and Sc is a scenario. Since a goal is intentional and a scenario is operational in nature, a requirement chunk is a possible way of achieving the goal.

A goal is defined as "something that some stakeholder hopes to achieve in the future" [Pli98]. In our approach, a goal  is expressed as a clause with a main verb and several parameters, where each parameter plays a different role with respect to the verb. A detailed description of the goal structure can be found in [Rol98]. An example of a goal expressed in this structure is the following :

Provide verb (efficiently) quality (electricity) object (from PPC producer) source (to our non eligible customers) beneficiary (using PPC network) means (in a normal way) manner
A scenario is "a possible behaviour limited to a set of purposeful interactions taking place among several agents" [Pli98]. It is composed of one or more actions, an action being an interaction from one agent to another. The combination of actions in a scenario describes a unique path. A scenario is characterised by an initial and a final state. An initial state attached to a scenario defines a precondition for the scenario to be triggered. A final state defines a state reached at the end of the scenario. We distinguish between normal and exceptional scenarios. The former leads to the achievement of its associated goal whereas the latter fails in goal achievement.

Requirement chunks classification and abstraction levels : We have introduced three levels of abstraction called contextual, functional, and physical. The contextual level identifies the services that a system should provide to an organisation and their rationale. The functional level focuses on the interactions between the system and its user to achieve the needed services. Finally, the physical level deals with the actual performance of the interactions. Each level corresponds to a type of requirement chunk. As a result, we organise the collection of requirements in a three level abstraction hierarchy.

Relationships between requirement chunks: There are three types of relationships among Requirement Chunks (RC) namely, the composition, alternative, and refinement relationships. The first two of these lead to a horizontal AND/OR structure between RCs. These are extensions of conventional AND/OR relationships between goals. AND relationships among RCs link together those chunks that require each other to define a completely functioning system. RCs related through OR relationships represent alternative ways of fulfilling the same goal. The third kind of relationship relates requirement chunks at different levels of abstraction. The refinement relationship establishes a vertical link between requirement chunks.

[Top]

The CREWS-L’Ecritoire approach aims at discovering/eliciting requirements through a bi-directional coupling of goals and scenarios allowing movement from goals to scenarios and vice-versa. As each goal is discovered, a scenario is authored for it. In this sense, the goal-scenario coupling is exploited in the forward direction from goals to scenarios. Once a scenario has been authored, it is analysed to yield goals. This leads to goal discovery by moving along the goal-scenario relationship in the reverse direction.

The exact sequence of steps of the CREWS-L’Ecritoire process is as follows :

1. Initial Goal Identification
repeat
       2. Goal Analysis
       3. Scenario Authoring
       4. Goal Elicitation through Scenario Analysis
until all goals have been elicited
It can be seen that goal elicitation and scenario authoring are complementary steps and goals/requirements are incrementally discovered by repeating the goal-analysis, scenario-authoring, goal-elicitation-through-scenario-analysis cycle. Each of the three steps of the cycle is supported by mechanisms to guide the execution of the step.

The guidance mechanism for goal analysis is based on a linguistic analysis of goal statements. It helps in reformulating a narrative goal statement as a goal template as introduced in the previous section. The mechanism for scenario authoring combines style/content guidelines and linguistic devices. The former advise authors on how to write scenarios whereas the latter provides semi-automatic help to check, correct, conceptualise, and complete a scenario. Finally, for goal elicitation through scenario analysis, we defined enactable rules offering three different goal discovery strategies, one for each of the relationships between <goal-scenario> pairs. namely, refinement strategy, composition strategy, and alternative strategy. The first of these discovers goals at a lower level of abstraction than a given goal ; the second discovers goals ANDed to the original one ; the last discovers goals ORed to the original goal.

[Top]

In the requirements elicitation process, goal discovery and scenario authoring are complementary activities. Once a goal is discovered, scenario authoring can be done, followed by goal discovery. These goal-discovery/scenario-authoring sequence is repeated to incrementally populate the requirement chunks hierarchy.

The requirements elicitation process can be viewed as a flow of steps : each step starts with a given goal and describes a scenario as a possible concretisation of the goal. Clearly, a step results in a complete requirement chunk. In order to progress, a new goal has to be determined. This is done in the goal discovery activity through an analysis of the scenario. Thus, it can be said that the discovery activities regulate the flow of authoring activities.

It has been shown that flow control is based on strategies [Dar93, Pot94, Sis97]. We identify three strategies, namely refinement, composition and alternative strategies. Upon the completion of a step, any strategy can be dynamically chosen. Thus, there is no statically imposed linear order of the flow of steps. This flexibility in strategy selection is the main advantage of the step-flow process. However, it shall be noticed that goal discovery in this approach is about sub-goal discovery, not super-goal discovery.

The three discovery strategies exploit the three types of relationships identified among requirement chunks in the previous section. Therefore, given a pair <G, Sc> (where G is a goal, and Sc is a scenario) :

- the composition strategy looks for goals Gi which are ANDed to G,
- the alternative strategy searches for goals Gj which are ORed to G,
- the refinement strategy aims to discover goals Gk at a lower level of abstraction than G.
Thus, when a step is completed and the requirement chunk is expressed, the Requirement Chunk Author (RCA) has three options to proceed in the process.

We associate guiding rules with each of the strategies to discover new goals. Therefore, composition (alternative) rules help in discovering goals ANDed (ORed) to G. All these goals are at the same level of abstraction. The <G,Sc> chunk is processed by the refinement rules to provide goals at a lower level of abstraction than G. This is done by considering each interaction in Sc as a goal. Thus as many goals are produced as there are interactions in Sc.

EcritoireTool.gif (8308 octets)

As shown in the figure above, the rules are implemented in the CREWS-L’Ecritoire software environment and automatically enacted on request [Taw97]. This facilitates the use of the approach and limits the effort required to apply it. Using CREWS-L’Ecritoire, the RCA is automatically guided in a flexible manner to elicit requirements systematically.

[Top]
 

Our approach to evaluate CREWS-L'Ecritoire was structured around three questions as follows : The WHAT question
We answered the first question by identifying the main features of the CREWS-L’Ecritoire approach, defining hypotheses to evaluate, and associating features with the hypotheses relevant to them. We identified the six following  features which we believe, constitute a full coverage of the approach:
        - F1 : Scenario Authoring
        - F2 : Goal Formulation
        - F3 : Goal Discovery
        - F4 : Organising the Goal-Scenario Collection
        - F5 : Guiding the Requirements Elicitation process in a Computer Based Environment
        - F6 : Working in a Top-Down Manner
Some of these features have been further refined into sub-features, 10 in total. For each feature/sub-feature we identified one or several hypotheses that are the subjects of the evaluations. We identified 25 hypotheses [Par99]. Each hypothesis represents an aspect of scenario-based requirements elicitation that we expect CREWS-L’Ecritoire to improve. Hypothesis evaluation therefore, tells us whether or not the expected improvements have, in fact, been realised.

The WHEN question
To perform the evaluations we conducted experiments of four kinds : workshops, empirical study, case study and tutorial. We conducted three workshops with the participation of 52 system development experts drawn from French companies [Rol98]. Two empirical studies, one on the use of the 'scenario authoring approach’ [Ben99] and a second one on the use of the ‘goal/requirement discovery approach’ [Taw00] were performed with students, 69 in the first case and 41 in the second one. One industrial case study on ‘business process reengineering’ was conducted in a large electricity supply company [Rol99a]. Finally 6 days of tutorials, based on an intensive use of the tool environment, were performed with different groups of either students or professionals [Par99].

The HOW question
In order to determine whether a hypothesis could be validated or not, we have adopted an approach based on metrics. We identified evaluation criteria and defined metrics in order to measure each hypothesis in a given experiment. This means that the evaluation frame is a 5-tuple of the form :

< experiment, feature/sub feature, hypothesis, criteria, metric>.
There is one result for each 5-tuple which represents an atomic evaluation. These atomic results form the basis of more global evaluations and measurements.
The figure below sums up our evaluation approach and gives an example of atomic evaluation. In the empirical study of the scenario authoring approach, an hypothesis related to the feature ‘scenario authoring’ is that ‘the use of style guidelines leads to scenarios with less terminological errors’. The criterion that can be used to evaluate this hypothesis is the ‘quality improvement’. To evaluate how style guidelines improved the quality of the scenarios terminology in the empirical study, the measure which is proposed is a difference between the proportion of terminological errors per number of actions with and without style guidelines. Overall, the result obtained is yes, because with guidelines subjects make 60% less terminological errors; a deeper analysis of this result can be found in the evaluation reports.

Overview of CREWS-L’Ecritoire evaluation approach

Evaluation Results
A complete report of the evaluation results can be found in [Par99]. The following list provides you with samples of our main findings:

Detailed analysis of the experiments results can be retrieved by downloading our papers from the CREWS official web site:
  • Workshops (ps) (pdf)
  • Empirical evaluation of scenario authoring (ps) (pdf)
  • Empirical evaluation of goal formulation and discovery (ps) (pdf) [shorter version (ps) (pdf)]
  • ELEKTRA case study (ps) (pdf)
  • Overall evaluation results, including tutorial results (ps) (pdf)
  • [Top] The Information Systems community is merely concerned by the need for adaptability and extendibility of existing methods to meet the changing needs of practice. Method engineering [Welk92a], [Harm94] represents the effort to improve the usefulness of systems development methods by creating an adaptation framework whereby methods are created to match specific organisational situations.

    In fact the entire process modelling community realised quite early that even though process models were prescriptive, in actual practice departures from the prescription occurred [Hidd94, Rus95, Wije90, Aae92, Your92]. Therefore, a concerted effort was put in to allow process models to respond to those departures. One approach was to assume prescriptive models and then, modify them to accommodate real processes. This modification could be achieved in two ways. First the extent of deviations from the prescription that could be allowed was modelled as constraints [Cug95, Cug96, Cug98]. Any actual deviation that satisfied the constraint was therefore manageable and the process enactment mechanism could handle it. This way of handling deviations took the prescriptive approach to its logical conclusion : it prescribed the deviations allowed in a prescription. The second way of handling deviations is to allow changes to be made in the prescription as and when they are needed [Dow94, SiS96, Jac92, Fin94, Ban93, Bel94]. Thus, a dynamic change of the basic prescription is allowed.

    An altogether different approach was taken in the contextual model [Gha97, Rol95, Poh96, Bub94]. Here the attempt was to relax the prescription given by a process model. Thus, the process model did not always specify what must be done but contained some specification of what can be done. The process model therefore, contained a number of alternative ways of doing a task and a selection of the particular alternative was done dynamically, depending upon the situation in which the product was found. However, the contextual model could consist of both alternatives as well as prescriptions. Whenever such alternatives were available, the net effect was that the process model could be dynamically built, even as the process was being performed. The major difference between the two approaches is that whereas handling departures from prescriptions is an exception handling activity in the former, selection from alternatives in the latter is a normal activity envisaged in the process model itself and supported by a dynamic selection mechanism. Thus, support for real processes is provided in a more natural way.

    CREWS proposes to relax the prescription of a process model even further [Rol00, Ral99]. Our proposal is based on our experience with the contextual model that we conducted with four groups of master students. The experiment consists of using the ten methods described with the contextual model in [Pli94] to develop application case studies within the process centred environment MENTOR [Sis96]. Our experience was that a key discriminant factor in real processes is the product situation. This situation has a strong bearing in selecting the task best suited to handle it and also the strategy to be adopted in carrying out this task. For example, consider a process for doing requirements engineering using goal-scenario coupling. Assume that a goal G has been elicited. Now, it is possible to either explore alternative goals of G or to write a scenario for it. Thus, the process model must reflect this choice and the requirements engineer would dynamically choose between one of these alternatives. It can be seen that G provides a basis for a discriminant choice in what task is to be done next. Now, consider that a fully developed scenario has been written out and goals are to be determined by scenario analysis. That is, the next task to be done is known. However, it is possible to discover goals that are exceptions or obstacles to G or sub-goals of G using the alternative or the composition discovery strategies. Again, these strategies for eliciting goals need to be reflected in the process model so that the right one can be dynamically chosen depending on the nature of the scenario. Thus, the product situation also provides a basis for a discriminant choice in what strategy is to be adopted in performing a task. Evidently, a process model that captures all alternatives of tasks and strategies is needed to support processes. Such a model needs to be backed up by a dynamic selection mechanism of tasks and strategies. In the paper we propose to represent task and strategies alternatives as a labelled directed graph called a map and provide support in alternative selection through guidelines. The CREWS method has been modelled following these lines and implemented in an electronic format available at the URL : http://panoramix.univ-paris1.fr/CRINFO/CMB/.

     The salient features of our approach are :

    - explicit recognition of the role of strategies in process modelling,
    - a non-prescriptive model of strategies and tasks containing alternatives only from which real processes can be built,
    - dynamic process construction is the rule rather than an exception.
    As indicated above, the non-prescriptive model is a labelled directed graph called a map. The map uses two fundamental notions, a process intention or intention for brevity, and strategy. An intention captures in it the notion of a task that the application engineer intends to perform whereas the strategy is the manner in which the intention can be achieved. The nodes of the map are intentions whereas the edges are labelled with strategies. The directed nature of the map identifies which intention can be done after a given one. The only way in which a process can be built is dynamically, through the use of guidelines for selection among alternatives. Only after the task and the strategy have been decided is there a need for a guideline to achieve the task.

     There are three guidelines associated with the map :

    - intention selection guidelines for determining all succeeding intentions of a given one,
    - strategy selection guidelines for determining the strategies from which one is selected,
    - intention achievement guidelines for defining the way in which an intention can be achieved. Thereafter, the enactment mechanism is invoked to actually carry out the tasks.
    We view a map as containing a panel of process prescriptions from which, by dynamic selection, the particular one that is best suited to the product situations as they emerge is selected. In this sense, the map is a multi-model with dynamic process modelling capability.

    [Top]


    The CREWS Consortium
     
    RWTH, Aachen, Germany (Coordinating Partner)
    Prof. M. Jarke, Dr. K. Pohl

    City University, London, United Kindom
    Prof. A.G. Sutcliffe

    Facultés Universitaires Notre-Dame de la Paix, Namur, Belgium
    Prof. P.Y. Schobbens

    Université Paris 1-Sorbonne, Paris, France
    Prof. C. Rolland

    CrewsTeam.jpg (38105 octets)


    Staff

    Colette Rolland
    Carine Souveyet
    Jolita Ralyté
    Mustapha Tawbi
    Camille Ben Achour

    CREWS Reports
    delivered by CRI, University Paris 1 - Sorbonne
    [All the CREWS reports can also be downloaded from the CREWS website.]

    2000

  • Evaluating the CREWS-L'Ecritoire Requirements Elicitation Process [ps][pdf]

  • Mustapha Tawbi, Fernando Velez, Carine Souveyet, Camille Ben Achour
    Submitted to REFSQ'2000, International Workshop on Requirements Engineering Foundations for Sofwtare Quality.
  • From Conceptual Modelling to Requirements Engineering [ps][pdf]

  • C. Rolland, N. Prakash
    Annals of Software Engineering, Special Issue on Comparative Studies of Engineering Approaches for  Software Engineering; to appear.
  • A multi –Model View of Process Modelling [ps][pdf]

  • C. Rolland, N. Prakash. A. Benjamen
    Requirements Engineering Journal, to appear.


    1999

    1998 1997 1996

    References
    [Follow the links to identify quoted papers which are part of the CREWS reports]


    Last update : 15 Febr. 2000
    This page is maintained by Camille Ben Achour