e grand vision is that with completion of a method, a button press will generate a software system that realizes this method in full. We are not there yet. Method authoring is a long and excruciating process. The resultant method definition is complicated, and typically spans hundreds of different kinds of contents, each with its tangled life cycle prescription.
We do not have the means yet for fully automatic translation of these into a software system. But, much of the remaining manual work is routine, and more importantly requires absolutely no programming.
In principle, the approach we offer is 1) To augment SPEM with further meta-modeling capabilities to support enactment at the level we seek; and 2) To augment method authoring tools with the capabilities to instantiate the richer model, i.e., the means and editors which will produce an enhanced method, specifically, including
• A data model for “structured deliverables”; and
• An enhanced state machine description of lifecycles for these deliverables—in particular we provide the means for changing the structure of the deliverable in accordance to its state in its own lifecycle and the current state of the project.
The original SPEM allowed a method to define the kind of tool to be used in each task. For example, in OUM, might be recommended that the requirement gathering task is carried out with Doors12 or RRC13, while HPQualityCenter14 or Rational Function Tester15 are recommended for testing tasks. Another enhancement we offer is that we allow tools can be defined not only for method wide phases, but also for the lifecycle of each deliverable. This makes it possible for the consultant assistant to not only create test cases (which are kind of structured deliverables), but also pump these into Rational Team Concert for tracking and to Rational Quality Manager for further management of the test case.
Currently, we have not implemented yet an extension to the method authoring tools. Instead, our OUM and SAP instantiation of our approach was carried out manually, relying on knowledge extraction sessions with Subject Matter Experts (SME)—individuals who are proficient with the method and its application, almost at the level of the original method author. In these sessions, we capture their view and understanding of the deliverables, in the form of structured deliverables, and the relationships among these. Further, these sessions are to extract knowledge of the lifecycle of each of the deliverables and changes to its structure during the lifecycle. The output is an annotated model in Eclipse’s EMF (Core)16 format—from which the process of generating the environment starts.
2.1. Details of the Meta Model
Consultants’ documents are complex work products with a lot of text, lists and tables as well as images for flows and diagrams for architectures, etc. In this section, we describe in greater detail how these deliverables are made into “structured deliverables”.
Figure 2 depicts the meta-model we use. This metamodel can be instantiated to a specific model for any document artifact. An adaptation of our system to a specific method is carried out by instantiating the meta model for all types of document artifacts that the method defines.
A principal entity in the figure is Method Deliverable, which represents a sort of deliverables of a method in a raw, unstructured form, e.g., the “conversion strategy document” sort of documents, which is part of the Oracle Unified Method. In contrast, stands the Structured Deliverable entity, which serves a similar purpose, except that it captures commonalities of deliverables of the same sort. For example, the one-to-one association link between these two manifests our desire to replace as much as possible entities of the sort of Method Deliverable by Structured Deliverable entities.
As shown in the figure, each Structured Deliverable has a number of Sections. First of these are the mundane Rich Text Section, Image Section, and Document Management Section (in charge of storing author name, versions, title, etc.). It should not be forgotten that this list occurs at the meta model level; a specific instantiation of Structured Deliverable will typically require that specific, named sections of a certain type should occur. For example, a use case deliverable (which is an instance of Structured Deliverable), will require a section named “use case details” which would be of type Rich Text Section, and in deliverable of type Business Process would require a Business Process Flow Diagram, which is of type Image Section. Form Section serves a similar purpose, aggregating various scalar values and attributes which may be associated with a Structure Deliverable, e.g., a Requirement document, would include a form with fields such as Priority.
The References Section makes it possible to associate arbitrary files and resources with the deliverable, either directly as an attachment, or indirectly as a Unified Resource Identifier (URI).
More interesting is the Cross Deliverable Section, which makes it possible to express m-to-n sort relationships between deliverables as dictated by the specific methods. For example, according to the Oracle Unified Method, Business Process Model deliverables come in two varieties, level 1 and level 2, and each level 1 model must include a number of level 2 models as well as Project deliverables, and will accordingly include at least two Cross Deliverable Sections.
Finally, of particular interest is the Custom Section, which can be thought of as the equivalent in the modeling of deliverables of prototypes as in the Self17 programming language and dynamic typing. Specifically, an instantiation of any Structured Deliverable, may allow for the consultant who creates a deliverable of this kind
Figure 2. A meta class diagram for the structured deliverable notion.
extra freedom, in that this individual will be able to add any number of Custom Sections which may take a nested tree structure. For example, the Strategy Document and its subkinds, e.g., Application Strategy Document, contain, in addition to their own idiosyncratic list of structured sections, a Custom Section, which allows for a creative, yet partially structured description of the strategy.
2.2. Example: The Key Performance Indicator Deliverable
In this section, we show how Figure 2 is instantiated for a certain deliverable, specifically the Key Performance Indicator (KPI)—one of the most central deliverables of the OUM.
A KPI is a measure of performance specifically related to critical success factor or business goal, which is used by management to steer and improve operations.
We are mostly interested in KPIs that pertain to the value a customer gains by implementing the Oracle solution. For example, introducing an enterprise application to a bank, may include a KPI of the time to process a check deposit. There are also KPIs prescribed by OUM for the consulting phase, e.g., total design time, total cost of deployment, etc.
IBM sell of an enterprise application to a customer begins by comparing the current customer KPI value with the standard or usual value in the industry to which the client belongs. The difference between these two values is what guides the decision of which business processes are to be taken over by the Oracle application and which changes of internal processes are to be implemented.
A customer engagement thus begins with a definition of cross-deliverables, which connect each of the KPIs with the pertinent business processes.
KPIs can be used both during consulting, where they yield current projections on customer benefits, and after deployment, times at which they reflect actual benefits achieved.
Using KPIs after deployment is a matter of feeding into this definition numerical values obtained from running the customized Oracle application, as implemented by the programmer according to business process specification written by the consultants as per the OUM. The challenge that MATCON meets is that of updating the KPI numbers even during consulting time while some of the business processes are being adjusted.
Figure 3 presents the instantiation of Figure 2 that we use for upgrading the KPI notion from deliverable to structured deliverable.
In the top left corner of the figure we see class KPI, which is an instantiation of the meta-class Structured Deliverable. To its right we see that a KPI is associated with States, an instantiation of the meta level Lifecycle. The particular instance of States associated with a particular instance of KPI, defines the method phase for
Figure 3. A class diagram for the KPI deliverable.
which this KPI applies.
Below these we see that a KPI has a number of sections, all of which are instantiations of appropriate section classes in the meta model. Specifically, the figure tells us that a KPI has these sections:
• DM (an instantiation of Document Management);
• Overview (an instantiation of Form including three instantiations of the Simple Attribute meta class);
• Three rich text sections: Description, Calculation Parameters, and Calculation Formula;
• Attachments (an instantiation of References); and, most importantly• Five Cross Deliverable sections which connect the KPI to other structured deliverables.
Of special interest are the Phase KPI Target and Project Benefit cross deliverable links. These two links make it possible to automatically update overall project benefit based on the numerical value of the performance indicator and the phase KPI target: different KPIs come to play depending on the phase the project is in. Thus, in each phase, the project benefit is computed based on a different set of KPIs.
Since our meta model is concerned with documents rather than formulae, it cannot represent this automatic computation. Instead, our code generator has a hard coded component which is responsible for generating code for automatic updates of the project benefit out of the KPI and the phase KPI target, regardless of the meta model and models fed into the generator.
2.3. Model Transformation
We now turn into the description of MATCON model transformations. According to Czarnecki and Helsen’s taxonomy of model transformations, our approach is of the hybrid nature. There is 1-to-n correspondence between source and target elements. Also, our transformation is what is called structure driven.
All model transformations we apply are implemented in XSLT18 using variables, logic and parametrization and XPath19. Recall that the model of a specific method and the model of the structured deliverables of the specific method are recorded in an ECORE representation. This representation is the starting points for the application of Matcon. We also note that the ECORE model uses various annotations to guide these transformations.
Figure 4 shows how these inputs are processed, and how a full consultant supportive environment which enacts the method is created as an output of the transformations.
The transformation process starts at the left hand side of the figure, which includes a method (e.g., OUM, ASAP or SOMA) specification in the SPEM format, along with its MATCON augments which are recorded in annotated ECORE.
This recorded information is then pumped through a number of largely independent XSLT converters, which produce the various output products at the right hand side of the figure.
In particular, we have the following converters:
• Model-to-DB converter which generates a schema of the database and scripts to create it for storing the information captured by consultants while working on their deliverables.
• Model-to-RESTful services converter which generates Java classes implementing REST based web services over database to get and post information captured by consultants and stored in database, e.g., a specific KPI and all its parts.
• Model-to-GUI converter which automatically generates Java script dojo widgets for information entry editors (for particular structured deliverables) and sections (for particular parts of structured deliverables), e.g., KPIEditor.js and KPIDescriptioRichTextSection.js files.
• Model-to-External tools Services and Interfaces converter which generates the Java and Java script code, for example, for implementing integration with RTC, RQM, RAM and Oracle BPA tools.
• Model-to-Project Management transformation converter which generates the three parts of our project management engine, namely a dashboard definition, work items definition and project definition.
Later on dashboard definition makes it possible to instantiate dashboards to track projects working in accordance with the method. Work items definition allows instantiating particular work items to be assigned to consultants. Project definition allows instantiating roles, phases and other project relevant data.
• Model-to-Document Generation Services converter automatically generates required documents with appropriate styles and content.
• Model-to-Asset Repository Services and Interfaces converter generates code which implements various duties required for model-driven asset repository, e.g., contextualized search functionality, harvesting, cleansing, etc. services relevant to particular method assets.
In this section we shed additional light on the implementation of the MATCON approach to produce IBM’s solution workbench for Oracle.
The Oracle Unified Method (OUM) is Oracle’s standards based method. Oracle claims that OUM is rapid, broadly adaptive, business focused and that it includes project and program management framework20. Yet, with 7 phases, 105 activities, 347 tasks, 1649 steps, 62 different roles, 135 templates and 194 artifacts, the method is far from being simple.
IBM GBS adapted OUM so as to make it possible for IBM consultants to apply the method more effectively. First, Oracle’s method definition, including overview materials, guidelines, templates, tailored work breakdown structure, was reformulated using the Object Management Group SPEM specification. Second, IBM augmented the method with information drawn from Rational Unified Method and IBM’s own practice and experience in applying the method. The result is called IBM OUM Methodology.
The entire process was carried out using the Rational Method Composer (RMC).
The application of the MATCON approach was carried out by extracting method and deliverables information from human experts. This knowledge extraction process lasted about three months, and involved a MATCON author questioning asset experts, method experts and practice experts, and recording this extracted information in the ECORE format.
Over a hundred deliverables including their lifecycle were thus captured in our meta model representation. Later on we applied the transformations described above to generate a full blown Jazz based environment , a global collaborative platform.
We used Rational Team Concert (RTC) (one of Jazz’s applications) for the project management engine, including
Figure 4. Method to model to environment.
dashboard capabilities and work item management. RTC was also used for implementing a discussion component to the work product deliverables, and for implementation of their approval cycle process.
Rational Asset Manager (RAM) was employed to manage the reuse of the structured deliverables. The APQC/PM process map was used to identify the process we are working on. We used Rational Quality Manager (RQM), while making our Use Cases into Test Cases which can be tested using RQM.
Figure 5 shows an example how one structured delivery—Business and System Objective (BSO) is being transformed into a working environment using MATCON. The figure also shows the rather polished interface for information entry for each model entity. This interface allows consultants to add, delete and edit information and links to other entities defined in the system.
Traversing the figure left to right, top to bottom, we see the inception of the BSO in the Rational Method Composer. Next, this description is enhanced and given an annotated ECORE representation by MATCON. To generate this representation, knowledge was extracted from the “subject matter expert”.
Applying the XSLT converters we generate a full blown consultant assistant, which in a web-2.0 collaboration fashion, supports the lifecycle of the BSO, including whatever editing operations are necessary. Working on this artifact automatically creates a Work Item representive in the Project Management environment (RTC).
In the figure we see a specific instance of the BSO, a General Ledger and Account Payble document. Furthermore we see that this BSO object connects to Rational Team Concert, which manages the associated deliverable and its approval lifecycle.
Note that the deliverables of this BSO, just as all deliverables are generated automatically, and mapped to the formal deliverables specified by the method for the project. These are the same unstructured documents produce with the absence of MATCON. This familiarity saves time for the consultants and guarantees that all project documents will be consistent with the template.
We would have liked to run a controlled experiment, in which we carry out an enterprise application implementation (say), with and without MATCON. This wish is futile. Other than the difficulty of keeping all other contributing factors equal, we cannot forget that the introduction of a new enterprise application—a typically $10 million project and circa two years duration —is not only traumatic for an organization, it is rather risky with surveys indicating failure rate to be between 20% to more the 50% . The bad experience that the mighty HP had  that caused it to lose $400 million in the process and to let go of four senior executives is not encouraging. And,
Figure 5. MATCON, Method composer and the generation of cosultant assistant.
a recent survey21 tells us that despite the very structured and very planned process that established methods provide, only 40% of companies had their measurable business benefits expectations achieved at more or less satisfying level. Worse, about 40% of companies surveyed had to forego more than half of their expectations.
Instead, in this section we shall 1) provide data on its penetration levels; 2) give measures of the incurred savings, at least in the eyes of the suppliers of consultation services; and 3) explain some of the reasons why MATCON improves the process in a tangible way.
4.1. Penetration and Estimates on Savings
The SAP solution workbench was developed on 2008. The culmination of this project was in a large deployment (probably the largest of its kind) involving 200 consultants, substitute 1100 different IBM applications on a worldwide level introducing one global SAP enterprise solution. This project is at its final stages now.
Development of the Oracle solution workbench commenced at the beginning of 2009. Ten months later it was deployed. Around 150 employees were trained with the system. Currently, it is being used in large scale ERP projects in the US, totaling 45 million dollars employing some 80 consultants.
The use of MATCON enabled IBM to apply a conservative 10% reduction in head count and other expenses in these projects. The decision to apply such a reduction was based on internal estimates of various contributing saving factors.
It was estimated that consultants’ efficiency will improve by 20% - 30% and that IBM’s internal cost to store and manage associated assets will decrease, thanks to 60% cut of redundant or unwanted data.
It was also estimated that the use of standardized solution and environment will lead to 60% - 70% saving in training costs. This savings should allow consultants to focus more on defining business solutions rather than learning tools.
And, by integrating the complete project lifecycle around a single, integrated, toolset, rather than multiple tools for each of the tasks in the consultation, it was estimated that an additional 15% work can be pushed to global delivery centers, with consequent cost saving.
Standardization and unification of the tools and assets across all projects and for the entire lifecycle, enhances flexibility, and is expected to permit consultants to easily and quickly move between projects.
Probably thanks to all of these, it was decided that all forthcoming IBM Oracle projects in the United States will be using it. Overall, the plan is to globally employ it in 30% of all projects in 2010 and reach a 70% penentration level by 2012.
4.2. Improvement on Project Success Rates
What are the factors that determine the prospects of success of an ERP project? One may jump to a conclusion that the answer is subject to heated and not very scientific debate. However, Ehie and Madsen  carried out a study of hundreds of projects in which eight such factors were identified and then their relative importance was determined by means of linear regression. The most crucial (with ≈0.7 Pearson correlation) factor turned out to be top management support.
MATCON promotes managerial support in two ways. First, the web environment that MATCON provides allows realtime monitoring of consultants’ activity across the ERP implementation. Thus, unlike previous implementations of ERP methods, managers can always stay on top of the progress of the project, guiding it away from joining the sad statistics of more than 50% failures.
Second, and more important is the fact that this monitoring is directly linked withand updates directlyKPIs—the estimates (or benchmarks) on financial benefits to the organization22. The Matcon technique provides a solution to most critical aspect of ERP implementation, that is, in tying benchmarking to implementation, and enabling direct tracking of measurable performance indicators through out the ERP implementation and after it.
We thus have that the use of Matcon enables simultaneously risk management and constant monitoring expected benefits.
According to Ehie and Madsen’s scholarly work, the second most important factor detrimental to ERP success is quality of consulting services. We identify three ways in which MATCON should enhance this quality.
1) Better adherence to method. Methods are not only structured. They are also very complicated. Each stage within these includes many components, including purpose, relationships, roles, inputs, outputs, inner steps, illustrations, checklists, etc. Without MATCON, these are mostly words, subject to understanding, interpretation, and good will obedience by humans. MATCON translates much (but not all) of these definitions into a workbench, that encourages consultants to follow these more accurately, and conversely, use an appropriate method adaptation as necessary. It is a Gartner estimate that a full lifecycle enactment is one of the most important factors to derive the full value from an ERP implementation .
2) Greater reuse and opportunity to offer best practices. The structured organization of consultation data that MATCON offers makes it possible, for the first time, for the consulting firm to reuse this data, and let consultants supply clients with best-practice implementation from previous projects.
In the past, IBM’s Oracle team spent thousands of man hours to clean a large number assets, so that they could be used on future projects. Internal IBM Global Business Services (GBS) estimates, MATCON will save 60% - 70% of the effort to harvest clean assets back to the asset repository23.
3) More cohesive team work. Also, this data structuring promotes better full lifecycle application support. That is, more organized communication between the individuals working in different stages of the project: The business analyst team defines the business processes, which are then delivered to the requirement engineers (also called “functional leads”). Without MATCON, this delivery was by electronic mailing of MS-Word documents. With MATCON storing these documents and structuring them, the requirement engineers can place electronic ties between the locations in the requirement documents and the relevant components of the business processes documents that initiated their work. Similarly, MATCON offers a semi-automatic translation of Use Cases to Test Cases and their testing data as generated by the quality assurance team.
4) More effective collaboration with the client’s IT department. Most ERP projects do not materialize in environments void of any computerization. It is important to explore current IT department knowledge, gather requirements, perspectives and current practice, organize these as per the method guidelines, and reuse it appropriately. Again, MATCON structured approach is valuable in making these more effective.
We believe these are pivotal to quality. And, as mentioned above, IBM’s GBS internal estimates are that overall, the quality of the consultant service is expected to improve by 20% to 30%.
The third factor according to this study is appropriate application of project management principles, including scope definition, team setup and responsibility allocation, and performance objectives guided mode of work. It should be clear that MATCON (together with the method it implements) is instrumental to these.
5. Future Work
We stand still to see the application of MATCON to more and more methods.
Remaining research challenges include better tools for elicitation of knowledge from experts, and enhacement of method editors to support MATCON augmented models. An interesting and intriguing issue remains regarding the proper modeling/meta-modeling of a method together with its specialization modes.
We are in process of collecting actual benchmark numbers based on implementations and usage patterns, and of collecting both clients and suppliers answer to questionnaire. Based on these, it would be interesting to see whether MATCON process can be further improved.
We wish to thank the team members of the Solution Workbench for Oracle project from IBM Research Pietro Mazzoleni, Vibha Sinha, Senthil Kk Mani, Richard Goodwin, Rakesh Mohan, Biplav Srivastava, Manisha Bhandar, Juhnyoung Lee, Debdoot Mukherjee, Sweefen Goh, Shyh-Kwei Chen, Pankaj Dhoolia, Amit Fisher, Aubrey J Rembert, Jeaha Yang, Mangala Gowri, Rema Ananthanarayanan, our IBM GBS partners Dennis Conrad and Maharshi Desai for great collaborative work on the implementation of the Solution Workbench for Oracle. IBM colleagues Avi Yaeli, Shlomit Shachor, Chris Sibbald, Kai-Uwe Maetzel, Erich Gamma, Peter Haumer, Bingxue Xu helped both fire up and temper our views. Our families provided patience and support.
*Work done in part while author was on sabbatical leave with IBM Research-Haifa.
1Rational Rhapsody, http://www-01.ibm.com/software/awdtools/rhapsody/
2SAP is the world’s largest business software company with more than 47,578 employees at sales and development locations in more than 50 countries worldwide. site: sap.com
3Oracle provides the world’s most complete, open, and integrated business software and hardware systems to more than 370,000 customers including 100 of the Fortune 100 that represent a variety of sizes and industries in more than 145 countries around the globe. site: oracle.com
5Software and Systems Process Engineering Meta-Model Specification, OMG standard, 2008.
6Manifesto for Agile Software Development, Agile Alliance, 2001.
7Rational Method Composer, IBM, http://www-01.ibm.com/software/awdtools/rmc/
8Eclipse Process Framework, http://www.eclipse.org/epf/
9P2 Toolkit: Prince2 Support Resources and Documentation, http://prince2.privacyresources.org/
10Rational Team Concert, IBM, http://www-01.ibm.com/software/awdtools/rtc/
11Incidentally, the acronym sounds a bit like the word “recipe'' in Hebrew.
12DOORS, IBM, http://www-01.ibm.com/software/awdtools/doors/
13Rational Requirements Composer, IBM, http://www-01.ibm.com/software/awdtools/rrc/
14HP Quality Center, https://h10078.www1.hp.com/cda/hpms/display/main/hpms
15Rational Functional Tester, http://www-01.ibm.com/software/awdtools/tester/functional/index. html
16Eclipse Modeling Framework, http://www.eclipse.org/modeling/emf/
18XSL Transformations (XSLT) Version 1.0, W3C, 1999, http://www.w3.org/TR/xslt
19XML Path (XPath) Language Version 1.0, W3C, 1999, http://www.w3.org/TR/xpath 20Oracle Full Lifecycle Method for Deploying Oracle-Based Business Solutions, Nov 2009.
212010 ERP Report, Panorama Consulting Group, http://panorama-consulting.com/resource-center/2010-erp-report/
22Leading market analysis firms seem to agree that these financial gains (“shareholder value” in the lingo) are indeed tied with these monitoring abilities. In their words (in reference to Matcon): “... outcomes rather than process...” (Gartner 2009), “... link to shareholders value is unique” (AMR Research, 2009), “tying benchmarking to implementation” (Forester, 2009), and “... tie shareholder value objectives down to the screen and field level within the Oracle applications...” (TBR, 2009).
23Note that GBS is peer to, and different from IBM Research, our division.