Due to the rapid growth of technology in recent days, competition in the software market has greatly intensified. Customers now expect more from software vendors, while vendors try their best to satisfy customers’ needs by introducing good products and services to protect their market share.
However, in reality, many software vendors only describe their products from a business perspective to sell their solutions and hide any complexities that might be involved in integrating or customizing their solutions to fit a customer’s environment  . The only aspects that are usually presented to customers are those concerning the functionality of their systems.
Nowadays, many software vendors employ the SOA style  in their solutions, which represents a system as a composition of several functional layers. One of the significant layers in many SOA-based enterprise solutions is the orchestration layer; this captures the actual processes of an organization and models them according to a standard notation commonly known as Business Process Modeling (BPM) notation. BPM is a mechanism that separates all business aspects from the underlying technical aspects of a system to capture an organization’s processes and achieve its business objectives. The paradigm falls within the basics of software design principles that have introduced the notion of a separation of concerns  .
A number of BPM solutions are currently available, from open-source solutions to proprietary ones. However, these solutions only describe general functional characteristics of their products, despite the possibility there might be incompatibility issues with certain environments. This makes the selection of one solution or another a challenging task. Therefore, this work comprised a thorough investigation into the environmental characteristics of BPM solutions regarding their functional, architectural, and usability dimensions to help organizations select the most appropriate solution to fit their environments. This study targeted the most commonly recognized solutions in the Saudi Market because the selection of solutions is currently either based on reviewing information given by vendors about their provided solutions, which are usually very brief and general, or made by following the brand names of solution providers. This work will lead to the development of a supporting tool that will help customers systematically explore and identify the best solutions to fit their environments.
It is important to note that the list of solutions considered for the investigations is not meant to be exhaustive; however, it does cover a variety of solutions currently taking place in the Saudi market that can be used to validate our characterization work and the proposed tool. The solutions that will be covered are: Process Maker, Bonita BPM, ARIS Express, Bizagi BPM Suite, and Oracle BPM Suite.
The remainder of the paper is organized as follows: the second section presents related work and provides an overview of current and previous studies in the BPM field. The third section provides an overview of the main characteristics that were assessed under each of the main category dimensions.
The fourth section then presents an evaluation of BPM solutions against each characteristic in our taxonomy. Finally, a conclusion and future work will be presented and discussed in section five.
2. Related Work
Bernroider and Bernroider  (2008) examined business process management tools with multiple functionality attributes. Their assessment framework was based on the Utility Ranking, and Analytical Hierarchy Process (AHP) methods were employed. The open-source software tools included in the assessments were: ARIS PF, ARIS DP, uEngine, YAWL, jBPM, Runa, and AnaXagora. The authors provided a comprehensive list of eligible criteria for the assessment followed by an evaluation of the tools in terms of their features and development, with a focus on the commercial market leader. Lodhi et al. (2011)  provided an overview of current active research areas and challenges pertaining to business process modeling. To provide a better understanding of the context of the business process their report discussed business process foundations starting from very basic definitions and then described in detail the business process lifecycle from a post-execution perspective. They then discussed active BPM research, which encompasses many areas and helps to delineate future research directions. The report also considered the challenges in BPM and provided suggestions for research needed to assess the effects of business processes and their context on enterprise operations. Finally, their report discusses how changes in technology and the context of business processes require a different set of models to illustrate phases of the business process lifecycle and address current challenges  . Craggs (2011)  published a report for Lustratus Research Limited  that compared three BPM vendors: Pegasystems, IBM, and TIBCO. The report discussed current market needs and assessed each of the three vendors to identify the main differences between them in terms of functionality, characteristics, and solution extension attributes. Their assessment focused on buyers’ needs and assessed the comparative strengths and weaknesses of each of the three vendors’ offerings based on the time to value, TCO implications, risk, and broader value potential. In contrast, the current investigation not only focuses on what vendors provide or what buyers need, it develops a framework for users to evaluate many vendors’ solutions to obtain the most appropriate solution for their environment. Our investigation and Cragg’s report have similarities in that they both focus on the vendors’ functionality. A key difference is that we also cover the architectural and usability categories and different characteristics. Abdulgader et al.  conducted research (2013) on three BPM solutions: ProcessMaker (free open-source version), Bizagi (academic version), and Joget (community version). Their investigation primarily focused on process modeling capability, monitoring, process engine, and business activity monitoring. They compared the three solutions in terms of process modeling, business rules, user and authorization management, forms creation, and database management and creation. Theyemployed the Workflow Reference Model and found that all three systems exhibit satisfactory performance during development  as conducting the case study using each tool was generally simple. Similarly, in (2013) Nafie and Talab  compared two workflow tools: Arabdox and Bizagi Express, focusing on the most important features of workflow automation offered by both vendors. The features compared were: process modeling, business rules, user and authorization management, forms creation, and database management and creation. The case study for this comparison was pension file processes in the National Pensions Fund (NPF) “which pass through many workstations and designed forms with Arabdox process designer and Bizagi modeler”  . Shitkova  (2014) reviewed current research on the usability of business process modeling tools, concluding, “The results of the literature review show, that although a number of research papers mention the importance of usability for BPM tools, real usability evaluation studies have rarely been undertaken”  . Finally, based on the literature review, possible research directions in the usability of BPM tools were suggested.
Whether the surveyed works highlighted the importance of understanding BPM as the selection of one solution or another remains an open question. However, the architectural and usability aspects were not thoroughly covered in the surveyed work in a manner that users can understand and utilize. Hence, a thorough investigation is needed which is the key contribution of this work.
3. Characterizing BPM Solutions
The main scope of our investigation is to facilitate the identification of suitable BPM solutions for a given environment. This will enable customers to examine and select a solution that might fit their environment and fulfil their requirements.
Every BPM solution exhibits a number of characteristics that must be explored prior to considering their utilization. In this section, we highlight the key characteristics that need to be examined to make an informed decision about the suitability of a solution for the environment within an organization.
Table 1 provides an overview of the main characteristics assessed and classified under each of the main category dimensions.
A description of the characteristics is provided in the following sections.
3.1. The Architectural Characteristics
A Platform in an IT field has many definitions: the IEEE/ISO definition describes it as “a combination of an operating system and hardware that makes up the operating environment in which a program runs”  . It is widely known that different products and solutions may require disparate hardware or software environments. It is helpful if the solution is compatible with different OS. Moreover, each solution is based on a specific language or environment; for example, companies that depend on PHP and use Apache prefer a BPM solution that is suitable for these environments such as ProcessMaker. Therefore, it is important for users to identify the environment and programming language that supports specific solutions for implementing services and applications.
3.1.2. Support Integration
To ensure user satisfaction, BPM vendors should consider whether the infrastructure of BPM solutions meet enterprise needs. It is very important that the solution can integrate with existing solutions and key related technologies in the market.
Table 1. Summary of the evaluation characteristics reference framework.
For BPM solutions, integration means “the ability to automatically connect to and invoke backend systems and databases as work items flow through the processes”  .
3.1.3. Support Integration with External Database
Some companies depend on a specific database for their enterprise work and they need to know which database BPM solutions require or support  rather than the solution packaged DB. For example, some BPM solutions support MySQL, Oracle or SQL Server. At the same time, there are other solutions that may not support the same databases or perhaps only one or two of them.
3.1.4. Support Model Reuse
If BPM solutions provide the option of reusing the existing process model rather than creating a new model, the effort needed for business process modeling might be reduced which could lead to an increase in the quality and flexibility of the model. It is rare for BPM solutions to reuse existing process models or objects, but it would be an advantageous feature if the solution, e.g. ARIS Express, could support this feature.
3.1.5. Support Cloud Services
During the last few years many technologies have grown and evolved, among which cloud services have become both extremely popular and important. The cloud has many definitions. For instance, the NIST identify cloud computing as “a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction”  .
Cloud services must be considered for use in BPM solutions to make them faster and more powerful. By using such services, it becomes possible to easily access and use business process solutions over the Internet to manage, create, modify and customize business processes online. This means that the cloud can offer high availability.
There are many BPM solutions that support cloud services such as ProcessMaker, Bizagi BPM Suite, and Oracle BPM Suite. However, some BPM solutions support cloud services features in their enterprise/commercial products but not in free or open source versions.
3.1.6. Support Compatibility with Service Oriented Architecture (SOA) Model
As mentioned previously, many BPM software vendors now employ the Service Oriented Architecture (SOA) style  in their solutions. The concept of SOA is based on the principle of building and developing reusable business services that are available for consumption by any application as a standards-based service. Therefore, SOA technologies can address the modern requirements of organizations.
Each vendor approaches SOA based on their needs; some focus on external integration, some focus on workflow, and some may focus on internal integration. Users (business users and IT professionals) of BPM solutions should consider SOA if they are modeling processes because BPM has become much more powerful with SOA.
There is a big gap in traditional software development approaches between modeling processes and IT, SOA can therefore align IT with the business process to effectively reduce this gap  . If a BPM solution is compatible with SOA, it may enable process modeling to be conducted using different solutions or tools.
3.1.7. Support Plugin Architecture
To achieve flexibility and extendibility of BPM solution features, it should support plugin architecture. A plugin can help the solution easily integrate with any new features that are not yet supported or available in the tool without breaking the tool’s fundamental core features.
3.1.8. Support Modeling Standard
Like the modeling solution tool, for which there are many choices, the modeling language with its semantics and syntax must also be selected. It is essential for the BPM tool to be able to verify and support different modeling notation standards and languages, e.g. BPMN 2.0, BPQL, BPEL, EPC, UML, and XML. Some BPM solutions can support multiple modeling standards while others may only support one.
3.1.9. Support Portability
It is useful for BPM solutions to be portable on various platforms and devices such as smartphones and tablets (not restricted to the desktop). This will enable users to reach their business models using any device at their convenience.
We can find various BPM solutions available for many platforms and devices, although somemay only be available for one type of platform. For example, Bonita BPM is available for Windows, Linux, and Mac operating systems while Oracle BPM Suite is available for Windows, Linux, Mac, Solaris, AIX, HP-UX operating systems, and mobile devices.
3.1.10. Support Directory Management (Integration with Active Directory)
Active Directory (AD) is an advanced hierarchical network directory management service that was introduced and developed by Microsoft for Windows domain networks. It provides secure, structured, hierarchical data storage for different kinds of objects in a network such as users, systems, sites, computers, printers, and services. Moreover, it allows administrators to manage and organize those objects. By using Lightweight Directory Access Protocol (LDAP), which is a “set of protocols for accessing and maintaining information directories over an internet protocol network”  , it provides fast access to all the information and objects that form part of the organization or enterprise   .
Integration with the AD and the LDAP enables the user to take different actions, logging directly into an LDAP or Microsoft AD server by authenticating. With the user’s list in AD or LDAP, BPM solutions might also support importing or synchronizing a user’s information and comparing the user’s lists in the solution.
3.2. The Functionality Characteristics
3.2.1. Support Process Modeling
In our study, we focus on business process modeling (BPM). Therefore, the solutions in the comparison should essentially provide and support this feature, otherwise it will not be considered a BPM solution and will only be conceived of as a design tool.
We have previously defined Business Process Modeling based on Bloomberg and Schmelzer’s definition, which describes BPM as “a set of practices or tasks that companies can perform to visually depict or describe all the aspects of a business process, including its flow, control and decision points, triggers, and conditions for activity execution, the context in which an activity runs, and its associated resources”  .
3.2.2. Support Business Process Analysis
Business Process Analysis (BPA) “is the analysis of various business operations classified into processes, or series of related tasks, where observation revolves around the specific ways in which these processes happen along a life-cycle from beginning to end”  . It is used to generate the information necessary for organizations to make decisions about business processes.
Business process analysis is essential for presenting how the business meets its fundamental objectives. Moreover, it is used to understand the context of work and each individual process. Business process analysis can also help in understanding and capturing business problems, identifying potential improvements within the process, and determining whether they should be carried out.
Business process analysts perform analysis at the design stage using BPM solutions. However, the solution is sufficiently powerful to analyze complex processes that would increase the quality and improve the performance and productivity of the business.
There are various types of business process analyses used to assist business leaders that will enable them to analyze processes in several ways, such as through diagramming, advanced visual interfaces, flow charts, benchmarking, and through an analysis of performance and scalability.
3.2.3. Support Process Mapping
Process mapping was developed to facilitate analysis of the business process. It is a canvas area in which business processes are visualized to present what the business does by considering specific standards, responsibilities, and roles, e.g. designing and using Business Process Model and Notation standards (BPMN 2.0). The visual representation in process mapping provides better analysis and understanding to improve business, which makes the enterprise much more effective. This visual representation also shows the activities of all the participants in a process, which might include people, computer applications, departments, roles, and external enterprises or organizations  . Thus, process mapping makes understanding of the process much easier without the need to read a long description.
The difference between BPM and process mapping is that in process mapping the tool or solution concentrates more on the documentation because it presents and demonstrates how the work is carried out by understanding each process and how this relates to other processes. At the same time, it is not necessary to present details regarding how this should be done.
3.2.4. Support the Creation of Process Documentation
It is very important that a user has the ability at process level to include information about the processes and detailed information and descriptions of the elements in the model. This will make the process document easy to read and understand.
Once the model and documentation are complete, the modeling software can generate complete documentation for the business process in any format the user prefers, e.g. .doc, .ppt, html, pdf, and SharePoint. Moreover, it is important that those generated documents can be downloaded for storing or printing or for sharing with other users in the same business.
It is not necessary for the BPM solution to support all types of document format, but it would be much better and more powerful if the solution can support various kinds of format rather than being restricted to Portable Document Format (PDF) or MS Word (DOC).
3.2.5. Support User Management and Authorization
User management and authorization is a must-have feature in BPM solutions; it enables the customization of processes by creating roles, groups, and departments. This allows users to assign roles that match their actions and functions in the organization or enterprise. The roles of users define what actions can be carried out and which parts of the BPM solution users can access.
A user with an administrator’s role has all the permissions by default, which means that they can carry out all actions and access all parts of the BPM solution. Those actions include creating, editing, and deleting users and processes.
3.2.6. Support Management of Documents
Document management refers to the ability to integrate and control the process of documentation and saving documents. In BPM solutions, users should have the ability to easily save and retrieve documents and files that have been uploaded or previously created within a process.
3.2.7. Support Executive Dashboard
The dashboard can be defined as “a collection of key reports in a visual format that is designed for various user functions”  .
In BPM solutions, the dashboard should provide robust real-time process monitoring, analytics, and create reports and charts to show critical business measures and KPIs so that a business can be run and managed easily. An administrator can easily use the dashboard to see what role each user has and check up on productivity in the organization. The dashboard is essential for measuring the effectiveness of processes and determining how to improve them in the future.
3.2.8. Support KPI
KPI stands for Key Performance Indicator; it is a type of performance measurement. “A KPI is an element usually shown in a dashboard, which allows decision makers to see data about the performance of users and processes”  . The success of an organization or the success of a specific activity or process is evaluated using KPIs.
3.3. Usability Characteristics
3.3.1. Usability of the User Interface (UI)
The User Interface (UI) is the “visual part of computer application or operating system through which a user interacts with a computer or a software”  . The User Interface can determine the commands given to computers or programs and how the information is displayed on the screen. Moreover, it can determine the usability and functionality of software for the user.
The user interface in BPM solutions displays work information and items in the workplace to users during runtime. Models, charts, graphs, and maps are displayed to represent data; therefore, the user interface is an essential part of BPM solutions. A poorly designed user interface will decrease the value of a powerful tool. The interface should provide the user with an enhanced description for components such as toolbars, menus, fields that require input, and dialog boxes.
Most BPM vendors provide a customized user workplace for their users. This contains and displays all the objects that form the user interface such as an organized toolbar and tooltips with explanations.
We will evaluate the user interface in terms of the support and efficiency of displayed items and the availability of toolbars, menus, and drag and drop menus that help during the modeling of the business process.
3.3.2. Support Business Process Templates
BPM solutions sometimes offer various business process templates for their users; if the user can find a template that matches their needs this will reduce the cost and effort of building a new process model from scratch. Business process templates are sometimes offered by BPM vendors for free; however, some may not or may provide them only for their enterprise or commercial version.
In our study, we will evaluate the availability of a business process template that can facilitate the modeling of business processes to users, saving them time and effort.
3.3.3. Support Learnability
Learnability is a fundamental usability characteristic and plays an essential role in the success of software products. Shitkova defines learnability as the “ease of learning the functionality of software”  . Thus, learnability can be measured by how easily the user can learn about and deal with the functionality of the solution. BPM solution vendors should design solutions to support users in terms of their capabilities and needs. Moreover, vendors should provide clear documentation, tutorials, and webinars to users to help them learn and easily find the information they need about the solutions.
3.3.4. Flexibility of Modeling Solution
BPM solutions should be flexible and easy to use: they should be clear and understandable for all kinds of users. Flexibility in using the solution is an essential aspect of this and is one of the usability requirements of any form of software.
Providing a familiar interface with features from BPM vendors can facilitate the flexibility of solutions. Moreover, flexibility can be embedded into the interface layout by making the commands and menu options easy to find within business models and creating forms, thus providing easy integration with other systems and solutions and easy management for models, users, and documents.
This characteristic will be measured by comparing selected BPM solutions while using them to create the process model for the case study in our work.
4. Evaluation of BPM Solutions
To evaluate the solution against each characteristic and category, we will follow the same method used in the survey by Meidan et al. on Business Process Management suites  . The evaluation method they designed will facilitate aquantitative evaluation of each characteristic, category of characteristics, and overall BPM solutions in our investigation.
4.1. The Criterion Level
The criterion level is obtained by assigning points to each criterion based on a specific scoring scale. In this case the evaluation score for each characteristic will be a value either 0 or 1 as shown in Table 2.
4.2. The Partial Score (PS)
This is obtained per group of criteria. It is calculated by adding the resultant scores of all characteristics of the main categories, dividing the result by the maximum score of the category of the characteristics, and then multiplying the result by 10.
“Equation (1)” shows this calculation, where (n) represents the number of characteristics in the category, (s) represents the score of the characteristics, and (max) represents the maximum score of the category.
4.3. The Final Score (FS)
This is the final score for the BPM solutions and is obtained by adding the partial scores of all the main categories of each characteristic. “Equation (2)” shows this calculation, where (n) represents the number of the category, and (ts) represents the total maximum score of all the categories of the characteristics which, in our case, will equal (30).
The total score of (PS) and (FS) listed in Table 3 will be calculated out of 10.
The following charts present a summary of the evaluation for selected BPM solutions against the characteristics for each category and a summary of the total evaluation rate.
The chart in Figure 1 shows that ProcessMaker, Bonita BPM, and Oracle BPM Suite have an equivalent score in the evaluation of architectural characteristics after applying the equation of the partial score (PS) based on the number of characteristics and their evaluation. All three solutions achieved the highest score, which is (8.89) out of an overall (10). Among the others, Bizagi BPM Suite achieved (4.44) and ARIS Express achieved (1.11) out of (10).
Figure 2 shows that Oracle BPM Suite and Bonita BPM have an equivalent score in the evaluation of functional characteristics after applying the equation of the partial score (PS) based on the number of characteristics and their
Table 2. The evaluation scores/points for characteristics.
Table 3. Evaluation summary of BPM solutions against the characteristics.
evaluation. Both solutions achieved the highest score, which was (10) out of an overall (10). Among the others, Bizagi BPM Suite achieved (7.5), ProcessMaker achieved (6.25), and ARIS Express achieved (2.5) out of (10).
Figure 3 shows that Bizagi BPM Suite and Bonita BPM have an equivalent score in the evaluation of usability characteristics after applying the equation of
Figure 1. Summary of the scores of architectural characteristics.
Figure 2. Summary of the scores of functional characteristics.
Figure 3. Summary of the scores of usability characteristics.
the partial score (PS) based on the number of characteristics and their evaluation. Both solutions gained the highest score, which was (10) out of an overall (10). ProcessMaker achieved the second highest score, which was (7.5) while both the Oracle BPM Suite and ARIS Express achieved (5) out of (10).
In the previous charts, the strengths and weakness of each BPM solutionat different levels were evident when evaluating the solutions against all of the characteristics. In terms of architectural characteristics, ProcessMaker, Bonita BPM, and Oracle BPM Suite achieved the highest score. In terms of functional characteristics, Bonita BPM and Oracle BPM Suite achieved the highest score. Finally, in terms of usability characteristics, Bonita BPM and Bizagi BPM Suite achieved the highest score.
Figure 4. Total Evaluation Rate for All Classified Categories and their Characteristics.
Based on the given evaluation score and after applying the equation for the final score (FS) based on the number of main categories and their partial score (PS) evaluation, it is clear from Figure 4 that Bonita BPM achieved the highest average score from all five solutions with (9.63) out of an overall (10). Oracle BPM Suite achieved the second highest score of (7.96), followed by ProcessMaker with (7.54), Bizagi BPM Suite with (7.31), and ARIS Express with (2.87) out of (10).
The selection of an appropriate solution for an organization requires an in-depth investigation and analysis of numerous well-known solutions in the Saudi market. Therefore, we proposed a characterization framework for BPM solutions through the investigation of several BPM solutions and their characteristics.
The categories of the classification were structured into three main dimensions: architectural, functional, and usability. These three category dimensions were discussed and explained in detail by focusing on the role each one plays in the BPM solution.
A taxonomy of the available solutions in the Saudi market was constructed to show that our characterization was sound in practical terms.
This study was limited to open-source solutions used in the Saudi market between 2013 and 2018. Other vendors that fall outside these selection criteria were omitted.
Similar characteristics that may be available in all the solutions were not considered in this investigation. Moreover, we omitted some of the characteristics from our contribution because specific requirements were needed such as the availability and placing of large infrastructure.
As part of a future plan, a supporting tool will be developed to automate the selection process. This will be based on the taxonomy generated in this work and will examine the practical applicability of our approach.
The authors of this work would like to thank the College of Computer & Information systems and the Deanship of IT at Umm Al-Qura University for their useful help and support in accomplishing this work.