Integration at a Glance: an Essential Guide to Enterprise Application Integration

Editor’s note: Andrei describes major approaches to integrating enterprise applications and shares best practices to prevent integration risks. If you’re looking to establish the connection between your enterprise apps smoothly and cost-effectively, be sure to look at ScienceSoft’s enterprise application integration services.

To add to the inconvenience of constant switching between different independent systems, a fractured application environment often duplicates the data and functionality, the effort of their users and money needed for their maintenance. Integration of the enterprise applications can win back lost transparency and order to your business processes as well as reveal the hidden potential of the existing systems.

This article explores the benefits you can expect from integration, the options you have and the important points you’d better not overlook if you want to succeed.

Consider integration benefits

The expenses, managerial efforts and risks related to integration are high, that’s why CIOs prefer not to rush into integration projects. However, the benefits that can be gained from enterprise application integration look impressive. They include:

A 360-degree view of business aspects

Integration can allow for a comprehensive view of business components - be it customer management, supply chain management, asset management, etc.

The integration of data from multiple customer touchpoints dispersed across different company divisions (several CRMs, customer portal, physical office/store, multiple marketing channels, etc), allows getting a complete picture of customers’ interactions with the business and delivering advanced loyalty programs. Here’s an example of a similar integration project we’ve completed successfully.

Having connected CRM and digital analytics data, Domino’s managed to come up with a more advanced marketing strategy. As a result, their monthly revenue increased by 6% and ad operations costs were reduced by 80% YOY. 

The integration of an order management and a customer service system can help the customer service department get the latest info on customers’ orders, their activity periods, product returns, etc., and tailor support services more appropriately.

In supply chain management, access to the consolidated info from an accounting system, inventory management software and a vendor portal makes it much easier for category managers to develop long-term category development strategy.   

Alignment of business processes

By connecting, for example, fragmented inventory and vendor management applications, a business can plan to restock supplies more accurately or manage sales more efficiently, for example, raise prices to avoid an out-of-stock situation when the demand is growing, but the supplies are delayed.     

User convenience

The constant switching between apps is not only inconvenient but also time-consuming and causes effort duplication and an increased number of errors. For example, the integration of CRM and a document management solution can simplify the life of sales teams and increase their efficiency. While having CRM as the main tool in the process of working on a deal, they can quickly find information (terms, service scope, cooperation models) relating to similar cases.

More possibilities to use modern techs

Integration allows introducing IoT, AI-based and data analytics tools into core business processes. Also, aggregating enterprise-wide data and ensuring its consistency, you can improve training possibilities for machine learning algorithms.

Identify the level of integration

To get things started, you should understand the type of integration you need. The choice depends greatly on what you expect to achieve with a future integration project.

Do you need your multiple applications to use one consistent set of data?  With data integration, the applications can work independently, each having their own data store, but also reuse the info from other apps if needed to ensure data consistency across the organization.

Do you want some function to be beneficially reused across the organization? With a shared business function, some elementary business functions (social security number validation, payment confirmation, etc.) can be reused across multiple enterprise applications to avoid functionality duplication, simplify updates and reduce expenses on the development of new applications. For example, Travelex, a world-famous provider of financial services, managed to lower application development costs and time by 30% through the implementation of re-usable components.

Do you want to create a new business process while reusing the logic of the existing applications? With the implementation of a distributed business process, you can leverage the already existing code of your applications in new ways. A separate component to manage integration (also known as a choreographer) can coordinate the work of a composite workflow. For example, establishing links between vendor stocks, shipping partners and own inventory, a retailer can provide advanced info on shipping terms and order status to its customers. This integration type helps to increase the value of previous investments and reduce expenses on app development and maintenance.

Do you want to relieve users of switching between multiple app screens? With presentation integration, enterprise applications are displayed to users via a common interface, streamlining the work and reducing errors.  

The integration type chosen becomes the basis for a future architectural decision, communication patterns, and connection points of a future integration solution.

Choose an appropriate communication pattern

To quickly illustrate the kinds of solutions that integration architects build, let’s consider the three most popular approaches to building integration solutions so far - an old and trusted shared database, much hyped-around service-oriented architecture (SOA) and increasingly popular robotic process automation (RPA).

Shared database

The oldest option is to integrate several applications via the introduction of a shared database. Under this scenario, a set of applications reuses the same data from one shared database. The estimated project price varies between $1 million and more than $10 million for mid and large corporations depending on the solution size and complexity.

Benefits:

• Exceptional data consistency  – all data changes are immediately available.
• Relatively quick response time – there are no additional abstraction layers (interfaces) for data to pass through.

Limitations:

• A single point of failure – in case the database stops or fails, all the connected applications become partially or completely unavailable.
• Complicated changes – the changes to the database require harmonization with all the applications that use it.  
• Elevated security risks – external applications have direct access to the stored data.
• Ban on ready-made products – any platform-based application, like SAP ERP or Salesforce CRM, have their own databases and cannot be integrated this way.

Service-oriented architecture (SOA)

The SOA-based approach implies creating independent, agile and reusable web services (most often they are REST services or SOAP-based services). One system (for example, an ecommerce solution) can make up one service or a set of services, each encompassing one elementary function (for example, product catalog, active clients, etc.). Web services expose their interfaces to communicate with other systems (say, marketing CRM).

The average cost of service implementation is around $50,000. SOA’s ROI primarily depends on the number of times the services will be reused. Thus, a business should seriously consider business functions and services they want to separate. The implementation of SOA also allows for the rapid introduction of new services, easier modifications of the existing ones, and the ability to expose the application to external systems.

At the same time, significant initial investments, complicated configurations and change management, demanding network and system monitoring make up the dark side of SOA solutions. While 53% of businesses can’t justify their expenses on SOA, some report up to 200% ROI (Business Value of SOA, IBM’s report).

When you opt for SOA-based integration, you’ll need to choose between two popular ways of its implementation – point-to-point integration and enterprise service bus (ESB). Let’s have a look at the details around them.

Point-to-point integration

The services communicate with each other directly.

Benefits:

• Relatively fast and easy integration - when you have few services to coordinate.
• Better flexibility – the system is more flexible because an abstraction level (API) is added.
• Low cost – the cost of point-to-point integration may start from as low as $3,000 when only minimal customizations are required.

Limitations:

• Troubled scalability – the services are tightly coupled, and any new component in the system will require changes in the APIs of all earlier integrated components that will use data from the newly added one.
• Usage of standard protocols – the loosely coupled interfaces require to follow the consistent format of data exchange.
• Vulnerability to overload – a service can get overwhelmed with processing the increased number of incoming requests and have too few resources to perform its functions. As the messages have just one addressee, every interrupted call will result in a lost message.  
• Complex maintenance – with 5 services integrated, there are 10 connections to maintain, with 9 services – 36. The systems of 15, 20 or 100 of integrated services will make complicated environments hard to design and maintain.

Enterprise service bus (ESB)

In this case, the integrated services communicate through a central mediator. This integration server takes over all the responsibility for communication between services and undertakes message processing, enhancement, and routing, transaction management, process choreography, security management. Special adapters allow the applications to communicate with the integration server translating or transforming the message data (e.g., XML to JSON, JMS to HTTP, XML to JAVA objects).

The pricing of ESB implementation is extremely diverse. A business can employ an open-source ESB solution for free. In this case, the costs will only comprise support expenses (in-house / available commercial / outsourced). Another option is to use commercial ESB solutions. The price of a standard 16 core one-year licenses for an ESB solution (including basic support services) may range from $25,000 up to $180,000 depending on the vendor. Among the most popular commercial ESBs are Microsoft BizTalk, Oracle ESB, IBM WebSphere ESB and Mule ESB.

 Benefits:

• Resilience to load – the messages are not dependent on the specific addressee, which allows for load balancing and failover.
• Reinforced security – ESB usually protects services from unauthorized access with in-built authentication and authorization, encryption, and tokenization mechanisms.
• Quick updates and scaling – there’s no need to rewrite all APIs as in the case with point-to-point integration. Only a changed service’s adapter needs an update.
• Integration of legacy software  – the components are not obliged to communicate using modern protocols like REST or SOAP, which aging software is often incapable of.  
• Simplified support – due to increased visibility and transparency of the request journey.

Limitations:

• A possible total freeze-up – similarly to the case with a shared database, the stop of ESB may paralyze the whole enterprise IT structure.  
• Slightly reduced performance – due to an increased number of intermediary components (adapter, router, format transformer, choreographer, etc.).
• Middleware licensing costs – may require significant spendings for complex systems.

Robotic process automation (RPA)

The software robots can imitate users, typing, clicking and copy-pasting data from one application to another, exactly the way a human would do. Up to 45 % of business activities can be automated this way.

The RPA software pricing starts from $10,000. However, HFS’s research shows that software costs represent only 25-30% of total RPA costs.

Benefits:

• No interferences implied – no programming and changes in the existing applications are required.

• No need for APIs – the integration happens at the level of the user interface.
• Quick implementation – An RPA solution can be introduced within just several weeks. 
• Rapid ROI - case studies on robotic process automation estimate its ROI from 30% up to 200% already over the first year after implementation.

Limitations:

• Troublesome changes – any minor modifications in the application (user interface, data, etc.) will require adjustments in the robot’s work.
• Limited area of application – RPA can be applied only to standard business processes with clear triggers where there’s a clear positive gap between robot and human productivity.

A combination of several approaches

It’s a rare case when a single communication pattern fits all the required connections across a complex enterprise environment. So, businesses usually end up with a combination of several integration options. For example, a group of shipping services (such as shipment notice, shipment tracking, rate and route services) can communicate directly (point-to-point) with one another. While as a group (a shipment management module), they have a common interface to receive and answer queries from other systems/modules (order processing, customer service) via ESB.

Eliminate potential integration risks  

Integration has a lot of points that need careful consideration if you want to make it a success. A fail-proof formula during the integration process will require you to: 

Ensure data quality

Integration solutions are prone to data quality issues. The reasons for that are numerous, for example:

- Integrated applications employ incompatible data structures.

- Wrong data from one system (where it may be manually entered) can move to another system and infiltrate the whole system.

- Data can be duplicated.

- Data can be lost or damaged while travelling through different integration layers and between different applications.

Bad-quality data will result in wrong decisions based on it, damage to the client’s experience, process interruptions and more. To avoid these risks, take care of proper mechanisms for data validation and transformation, introduce compensation events for failed or incomplete long-running transactions, design data structures to be flexible and open for new tables and fields. Also, consider the implementation of common data rules, such as the canonical data model (CDM), across the entire organization, especially if you have access to employees both knowledgeable of your business specifics and having essential data modelling skills. 

Take care of information security

Being integrated into one system, the info becomes easier to reach. To prevent security-related issues, be careful and selective when providing direct access to the data for other systems.

Also, integration often implies remote communication via the network, which also poses serious privacy and security risks. So, make sure you use proven authentication and authorization protocols, think out security policies and access control, and consider additional encryption mechanisms.

Plan comprehensive quality assurance (QA) activities

Integration provides a field for errors and failures due to the evolvement of new connections, decoupling of processes, the introduction of asynchronous messaging, time-triggered and event-driven transaction. Testing an integration solution entails working with multiple languages and communication protocols. The existence of legacy applications can complicate the creation of a shared test environment. Besides, multiple stakeholders (business partners, vendors of a packaged solution, etc.) may need to take part in the QA process.

To overcome these challenges, get a professional testing team experienced in integration projects, with good knowledge of enterprise application integration patterns and middleware, and a thorough understanding of your integration design and architecture.

A QA and testing team should make sure that applications work as expected in silos, before being integrated. The testing process should also include verification of all uni- or bidirectional queries between the connected interfaces, checking the completeness of distributed transactions, approval of appropriate exceptions handling, assuring the work of the failover and recovery mechanisms.

Performance testing should also be given enough attention to. Low performance of one component can drag down the whole integrated system. The testing team should check if the chosen architecture and technologies guarantee the appropriate performance of each component if the integrated system works well within established bandwidth limitations, and the system components are capable of fail-free processing of large data sets and handling an increased number of requests.  

The final curtain

Integration is a handy way to tackle the inconvenient and time-consuming nature of standalone applications, eliminate data discrepancy and leverage comprehensive info to add to the consistency of business processes.

To get a rock star integration solution:

• Begin with a clear definition of your integration intent.
• Decide what applications to integrate.
• Consider available integration concepts and communication models between the connected components.
• Pay due attention to the increased security and performance risks.
• Assure enterprise-wide data quality.
• Make sure that QA activities thoroughly cover the integration process.