Application Integration: Steps, Skills, Success Factors
With 32 years in IT and practical knowledge of over 20 business areas, ScienceSoft helps businesses with integration of custom and third-party on-premise, cloud and web applications.
Application Integration: the Essence
For enterprises, application integration enables automated data flows between a company’s internal business applications (Enterprise Application Integration (EAI)), internal applications and applications of customers, suppliers and business partners (Business-to-Business Application Integration (B2B)). For software product companies, integration binds several software products into a unified product platform.
Today, the interest in integrating software applications gets intensified by:
- The necessity to remove data silos.
- Supplier and customer relationships expansion.
- The need to get new functionality quickly.
- Mergers and acquisitions.
- Company-wide data analytics.
The application integration stages and their timelines can vary depending on the number of applications integrated, their architecture, required changes in legacy software, the chosen integration approach, complexity of security and compliance requirements, and more. Below are the most common steps of integration projects.
Step 1. Integration scoping and planning
Duration: ~3 weeks
At the stage, the integration project team:
Elicits business requirements for a new integrated system.
Defines the systems/processes to be integrated and data to be shared.
Analyzes the software to be integrated, understanding its capabilities and constraints.
Elicits infrastructure capabilities and constraints.
Figures out how data should flow and whether it needs to be transformed into a different format.
Checks how many communication protocols between integrated applications will be used.
Defines business metrics to measure integration initiative success (e.g., improved order processing/delivery time, reduced transaction errors, optimized spending on storage, compute and app maintenance).
Describes compliance requirements for a new integrated system.
Hint: (for large-scale enterprise-wide strategic integration initiatives) Have a long-term enterprise integration strategy and high-level enterprise integration architecture plan. While requiring higher initial investments, it will help reduce integration failures and avoid expensive maintenance of multiple technologies later.
Step 2. Integrated system design
Duration: 3-5 weeks
The integration team aligns the prepared business requirements for an integration solution and defines its detailed technical requirements:
Describes the requirements for the integrated system (availability, integrity, scalability, maintainability, performance, persistence) based on business needs and the integrated applications.
Describes security requirements for the integrated system.
Decides on an appropriate app integration approach:
Common data storage
- Combining data from several applications into a shared databases.
Note: Platform-based applications, like Salesforce or SAP ERP have their own databases and cannot be integrated this way.
Direct (point-to-point) integration
- Out-of-the-box integration (mostly, for third-party software products).
- Hand coding point-to-point interfaces between systems (mostly, for legacy/niche integrations).
- Using Enterprise Service Bus (ESB) as a central mediator for communication between services. ESB undertakes message processing, transformation/modification, and routing, transaction management, process choreography, security management. ESBs can be bought or built from scratch (mostly, in case of niche integrations).
- Using an integration platform as a service (iPaaS). iPaaS includes all or a combination of the following integration services: API management, orchestration, messaging, translation and transformation, adapters. iPaaS can be adopted entirely or partially.
Using specific software (‘robots’)
- Using specific software (‘robots’) to imitate human interactions with computer systems (typing, clicking and copy-pasting data from one application to another).
See pros and cons for each integration style in ScienceSoft’s Enterprise Application Integration Guide and Overview of Cloud Integration Approaches.
Hint: Invest in reuse. When SOA architecture is put in place, the reuse of a solution can result in up to 200% ROI.
Hint: It’s rarely possible to solve all integration needs with a single integration approach, especially, when it comes to complex integration with more than two applications involved.
Step 3. Selection, evaluation and comparison of tech stacks for the selected integration approach
Duration: 3-4 weeks
A BA and a solution architect shortlist 2-3 most suitable tech stacks for the chosen integration approach (e.g., for an ESB-based integration – Talend ESB and SAP PI), for iPaaS-based integration – Azure iPaaS or Informatica iPaaS). After that, together with the project stakeholders, they weigh up all the options and choose the best strategic ones.
Hint: When you need your external partner to connect, make sure they have the required infrastructure and skills to use the chosen integration technologies and support the planned communication standards.
Step 4. App integration implementation and testing
Duration: 6-12 months to develop a new integration solution
Further steps will differ depending on the chosen integration approach and tech stack. Below, we describe the process for two most popular integration types.
- Designing and installing custom APIs or implementing ready-to-use APIs.
- Building or buying a glue layer (message processor/ESB) that will handle message routing, modification, aggregation and changing data mapping.
- Selecting or building adapters for your applications to communicate with the glue layer.
- Orchestrating integration:
- Configuring triggers for requests sent to the service.
- Configuring invokes for requests sent from the service to the target.
The QA team validates the integration:
- Verifies all uni- or bidirectional queries between the connected applications. Special attention should be given to testing flow which requires calling multiple services in sequence or asynchronously which is known as orchestration testing.
- Approves the work failover mechanisms and recovery mechanisms which is of a big importance in case of integration projects.
- Performs security tests. API and web services are highly vulnerable to security attacks and we need to be absolutely sure about the security of the exposed web service depending on the architecture of the web service and the nature of its use.
Hint: Most source and target systems are business critical and therefore cannot be taken offline. As a result, a detailed discussion of a test plan and a lot of co-coordination is required.
- Plans the app integration project, assigns integration project tasks and responsibilities to the integration team and controls tasks’ execution.
- Makes sure the project is being implemented on time, on budget, and complies with the set quality metrics.
- Reports to the key stakeholders on the software integration project progress.
- Elicits, clarifies, prioritizes, and documents the integrated system’s and the integration solution’s requirements.
- Chooses the integration approach (in collaboration with integration architect).
- Chooses the integration approach and architects the integration solution to meet the business and technology requirements.
- Develops the to-be integrated apps architecture transformation and modernization roadmap (if necessary).
- Designs frameworks and processes to support API development and management.
- Builds custom integration code (APIs, ESB, etc.). In case of pre-built integration, sets up triggers, chooses resulting actions, and selects data to be shared.
- Configures the integration solution infrastructure.
- Automates development and release processes by introducing a CI/CD pipeline.
- Monitors the integrated system security, performance, and availability, etc.
- Designs and implements a test strategy, a test plan and test cases for the integration solution and the integrated system to assure that the functional, security, and compliance requirements are met.
- Creates and conducts manual and automated test cases.
- Provides test summary reports.
Note: They should have a good understanding of the to-be integrated application software and be technically capable to understand integration architecture elements (e.g. messaging technologies used).
The entire app integration process is in-house
- Full control over the integration process and team work.
- The lack of sufficient integration experience can compromise the integration quality, lead to a rework and significantly prolong the project timelines.
- All hiring and managerial efforts are on your side.
Partial or full outsourcing of technical resources
- Quick integration project start.
- Quick access to specific tech knowledge when required.
- Project team coordination, control over quick and quality integration are partially or totally on your side.
Full integration process outsourcing
A vendor is responsible for end-to-end integration, including:
- Providing specific skills, team ramp-up and management.
- Integration solution design.
- Quality of the project results.
- High vendor risks.
Azure Integration Services
Best for: hybrid integration
- Among leaders in 2020 Gartner Magic Quadrants for Enterprise iPaaS and Full Life Cycle API Management.
- Comprehensive set of integration capabilities covering applications and data with over 370 connectors.
- Serverless integration via Azure Functions.
- Support of multi-cloud, on-premises and edge computing environments.
- Azure Integration Services is part of the larger Azure offering, giving you immediate access to any other Azure services you need, e.g., Azure Data Lake Store for storing large amounts of unstructured data.
AWS Integration Services
Best for: data integration
- Comprehensive messaging suite comprising Amazon SNS (fully managed pub/sub messaging), Amazon SQS (fully managed message queues), Amazon MQ (managed message broker service for Apache ActiveMQ).
- AWS AppSync (to create flexible GraphQL APIs to securely access, manipulate, and combine data from one or more data sources) and API Gateway (to create and maintain secure REST APIs at any scale in serverless and web development).
- AWS Step Functions and AWS Lambda for serverless development.
Informatica Intelligent Cloud Services
Best for: enterprise application integration
- A leader in Gartner’s Magic Quadrant for Enterprise iPaaS for 7 years straight.
- Unified user experience across all data management services.
- The AI-driven CLAIRE™ engine to accelerate and automate data governance processes.
- Multi-cloud and serverless environments.
- The industry’s highest number of security certifications among cloud app integration solutions, including SOC2, SOC3, HIPAA, and HITECH.
Starting at $2,000/month, free 30-day trial.
SAP Cloud Platform Integration
Best for: specialized enterprise application integration
- Among leaders in 2020 Gartner Magic Quadrant for Enterprise iPaaS.
- SAP’s API Business Hub offers access to rich integration content packs with over 1,500 prebuilt, ready-to-run integrations, APIs and events that target specific business processes.
- Targets diverse business operations and customers across 25 industries.
- Connects cloud applications with SAP and non-SAP cloud and on-premises applications.
The costs will vary depending on the specifics of the integration project. Major cloud application integration cost factors include:
- The number of applications to be integrated and the number of integration points.
- The chosen integration approach.
- The required modifications of the integrated applications (both on the technical level and on the app logic level).
TYPES OF COSTS RELATED TO APPLICATION INTEGRATION
Integration solution infrastructure
- Licensing for middleware (e.g., third-party ESB)
- Infrastructure for SOA implementation (service connector, service descriptor, asynchronous response handler, service interceptor, etc.).
Skills development / acquisition
Training of in-house staff / hiring new experts (either on the outsourcing or in-house basis) for:
- Building custom adapters/APIs or tailoring off-the-shelf adapters.
- Implementing the integration solution: message broker/ESB/iPaaS (if required).
- Designing and implementing application changes (if required).
- Introducing/updating compliance requirements to the new integrated system.
- Ensuring data quality – data validation and standardization.
- Planning and executing tests for the integration solution, the integrated components and and the integrated system on the whole.
- Monitoring performance, availability, security, data quality, etc. and troubleshooting during and after integrated system implementation.
(for enterprises) Business costs
- Lost productivity and downtime as a result of switching to new business processes, integration ramp-up, and business user training / knowledge transfer.
Application Integration Consulting and Implementation by ScienceSoft
Application integration consulting
- Elicit and document business and technical requirements for the integrated system.
- Choose the integration approach and the tech stack.
- Plan required modifications of the integrated applications.
- Help comply with the required security certifications/regulations, including HIPAA, HITECH, SOC2, SOC3, and more.
- Deliver a detailed app integration project roadmap.
Application integration implementation
Our team takes over:
- Integration project planning: requirements elicitation and prioritization; application and infrastructure analysis.
- The integrated system’s architecture design.
- Migration and modernization (re-engineering, re-architecting) of the applications before integration (if required).
- Integration solution development.
- Integrated system implementation and testing (at system and component levels).
- After-launch support, user and admin training.
- Continuous management and evolution of the integrated ecosystem (on demand).
A global IT consulting and IT services company headquartered in McKinney, TX, ScienceSoft provides end-to-end application integration services. Proficient in working with both modern and legacy techs and architecture patterns, we’re ready to handle most complicated application integration and modernization challenges. As the holder of ISO 9001 and ISO 27001 certifications, we rely on mature quality management and guarantee the cooperation doesn't pose any risks to the customers' data security.