Application Integration: Steps, Skills, Success Factors
With 33 years in IT and practical knowledge of over 20 business areas, ScienceSoft helps businesses with integration of custom and third-party on-premise, cloud and web applications.
Application Integration: the Essence
The application integration of medium and high complexity takes around 9-15 months, including steps like integration scoping and planning, integrated system design, tech stack selection, integration implementation and testing. The project requires a team of a project manager, a business analyst, an integration architect, developers, QA and DevOps engineers.
In software development since 1989, ScienceSoft helps build reliable, secure and convenient integrations between custom and/or commercial systems.
The application integration stages and their timelines can vary depending on the number of applications integrated, their architecture, required changes in legacy software, the chosen integration approach, complexity of security and compliance requirements, and more. Below are described the typical steps we at ScienceSoft take to perform application integration.
Integration scoping and planning
At the stage, ScienceSoft's team:
- Elicits business requirements for a new integrated system.
- Defines the systems/processes to be integrated and data to be shared.
- Analyzes the software to be integrated, understanding its capabilities and constraints.
- Elicits infrastructure capabilities and constraints.
- Figures out how data should flow and whether it needs to be transformed into a different format.
- Checks how many communication protocols between integrated applications will be used.
- Defines business metrics to measure integration initiative success (e.g., improved order processing/delivery time, reduced transaction errors, optimized spending on storage, compute and app maintenance).
- Describes compliance requirements for a new integrated system.
Best practice: (for large-scale enterprise-wide strategic integration initiatives) We prefer to discuss (or outline, if there's yet none) a long-term enterprise integration strategy and high-level enterprise integration architecture plan of our clients. While requiring higher initial investments, it will help reduce integration failures and avoid expensive maintenance of multiple technologies later.
Integrated system design
ScienceSoft's team aligns the prepared business requirements for an integration solution and defines its detailed technical requirements:
- Describes the requirements for the integrated system (availability, integrity, scalability, maintainability, performance, persistence) based on business needs and the integrated applications.
- Describes security requirements for the integrated system.
- Decides on an appropriate app integration approach:
- Common data storage:
- Combining data from several applications into a shared databases.
- Note: Platform-based applications, like Salesforce or SAP ERP have their own databases and cannot be integrated this way.
- Direct (point-to-point) integration:
- Out-of-the-box integration (mostly, for third-party software products).
- Hand coding point-to-point interfaces between systems (mostly, for legacy/niche integrations).
- Mediated integration:
- Using Enterprise Service Bus (ESB) as a central mediator for communication between services. ESB undertakes message processing, transformation/modification, and routing, transaction management, process choreography, security management. ESBs can be bought or built from scratch (mostly, in case of niche integrations).
- Using an integration platform as a service (iPaaS). iPaaS includes all or a combination of the following integration services: API management, orchestration, messaging, translation and transformation, adapters. iPaaS can be adopted entirely or partially.
- Using specific software (‘robots’):
- Using specific software (‘robots’) to imitate human interactions with computer systems (typing, clicking and copy-pasting data from one application to another).
- Common data storage:
See pros and cons for each integration style in ScienceSoft’s Enterprise Application Integration Guide and Overview of Cloud Integration Approaches.
The potential reusability of an application functionality is one of our key consideration points when choosing an optimal integration approach. When SOA architecture is put in place, the reuse of a solution can result in up to 200% ROI.
Note: It’s rarely possible to solve all integration needs with a single integration approach. For the majority of our clients, especially, when it comes to complex integration with more than two applications involved, we make up integration solutions that combine several integration approaches.
Selection, evaluation and comparison of tech stacks for the selected integration approach
ScienceSoft's BA and solution architect shortlist 2-3 most suitable tech stacks for the chosen integration approach (e.g., for an ESB-based integration – Talend ESB and SAP PI), for iPaaS-based integration – Azure iPaaS or Informatica iPaaS). After that, together with the project stakeholders, they weigh up all the options and choose the best strategic ones.
Best practice: When our clients need their external partner to connect, we always keep in mind to discuss if their partners have or ready to employ the required infrastructure and skills to use the chosen integration technologies and support the planned communication standards.
App integration implementation and testing
Further steps will differ depending on the chosen integration approach and tech stack. Below, we describe the process for two most popular integration types.
- Designing and installing custom APIs or implementing ready-to-use APIs.
- Building or buying a glue layer (message processor/ESB) that will handle message routing, modification, aggregation and changing data mapping.
- Selecting or building adapters for your applications to communicate with the glue layer.
- Orchestrating integration:
- Configuring triggers for requests sent to the service.
- Configuring invokes for requests sent from the service to the target.
The QA team validates the integration:
- Verifies all uni- or bidirectional queries between the connected applications. Special attention should be given to testing flow which requires calling multiple services in sequence or asynchronously which is known as orchestration testing.
- Approves the work failover mechanisms and recovery mechanisms which is of a big importance in case of integration projects.
- Performs security tests. API and web services are highly vulnerable to security attacks and we need to be absolutely sure about the security of the exposed web service depending on the architecture of the web service and the nature of its use.
Best practice: When we deal with integration of business critical that cannot be taken offline, we pay special attention to co-oordinadination and detailed discussion of a test plan.
Application integration consulting
- Business and technical requirements for the integrated system.
- The integration approach and the tech stack.
- Description of the required modifications of the integrated applications.
- Guide on compliance with the required security certifications/regulations (HIPAA, HITECH, SOC2, SOC3, and more).
- A detailed app integration project roadmap.
Application integration implementation
- Integration project planning: requirements analysis; application and infrastructure analysis.
- The integrated system’s architecture design.
- Migration and modernization (re-engineering, re-architecting) of the applications (if required).
- Integration solution development, implementation and testing.
- After-launch support, user and admin training.
- Continuous management and evolution of the integrated ecosystem (on demand).
ScienceSoft as a Reliable Application Integration Partner
ScienceSoft offered a simple, yet well-functioning, way to integrate our financial portal with the data management and accounting systems, which allowed us not to get stuck with an overcomplicated data synchronization process.
The team was always responsive to all our questions, and we truly appreciated such a collaborative approach.
Cindy White, Director of IT, Texas Methodist Foundation
- In IT since 1989.
- 2 days – 2 weeks to start a new project.
- Integration and database architects with 7-20 years of experience.
- Over 10 years of experience in cloud development and migration.
- 9 years in applying DevOps practices.
- 19 years in cybersecurity. All-level security configuration and monitoring (infrastructure, application logic, security testing, compliance).
- Expertise in more than 30 industries.
- ISO 9001 certified quality management, ISO 27001 certified information security management.
- Compliance with HIPAA, GAMP, PCI DSS, and more.
- Microsoft Gold Partner with 9 Gold Competencies.
- AWS Select-tier Consulting Partner.
- Additional AWS and Azure discounts and priority access to AWS and Azure technologies and services.
Selected App Integration Projects by ScienceSoft
Enterprise Application Integration Consulting for a Multi-Billion Corporation
ScienceSoft’s team prepared four integration scenarios for multiple disintegrated CRMs, ERPs, POSs and ecommerce solutions, supply chain management systems, etc. The suggested integration solutions allowed for clean, accurate and consistent data, timely available and safely transferred across the systems.
Application Integration to Drive Digital Transformation in the Healthcare Business
ScienceSoft automated paper-based document management, order and warehouse management workflows with custom software solutions. The delivered solutions have largely catalyzed customer experience optimization.
Development of a Supply Chain E-Collaboration Platform for 1,700-Store Retailer Auchan
ScienceSoft helped to piece together an e-collaboration platform that integrated real-time supply chain data provided by buyers, accountants, vendors, store managers, etc. The platform provides visibility into buying and selling processes, helps minimize out-of-stock levels, and facilitates demand planning.
Development of a BI Solution for the Producer of Phytotherapy Products
ScienceSoft developed a full-scale BI solution that integrated the Customer’s disparate data sources and transformed them into actionable and easy to digest reports to improve decision-making.
- Plans the app integration project, assigns integration project tasks and responsibilities to the integration team and controls tasks’ execution.
- Makes sure the project is being implemented on time, on budget, and complies with the set quality metrics.
- Reports to the key stakeholders on the software integration project progress.
- Elicits, clarifies, prioritizes, and documents the integrated system’s and the integration solution’s requirements.
- Chooses the integration approach (in collaboration with integration architect).
- Chooses the integration approach and architects the integration solution to meet the business and technology requirements.
- Develops the to-be integrated apps architecture transformation and modernization roadmap (if necessary).
- Designs frameworks and processes to support API development and management.
- Builds custom integration code (APIs, ESB, etc.). In case of pre-built integration, sets up triggers, chooses resulting actions, and selects data to be shared.
- Configures the integration solution infrastructure.
- Automates development and release processes by introducing a CI/CD pipeline.
- Monitors the integrated system security, performance, and availability, etc.
- Designs and implements a test strategy, a test plan and test cases for the integration solution and the integrated system to assure that the functional, security, and compliance requirements are met.
- Creates and conducts manual and automated test cases.
- Provides test summary reports.
Note: They should have a good understanding of the to-be integrated application software and be technically capable to understand integration architecture elements (e.g. messaging technologies used).
The entire app integration process is in-house
- Full control over the integration process and team work.
- The lack of sufficient integration experience can compromise the integration quality, lead to a rework and significantly prolong the project timelines.
- All hiring and managerial efforts are on your side.
Partial or full outsourcing of technical resources
- Quick integration project start.
- Quick access to specific tech knowledge when required.
- Project team coordination, control over quick and quality integration are partially or totally on your side.
Full integration process outsourcing
A vendor is responsible for end-to-end integration, including:
- Providing specific skills, team ramp-up and management.
- Integration solution design.
- Quality of the project results.
- High vendor risks.
Benefits of Application Integration with ScienceSoft
ScienceSoft introduces data governance rules, error message management, message completeness verification, roll-back of distributed transactions to ensure consistent and comprehensive data across the integrated system.
Advanced integration security
We use sophisticated access control rules, industry-proven protocols for authentication and authorization, “safer” outbound connections, and more.
System performance management
We pay special attention to planning system capacity and middleware, design scalable architecture, introduce priority queues, effectively use timeouts and sync calls, logging and caching, conduct load and performance testing, network latency problem assessment.
Optimal integration journey
ScienceSoft starts with a feasibility study of its technical, economic and operational aspects of your integration initiative. Our consultants with 7-20 years of experience in the field investigate whether the project is worth your time and money and which integration scenario will bring max benefit.
At ScienceSoft, we often choose the following technologies for our application integration projects:
Azure Integration Services
Best for: hybrid integration
- Among leaders in 2020 Gartner Magic Quadrants for Enterprise iPaaS and Full Life Cycle API Management.
- Comprehensive set of integration capabilities covering applications and data with over 370 connectors.
- Serverless integration via Azure Functions.
- Support of multi-cloud, on-premises and edge computing environments.
- Azure Integration Services is part of the larger Azure offering, giving you immediate access to any other Azure services you need, e.g., Azure Data Lake Store for storing large amounts of unstructured data.
AWS Integration Services
Best for: data integration
- Comprehensive messaging suite comprising Amazon SNS (fully managed pub/sub messaging), Amazon SQS (fully managed message queues), Amazon MQ (managed message broker service for Apache ActiveMQ).
- AWS AppSync (to create flexible GraphQL APIs to securely access, manipulate, and combine data from one or more data sources) and API Gateway (to create and maintain secure REST APIs at any scale in serverless and web development).
- AWS Step Functions and AWS Lambda for serverless development.
Informatica Intelligent Cloud Services
Best for: enterprise application integration
- A leader in Gartner’s Magic Quadrant for Enterprise iPaaS for 7 years straight.
- Unified user experience across all data management services.
- The AI-driven CLAIRE™ engine to accelerate and automate data governance processes.
- Multi-cloud and serverless environments.
- The industry’s highest number of security certifications among cloud app integration solutions, including SOC2, SOC3, HIPAA, and HITECH.
Starting at $2,000/month, free 30-day trial.
SAP Cloud Platform Integration
Best for: specialized enterprise application integration
- Among leaders in 2020 Gartner Magic Quadrant for Enterprise iPaaS.
- SAP’s API Business Hub offers access to rich integration content packs with over 1,500 prebuilt, ready-to-run integrations, APIs and events that target specific business processes.
- Targets diverse business operations and customers across 25 industries.
- Connects cloud applications with SAP and non-SAP cloud and on-premises applications.
The costs will vary depending on the specifics of the integration project. Major cloud application integration cost factors include:
- The number of applications to be integrated and the number of integration points.
- The chosen integration approach.
- The required modifications of the integrated applications (both on the technical level and on the app logic level).
Types of costs related to application integration
Integration solution infrastructure
- Licensing for middleware (e.g., third-party ESB)
- Infrastructure for SOA implementation (service connector, service descriptor, asynchronous response handler, service interceptor, etc.).
Skills development / acquisition
Training of in-house staff / hiring new experts (either on the outsourcing or in-house basis) for:
- Building custom adapters/APIs or tailoring off-the-shelf adapters.
- Implementing the integration solution: message broker/ESB/iPaaS (if required).
- Designing and implementing application changes (if required).
- Introducing/updating compliance requirements to the new integrated system.
- Ensuring data quality – data validation and standardization.
- Planning and executing tests for the integration solution, the integrated components and and the integrated system on the whole.
- Monitoring performance, availability, security, data quality, etc. and troubleshooting during and after integrated system implementation.
(for enterprises) Business costs
- Lost productivity and downtime as a result of switching to new business processes, integration ramp-up, and business user training / knowledge transfer.
A global IT consulting and IT services company headquartered in McKinney, TX, ScienceSoft provides end-to-end application integration services. Proficient in working with both modern and legacy techs and architecture patterns, we’re ready to handle most complicated application integration and modernization challenges. As the holder of ISO 9001 and ISO 27001 certifications, we rely on mature quality management and guarantee the cooperation doesn't pose any risks to the customers' data security.