Cloud Application Integration: Stages, Options, Timeline, and Costs
ScienceSoft has been providing cloud application development and integration services since 2012.
СLOUD APPLICATION INTEGRATION: THE ESSENCE
Cloud application integration is the process of:
- connecting cloud-hosted applications with other cloud or on-premises software within one organization, including among distributed facilities;
- connecting cloud-hosted applications with other cloud or on-premises software between different companies;
- (for SaaS vendors) integrating a set of cloud-hosted software products.
The key value of integration resides in real-time data exchange and process synchronization.
The steps of the cloud application integration process and their timelines will differ depending on the number of applications integrated, their architecture, required legacy software modification, integration approach, complexity of security and compliance requirements, and more. Below, we describe the most common steps of cloud app integration projects.
Step 1. Cloud app integration planning and design
Duration: 3-5 weeks
Integration scoping and planning
At this stage, the integration team:
Defines the systems/processes to be integrated and data to be shared.
Elicits existing concerns from the business side (security, compliance, business process disruption during integration activities, availability of resources to operate and manage the integrated system).
Defines infrastructure capabilities and constraints.
(when legacy systems are involved) Audits legacy software to be integrated, analyzing its capabilities and constraints.
Figures out how the data should flow and whether the data needs to be transformed into a different format.
Checks how many communication protocols between integrated applications will be used.
(optional, mostly for strategic integration initiatives) Examines existing/defines new types of standards that will be used in the integration architecture (e.g., for application interfaces, message formats, process models, an enterprise metadata model).
Defines non-functional requirements for the integrated system (performance, availability, security, etc.).
Defines compliance requirements.
Checks if there are any ready integration solutions or integration interfaces available.
Note: It’s also a best practice to define business metrics to measure integration initiative success (e.g., improved order processing/delivery time, reduced transaction errors, reduced time to introduce a new software solution, etc.).
Integration approach selection
The team decides on an appropriate cloud app integration approach from 4 major types described below. The choice between direct and mediated integration approaches will depend on the number of application to be integrated, planned short-term and long-term investments, overall IT strategy (for enterprises)/business strategy, non-functional requirements to the integrated system (performance, availability, security), the necessity to integrate with legacy software, and more.
Direct (point-to-point) integration
- Using tools and/or out-of-the-box integration provided by applications’ vendors.
- Using custom code.
- Using Enterprise Service Bus (ESB). ESB is a central mediator (integration server) that takes over the responsibility for communication between services and undertakes message processing, transformation/modification, and routing, transaction management, process choreography, security management. Special adapters allow the applications to communicate with ESB translating or transforming the message data (e.g., XML to JSON, JMS to HTTP, XML to Java objects).
- Using an integration platform as a service (iPaaS). An iPaaS solution is a set of cloud services that often includes API management, orchestration, messaging and event management services.
It’s a rare case when a single integration pattern fits all the required connections when several apps are to be integrated. So, often a combination of integration approaches is used.
Optional: evaluation and comparison of tech stacks for the selected integration approach
A BA and a solution architect shortlist 2-3 preferable integration tech stacks (e.g, for an ESB-based integration – Talend ESB and SAP PI, for iPaaS-based integration – Azure iPaaS or Informatica iPaaS) and rate strengths and weaknesses of each option. Together with the project stakeholders, they decide on the most feasible option with the best strategic potential.
Note: If you also expect your external partners to connect, make sure they have required infrastructure and skills to use integration technology and support the planned communication standards.
Step 2. Cloud app integration solution implementation and testing
Duration: 5-9 months to develop a new integration solution
The steps of this stage will differ depending on the chosen integration approach. Below, you can find exemplary scenarios for two most popular integration approaches.
- Designing and installing custom APIs or implementing ready-to-use APIs.
- Building or buying a glue layer (message processor/ESB) that will handle message routing, modification, aggregation and changing the mapping of the data.
- Creating connections – select or build adapters for your applications to communicate with the glue layer.
- Orchestrating integration:
- Configuring triggers for requests sent to the service.
- Configuring invokes for requests sent from the service to the target.
The QA team works on:
- Verification of all uni- or bidirectional queries between the connected interfaces.
- Checking the completeness of distributed transactions.
- Approval of appropriate exceptions handling.
- Assuring the work of the failover and recovery mechanisms.
- Checking how well the integrated system works within established bandwidth limitations.
- Checking how good the system components are at fail-free processing of large data sets and handling an increased number of requests.
Step 3. Integrated system support and evolution
The support team monitors the integrated system performance and handles its scale-ups (the number of users) and scale-outs (the number of the integrated applications).
- Plans the cloud app integration project, assigns tasks to the integration team and oversees tasks’ execution.
- Makes sure to provide stated project deliverables on time, on budget and of the required quality.
- Timely updates required stakeholders on the integration project progress.
- Communicates with cloud providers, SaaS vendors (in case of enterprise apps integration), and other vendors.
- Gathers, clarifies and documents the integrated system requirements.
- Together with integration architect, defines the integration approach.
- Designs the integration between cloud-hosted and other cloud/on-premises apps to meet the technical and business requirements.
- Develops an app architecture transformation and modernization roadmap (if necessary).
- Develops technical documentation relating to the integrated system architecture.
- Creates a strategy for disaster recovery and data integration.
- Designs frameworks and processes to support API development and management.
- Builds custom integration code (APIs, ESB, etc.). In case of pre-built integration, sets up triggers, chooses a resulting actions from the other app and selects data to be sent.
- Configures the integration infrastructure.
- Automates and streamlines development and release processes by introducing CI/CD pipelines.
- Monitors the integrated system performance, availability, security, etc.
- Designs and implements a test strategy, a test plan and test cases for the integration solution, the to-be-integrated applications, and the target system on the whole.
- Assures that all availability, performance, security, and compliance requirements of the integrated solution are met.
- Checks the mechanisms of data validation and transformation, compensation events for failed or incomplete long-running transactions.
- Provides test summary reports.
The entire cloud app integration process is in-house
- Direct supervision over the integration process and the team’s work.
- The integration project can be delayed or compromised due to the lack of the required resources or their insufficient experience.
- (for enterprises) The IT department may have to give up other priority projects to build the integration.
- All hiring and managerial efforts are on your side.
Partial or full outsourcing of technical resources
- Quick integration project start.
- Optimal human resource usage at peak times.
- Quick access to specific tech knowledge when required.
- Project team coordination, concerns about quick and quality integration solution delivery are partially or totally on your side.
Full integration process outsourcing
A vendor assumes full responsibility for the integration project, including:
- Team ramp-up and management.
- Integration solution design.
- Establishment and coordination of the integration process.
- Quality of the project results.
- High vendor risks.
Azure Integration Services
Best for: hybrid integration
- Among leaders in 2020 Gartner Magic Quadrants for Enterprise iPaaS and Full Life Cycle API Management.
- Comprehensive set of integration capabilities covering applications and data with over 370 connectors.
- Serverless integration via Azure Functions.
- Services and tools support multi-cloud, on-premises and edge computing environments.
- Azure Integration Services is part of the larger Azure offering, giving you immediate, integrated access to any other cloud services you need, e.g., Azure Data Lake Storage for storing large volumes of unstructured data.
AWS Integration Services
Best for: data integration
- Comprehensive messaging suite comprising Amazon SNS (fully managed Pub/Sub messaging), Amazon SQS (fully managed message queues), Amazon MQ (managed message broker service for Apache ActiveMQ).
- AWS AppSync (to create flexible GraphQL APIs to securely access, manipulate, and combine data from one or more data sources) and API Gateway (to create, maintain, and secure REST APIs at any scale in serverless and web development).
- AWS Step Functions and AWS Lambda for serverless development.
Informatica Intelligent Cloud Services
Best for: enterprise application integration
- A leader in Gartner’s Magic Quadrant for Enterprise iPaaS for 7 years straight.
- Unified user experience across all data management services.
- The AI/ML-driven CLAIRE™ engine to accelerate and automate data governance processes.
- Cloud-native and scalable integration.
- Multi-cloud and serverless environments.
- The industry’s highest number of security certifications among cloud app integration solutions, including SOC 2, SOC 3, HIPAA, and HITECH.
Starting at $2,000/month, free 30-day trial.
SAP Cloud Platform Integration
Best for: specialized enterprise application integration
- Among leaders in 2020 Gartner Magic Quadrant for Enterprise iPaaS.
- SAP’s API Business Hub offers access to rich integration content packs with over 1,500 prebuilt, ready-to-run integrations, APIs and events that target specific business processes.
- Connects cloud applications with SAP and non-SAP cloud and on-premises applications.
The costs will vary depending on the specifics of the integration project. Major cloud application integration cost factors include:
- The number of applications to be integrated and the number of integration points.
- Cloud app integration approach (development of custom integration solution from scratch will require approximately ~4-6 FTE).
- Integration-readiness of the applications (both on the technical level and on the app logic level).
Types of costs related to cloud application integration
Integration solution infrastructure
- Licensing for middleware (e.g., third-party ESB)
- Infrastructure for SOA implementation (service connector, service descriptor, asynchronous response handler, service interceptor, etc.).
- Additional computing to run new software
Skills development / acquisition
Re-training of current resources / hiring of experts (either on the outsourcing or in-house basis) for:
- Building custom adapters/APIs or tailoring off-the-shelf adapters
- Designing and coding application modifications (if required)
- Implementing the integration solution: message broker/ESB/iPaaS (if required)
- Introducing/updating compliance requirements
- Ensuring data quality – data validation and standardization
- Planning and introducing various types of testing for separate components and the integrated system on the whole (incl., development and maintenance of automated tests).
- Monitoring performance, availability, security, data quality, etc. and troubleshooting during and after integrated system implementation
(for enterprises) Business costs
- Establishment of cross-department teams for integration planning and evolution.
- Lost productivity and downtime associated with transitioning to new business processes, integration ramp-up, and business user training / knowledge transfer.
CLOUD INTEGRATION CONSULTING AND IMPLEMENTATION BY SCIENCESOFT
With 9 year experience in cloud computing and hundreds of successful integration projects completed, ScienceSoft can help with:
Cloud application integration consulting
Cloud application integration implementation
A global IT consulting and IT services company headquartered in McKinney, TX, ScienceSoft provides end-to-end application integration services. We leverage our expertise to execute industry- and function-focused integrations and keep your integration journey quick and smooth.