Application Integration on Amazon Web Services (AWS): Steps, Roles, Techs
Having 32 years in IT and 8 years of experience in AWS consulting and related services, ScienceSoft advises on and carries out AWS-based app integration.
Application Integration on AWS: The Essence
Application integration on AWS is enabled via point-to-point API communication or via message- or event-driven communication, leveraging respective AWS services. The core AWS services used for integration include Amazon Simple Notification Service (SNS), Amazon Simple Queue Services (SQS), Amazon API Gateway, AWS EventBridge, etc.
A typical AWS-based integration process consists of analyzing the to be integrated applications, designing the integrated system, implementing and testing the integration. It can be carried out by a team of a project manager, a business analyst, an AWS solution architect, developer(s), QA engineer(s), and a DevOps engineer.
The stages and timelines of an AWS-based application integration project can vary depending on the number of applications under integration and their complexity, the chosen integration approach and architecture, required changes in legacy software, and more. Below are the most common steps of AWS-based app integration projects.
Step 1. Analyzing the to-be-integrated applications
Duration: ~1-3 weeks
- Describing key architecture characteristics (modular architecture, APIs, etc.) and constraints (rigid architecture, lack of up-to-date documentation, etc.) of software to be integrated.
- Estimating the required investments and value that AWS-based application integration can bring considering your integration goals and scope, the specifics of the apps to be integrated and the existing IT environment.
- Defining business metrics to measure AWS-based integration success (e.g., improved order processing or delivery time, reduced transaction errors, optimized spending on storage, compute and app maintenance).
Step 2. Designing the integrated system and points of integration
Duration: ~1-3 weeks
- Identifying APIs and build a repository of the available APIs.
- Mapping out business workflows driven by the to be integrated apps and identifying their integration points.
- Designing the architecture of the integrated system:
- Mapping data flows between the apps to be integrated.
- (if required) Planning data transformations into a different format.
- Planning security measures for data, its location, and its transmitting processes.
- Defining the required performance of the future integrated solution.
- Choosing the integration approach (API-based, event/message-based, etc.), choosing the AWS services to use, and deciding what parts of the integrated system they will replace.
- (optional) Planning a pilot project to prove the effectiveness of the chosen AWS integration services.
- (if required) Planning code adjustments in the applications under integration.
Note: App modernization may take from 2-3 weeks to 2-3 months.
Step 3. Implementing and testing the integration
Duration: depends on the chosen integration approach and the characteristics of the to-be integrated applications
- Implementing the required code-level changes to the applications under integration.
- Connecting applications:
- Setting up the log messages/events to monitor the performance of the integrated solution (using Amazon CloudWatch, AWS X-Ray, or any other preferred observability solution).
- Testing the integration, including:
- All uni-, bidirectional, and asynchronous (orchestration) queries between the connected applications to ensure the completeness of distributed transactions and data consistency.
- Failover and recovery mechanisms so that no issues can spread over and disrupt the work of the integrated apps.
- The performance, scalability, and security of the integrated solution.
- Tracking the value of the integration project based on the defined business goals for application integration.
- Plans the AWS-based app integration project, assigns integration project responsibilities and tasks to the team, and controls the tasks’ execution.
- Makes sure the integration project is implemented on time and budget and in compliance with the set quality metrics.
- Reports on the AWS-based integration project progress to the key stakeholders.
- Elicits, prioritizes, and documents the requirements for the AWS-based integrated system.
AWS Solution Architect
- Designs the integration architecture to meet the business and technology requirements.
- (if required) Develops the architecture transformation and modernization roadmap for the to be integrated apps.
- Designs frameworks and processes to support API development and management.
- Defines AWS services to employ.
- Configures integrations (e.g., in case of Amazon AppFlow, AWS Step Functions), builds custom integration code and APIs.
- Plans, configures, and optimizes the AWS-based integration solution infrastructure.
- Automates development and release processes by introducing a CI/CD pipeline.
- Monitors the integrated system security, performance, and availability, etc.
- Designs and implements a test strategy, a test plan, and test cases for the AWS-based integration solution and the integrated system to assure that the functional, performance, security, and compliance requirements are met.
- Conducts manual and automated testing.
- Tracks test results and reports bugs.
The entire AWS-based integration project is in-house
- Full control over the AWS integration project progress.
- Easy team communication.
- Need to hire specific resources with experience in AWS services, modern integration patterns, legacy app modernization.
- All hiring and managerial efforts are on your side.
The AWS-based integration project is outsourced
- A vendor assumes full responsibility for the team assembly and management and the quality of the AWS integration project delivery.
- Established frameworks for AWS-based app integration.
- High vendor risks.
A mix of in-house team and outsourced consultancy for AWS-based integration
- In-house team has a deep understanding of internal business processes and the software environment.
- Outsourced consultancy provides expert guidance for the most challenging integration activities, shares specific AWS knowledge, trains your team to maintain the integrated solution efficiently.
- Risks of consultancy vendor selection.
- Requires time and experience to establish smooth collaboration and communication between the in-house and consultancy teams.
- Partial or total project team coordination, quality control and risk management are required from your side.
Basic integration cost factors
- The chosen integration approach (point-to-point, mediated integration, workflow-based integration).
- The number of integrated apps.
- The complexity of the new integrated workflow (whether it can be created codeless, with minimal code adjustments, or by writing custom code).
- The availability of ready APIs and connectors.
- Efforts for testing the co-coordination and data consistency between the integrated systems.
Costs of AWS services
The cost of some AWS services will depend on:
- Number of operations for a certain period (messages, events, delivery attempts, management calls, etc.).
- Message size.
- Using specific features (message transactions, de-duplications, etc.).
- Downtimes/interruptions caused by integration ramp-up.
- Business user training.
AWS Simple Notification Service (SNS)
High-throughput, push-based, many-to-many messaging between serverless functions, queues, and distributed systems.
- Strategies for reliable message delivery, e.g., dead-letter queues (messages that can't be delivered due to client/server errors will be saved for reprocessing or further analysis).
- Automated scalability up to millions of messages per second.
- FIFO (First-In-First-Out) topics enable message processing in a strict order and with no duplications.
- Message filtering and fanout.
- First 1 mln Amazon SNS requests per month are free, $0.50 per 1 mln requests thereafter.
- Published API requests are ~ $0.3 – 0.5 per 1 mln requests and ~ $0.02 – 0.03 per 1 GB of payload data.
- Subscription messages are ~ $0.01 – 0.02 per 1 mln messages and ~ $0.001 – 0.002 per 1 GB of payload data.
- Topic owner and subscription owner API requests are charged as Standard API requests.
AWS Simple Queue Service (SQS)
Sending, storing and receiving messages between software components at any volume, without requiring other services to be available.
- Standard message queues for maximum throughput, best-effort ordering, and at-least-once delivery.
- FIFO (First-In-First-Out) message queues to guarantee that messages are processed exactly once, in the exact order that they are sent.
- Automated scaling via increase in the number of listeners/consumers, the number of threads per client or combined, via action batching.
- Server-side encryption (SSE) can be used to encrypt each message body for sensitive data exchange.
1 mln Amazon SQS requests for free each month.
~ $0.40 per 1 mln requests if the number of requests doesn’t exceed 100 bln/month.
~$ 0.50 per 1 mln requests.
AWS API Gateway
Building and operating APIs.
- Dashboards to visually monitor calls to the services.
- Integration with Amazon CloudWatch to track metrics such as API calls, latency, and error rates.
- Throttling and cashing.
- Fine-grained API access permissions.
- API keys to control access for third-party developers.
- Custom plans that set throttling and request quota limits for each API key.
- Multiple release stages for each API version (alpha, beta, production). Each API stage can be configured to interact with different back-end endpoints.
- Running multiple API versions simultaneously allows applications to call previous API versions even after the latest versions are published.
1 mln HTTP API calls, 1 mln REST API calls, 1 mln messages, and 750,000 connection minutes per month for free for up to 12 months. If you exceed this number of calls per month, you will be charged according to API Gateway usage rates.
ScienceSoft’s Offering in AWS-based Application Integration
AWS-based application integration consulting
- Elicit business and technical requirements for the integrated system.
- Choose the integration approach and the tech stack.
- Plan required modifications of the integrated applications.
- Help comply with the regulations, including HIPAA, HITECH, SOC2, SOC3, and more.
- Deliver a detailed AWS-based app integration project roadmap, including the integration approach and AWS services to use.
AWS-based application integration implementation
Our team takes over:
- Requirements elicitation and prioritization for the app integration project.
- Application and infrastructure analysis.
- The AWS-based integrated system’s architecture design.
- Modernization (re-engineering, re-architecting) of applications before integration (if required).
- Integration solution development.
- Implementation and testing (at system and component levels) of the AWS-based integrated system.
- After-launch support, user and admin training.
- Continuous management and evolution of the AWS-based integrated system (on demand).
A global IT services company headquartered in McKinney, Texas, ScienceSoft provides end-to-end application integration services.
- 8 years of experience with AWS-based infrastructures.
- The status of Amazon Web Services Select-Tier Consulting Partner.
- AWS Certified Solution Architects, AWS Certified SysOps Administrators, etc., on board.
As the holder of ISO 9001 and ISO 27001 certifications, we rely on mature quality management and guarantee that the cooperation with us doesn’t pose any risks to our customers’ data security.