Healthcare Data Warehouse: Overview
ScienceSoft has been providing a full range of data warehousing services since 2005.
Data Warehouse in healthcare: The fundamentals
A healthcare data warehouse is a centralized repository for a healthcare organization’s data retrieved from disparate sources, processed and structured for analytical querying and reporting. The healthcare data warehouse integrates with a data lake, ML and BI software. Implementation costs for the healthcare DWH start from $200,000 for a midsize healthcare company.
A data source layer
– healthcare data from internal and external data sources (ERP, EHR/EMR, CRM, claims management system, pharmacy management systems, etc.).
A staging area
– intermediate temporary storage, where healthcare data undergoes the extract, transform and load (ETL) or the extract, load and transform (ELT) process.
Data storage layer
– includes centralized structured storage. It may also have data marts – healthcare DWH subsets oriented to a specific business line (HR, accounting, etc.) or department (radiology, intensive care, pediatrics, etc.).
Analytics and BI
– business analytics, data mining, data reporting and visualization tools.
Data integration
- Ingesting structured, semi-structured, unstructured healthcare data (from EHR systems, ERP, HR management systems, public medical databases, claims management systems, etc.).
- ETL/ELT-based healthcare data integration.
- Full and incremental healthcare data extraction/load.
- Controlled healthcare data loading/management.
- Healthcare data transformation of varying complexity (data type conversion, summarization, etc.).
- Healthcare data loading and querying using SQL.
- Big data ingestion.
- Streaming data ingestion.
Data storage
- Integrated, historical, summarized, subject-oriented healthcare data storage.
- Protected Health Information (PHI) storage.
- Metadata storage.
- Options of healthcare data storage environments (cloud, on-premises, hybrid).
Database performance and reliability
- Elastic scaling of storage and compute resources.
- High-performance query processing due to healthcare data indexing, materialized view support, result-caching.
- ML capabilities to dynamically manage performance and concurrency.
- Automated data backup across various regions and zones within the cloud environment for fault tolerance and disaster recovery.
Security and compliance
- Granular row and column level security control.
- Multi-factor authentication.
- Healthcare data encryption at rest and in transit (including backups and network connections).
- Dynamic healthcare data masking.
- Ongoing threat detection and vulnerability assessment.
- Compliance with healthcare regulations (HIPAA, FDA, HITECH, etc.).
A data lake
Data lakes serve as a cost-effective repository of semi-structured and unstructured healthcare data at any scale (radiology images, audio/video recordings, streaming healthcare data from wearables and devices, etc.). The data lake keeps data before it is queried by the data warehouse, which stores only highly structured healthcare data ready for analysis. Data stored in the data lake can be further used to develop ML models (for example, for medical imaging diagnosis).
Machine learning (ML) software
This integration enables training ML models on structured healthcare data from the data warehouse. ML-powered advanced analytics helps predict clinical outcomes, deliver personalized healthcare recommendations, improve appointment scheduling, make grounded decisions about hospital spending, etc.
Healthcare data structured in the data warehouse is visualized and reported in immersive reports (hospital annual report, average hospital stay report, etc.) and interactive dashboards (patient demographics, physician allocation, insurance claims, etc.). Self-service BI tools facilitate shorter time-to-insight.
What determines medical data warehousing success
Proof of Concept
Validate your healthcare DWH solution with a PoC to better understand its real potential and get real-life user feedback.
Healthcare DWH scalability and flexibility
To instantly upload any type (structured, semi-structured, unstructured) and amount of healthcare-related data to efficiently address new data analytics objectives.
Focus on security and healthcare data protection measures
Store and process sensitive patient data within highly secure environments (AWS, Microsoft Azure, Google Cloud or private servers), ensure all-time data encryption and dynamic data masking, restrictive data access, multi-factor user authentication, healthcare DWH vulnerability assessment and penetration testing, etc.
Well-established data quality management
To ensure high quality of data delivered from diverse data sources, conduct a comprehensive data warehouse system analysis and design robust data governance practices. It will help deal with such common data quality challenges as different encoding formats, attribute measurements of different data source systems, conflicting key fields, etc.
Healthcare data warehouse investments
The cost of a healthcare data warehouse implementation project varies for healthcare organizations of different size as follows:
*Monthly software license fee and other regular fees are NOT included. |
|
Healthcare data warehouse key cost drivers:
- Number of healthcare data sources (ERP, EHR/EMR, CRM, claims management system, pharmacy management systems, etc.).
- Healthcare data disparity (for example, difference in data structure, format, and use of values) across various source systems.
- Complexity of healthcare data (for example, big data, streaming data).
- Volume of healthcare data to be processed.
- Healthcare data security requirements.
- Number of healthcare data tables and columns used for analysis.
- Healthcare data warehouse performance requirements (velocity, scalability, fault tolerance, etc.).
Key financial outcomes of a healthcare DWH
As a central element of a BI solution, healthcare DWH enables consolidation of disparate healthcare data sources into a structured healthcare data repository ready for analysis to improve business and clinical decision-making and to:
|
Improve health outcomes. |
|
Accelerate data-driven labor management. |
|
Improve healthcare resource management. |
|
Personalize care delivery and improve patients’ experience. |
|
Decrease healthcare operating costs. |
SUCCESS STORIES OF HEALTHCARE DATA WAREHOUSE IMPLEMENTATION
DWH and BI Implementation for 200 Healthcare Centers
ScienceSoft assisted in developing an analytical data warehouse to allow healthcare centers and retirement homes to analyze and report data on medication inventory, clinical services, etc. from 200 databases.
Implementation of a DWH and Analytics Solution for 500+ nursery homes
To establish a standardized and comprehensive role-based reporting for a US company that renders services to 500+ nursing homes, ScienceSoft developed a DWH solution with a universal analytical cube.
DWH and BI Implementation for a Medical Provider of Mobile Diagnostic Imaging Services
ScienceSoft implemented a data warehouse and a two-analytical-cube solution to enable a US company that renders mobile X-ray, ultrasound, echocardiogram, EKG, and bone density testing services to 800+ facilities to track the efficiency of provided services.
Amazon Redshift
Best for: big data warehousing
Features
- Integration of all healthcare data types (structured, semi-structured, unstructured) for storing and SQL-querying.
- Integrations with the AWS ecosystem (including S3, AWS Glue, Amazon EMR) and third-party tools (Power BI, Tableau, Informatica, Qlik, Talend Cloud).
- Federated queries support.
- ML capabilities for optimized performance under varying workloads.
- Separate scaling of compute and storage.
- Healthcare data encryption and fine-grained access control.
- HIPAA-compliant.
Pricing
- On-demand pricing – $0.25 – $13.04/hour.
- Reserved instance pricing offers saving up to 75% over the on-demand option (a 3-year term).
- Data storage (RA3 node types): $0.024/GB/month.
Azure Synapse Analytics
Best for: advanced data analysis
Features
- SQL-querying of structured, semi-structured, unstructured healthcare data.
- Native integrations with a data lake, operational databases, BI and ML software.
- Integration with third-party BI tools, including Tableau, SAS, Qlik, etc.
- Separate billing for compute and storage.
- Healthcare data encryption, dynamic healthcare data masking, column- and row-level security.
- HIPAA-compliant.
Pricing
- Compute on-demand pricing – $1.20–$360/hour.
- Compute reserved instance pricing allows saving up to 65% over the on-demand option (a 3-year term).
- Data storage: $122.88/TB/month.
Oracle Autonomous Data Warehouse
Best for: hybrid healthcare DWHs
Features
- Querying across multiple healthcare data types (structured, semi-structured, unstructured).
- Built-in connectivity to Oracle Cloud Infrastructure Object Storage, Azure Blob Storage, Amazon S3.
- Integration with Oracle Analytics Desktop and third-party BI tools (Microsoft Power BI, Tableau, MicroStrategy, Qlik, etc.).
- Healthcare data encryption, privileged user and multifactor access control.
- Independent scaling of storage and compute.
- HIPAA-compliant.
Pricing
- Compute costs: $1.3441/CPU/hour
- Data storage: $118.40/TB/mo (in the public cloud).
Healthcare DWH implementation with ScienceSoft
Since 2005, ScienceSoft has offered IT solutions to healthcare organizations and provided a full range of data warehousing services to help them build robust healthcare DWHs and support the decision-makers with high-quality healthcare data.
Healthcare DWH consulting
- Eliciting requirements for a future healthcare DWH solution.
- Designing a healthcare DWH implementation/migration strategy.
- Outlining the optimal healthcare DWH vendors, technology stack and its configurations.
- Advising on healthcare data integration and data quality procedures.
- Conducting admin training
Healthcare DWH implementation
- Healthcare data storage needs analysis and DWH solution architecture design.
- Healthcare data sources (ERP, EHR/EMR, CRM, claims management system, pharmacy management systems, etc.) integration into a healthcare DWH.
- DWH platform integration into the data environment (a data lake, big data platform, BI tools, etc.).
- Set up data management and metadata management procedures.
- Conduct healthcare data cleaning and data migration.
- User training.
- DWH support and evolution (if required).
About ScienceSoft
ScienceSoft is a global IT consulting and IT service vendor headquartered in McKinney, TX, US. Since 2005, we have provided a full range of data warehousing services to help healthcare organizations build from scratch or enhance their existing data warehouse platforms within the set timeframes and with minimal investments. Being ISO 13485 certified, we design, develop and test high-quality medical IT solutions according to the requirements of the FDA and the Council of the European Union.