Big Data Platform MVP Development for a Global Consulting Company
The Customer is an international company that provides business and technology consulting services for large-scale construction projects in oil&gas, chemical, power, mining, pharmaceutical, and other sectors.
The Customer wanted to develop from scratch a complex platform to streamline the delivery of consulting services to their clients. In a long-term perspective, the platform was expected to be enriched with machine learning (ML) capabilities to provide the Customer’s clients with recommendations on their construction projects. The Customer lacked competencies to develop the solution in-house and was looking for a reliable vendor to deliver the MVP.
ScienceSoft provided the Customer with dedicated specialists to build the front end and the back end of the solution. Our team worked with the Customer’s project manager based on the Agile methodology with one or two-week sprints.
In close collaboration with the Customer, ScienceSoft’s experts have prioritized the solution’s functionality, based on the software requirements specification. Among the priority features were the secure storage of voluminous client data, client data archiving, and advanced data processing capabilities. When designing the solution architecture, the team took into account the scalability requirements and used Docker containers with Docker Swarm to achieve that.
In the course of 10 months, ScienceSoft delivered a complex MVP based on Delta Lake with a mechanism for multi-layered data storage. The platform enabled quick processing of heterogeneous data on the Customer’s projects from multiple sources and the possibility to track the record of the data added, modified and deleted. The Customer could extract any of the existing project versions in JSON, CSV, Excel, PDF formats, amend, and compare it with other versions, which facilitated the work on long-lasting construction projects. All alternations made in the versions could be automatically highlighted with different colors, making it convenient for the users to analyze the changes. Besides, ScienceSoft’s experts set a notification system to inform users on successful completion of certain operations, e.g., the compression of a large file.
The Customer was completely satisfied with the MVP that enabled collaboration on voluminous project data and facilitated the delivery of consulting services to the Customer’s clients. The delivered solution is currently being enriched with machine learning capabilities.