Back to The EDiT Journal
EDT&Partners - Octopus BI
EDT&Partners helped OctopusBI optimize its AWS data infrastructure, improving speed, scalability, and cost efficiency through differential data migration, Lambda deployment, and Redshift optimization.
EDT&Partners
In this article
1. Introduce Differential data migration to the agent:
2. The Data Engine Service deployment as a Lambda functions
3. More efficient data storage
4. About Octopus
This article outlines how EDT&Partners supported OctopusBI in optimizing its data architecture through AWS-powered improvements. By introducing differential data migration, deploying the Data Engine Service via Lambda functions, and enhancing data storage in Redshift, EDT&Partners helped reduce costs, improve scalability, and accelerate performance. The collaboration reinforces EDT’s expertise in driving cloud innovation for leading EdTech providers.
1. Introduce Differential data migration to the agent:
Review current algorithm to optimize data usage. The potential advantages of this improvements include:
- Faster execution time for the agent
- Faster upload time to AWS S3
- Decreased data transfer cost
- Increases the ability to efficiently scale the number of clients
2. The Data Engine Service deployment as a Lambda functions
The data can be queried from S3 using pagination, transformed and uploaded in Redshift:
Data Engine
Single instance of AWS Fargate. Continuously querying, transforming, and inserting massive amounts of data.
- Lambda is a serverless service, allowing for concurrent execution, and a lower execution cost.
- Introducing pagination increases concurrency and decreases execution cost.
- Differential data introduced in step one will significantly decrease the execution time and cost of this operation.
3. More efficient data storage
Better storage and implementation of clients Data in Redshift.
Cloudwatch
Amazon S3
Lambada
4. About Octopus
OctopusBI’s mission is to empower education businesses with invaluable insights from data to transform future outcomes. At its core, OctopusBI is a Business Intelligence platform for education and a master connector that solves data problems for a holistic view of student, classroom and business operations. Currently, OctopusBI goes to market mostly in the K12 segment with two differentiated products: OctopusBI Premium and Tentacle.
Currently OctopusBI is presently expanding its user base, workloads and use cases, presenting a number of technology challenges and concerns around the future growth stages for the company.
.png)
.png)
