EDT&Partners - Octopus BI
Table of contents
1. Introduce Differential data migration to the agent:
2. The Data Engine Service deployment as a Lambda functions
3. More efficient data storage
4. About Octopus
1. Introduce Differential data migration to the agent:
Review current algorithm to optimize data usage.
The potential advantages of this improvements include:

Faster execution time for the agent

Faster upload time to AWS S3

Decreased data transfer cost

Increases the ability to efficiently scale the number of clients
2. The Data Engine Service deployment as a Lambda functions
The data can be queried from S3 using pagination, transformed and uploaded in Redshift:
Data Engine
Single instance of AWS Fargate.
Continuously querying, transforming, and inserting massive amounts of data.
- Lambda is a serverless service, allowing for concurrent execution, and a lower execution cost.
- Introducing pagination increases concurrency and decreases execution cost.
- Differential data introduced in step one will significantly decrease the execution time and cost of this operation.
3. More efficient data storage

Better storage and implementation of clients Data in Redshift.

Cloudwatch

Amazon S3

Lambada
4. About Octopus

OctopusBI’s mission is to empower education businesses with invaluable insights from data to transform future outcomes. At its core, OctopusBI is a Business Intelligence platform for education and a master connector that solves data problems for a holistic view of student, classroom and business operations. Currently, OctopusBI goes to market mostly in the K12 segment with two differentiated products: OctopusBI Premium and Tentacle.
Currently OctopusBI is presently expanding its user base, workloads and use cases, presenting a number of technology challenges and concerns around the future growth stages for the company.