Migrating to cloud-based Oracle Autonomous Data Warehouse: Latvenergo Story

In 2022, Latvenergo moved its Billing analytics from an on-premise Oracle data warehouse with a legacy replication tool to a cloud-based Oracle Autonomous Data Warehouse (aDWH) and GoldenGate. This resulted in the possibility of acquiring fresher data, happier BI users, and teaching tech people to consider business needs and the impact of costs, among other things. This piece delves into the migration journey, detailing the challenges, decisions, and outcomes.

Latvenergo Group is one of the leading energy suppliers and the leader in green energy generation in the Baltics. In 2021, Latvenergo was recognized as the most valuable company in Latvia for the thirteenth time in a row. Scandic Fusion's relationship with Latvenergo spans over 16 years, bearing witness to the company's evolving data analytics and management needs. It's been a journey of collaboration, navigating challenges, and mutual growth.

The Challenge

In 2022, Latvenergo faced a challenge: their on-prem resources were approaching limits. As a result, they had slower and less reliable data loads, lag during BI queries, and a dip in user satisfaction. Besides that, it was known that DbVisit, a third-party replication engine and the foundation of Oracle Billing real-time analytics, would no longer be supported. Given its crucial role, the issue with DbVisit, combined with the strain on the on-prem resources, served as a catalyst for a change, and finding a timely solution became paramount.

Over the years, Scandic Fusion has been working with Latvenergo on multiple projects, while in parallel, Latvenergo has been building its internal competencies, building many things in its BI ecosystem on its own. This time around, the challenge at hand had turned into a pressing business need to solve highly technical tasks within a limited time, requiring high capacity and high competence. Therefore, Latvenergo turned to Scandic Fusion for assistance.

^"We had reached a situation where the growing amount of data loading to DWH could no longer be done in a nightly time window, and on top of that, the DbVisit technology used for online data flow was rapidly approaching the end of the support period. We were at a crossroads where we had to pick the right direction - to find a solution that solves all our problems in the shortest time. As the correct direction, a decision was made not to go north or south but up in the clouds - to migrate our most complex solution to Oracle Cloud infrastructure, which was a big challenge due to its scale and complexity. Considering the situation, the involvement of professionals from long-term partners Scandic Fusion was urgently needed.

_{Rolands Bērziņš, IT Project Manager at Latvenergo}

The Solution & Implementation Journey

Why Oracle's aDWH?

While the thought of going to the cloud had been in the air for some time, there was never an urgency for it. The situation with Oracle Billing and especially with the discontinued support for DbVisit pushed Latvenergo towards the choice because they had to ensure the system worked.

Several alternative solutions with third-party replication tools to other vendors’ data warehouse platforms were considered, but migrating to a non-Oracle platform was too expensive and the considered alternatives were less capable in the context of the specific requirements for the case at hand. Besides that, Latvenergo’s team has extensive knowledge of Oracle DWH/ELT/BI technologies, which meant there would be less of a need for a steep learning curve to gain acquire new technology know-how, saving time and money. Furthermore, the total cost of ownership (TCO) for the aDWH with GoldenGate replication and Oracle Data integrator was found to be more economical than scaling or separating the existing Oracle’s on-prem ecosystem.

In short, choosing Oracle aDWH meant:

• reuse of existing know-how;
• advanced capabilities; 
• lower immediate costsand TCO.

The transition from on-prem setup to the cloud wasn't without its technical intricacies, and therefore, it’s even more important that Latvenergo approached this project with an open mind. It was reinforced by the discovery of aDWH’s efficiency and adaptability.

The Power of aDWH

The technical benefits of aDWH were evident, its power coming from two main things: data compression and parallel execution with autoscaling.

While many might not pay attention to it, data compression is actually really important. One of the challenges Latvenergo faced was speed. For BI users, the interaction with the reporting platform had become slow and burdensome. Data compression significantly enhanced data retrieval speeds with aDWH leveraging Exadata Hybrid Columnar Compression (EHCC).  

Additionally, the autoscaling feature provided a new level of flexibility, boosting CPU power during peak periods. And a good thing about the cloud is that you don't have to worry about running out of disk space.

However, as with any technology, aDWH presented its own set of challenges. Certain sequential operations showed slower performance compared to on-prem solutions, and the autonomous self-tuning capabilities were not fully realized in real-time executions. Nevertheless, other components compensated for these disadvantages.

As a result, we are proud to say that we made it happen. We managed to move on-prem DB to cloud, while BI stayed on-prem. ETL finishes on time, data is fresher, and BI users are happier due to more dedicated capacity, which results in reduced query time.

Lessons Learned and Looking Ahead

^"A project like this - migration of a business-critical system to a new technology platform in a limited time frame - sets high expectations. Even with our long-standing partnership with Latvenergo Group in data analytics, we are grateful that Latvenergo invested their trust in our team of professionals and believed that we could help them achieve the project goals. During these 6 months we time-travelled through Dunning-Kruger effect reinforcing the importance of knowledge and close collaboration with the customer's team!
‍
_{Kārlis Vītols, CEO & Data Analytics Architect at Scandic Fusion}

While the transition journey had its share of hurdles, the outcome was a more robust, scalable, and efficient data management system for Latvenergo. And for us, at Scandic Fusion, a pool of lessons learned for future projects:

• With big projects like this and the time pressure to execute ASAP, 100% pre-planning is impossible. The best way forward is by small steps, and it’s good to have competent people on the client side with open minds and ready to adjust to the situation.
• Take the time todo proper research before starting the project. If the initial research is too narrow, today’s expectations might not match future abilities.
• It’s always a good idea to start with a small experiment because it gives you the space for trial and error, and an option to choose the best next step.
• Last but not least, continuously improving your tech knowledge during and outside the project is very important.