Beyond the Hype: How Machine Learning is Transforming Energy Markets
The energy sector stands at a fascinating crossroads where traditional optimization meets cutting-edge machine learning. In a recent conversation our experts Diego Roa and Viking Björk Friström shared their insights on where the real opportunities lie—and why many organizations are still struggling to bridge the gap between promising prototypes and production-ready solutions.
The Production Gap Challenge
Both experts see a persistent challenge across the industry: teams excel at building models but struggle with productionization. For example many data science teams remain isolated, working primarily in Jupyter notebooks without the software engineering rigor needed for reliable, scalable systems. "There's always this structural issue with data science teams," Diego explains. "They hire talented data scientists who know how to build models, but there's a gap when it comes to really building products—end-to-end solutions with proper testing, monitoring, and deployment practices." Viking echoes this concern from an operational perspective, particularly around observability. "People have figured out how to put models into production quickly, but then they sort of forget about them and move on to the next problem. There's often no continuous follow-up on model performance."
Where Energy Gets Interesting: Trading and Grid Optimization
Energy presents unique technical challenges that go far beyond traditional ML applications. Viking describes how the sector is experiencing a significant shift, particularly in trading algorithms. "Intraday energy trading is becoming very similar to stock market trading," he notes. "You can basically take fast frequency trading algorithms from financial markets and apply them directly to energy markets—and it works." However, the broader energy landscape remains a complex optimization problem at massive scale. Viking frames it perfectly: "The whole grid is just a big optimization problem whether you're zooming into a small generator or looking at the national grid, it’s all about optimizing somewhere. You have people who want energy, producers who create it, and it all needs to be balanced while respecting the physical constraints of transmission lines."
The Netlight Advantage: Product Mindset Meets Technical Depth
Both experts see the key in bringing a product-first mentality to machine learning projects, as per Netlight’s mindset. Diego emphasizes how this approach prevents teams from getting caught up in technical novelty without business impact. " Sometimes, teams jump headfirst into building something cool, but they lose sight of the business problem they’re trying to solve. Do we even need ML here? Or could a simple rule-based system be just as effective?” Diego reflects. This practical approach extends to implementation strategy. Rather than replacing existing tools entirely, the focus is on improving processes. Even when teams continue using Jupyter notebooks, Diego advocates for better practices: centralized code libraries, proper review processes, staging environments, and quality checks.
The Simulation Revolution
Simulations are particularly powerful tools for exploring ideas in the energy field. Viking has recently worked for example on sophisticated grid simulations using frameworks like PyGrid, developed by MIT, which allow teams to model complex scenarios: "You can simulate where to optimally place a data center from a grid perspective, or model how adding battery storage would affect grid stability." For Diego, simulation extends beyond technical modeling to business case validation: "We're seeing a lot of companies building similar simulations with different time series to prove or kill business ideas."
Looking Ahead: Integration Coupled With Innovation
Perhaps the most insightful takeaway is that success in energy ML isn't about having the most advanced algorithms—it's about integration and practical application. "Energy is chaotic, beautiful, and unforgiving. It’s an optimization challenge with real-world consequences, and where you look for solutions often determines your ability to make a difference,” Viking says. His experience with mixed linear programming often outperforming genetic algorithms illustrates this perfectly: "We start with very simple optimization models and iteratively add constraints until we have something that reflects reality. Sometimes the classical approaches work better than the fancy ones." "ML is so use-case dependent.” Diego agrees, “You can build any model, but if you're not using the right data or don't understand the business context, you won't generate value. It’s about reinforcing that connection between technical solutions and business needs.”
The Path Forward
As the energy sector continues its digital transformation, the most successful organizations will be those that combine deep domain expertise with rigorous engineering practices. The winners won't necessarily be those with the most sophisticated AI—they'll be the ones who can reliably deploy, monitor, and iterate on solutions that solve real problems. The path for ML in energy isn’t a straight line, but one thing is obvious from this discussion: success requires balance. The right tools, disciplined processes, and a collaborative mindset are what push ML efforts beyond prototypes into solutions that truly light up the grid. Combining theoretical depth with a product-oriented approach, teams can tackle this complex, dynamic sector with agility and purpose. Because at the end of the day, energy isn’t just about powering systems. It’s about solving problems that connect us all.
