Adaptive Design to Improve Customer Satisfaction
The right customer data and responsive design principles guide program optimization. Utilities are finding that customer expectations around energy programs are rising. Other industries have raised the bar for customer experience and digital engagement, with advanced systems to centralize and coordinate program and customer data. As a result, CIOs are often pulled into critical system decisions surrounding data, deciding how to best leverage systems and data segments to simultaneously improve customer satisfaction metrics and deliver improved energy program results. Beyond knowledge of their systems and data requirements, today’s utility CIOs need modern analytic approaches to make sense of this data and enable utilities to deliver products and services that perfectly fit customer desires.
Enter Adaptive Design
Retailers, financial service providers and many other customer-facing companies are constantly making subtle changes in their customer messaging content to conduct thousands of small consumer experiments. The companies then use the data they collect to simultaneously improve the next customer outreach effort while testing new concepts; this approach is often referred to as “Test and Learn” and is done at scale.
Today’s utility CIOs need modern analytic approaches to make sense of the data and enable utilities to deliver products and services that perfectly fit customer desires
Over the past five years, working closely with our utility customers, Nexant has pioneered the adaptation of this concept specifically for utility Demand Side Management (DSM) programs, which include energy efficiency, demand response, and behavioral modification programs. We call it the Adaptive Design Process (ADP). ADP tests the efficacy of DSM program tactics by conducting many small experiments during the implementation of utility energy conservation programs. By continuously testing interventions and adjusting the programs in response, DSM program managers can accelerate the optimization of their processes while a program is still running, without waiting for a final evaluation.
The success of ADP is dependent on the ability to keep up with customer intentions. To achieve this goal, the ADP approach takes effective entrepreneurial tactics from Eric Ries' "Lean Startup" methodology: constantly observe and measure customer behavior and acceptance, then make quick decisions about whether to keep building out existing features, kill the ones that are not working, or introduce new features based on any unexpected customer behavior.
The ADP approach also makes use of "design thinking," a creative approach to product design used effectively by organizations like IDEO and Stanford's d.school. Design thinking focuses on empathy to drive design as opposed to starting with a solution and forcing customers to embrace it. Customers almost never know what they want; in fact, they usually want something just like the thing they have-only better. Therefore, it becomes crucial to notice gaps in usability and to look for implicit needs by observing how people work around problems.
For utilities with disparate and siloed data sources, the process of testing or piloting many programs simultaneously is a bit more difficult than for startups. In order to see what customers really want, it is important for utility CIOs to support a centralized system that can leverage multiple data sets, with a structure that allows you to test often and to integrate unstructured, ethnographic data to get a more accurate picture of consumer behavior and preferences.
Modern Ways to Measure Program Impacts
Utilities typically focus on measuring energy savings estimates by using either engineering models or empirical observations. Engineering-based estimates measure the difference of energy use before and after a specific treatment. These are good for assessing simple savings but not to measure many other types of impacts, such as those resulting from behavioral changes. Empirical studies document observable effects such as a bill comparison before and after receiving a treatment, but are not able to measure unobservable impacts (e.g. if the number of people living in a house changes or if a behavioral change is due to a new job with a new schedule).
CIOs can ensure that evaluations are more robust by freeing data to support randomized control trials (RCTs), a well-known form of experimental research. RCTs work by evaluating a treatment group (those who receive a treatment) against a control group (those who do not). When applied to energy efficiency programs, a well-designed RCT can help an evaluator answer the question –“What would have happened in the absence of the program or specific intervention?"
Accelerating Program Optimization and Improving Cost-Effectiveness
The typical program cycle can take multiple years, from identifying an opportunity, to picking a solution and piloting or testing it, to improving the program based on program outcomes. Even if the program is deemed successful, it cannot be said that it is “optimized” since there is no assessment of the opportunity cost or the potential benefits of all the untested solutions compared to the implemented option.
The ADP identifies and tests multiple opportunities and design concepts simultaneously and adjusts the program responsively. Program managers and implementers can optimize the offering to any chosen combination of program outcomes, such as increasing per-customer savings, maximizing participation, or improving overall cost-effectiveness. Thus, programs immediately reap the benefits of improvements to their effectiveness and resource utilization.
The Right Systems and Services
Utilities already have the data and technology platforms to keep up with customer expectations; the way to create value from existing data platforms is to take advantage of today's extended program cycles to glean rich and actionable insight about their customers through the Adaptive Design Process. Not only does this serve customers better, it also maximizes cost-effectiveness by pursuing the optimal intervention combinations for each market segment.
To create a culture of innovation, utilities need to nurture curiosity-about their customers, about technology, about possible new outcomes. Adaptive design allows utilities to foster a new perspective on value creation, discover end-user needs, test expectations, and ultimately learn more about their customers to ensure their own future success.