Why the sustainable provision of low-carbon electricity needs hybrid markets

Deep decarbonization of energy systems poses considerable challenges to electricity markets and there is a growing consensus that an energy-only design based on short-term marginal cost pricing cannot deliver adequate levels of investment and long-term coordination across actors and sectors. Based on the instructive example of the evolution of European electricity market designs, we discuss several shortcomings of energy-only markets and illustrate how ad-hoc policies that intend to address them have limitations of their own, notably a lack of systemwide coordination. Second, we describe how the sheer scale and nature of deep decarbonization targets requiring massive investment in capital-intensive low-carbon technologies exacerbate these issues. Ambitious emission reduction targets thus require an evolution of market design towards hybrid regimes. Hybrid markets separate long-term investment decisions from short-term operations through a balanced and differentiated use of competitive and regulatory design elements to coordinate and de-risk investment. Finally, a historical analysis of the evolution of different electricity market designs shows how hybrid markets constitute contemporary forms of long-run marginal cost pricing that are appropriate for meeting deep decarbonization targets with reduced uncertainty and hence lower private and social costs.


Introduction
In energy-only markets (EOM), competitive short-term prices should drive the cost-effective use and dispatch of existing generation assets in the short run as well as the coordination of capacity investments and shutdowns towards the socially optimal generation mix in the long run.In principle, all assets break even and recoup their fixed investment costs in the long-run equilibrium.This holds even in the presence of large shares of intermittent assets with zero short-run marginal cost as long as a sufficient number of scarcity hours are allowed for (Hogan, 2022).Yet, there is a growing consensus among scholars and practitioners that both the ideal and current market designsrespectively, a pure EOM and an EOM supplemented by various ad-hoc policiesfall short of short of ensuring security of supply and the deep decarbonization of energy systems as economically as possible and on schedule (e.g.Roques and Finon, 2017;Newbery, 2018;Blazquez et al., 2020;Joskow, 2022;Wolak, 2022).Similar concerns have recently become apparent also in regulatory and political discussions in the EU, reinforced by concerns about the speed of the energy transition and the security of energy supply in a context of unprecedented high price levels in energy markets (CEER, 2021;ACER, 2022;EC, 2022).As we will see, this debate is hardly new but assumes greater urgency in this context.
Against this background, the objectives of this paper and its contributions to the literature are threefold.First, the paper establishes a diagnosis of the shortcomings of the ideal and current market designs based on a literature review (Section 2).It identifies four main issues, namely security of supply externalities, innovation externalities and industrial or social preferences, climate change externality, and missing long-term markets for electricity provision.The main focus of this paper is on the EU.Yet, other jurisdictions share similar features, and our analysis remains relevant in these instances too.On its own, each issue is amenable to specific corrective intervention, but in practice such ad-hoc remedies also have limits and are added on top of one another without sufficient systemwide coherency and coordination.The ensuing multilayered policy environment conveys conflicting signals and suffers from adverse interactions, making it difficult to navigate for market participants and investors.Taken together, these issues thus challenge the idea that free market provision delivers first-best investment outcomes.
As a second contribution, the paper describes how the energy transition exacerbates the problem of adequate investment in low-carbon generation capacity (Section 3).That is, although this problem is inherent to an EOM, the issues identified previously are magnified by the permeation of intermittent renewable energy at scale and the urgency to transform the energy mix with the need for sizable capital-intensive investment and decommissioning.We illustrate two important aspects: first, investment is hindered or made more expensive because of higher capital costs resulting from high uncertainty in the short to medium term (market prices and rents become more volatile, making risks more difficult to hedge) as well as in the long term (future market conditions and prices are deeply uncertain, implying risks that for structural reasons such as unknown or unknowable probability distributions cannot be hedged).Second, decarbonization entails the risk of proliferation of uncoordinated ad-hoc remedies on top of short-term markets to achieve political targets.Absent a design overhaul that clarifies the roles of markets and ad-hoc regulations, this further lowers the performance and consistency of the policy patchwork, which negatively affects the generation mix and increases generation costs.
In line with other scholars (e.g.Roques and Finon, 2017;Joskow, 2022), we next briefly describe a bifurcated evolution of the current market design into a hybrid market model which has the potential to overcome the identified shortcomings and deliver on deep decarbonization targets (Section 4).A hybrid market consists of two modulesa long-term module that separates and de-risks investment decisions from short-term prices via long-term contractual arrangements, and a short-term module which harnesses the forces of competitive wholesale markets to exploit existing assets cost-effectively as at present.We outline key challenges and tradeoffs associated with the design and articulation of these modules.
Finally, as a third contribution, the paper discusses the conceptual basis for hybrid markets in a historical and international perspective as a solution combining centralized and decentralized elements to achieve socially optimal long-run marginal pricing, which Boiteux established as the appropriate normative reference for electricity provision (Section 5).Hybrid designs are in fact contemporary forms of long-run marginal cost pricing that are fit for today's policy context and political targets.They differ substantially from the regulated systems that prevailed before the 1990s, whose underperformance led to the deregulation of electricity markets.In a nutshell, if the priority at that time was operational efficiency, today it is rapid and massive investment in low-carbon generation.Hybrid markets therefore neither constitute a radical departure from current practices nor an abandonment of the benefits that 25 years of market liberalization have brought.The challenge is instead to permit a more coherent, integrated application of existing economic instruments in conjunction with competitive wholesale markets to achieve the low-carbon transition and security of supply.

Issues with current and target market design models in the EU
In this section, we identify four main issues with both the EOM and the current market design in the EU (an EOM flanked by various ad-hoc policies) in driving long-term investment towards decarbonization and reliability targets in a timely and cost-effective manner.Although these issues are in part interrelated and mutually reinforcing, 1 we discuss them here at first in isolation to clearly delineate and diagnose market design shortcomings.We also argue that, while on its own each issue is amenable to specific ad-hoc remedies, taken together they challenge the idea that free market provision delivers first-best solutions.These issues are not fundamentally new, but we will explain how they are magnified in decarbonized energy systems (Section 3).
In an EOM, the market clearing price equals the marginal producer's variable cost outside of scarcity hours.When supply is scarce relative to load and generation capacity is fully utilized, the market price should be able to rise above the variable cost of the last (costliest) available generation unit.Such scarcity pricing is needed to ensure that a long-term equilibrium exists in which all generators recoup fixed costs.In this equilibrium, the energy mix and overall capacity are welfare optimal given relative technology costs and demand fundamentals.The equilibrium price reflects not only the variable cost, but also the opportunity cost of capacity (i.e. a scarcity premium is de facto factored in the energy market price).Short-term prices should therefore efficiently guide the dispatch of capacity units in the short run and the coordination of capacity investments and closures leading to the socially optimal generation mix in the long run.
While short-term pricing has proven to work well in the optimization of dispatch, its ability to steer the long-term mix adequately however rests on a set of demanding assumptions that do not hold in practice (e.g.Rodilla and Batlle, 2012).In fact, relaxing idealistic investor behavior assumptions (e.g.perfect rationality and information, perfect coordination between investment and decommissioning decisions) can generate energy mix trajectories that considerably deviate from optimality, even in the first best case of an efficient EOM with robust carbon pricing (e.g.Kraan et al., 2019;Lebeau et al., 2021).As we discuss below, various other externalities and market failures also warrant internalization and corrective intervention.

Issue 1: Security of supply externalities
The issue.Underinvestment in capacity required to ensure generation adequacy is commonly attributed to missing moneythat is, revenues or inframarginal rents that generators receive in energy markets are insufficient to cover investment costs in full (e.g.Joskow, 2008a;Cramton et al., 2013;Newbery, 2016).As Fabra (2018) summarizes, however, there are two contrasting views as to what the underlying reasons for the missing problem are.The first claims that price caps are the root cause that stifles scarcity pricing and thus must be lifted in line with a direct application of the EOM paradigm (e.g.Hogan, 2022).The second contends that capacity and reliability have intrinsic public good values that cannot fully be priced even in idealized markets due to security of supply externalities that warrant internalization through 1 For illustration, let us consider one example of issue interrelation.High costs of capital (e.g.due to missing long-term markets, Issue 4) may deter investment in zero/low-carbon technologies and instead foster fossil/conventional technologies as the former are more capital-intensive than the latter.This in turn can undermine the effectiveness of carbon pricing (Issue 3), i.e. for a given carbon price the cost-optimal mix comprises less low-carbon assets the higher the capital cost (Hirth and Steckel, 2016).This implies that complementary instruments are needed to lower capital costs by reducing and spreading out remuneration risks (Steckel and Jakob, 2018) to allow for an adequate deployment of specific technologies for reliability (Issue 1) or decarbonization (Issue 2) purposes.specific mechanisms (e.g.Abbott, 2001;Joskow and Tirole, 2007;Keppler, 2017;Holmberg and Ritz, 2021).
These externalities essentially arise because security of supply has public good features, 2 i.e. one person's electricity consumption affects others' utilities without proper economic feedback (e.g.side-payment) and inter-person communication on utility impacts, see Keppler (2017) for more detail.This is because the knock-on effects of a capacity shortfall and interdependencies between individual utility functions are not fully accounted for in private decision-making that security of supply cannot fully be turned into a private good. 3These externalities are rooted in the involuntary, unexpected, or uncontrolled nature of enforced load-shedding during stress periods.Loss of load therefore entails a system-cost externality or social cost on top of the lost surplus of rationed consumers (Fabra, 2018;Holmberg and Ritz, 2021). 4 There are thus three reasons that provide support to the second view of the underlying causes of the missing money problem.First, price caps can be justified when regulators seek to preempt potential exercise of market power (Fabra, 2018) or are averse to inequality across agents in the market, i.e. concerned not only with allocative efficiency but also redistribution (Dworczak et al., 2021).Second, they do not by themselves create missing money, but only in conjunction with exit barriers or other limitations to the number of scarcity hours.Without such constraints, capacity retention by producers could generate sufficient scarcity hours to recuperate missing money even with price caps (Stoft, 2002;Keppler, 2017).Third, even in idealized markets with full information where both generators and consumers express their true costs and preferences, security of supply externalities imply that the social willingness-to-pay for additional capacity is greater than the corresponding private willingness-to-pay.Because the social costs of supply interruption exceed the value that can be captured in an EOM by the marginal capacity provider, overall capacity will always be lower in an EOM than in the social optimum. 5 Ad-hoc remedies and their limits.In practice, capacity remuneration mechanisms (CRM) ensure that predefined levels of systemwide capacity are attained in line with political or social objectives (Joskow,  2008a; Cramton et al., 2013). 6This is by and large a logically consistent policy response to address security of supply externalities (e.g.Batlle and Pérez-Arriaga, 2008;Keppler, 2017;Fabra, 2018;Aagaard and Kleit, 2022).CRMs cover a broad range of tools providing different forms of remuneration and levels of certainty over different timeframes for different technologiessee Bublitz et al. (2019) for a review.However, due to the sensitivity of electricity systems and their peak capacity needs even to small changes in the mix, demand, or contextual factors such as the availability of demand response or storage, some CRMs have generated volatile price signals, which inevitably raises capital costs (Section 2.4).

Issue 2: Innovation externalities and industrial & social preferences
The issue.In practice, various economic or political considerations outside the realm of energy markets prevail, reflecting innovation or industrial policies and social preferences for specific technologies.They should partly dictate and drive the energy mix structure as there is no reason that the mix resulting from market forces alone coincides with the desired one.Targeted policies and regulatory interventions are thus justified to factor in these considerations and internalize associated externalities, market failures and social preferences.
Ad-hoc remedies and their limits.Targeted support schemes for variable renewable energy (VRE) technologies are typical examples of such ad-hoc policies, e.g.feed-in tariffs (often with priority dispatch) or premia, and contracts for difference.They aim to support the deployment of promising but not yet market-ready technologies in a bid to bring down their costs through economies of scale, capture innovation/ learning spillovers not internalized even with efficient carbon pricing, steer the mix towards predefined targets, and de-risk investment in relatively more mature yet capital-intensive low-carbon technologies to lower finance costs (e.g.Fischer and Newell, 2008;Newbery, 2021b). 7However, because the mix is not exclusively market-driven but also steered by policymakers and regulators towards exogenously set targets, market revenue is by construction (i.e. policy choice) insufficient for all generation units to break even and recoup capital costs (Brown and Reichenberg, 2021).A particularly stark example of the law of unintended consequence is provided by the fact that VRE capacity deployed at high levels faces decreasing market returns (below average prices) due to correlated generation, i.e. the larger near-zero marginal cost VRE infeed with autocorrelation, the more depressed market prices by way of the merit-order effect, and thus the lower their market revenues (Joskow, 2011;Hirth, 2013;Blazquez et al., 2018;OECD, 2019;Eising et al., 2020).Such cannibalization of own revenues is independent from but exacerbated by the non-dispatchability of VRE. 8  Reaching renewable and decarbonization targets may in turn necessitate continued support, for otherwise market forces alone are 2 They also arise because of the complexity of the good 'security of supply', which depends on social preferences, political circumstances, the state of technology as well as behavioral and informational factors that contribute to the relative inelasticity of the short-term electricity demand function.All these aspects create transaction costs that frustrate the formation of a wellfunctioning market and adequate pricing (Coase, 1988;Keppler, 1998).
3 This applies to capacity subscriptions which are competitive market-wide mechanisms for self-rationing in which consumers choose and pay for a level of firm supply (Doorman, 2005;Doorman and Botterud, 2008).Decentralizing security of supply decisions can partly mitigate free riding, correct for limited forward contracting on the demand side and reveal individual preferences for supply adequacy, but interdependencies and transaction costs however still impede the emergence of the first best, and the key issue of equity remains sidestepped. 4Joskow and Tirole (2007) argue that market mechanisms cannot fully capture the social costs of a network collapse as consumers can do nothing to escape its consequences and generators cannot profit from it.Wolak (2022) argues that a reliability externality arises as no retailer or customer faces the full expected cost of not procuring adequate energy forward since random curtailments are imposed in case total supply is less than total demand. 5In the absence of adequate or sufficient long-term hedging options (Sections 2.4 and 5.2), a structural investment gap persists in competitive markets because investors consequently self-hedge by systematically underinvesting.
One reason for this is that the risks of under-and overinvestment are asymmetric: the former accrue in terms of profits foregone (and this only if underinvestment does not induce scarcity pricing) that are second order compared to the latter as overinvestment always results in lower or even near-zero prices (Keppler, 2017).
the overall cost of the electricity system above the cost of a centralized system with an appropriate level of capacity.Back-up is warranted only for those installations where the risk of massive externalities is so high (e.g.hospitals, data centers) that it outweighs any concerns about the economic efficiency of electricity supply. 7As Fischer and Newell (2008) point out, due to knowledge spillovers, optimal policy cannot be a single instrument (e.g.carbon pricing) and involves a portfolio of different instruments targeted at emissions, learning and R&D. 8 This effect tends to be more pronounced for solar PV than for wind generation due to the stronger concentration during a limited number of hours and thus a larger downward impact on prices.As an aggravating factor, some support schemes do not disincentivize VRE infeed when prices are near zero or negative.Storage deployment could mitigate cannibalization by spreading out VRE output over time via arbitrage (Ekholm and Virasjoki, 2020).
likely to undershoot those targets and fall short of inducing the desired mix.Because VRE market value tends to decrease faster than generation costs as installed VRE capacity rises (Bigerna and Bollino, 2016;Green and Léautier, 2017), sole market remuneration would constrain VRE entry at lower economical levels.Crucially, this holds even in the presence of technology cost reductions and adequate carbon pricing (Hirth, 2015;Kraan et al., 2019).In fact, keeping the current market design unchanged, the systemwide gap between energy market sales and generation costs is projected to widen over time (e.g.IEA, 2018). 9 The key issue with support schemes is that they target specific assets, eschewing systemwide coordination with sufficient coherence between policies and technologies.First, they are not innocuous for those unsupported market segments because revenue inadequacy and uncertainty increase for all assets (e.g.Llobet and Padilla, 2018).VRE deployment lowers average prices and increases price dispersion, making it harder to hedge for all assets (Section 3).Second, long-lived units whose investment decisions were made prior to support introduction may become stranded.Last but not least, some schemes have proven to be inadequately designed in that they distort short-term operations, e.g. by maintaining generation incentives even when and where it is economically ineffective (i.e.price < variable cost).VRE support should thus be adjusted to capture the full VRE cost structure and implied systemwide flexibility needs (Blazquez et al., 2018;Huisman et al., 2022), address location and dispatch distortions (Newbery, 2021b) as well as reflect both the carbon value embedded in the electricity price and the extent of fuel switch from coal to gas (Abrell and Kosch, 2022).

Issue 3: Climate change externality
The issue.In principle, carbon pricing can internalize the social cost of carbon emissions and should be the main policy driver for decarbonization.In the electricity sector, a robust carbon price signal is deemed central to reducing emissions efficiently both on the supply and demand sides (e.g.Petitet et al., 2016;Bergen and Munoz, 2018).In the EU, the emissions trading system (ETS) has been instituted as the backbone of the climate-energy policy package to convey both short-run operation and long-run investment signals.Yet, the EU ETS has so far fulfilled this role only at the margin especially with regard to the long-term signal.As Tvinnereim and Mehling (2018) argue, carbon pricing has proven useful where it can incent marginal optimization (e.g.fuel-switch for electricity production) but prices have largely remained below levels that could spur innovation and investment for decarbonization.Below, we delineate two categories of reasons for why this has been the case and might continue to be so.
Ad-hoc remedies and their limits.The first category of reasons relates to unintended policy interactions and partial policy responses.Indeed, related policies such as VRE support schemes (Section 2.2) undermine the carbon price signal (in terms of level, volatility, and credibility) and associated low-carbon investment incentives.These policies eat away at the demand for emission permits independently of the permit price, eroding the stringency of the emission cap and depressing prices (Richstein et al., 2015a;Burtraw and Keyes, 2018;Borenstein et al., 2019;Chèze et al., 2020;Aune and Golombek, 2021).While low carbon prices may appear virtuous, they rather reflect insufficient policy coordination and impaired cost-effectiveness of the whole policy package (i.e.unexploited synergies, conflicting objectives).Since targeted policies drive most emission reductions, carbon prices are artificially kept at low levels and true policy costs are not transparent. 10This adds to the relatively low price explanatory power of fundamentals (e.g.Friedrich et al., 2020;Quemin and Pahle, 2022), denting confidence in the ETS being the key tool to drive low-carbon investment.In fact, the ETS acts more as a backstop ensuring the overall emission target is met in case targeted policies are not sufficient or underperform.
To tackle the structural issue of low carbon prices and embed some price resilience into its ETS, the EU introduced a supply-side mechanism in 2019, the market stability reserve (MSR).The MSR has started to absorb the historical permit overhang and contributed to pushing prices up to a new regime (e.g.Quemin and Trotignon, 2021).However, it has limited capacity to stabilize prices in the face of demand shocks, and its medium-to long-term market impacts are uncertain and hinge on market behaviorsee Perino et al. (2022) for a literature review.Specifically, its core design does not enhance synergies with other policiesit may even be counterproductive and engender a form of green paradox (e.g.Gerlagh et al., 2021) nor price stabilityit may even create volatility of its own (e.g.Richstein et al., 2015b;Quemin, 2022).As its implications lack both transparency and simplicity, e.g.compared to a price-based control (Newbery et al., 2019;Flachsland et al., 2020), the MSR thus appears to be an ad-hoc fix introduced without sufficient policy coordination. 11he second category of reasons relates to other externalities or issues that can make it necessary to have other policies in place to complement carbon pricing.First, even if an ETS is designed to account for policy interactions and smooth out demand shocks efficiently, firms' decisions upstream may still be distorted by transaction costs (Baudry et al., 2021) and behavioral factors such as risk aversion (Kraan et al., 2019), imperfect information (Lebeau et al., 2021), limited foresight (Quemin and Trotignon, 2021) or forecast errors (Aldy and Armitage, 2022). 12econd, even if efficient carbon prices are passed on to consumers, downstream pricing may not align with the social cost of carbon due to preexisting distortions (Goulder et al., 2016;Borenstein and Bushnell, 2022).Last but not least, other policy targets or externalities (e.g.learning spillovers) call for a policy portfolio approachsee Goulder and Parry (2020) for a review.In the electricity sector, other ad-hoc remedies complementing the ETS in conveying signals for investment and retirement include carbon contracts for difference to spur industry decarbonization by removing carbon price risks (Richstein and Neuhoff, 2022) and technology phaseout policies to prevent carbon lock-in through long-lived assets (Geels et al., 2017) notably in a bid to shut down coal-fired plants (Osorio et al., 2020).Again, these policies are often targeted at specific technologies or units and tend to be designed in silos, entailing a risk of policy overlap that increases overall policy costs as well as insufficient coordination that blurs the path to a decarbonized mix.

Issue 4: Missing long-term markets
The issue.In an EOM, short-term prices are supposed to guide longterm investment and shape the future mix efficiently.Spot prices are 9 Specifically, assuming high technology cost reduction, Hirth (2015) finds that the optimal VRE share that would emerge in a pure market setup remains below 25% and is non-monotonic with the carbon price (a high price spurs investment in baseload low-carbon technologies such as nuclear that reduce VRE profitability).Even if investment in such technologies is hampered by high capital costs or risks (Issue 4), the optimal VRE share could only reach a maximum of 45%.Moreover, the IEA estimates that for given investment paths, energy sales may only cover up to 50% (or 60% with 'high' carbon prices) of long-run generation costs in 2030 (IEA, 2018; Fig. 10.21).however extremely volatile, implying significant risks for investors.Investment performance in an EOM therefore crucially hinges on the extent to which investors can hedge long-term risksand symmetrically the extent to which consumers or someone on their behalf are willing to enter into long-term contracts.Risk and risk aversion are not an issue per se provided that markets are complete, i.e.Arrow-Debreu securities exist for every possible state of nature and agents can trade and transfer risk via adequate hedging instruments (Willems and Morbee, 2010;Léautier, 2016;Abada et al., 2019).Simply put, market completeness corresponds to an ideal situation where all relevant risks can be traded over all relevant time horizons (i.e.all price, volume and revenue risks could feasibly be hedged for any asset over its entire lifetime including construction time).
In reality, however, energy-related hedging markets are severely incomplete (e.g.Rodilla et al., 2015;de Maere d'Aertrycke et al., 2017;Roques and Finon, 2017), an issue which is referred to as missing long-term markets (e.g.Newbery, 2016;Wolak, 2022).That is, long-term hedging instruments do not emerge spontaneously in financial markets, which exhibit limited efficiency at pricing some types of risks and/or where counterparties are not keen to develop or enter into relevant instruments.Although both producers and consumers are risk averse and would prefer price certainty, they only sign hedging contracts over a few years at most.There is thus a large gap between available contract maturities and investment timeframes.
That the power sector is particularly subject to market incompleteness has much to do with the preceding issues, notably the semi-public good nature of electricity as a product, its specificities (multiple services and technical constraints with strong intertemporal dependencies) or its high price volatility (in part due to limited storage and demand participation).Insufficient long-term hedging is also due to consumers' perception of paternalist intervention or regulatory insurance in case of price spikes (Vázquez et al., 2002;Batlle and Pérez-Arriaga, 2008), uncertainty about long-term fundamentals or relative technology costs (Newbery et al., 2018), and regulatory risk that is unhedgeable in nature (Abada et al., 2019). 13As Newbery (2016) notes, generators and consumers are poorly equipped to deal with uncertainty about future regulatory choices when "politicians and/or regulators are not willing to offer hedges against future market interventions that could adversely affect generator profits".In other words, market counterparties to hedge risks beyond a year or two are simply missing both on the producer and consumer sides.
Because of missing long-term markets, risk-averse agents cannot fully hedge their risk and price exposure, especially for long-lived capital-intensive assets.In this context, even an idealized EOM can lead to starkly inefficient outcomes (Newbery and Stiglitz, 1984). 14If perceived risks cannot be arbitraged out or partially spread and shared with other agents, risk-averse producers will utilize risk-adjusted probabilities to gauge investment value and truncate risk profiles to reflect untradeable risk (Willems and Morbee, 2010;de Maere d'Aertrycke et al., 2017).Impaired risk-taking capabilities lead to a crowding-out of long-term private investment (e.g.Mays et al., 2022).Downside price risk weighs heavily on investment decisions, which in turn distorts the energy mix towards less capital-intensive, less risky technologies and increases both capital and average production costs (Neuhoff and de Vries, 2004;Ehrenmann and Smeers, 2011;Peluchon, 2021). 15In short, market incompleteness drives a wedge between private investors' and socially optimal discount rates, implying that in an EOM the private cost of capital remains too high to drive long-term investments in line with reliability, sustainability and affordability goals.This gap in turn justifies specific regulatory intervention to reduce private discount rates and thus financing risks (e.g.Cherbonnier et al., 2022).
Ad-hoc remedies and their limits.Regulators have taken steps to tackle risk-sharing issues by stepping in as long-term market makers. 16 That is, they introduce contract mechanisms which provide stable (e.g.renewable supports) or additional (e.g.capacity) remuneration in place or on top of energy sales over a certain time horizon.The aim is to reduce financial and regulatory risks by facilitating implicit hedging between producers and consumers, in turn reducing costs of capital and deploymentsee May and Neuhoff (2021) for a quantified analysis.
As with VRE supports (Section 2.2), the core issue with all long-term contract schemes is that they are not market-wide and lack systemwide consistency.Focusing on capacity remuneration mechanisms, we can highlight other specific issues.While CRMs have the potential to stabilize revenues and reduce investment risk relative to an EOM (e.g.Petitet et al., 2017;Abani et al., 2018), they provide limited incentives to invest in capital-intensive, low-carbon assets for at least three reasons. 17First, prices for capacity contracts, especially for annual ones, have been volatile (Spees et al., 2013;Bhagwat et al., 2017;Bublitz et al., 2019;Duggan, 2020).Second, multi-year fixed-price contracts for new assets or refurbishments only cover a fraction of the asset lifetime and remuneration only starts at commissioning (no coverage of construction risks or support during the construction phase which can be substantial for some assets).Third, they are a construct poorly suited to remunerate intermittent technologies.In other words, these schemes are currently primarily designed for energy systems dominated by thermal generation but not for a future capital-intensive low-carbon mix (Joskow, 2019;Wolak, 2022).

Energy-only markets under deep decarbonization: From imperfect investment signals to market breakdown
In this section, we specify three channels through which the deep decarbonization of electricity and energy systems dominated by high shares of capital-intensive and/or intermittent resources is bound to exacerbate the design issues identified previously.
Security of supply in deeply decarbonized systems.The ability of decentralized markets with dominant dispatchable fleets to maintain generation adequacy has been limited by security of supply externalities and hence missing money.As energy systems become increasingly VREbased, non-dispatchability, intermittency and autocorrelation 13 Other related reasons include asymmetric willingness to contract between suppliers and generators as the former face strong customer switching risk with retail competition, a hold-up problem between generators and consumers, asymmetric information about what future prices would be, and exponential increases in counterparty default risk and cost of guarantee with contract duration. 14Even if spot market revenue is adequate (no missing money), it may not be perceived to be so by generators or their financiers (Newbery, 2016).Moreover, even if short-term prices were efficient, there would still be the issue of price backpropagation to longer investment timeframes, i.e. efficient short-term prices are essentially irrelevant if they cannot be properly conveyed to and appraised by investors.Such backpropagation will fail if significant risks cloud the investment path and cannot be adequately hedged (Abada et al., 2019).option value in deferring decisions to invest in new plants (e.g.Dixit and Pindyck, 1994;Rios-Festner et al., 2020). 16A classic private lever to address risks is self-insurance, various forms of which (diversification, size increase, vertical integration between generation and retail) have been used by utilities through reconfiguration (mergers & acquisitions) or internal growing.Although vertical integration tends to outperform liberalized futures markets in restructured systems in terms of risk sharing (Chao et al., 2008;Aïd et al., 2011), self-insurance levers are limited in scope by competition rules and, more fundamentally, cannot insulate capital-intensive investments from strong price volatility, downside price risks and unexpected changes in costs or economic conjuncture. 17Petitet et al. (2017) find that investment-decommissioning decisions and prices are less sensitive to risk aversion with a CRM relative to an EOM.Hary et al. (2016) and Abani et al. (2018) find similar results, showing that CRMs can also reduce investment cycles that otherwise appear in an EOM (Arango and Larsen, 2011; Section 5.3).exacerbate these issues, which strengthens the need for flexibility such as storage and demand participation (e.g.Huisman et al., 2022). 18This calls for a profound rethink of traditional approaches to ensuring security of supply (e.g.Newbery et al., 2018;Joskow, 2019;Duenas-Martinez et al., 2021;Billimoria et al., 2022;Wolak, 2022).First, (peak) demand net of VRE infeed is more variable and exhibits steeper ramps.Moreover, correlated generation entails risks of widespread supply shortage, and the definition of standby capacity requirements for reliability can no longer assume statistical independency.The focus thus shifts from generation to system adequacy.That is, a systemwide approach covering the entire supply chain across various sectors becomes necessary to address increasingly interconnected issues, be they due to extreme weather events triggered by climate change (e.g.massive gas delivery failures in Texas in 2021's cold snap) or geopolitical turmoil (e.g. the gas supply crisis in Europe in the wake of the Ukrainian war.) Large investment needs and deep uncertainty.Whatever the deep decarbonization scenario considered, sizable low-carbon investments are necessary in the coming decades, both upstream (e.g.VRE, hydro, nuclear, storage) and downstream (e.g. demand response, electrolyzers, heat pumps). 19Importantly, since most of these technologies are capitalintensive with low or near zero variable costs possibly with long lifetimes, capital cost will be the main component of total generation costs.Ensuring that these investments are made in a timely way and at lowest cost, ideally in a coordinated and systemwide approach, poses significant challenges to market and policy design.One reason for this is that capital-intensive investment is particularly vulnerable when exposed to non-hedgeable risks, and the time to recuperate finance costs often exceeds financiers' willingness to lend without firm guarantee. 20Absent de-risking instruments over relevant timeframes, this causes finance costs, and in turn generation costs, to rise dramatically.With its shift towards high fixed cost technologies, deep decarbonization thus poses challenges to liberalized markets even when achieved by means of dispatchable low-carbon assets.
We highlight two sources of uncertainty and risk in any deep decarbonization scenario which magnify the missing long-term market issue.First, market prices and revenues become lower and more volatile, i.e. more difficult to hedge for cost recovery, precisely at a time where the need for stability and predictability is higher than ever.There will be many more hours per year of very low prices when VRE availability is high relative to load, and more hours of very high prices when their availability is low and dispatchable capacity is constrained, implying a fatter-tail asymmetric price risk (Huisman et al., 2022). 21As a result of this bimodal price distribution, inframarginal rents required for full-cost recovery need to materialize during a small number of hours with very high and volatile prices, increasing both financing risk and capital cost.This holds for all technologies, and particularly for capital-intensive ones (e.g.Tietjen et al., 2016;Cramton, 2017;Peluchon, 2021;Joskow, 2022;Mays and Jenkins, 2022). 22Crucially, flexibility provision in the form of demand response, long-duration energy storage or dispatchable low-carbon generation can only partly reduce this effect (Junge et al., 2022). 23 Second and at a more fundamental level, the future energy mix, market conditions and price distributions remain deeply uncertain today.For a given end-point target, there is a multiplicity of transition pathways with different combinations of energy carriers, generation technologies, and levels of demand (energy efficiency gains, electrification, behavioral change) or flexibility (electrical vehicles, storage).The future cost or social acceptability of low-carbon technologies are also deeply uncertain.The mix and key policy-economic factors in 20-40 years' time remain elusive, making it impossible to assign objective probabilities to given scenarios and associated price distributions, or to enumerate all of them (e.g.Abada et al., 2019;Joskow, 2022).
The need for higher systemwide coordination and policy coherency.The current regulatory framework for electricity markets in the EU is not conducive to efficient levels and coordination of investments due to a systemwide lack of coherence.Adequacy, technology, innovation and decarbonization policies address separate issues (Section 2) but are largely designed in policy-making silos.The uncoordinated implementation of ad-hoc remedies both within and across countries creates issues of its own, resulting in a multilayered policy patchwork that is difficult to navigate and prone to complex or undesirable interactions.This is notably exacerbated by the integration of VRE sources (de Vries and Verzijlbergh, 2018) and casts doubt on the ability of the current design regime to achieve political targets in the EU at least cost and on schedule.There is a growing divide between political guidelines for investment drivers in principle (i.e.competitive short-term prices) and actual investment drivers.In fact, only a vanishingly small fraction of new capacity additions in recent years have proven to be fully merchant (i.e. based on expected market revenues alone) while the bulk of investments have largely materialized through specific support policies and contracts (e.g.CompassLexecon, 2021;IEA, 2021).
The scale and urgency of decarbonization in terms of new investment and decommissioning of existing capacity imply that policymakers will be ever more likely to intervene to ensure some control over the energy mix.The aforementioned concerns about price volatility and capital costsmagnified by policy interaction and complexitywill intensify absent a design overhaul that clarifies the roles of markets and regulation in driving the energy transformation.Crucially, this overhaul will also need to enhance systemwide coordination through sector coupling, both horizontally across energy sources or carriers and vertically via end-usage electrification, with large economies of scale for some infrastructures.We turn to this issue in the next section.

The need for a design overhaul in the form of hybrid markets
In this section, we take stock of both the tension between the two purported coordination roles of wholesale marketsshort-term operational efficiency and long-term dynamic efficiencyand the diagnosis in Sections 2 and 3. To overcome the identified issues, we outline a design overhaul in the form of hybrid markets implementing a coherent systemwide framework.
18 Such flexibility increases VRE supply-security value.Note also that as the reliance on electricity increases with electrification, interdependencies and thus the consequences of security-of-supply externalities will intensify. 19To illustrate, the IEA (2021) estimates that a tripling of investments worldwide is needed in the coming decade to transition to decarbonized power systems: over 2021-2030 (2031-2040) average annual investments amount to roughly 1.2 and 0.65 (1.3 and 1.2) trillion 2019 USD in generation and network + storage respectively. 20Capital-intensive investments are also often irreversible, hence prone to sunk costs and opportunism. 21Technically, the load-duration curve rotates to the southwest around its intersect as the VRE share rises because of a small reduction in maximum residual load, reduced full-load hours for baseload plants, VRE overproduction and negative prices, as well as increasing load gradients. 22As the VRE share rises, the whole fleet will be impacted by higher revenue volatility, not just peak units as at present.As previously mentioned, VREdominated systems will also be more susceptible to extreme climate and weather fluctuations, further adding to increased price volatility. 23Relatedly, Sepulveda et al. (2018) show that firm dispatchable low-carbon generation (BECCS, nuclear, hydro) lowers decarbonized electricity system costs.Flexible assets (batteries, demand response) do not obviate the need for and value of firm resources, hence the importance of having a broad portfolio and policy support.
Central to the concept of hybrid markets is the role of the visible hand of public intervention as a coordination tool at the core of an integrated investment framework (Finon and Roques, 2013).Hybrid markets rest on a dual approach to market design whereby long-term investments are dynamically coordinated in a specific module that is separated frombut complements and works alongside witha short-term module for dispatch and balancing operations.While the long-term module has a regulatory dimension, competitive forces are an integral part of both modules.A hybrid market does not constitute an abandonment of competition per se, but rather a departure from the current exclusive focus on competition based on short-run marginal costs.The core characteristics of the two modules can be delineated as follows.
Long-term investment planning and procurement module (competition for the market).The first fundamental objective is to hive off and de-risk investment decisions from volatile and remuneration-wise insufficient short-term price signals.The second is to organize and steer the evolution of the mix towards political targets in a structured, coordinated investment framework that helps spur innovation in not yet marketready technologies.The module addresses security of supply externalities and deep decarbonization commitments jointly.It is typically broken down into three stages: the definition of the planning process and identification of system needs, the definition of long-term contractual arrangements (LTCA), and the organization of LTCA competitive procurement and the management of the interface with the short-term module.
Short-term dispatch module (competition in the market).The module rests on short-term markets based on marginal cost competition to carry out dispatch and balancing operations cost-effectively as at present.There are margins for improvement, especially to accommodate the expansion of decentralized and intermittent assets in a more flexible and efficient manner.
In practice, multiple variations of the above market architecture can be conceived.While energy economists increasingly converge on the need to reform the current market design paradigm, notably in some form of hybrid regime (e.g.Newbery et al., 2018;Joskow, 2022;Wolak, 2022), views diverge on specific implementation and design issues (e.g.centralized vs. decentralized approaches).It must also be noted that practical design variations of hybrid regimes will always be context dependent, i.e. a function of the intensity of jurisdiction-specific characteristics and issues.We leave a detailed typology of hybrid markets for future work, 24 and here only briefly sketch out feasible options and associated tradeoffs in the design of the new long-term module and its articulation with the improved short-term module.
Planning and coordination.The first step is to introduce a systemwide planning process to define energy and investment needs in a coordinated way in line with decarbonization and security of supply targets.Its scope (e.g.share of needs covered, low-regret approach), nature (informative vs. binding) and timing (early vs. late intervention) are customizable. 25 Competitive procurement.Once system needs are determined, LTCA must be defined and procured competitively.Procurement can be decentralized, centralized or combine both approaches.Its format and scope involve key design choices (e.g.technology neutral or specific auctions, differential or identical treatment for new and existing assets). 26 LTCA design.Contract design should ensure adequate long-term risk sharing to reduce capital and investment costs by trading off long-run uncertainty with the visibility that investors need as well as a seamless, undistorted interface with short-term markets notably by conveying economically effective operation incentives.LTCAs can have different formats, parameters and standards, 27 and need to indemnify against regulatory risk to generate genuinely new space for Pareto improvements.Such codification of uncertainty will ensure that LTCAs are more than simple risk transfer mechanisms between producers and consumers.
Upstream-downstream articulation.Financial balance of the long-term module between its upstream (investors, generators) and downstream (suppliers, final consumers) ends must be carefully orchestrated to ensure a smooth functioning of the hybrid architecture, recover LTCA costs, and allocate risks in a socially efficient and acceptable manner.Retail pricing must reflect the long-term upstream generation cost efficiently and transparently to ensure sufficient long-term visibility to guide investment downstream while maintaining some exposure to short-term system marginal cost to incentivize demand response. 28 Finally, it is worth noting that transitioning to a hybrid design regime is an evolution rather than a revolution, notably as hybrid market modules continue to rely on market forces (competition for and in the market).In a partial, haphazard, uncoordinated, and frequently unacknowledged manner, these modules serve already today in Europeand in a more explicit and integrated manner, hybrid designs are already in operation in Latin America (Roques and Finon, 2017). 29The challenge today is to combine and integrate these modules in a coherent fashion to ensure a low-carbon transition at least cost and high security of supply.

Hybrid markets as a contemporary form of long-term marginal cost pricing
Hybrid markets as described in the previous section are more than just a collection of measures to respond pragmatically to the twin challenge of deep decarbonization and security of supply.They constitute in fact the contemporary form of long-run marginal cost pricing in electricity markets that is appropriate not just for Europe but also for all jurisdictions facing comparable challenges.We illustrate this by means of a brief historical and international perspective on the evolution of market design over time and across jurisdictions. 24A precise description and typology of hybrid market designs is beyond the scope and contributions of this paper.More importantly, it requires dedicated work, and two specific companion papers are in preparation.The first is empirical and focusses on experiences with and proposals for hybrid market designs in Europe and elsewhere.The second is conceptual and categorizes different forms of LCTA in function of the characteristics of the underlying risks they are supposed to hedge for different technologies, markets and operators. 25For instance, Corneli (2020) argues that planning should be centralized to address significant informational and computational issues due to multiple complementarities and transaction costs in deeply decarbonized systems.It should also coordinate generation and transmission investments jointly (e.g.Chao and Wilson, 2020). 26For more detail on how to organize competitive procurement (i.e.competition for the market) and the associated design tradeoffs, see inter alia Laffont and Tirole (1993), Fabra and Montero (2022) and Iossa et al. (2022).See also Reus et al. (2018) for a centralized clearing proposal jointly minimizing expected energy cost and risk exposure. 27Although contracts for difference (CfD) have good hedging properties and maintain participation in wholesale markets in general, their type and parameters can depend on technology, e.g.reliability options for controllable units (Newbery et al., 2018), CfDs based on installed capacity (Newbery, 2021b) or on hourly generation profiles (Marambio and Rudnick, 2017) for VRE, capacity payments remunerating storage power (Joskow, 2022) or a hybrid regulated asset base model for nuclear assets with long life and construction times (Newbery, 2021a). 28The increasing importance of capital costs in low-carbon electricity systems could for instance be reflected in retail tariffs through an increasing share of fixed subscription per kW. 29In Brazil and Chile, an investment planning module is used to grant longterm contracts competitively, together with a competitive short-term dispatch module (e.g.Muñoz et al., 2021;Tolmasquim et al., 2021).

Marginal cost pricing in electricity markets
Electricity provision has always posed a distinct challenge to the normative ideal of achieving socially optimal equilibria through decentralized individual decisions coordinated through the gradual process of converging short-run prices.At the heart of the matter is, in the terminology of the seminal contribution by Boiteux (1960), the difficulty to have short-run and long-run optimal prices coincide, i.e. to combine efficient short-run marginal cost pricing with full cost recovery in an industry producing a non-storable good. 30In the electricity sector, two solutions can be delineated.The first delegates generation and investment decisions to a regulated entity that sets regulated tariffs (including in peak demand periods) at the level of long-run marginal cost (the cost of an additional capacity unit plus variable costs).Regulated systems can ensure sufficient funds to finance an adequate level of capacity to cover demand with a high level of certainty but provide limited incentives for generators to improve efficiency and innovate.The second lets individual generators guided by the principle of profit maximization in liberalized markets choose a level of capacity inducing enough scarcity hours with high prices to recuperate fixed costs.Symmetrically to regulated systems, liberalized markets harness competitive forces to improve operational efficiency but create uncertainty with respect to capacity investment on top of an incompressible number of possibly socially unpalatable high-price scarcity hours.
The conceptual principle of long-run marginal cost pricing was set out in its canonical form by Boiteux (1960).Its reference case assumes that total capacity is adjustable and that prices (or tariffs) are set by a regulator (or welfare-maximizing monopolist).Short-run prices (or optimal tariffs) are simply fixed so that they correspond with the long-run marginal cost of expanding capacity by one additional unit: "Under the theory of selling at marginal costs, prices must be equal to the differential costs for existing plant.Plant is of optimum capacity when the differential cost and the development cost are equal, that is to say when differential cost pricing covers not only working expenses but also plant assessed at its development cost."(p.167) [In turn], "provided there is an optimal investment policy, short-term pricing is also long-term pricing and there is no longer any contradiction between the two."(p.165, our emphasis) Moreover, with demand varying through day, week or year, and assuming capacity is adjustable in the long run, Boiteux also arrives at the principle of differentiated pricing during peak and off-peak hours (respectively, level and off-level hours in his terminology): "During the off-level hours, the rate charged will cover energy costs only.The level hours bear rates which will also cover daily power charges assessed at development cost when the level [of capacity] is adjusted to demand."(p.176) Notably, the optimality of peak-load pricing holds true irrespective of generation technologies.Boiteux's analysis was then taken up and furthered by various economistssee inter alia Crew et al. (1995) for a thorough literature review and Green (2006) for a useful primereventually culminating in the paradigm of competitive spot electricity pricing.In principle, inframarginal rents accruing to generators in addition to high prices in extreme peak demand hours should provide sufficient revenues to finance capacity and recoup full costs.Due to enforced scarcity during those hours, prices would equal the value of lost load (VoLL). 31 On paper, it all adds up.VoLL-pricing exhibits a strong structural identity with Boiteux's peak-load pricing, i.e. revenues for adequate capacity are generated during peak demand hours.While VoLL-pricing rests on the profit-maximizing behavior of competitive generators in deregulated markets and Boiteux considered a benevolent welfaremaximizing monopolist, both approaches have prices equate short-run marginal costs (SRMC) outside peak hours and long-run marginal cost (LRMC) during peak hours, during which full costs are recuperated and budget constraints are satisfied for individual firms and the system overall (no missing money).
Finally, when comparing regulated systems, deregulated markets and later hybrid markets, it must be noted that these approaches cannot be distinguished by the mechanics of equating costs with revenuestheir theoretical justifications all postulate full cost recovery without excess profitbut with respect to the dynamic incentives for generators, the conditions ensuring cost recovery, as well as the validity of the underlying assumptions.A decisive difference therefore relates to the level, number of hours, volatility, predictability and ultimately acceptability of high-price hours. 32Specifically, it is key whether full costs are recuperated during all operating hours (hybrid market with LTCAs), a subset of suitably defined peak-demand hours (Boiteux's welfare-maximizing monopolist) or an even smaller subset of scarcity hours (liberalized market relying on SRMC and VoLL-pricing).

From regulation to liberalization and towards hybrid designs for lowcarbon electricity markets
In the 1980-90s, technological and institutional changes held out the hope that SRMC-based competition rather than tariffs set by a regulated welfare-maximizing monopolist could achieve optimal capacity levels.On the technological front, new possibilities arose with the advent of combined cycle gas turbines with relatively low fixed costs and low-cost computing power allowing for rapid resolution of market-clearing algorithms.On the institutional front in Europe, North and Latin America and Australia, political and economic preferences prevailing at the time were oriented towards introducing competition and privatization to reduce government intervention and improve operational efficiency.In the electricity sector, the intent was to lower prices through more efficient operations of power plants, fossil fuel use and wholesale markets, as well as end corporate slack and induce a new technological dynamism (e.g.Batlle et al., 2010;Pollitt, 2012;Borenstein and Bushnell, 2015;OECD, 2019). 33 It is within this context that electricity economics converged around the paradigm of SRMC pricing (Joskow and Schmalensee, 1983;Schweppe et al., 1988;Stoft, 2002).Joskow (2008b) summarizes the main features of this "textbook architecture" as: privatization of state-owned monopolies; vertical separation of potentially competitive segments; horizontal restructuring to make generation more competitive; integration of transmission grids and network operations; competitive spot markets for energy and operating reserves; institutions to integrate demand responses; competitive allocation of transmission capacity; unbundling of tariffs to allow for competitive retail services or 30 A rich literature going back to discussions on the setting of optimal rates for various non-storable services since the 20th century has provided ample commentary on this issue.See also Keppler (2017) for a historical perspective. 31VoLL reflects the unit cost of involuntary or unplanned demand reduction.
In the long run, entry & exit ensures that the number of VoLL-hours multiplied by VoLL covers the gap between the revenues from wholesale markets during off-peak hours and the cost of an additional unit of investment. 32In fact, social and political acceptability of high prices goes beyond simply scarcity hours, as attest uncoordinated government interventions in the wake of the ongoing energy crisis in the EU. 33In the EU, the move towards integrated/coupled national markets was also a strong policy driver for change (e.g.Pollitt, 2019).In the US, Borenstein and Bushnell (2015) argue that the greatest political motivation for restructuring was rent shifting, and not efficiency improvements per se.distribution monopolies obliged to source through competitive markets; and competent regulatory agencies.
This new paradigm quickly caught the imagination of regulators and policymakers worldwide, driven in part by first movers like the UK in Europe or Chile in Latin America.They unbundled vertically integrated monopolies, separated electricity generationwhich was deemed fit for market allocationfrom transmission and the supply of ancillary services, created competitive wholesale markets and instituted retail competition.While certain drawbacks became visible almost immediately (see below), liberalized markets delivered on the main promise of enhanced efficiency and proved to be very good at "sweating assets" (Newbery), i.e. utilizing existing assets cost-efficiently through meritorder dispatch.Generally speaking, they did rather well yielding efficiency gains and better access to services (e.g.Batlle et al., 2010;Pollitt, 2012;Borenstein and Bushnell, 2015). 34At least initially, generation costs declined due to the combined effect of enhanced efficiency (in systems often subject to overcapacity initially), lower cost of new generation technologies and a decrease in interest rates.The net gains, however, were small in the order of 5% of total costs and not readily visible to customers and some were even left worse off due to redistributive effects (Pollitt, 2012(Pollitt, , 2019;;Cicala, 2022). 35 Efficiency gains notwithstanding, various issues regarding generation adequacy soon appeared.A large body of literature quickly started documenting various market failures and externalities that could deter investment, notably in peaking units, and impair generation adequacy already in the late 1990s and early 2000s (e.g.Hirst and Hadley, 1999;Pérez-Arriaga, 2001;Vázquez et al., 2002;Woo et al., 2003).This literature expanded and eventually covered most if not all jurisdictions which liberalized their electricity sectors, whether it be in the US (e.g.Chao et al., 2008;Joskow, 2008a;Borenstein and Bushnell, 2015), the EU (e.g. de Vries, 2007;Finon and Pignon, 2008;Roques, 2008;Cepeda and Finon, 2011) or Latin America (e.g.Batlle et al., 2010;Rodilla and Batlle, 2012).In other words, SRMC pricing alone fell short of sustaining investment in line with an adequate reserve margin, exposing the need for complements through various forms of capacity remuneration mechanisms (Section 2) or alternative price adders to improve scarcity signals (e.g.Hogan, 2013).
Failure of deregulated markets to ensure generation adequacy is attributable to various reasons already covered in Section 2, including insufficient dynamic coordination between investors 36 and uncertainty about the number of VoLL hours. 37More generally, a SRMC-based market design is intrinsically sensitive to external shocks related to insufficient spontaneous long-term contracting on the demand side (Vázquez et al., 2002;Batlle and Pérez-Arriaga, 2008;Chao et al., 2008).In turn, ad-hoc intervention is inevitable when prices are significantly above or below LRMC, creating situations of dire over-or under-coverage of fixed costs, possibly with large politically unpalatable transfers.This severely impedes the sustainability of such a design, as illustrated by the ongoing energy crisis and flurry of uncoordinated national measures in the EU to contain price impacts on affordability and competitiveness.
Some still point to the full rollout of SRMC pricing as the way forward to correct all existing issues and limitations (e.g.Littlechild and Kiesling, 2021;Hogan, 2022).Yet, recent experience in jurisdictions with a design in place closest to this textbook model suggests that they too are far from immune to the problems identified.In Texas for instance, administered scarcity price adders do improve on short-term market and investment efficiency, but fall short of ensuring adequate long-term reliability investment (Zarnikau et al., 2020;Bajo-Buenestado, 2021).In Australia, predicted shortfalls in generation due to the national market's limited performance in ensuring capacity investment led to the introduction of a retailer reliability obligation, which is triggered if the market operator forecasts a generation gap (Simshauser, 2019(Simshauser, , 2021)).Finally, in Alberta, some missing money and price manipulation problems still exist today (Brown and Olmstead, 2017) and a capacity market was even considered for a time (Brown, 2018).
A workable solution is neither found in the conceptually still dominant paradigm of short-run marginal cost pricing (e.g.Blazquez et al., 2020) nor in the regulated systems of old.Today's context both demands and enables a third way in the form of hybrid market designs to ensure that the normative ideal of LRMC pricing can achieve deep decarbonization targets, address system adequacy concerns and foster technological dynamism.Hybrid designs seek to leverage the best of both worlds by combining deregulated and regulated features to exploit their relative strengths and weaknesses in a coordinated manner (see Section 4).
Today's context casts an old debate in a new light, i.e. how best can policymakers and regulators organize energy systems to balance the often-conflicting objectives of efficiency, equity and innovation (Newbery, 2018).This renewed debate unfolds in the EU (e.g.Roques, 2021), the US (e.g.Duenas-Martinez et al., 2021;Gruenspecht et al., 2022) and Latin America (e.g.Muñoz et al., 2021;Tolmasquim et al., 2021) alike. 38While the related literature is still nascent, we highlight a few specific designs.In the US, proposals can notably be found in Woo et al. (2019), Woo and Zarnikau (2019) and the contributions to a joint RFF-WRI workshop (2020).In the EU, policy discussions have recently intensified on the back of mounting affordability issues and the need to change how consumers pay for energy to better reflect the present and future energy mix (e.g.Batlle et al., 2022;CERRE, 2022).In this context, competitively-allocated long-term contracts can allow customers to lock in part of their energy costs at fixed prices while reducing investment risks in low-carbon generation.

Conclusion and policy implications
This paper discusses the challenges of electricity provision through decentralized energy-only markets (EOM) and indicates the limitations of the current EU market design regimean EOM based on short-run marginal cost pricing flanked by various uncoordinated ad-hoc policiesin ensuring deep decarbonization and security of supply at least economic cost and on schedule.While benefits of competitive dispatch are widely recognized, the current system has failed in bringing about adequate amounts of capacity investment.The paper spells out the 34 Pollitt (2012) provides a literature review of international experiences with liberalization reforms while Batlle et al. (2010) focus on Latin America and Borenstein and Bushnell (2015) and Bushnell et al. (2017) on the US. 35 Borenstein and Bushnell (2015) point the unsustainability of induced rent transfers and argue that electricity rates were mostly reduced by exogenous factors such as advances in generation technologies and changes in gas prices rather than by restructuring itself (the former affect the counterfactual against which the latter must be assessed). 36The first years following restructuring in the US are a case in point.Hill (2021) demonstrates excess entry of gas-fired generation during this period, paving the way for an investment cycle (capacity overinvestment entailed underutilization and shrinking margins that led to a subsequent sharp drop in investment).Excess entry is attributed to a coordination failure among firms with incomplete information, to a contagion effect and to overconfidence in one's ability to be the lowest-cost producer prompted by a decline in gas technology cost and prices. 37VoLL-hours resulting from limited generation capacity relative to peak demand was originally supposed to be limited to a few high-price hours per year thanks to the advent of CCGT as peaking units.With high variable costs relative to fixed costs, the shortfall to be financed during peak hours seemed manageable.Things turned out rather differently, reflecting a tension between the fact that VoLL-hours seldom arise, creating risks not readily assumed by investors, yet occur regularly enough for policymakers to be unwilling to accept the situation.