Stakeholders’ Role in Regulatory Outcome Evaluation: A Closer Look at Brazil’s RIA Guidelines

By Denis Guimaraes (AGPR)¹

Introduction: RIA and ROE – ex ante and ex
post Tools

As noted in
our previous article, in 2018 the Brazilian government published its first
Regulatory Impact Analysis (“RIA”) Guidelines (the “Guidelines”) for
rulemakers.²

In addition
to RIA, whose main topics of interest have already been explored, it should be
noted that the Guidelines also deal with another topic relevant to the
regulatory and antitrust community, namely regulatory outcome evaluation
(“ROE”).          

ROE is “the
systematic evaluation process of an intervention to determine whether its
objectives have been achieved.” Thus, unlike RIA, which is a form of ex ante
policy analysis, ROE is ex post. A key consequence of this difference is
that ROE should not be confused with RIA inspection or monitoring processes.³

Our
preceding article included a list of topics to be addressed in an RIA report.
After comparing the available alternatives and justifying the measure chosen to
address a regulatory problem, the RIA report must present a strategy for
implementing that measure, including how any inspection and monitoring should
work.

Basically,
the inspection strategy consists of defining how the practices of those subject
to a regulation will be followed, so that the regulator can check whether their
obligations are being met. The monitoring strategy, in turn, consists of
defining indicators (quantitative or qualitative) that can be efficiently used
(in terms of time and cost) by the authorities to keep track of the impact of
the chosen regulatory measure after its implementation.⁴

Therefore,
the ex ante RIA report sets out inspection and monitoring strategies as
a foundation for ROE, an ex post evaluation of the actual “performance
of the adopted or amended regulation, considering the achievement of desired
objectives and outcomes, as well as other impacts observed on the market and
society, resulting from its implementation.”⁵

Cases and
Timing

After
distinguishing RIA from ROE, we will now discuss, with respect to ROE: (i) the
main kind of cases in which it should be conducted; (ii) its timing; (iii) the
government’s interest in stakeholder engagement; and (iv) its “principle-based
approach,” focused on “reducing administrative burdens or promoting
competition.”⁶

The Guidelines
state that ROE should be conducted at least in two kinds of cases: (a) RIAs of
Level II, i.e. when the complexity of the case or an expected significant
regulatory impact demand the formulation of a complete RIA report; and (b) when
RIA has been dismissed by the regulatory authorities due to reasons of
urgency.  

The rules
regarding timing for conducting an ROE are: (a) when a regulation is analyzed
through a RIA report of Level II, the regulation itself should set the ROE
deadline; (b) when a regulation is enacted without RIA due to reasons of
urgency, the ROE should be conducted within two years of such enactment.

Stakeholder
Engagement

Such periods
of time (case-by-case or two years) between the enactment of a regulation and
the conclusion of an ROE may seem quite long to stakeholders that have invested
in providing input to the RIA. However, in fact, stakeholders do not simply
need to wait for the authority conducting the ROE to see the actual impact of
the regulation on their activities – and then wait again to see whether the
authority has learnt from the ROE and will use this knowledge to feed into the
next RIA to amend the regulation or create a new one. The section of the
Guidelines on ROE also makes clear the importance of stakeholder engagement,
both in the main text and in the references.

The
Guidelines refer (among several other important publications⁷) to the
UK’s Magenta Book⁸ to
summarize three main kinds of ROEs that can be conducted: (i) the process
evaluation; (ii) the impact evaluation; and (iii) the economic evaluation. In
the first two cases, the importance of stakeholder engagement is very clear.
The Magenta Book states that “[p]rocess evaluations will often include the
collection of qualitative and quantitative data from different stakeholders,
using, for example, group interviews, one to one interviews and surveys.”⁹ Regarding
impact evaluation, the Book advocates that the ROE look for changes “across
different individuals, stakeholders, sections of society and so on, and [to]
compare [them] with what was anticipated”¹⁰
in the RIA.

Thus, it
should be noted that stakeholder engagement should not be seen as a static
action taking place during the RIA and/or ROE, but should preferably be a
continuous activity of market observation and interaction with the regulatory
authorities.¹¹

Institutional
Changes, ROE/RIA, and Competition Advocacy

Indeed, the
Brazilian Office for Public Policies (“SAG”) had been pointing out the
importance of stakeholder engagement not only in different Guidelines they had
been coordinating up to the end of the last federal administration in December
2018, but also in events that have included the participation of the private
sector.

The new
federal administration that took office in January 2019 has promoted some
institutional changes, but the work of promoting best practices among Brazilian
regulatory entities carried out under the last administration should continue.
This will occur either through the SAG, and/or through the reformed Ministry of
the Economy (resulting from the merger between the four previous Ministries of
Finance, Planning, Industry and International Trade, and Labor), where former
SAG members have been allocated to the Executive-Secretariat (right below the
Minister of the Economy), and have already started to develop further work on a
regulatory improvement agenda.

It can be
anticipated that one of the focuses of that agenda will be working jointly with
the reshaped secretariats for Competition Advocacy and Competitiveness (“SEAE”)
and Evaluation, Planning, Energy, and Lottery (“SECAP”). These secretariats
must (among other duties) (i) promote competition in regulated sectors; and
(ii) analyze the regulatory impact of public policies. While (ii) should
include both RIA and ROE, it must be recalled that they include the analysis of
possible antitrust issues that may be part of the regulatory problem.¹² In other
words, the Regulatory Improvement Agenda acknowledges the already quoted OECD
principle-based approach that promoting competition should be, in general and
when the market features permit, a more efficient way of (not) regulating the
markets, leaving specific regulatory measures (and imposing administrative
burdens) to the cases in which they are indispensable, and according to RIA and
ROE, where applicable.  

Conclusion               

In conclusion, these are the main ROE features we wanted to address. We expect that the private sector will engage in further developing this know-how jointly with the Brazilian authorities.

Click for the full PDF.

1 AGPR, Sao Paulo: www.agpr.consulting.

2 RIA guidelines and RIA guidebook for Regulatory
Impact Analysis / Office for Public Policies – OPP [et al.]. Brasília:
Presidency of the Republic of Brazil, 2018 (“OPP 2018”). The Guidelines have
since been officially published in English.

3 OPP 2018, p. 83.

4 OPP 2018, p. 88.

5 OPP 2018, p. 11.

6 Regulatory Policy Outlook 2015 / OECD. Paris:
OECD Publishing, 2015, p. 120.

7 “What We Heard Report on Regulatory Reviews and modernization stakeholder consultations / Treasury Board of Canada Secretariat”, April 2, 2019; “OECD Best Practice Principles on Stakeholder Engagement in Regulatory Policy (Draft for Public Consultations)),” OECD Directorate for Public Governance, 2017; “Stakeholder engagement examples by stage in the policy cycle / OECD Pilot Database on Stakeholder Engagement in Regulatory Policy,” OECD Directorate for Public Governance, available at http://www.oecd.org/gov/regulatory-policy/pilot-database-on-stakeholder-engagement-practices.htm.  Guidelines on Stakeholder Consultation / European Commission Better Regulation Guidelines Chapter VII, 2015.

8 The Magenta Book: Guidance for Evaluation. HM
Treasury, London (2011).

9 Id.,
at p. 18.

10 Id. at p.19; see also Avaliação de políticas
públicas: guia prático de análise ex post, volume 2 / Casa Civil da Presidência
da República [et al.]. Brasília: Casa Civil da Presidência da República, 2018,
p. 14.

11 Avaliação de políticas públicas: guia prático de
análise ex ante, volume 1 / Casa Civil da Presidência da República, Instituto
de Pesquisa Econômica Aplicada. Brasília: Ipea, 2018, p. 160.

12 Institucionalização e Prática da Análise de
Impacto Regulatório no Brasil / IBRAC – Comitê de Regulação do Instituto
Brasileiro de Estudos de Concorrência, Consumo e Comércio Internacional. São Paulo: versão preliminar, 24 de junho de
2019, pp. 224-236.