Tutorial on Fiscal Transparency Portals: An User-Centered Development

Citation preview

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

In line with the core tenets of fiscal transparency, information pertaining to how governments collect taxes; acquire debt; and allocate, spend, invest and manage public resources should be made available to the public in a clear, reliable, frequent, timely, relevant, comprehensive and accessible manner. Information technology advancements mean that this is now more achievable than ever before. This is also facilitated by legislation in a large number of countries now guaranteeing the right to access information; the existence of internal financial management information systems, whether comprehensive or not; and transparency and open data technological innovations. All of these factors have unlocked an enormous potential for growth.

Tutorial on Fiscal Transparency Portals November 2019

Written and developed by: - Tarick Gracida, Technology and Communications Coordinator. [email protected] | @TarickGracida - Lorena Rivero del Paso, Manager for Cooperation and Technical Collaboration. [email protected] | @lorenardp With gratitude to Raquel Ferreira for their useful comments and suggestions. Translation to English: Sofía Leal de la Rosa Style correction: Erica L. Wood

Acknowledgements Throughout the years, the Network has witnessed the multiple efforts, many extra work hours, frustrations and celebrations behind each development of a fiscal transparency portal. We’d like to express our recognition of and, above all, our gratitude to those we call champions of transparency, who from their desks own the battle and serve as a source of inspiration to others in the search of ways to build better countries for the people. You know, in each country of the Network, who we are talking about. Thank you!

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

Fiscal transparency portals have emerged in recent years as a digital solution to release information to the public in a proactive, simple and consolidated manner, in formats that allow for faster and easier analyses. These portals decrease the asymmetry of information between government and society, thereby facilitating a genuinely informed dialogue that has the potential to improve public finances and consequently enhance service delivery and development. The publishers of such portals however frequently face challenges with their configuration and implementation; and subsequently with their dissemination, acceptance and use by wide audiences. This often emanates from a mismatch of expectations from those that supply, and those that demand fiscal information. Currently, numerous government and civil society institutions worldwide are developing or planning to develop some form of fiscal transparency portal, or launching new platforms for already existing ones. Alternatively, in many other institutions, the frequent analyses of results is leading to redesigns in the manner in which fiscal information is published. Regardless of the form of the portal, this tutorial has been developed to assist in bridging the gap between the supply and demand for fiscal information, by proposing a route to publish fiscal information that reflects user needs and options. To do this, the tutorial presents a series of techniques to identify and segment various user audiences, thereby enabling user experience-driven development; an array of considerations for creating a plan that examines the publishing team’s capabilities (e.g. human, technological and informational); and a number of tools to facilitate an iterative design process¹ based on monitoring and evaluation.

Global Initiative for Fiscal Transparency www.fiscaltransparency.net | [email protected] / @fiscaltrans Network Director: Juan Pablo Guerrero | [email protected] | @jpga63 1. Iterative design is a software development methodology based on small, repetitive stages (iterations) as part of a cyclical process of investigation, conceptualizing, prototyping, user testing and refining.

I. Objective

has also benefited from the experiences of Chile, Colombia, South Korea, Guatemala, Ireland and the United States.

The main objective of this tutorial is to provide a basic guide to publishing fiscal information using a user-centered approach. To do this it describes the concepts and actions to be considered throughout the process.

This tutorial systematizes the process of developing or updating a fiscal transparency portal or platform using a user-centered approach. It forms part of the network’s practical tools for achieving greater fiscal transparency globally.

By the end of this tutorial, participants will be able to: Understand the importance of publishing fiscal information designed for target audiences. Recognize the different stages and steps required to develop a fiscal transparency portal or platform with a user-centered approach. Identify potential users and segment the audience of a fiscal transparency portal. Discover tools that assist in leveraging current technological progress for a user-centered publication.

04

Acquire techniques to monitor the results of a publication, or of an objective-oriented iteration.

II. Target audience This tutorial is aimed at those responsible for managing or implementing fiscal transparency policies. While it focuses mainly on providing governments with tools, being that they are the primary generators and publishers of such fiscal data, its contents can also be useful to civil society organizations that seek to use digital tools to disseminate analysis and interact with particular communities for advocacy purposes.

III. Methodology Despite the absence of vast literature on fiscal transparency portals, the GIFT network, through its constant exchange of research and experience, has witnessed and supported the evolution of numerous portals. The network has participated in the creation and improvement of portals in Argentina, Brazil, El Salvador, Indonesia, Mexico, the Dominican Republic, South Africa and Uruguay. Moreover, the network

III.1. Instructional design The tutorial consists of nine modules, each containing videos and a supporting document. Its theory is based on methodologies and techniques applied in social sciences (conceptual framework); international examples; and recommendations aimed at facilitating the process of implementation. Its structure has a sequential design, following the premise that omitting certain steps could lead to a less successful implementation. If a publishing team has however assessed its position and considers only certain topics as relevant, it can consult these modules separately.

05

The MODULES are organized as follows 06

1 2 3 4 5

Introduction to fiscal transparency portals and the importance of taking the user into consideration.

This introductory module presents what fiscal transparency portals are and what their purpose is. It also defines the analytical segmentation of six basic elements of fiscal transparency portals, recognizing their progress and limitations in terms of the supply of information, interaction with the demand, publication formats, update mechanisms, design and communication channels.

Introduction to user-centered design

The essential goal is to generate portals and platforms that not only account for users in their development, but that engage them. This module presents the user-centered design philosophy, as well as its implementation steps. User-centered design, which through five iterative development levels will be the axis of the following chapters, aims to achieve a high level of satisfaction in terms of user experience, allowing different audience segments to meet their goals and needs with minimum effort as they interact with the portal.

Definition of a general purpose, objectives and indicators

This module explains the importance of having a clear purpose prior to the development or the upgrade of a portal and provides tools to generate one. Fiscal transparency can have different objectives and benefits; and as such, the objectives of fiscal transparency portals and platforms can vary according to a country’s context, priorities and public finance management capabilities. Despite this, having objectives facilitates the prioritization of a critical route, providing for the establishment of indicators to measure progress and, further on, to evaluate results and inform subsequent iteration.

Identifying and segmenting the audience

This module provides tools and techniques to conduct research on, understand and engage the demand for certain outputs of portal development from a user-centered design perspective, with the purpose of identifying and segmenting different potential users (audiences) from within and outside government. Furthermore, the “personas and scenarios” technique will be put forward as a method to segment different profiles. Finally, the benefits of applying a progressive disclosure of information based on the identification and segmentation performed will be explained.

Internal coordination for external publication 5.1 Team Coordination

Before developing or upgrading a fiscal transparency portal or platform, an assessment should be undertaken of the current resource status, including in respect of the availability of tools, team capabilities, and other complementary resource availability. In this section, frequently asked questions within the GIFT network will be used to explain the importance of coordination among the different areas that generate the data to be published, allowing for sustainable publication. This module also discusses existing internal organization options.

07

5.2 Systems and data connectivity

To enable publication, it is necessary to map where the data requiring extraction is stored, as well as to note its structure and storage format. This information is required to design a publication that releases data that meets the main objectives, while considering its disaggregation, integrity and comprehensiveness.

08

6 7 8 9

Development of the portal or platform

Structure design, information design and visual design are the three architectural levels in a portal or platform. Aside from these three levels, this module addresses concepts like user interface (UI) and user experience (UX), and introduces tools which can be used during the prototype testing and refinement stage prior to the conclusion of programming, and which, in a cyclical manner, ensure a more user-centered development. Finally, this module presents a developer’s toolkit containing tools, sources of inspiration, outputs, open code, trends, libraries, and other helpful resources.

Amplifying the audience

Fiscal information publishers, as well as development teams, constantly face a lack of engagement from users within fiscal transparency portals and platforms. To handle this issue, this module proposes--in four steps and as part of a comprehensive strategy in the development of a user-centered portal--strategies for communication and engagement, as well as tips and examples on implementation.

Monitoring and evaluation to improve

What can be measured can be improved! This is why it is important to monitor both the initially proposed indicators and the functional/operational activity of the platform. In this penultimate module, a handful of monitoring methods will be explored, including the use of Google Analytics as a tool to enable analyses of user behavior. Subsequently, methodological options are introduced to allow for the evaluation of a publication’s processes, cost-benefit and impact, providing for the gathering of evidence to inform decision making on subsequent improvements.

Iteration to accomplish a living portal

A fiscal transparency portal developed or upgraded through user-centered processes becomes a living and reactive portal that adapts in an agile and constant way according to objectives and results over time. Accordingly, such reactive portals guard against the possibility of inconsistencies due to changing local context, international advances in related matters and technological evolution, among other factors.

III.2. Limitations of the tutorial This tutorial presents standard techniques and focuses on sharing open tools. Its successful implementation will however depend on resource capabilities and local contexts. It is of paramount importance that publications are customized to the context and expectations of their respective country, community or target audiences. Regarding the technological aspects of portal creation, while the tutorial does suggest certain useful tools and applications, its main purpose is to reflect on how a fiscal transparency tool can be planned-- rather than to simply define the technologies that should be used in its making. It is essential that readers consider that disruptive techniques have begun to emerge at a faster speed than ever before. Therefore, publishers must always keep account of the user’s varying needs, and react to them by iterating with respect to change. Finally, while the tutorial offers a general overview of the different topics to consider when creating or upgrading a fiscal transparency portal, it should not be considered as being a complete guide. Publishers will however acquire information to identify the different fields of expertise necessary for the process of implementation.

09

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.1

Introduction to fiscal transparency portals and the importance of considering the user

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

For more than a decade, governments have been looking for effective ways of making fiscal information available online, seeking to satisfy the demand for transparency and responsibility in public finance management. Progress in government openness and digitalization, including the development of financial management information systems, makes this goal more accessible than ever before.

1. Portals and platforms Fiscal transparency portals can adopt various forms based on a country’s context, and can contain different information. A fiscal transparency portal can, in turn, be composed of various platforms, each aimed at satisfying different objectives and audiences. Structure example of a fiscal transparency portal with different platforms.

In this context, fiscal transparency portals emerge as a type of digital tool built on new technological possibilities, acting as consolidated entry points to fiscal information that is subsequently more clear, reliable, frequent, timely, relevant, comprehensive and accessible.

04

The development of portals is strongly aligned with general technological developments, as well as country specific internal systems’ progress. From the basic availability of non-structured online files to the real-time dissemination of open data, technology has a significant impact on the variety of methods available for publication. It is acknowledged that much has been said in recent years about the importance of civic technology (civic tech¹) for achieving open government, and that limited impact has been seen from its implementation. Accordingly, it is important to recognize that the use of technology as an aim in and of itself, is not useful in supporting public engagement. That being said, if used strategically, technology can be the most powerful tool for fiscal openness. In this sense, the development of tools that are made with and for the user--such as fiscal transparency portals--acquires special relevance in the context of civic tech.

1. Civic tech focuses on informing people, connecting them with each other and getting them to engage with their government in order to work together for the public good. (CitizenLab,https://www.citizenlab.co/blog/civic-tech/whats-difference-civic-tech-govtech/).

Fiscal Transparency Portal

Platform on budget and spending

Platform on public investments

Platform on income from state-owned enterprises

Platform on non-financial performance

Platform N

While a portal offers up its content in an aggregated manner that seeks to invite exploration, the experience is delimited and predetermined by a set of decisions about what is necessary, relevant and useful, and then presented in a consolidated way to all types of users. Platforms, by contrast, place these decisions back into the hands of users, creating innumerable ways of interacting with information and data, rather than just one way within a single interface. In this sense, there is a huge conceptual and practical difference between referring to a portal as a monolithic entity and conceiving it as a structure composed of platforms with different scopes, objectives, and users, which can ultimately facilitate its design processes, construction, monitoring and evaluation. Example structure of a portal with platforms: Mexico’s Ministry of Finance and Public Credit’s fiscal transparency portal in its 2018 version. Budget

Programs

Evaluation and performance System

Public Investment

National Development Plan

Educational infrastructure

Transfers to subnational governments

Open contracting

05

The portal is composed of thirteen platforms with different topics, which are all simultaneously linked to each other. Each one was developed based on audience analyses and user testing taking place during its design, implementation and monitoring processes; which included focus groups, interviews, field studies, prototype use observations, surveys and use statistics. As an example, the following image shows how in the “budget program sheet” users can be linked with information from the different platforms, including: approved and spent budgets, investment projects, performance indicators, external evaluations and procurement made under the program--all this, with specific open data downloads.

2. Basic elements of fiscal transparency portals Broadly speaking, fiscal transparency portals are defined by the six basic elements that constitute them: 1. Supply of information. This refers to the information contained in a portal, including its platforms. Additionally, it considers the completeness and granularity or disaggregation of the information presented. In order for a portal to be considered user-centered, the supply of information must be responsive to the demand for it. 2. Interaction with the demand. Considering users’ interests and needs has become increasingly important to publishing teams aiming to generate products that are useful and widely utilized by the audience to which they are addressed. Government’s interaction with users (the demand), when it exists, can vary from being with selected groups related to fiscal issues, or being with potential users that are not specialists on the subject. 3. Publication formats. The digitalization of fiscal transparency--that is, the online publication of information--allows users to access and download files related to different phases of the budget cycle. These files, available online and/or in downloadable form, can vary in degree of openness from those being in non-structured and proprietary formats to those that have open and interoperable² data. Additionally, having online publishing portals allows for the possibility of offering interactive visualizations that make interactions more dynamic and attractive for users.

06

- Performance information - Evaluations - Matrix of indicators

Program objective

Indicators

4. Mechanisms and update frequency. Having timely information is one of the characteristics of significant fiscal transparency. The possibility of creating a timely publication depends largely on the degree to which site updates--based on internally generated and stored data--are automated. 5. Design. The visual development of the interface with which users interact, beyond aesthetic considerations, is key for achieving a positive experience between a portal and its users, taking into account their goals, capabilities and preferences. 6. Communication channels. There are several possible channels for communication between the publishers of a fiscal transparency portal and its various audiences, and many different objectives for this communication, such as informing, consulting and engaging, among others. The user’s role is determined through the selection of these different channels and objectives, resulting in either unidirectional communication channels or two-way interaction mechanisms³.

2. To learn more about the degrees of openness, consult the “Open Data Tutorial” by Global Initiative for Fiscal Transparency and Mexico’s Ministry of Finances and Public Credit (http://bit.ly/OpenDataTutorial), more specifically the module “What is Open Data?”. 3. With two-way communication, the publisher enables channels to transmit published information that simultaneously also serve as feedback mechanisms. This feedback can relate to different aspects of the publication or even to core financial management.

07

3. Progress in fiscal transparency portals according to their basic elements

08

Variations and iterations in each of the six basic elements of fiscal transparency portals allow us to segment progress into three generations of iteration. Readers may recognize elements from different generations in the same portal according to the concept description. It is worth mentioning that this segmentation is only for the purpose of analytically grouping some of the basic characteristics of these portals within this tutorial, thus enabling implementers to identify development opportunities and good practices, and achieve a sustainable, user-centered publication.

1. Supply of information In most cases, initial fiscal transparency portals focus their efforts in publishing information that is already systematically available as part of government reports or similar official sources elsewhere, such as executive budget proposals, approved budgets, year-end reports, in-year reports or reports on government priority programs--all with a very limited degree of disaggregation, which makes in-depth analysis difficult. The main contributions provided by fiscal transparency portals relate to their illustrative additions, such as the provision of graphics or plain language explanations for the contents of a budget document. For example, the 2012 version of El Salvador’s portal contained, on-line for the first time, public finance reports generated for the congress/parliament, as well as the citizen version of the budget published as a non-interactive booklet.

09

Fiscal transparency portal from El Salvador in its 2012 version.

Budget guide from El Salvador from 2012 in its citizen version.

2. Interaction with the demand

3. Publication formats

Generally, in first-generation portals, the publication is based on an interpretive exercise of governmental transparency obligations and an analysis of the information already produced within government, for the purpose of compliance with fiscal responsibility norms. Some governments incorporate external voices--in addition to governmental ones--during the publication process, which in most cases are from specialized academia, press and civil society organizations (CSOs) interested in analyzing different aspects of public spending, such as fiscal sustainability, revenue resources and budget allocations.

In the case of first-generation portals, downloads occur in closed5 and/or non-structured6 formats. This represents a great advance regarding the alternative, in which only some people can have access to budget and fiscal information, and even then, perhaps, only through an explicit request for information.

An example of this, is the 2011 version of the portal published by Mexico’s Ministry of Finances and Public Credit, in which eight CSOs specializing in public finance and accountability (represented by a collective) participated in the process of defining and prioritizing content. Through established meetings and working groups, the team in charge of development sought how to firstly better respond to the need for information and, secondly how to display and use the portal to improve Mexico’s score on international indexes on fiscal transparency. This interaction laid the foundation for a collaborative relationship, building trust over time, and gradually facilitated an increase in political support for the portal4, given its international recognition.

An example of this is the 2018 version of Kosovo’s fiscal transparency portal, where, in addition to finding spending visualizations, people can download and consult the data in Excel spreadsheets and PDF files.

11 Fiscal transparency portal from Kosovo in its 2019 version.

Approved budget from Kosovo from 2019 in its citizen version.

4. Mechanisms and update frequency Considering the development of financial management systems, the first generation portals tend to be updated though manual interventions. In some cases, these manual interventions consist only of uploading a database from an internal system onto a content manager, while in other cases, even a manual integration of databases is required. Depending on the degree to which manual intervention is necessary, and on political decision making, the update frequency can match or be greater than the frequency of legally mandated public finances reports. This pertains to portals in which approved budget information is updated at the beginning of the year and paid budget information is updated upon delivery of the year-end report.

Fiscal transparency portal from Mexico in its 2011 version.

4. To learn more about the process of development behind Mexico’s portal, consult the blog post Weaving a New Narrative: How fiscal transparency Reforms Took Hold in Mexico (http://www.fiscaltransparency.net/blog_open_public.php?IdToOpen=5033) and 10 Lessons from the Implementation of a Fiscal Transparency Platform (http://www.fiscaltransparency.net/blog_open_public.php?IdToOpen=3023).

5. An open format is a specification for storing digital data that is usually published and promoted by an organization with open standards, and that is free of legal and economical restrictions for its use. To learn more about open publication formats, consult the Open Data Tutorial developed by GIFT and the Ministry of Finance and Public Credit of Mexico in: bit.ly/OpenDataTutorial. 6. Structured data refers to data that can usually be found in most databases. These are files containing information typically shown in rows and columns with titles. Such data can be organized and processed easily by machines.

5. Design The visual design in the first generation of portals is developed with the purpose of achieving aesthetic improvement. It is very common that graphic implementations are developed and also authorized by teams without taking users into consideration. Sometimes, approval is even carried out based on the personal tastes of the directors responsible for development. As an example, the following image shows the 2008 version of Brazil’s fiscal transparency portal. There is an obvious emphasis on increasing the provision of information, which could lead to visual and content saturation for the audience. In its redesigned 2018 version, the government sought to improve user experience.

6. Communication channels When it comes to communication, we can observe that first-generation channels are unidirectional, wherein the publisher provides information to a passive actor in the portal, without seeking interaction or feedback from users. In these cases, communication is seen as an additional action to be carried out after publication and is not part embedded in project design. It is however key to distinguish between communication being used as a tool to provide useful information in an objective, timely and open manner; as opposed to being merely a tool to promote an institution, government or person.

1st Supply of information

The following figure summarizes the characteristics of first-generation portals in the six categories.

While disaggregation of the supplied information can vary widely in the second generation in terms of this concept, publishers normally go beyond the provision of legally required and aggregated information--for example, through the publication of platforms dedicated to program evaluations, budget literacy, procurement, etcetera. This tends to be supported by stronger financial management information systems and the systematic integration of a culture of transparency within the government organization. 2. Interaction with the demand

Fiscal transparency portal from Brazil in its 2008 version.

12

1. Supply of information

Interaction with the demand Publication formats Mechanisms and update frequency Design Communication channels

Generation

The evolution of modern technology allowing for the storage and processing of large amounts of data has given way to the incorporation of people from new communities from within and outside government--who have the interest and capacity to process large amounts of data. These people can have different backgrounds and objectives for the use of such data, whether it be for academic, journalistic, or social advocacy purposes, among others. Second-generation portals recognize the existence of these groups--which are external to the community of financial analysts at large--and seek to meet their needs. As an example of this, consider the meetings held by the Dominican Republic’s Ministry of Finance starting in 2017, which included universities, economic journalists, CSOs, private sector organizations and those responsible for the access of information, so as to introduce the fiscal transparency portal and discuss their specific needs.

Little additional information available from that already contained in existing government reports. Engagement with specialists in fiscal matters, such as CSOs, academia and press (not in all cases). Closed formats and visualizations with no possibility of information crossing.

Dominican Republic.

3. Publication formats The open data revolution penetrates the core of open government’s publication priorities, including fiscal openness, and as such is central to second-generation publication formats.

Manually updated data. Design not developed by specialists. Few, unidirectional communication channels.

An example of fiscal data openness can be found by looking to Indonesia’s Ministry of Finance, which developed a portal fully dedicated to open data.

13

Fiscal transparency portal from South Africa in its 2019 version. Open fiscal data portal from the Ministry of Finance of Indonesia in its 2019 version.

For another case, we can look to the Ministries of Finance from El Salvador, Argentina and South Africa, which integrated a section dedicated to open data within their fiscal transparency portals.

4. Mechanisms and update frequency Fiscal transparency policies begin to consider the process of data release, that include capacity and developing technological tools for the extraction and publication of data from its sources. This connectivity allows for more frequent updating and the publication of more disaggregated data. In what we consider the second generation, the automatic update of timely data is sought after; however, this may not be possible for all data, considering the need to integrate or consolidate information from different sources. This distinguishes the second generation from the third generation.

14

5. Design

Fiscal transparency portal of the Ministry of Finance of El Salvador in its 2019 version.

The second generation in terms of visual design is linked to the pursuit of “citizenizing” the portal’s contents. The general premise lies in thinking about the user, though still without engaging the different audiences, or considering their capabilities and preferences. It is in this second generation where visual design detaches itself from the formality of institutional graphics--less serious typographic styles are implemented and the use of illustrative and audiovisual sources is increased. 6. Communication channels

Fiscal transparency portal of the Ministry of Finance of Argentina in its 2019 version.

The second generation of communication refers to the opening of additional channels to communicate fiscal information to users. For example, the citizen’s budget is no longer just published on the portal, but also disseminated on social media. This represents an important step for communication, since it recognizes the need to disseminate content so that its existence and potential uses are known. However, in this second generation, communication channels are used only unilaterally for the provision of information to the user, and still not for interaction.

15

As before, the following figure summarizes the characteristics of second-generation portals in the six categories.

1st

Generation

2nd

Generation

Supply of information

Little additional information available from that already contained in existing government reports.

Disclosure of information beyond what is legally required and with more disaggregation.

Interaction with the demand

Engagement with specialists in fiscal matters, such as CSOs, academia and press (not in all cases).

The voices of open fiscal data analyses communities are incorporated.

Publication formats

Closed formats and visualizations with no possibility of information crossing.

Addition of open fiscal data.

Manually updated data.

Some connections to internal systems.

Design not developed by specialists.

Design developed by specialists is dedicated only to aesthetic purposes meant to attract users without engaging them in the process.

Few, unidirectional communication channels.

Addition of social media with unidirectional use of informative content publication, but without interaction.

Mechanisms and update frequency Design Communication channels

Romero, de la Mora and Ruiz (2016) examined the actions undertaken by governments as they addressed problems within the realms of the impact and lack of use of fiscal transparency portals. Subsequently, they identified the following measures to address the issues: the provision of tools and resources to help users interpret unprocessed data; online tools that are useful to non-expert users; linked and cross-referenced information between websites that offer orientation for data users; and access to basic data for users; as well as the development of better feedback cycles from users and report mechanisms. As can be seen, whether they are experts in fiscal matters or not, the common denominator of these actions is the centrality of the user. Portals whose publication processes--policy design; data formats and prioritization; and experience design--are based on the user; iterated according to a continuous analysis of the demand through monitoring; and designed with two-way communication and engagement tools are what we call third-generation fiscal transparency portals.

1. Supply of information

16

In a nutshell, we’ve witnessed fiscal transparency portals that range from its about information in the first generation, to its about data in the second generation.

As illustrated by de Renzio and Mastruzzi (2016), the lack of information disaggregation, difficulty in cross-referencing between data sets and the inability to carry out comparative analysis proved to be among the greatest limitations to the increased use of fiscal information, denoting a gap between the characteristics of government-provided fiscal information and the information needs of CSOs. Unlike what happens in the second generation, where disclosure of more information is privileged, the publication processes of the third generation are based on the detection of the demand’s needs and the discovery of links between data that facilitate a comprehensive contextual understanding of topics. For example, in the “Evaluation” section of Uruguay’s Planning and Budget Office’s fiscal transparency portal it is

17

possible to read all external evaluations of public policies. Each evaluation sheet also includes the relationship between public policy and the UN’s sustainable development goals, as well as links to information on related budget programs and each responsible agency.

2. Interaction with the demand In the third generation, the fact that the demand for information can have different purposes beyond financial management, and that it can help to improve the design and implementation of public policy and service delivery, is recognized. Based on this, it is accepted that the demand comes from different profiles--such as, CSOs with an agenda focused on specific areas such as social justice, gender, equality, environment, etcetera--and that it should, therefore, be placed at the center of decision making.

18

Such is the case of Vulekamali, South Africa’s fiscal transparency portal, which was conceptualized and developed in a partnership between the National Treasury and Imali Yethu, an open coalition of CSOs. Iterations for improvement of the portal and its contents result from public engagement through data exploration and budget literacy events, which are carried out in universities and public spaces across the country.

compatible and comparable. In addition, these standards grant a methodological guideline for the construction and publication of data that has not yet been opened7. GIFT and the Open Contracting Partnership, in a coordinated effort, have developed a link between the Open Fiscal Data Package and the Open Contracting Data Standard, with the hope that the publication of this data will facilitate traceability between the processes of budget allocation and the execution of public contracts.

Open Fiscal Data Package (Stages of the transaction)

Open Contracting Data Standard

Executive Budget Proposal

Approved

Planning

Apportionment

Reservation

Initiation (Tender)

Commitment

Award

Adjusted

Contract

Verification (Accrued)

Payment order

Payment

Implementation

(Stages of the contracting process)

19 4. Mechanisms and update and frequency As a consequence of the greater supply of information and the attention placed on demand needs, it is necessary to minimize manual interventions to maintain information integrity. Thus, in the third generation, the connection to internal systems is vital to guarantee timeliness and sustainability. In the second part of the fifth module, titled “Systems and data connectivity” this topic is further explored.

3. Publication formats As with other variables, publication formats are not static but rather depend on the demand of information. They seek to accommodate a range of audiences, presenting different consumption mechanisms that are legible to machines as well as to people. It should be noted that technological development allows the presentation of information in different formats for visualization, download, use and reuse, simultaneously by linking different structured open data sources. Regarding open data, in accordance with the supply of information, the use of open data standards is privileged. An open data standard is a homogenization of information in a format structured through unique templates that facilitates crossing between data sets. The structure of this information is thus critically important if the wide use of data is sought. In particular, it allows data to be linked with other data of the same type to provide context. Standardizing information allows broader communities to identify the same meanings when identical concepts are used, even in different contexts; hence, the information is

As can be seen in the case of Argentina’s open budget portal, open data on budget execution is published daily with a disaggregation to the line-item level--a detailed category by object of spending in economic classification. This would not be possible without automatic updates from internal systems. Fiscal transparency portal from the Ministry of Finance of Argentina in its 2019 version.

7. To learn more about this topic, check the Open Data Tutorial - Opening and promoting use of budget data: http://bit.ly/OpenDataTutorial

5. Design Visual design, like the other characteristics of third-generation portals, is conceptualized, created, tested and refined based on research and engagement with different potential users to determine their capabilities and, especially, their preferences. Achieving a positive and useful user experience is the basis for development. The various iterations of Mexico’s portal, the Intelligent Subsidies Platform of the National Government, offer a number of ways to find information about subsidies delivered by government programs and the requirements to access them.

20

Prior to determining the design and technology to be used, an analysis of potential users was carried out that included considering their goals and capabilities. This analysis was made possible through a number of interviews that took place in cities across the country, which were made possible through a collaboration with SocialTIC, a CSO dedicated to promoting the use of technology for social advocacy. As a result of this research, the use of natural language processing--an artificial intelligence mechanism--was selected in order to develop an information search platform that adapted to the ways in which users formulated their questions in the search engine and chat box. 6. Communication channels In the third generation, users stop playing a passive role, and instead take on a central, active role that requires useful information. Thus, communication channels become mechanisms of two-way interaction where publishers provide information, creating opportunities for engagement. Vulekamali, South Africa’s fiscal transparency portal, uses social media to actively encourage analyses from the population. This analyses is incorporated into the contributions section of the website.

The following chart summarizes the characteristics of third-generation portals in the six selected categories.

1st Supply of information Interaction with the demand Publication formats

Mechanisms and update frequency Design

Communication channels

Generation

2nd

Generation

3rd

Generation

Little additional information available from that contained in already existing government reports.

Disclosure of information beyond what is legally required and with more disaggregation.

Detection of the needs of the demand, and data crossing to provide users with context about published information.

Cooperation with specialists in fiscal matters, such as civil society organizations, academia and press (not in all cases).

The voices of open fiscal data analyses communities are incorporated.

The demand is recognized and incorporated in a way that accounts for diverse topics and profiles (e.g. gender, environment, equality, education, etc.).

Closed formats and visualizations with no possibility of information crossing.

Addition of open fiscal data.

Structured open data is published in a way that seeks to attend to different audiences by presenting different consumption mechanisms that are legible to machines as well as to people.

Manually updated data.

Some connections to internal systems.

Connections to diverse systems, and fewer manual updates.

Design not developed by specialists.

Design developed by specialists is dedicated only to aesthetic purposes meant to attract users without engaging them in the process.

The demand is engaged in development, and there is a desire to achieve a positive experience for users.

Few, unidirectional communication channels.

Addition of social media with unidirectional use of informative content publication, but without interaction.

Communication channels are implemented as mechanisms for two-way interactions, where publishers generate opportunities for engagement.

This process of evolution, as with all process sciences, has been neither linear nor globally homogeneous. Even portals that are considered the “most advanced” continue to evolve. Regardless of the state in which fiscal information is found within ministries of finance, it is possible and necessary to aspire to third-generation portals; shorten the pathway to achieving an effective publication; ensure that existing information within the government responds to the needs of users; and take into account the rapid growth of current technology, with the user involved in and at the center of development. In the following module, an introduction to user-centered design is put forward, a design method focused on solving the needs of the end users of a product, which, through the implementation of a cyclical process of stages, serves as a guide and strengthens the development or upgrade of a fiscal portal--from planning and conceptualizing to post-launch monitoring--facilitating the processes of constant improvement.

21

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.2 Introduction to user-centered design (UCD)

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

Digital projects within governments can be considered as being internal management, service delivery and/or public communication investments. Governments that are focused more on processes than results, however tend to minimize the importance of considering what users want or need. As Romero, de la Mora and Ruiz (2016) state, when faced with a lack of use following the development and launch of a portal, governments are forced to use additional human and financial resources. Sometimes, however, consequent upgrades, or even total redesign processes, are carried out once again without engaging users.

04

As Benyon (2014) states, in the human-computer interaction field of study, a product has a greater possibility of satisfying the users’ expectations when they are engaged during development. The engagement of users not only facilitates an understanding of their needs and capabilities, but also serves as an early alert system for detecting possible positive and negative aspects that the development team may otherwise have overlooked in vital areas such as usability and accessibility. Different problems that occur during and after the development or update of a publication--such as if a government extends the implementation timeframes, thus delaying publication; if the demand is not taken into account in a planned manner to obtain useful feedback; or if requirements are not clear, and consequently the final product does not have the expected specifications for the publication of content--can all be addressed through the appropriate, step-by-step implementation of an iterative process with user engagement.

With regard to government’s responsibility to meet the information demands of the population, usability.gov (n.d.) states that by adopting a user-centered approach for developing government portals, government agencies: improve the quality of the information they have by making it more useful and usable, save monetary resources in the long term and increase their credibility by being more transparent during development. User-centered design (UCD) is a method that focuses completely on addressing the needs of the final users of a product, whether related to editorial design, urban design, architectural design or web design, among others. This iterative design process is not the same as designing by “thinking about the user”, because it requires engaging the users that a product is being created for, during development. The goal of UCD is to achieve a positive product user experience allowing users to resolve their specific needs with a high degree of satisfaction, with minimal effort; and developers to evaluate and adjust the design until this result is achieved.

The five levels of user-centered design When designing a project using the UCD method, five levels are transited (adaptation of the authors, based on Jesse James Garrett, 2011): CONCLUSION CONCRETE

VISUAL DESIGN INFORMATION DESIGN STRUCTURE DESIGN REQUIREMENTS STRATEGY

CONCEPTUALIZATION ABSTRACT

As can be seen in the above chart, the lowest, most abstract level is Strategy, that is, the first phase of the project. In terms of fiscal transparency portals and platforms, this level is where the what for and with what questions steer the project being developed; in other words, this is where the general purpose of the portal or platform is defined, inclusive of its objectives and goals. The Requirements level is where the portal’s scope and content requirements are defined, and where its functional specifications are established. Some of the characteristics--or requirements--that are documented at this level, and that determine specific functionality, could include examples such as when a user interacts with a spending graphic by downloading the open data for a more detailed analysis, or when, as a user analyzes an investment project, they can choose to visualize it on a georeferenced map to get a clearer idea of the possible beneficiaries. The third level is Structure Design, which focuses on the possible navigation paths available to users in order for them to move from one section of published information to another. The types of structures will be discussed later in this tutorial. The penultimate level is Information Design, where proportions and distribution plans for elements—text blocks, buttons, menus, etc.—are developed through the use of a visual hierarchy that helps users easily find what they are looking for. Finally, the concluding and most concrete level is Visual Design, wherein decisions are made concerning the fonts, colors, images and other elements with which users will interact.

05

As development progresses from one level to the next, the problems to be solved become more concrete and their level of abstraction diminishes; decisions, too, become more specific and detailed. For example, in the Strategy phase, the portal or platform’s visual design should not be a concern; instead the focus should be on how it will be inserted as an interface to accomplish the project’s general purpose. Tip: Any portal intended for publication is made for the purpose of reaching users; therefore, it is vital to not only think about users, but to actively engage them in development through a user-centered process.

Steps to implement user-centered design As the UCD process requires small repetitive stages or iterations, to carry out this method and transit through the five levels, it is necessary to understand each step in order to implement it. The steps for executing an UCD are planning, conceptualizing, creating, prototype testing and refining, launching, monitoring, and finally evaluating, which together form an iterative design, and which will be the axis for the following modules of the tutorial.

06

The first step in an iterative design when developing or updating a fiscal transparency portal or platform is planning, which is closely linked to the Strategy phase. One of the most common failures when launching a portal is the lack of demand. Although this can be due to a variety of reasons, the most common one is that the simple key questions of What do we wish to achieve? and What are the project’s general purpose and objectives? are not asked until after the first lines of code have been written, and other binding decisions taken concerning for instance what information will be published together with the applicable data sets, and what color palette will be used. These essential elements of planning will be addressed in the following module of the tutorial.

PLANNING The second step, associated with the Requirements phase, is conceptualizing. In this step, for the project to be successful, it is not only necessary to think about users, but to engage them, gathering as much information as possible about them, their needs and capabilities. In the module “Identifying and segmenting the audience”, various tools are introduced to accomplish this step.

PLANNING

CONCEPTUALIZING

PLANNING

CONCEPTUALIZING

CREATING Prototype testing and refining

When the prototype testing and refining stage is completed and the fiscal transparency portal’s programming is finished, it can be taken through to the next step: launching. This step of the project must be linked with communication and engagement strategies.

PLANNING

CONCEPTUALIZING

CREATING

LAUNCHING

What can be measured can be improved; therefore, the next step is monitoring. To measure and understand if the portal or platform is contributing to the fulfillment of the publication’s general purpose, it is necessary to gather information post-launch. In the module entitled “Monitoring and evaluation to improve”, multiple tools are introduced to accomplish this goal.

PLANNING

CONCEPTUALIZING

CREATING

LAUNCHING

MONITORING

With the information gathered through monitoring, the developer team can begin evaluating the portal or platform against the technological requirements and project’s objectives to determine strengths, weaknesses and necessary improvement actions, whether they are related to the portal’s conceptualization, processes or impact.

PLANNING

CONCEPTUALIZING

CREATING

LAUNCHING

MONITORING Evaluating

PLANNING

CONCEP TU A

CREATING

Evaluating

M

ON

IT O R I N G

H UNC ING LA

G IN LIZ

CONCEPTUALIZING

Once a portal prototype is finished, prior to going any further, testing and refining it with users is necessary. This is a very important midpoint, where usability testing can be implemented using techniques such as observation or focus groups, a topic which will be addressed in a later module. This testing stage will yield information that is valuable throughout the cyclical process of refinement, prior to publication.

Once the evaluation is finalized, a cycle of iterations occurs to help further improve the portal or platform.

Once the project is conceptualized with the demand engaged, the next step is creating the portal or platform. The three remaining levels of user-centered design--Structure Design, Information Design and Visual Design--come into play at this stage.

PLANNING

Tip: This is when, based on the information gathered during conceptualization, the drafting of engagement strategies begins, which will occur simultaneously with the development of communication strategies. These sets of strategies will accompany the portal at and after its launch in order to achieve a comprehensive product. These strategies are further discussed in later modules.

Prototype testing & refining

C R E ATI N G

07

In some cases, the lessons learned through iteration can allow for scaling the portal. An example of this might be the creation of different internal platforms that meet the diverse needs of users from different audiences with respect to specific areas of the budget, such as those related to gender, elementary education, natural disasters, etcetera.

PLANNING CONCEP TU

G

IT O R I N G

Prototype testing & refining

IN

M

ON

A LIZ

Evaluating

NCHING U LA

C R E ATI N G

SCALING 08 With this iteration cycle, we close the second introductory module of this tutorial. Now we can start developing a fiscal transparency portal with user-centered design! In the next module, which is linked to planning (the first step of iterative design), discussion topics include the nuances of having a general purpose for publication, accompanying objectives and indicators, together with methods to define them.

%

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.3 Defining objectives and indicators

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

While some countries have committed to protecting access to public information as a fundamental right¹, proactive transparency, which generally also has legal foundations, entails that anyone interested can have immediate access to public information and avoid the costs of submitting a request or participating in administrative procedures (OECD, 2011). With this in mind, the proactive publication of fiscal information can have strategic objectives that enrich and extend the protection of rights. As stated previously, it is important to be clear about the objective of publishing a fiscal transparency portal or platform. This is necessary to allow the portal to be subsequently evaluated, thereby providing opportunities for improvement. Determining the reason behind the development of the project--that is, defining the general purpose of the portal, with its specific objectives and indicators--is an essential first step; referred to here as planning.

M

ON

IT O R I N G

Having a unified objective motivates people to collaborate towards shared results, sets the direction for subsequent transformation activities and facilitates coherence between the involved areas and organizations. A general purpose or objective must be disseminated within the project’s leading organization--whether it’s a ministry of finance, a government-civil society coalition or some other form of leadership--from groups at management levels to those in charge of the technical processes. The following are examples of some general objectives that have been put forward for a fiscal transparency portal:

Prototype testing & refining

G

04

HING UNC A L

IN LIZ

Evaluating

A portal’s objective or purpose can vary depending on the state of each country’s public financial management. However, the primary intention is for the implementing team to be able to properly identify the portal’s requirements by considering outstanding social needs. The information and form of publication can vary based on a portal’s general purpose, specific objectives and performance indicators.

Facilitating the monitoring of a specific public service Improving public engagement and interaction Providing certainty about the economy’s position for financial markets Accompanying and strengthening public financial management reforms

PLANNING

CONCEP TU A

How to define the objectives of a fiscal transparency portal

Tip: Keep in mind that a portal can be composed of different platforms with information and structures that serve different objectives and audiences. This topic will be addressed later in the tutorial.

C R E ATI N G SCALING

Defining specific objectives Objectives should describe the desired results of the publication, and should be used in project planning either by governments, by civil society organizations (CSOs) or by coalitions. To produce the greatest benefit, these objectives must be defined at the beginning part of the project’s life cycle, be specific and measurable, and comply with time, budget and quality requirements. A project must have a general objective (outcomes), intermediate objectives (products or outputs) and immediate objectives (inputs or activities), all of which should be consequently achieved. Often, however it will be realized that the successful publication, use, or impact of the information, requires regulatory or process reforms--which will imply more complex policies or actions than the mere publication. There are a range of specific techniques that support the planning and monitoring of policies and projects with a performance-oriented approach. These help 1. Article 19 of the Universal Declaration of Human Rights recognizes the access to public information as a human right. The Declaration was proclaimed by the United Nations General Assembly and signed by 48 countries in 1948.

Inputs or activities Products or outputs

Outcomes

05

ensure that any activities, outputs and publishing results have a sequential logic, and facilitate the delineation of indicators for each level. Below two of the most commonly used techniques are presented: Logical framework methodology – This technique is based on the identification of a problem, including its roots and consequences; the transition towards objectives; and the hierarchical structure of inputs, outputs and outcomes. Normally, the logical framework matrix, which helps to establish indicators for each level of the hierarchical structure, will stem from this identification process. To learn more about this methodology visit the Manual for the Logical Framework Methodology for planning, monitoring and evaluation of projects and programs from the Economic Commission for Latin America and the Caribbean (CEPAL) here.

The theory of change - This methodology is based on the identification of a chain of causality that links objectives, inputs, products, intermediate results and impacts, which in turn facilitates the definition of hypotheses to be verified. Unlike the logical framework methodology, the theory of change offers a “general overview” and summarizes the work at a strategic level.

Tip: It is a common error to consider user or page visit counts as results indicators. While these are significant metrics to monitor, it is important to keep in mind that they say little about the actual impact of a portal.

Monitorable - Their metrics must result from a specific calculation linked to the data sources being utilized. Indicators can have different dimensions such as quality, efficiency, effectiveness or satisfaction.

Tip: One way of measuring the audience’s level of satisfaction is by conducting user surveys about the portal’s use and utility using objective-oriented questions. These surveys can be kept on the site/s, with results extracted periodically to allow for an analysis of trends regarding the improvement or deterioration of satisfaction.

To learn more about this methodology, visit the following guide for the Theory of Change, Indicators and Impact Measurement by Abdul Latif Jameel Poverty Action Lab (J-PAL) here.

Application example relating to the module on defining objectives and indicators

06

Tip: No matter which technique is used, the audience can participate in defining the objectives of the portal or platform. This can be facilitated by opening up the process to public inputs. Examples on how to do this: • Publish a draft of the strategy document on the leading organization’s site and accept comments through online tools (such as Google Survey, Typeform, Survey Monkey, etc.). • Create a collaborative document for each phase where those who are interested can comment and propose changes (by using Google Docs, for example). • Open a public invitation for co-creating strategy (bearing in mind there is a maximum capacity for participants that will allow work to be guided).

The following exemplifies this module with a case related to performance evaluation, which was inspired by real cases discussed within the GIFT network. Main problem: Civil society organizations, academia and the legislative branch face difficulties when trying to use government programs’ performance information to base their studies and analysis. Roots of the problem: • The performance information is published by each government agency in its own portal (scatter).

Defining indicators

• The published information is generated in a different format by each agency (heterogeneity).

Indicators assist in assessing whether the performance-oriented actions taken are having the expected impact and in redirecting them, if they have not.

• The published information lacks the necessary details--methods of calculation, sources of information, etcetera--to enable a rigorous analysis (comprehensiveness).

Indicators must always be: Clear - They must be accurate, unambiguous and related to the objective. In cases where specific measurement is not feasible or the necessary data is not available, it is possible to use proxy² indicators (or indirect measurements).

• The indicators are only published with the approval of the budget; monitoring throughout the budget execution is not published and the results are only published with the year-end report, and even then, only for selected indicators (availability).

Relevant - They must reflect the importance of achieving the objective and provide information about what should be measured. Accordingly, they should be appropriate for their respective level--that is, as markers of immediate, intermediate or outcomes objectives.

2. A proxy indicator is a measurement or indirect sign that approximates or represents a phenomenon in the absence of a direct measurement or sign.

• The published indicators are low-quality and management-oriented (quality). • CSOs and legislative branch actors ignore the information existence (knowledge) and/or how to use it (capability).

07

Outcomes objective: CSOs, academia and the legislative branch have fiscal performance information (financial and non-financial) that supports the discussion of the budget in the different stages of the budget cycle. - Indicator 1: Percentage of CSOs, academia and legislative branch actors that report that the published information has facilitated their analysis of government performance in specific issues3. - Indicator 2: Percentage of budget with non-financial performance information updated according to its measuring frequency. Intermediate objectives: • Consolidate the financial and non-financial performance information of the different government agencies into a single point. - Indicator 1: Progress percentage in the development of the fiscal transparency portal’s first iteration. • Establishing a homogeneous format. - Indicator 2: Percentage of administrative units or departments that report their non-financial performance information in the structure established by the Ministry.

08

Immediate objectives: • Managing the development of a user-centered fiscal transparency portal. - Indicator 1: Progress in the implementation of a user-centered methodology for the new fiscal transparency portal. - Indicator (n)...

%

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

3. This case includes a proxy indicator due to the difficulty of measuring the “performance-oriented approach of budget discussions”.

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.4 Identifying and segmenting the audience

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

Repeatedly, statements are made about the public’s lack of interest in the information published on fiscal transparency portals or platforms. In addressing this, it’s important to ask whether users’ information needs were identified prior to portal development. It is possible that there is an incomplete understanding of our base of potential users, as well as of their interests. It is important that the following questions are asked: • What type of information arouses interest or is useful to the population? • Who could the users or user organizations be? • How frequently do they need and consult the information? • What terminology makes it easier for them to find what they need? • What sources do they currently use to find or request that information? • In what format would the published information be most useful to them?

04

IT O R I N G

05

• User capabilities Prototype testing & refining

G IN

M

ON

LIZ

Evaluating

In a UCD process there are two key elements to consider: user capabilities and goals.

H I NG UNC LA

TU A

G IN

CONCEP

LIZ

PLANNING

As mentioned earlier in the second introductory module, UCD is a method that focuses completely on providing solutions for the needs of those who will become a portal’s final users. It is not about “thinking of users” throughout design, but rather engaging them during the process in order to truly understand them. In any PLANNING development of technology, including HING CONCEP the creation of fiscal transparency UN C TU LA A portals and platforms, the best results are achieved with user Prototype Evaluating testing & participation during the refining conceptualizing, creating, prototype ON IT O R I N G CR E ATIN G testing and refining, and monitoring stages. SCALING M

Students, journalists, government program beneficiaries, private sector groups, social activists or officials from different levels of government are just examples of some of the potential users of a fiscal transparency portal, each with different interests and information needs. In this fourth module, tools are introduced to identify the demand within and outside of government, which will simplify decision making on prioritizing the information, which formats to privilege, what update frequency to implement and which language to use. This falls within the conceptualizing stage.

4.1 Identifying and segmenting audiences with a UCD approach

C R E ATI N G SCALING

It is worth mentioning that while a fiscal transparency portal can contain different platforms for meeting the various data needs of users based on the information available within the government, when developing the portal amid the conceptualizing and creating stages, it is necessary to identify not only the different audiences in play, but also the users’ capabilities and goals.

At the Requirements level of a UCD, the portal’s content requirements, and their scope, are defined, where the functional specifications are established considering users’ capabilities. In other words, to achieve a positive user interaction, during the development and update stages of the portal or platform it is vital to consider the audience’s capabilities.

VISUAL DESIGN INFORMATION DESIGN STRUCTURE DESIGN REQUIREMENTS STRATEGY

These are some examples of such capabilities: - Level of understanding concerning fiscal matters - Preference for certain information formats - Technological abilities; use of internet, devices, apps, APIs, databases, social media, etcetera. • User goals

This module contains the way to identify and segment audiences with a user-centered design approach, user research and segmentation tools, and finally, small tips on progressive disclosure.

Once the different types of users are identified considering their capabilities, the demand can be segmented. To help facilitate users’ various goals, the portal’s audience should be grouped based on similar profiles, or segmented. The aim of segmentation is to reach a positive usability result by allowing users to achieve their goals or accomplish tasks with a high degree of satisfaction and minimal effort.

4.2.1 How to find the demand

User type Expert

Non-expert

Engage me

Facilitating

Casual user

Show me

Informing and guiding

Tell me

Instructing and directing

EXAMPLE OF APPLICATION 06

Budget literacy allows for building the capacities of those users who wish to engage but do not have the knowledge necessary to do so. Two interesting cases of budget literacy implementation come from the Planning and Budget Office of Uruguay, which engages secondary education students through budget games that enable them to understand the structure and contents of the national budget; and the Institute for Socio-Economic Studies of Brazil, INESC, which provides training on public budgets to young indigenous and quilombola people, with special focus on the areas of health, property rights, socio-environmental agenda and food security.

It is important to take into account that during the development of a fiscal transparency portal or platform, local context needs to factor into the methods used in locating the demand. Decisions regarding which of the following tools to implement should be made with the aim of reaching a highly representative value of different user needs. Requests for access to information In countries with legislation that touches on transparency and access to public government information, people can submit requests to the authorities of the different branches, autonomous bodies, and subnational governments to access information about the activities they perform. Analyzing these requests for information can be useful in detecting the concerns and main demands the public have for fiscal information. It is also important to know not only what interested parties are asking for, but also to understand how they ask for it. Occasionally, the requested information may already be in the public domain; however, if several requests are made for this information, it is likely that users are seeking it in more colloquial terms; that its location is not intuitive for the user; or even that the formats in which the information is being published does not address the needs of the demand. Dialogue with the demand Identifying interested people and organizations, meeting with them directly and listening to their information needs and capabilities will streamline future joint collaboration processes and strengthen the demand. Some strategies to build these channels with different groups are: a) Identifying organizations or academic institutions that consider fiscal matters as part of their agendas within and outside government. Participating in forums with specific communities--representatives of the press specialized in financial matters, economics, public administration or accounting academia; specialists in data science oriented toward civic advocacy; civil society organizations with a public policy agenda, etcetera--is also recommended to gather their inputs and work with them directly on a collaborative agenda. Maintaining a continuous dialogue with the specialized demand will prove very useful in targeting efforts since they are the key users of fiscal information. However, this has obvious limitations regarding the capability to engage a wider and more general audience, and with those, a limited representational value. Therefore, the following additional tools are recommended to engage with the demand:

4.2 Research and segmentation tools To foster a more thorough understanding of the methods available for gathering and integrating information on the different potential audiences, six tools on how to find the demand are introduced below. The personas and scenarios technique, which is frequently used as part of UCD processes to enable the segmentation of all the gathered information, will then be explained.

b) Performing consultation exercises such as surveys or focus groups, which expose non-expert audiences to fiscal information so as to learn their perspectives, as well as their degrees of understanding and interest. c) Identifying and establishing communication with communities that are not expressly related to fiscal issues to encourage the exploitation of fiscal information in specific agendas. For example, engagements can be held with people who are part of a register of government program beneficiaries, journalists or groups interested in thematic agendas such as gender, environment, poverty reduction, entrepreneurship, etcetera.

07

Online search trends User trends in online searches are traceable at country level. This tracking is possible with the tool Google Trends, which can demonstrate both which budget issues generate more interest in the population and which words are being used to carry these searches out.

Another method of searching, which is less technical is by topic--that is, by trying to identify which functional classification of the expenditure, such as health, water, education, housing and public transport, people are most interested in.

This tool, when properly used, can provide broad guidance on people’s interests in fiscal issues, as well as on the terms most used to research them.

08

Using South Africa as an example, we can filter all the searches carried out in the country during 2018.

Portal and platform analytics If a site already exists where information and fiscal data or reports are being published, it is necessary to examine its usage statistics, regardless of format. Namely, it is advisable to observe current usage trends in order to detect which information is used most often, as well as which is used least often, and why. One of the tools most commonly used to do this is Google Analytics. By installing this application in the portal, the following can be identified: a) The most visited sections or where users stay the longest.

Afterward, we can compare search terms like government budget, public works, government program, public service and performance indicators. Google provides a comparison for the chosen year, as well as a comparison broken down by subregion or city, in some cases. It also displays individual results from the consultation of each chosen term. Adjustments to the filters of analysis can be made, and the information found can even be downloaded in CSV format.

b) The most downloaded or consulted reports or documents.

c) User behavior on the portal, from their arrival source, in some cases, to the last page they visit.

09

d) The days and times the most visits occur. e) The devices used to connect, whether mobile or desktop, as well as the browser used for connecting (Safari, Firefox, Chrome, etc.).

f) Socio-demographic data on users, such as age range, gender and country, among others.

Example of Facebook:

analytical

information

in

Example of analytical information in Twitter:

If there are already platforms or a portal specialized in fiscal information, it is advisable to install and frequently use this tool, which contains and provides timely information about the users, even in real time.

10

Social media Social media is another important source of information on the demand’s interests; it is not only a space for direct interaction, but also an analytical tool to gauge content and trends. Trends in social media allow us to better understand the different audiences’ needs.

It should be born in mind that each population has different preferences regarding social media; and that it is thus important to identify which channels are relevant for a country given its current context.

Monitoring the social media of other ministries, organizations or communities interested in the matter is also advisable, as it helps to recognize trends beyond the limits of a single organization and in identifying the content that generates the most interest.

In the case of Mongolia, the analytical information provided by Facebook had a high representative value, since it was utilized by 72.12 per cent of the country’s internet users; in contrast to only 6.07 per cent that used Twitter.

11

https://gs.statcounter.com

Some media sites, like Facebook or Twitter, have their own analytical tools. Using these tools, growing trends, clicks, interactions, etcetera can be assessed in order to evaluate the information with the highest level of engagement, and to simultaneously create more effective communication strategies.

Finally, a great deal of knowledge and experience has been gathered by different international agencies focused on encouraging better practices in terms of public finance management and fiscal transparency. Taking international sources into account International sources provide resources to assist not only in identifying the immediate demand for existing information, but also in building long-term strategies that include processes to improve financial management.

The following are among the main international initiatives: a) The International Budget Partnership (IBP) uses fiscal analysis as a tool to improve effective governance around the world. The IBP publishes the Open Budget Index biennially, which evaluates the degree of openness of eight main documents within the budget cycle. The results of the index help provide a reference of what information is not yet published, identify areas of opportunity, strengthen openness mechanisms and prioritize information disclosure.

It is important that each created persona expresses sufficient empathy and consideration so as to facilitate a better understanding of the users being represented; though, it is advisable not to add too many personal details, since that could lead to their ceasing to represent the user group. As Don Norman has expressed, personas need only be realistic; they don’t have to seem real or even be accurate as long as they help to exemplify target end users.

b) The Fiscal Transparency Code of the International Monetary Fund (IMF) identifies good fiscal transparency practices for growth as a fundamental element of good governance.

A persona is usually made up of: •A simulated name • Demographic information (age, gender, ethnic background, family status, etc.) • A photo that represents the group of users • A group profile (public official, economics student, data analyst, etcetera.) • A career title and job responsibilities • Possible interests and objectives when navigating the portal • Technological preferences

c) The Budget Transparency Toolkit of the Organization for Economic Co-operation and Development (OECD) offers a gateway to the various global budget and fiscal transparency institutions, official instruments, standards and guidance material. The Toolkit was designed with the participation and collaboration of the broader global community of budget and fiscal transparency institutions--namely, the IMF, World Bank Group, IBP, International Federation of Accountants (IFAC) and Public Expenditure and Financial Accountability (PEFA) Program, all members of GIFT--and with the coordination of the GIFT team.

12

Example of a Persona - 1

d) The Global Initiative for Fiscal Transparency (GIFT), as part of its core work streams, publishes case studies and research documents that help to identify better practices at the international level. Additionally, it works with its stewards and partners to harmonize fiscal transparency norms and standards, like the High-level Principles on Fiscal Transparency, the Principles for Public Participation in Fiscal Policies and the Guide on Public Participation Principles and Mechanisms.

Sarah Econobel "Economics and astronomy are fun!”"

These six tools for “finding the demand”--requests for access to information; dialogue with the demand; online search trends; portal and platform analytics; social media; and international sources--will not only help in the conceptualizing stage of building a fiscal transparency portal or platform, but also in the prototype testing and refining stage, as well as in the launching and monitoring stages. 4.2.2 Personas and scenarios technique Within a UCD process, the personas and scenarios technique helps to analyze and subsequently segment all the data gathered about the users, their capabilities and their goals. This technique is executed based on the personification of objectives, capabilities and behaviors of segmented user groups through the creation of fictional characters that represent them, called personas. Once these representations are created, the possible interactions of the personas against hypothetical situations, called scenarios, are analyzed. To be effective, the personas must (usability.gov, 2019): • Represent an important group of users for the portal or platform. • Express and focus on the main needs and expectations of the most important groups of users. • Provide a clear image of the users’ expectations and how they will probably use the portal. • Help discover universal characteristics and functionalities. • Represent the patterns of real people with backgrounds, goals, values and capabilities.

Goals . Opening a blog . Improving public services for people . Traveling more . Increasing public participation in fiscal processes

Demographics

Hobbies

Age 38 Education Economist Civil Status Lives with her boyfriend Position Budget Director, Ministry of Finance

Reading Space camps

Technology

Specific skills

Internet Social media Games Online shopping Programming

Data use Public finance understanding

Frustrations . Little time to read . Little disaggregation of spending open data in the country . Bad usability in the Ministry of Finance fiscal transparency portal

13

The personas created for the UCD process of a fiscal transparency portal or platform will come alive and make sense once they interact with different scenarios. Scenarios are hypothetical interaction situations with the portal, or rather, small instances that describe how users carry out the typical tasks that they seek to accomplish. The description of the interactions between the personas and portal helps in understanding, identifying and anticipating the main, step-by-step paths that different users will take when navigating it later.

Example of a Persona - 2

Paul Codevila "We are made of algorithms!” Demographics

Hobbies

Age 29 Watching series Education Programmer Climbing Civil Status Married, no children Social outings Position Project Manager, Software as a Service

Specific skills

Technology

Goals

14

Internet Social media Games Online shopping Programming

. Buying an apartment . Maintaining a healthy body . Creating a private app to sell data analysis . Building capacity to become more competitive at an international level

Data use Public finance understanding

Frustrations . Internet speed in his country . Rainy days . New code

Example of a Persona - 3

Diane Hedge "There are more good people!” Demographics

Hobbies

Age 33 Education Attorney Civil Status Married, with children Position Health Sector Coordinator, Activist, Civil Society Organization

Theatre Strolling in the park YouTuber of short stories

Technology

Goals . Impacting the lives of the poor by advocating for a comprehensive healthcare system . Saving money for her children’s college studies . Diminishing corruption within the Health Sector

Tip: It is advantageous for technical team members to grasp the value of personas. When members of the development team reach a point of empathy with the project’s personas, they will start to propose solutions as if those personas were real and not merely imagined. Well-constructed personas become part of the team.

Internet Social media Games Online shopping Programming

Specific skills Data use Public finance understanding

Frustrations . Neglect of the population . Having no time for her family . Bureaucracy

4.3 Progressive disclosure Progressive disclosure is a strategy of publication based on information prioritization. It entails favoring relevant content for users and intentionally omitting excessive or overly detailed information, which causes overload and hinders users from achieving their goals on the portal or platform. As an example, when a user arrives at the home page of a fiscal transparency portal, it is not advisable to present them with a historical archive of the citizen versions of the budget or full explanatory information about this type of publication; instead, it is preferable to show users only one link to the latest citizen version of the budget, or to citizen publications. From there, it is possible to reveal the remaining content by using a “read more” link, through the process of progressive disclosure. This method enables developers to understand and meet the needs of the many audiences of a portal or platform during the different stages of its creation, with specific consideration for users’ needs and capabilities.

Application example The fiscal transparency portal of El Salvador displays an explanatory video, inviting users to learn more about the subject; interactive graphics of the current budget year with possible navigation to greater detail; and buttons that allow data to be downloaded for deeper analysis in each of its main sections. In this way, it addresses three different audience profiles within a single screen.

15

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.5 Internal communication for external publication

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

Continuing with the conceptualizing stage, team coordination and data connectivity systems are presented in this module.

PLANNING CONCEP

Prototype testing & refining

G IN

M

IT O R I N G

LIZ

Evaluating

ON

H I NG UNC LA

TU A

C R E ATI N G SCALING

5.1 Team coordination The assignment of clear roles in the development of a fiscal transparency portal may seem like a basic requirement, however it is one that has often not been met. A lack of clear leadership and defined roles within the teams responsible for generating fiscal information and public finance can lead to incomplete and limited implementations, or even duplicate tasks, potentially resulting in fragmented publications and the administration of parallel content, design and communications. Cases where the approved budget was published on one portal, while spending was published on another, with different visualizations, formats and disaggregation, provide a clear example of how this can practically manifest itself. This sort of situation creates confusion for users and internal inefficiency, since it requires double the maintenance. As previously mentioned in the description of fiscal transparency portal evolution, one of the pillars of third-generation portals is linked information. Therefore, it is imperative to be conscious of the need for coordination between the areas managing the project from the very beginning. SILOS

TEAM

04

05

To have possible, specific and real requirements, whether technological or informational, constant coordination between the project leader team, the technology team and the additional areas that manage the information to be published, is vital. This implies that within the ministry, a decision has been made at the highest level to promote a portal project, and that a project leader has been appointed with the authority to coordinate the areas that generate and publish relevant information. Coordination allows the team to share a vision of: What is being built Although team members may have personal clarity on what is being built, this understanding can vary from one member to another. The vision of the team should be aligned with the defined objective.

What is NOT being built Understanding what is not being built can help in determining, for example, which objectives are being left for a later iteration, as well as the priorities of the development.

The following section delves into team structure and coordination based on a number of frequently asked questions made by governments and civil society organizations within the context of the GIFT network. 1. In which part of the organizational structure should the publication’s leading team be located? Firstly, it is important to reiterate that the information included in a fiscal transparency portal can emanate from several different areas, sources and systems and vary in nature. Accordingly, there is no correct or incorrect answer for which area should take the lead on the publication. Although these sites have often been led by the budget area or department, there are examples of sites led by a transversal area of an organization. Ultimately, what is important is that the leading area has the institutional support necessary to achieve high-level coordination in the development of a publication. The leading area should, to a greater or a lesser extent: • Oversee or support the development of the portal or platform’s key content, including by carrying out encounters with potential users to ensure that content delivery is user-centered.

06

• Prioritize site iterations, which will depend mainly on the demand, the existence and maturity¹ of information, and any possibilities with regard to available materials and human resources. • Head the development of technological requirements in coordination with the area that manages the information to be published, including use cases. • Establish the characteristics of the data to be collected from the areas generating information. Tip: It is important for there to be leadership in the coordination of data and information collection in order to have alignment and standardization between content and its consumption for the user. • Authorize the visual design of the development--in coordination with the communications area if existent within the ministry--taking into account the test with users. 2. How many people should make up the team leading the publication? In reality, the number of people involved will vary depending on the dimension and complexity of the project. These publications however do require the work of multidisciplinary teams that must include: • Project management, which coordinates the technological and substantive areas, handles delays, and facilitates communication between teams. 1. By maturity, we mean the level of consolidation and systematization of data collection and its processes within government.

• Graphic and web design, which develops the visual and interactive aspects of the platform. As we will discuss further ahead, design is a crucial part of a project’s success. • Data analysis. In the era of open data publication, it is important that the leading team can: standardize the way in which data is formatted and acquired from different areas or systems; use the data for storytelling in coordination with the design profiles; and carry out tests on the correct publication of data, in coordination with the area that manages the data, before an initial launch takes place. 3. What is better: internal development or outsourcing? To answer this question, those interested in developing a publication must gauge the capabilities of the internal team and the availability of resources for outsourcing; a decision can then be made based on those factors. A common problem with hiring external service providers is that often the service agreements do not include internal team capacity building to enable the continued platform maintenance by internal resources, once the external services have been concluded. For example, it’s not uncommon to find that the portal uses a programming language not typically used within the organization, or that detailed operation manuals have not been included. The result of this is that the publication becomes outdated in the medium or long term, or that a dependency on the external supplier is generated, that can become a limiting factor if there are no further resources available to contract maintenance services.

Tips: 1) When opting to hire a third-party developer, the contract should, in addition to development, make provision for usability testing with potential users and post-test refinement; delivery of complete documentation related to development; and training on the management of both content and code. 2) If development will be carried out externally, the contract requirements should include the characteristics, programming languages and databases of the server to be used; ensure that they are supported by existing internal infrastructure; and that they allow for simple maintenance and internal escalation.

4. How can we improve communication between the substantive areas and the team in charge of technological development? It is possible that these teams use different technical terms and have a different understanding about the challenges to be faced at each stage of development. Therefore, constant and fluent communication, flexibility and adaptation based on user testing are all key to a project’s success as well as to avoiding delays. To facilitate communication and development, it is advisable to approach

07

the project as an agile software development. Agile software development simply refers to the creation of and response to change in software development with an orientation to people, which stems from the following manifesto:

Example of a paper board:

Individuals and interactions

over processes and tools

Working software

over comprehensive documentation

Customer collaboration

over contract negotiation

Responding to change

over following a plan

Example of an online board for the development of the Open Fiscal Data Package:

That is, while there is value on the items on the right, we value the items on the left more.2

08

09

While agile software development goes beyond tools, two main techniques for the agile management of technological projects are scrum and kanban:

SCRUM

KANBAN

A form of agile project management based on “sprints”. Sprints are time intervals of a maximum of one month in which parts of a product are developed or potentially made deliverable based on the determined priorities. Short daily meetings are held for planning, with changes for production, reflections and evaluations occurring at the end of each sprint.3

Kanban is a visualization method that helps track the workflow across teams. It seeks to show inefficiencies in the workflow. Unlike scrum, in which there are sprints, kanban is a continuous process. Each partial development is packaged for launch as soon as it is finished.4

There are hundreds of tools that can help in monitoring agile software development, from paper boards to technological platforms, such as Mural, GitHub, MeisterTask, Trello and GitScrum. A comparison of the different platforms currently available can be found in this blog.

2. The manifesto was first devised by 17 leaders of opinion in software development, who upon meeting in Snowbird, Utah, discovered that they were facing many of the same problems, that their approaches for software development were based excessively on plans, and that they felt as if they weren’t obtaining the types of results they could reasonably achieve. To learn more about agile software development and its twelve principles, consult: www.agilealliance.org 3. To learn more about the technique, see the scrum official guide at: https://www.scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-Spanish-SouthAmerican.pdf#zoom=100 4. To learn more about how kanban works, see this blog: https://medium.com/the-super-serious-lab/beginners-guide-to-kanban-6d1ed3babe86

5.2 Systems and data connectivity Connecting the publication of data to internal systems is an important element for portal sustainability for two main reasons: 1) Consistency, in order to avoid human manipulation of data that can generate errors in the publication. 2) Efficiency, since avoiding intermediate steps within the publication is more productive. This becomes more noticeable as the publication grows, whether by number of data sets or by frequency of publication. Even if the associated internal financial management information systems are not unified, it is possible to visualize different data sets from several different sources in the same platform. This can be achieved by establishing a data warehouse and/or APIs, which will not affect the operation of transactional systems or generate security vulnerabilities.

This is evident in the 2018 version of Mexico’s Public Works platform, which displayed physical investment projects georeferenced on an interactive consultation map, with data from three different systems: the Investment Project Planning System, the Investment Project Monitoring System and the Budget Integration System.

objectives, where it’s located, what characteristics it has and if there are shared IDs that will be used to connect them. That is, we must map: 1) the systems containing the data and 2) the specific fields that require extraction. A simple solution is to use mapping templates. To do this, a spreadsheet should be generated and a column assigned for each applicable element that needs to be mapped. This will also help identify if there are any missing data fields. The importance of this increases with project complexity involving more areas and systems. Hypothetical example of mapping for the construction of a platform on investment projects:

Public Works platform of Mexico in its 2018 version.

Another method is demonstrated by El Salvador’s portal, launched in 2018, which stores all the information behind the publication in its integrated financial management system (SAFI), a set of subsystems interrelated and integrated in its centralized and decentralized operations, wherein each subsystem plays a role in normative centralization.

Field

System containing the data

Name of the field in the system

Field characteristics

Necessary algorithm

Project ID

Investments system

PROJECT_ID

Alphanumeric

N/A

Project name

Investments system

PROJECT

Alphanumeric

Responsible ministry ID

Budget system

SECTOR_ID

Catalog

Notes

N/A

N/A

10

11 Responsible ministry

Based on the experiences of different countries, data extraction from different sources is normally required for a consolidated publication. This should however not hinder the publication. The important thing to understand is which data is relevant to the achievement of the established

Catalog

N/A Various points, separated by ; Various points, separated by ;

LATITUDE

ISO 6709:2008/Cor 1:2009

N/A

LONGITUDE

ISO 6709:2008/Cor 1:2009

N/A

Total budget

Budget system

N/A

Numeric

budget adjusted per year + n estimated budget subsequent years

Approved annual budget

Budget system

APPROVED

Numeric

N/A

Budget paid in the year

Treasury system

PAID

Numeric

N/A

Total paid budget

Treasury system

N/A

Numeric

Physical advance

Investments system

PHYSICAL ADVANCE

Numeric

X of partial advances

Investments system

COST-BENEFIT

URL

N/A

Various points, separated by ;

Investments system

ENVIRONMENTAL IMPACT STATEMENT

URL

N/A

Various points, separated by ;

Longitude

Data mapping

SECTOR

Investments system Investments system

Latitude

Tip. See this technical note from the International Monetary Fund on how to design a financial management information system with a modular approach: https://www.imf.org/~/media/Files/Publications/HowToNotes/HowToNote1903.ashx

Budget system

Link to pre-investment studies Link to environmental impact studies Fila N...

System...

n

n

paid paid per year

Use tight budget

Use paid budget

A mapping spreadsheet can be downloaded from the Open Fiscal Data Specification or the Open Contracting Data Standard; a column should be added to identify the systems from which each data field will come. Disaggregation A common misconception in the publication of data, is that in order for data to be better understood by users, it must be more aggregated. Another is that most users will not have the ability to use big databases. However, aggregated data unfortunately generates more limited interaction opportunities and, beyond that, as Renzio and Mastruzzi (2016) state, users generally want more disaggregated and accessible data. Technology and digitalization provide the ability for external analysis, consultation and visualization.

Data integrity Data integrity refers to the accuracy and completeness of the information in a database, without variations or alterations on the original information, which results in the publication of accurate information that generates trust in users. Fiscal processes are however complex, with many different information consolidation systems and mechanisms. This can hinder complete fiscal transparency. It is thus essential to keep these types of possibilities in mind, and seek to consolidate information in advance through compilation processes that have guidelines and standardized structures. These should be implemented during the design phase. When data integrity is met correctly and in a planned manner, the stored data remains complete and accurate, regardless of its update frequency.

For example, imagine a neighborhood organization interested in improving local health services. They would need data on health spending in the community, perhaps observing the budget of the closest hospitals. They would also need to know what constitutes that expenditure--how much is allocated to each line-item of current and capital expenditure--among other relevant data. Published aggregated health spending data with a functional classification only, or even up to the sub-function level, would severely limit the ability of the organization to use the data.

12

To solve this paradox of the required degree of information disaggregation, it is necessary to revert to the concept of progressive disclosure previously mentioned. This could entail the use of consumption visualizations and filters that allow users to consult the data they interested in without overwhelming them with massive data volumes, displaying the information along multiple screens based on the users’ different needs, goals and capabilities. Importance of the use of catalogs and codes While working on the publication process, it is possible to encounter suboptimal situations, such as realizing that one of the systems has an outdated catalog of administrative classification codes or that the nomenclatures of some classifications are not coming from a catalog and thus require cleaning and standardization. Chapter 6 of the Open Data Tutorial: “Opening and Use of Budget Data” provides information in this regard. The probability that the systems contain identifiers that connect processes, catalogs and standardized codes, is also increased through enhanced coordination between the different teams. The improvement of these data links will also prove useful for the general enhancement of financial management.

Tip: Always consider data integrity, that is the maintenance and guarantee of the accuracy and consistency of data during its life cycle. This is a critical aspect for the design, implementation and use of any system that stores, processes or recovers data.

Data comprehensiveness

13

Comprehensive information is at the core of quality published data. When discussing fiscal information, it must be presented in a consolidated manner in gross terms across the entire central government. It is important to provide the user with clarity on the included data, as well as on the data which is not part of the publication. This will avoid misinterpretations and facilitate communication between publishers and users. To learn more about data comprehensiveness, consult the pillars of the Fiscal Transparency Code of the International Monetary Fund6. Tip: If it is not possible to publish all the expected data and achieve your objectives in the first iteration, consider adding the work plan of future iterations on the site.

An example of this is connecting the stages of a transaction5 within e-procurement systems.

5. Pattanayak, Sailendra. Expenditure Control: Key Features, Stages, and Actors Prepared by Sailendra Pattanayak Fiscal Affairs Department. International Monetary Fund, 2016.

6. Fiscal Transparency Evaluation of the International Monetary Fund https://www.imf.org/external/np/fad/trans/

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.6 Portal development

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

As mentioned in the introductory module on user-centered design, the three levels included in the creating stage of development are Structure Design, Information Design and Visual Design.

6.1 Portal architecture To demonstrate in a simple way what structure design and information design are, we’ve included the following graphical representations:

PLANNING

Structure design

CONCEP

TU

IT O R I N G

A

Prototype testing & refining

G IN

M

ON

H I NG UNC A L

LIZ

Evaluating

Information design

C R E ATI N G

SCALING 6.1.1 Structure design In this module these three levels suggested by Garrett (2011) will be discussed, as well as tools and tips for the prototype testing and refining stage, which occurs prior to the project’s final programming.

04

Finally, a compendium of tools available to the team in charge of the project: a toolkit on open source, typographies, wireframes, images, maps, etcetera, is presented.

In structure design, the possible content and navigation paths the portal will include are defined--that is, the routes users can take to move from one section to another and back, or from one piece of information to another related piece of information, allowing them to delve into or change between topics. Here, the personas will enter into play with the different scenarios, which will assist in anticipating users’ possible behaviors in the portal, as well as how the portal will adapt and respond to this behavior. While there are many different types of structures, the following two examples reveal how different structures can lead to different results. Hierarchical structure This structure allows users to delve through ramifications from root points, in only an ascendant or descendant manner. In this structure, usually called a tree structure, information becomes more specific with each transition downward between the root points.

Own elaboration based on Garret (2011)

Matrix structure This structure allows for movement from one point to another along one or two dimensions within a publication, making navigation paths more flexible for users and more customizable based on their different needs, technological capabilities and levels of understanding regarding the content. For example, by clicking on the information of a certain government agency, a user can analyze companies that are linked to it and from there, navigate directly to contracts. In another example, for a user who might not know which government agency is linked to a particular company of interest, it would be possible to first access treasury payment data, then the company’s information, and from there navigate to the related government agency.

05

Tip: When designing a portal’s structure all of the members of the portal’s development team should be engaged. This process can be completed with post-it notes, drawings of structures or online tools. The best tool for doing this will depend on the availability of face-to-face meetings.

It is important to take into account the different electronic devices, such as tablets, laptops or smartphones, with which users can navigate the portal or platform. This factor can influence the developing team’s decisions on structural specifications--for example, on how queries (consultations to the databases on the portal’s backend) will be programmed in order to avoid consuming all of the users’ internet data when connected through their smartphones.

Tip: While it is possible to develop wireframes through hand drawings, digital tools are recommended as they facilitate a responsive design for multiple devices and resolutions.

6.1.2 Information design Once the structure is defined, information design should begin, in which wireframes--the proportions and element distribution plans, including text blocks, buttons, menus, etc.--are developed through a visual hierarchy that helps users achieve their objectives in a simple manner.

06

These wireframes will guide users in achieving their goals along one or several pages, giving them a sense of where they are in the portal and what options they have to achieve their tasks. This is the reason the wireframes should be developed based on the structure design step, seeking to anticipate the user's most common browsing intention probabilities. Usually, wireframes are a consultation document for the whole development team, since they include the navigation specifications of each component of the portal and their relation to each other--that is, they contain all the decisions and details of how the general vision of the structure will be implemented in the interface. In addition to this, they allow for quick verification that the publication’s development remains correctly linked to the conceptualizing stage.

When developing wireframes, it is important to keep in mind the progressive disclosure strategy (addressed in the final part of module four) by displaying information along multiple screens based on the different needs, goals and capabilities of the users.

Before addressing visual design (the third component of the creating stage), it is necessary to understand and differentiate between two basic concepts that allow the different audiences of a publication to have positive interactions: user interface and user experience.

07

User interface (UI) is the contact point, space or device with which users interact. In terms of our subject, the interface through which different audiences will interact with the information is the fiscal transparency portal or platforms, including all images, fonts, disposition, color palettes, etcetera.

Example 1

On the other hand, user experience (UX) deals with user research, structure and information design, prototypes, personas and scenarios, testing, etcetera. In other words, UX deals with all the elements of UCD that are used to ensure that the user’s interaction with the interface (the fiscal transparency portal or platforms) is positive. Example 2

Downloadable report

Downloadable report

UX is often confused with the usability of a publication, even though the latter is only one of the seven elements that contribute to user experience (Morville, 2004). To achieve a positive user experience, a portal should be: Useful - The creation of a portal or platform should not to be impulsive or decided on a whim; it should solve the real needs of users. Usable - It should be an interface that works. Note that usability is necessary but not sufficient on its own. Desirable - Users should find the portal attractive. It should, through the proper use of visual elements, be capable of persuading different audiences. Findable - It should feature good structure and information designs that facilitate tasks. Accessible - It should be easily used and navigated by users with disabilities, enabling technological adjustments such as screen readers, keyboard navigation, monitor configuration, etc. Credible - It should provide clear, reliable, frequent, timely, relevant, comprehensive and accessible information.

08

Here are five small tips for the visual development of a fiscal transparency portal or platform: 1. Do not imitate. It is very common in the development of fiscal transparency portals for a main source of inspiration to be the portals of other countries. As previously mentioned the context of each country is however unique, together with the needs and capabilities of users. While it is positive to be inspired by good international implementations, imitating or replicating a portal can result in a product that is not attractive to its audience, or even worse, not functional from a contextual standpoint. 2. The format speaks volumes. When buying soap, a coffee maker or any other type of product, its packaging is one of the most influential factors when forming an opinion on it. If the graphic design of the product is bad, it generates doubts about the quality of the product. Visual design speaks volumes, not only about the product, but also about the company or institution behind it. Imagine users entering an online shopping portal with low-quality graphic design. Isn’t it possible they would feel distrust about entering their credit card information? The same thing happens in the fiscal transparency field, the format of a publication speaks volumes, beyond numbers and fiscal transparency; it reflects the quality and professionalism of the institution. Which of the following graphics represents an institution with greater credibility and strength in its data?

Valuable - It should help users to achieve their tasks and meet their needs. These characteristics can be met in the development or upgrade of a fiscal transparency portal through the correct implementation of a cyclical, UCD-based iteration process.

Ejecución presupuesto Budgetdel Spending

2019 Budget Education Health

6.2 Visual design of the portal

Environment Public transport Security Social welfare Water and sewage Roads and bridges

Visual Design is the least abstract level of the creating stage, wherein the guidelines for a publication’s development become specific and detailed. It is at this level that the design of the interface, the visible face the user will interact with, begins. It is common for visual design to be interpreted as an aesthetic subject only; however, it is important to remain cognizant that this interface is the final window of all the work developed in the previous levels. It is not only about creating a pleasant and attractive graphic, but about generating a functional and effective visual solution, taking into consideration users’ capabilities, preferences and goals.

CONCLUSION CONCRETE

VISUAL DESIGN INFORMATION DESIGN STRUCTURE DESIGN REQUIREMENTS STRATEGY

CONCEPTUALIZATION ABSTRACT

Urbanization Housing Energy

It is crucial to develop fiscal transparency portals and platforms with a professional visual design that suits the capabilities and goals of users and does not generate suspicion regarding the quality of the information contained in the publications. In the toolkit included in this module, sources for visual inspiration beyond fiscal transparency portals are presented. It is important to always analyze the latest worldwide graphic trends that are generating positive engagements. 3. Always respect the institutional or corporate identity. Before commencing visual design, it is important to find out whether there are any institutional aesthetic guides or guidelines, such as standards for logo usage, an official color palette, fonts, recommended image style, etcetera. These elements are found in compilations better known as corporate or institutional identity manuals.

09

Respecting the visual image of an institution or fiscal transparency channel allows users to immediately identify the different products and platforms that are being developed, communication publications included.

video explanations,

interactive presentations.

Corporate identity manuals do not seek to restrict the creativity of the visual design, but rather to facilitate a supportive framework within the broader objective for development. Order generates trust:

Video from the Planning and Budget Office of Uruguay.

Graphic animation from the Ministry of Finance of Argentina.

When developing or updating a fiscal transparency portal or platform, as when drafting the communications related to it, text saturation should be avoided. It is important to bear in mind that we live in an increasingly visual society. Neuroscientists from the Massachusetts Institute of Technology discovered that the human brain can process whole images exposed to the eye in a mere 13 milliseconds; evidence of the brain’s incredibly fast processing speed (Potter et al., 2013).

10 Corporate stationary example.

Corporate stationary example.

4. Make use of user-centered formats. As addressed in the fourth module on segmenting audiences, to attain better engagement, it is important to develop content in formats that align with the goals and capabilities of users, taking advantage of one or several of these tools to achieve a positive experience between the user and the portal: static graphics,

infographics,

booklets,

5. Use plain language. Plain or simple language, commonly called citizen language, facilitates the public’s understanding of fiscal contents, avoiding technicalities as far as possible and disaggregating information in a simple manner. Bear in mind that when researching the demand, the terms most used by a portal’s audience to find fiscal information can easily be identified. Discovering these can help to better empathize with users. It should also be considered that specialized public officials within a ministry may understand the nuances of their own area without understanding the nuances and information of another area. Similarly, a person from civil society or the general population may also not fully understand every topic. In both cases, this generates a lack of interest in the published fiscal information. It is thus recommended the implementation of plain language as a communication policy for the creation of portals, platforms and communication pieces.

6.3 Prototype tests The possibilities that different users can conjure for a portal or platform are greater than those that a developing team can imagine on its own. It is very common to find surprises in the development process regarding definitions, that may otherwise have seemed obvious. This is the reason, testing and refining a functional prototype is crucial prior to a portal’s final programming.

Vulekamali poster.

Infographic from the Public Service Accountability Monitor (PSAM).

People’s Proposed Budget from the Department of Budget and Management of the Philippines.

11

Schedule and agenda. Tests normally last around 90 minutes, which is why the order of implementation should be prepared--project presentations, instructions for users, etc.--and the spaces between one test and the next should be calculated.

PLANNING CONCEP

TU

IT O R I N G

Prototype testing & refining

G IN

M

ON

A LIZ

Evaluating

H I NG UNC LA

C R E ATI N G

SCALING These tests can be carried out not only after the prototype is finished, but in all three phases of the creating stage, in order to cyclically refine it prior to completion. Refinements can be implemented through usability testing. This includes techniques such as observation or focus groups, in which representative users try to complete tasks while the developing team listens, observes, documents and takes notes.

Test scenarios. A key objective in testing is to efficiently plan different scenarios, or tasks, for users to complete. Indicators. Depending on the scope of the testing, the dimensions to be evaluated must first be defined, as well as their qualitative and quantitative indicators. These dimensions can include satisfaction, fulfillment of the scenarios, or use and recommendations for improvement. For example, as Heinz (n.d.) suggests, to measure satisfaction on the prototype you can ask questions such as: In what scenario would you use the portal? Were you looking for any information in particular and did you find it? What was the first thing that caught your attention? What topics did you find most interesting? The indicators of fulfillment in scenarios can be measured either with quantitative metrics--for example, the successful completion of a task, error ratio, fulfillment time, etcetera--or with open questions such as: What distracted you while you were performing the task? or What elements confused you while trying to accomplish your task? Finally, tests can be concluded with measurements on use and recommendations for improvement: What information do you find difficult to understand? or What else would you like to find in the portal? Before implementation with users, it is advisable to carry out usability testing a couple of times with team members to verify the publication is functioning well.

12

Tip: Heinz (n.d.) recommends focusing on users individually through observation, since in focus groups there is a higher probability of obtaining answers influenced by the answers of the others.

Below is a list of some of the elements to consider for planning usability testing suggested by usability.gov (2019). Scope of testing. Depending on the stage of development, it must be decided what combination of aspects, such as navigation, content and visual design, will be evaluated. Devices. It should be determined whether testing will be performed for laptop computers, desktop computers, mobile phones or a combination of the above. Number of tests. After 5-10 tests have been carried out it will be possible to identify patterns and implement improvements.

13

6.4 Toolkit for developers: outputs, open code, libraries, etc. To conclude this module on the creation of fiscal transparency portals and platforms, the following compendium of tools can be consulted, it provides various sources of help and inspiration for development. Development • Free web fonts (it’s recommended to filter in “font properties” fonts with a number of styles higher than five): https://fonts.google.com/ • Open Data Tutorial (discussion ranges from what they are to how to implement them): http://bit.ly/OpenDataTutorial • Open Fiscal Data Package (simple, free and open technical specification to publish fiscal and spending data): http://www.fiscaltransparency.net/ofdp/ • Libraries of open source visualizations: https://d3js.org/ | https://developers.google.com/chart/interactive/docs/gallery

12

• Open license maps: https://www.openstreetmap.org/ • Free images libraries (it is very important to review the user license): https://unsplash.com/ | https://www.pexels.com/royalty-free-images/ | https://freephotos.cc • Vector image libraries (it is very important to review the user license): https://www.stockio.com/ | https://www.freevector.com/ • Stickies online collaboration board: https://mural.co • Sketching tool for structure design (sharable online): https://docs.google.com/drawings/ • Wireframe and prototype sketching tools: https://www.invisionapp.com/ | https://idoc.mockplus.com

%

• Free tools to carry out surveys: https://try.typeform.com | https://www.surveymonkey.com • Tool to generate color palettes: https://color.adobe.com/create/color-wheel/ Inspiration • Trends websites: https://www.awwwards.com/ | https://dribbble.com/shots/popular/web-design • Examples of JavaScript visualizations: https://threejs.org/ • Color trends: https://color.adobe.com/trends/Graphic-design • Digital visual design trends: https://theblog.adobe.com/search/design+trends • Corporate identity design trends: https://dribbble.com/shots/popular/branding • Illustration trends: https://www.creativebloq.com/search?searchTerm=illustration+trends | https://dribbble.com/shots/popular/illustration

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.7 Amplifying the audience

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

Step1 - Information on demand segmentation analysis Fiscal information publishers, as well as developing teams, face the difficult task of capturing user interest in a competitive digital context. To meet this challenge, module seven addresses, in four steps and as part of a comprehensive strategy in the development of a user-centered portal, communication and engagement strategies, as well as tips for, and examples of implementation. 04

To implement a comprehensive strategy and amplify the demand, as seen before, it is necessary to understand users, as well as their goals and capabilities. In this sense, the analysis done previously to identify the audience--not only thinking about users, but engaging them--is vital for decision making. Tip: For this task, it is highly recommended to considering and use the tools and examples presented in the fourth module of this tutorial, “Understanding and segmenting the audience”.

Step2 - Communication strategy planning To create an efficient and effective communication strategy for a fiscal transparency portal during and post launch, it is necessary to contextualize (on a country-by-country basis) the possible channels used to reach different audiences utilizing the information from “Step 1”. The communication strategy selected should contemplate both online channels and offline channels. Selecting online channels When selecting online channels, any decision made should be based on which channels are more popular with the content’s target population segment. Occasionally, one channel can be used with different messages in order to accommodate the interests, goals and capabilities of many different audiences at once. Tip: As introduced in the fourth module, it is possible to use tools like https://gs.statcounter.com to analyze which channels have a higher representative value in the country, whether it’s Facebook, Twitter, Qzone, LinkedIn, VK, etcetera.

Digital channel preferences should take note of the types of users--expert, non-expert or casual--to which communications are directed. Let’s look at a concrete example: Imagine there’s a platform that seeks to monitor the resources for infrastructure improvement delivered to public middle schools. In this case, three different immediate audiences to engage can be detected: that is teachers, guardians and teenage students. Here, the first two audiences are adults, both presumably belonging to a similar age range and used to using comparable online channels, though their motivations and interests may be different causing messaging to vary. On the other hand, teenagers, generally have distinct capabilities, interests and goals from those of adults, and will thus tend to select and interact with digital channels in a different way.

05

The International Association for Public Participation (iAP2) suggests various digital channels to encourage participation through a strategic lens depending on the intention, whether it be informing, consulting, engaging, etcetera.

Example Tools

Inform

Consult

Involve

Collaborate

Empower

Newsletter, email, video, website, social media, live stream meetings

Social media discussion, polls, blog, workbook, survey

Ideation, survey, polls, serious games, social media discussion

Online forums, document cocreation, mapping, Twitter chat, video meeting

Decision-making: Online voting, participatory budgeting Community action: Discussion forums Source: Haas Lyons (2017).

Something to highlight on the above chart by Haas Lyons (2017) is that a single channel, as in the case of social media, can be used for multiple purposes and not only to inform, as is often the case with many institutions.

06

Tip: Depending on the content and objectives of the chosen communication strategy, it is possible to use paid campaigns on social media, directed at the target population so as to achieve wider outreach and more precise engagement.

While it is very important to use online tools such as digital newsletters, social media, online discussion forums, e-chats, surveys, etcetera., even nowadays, any comprehensive communication strategy on fiscal information must contemplate offline channels in order to reach as many different audiences as possible.

Finally, in addition to these different channels, the guerrilla marketing technique can be implemented. This technique is based on the use of unconventional advertising to reach a high level of engagement at low cost, surprising audiences and sometimes getting them to share communications voluntarily, thus causing them to go viral.

Example of guerilla marking on a general budget topic. A pedestrian crossing signage is used as a bar chart with the spending functional classification.

Example of guerrilla marketing on one budget area. Applied in front of public education centers, the approved budget of the current year for primary education is detailed.

Selection of offline channels As with online channels, the selection of offline channels depends directly on understanding users. For example, research on different potential audiences can yield profile cases that do use the internet, but make little use of social media. To accommodate these audiences, offline channels are a great tool. It should be remembered that designing messages on fiscal topics is not just about adding graphic designs or animations to numbers or information; rather, it’s about generating effective and functional communications based on users. Common offline channels like billboards, posters, flyers, etcetera, can be used in communication strategies in addition to less used channels, such as service bills or tax payment notifications, in which communications related to the subject can be incorporated. Examples might include informative messages on energy investment projects in an electricity bill, collaborative messages on surveys for decision making in determined public works, or messages of empowerment in public participation spaces, etcetera.

Example of guerrilla marketing on a specific program of the budget. In the scale figure, the amount of resources that will be used for fertilizers to support agriculture in 2020, is communicated.

07

It’s important to remember that communication strategies, regardless of the channels selected, should aim exclusively to generate useful information for users through budget openness.

Tip: When developing messages (ever in accordance with good judgement), think “outside of the box”. In the private sector, for example, do marketers create messages talking about the ingredients of a cola drink, or do they talk about happiness? Are ads for watches about proper function and accuracy, or about the lifestyle associated with a certain brand? The information within the different user profiles will be very useful to members of the communications team.

Tip: Keep in mind that user research (discussed in the fourth module) is useful not only for developing the portal but also for developing communication strategies.

Step3 - Contents and formats selection and development The preceding two steps, “Information on demand segmentation analysis” and “Communication strategy planning”, are the ingredients for starting communication pieces using the formats necessary for the different selected channels.

08

For example, should user profiles determine the selection of offline channels such as posters and flyers, content must be generated for those materials in accordance with the profiles’ interests and capabilities regarding fiscal matters. In this way, it can be ensured that messages are useful and attractive for the different profile segments, or personas. This user-centered content selection and development, whether related to static or animated infographics, videos or images, also applies to online channels. The great advantage of online channels is analytics, which contribute to the iterative improvement of planned communications; in other words, the content developed and published in digital channels can be evaluated based on its effectiveness almost immediately, providing a great space for constant improvement. Published communications can be evaluated either individually or as a whole, comparatively, allowing analyses on which content different audiences find interesting, and which content they do not. Moreover, this analytical information allows for the creation of content for specific groups and target audiences.

A key point, whether applied to posters, tweets, billboards, newsletters or guerrilla marketing, is that the content of absolutely every communication within the scope of the strategy should have a link to disaggregated information in the fiscal transparency portal related to the topic addressed in the message, since the purpose of these communications is to amplify the portal’s audience. The user will find it demotivating, and will cease to navigate the portal or platform, if, for example, a communication of interest touting new green areas in a specific city then directs them to the current year’s budget for the Ministry of Urban Development and Transport. Every part of the communication strategy should be interrelated with the portal or platform to engage the audience effectively.

FROM THE USERS

FOR THE USERS

09

INFORMATION ON THE PORTAL

CONTENT THAT IS USEFUL OR OF INTEREST

N

IO AT

L RE

ER

FROM THE USERS

FOR THE USERS

CONTENT THAT IS USEFUL OR OF INTEREST

INTERRELATED INFORMATION ON THE PORTAL FORMAT

COMMUNICATION PIECE

INT

Step4 - Engagement strategies planning Conversing with and engaging the demand, as previously addressed in the fourth module, is not only necessary to identify possible audiences and segmenting them before the launch of a portal or platform, but also serve as a means for promotion and feedback post launch, so as to increase the demand for information from the portal in a sustainable manner within the user-centered iteration process. These engagement strategies are based on organizing events where the audience is, in some cases, civil society organizations and, in other cases, the general public. Fiscal information is quite broad and it touches all aspects of public policy. Given the objective of generating greater interest and impact

with different audiences, it is advisable that events focus on specific areas of information--for example, the spending in determined sectors such as gender, environment, education, etcetera, or the monitoring of public investment projects executed in local communities. Events focused on specific topics allow their target audiences to identify how public spending affects their daily lives, and, at the same time, grant specialists the ability to explore what they are interested in, since public spending can be applied to different contexts. Among other outcomes, these events assist in: 1) Discerning the information that users are looking for and whether or not the publishing organization has it. 2) Identifying any difficulty in understanding the information and fiscal data provided. 3) Distinguishing the types of analyses carried out by different groups of experts, as well as the tools they use for this purpose.

detection of inconsistencies between published information and reality 3) generation of feedback regarding the deficiencies of a fiscal platform and 4) identification of other useful information which could be published in fiscal portals or platforms.

Application example Learn more about how the citizens of Chile took to the streets to verify the country's public works through an invitation from the Fiscal Observatory Foundation. The data belongs to all the citizens

Application example

10

The Better Budget Dataquest is an event promoted by GIFT through its stewards and partners. In 2019, this data exploration event focused on three issues: gender, inequality and environment. In the case of Indonesia, it was implemented between #BetterBudget Seknas Fitra and the Ministry of Finance across three DATAQUEST universities with more than 80 participating students. for sustainable development Another example is that of Argentina, where the Civil Association for Equality and Justice (ACIJ) carried out a data exploration that resulted in journalistic GENDER INEQUALITY research being undertaken by one of the participating ENVIRONMENT teams, that focused on the under-implementation and overestimation of goals related to the Attention to the Mother and Child program. To learn more about this engagement strategy, visit: http://bit.ly/BetterBudget2020

MARCH-MAY 2020 http://bit.ly/BetterBudget2020

Depending on the needs of an institution, there are events that seek to promote the use and analysis of fiscal information, such as seminars, conferences, workshops and forums, as well as those that seek to increase the demand for fiscal open data, such as hack-a-thons and programmer camps, that work towards the collaborative development of applications and software, or data expeditions, boosting budget monitoring content and analysis on public spending. To conclude this module of the tutorial, the #DataOnTheStreets International Rally is introduced as an engagement strategy promoted by GIFT in different countries, since 2018. This activity invites people who are not specialists on fiscal topics (e.g. the general population) firstly, to verify information on the works and investment projects executed in their communities by using databases and public georeferenced visualization tools (maps), and secondly, to report their findings through their personal accounts on Twitter and Facebook. Four significant benefits of the #DataOnTheStreets Rally as an engagement strategy targeting different audiences are the: 1) promotion of information use 2)

Tip: It is common to hear that “users don’t come to portals”; however, do information publishers go after them? Hosting events is important, but it is also key to engage audiences by participating in the events organized by civil society, since through these events it is possible to learn what information is sought, how it is sought and the limitations the public encounters when trying to use published information.

11

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.8 Monitoring and evaluation to improve

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

What can be measured can be improved. The concepts of monitoring and evaluation share a dual control function. The difference between the two is that monitoring focuses on operational factors such as activities, resources, deadlines and even recent results in order to make timely modifications and corrections, while evaluation is based on the sphere of analysis with regard to achieving higher and broader objectives. Monitoring and evaluation as control elements Monitoring

04

• Addresses aspects of a more operative nature (activities, resources, deadlines, costs and recent results in the form of products and services). • Is done during the execution of a project. • Is continuous and permanent. • Is a systematic procedure to check the efficiency and effectiveness of execution. • Seeks to identify achievements and weaknesses so as to implement corrective measures. • Allows for performance improvement to ensure a project’s success.

Evaluation • Focuses on the achievement of results. • Is done in a project’s design, execution or maturation phase. • Is carried out in established periods or segments. • Allows conclusions to be drawn on a larger scale than monitoring--at the level of a public policy, for example.

8.1 Monitoring 8.1.1 Monitoring objectives based on indicators In this section, we revisit topics from the third module, that highlight the importance of having clear objectives in a project, as well as the causal relation of elements, with quantitative, and in some cases, qualitative measurements. The previous content on defining indicators won’t be repeated in this module. As it relates to monitoring, it is however necessary to ensure that established indicators include measurement frequencies in order to enable timely decision making. As such, it is advisable that activity or input indicators have a monthly or quarterly measurement frequency, while product or output indicators have a lower frequency. 8.1.2 Functional/operational monitoring Although, as previously indicated, a site’s number of users does not reflect its impact, the monitoring of how users interact with it provides an essential piece of information for the improvement process. The most commonly used tool for this is Google Analytics, a web analytics tool that offers grouped information on a site’s traffic according to the audience, acquisition, behavior and conversions which occur on it.

Source: Diplomado PbR, SHCP Mexico 2017

Monitoring and evaluation should not be confused with fiscalization. They should be thought of as allied tools that allow the detection of areas of opportunity for improvement. With this in mind, these processes, and all processes for that matter, should be designed and constituted in a way that facilitates decision making. Monitoring is carried out on two levels, one being that undertaken in respect of the achievement of objectives and the other on functional/operative areas. The first level refers to the monitoring of established indicators to measure immediate, intermediate and general objectives (results), which should be determined in the planning stage; while the second one refers to the operational monitoring of a site. It is important to note that, due to a portal project’s characteristics, both levels will be closely interrelated. Evaluation, on the other hand, focuses on different conceptual aspects of a project, such as its operation or impact. Different methodologies can be used for these, which are broadly mentioned later in this module.

Google’s training site provides details of how to install and use Google Analytics. However, since it is a very vast tool, this tutorial shares some recommendations on analytics that can be useful.

05

• Range of dates - When measuring a website’s performance, it is useful to break information down by date ranges or fixed periods of time. Using Google Analytics, one can select the range or period of time to be analyzed, as well as make comparisons between one set of results and another from a previous period. That is, comparisons can be made using the overlapping graphs of different months or weeks, as seen in the following image.

• Average time - This metric indicates the time a user has spent navigating through a specific page of a portal. It is relatively safe to assume that the longer this time is, the more interest there is in the content is. If the time that target users spend on a page is relatively short, it might suggest that the content it offers does not interest them. Although it’s possible to find this metric applied to the general site in the initial analytics panel, as can be seen in the following image, it is comparatively more useful to assess the average time users spend browsing compared by site section. This will prove very valuable in better understanding users’ interest in different topics.

To consult the breakdown of average time spent by site section, navigate through Google Analytics by selecting “Behavior” > “Site Content” > “All Pages”. Once there, the information is displayed as seen below:

06

07

• Number of visits - One of the main uses of Google Analytics is to see how many visits a portal or platform has received. From these visits, we can extract various kinds of data, including new visitors and page views. “New visitors” is a metric that refers to all the people who have visited a portal for the first time. Similarly, “page views” is a metric that reflects the number of times a given page has been opened. This data is required to measure the general use of the site.

• Devices - As mentioned in the previous module, it is possible to tell which types of devices visitors use to access a portal (desktop computer, tablet, mobile phone, etc.). With this information, the web design can be modified as required to guarantee navigation usability (responsiveness). For example, in the following image it can be seen that a large majority of users access this site through a desktop computer.

• Site speed - The loading time of each page provides very valuable data, since if the portal or platform is slow, or takes a long time to load its contents, the user will likely tire of waiting and leave the portal without having had any kind of interaction. Again, this data should be viewed and interpreted separately by section or platform, with the understanding that the sources of information and the type of connection can be different. Beyond these, Google Analytics provides a series of suggestions for improving site performance, which will be useful when it comes to making decisions on portal improvement.

08

09

• Browser - It may seem like irrelevant data. However, information on browser use is helpful when taking development decisions given that some visualizations or tools cannot be visualized correctly on all web browsers. Tip: It is advisable to periodically go beyond Google Analytics and review your site for possible broken links. Broken links generate the appearance of a lack of maintenance and regular updating, and can frustrate users. There are an infinite number of online tools to choose from that perform whole site inspections automatically and show where inactive links are located.

Surveys The possibility of conducting surveys with portal users or with the portal’s followers on social media should be considered, from time to time. These surveys are different from the sort conducted on potential users, since the portal’s post-launch users have developed a better understanding about its content and thus can provide better feedback as to whether the structure of the portal is logical or even if there are errors in functionality that the developing team has not been able to detect.

8.2 Evaluation The objective for evaluation might not always be clear, particularly for people who have participated in data gathering and the application of methodologies that end up forming part of reports, which are rarely, if ever, utilized. However, if portal implementers ask themselves critical questions, their evaluation can reveal the answers necessary for improvement in the short, medium or long term. Evaluation techniques related to the fiscal transparency of public policies and programs can be utilized in this regard. The Abdul Latif Jameel Poverty Action Lab (J-PAL), a global research center that works to reduce poverty by ensuring that policies are informed by scientific evidence, has identified different types of evaluation that can be applied to a project, depending on its needs and current phase (Dulfo et al., 2013). Based on these, below a handful of evaluation types are introduced that can help to identify the success of a portal or platform: • Project design evaluation - If a project’s goals and assumption are not reasonable, then there is a very small possibility that the solutions it generates are effective. This evaluation type implies reviewing how the theory of change or the logical framework is articulated in the planning stage. It can aid in identifying complementarity with other transparency and engagement activities that strengthen a comprehensive fiscal transparency policy, and help to avoid producing an isolated publication.

10

• Process evaluation - Also known as an implementation evaluation. This evaluation type analyzes the effectiveness of the operation of the project based on indicators, with baselines and targets. This evaluation allows for the identification of delays in activities or outputs that could call the achievement of higher objectives into question. • Impact evaluation - This evaluation type measures the success of a project based on its objectives, wherein “success” can be defined broadly or in a limited manner, and can help differentiate the least effective actions from the most effective actions. For example, imagine that you are implementing a platform meant to facilitate social monitoring of the budget related to hospital infrastructure. You will not only want to know if the platform was implemented and if it is operating properly, but also if the potential users have been able to use it for the intended objective and if it has contributed to improvements in the quality of such infrastructure.

%

• Cost/benefit and cost comparison evaluations a) Cost/benefit - quantifies the benefits and costs of an activity and places them on the same matrix (often assigning a monetary value to the benefits). b) Cost comparison - differentiates the development and implementation costs of similar initiatives.

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

MODULE.9 Iteration to accomplish a living portal

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

The term “iteration” refers to the repetition of a process applied to the result of a previous application, typically as a means to obtain successive approximations closer to the solution of a problem. As seen in the initial image of the tutorial (below), the user-centered design (UCD) process is an iterative model in which the lessons learned from monitoring and evaluation allow the subsequent iteration to be conceptualized. Additionally, these iterations allow for a project’s scalability over time based on the initially established objectives and availability of human and financial resources for growth, as well as the maturity and interoperability characteristics of the associated financial information systems.

Application example For an example of communication on site iterations, consider the Open Budget portal, published in 2019 by the Chilean Ministry of Finance. In a label ribbon at the top part of the portal, it clearly indicated that it was a “beta” version and allowed users to consult a road map for subsequent phases (https://presupuestoabierto.gob.cl/roadmap). This recognized both the efforts that were underway to achieve the integration of data and the challenges that existed in producing a more comprehensive data collection. It also communicated the next steps to be undertaken.

PLANNING CONCEP TU

G

IT O R I N G

Prototype testing & refining

IN

ON

M

04

A LIZ

Evaluating

NCHING U LA

C R E ATI N G

05

SCALING

If there are specific iterations planned, informing users is a positive way of exercising transparency and represents good communication. Provide users with information on the scope of the publication’s current version as well as the plans and schedule for site improvement.

By understanding and being aware of the relationship between a publication and the process of iteration, it is possible to further recognize that not only does technology evolve, but that society evolves. Needs can change with time, even when dealing with the same audience. Thus, to accomplish a living and reactive fiscal transparency portal, successive iterations are required. By recognizing the new starting point, and taking into account the findings from the monitoring and evaluating processes, the project team can reapply the UCD’s implementation steps, parting from conceptualizing.

Bibliography • Babich, N. (2017). Putting Personas to Work in UX Design: What They Are and Why They’re Important. Adobe Blog. Available at: https://theblog.adobe.com/putting-personas-to-work-in-ux-design-what-they-are-and-why-theyre-i mportant/ [Accessed 11 Feb. 2019]. • Benyon, D. (2014). Designing interactive systems: a comprehensive guide to HCI and interaction design. 3rd ed. Harlow: Pearson. • Fiscaltransparency.net. (2012). Gift Principles – Global Initiative for Fiscal Transparency. [online] Available at: http://www.fiscaltransparency.net/giftprinciples/ [Accessed 12 Apr. 2019]. • Fiscaltransparency.net. (2018). Data on the Streets International Rally – Global Initiative for Fiscal Transparency. [online] Available at: http://www.fiscaltransparency.net/blog_open_public.php?IdToOpen=6323 [Accessed 20 Nov. 2018]. • Frascara, J. and Kneebone, P. (2005). Diseño gráfico y comunicación. Buenos Aires: Infinito. • Global Initiative for Fiscal Transparency and Ministry of Finance and Public Credit of Mexico (2015). Open Data Tutorial: Opening and promoting use of budget data. [online] Global Initiative for Fiscal Transparency. Available at: http://www.fiscaltransparency.net/blog_open_public.php?IdToOpen=6102 [Accessed Feb. 2019]. • Haas Lyons, S. (2017). Digital Engagement, Social Media & Public Participation. [online] International Association for Public Participation - iAP2. International Association for Public Participation - iAP2. Available at: https://iap2canada.ca/resources/Documents/Newsletter/2017_social_media_white_paper.pdf [Accessed 14 Jul. 2019]. • Heinz, E. (n.d.). Prototyping for Web I: Information, UX, and Paper Prototypes. SkillShare. Available at: https://www.skillshare.com/classes/Prototyping-for-Web-I-Information-UX-and-Paper-Prototypes/5 42676627. • Imf.org. (2019). IMF - Fiscal Transparency Code. [online] Available at: https://www.imf.org/external/np/fad/trans/ [Accessed 12 Apr. 2019]. • International Budget Partnership. (2017). Open Budget Survey | International Budget Partnership. [online] Available at: https://www.internationalbudget.org/open-budget-survey/ [Accessed 12 Apr. 2019]. • Jesse James Garrett (2011). The elements of user experience: user-centered design for the Web and beyond. Berkeley, Ca: New Riders.

% ON

• Morville, P. (2004). User Experience Design. [online] Semantic Studios. Available at: http://semanticstudios.com/user_experience_design/ [Accessed 2 Sep. 2019]. • Nielsen, J. (1993). Iterative User Interface Design. [online] Nielsen Norman Group. Nielsen Norman Group. Available at: https://www.nngroup.com/articles/iterative-design/ [Accessed 16 Jul. 2019].

07

• Norman, D.A. and Draper, S.W. (2017). User centered system design: new perspectives on human-computer interaction. [online] Boca Raton; London; New York Crc Press. Available at: https://dl.acm.org/citation.cfm?id=576915 [Accessed 9 Jun. 2019]. • OECD Best Practices for Budget Transparency. (2002). [online] Organisation for Economic Co-operation and Development - OECD, pp.7–14. Available at: http://www.oecd.org/governance/budgeting/Best%20Practices%20Budget%20Transparency%20%20complete%20with%20cover%20page.pdf [Accessed 12 Mar. 2019]. • Plainlanguage.gov. (2019). [online] Available at: https://www.plainlanguage.gov/ [Accessed 10 Mar. 2019]. • Potter, M.C., Wyble, B., Hagmann, C.E. and McCourt, E.S. (2013). Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, & Psychophysics, 76(2), pp.270–279. • de Renzio, P. and Mastruzzi, M. (2016). How Does Civil Society Use Budget Information? Mapping Fiscal Transparency Gaps and Needs in Developing Countries. [online] Fiscaltransparency.net. International Budget Partnership / The Global Initiative for Fiscal Transparency (GIFT). Available at: http://www.fiscaltransparency.net/eng/resource_open_public.php?IdToOpen=20161215147 [Accessed 12 Jan. 2019].

08

• Rivero del Paso, L. (2016). Transparencia Presupuestaria Evolución y desafíos. [online] Secretaría de Hacienda y Crédito Público de México. Available at: http://www.fiscaltransparency.net/eng/resource_open_public.php?IdToOpen=20160420204. • Romero León, J., de la Mora, D. and Ruiz, L. (2016). How are Governments Disclosing Fiscal Information Online? [online] International Budget Partnership. International Budget Partnership. Available at: https://www.internationalbudget.org/publications/digital-budgets-how-are-governments-disclosingfiscal-information-online/ [Accessed 12 Jan. 2019]. • Usability.gov (2019). Usability.gov. [online] Usability.gov. Available at: https://www.usability.gov/ [Accessed 22 Jun. 2019].

%

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

TUTORIAL on FISCAL TRANSPARENCY PORTALS

A USER-CENTERED DEVELOPMENT

BIBLIOGRAPHY

% ON

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY

Bibliography • Babich, N. (2017). Putting Personas to Work in UX Design: What They Are and Why They’re Important. Adobe Blog. Available at: https://theblog.adobe.com/putting-personas-to-work-in-ux-design-what-they-are-and-why-theyre-i mportant/ [Accessed 11 Feb. 2019]. • Benyon, D. (2014). Designing interactive systems: a comprehensive guide to HCI and interaction design. 3rd ed. Harlow: Pearson. • Fiscaltransparency.net. (2012). Gift Principles – Global Initiative for Fiscal Transparency. [online] Available at: http://www.fiscaltransparency.net/giftprinciples/ [Accessed 12 Apr. 2019]. • Fiscaltransparency.net. (2018). Data on the Streets International Rally – Global Initiative for Fiscal Transparency. [online] Available at: http://www.fiscaltransparency.net/blog_open_public.php?IdToOpen=6323 [Accessed 20 Nov. 2018]. • Frascara, J. and Kneebone, P. (2005). Diseño gráfico y comunicación. Buenos Aires: Infinito. • Global Initiative for Fiscal Transparency and Ministry of Finance and Public Credit of Mexico (2015). Open Data Tutorial: Opening and promoting use of budget data. [online] Global Initiative for Fiscal Transparency. Available at: http://www.fiscaltransparency.net/blog_open_public.php?IdToOpen=6102 [Accessed Feb. 2019]. • Haas Lyons, S. (2017). Digital Engagement, Social Media & Public Participation. [online] International Association for Public Participation - iAP2. International Association for Public Participation - iAP2. Available at: https://iap2canada.ca/resources/Documents/Newsletter/2017_social_media_white_paper.pdf [Accessed 14 Jul. 2019]. Tutorial on Fiscal Transparency Portals November 2019

Written and developed by: - Tarick Gracida, Technology and Communications Coordinator. [email protected] | @TarickGracida - Lorena Rivero del Paso, Manager for Cooperation and Technical Collaboration. [email protected] | @lorenardp With gratitude to Raquel Ferreira for their useful comments and suggestions. Translation to English: Sofía Leal de la Rosa Style correction: Erica L. Wood

Global Initiative for Fiscal Transparency www.fiscaltransparency.net | [email protected] / @fiscaltrans Network Director: Juan Pablo Guerrero | [email protected] / @jpga63

• Heinz, E. (n.d.). Prototyping for Web I: Information, UX, and Paper Prototypes. SkillShare. Available at: https://www.skillshare.com/classes/Prototyping-for-Web-I-Information-UX-and-Paper-Prototypes/5 42676627. • Imf.org. (2019). IMF - Fiscal Transparency Code. [online] Available at: https://www.imf.org/external/np/fad/trans/ [Accessed 12 Apr. 2019]. • International Budget Partnership. (2017). Open Budget Survey | International Budget Partnership. [online] Available at: https://www.internationalbudget.org/open-budget-survey/ [Accessed 12 Apr. 2019]. • Jesse James Garrett (2011). The elements of user experience: user-centered design for the Web and beyond. Berkeley, Ca: New Riders. • Morville, P. (2004). User Experience Design. [online] Semantic Studios. Available at: http://semanticstudios.com/user_experience_design/ [Accessed 2 Sep. 2019]. • Nielsen, J. (1993). Iterative User Interface Design. [online] Nielsen Norman Group. Nielsen Norman Group. Available at: https://www.nngroup.com/articles/iterative-design/ [Accessed 16 Jul. 2019].

07

• Norman, D.A. and Draper, S.W. (2017). User centered system design: new perspectives on human-computer interaction. [online] Boca Raton; London; New York Crc Press. Available at: https://dl.acm.org/citation.cfm?id=576915 [Accessed 9 Jun. 2019]. • OECD Best Practices for Budget Transparency. (2002). [online] Organisation for Economic Co-operation and Development - OECD, pp.7–14. Available at: http://www.oecd.org/governance/budgeting/Best%20Practices%20Budget%20Transparency%20%20complete%20with%20cover%20page.pdf [Accessed 12 Mar. 2019]. • Plainlanguage.gov. (2019). [online] Available at: https://www.plainlanguage.gov/ [Accessed 10 Mar. 2019]. • Potter, M.C., Wyble, B., Hagmann, C.E. and McCourt, E.S. (2013). Detecting meaning in RSVP at 13 ms per picture. Attention, Perception, & Psychophysics, 76(2), pp.270–279. • de Renzio, P. and Mastruzzi, M. (2016). How Does Civil Society Use Budget Information? Mapping Fiscal Transparency Gaps and Needs in Developing Countries. [online] Fiscaltransparency.net. International Budget Partnership / The Global Initiative for Fiscal Transparency (GIFT). Available at: http://www.fiscaltransparency.net/eng/resource_open_public.php?IdToOpen=20161215147 [Accessed 12 Jan. 2019].

08

• Rivero del Paso, L. (2016). Transparencia Presupuestaria Evolución y desafíos. [online] Secretaría de Hacienda y Crédito Público de México. Available at: http://www.fiscaltransparency.net/eng/resource_open_public.php?IdToOpen=20160420204. • Romero León, J., de la Mora, D. and Ruiz, L. (2016). How are Governments Disclosing Fiscal Information Online? [online] International Budget Partnership. International Budget Partnership. Available at: https://www.internationalbudget.org/publications/digital-budgets-how-are-governments-disclosingfiscal-information-online/ [Accessed 12 Jan. 2019]. • Usability.gov (2019). Usability.gov. [online] Usability.gov. Available at: https://www.usability.gov/ [Accessed 22 Jun. 2019].

%

GLOBAL INITIATIVE FOR

FISCAL TRANSPARENCY