Global survey: Parliamentary voting data remains stuck in PDFs and hansards

Posted July 21, 2014 at 8:09am by kamilopblog

Some time ago, I surveyed all national parliaments in the world to see whether they record and publish results of plenary voting. In this post, I look at how exactly parliamentary voting data is provided. I also collected information about as much parliamentary monitoring organizations (PMOs) as I could find and to see whether and how they help open voting data up.

My sample includes 283 legislative chambers from 200 jurisdictions, mostly UN member states and some other territories (e.g. Taiwan, Hong Kong). There are two nation states included that no longer exist but voting results from their parliaments are still around: Czechslovakia (1991-1992) and the Fourth French Republic (1946-1958).

A chart below shows that knowing how your MPs decide is sadly still relatively rare worldwide. Only 90 legislative chambers publish at least some voting results. Moreover, many of these chambers publish results of very few votes. For example, about 20 votes per year are recorded in German Bundestaag while the number is well over 2,000 in the Czech Chamber of Deputies where every vote is taken as a roll call by default. Transparency of voting is generally lower in Africa and Asia and among non-democratic countries. But even in some old democracies, almost no voting data exist (e.g. in Austria, France, New Zealand or the Netherlands).

The next chart shows distribution of legislative chambers according to formats in which voting results are published. The question of formats is crucial since it directly determines whether and how easily can voting results be searched and re-used by media, PMOs, academics and other stakeholders. In the chart, formats are ordered from the most closed (photos) to the most open (structured data formats such as XML, JSON etc.).

Some legislative chambers provide data in multiple formats. I only include such cases once in the most open category. There are two types of PDF formats in the chart. The difference is in machine readability - it is much easier to mine data out of so called “native PDFs” than from scanned documents. Formats of voting data are visualized in the following map (red – voting results not published, purple – PDFs, orange – text files, yellow – HTML, cyan – tables, green – XML, JSON etc.). In case of bicameral parliaments, the most open data format available in any chamber is displayed.

In many cases, results of votes are reported in hansards or similar notes, minutes or transcripts of plenary sessions. This also greatly reduces usability of data since individual votes must usually be manually found and it is sometimes nearly impossible to systematically extract voting results out of these documents. In most legislative chambers, however, voting results are published separately from hansards.

Only very few legislative chambers provide bulk download or API (application programming interface) options. Here are some notable examples:

  • The most open parliament is probably the Swedish Riksdag which provides API and bulk files in HTML, CSV, XML, JSON and SQL formats,
  • Other countries with parliaments providing API access include Norway, Switzerland and Georgia (development of this API was actually supported by NDI),
  • Czech Chamber of Deputies provides CSV files aggregated by parliamentary term,
  • Brazilian Chamber of Deputies provides DBF files aggregate by parliamentary term,
  • Spanish Chamber of Deputies provides XML files aggregated by session,
  • Bulgarian National Assembly provides XLS files aggregated by session,
  • Swiss Parliament allows users to export multiple votes to CSV or XML files; the number of votes to export is, however, very limited.

Official parliamentary websites and databases are, however, not the only sources of voting data. Many amazing PMOs all around the world scrape and sometimes even manually record results of voting and republish this data in more useful ways. A map below includes 253 PMOs from 87 countries (red – no PMOs to be found, yellow – PMOs that do not publish voting data, cyan – PMOs that do publish voting data, green – PMOs provide API or bulk download).

Here are some honourable mentions of PMOs doing amazing work with voting data:

Keep in mind that this survey is intended as a conversation starter and not as a definitive picture of the current state of affairs. There is only so much a lone researcher can accomplish, I therefore expect to find many false negatives. I especially suspect that I missed some parliamentary API since they are sometimes very well hidden. As always, any feedback is greatly appreciated!

Kamil Gregor is a data analyst with and Masaryk University in Brno, Czech Republic.

Going to OKFest? Help us connect and share knowledge across the OpenGov community

Posted July 11, 2014 at 7:40am by gregbrownm

This post was co-authored by Mor Rubinstein and Lindsay Ferris

There is a growing consensus in the international open government movement that more effective networking and better information sharing could strengthen the community and contribute to real political change. If the open government community can engage in more meaningful communication about the projects we are working on, the policies we are pushing for, and the tools we are using, we can better support advocates and activists in their push for greater political openness.

Several sessions have been organized at this year’s OKFest to address the issue of improved knowledge sharing and to find a way forward. We would like to bring two of these sessions to your attention — and encourage you to participate!

With participation from a diverse number of organizations, a two-part discussion is being organized to gather ideas on these important issues. These sessions are being co-organized by a diverse array of actors, including Open Knowledge, Sunlight Foundation, mySociety,, ePantswo Foundation, World Bank, National Democratic Institute, and others. We have designed these sessions to gather feedback and solicit ideas from the international community; so, it seems appropriate to present these two discussions as a series of questions, rather than as a series of statements. The first session will address the following questions: Who are we? How does the international opengov community effectively map and connect the organizations and initiatives that make up the community? The second session will ask: How do we share knowledge? How can we use this knowledge to strengthen efforts to generate political change locally? We look forward to hearing your thoughts on how we can address each of these questions at OKFest.

The first session, Open Data Communities, will consider how to connect the global open government data community. Often, efforts to map the international community have been fraught with problems, namely outdated or incomplete lists, unstandardized information and limited funding. Following conversations at Transparency Camp (Sunlight Foundation’s annual opengov gathering), discussion sprang up about the possibility of creating a living platform that would provide an overview of the open government community and allow stakeholders to connect with other actors. Some work has been done towards this goal and we’re excited to share some of that with you in Berlin. Community buy-in and participation are critical to the success of this project and we need your help to make it work.  

The second session will consider how we can more effectively share knowledge, tools, and good practices to create political reform. At the national level, open government organizations or activities can sometimes be viewed as marginal actors, particularly when they operate in a narrow sector, such as parliamentary openness or open contracting. Yet, when multiple actors in this space coalesce, either internationally or domestically, across sectors on specific reform issues, the political leverage of this community can be strengthened. In this session, we will explore opportunities for improving collaboration in collecting open data sets relating to political information, sharing tools for analyzing and visualizing this data, and sharing knowledge and best practices —- specifically with a view as to how more effective networking and knowledge sharing can drive openness reforms.

We are thrilled to have two one-hour sessions to discuss these issues. At a minimum, we hope that these sessions will generate energy and excitement about what the opengov community can achieve if we more effectively share information. We encourage you to participate and look forward to seeing you in Berlin!

Case Study #12: Vouliwatch

Posted July 6, 2014 at 4:02pm by gregbrownm

Vouliwatch logo.jpg

Organization: Vouliwatch

Country: Greece

Government Level: National

Overview: Vouliwatch is a digital platform that engages Greek citizens with legislative politics and grants them with the opportunity to communicate, evaluate and hold elected representatives in the Greek and the European Parliament (MPs & MEPs) accountable.

Background: Vouliwatch was set up in March 2014. It is an independent, non for profit organisation aiming to promote public dialogue, knowledge, political participation and accountability between citizens and politicians.

The mission of the project is to encourage Greek citizens to engage in politics, as well as to increase accountability and transparency in the Greek political system. To achieve this, the Vouliwatch team will be cooperating with politicians and civil society in order to promote a culture of dialogue and understanding.

Vouliwatch incubates and cultivates a synergistic democratic culture that inspires institutional and technological innovation. Therefore, we are looking for new ideas, concepts and human networks to disseminate and improve our project.

Click here to read more.

Monitoring MPs attendance? Compare 33 countries!

Posted July 2, 2014 at 9:31am by kamilopblog

Is your organization monitoring attendance of MPs? Compare attendance rates in your country with rates from as much as 33 parliaments!, a Czech and Slovak parliamentary monitoring organization has been publishing regular reports on attendance rates in Czech and Slovak parliaments. This is one of the most common activity that PMOs do. Although we had some success (e.g. when we scraped an online calendar of an MP with low attendance and were thus able to account for 50 % of his absence), these reports became rather repetitive. Attendance is usually very stable over time so the topic could cease to be interesting for journalists and citizens after a while.

When we were looking for ways how to spice it up it occurred to us that it would be great if we were able to compare attendance of our MPs with their colleagues from other parliaments. Journalists love such international comparisons and citizens are often very receptive to reports of their country being in some respect worse then its neighbours. So if your organization also informs about MPs attendance I strongly suggest to try the same.

Thanks to new developments in standardizing and gathering voting data (especially by a new federation of expert and organizations called Poplus) it has became possible for me to create a rather comprehensive archive of voting results in national and regional legislatures. Some datasets come from national parliaments that provide data dumps (e.g. BulgarianCzechGerman), some from parliamentary monitoring organizations and experts that scrape voting data (e.g. Canadian Open North, Israeli Open Knesset, Norwegian Holder de ords, Czech and Slovak and some from academicians (e.g. John CareyKeith Poole and Simon Hix, Abdul Noury and Gerard Roland).

The chart below visualizes attendance rates in 33 national parliaments. These are average rates taken across all MPs and all votes in a given period of time. In bicameral parliaments (e.g. Brazil, Canada, USA), these are rates for the lower chamber. It should be noted that in many parliaments, results of only some (and sometimes only very few) votes are published so the rates can be skewed. This is especially the case of Germany, Italy or Philippines where only final votes on bill proposals are recorded. I believe, however, that the chart is still useful for a rough comparison.

As far as I know, this is the most comprehensive comparison to date. I am expecting to receive more voting data soon so the chart will be updated with rates from other countries, namely: former Czechoslovakia (1990-1992), Honduras, Serbia, Switzerland. I would very much appreciate any feedback on these rates, especially from experts on parliaments included. For example, I was rather surprised to find out that Scandinavian parliaments exhibit relatively low attendance and I have no idea why. Also, if you happen to have any raw voting data or attendance rates let me know!

Kamil Gregor is a data analyst with, a Czech and Slovak parliamentary monitoring organization, and Masaryk University in Brno, Czech Republic.

Monitoring parliamentary openness on the sub-national level: Czech experience

Posted June 27, 2014 at 10:37am by kamilopblog

In almost two years since the Declaration on Parliamentary Openness was drafted, many parliamentary monitoring organizations have realized its strength as a guideline for benchmarking openness of parliamentary data in various national parliaments and some of them have developed methodologies of capturing it. There are already comparative studies ranking selected parliaments according to their adherence to at least some articles of the Declaration.

The most prominent examples include a comparative study covering several Latin American congresses by the Latin American Network for Legislative Transparency that actually precedes the Declaration. Data availability of the Turkish and several Balkan parliaments was surveyed by a Serbian parliamentary monitoring organization Center for Research, Transparency and Accountability. Eastern European parliaments were also covered by a survey of the National Democratic Institute. And last but not least, a more tech-oriented methodology of data openness monitoring was developed by the Sunlight Foundation and applied to rank the US state legislatures.

Until today, however, there has been no attempt to measure parliamentary data openness on lower levels of government. At the same time, anecdotal evidence from all over the world seems to suggest that various regional and municipal parliaments and representative assemblies tend to be far less open than national parliaments., a Czech and Slovak parliamentary monitoring organization, has recently published a brand new methodology of measuring parliamentary data openness based on the Declaration and applied it to the 14 Regional Assemblies in the Czech Republic. The Czech Regions operate on the second level of government. Their population varies between 300,000 and 1,200,000 in a country of some 10 million and their combined annual budgets correspond to about 12 % of total public sector expenditures. The capital of Prague is one of the Regions.

Click here to read more.

La agenda “extraparlamentaria” de los grupos del Congreso

Posted June 27, 2014 at 9:03am by victoriahuertas

  • Hemos examinado si los grupos parlamentarios publican agendas de sus diputados
  • Los grupos se reúnen con lobbies, grupos de presión, empresas y sociedad civil
  • Solo UPyD publica los encuentros fuera de la actividad “oficial” parlamentaria, como con BlaBlaCar, Cairn Energy o Greenpeace
  • El PP publica entrevistas y alguna asistencia a actos y el PSOE la actividad parlamentaria

Conocer el día a día al completo de los 350 diputados y diputadas puede ser una misión casi imposible en la actualidad, ya que la gran mayoría de los parlamentarios no cuenta con una agenda pública,pero sí existen algunas excepciones, tanto de diputados como de grupos parlamentarios.

Reunión de UPyD con la Federación de Organizaciones en favor de Personas con Discapacidad Intelectual
Reunión de UPyD con la Federación de Organizaciones en favor de Personas con Discapacidad Intelectual. Imagen: FEAPS/Flickr

En este post nos centramos en las agendas de los grupos, más sencillas de localizar que las individuales, ya que de los siete grupos que hay en el Congreso, cuatro tienen página web (PSOECiUUPyD y PNV) y el PP tiene una cuenta de Facebook bastante activa.

En los casos de los grupos de del Grupo Mixto y de Izquierda Plural, al estar formados por tantos partidos, no tienen una web conjunta. Sin embargo, en este último caso cuentan con una web de apoyo, Once diputados, que resume su actividad parlamentaria en un boletín.

Click here to read more.

GitLaw: How The Law Factory turns the French parliamentary process into 300 version-controlled Open Data visualizations

Posted June 25, 2014 at 9:06am by regardsopblog

Law is Code!

Over the last few years, a number of people have explored the idea of inverting Lawrence Lessig’s metaphor “code is law”, looking at the evolution of laws through the lens of coding tools. The parliamentary process is indeed so similar to a collaborative software development workflow that it is only natural to try and use a version control tool such as git to track individual legislative changes.

The analogy between both processes is deep: in each case, there is a group of people collaborating on a textual artifact (bill or program source code), proposing changes (amendments or patches), adopting or rejecting them (through votes or pull requests), and iterating until a stable, public version is made available (by promulgation or release). This new paradigm to think about legislation paves the way for new, innovative approaches of law-tracking. Some exciting work has already been made, most notably in Germany: the BundesGit project invites citizens to propose their own legal modifications as “pull requests”, and Gregor Aisch produced an unprecedented visualization of modifications to one law over 40 years of amendments.

Click here to read more.