Some time ago, I surveyed all national parliaments in the world to see whether they record and publish results of plenary voting. In this post, I look at how exactly parliamentary voting data is provided. I also collected information about as much parliamentary monitoring organizations (PMOs) as I could find and to see whether and how they help open voting data up.
My sample includes 283 legislative chambers from 200 jurisdictions, mostly UN member states and some other territories (e.g. Taiwan, Hong Kong). There are two nation states included that no longer exist but voting results from their parliaments are still around: Czechslovakia (1991-1992) and the Fourth French Republic (1946-1958).
A chart below shows that knowing how your MPs decide is sadly still relatively rare worldwide. Only 90 legislative chambers publish at least some voting results. Moreover, many of these chambers publish results of very few votes. For example, about 20 votes per year are recorded in German Bundestaag while the number is well over 2,000 in the Czech Chamber of Deputies where every vote is taken as a roll call by default. Transparency of voting is generally lower in Africa and Asia and among non-democratic countries. But even in some old democracies, almost no voting data exist (e.g. in Austria, France, New Zealand or the Netherlands).
The next chart shows distribution of legislative chambers according to formats in which voting results are published. The question of formats is crucial since it directly determines whether and how easily can voting results be searched and re-used by media, PMOs, academics and other stakeholders. In the chart, formats are ordered from the most closed (photos) to the most open (structured data formats such as XML, JSON etc.).
Some legislative chambers provide data in multiple formats. I only include such cases once in the most open category. There are two types of PDF formats in the chart. The difference is in machine readability - it is much easier to mine data out of so called “native PDFs” than from scanned documents. Formats of voting data are visualized in the following map (red – voting results not published, purple – PDFs, orange – text files, yellow – HTML, cyan – tables, green – XML, JSON etc.). In case of bicameral parliaments, the most open data format available in any chamber is displayed.
In many cases, results of votes are reported in hansards or similar notes, minutes or transcripts of plenary sessions. This also greatly reduces usability of data since individual votes must usually be manually found and it is sometimes nearly impossible to systematically extract voting results out of these documents. In most legislative chambers, however, voting results are published separately from hansards.
Only very few legislative chambers provide bulk download or API (application programming interface) options. Here are some notable examples:
- The most open parliament is probably the Swedish Riksdag which provides API and bulk files in HTML, CSV, XML, JSON and SQL formats,
- Other countries with parliaments providing API access include Norway, Switzerland and Georgia (development of this API was actually supported by NDI),
- Czech Chamber of Deputies provides CSV files aggregated by parliamentary term,
- Brazilian Chamber of Deputies provides DBF files aggregate by parliamentary term,
- Spanish Chamber of Deputies provides XML files aggregated by session,
- Bulgarian National Assembly provides XLS files aggregated by session,
- Swiss Parliament allows users to export multiple votes to CSV or XML files; the number of votes to export is, however, very limited.
Official parliamentary websites and databases are, however, not the only sources of voting data. Many amazing PMOs all around the world scrape and sometimes even manually record results of voting and republish this data in more useful ways. A map below includes 253 PMOs from 87 countries (red – no PMOs to be found, yellow – PMOs that do not publish voting data, cyan – PMOs that do publish voting data, green – PMOs provide API or bulk download).
Here are some honourable mentions of PMOs doing amazing work with voting data:
- Probably the most zealous PMO in terms of the amount of voting data opened is KohoVolit.eu. Its founder Michal Škop wrote scrappers for Brazilian, Chilean, Spanish, Czech, Slovak and former Czechoslovak parliaments. They are currently not being maintained due to ScraperWiki reducing its free services. The data can, however, still be downloaded as an SQL database.
- PMOs that have built APIs of voting data include e.g. KohoVolit.eu (Czech, Slovak and former Czechoslovak parliament), It’s Your Parliament (European parliament), Congreso Visible (Colombia), Hvem stemmer hvad (Denmark), Biomi.org (Finland), Open Knesset (Israel), Sunlight Foundation (US Congress, US states) or Fundacja ePaństwo (Poland),
- My personal heroine Amira Yahyaoui and her team of volunteers from Al Bawsala have been manually collecting every voting record in the parliament of Tunisia since the Arab spring revolution in the country,
- Public Whip provides bulk voting data from the British parliament and Sunlight Foundation provides bulk voting data from the US states. US Congress roll call data since 1789 is available for download on the VoteView website maintained by Keith Poole, a political scientist studying the US legislative process.
- Tony Bowden and KohoVolit.eu are currently working on a Poplus component called VoteIt based on Popolo data standard and – more excitingly – on a global repository of voting data (imagine OpenStates project expanded to the whole world!).
Keep in mind that this survey is intended as a conversation starter and not as a definitive picture of the current state of affairs. There is only so much a lone researcher can accomplish, I therefore expect to find many false negatives. I especially suspect that I missed some parliamentary API since they are sometimes very well hidden. As always, any feedback is greatly appreciated!