6. Data providers

6.1. Overview

pandaSDMX supports a number of data providers out of the box. Each data provider is configured by an item in agencies.json in the package root. Data providers are identified by an uppercased string such as “ECB” or “OECD”. For each pre-configured data provider, agencies.json contains the URL and name of the SDMX API and potentially some additional metadata about the provider’s web API. The configuration information about data providers is stored in the dict-type class attribute _agencies of Request. Other data providers can be configured by passing a suitable json-file to the pandasdmx.api.Request.add_agency() which will be used to update the dict storing the agency configuration.

6.2. Pre-configured data providers

This section describes the data providers supported out of the box. The most salient distinction between data providers derives from the supported API: While OECD and Australian Bureau of Statistics (ABS) are only supported with regards to their SDMX-JSON APIs, all others send SDMX-ML messages. SDMX-JSON is confined to data messages. Hence, pandaSDMX features relating to structural metadata are unavailable when making requests to OECD or ABS.

6.2.1. Australian Bureau of Statistics (ABS)

SDMX-JSON only. Start by browsing the website to retrieve the dataflow you’re interested in. Then try to fine-tune a planned data request by providing a valid key (= selection of series from the dataset). No automatic validation can be performed as structural metadata is unavailable.

6.2.2. Eurostat

  • SDMXML-based API.
  • thousands of dataflows on a wide range of topics.
  • No categorisations available.
  • Long response times are reported. Increase the timeout attribute to avoid timeout exceptions.

6.2.3. European Central Bank (ECB)

  • SDMXML-based API
  • supports categorisations of data-flows
  • supports preview_data and series-key based key validation
  • in general short response times

6.2.5. International Monetary Fund (IMF) - SDMX Central only

  • SDMXML-based API
  • subset of the data available on http://data.imf.org
    • full IMF data access would require SOAP API support. This feature needs to be added.

6.2.6. Organisation for Economic Cooperation and Development (OECD)

SDMX-JSON only. Start by browsing the website to retrieve the dataflow you’re interested in. Then try to fine-tune a planned data request by providing a valid key (= selection of series from the dataset). No automatic validation can be performed as structural metadata is unavailable.

6.2.7. United Nations Statistics Division (UNSD)

  • SDMXML-based API
  • supports preview_data and series-key based key validation

6.2.8. UNESCO

  • free registration required
  • subscription key must be provided either as parameter or HTTP-header with each request
  • SDMXML-based API