Online Microdata Access Brief

In the past official statistics providers have given external researchers and analysts limited and tightly controlled access to the microdata from their censuses and surveys because of their duty to protect the privacy of their survey respondents. Typically this controlled access takes the form of in-house or remotely accessed data laboratories or research centres, or the provision of pre-confidentialized sample files. All of these scenarios typically involve a statistics provider’s staff having to do some form of manual review and vetting of the information generated in response to a data query before it is delivered back to the researcher.

Space-Time Research has a new, ground-breaking online application that enables official statistics providers to give external researchers and the general public much easier and faster access to microdata. This solution, already being used by the Australian Bureau of Statistics (ABS) for their 2006 Census and provides users with self-service application. This application lets users submit ad-hoc queries against microdata datasets and then the results are confidentialized “on the fly” before being returned to the user’s browser. The diagram below describes the process:

online-microdata

  1. The web application is accessed via the internet and your web browser. You log into the web application and given access to your authorized datasets.
  2. You create the query using the web application. You can choose combinations of fields at different levels of classification or geography and submit the query for execution. Also you can create your own custom fields (or recodes) which group together specific field values.
  3. The web application processes the query, generating a table of results which is then confidentialized “on the fly”. This uses a perturbation algorithm that minimizes statistical distortion while still giving a high level of disclosure control.
  4. The confidentialized records are then returned to your web browser in combination with metadata that helps the you correctly interpret the results.
  5. You can display the results as tables, thematic maps, or charts, as well as being able to generate extracts in formats such as Mircosoft Excel and SDMX. Also you can save the definition of the query for re-running at a later date.

    This sophisticated web application was developed by Space-Time Research in conjunction with the Australian Bureau of Statistics (ABS) and represents the latest version of our SuperWEB product line. It is the platform that underpins the ABS’ two main public-facing census applications, CDATA Online and TableBuilder which provides access to the 2006 Australian Census containing records for 20 million individuals and 7 million households.

    The Online Microdata Access solution helps official statistics providers deliver a better service to their user communities and, at the same time, reduces their costs and improves staff productivity:

    • Improved service – Our solution provides a powerful, fast and flexible way for users to generate ad-hoc cross-tabulations, charts, and maps from microdata datasets that can contain many millions of records. Users are able to select from all variables and all levels of detail within the dataset to get the exact output they are after.
    • Higher quality, confidentialized information – using “on the fly” disclosure controls to suppress or perturb data, our application can confidentialize a query’s results before they are displayed in a user’s browser or downloaded to a user’s workstation. At the same time, important metadata is incorporated into the output to help users correctly interpret the information they have generated;
    • Reduced costs and improved productivity – by allowing users to directly find answers to many of their questions, a statistics agency can cater for an increasing number of users with minimal increase in support staff and costs. It also allows a provider’s customer service team to focus their attention on the more complex and specialized requests they will still receive from researchers and analysts.

    Contact us

    To find out how our solutions meet your requirements, please contact us.