Privacy-preserving data publishing: attacks, countermeasures, and risk analysis

Health data, social networks, electricity consumption... Vast
quantities of personal data are collected today by private companies
or public organizations. Various legal, monetary, or visibility
incentives push data holders to envision sharing anonymized versions
of the collected datasets or machine learning models trained over
them. Indeed, sharing data or models at large, e.g., as open data, is
expected to bring strong benefits (strengthening, e.g., scientific
studies, innovation, public policies).

Privacy-preserving data publishing techniques are dedicated to the
sanitization of personal data before sharing in order to enjoy
(hopefully strong enough) privacy guarantees. Substantial progress
has been made during the last two decades, leading to a wealth of
privacy models and algorithms, the differential privacy family being
most prominent today [2, 6]. However, the rich body of work makes it
hard for non expert users to understand clearly the privacy
implications of choosing a specific privacy model or algorithm, or of
selecting suitable values for their privacy parameters.

In parallel to privacy-preserving data publishing techniques, the
study of attacks on sanitized data or models has grown tremendously
[7] (see, e.g., membership inference attacks [4, 8]). Attacks on
sanitized data and models have been used for auditing real-life
implementations of differentially private algorithms [5]. However,
although privacy auditing can help obtain empirical estimates of the
privacy guarantees offered by an algorithm and its privacy parameters,
using privacy auditing techniques for risk analyzis is non trivial
because of the large number of possible attackers and of the large
costs of each attack.

The main goal of this PhD thesis is to make use of today’s attacks on
sanitized data or models in order to allow the data holder to analyze
the risks related to one or more publications. This will require to
formalize the attackers (e.g., privacy games, formal models dedicated
to representing and analyzing security issues [3]), to structure the
space of attackers (e.g., generalization/specialization of attackers,
implications), and to design algorithms for exploring efficiently the
resulting space (e.g., multi-criteria optimization algorithms). In
addition to the core tasks of the project, the successful candidate
will contribute to the organisation of competitions where the privacy
guarantees of sanitization algorithms are challenged [1].