Estimating Effectiveness of Online Geographically-based Advertising Campaigns

Abstract

The effectiveness of online geographically-based advertising campaigns is estimated experimentally using a randomised experimental approach called a geo-experiment. In these experiments, a region of interest is partitioned into geographical-targeting areas called geos. The experiments are conducted in two distinct time periods where in the first time period there is no difference in advertising campaigns between geos, whereas during the second time period the campaigns for some selected geos are modified. The main concern is, which geos should be assigned to the treatment condition to serve the modified advertising campaigns during the second time period? It is a simple question with a not so simple answer in reality, especially in the presence of unobserved heterogeneity structure within geos. The issue therefore is to design a robust advertising campaigns which permits estimation of the effectiveness of the campaigns using geo-experiments. In this thesis, a conceptual model of geo-experiments is presented to improve our understanding of the potential impact of hidden heterogeneity on estimating the effectiveness of advertising campaigns. A theoretical framework based on theory of maximum likelihood estimation of misspecified model and Kullback-Leibler divergence is developed to study the implications of unobserved heterogeneity for inferences about estimated effects for geo-experiments. An important part of the framework is a proxy model linking the fitted model, with homogeneity structure within geos, and the assumed truth which includes unobserved heterogeneity. The theoretical framework plays a key role in approximating the behaviour of the estimated fitted model parameters. This saves having to do expensive Monte Carlo simulation all the time. The accuracy of the theoretical approximation is investigated for different campaign design strategies across different truth instances. The results reveal the advantage of design strategies based on unobserved covariates, such as social-grades, in reducing the variability of the approximation error and that designs based on spatial proximity may achieve some of the same benefit. Nonetheless, for the more complex truth instances investigated, none of the design strategies considered succeeds in avoiding bias due to unobserved heterogeneity.