Master’s Thesis: Spatial Regression with Deep Learning and Attention

I am excited to share that I have successfully completed and submitted my Master’s thesis, fulfilling the final requirement for my degree in Machine Learning and Data Science at Imperial College London.

At a high-level, the project focused on using Deep Learning to predict property prices (with TensorFlow). I was supervised by the inspiring, highly knowledgable and supportive Dr. J Martin. The thesis is titled:

Predicting Property Prices in New York City: Deep Learning with Attention for Spatial Regression on Areal Data.

Below, you’ll find the abstract. If you’re interested in reading the full text (~10,000 words), feel free to reach out via email — I’d be happy to share a copy!

Abstract

        Accurately predicting property prices is a challenging but important problem. An accurate understanding of property prices is important both as guidance to individuals (buyers gauging a fair price, and sellers evaluating their properties), and as signals of broader economic trends.
        This thesis approaches the task of property price prediction as a spatial regression problem. Specifically, we investigate the efficacy of deep learning with attention mechanisms in the task of spatial regression, as applied to a city-scale areal dataset. This methodological exploration is conducted with particular application to predicting residential property prices in New York City (NYC), at the city block level. Residential property prices in NYC are among the most expensive in the world, and so accurate property price prediction has significant financial implications.
        A comprehensive exploratory data analysis was undertaken to uncover the complexities of the NYC residential property market. Through the use of modern deep learning techniques, and statistical methods including Bayesian optimisation and spatial cross-validation, multiple models were developed and rigorously evaluated, and their performances compared.
        We propose the use of a deep learning model that incorporates an attention mechanism. Empirical results show that this model outperforms traditional approaches such as Geographically Weighted Regression, and demonstrates a more modest performance improvement over deep learning models without attention. Furthermore, inspection of the attention weights of the model offers possible additional interpretability and insight.
        We propose several future directions in this active research area, including the use of multi-head attention and alternative positional encodings, which may offer further improvements in predictive performance.

Keywords: spatial data, regression, deep learning, attention, property prices