The developing convergence of Artificial Intelligence and GIScience has raised a concern on the emergence of deep fake geography and its potentials in transforming human perception of the geographic world. Situating fake geography under the context of modern cartography and GIScience, this paper presents an empirical study to dissect the algorithmic mechanism of falsifying satellite images with non-existent landscape features. To demonstrate our pioneering attempt at deep fake detection, a robust approach is then proposed and evaluated. Our proactive study warns of the emergence and proliferation of deep fakes in geography just as “lies” in maps. We suggest timely detections of deep fakes in geospatial data and proper coping strategies when necessary. More importantly, it is encouraged to cultivate a critical geospatial data literacy and thus to understand the multi-faceted impacts of deep fake geography on individuals and human society.
Geospatial Artificial Intelligence (GeoAI), for its potential to provide groundbreaking capabilities to leverage GIScience with a series of Artificial Intelligence (AI) advances, such as natural language process, unstructured data classification, computer vision, or map style transfer (Hu et al., 2019; Kamel Boulos et al., 2019; Sirosh, 2018), has been hailed by both industry pundits (e.g. the seamless integration of deep learning functions in ArcGIS Pro, GeoAI solution launched on Microsoft Azure) and scholars alike (e.g. a series of GeoAI sessions in AAG annual conferences 2018, 2019, and 2020, Critical GeoAI session in AAG annual conference 2021, GeoAI workshops at ACM SIGSPATIAL 2017, 2018, and 2019, special issues on GeoAI sponsored by International Journal of Geographical Information Science and International Journal of Geo-Information). Such a wide applause of GeoAI is not the first time when GIS practitioners paid close attention to the use of AI in improving our capacity of understanding various geographical phenomena though; similar efforts can be traced back to mid-1980s (Couclelis, 1986; Estes et al., 1986; Nystuen, 1984; Smith, 1984). AI was a major driving force to form the subfields like automated geography (Dobson, 1983) and GeoComputation (Openshaw & Abrahart, 1996), which later became significant components of today’s prosperous GIScience. This early wave of AI in GIScience was well documented in Openshaw’s book “Artificial Intelligence in Geography” (Openshaw & Openshaw, 1997).
Besides the above-mentioned technical merits brought by AI, scholars have also witnessed problematic and unexpected implications of the convergence of AI and GIScience, such as fabricated GPS signals (Tippenhauer et al., 2011), fake locational information on social media (Zhao & Sui, 2017), simulated trajectories of online game bots (Pao et al., 2010), and fake photos of geographical environments (Isola et al., 2017). Even so, deep fake, as a problematic use of AI, has not widely proliferated in GIScience yet. Deep fake is often referred to as the deceptive and/or misleading synthetic media (e.g. image, audio, or video) that are created by AI. The deep fakes of politician speech and celebrity pornography spreading on social media that have received wide public attention in recent years. It has been regarded as a serious threat to individual privacy and national security (Chesney & Citron, 2019; Sayler & Harris, 2019), and has thus incurred responses from both industry and government to restrain its use. Big tech companies, including Amazon, Facebook and Microsoft, have jointly launched a deep fake detection challenge (DFDC, 2019), and Microsoft published a tool to identify artificially manipulated media (Burt, 2020). The proliferating misuse of AI also brought up serious concerns with the appearance of deep fakes in geography. For example, the automation lead at the National Geospatial-Intelligence Agency (NGA, known as the National Imagery and Mapping Agency from 1996–2003), a combat support agency under the United States Department of Defense and a member of the United States Intelligence, openly unveiled that AI was used to manipulate scenes and pixels to create artifacts on satellite images for malicious purposes (Tucker, 2019). Due to the often-sensitive nature of deep fake satellite imagery in similar settings as such, we could not get convenient and safe access to existing deep fake satellite images for this study and publication. Even though, we cannot ignore the appearance, or underestimate the development, of deep fake in satellite images or other types of geospatial data.
While many GIS practitioners have been celebrating the technical merits of deep learning and other types of AI for geographical problem solving, few have publicly recognized or criticized the potential threats of deep fake to the field of geography or beyond. Therefore, we would like to take the lead to explore the potential influences of deep fake on geospatial data and GIScience. Indeed, the emergence of deep fakes in GIScience is inevitable just as “lies” are essential in maps. As Monmonier (1991, p. 1) argued, “Not only is it easy to lie with maps, it’s essential … To present a useful and truthful picture, an accurate map must tell white lies.” Therefore, expecting the proliferating deep fakes of geospatial data, it is necessary to develop proper coping strategies and critically analyze their complicated social implications. Thus, in the remaining sections of this paper, we review the development of fake geography since before AI and introduce the basic technical details of deep fake in relevance to geography today. Then, we detail a case study of fake satellite images of Tacoma, Washington in order to closely examine the algorithmic mechanism of deep fake techniques in simulating fake satellite images – a primary type of geospatial data. Next, we introduce a feasible detecting approach to assessing the authenticity of a satellite image. We conclude this paper with a summary of our research findings and a critical discussion of “fake” from a broader humanistic geography perspective.
Although the term “Fake Geography” first appeared to describe an AI-generated fake digital geographical environment and warn us of its detrimental effects (Maclenan, 2018), its theoretical connotation and potential significance for geography is far more profound and broader. We could trace the origin of fake geography all the way back to the false or mythological interpretation of the world that could be illustrated from some ancient maps such as the Babylonian cuneiform map in the 5th century B.C. However, in this paper we situated fake geography in the context of modern science and technology. In doing so, we realized the importance of the correspondence theory of truth as epistemological guidance on the determination of what is true and its opposites (David, 2016). This theory is premised on a clear binary relationship between an object and its measurement: if the measurement is in correspondence with the object, a truth is thus established and the object can be represented by the measurement. For those measurements that are not deemed as “truth,” various terms have been used, such as “error,” “false,” “outlier” or “anomaly” that are often used to indicate inconsistent measurements in scientific research, and also “lie,” “fake,” “misinformation” or “disinformation” that are commonly used in public media and political debates to describe a deliberately generated inconsistent representation.
Monmonier is one of the first geographers whose work can be enlightening for today’s debates on fake geography. In his famous book “How to lie with maps,” a variety of ways in which maps (or geospatial data) distortedly represent the real world have been systematically explored (Monmonier, 1991). Early fake geographies would also include, for example, propaganda maps in wartime that distortedly illustrated the real battle situations in order to shake the enemy’s morale (Herb, 2002); fictitious geographical entries, also called paper towns, phantom settlements, or trap streets, that are labeled on the map to help unveil copyright infringements (S. Zhang, 2015). It is worth noting that the term “lie” in the book title cannot be simply taken as some negative intentions in map making. Indeed, cartographic generalization is a type of “white lie” – any map needs to simplify and thus reduce the complexity of the real-world phenomenon in order to enable an efficient and legible visual communication.
Monmonier’s book, republished several times by now, has influenced generations of cartographers and GIScientists. It did not foresee, but inspired us to understand more critically and holistically, the emerging “lies” or fake geographies in today’s data-intensive and networked environments. For example, GPS signals were spoofed to mislead superyachts off the course (Shepard et al., 2012), and selfies in fake scenery spots (e.g. beach, national parks) were shared on social media to show off “fakations” (a.k.a. fake vacations) (M. Zhang, 2015). Starting from 2017, Zhao and his collaborators have conducted a series of studies on location spoofing and its existence on multiple digital platforms, such as Twitter (Zhao & Sui, 2017), Facebook (S. Zhang et al., 2020) and the online mobile game Pokémon GO (Zhao & Zhang, 2019). Location spoofing, a relatively new geographical phenomenon of fake geography, refers to a deliberate inconsistency between the reported geospatial information and the ground truth. Zhao and Sui (2017) also proposed a detection approach through combining time geography principles and the Bayesian statistics, and further explored Twitter users’ intentions in generating fake geo-tags. Zhao and Zhang (2019) further explored the spoofing issues in Pokémon GO and discussed its underlying social implications. As indicated by this study, although location spoofing was considered as cheating by the game company as well as some game players, it can be used to overcome the spatial disparity of game resources (e.g. between black and white neighborhoods in New York City) and promote fairness in accessing game resources. Moreover, S. Zhang et al. (2020) examined a cyber protest on Facebook. During this protest, the AI-powered recommendation algorithm referred the posts about the protest to Facebook users who may be interested in this topic. As a result, a great number of Facebook users remotely spoofed their location check-ins to show their support to the local protesters. AI plays an increasingly significant role in building fake geographies that are essential to the recent debates on misinformation and post-truth (Maclenan, 2018; Oscar, 2018; S. Zhang et al., 2020).
The fast penetration of AI in various areas of today’s society is driving fake geography to another level, deep fake geography, which has triggered heated debates on its controversial capacity and unforeseeable impact on society. The NGA, as mentioned earlier, has seriously reminded us of the risk of deep fake satellite images being used as a terrifying AI-powered weapon (Tucker, 2019). Considering the increasing number of fake satellite images emerged during the past two years, such as satellite images of night light in India during “Diwali” – a Hindu festival of lights (Kundu, 2019) or of fake fire in the central park of New York City (Markuse, 2019), it is highly likely in the near future if not yet that deep fake techniques could be implemented to create fake satellite images containing uncannily real landscape features. If so, deep fake can potentially develop into a new mode of unpredictable and even terrifying fake geography (Kwok & Koh, 2020; Maclenan, 2018; Tucker, 2019).
To understand such a new mode of fake geography, it is necessary to comprehend the basic algorithm of deep fake techniques in making fake geospatial data and thus to inspire us to explore possible detection approaches. From an algorithmic perspective, deep fake techniques primarily rely on Generative Adversarial Networks (GANs), which is a class of unsupervised deep learning algorithms that can simulate synthetic media (e.g. image, video, audio) that appear authentic (Charleer, 2018; Oscar, 2018). The GANs generate two networks – a “generator” and a “discriminator”; and enable them to contest with one another through a multiple-epoch training process. In the training process, the generator creates a latent space of candidate datasets, and then the discriminator evaluates whether the candidate datasets are qualified by satisfying an evolving statistical characteristics criterion. The candidate data from the generator, after several training epochs of tuning, can reach an acceptable similarity to the required statistical characteristics (Goodfellow et al., 2014; Salimans et al., 2016). Similarly, if we use a GAN to simulate geospatial data, the GAN’s generator will create candidates of geospatial data and ask the discriminator whether the candidates meet the characteristics of a typical geospatial data. Here, the geospatial data can be as simple as a point, polyline or polygon, or relative complex data like satellite images, or even 3D point clouds. After several epochs’ training, the candidates could eventually meet the criteria of qualified geospatial data. At this stage, the candidates, recognized as seemingly authentic geospatial data, embody a new mode of fake geography.
With a thorough review of the existing deep fake detection methods, we categorized these methods into two groups based on the detecting feature selection process – manually defined or automatically extracted (Afchar et al., 2018; Galbally & Marcel, 2014; Hsu et al., 2020; Matern et al., 2019; Zhu et al., 2017). The detection methods using manually defined features were developed prior to those using automatically extracted features. Galbally and Marcel (2014) proposed 14 general image quality metrics to distinguish between legitimate face images and impostor samples generated by deep learning algorithms and achieved competitive results. When it comes to videos, by summarizing visual artifacts arising from global consistency, illumination estimation and geometry estimation, Matern et al. (2019) were able to recognize face manipulations in videos with acceptable accuracy only using shallow classifiers such as logistic regression models. For detection methods using automatically extracted features, Hsu et al. (2020) proposed a two-step approach for general fake image detection. The first step extracts the discriminative features using the common fake feature network (CFFN) learning process, and the following step feeds these salient features into a small convolutional neural network (CNN) concatenated to the last convolutional layer of CFFN. This method achieved a precision at least 0.92 on a variety of image datasets generated by state-of-the-art GANs, significantly superior to other existing fake image detectors.
Despite the significant progress in deep fake detection, specific methods for detecting deep fake satellite images have not been explored yet. We thus designed an empirical study to closely examine deep fake techniques and explore feasible means to detect deep fakes in satellite images. Since no existing GANs-generated satellite image has been publicized or easily accessible, this empirical study began with our own experiment of simulating a baseline dataset of satellite imagery of Tacoma, Washington. The simulated satellite images were developed on the basic urban structure on the CartoDB basemap, but with the landscape features extracted from two other cities, Seattle, Washington and Beijing, China. Such GANs-generated satellite images could be viewed as fake since the displaying landscapes did not exist in the real world and hence were used experimentally in this study for testing our deep fake detection approach. It has never been our objective to show how to fake satellite images; in fact, we acknowledged that the satellite image simulation process not only provided a baseline of simulated or fake satellite images, but also offered a demonstrative example of the essential deep fake mechanism. The baseline dataset enabled us to analyze the characteristics of fake satellite images, thereby facilitating the process of proposing an approach to detecting the deep fakes in satellite images.
Cycle-Consistent Adversarial Networks (CycleGAN), as a popular model of GANs, is frequently adopted for generating deep fakes (Zhu et al., 2017). In our study, we used CycleGAN to translate the basemap of a city to satellite images into landscape features of other cities. If the newly simulated images embodied any fake geographical environment but appeared to be real, we would consider them as fake images.
Specifically, CycleGAN translates between two different domains (i.e. X and Y), where X and Y should share some underlying relationship. CycleGAN aims to learn the relationship by developing two mapping functions, G: X → Y and F: Y → X. Two associated adversarial discriminators DY and DX are developed to facilitate the mapping between the two domains by encouraging the G and F functions to generate the output indistinguishable from the corresponding domain (i.e. Y and X, respectively). CycleGAN aims to solve Equation (1) (Zhu et al., 2017): G∗,F∗=LG,F,Dx,Dy=LGANG,Dy,X,Y+LGANF,Dx,Y,X+βLcycG,F(1)
where β is a parameter that controls the relative importance of the two losses: adversarial losses LGAN and cycle consistency losses LcycG,F,which are defined as follows: LGANG,Dy,X,Y=Ey∼pdatay(logDYy+Ex∼pdataxlog1−DYGx(2) LcycG,F=Ex∼pdatax∥FGx−x∥1+Ey∼pdatay∥GFy−y∥1(3)