Could language reconstruct a map of the world?

Methodology

Data and representation

1. Embedding model. GoogleNews-vectors-negative300: word2vec (skip-gram with negative sampling) trained by Google on ~100 billion words of Google News text, circa 2013 (Mikolov, Sutskever, et al., 2013). Vocabulary of 3 million tokens, 300 dimensions per vector, with frequent multiword phrases merged into single tokens (New_York, Buenos_Aires). Under the distributional hypothesis (Harris, 1954), cities mentioned in similar news contexts receive similar vectors — the cosine computation itself sees only text. One honest qualification: choosing which token represents an ambiguous name was adjudicated with geographic ground truth in hand (step 3) — a fully blind pipeline would have put St Petersburg in Florida — so "no geography enters" holds for the similarity computation, not for every judgment call around it. Two caveats follow directly from the training data: what drives similarity is news salience, not geography (a shared war, league or stock exchange pulls cities together as surely as a shared border), and the corpus is frozen in ~2013, so prominence and usage reflect that period's coverage. That geography is recoverable from this text at all is a long-known effect (see Related work): the signal rides on how cities are described — country names, climate words — never on coordinates.
2. City list. 124 cities curated by hand: metropolitan-area population roughly >1 million and global prominence, deliberately balanced across 9 regions rather than ranked by population — a strict top-N by population would be dominated by Chinese and South-Asian metros, and the map would lose most of the regions a general reader recognizes. Region labels are used only for colouring — they play no role in the computation.
3. Token resolution. word2vec knows only tokens, not "cities", so each city is mapped to the vocabulary entry that best carries its meaning. Multiword names are the hard part: GoogleNews merged word pairs into single tokens only where they co-occur unusually often, so coverage is arbitrary — New_York and Buenos_Aires exist, but Mexico_City, New_York_City, New_Delhi and Ho_Chi_Minh_City do not (verified against all 3M entries). The resolver walks an ordered candidate list and takes the first hit; the non-obvious choices:
   • Mexico City → mean of Mexico + City, as no phrase token exists. The all-caps MEXICO_CITY (from agency datelines, "MEXICO CITY (Reuters) —…") was tested and rejected: dateline tokens keep company with other datelines, so their cosines run uniformly low and push the city to the map's edge.
   • Ho Chi Minh City → Saigon: the full name fragmented into pieces, while Saigon — still common in news and historical coverage — is a well-trained token with the right meaning.
   • St Petersburg → Saint_Petersburg, not the commoner St._Petersburg — which is dominated by St. Petersburg, Florida (cos to Tampa .68 vs Moscow .60).
   • Kyiv → Kyiv: this spelling already had its own token in the 2013 corpus, years before it displaced "Kiev" as the standard English form (most news style guides switched around 2019). New York and Delhi fall back to New_York and Delhi.
   Every token still fuses all senses of its string — Treaty-of-Lisbon coverage drags Lisbon off Iberia, and Santiago and Hyderabad each name several cities. No resolver can untangle that; entity-level embeddings could (Beyond word2vec, below).

Similarity

4. Similarity matrix. Vectors are L2-normalized and all pairwise cosine similarities computed, cos(u,v) = u·v / (‖u‖ ‖v‖) giving a symmetric 124×124 matrix (7,626 unique pairs). Observed range −.06 (Houston–Lisbon) to .86 (Sydney–Melbourne), M = .30.
5. Dissimilarity: why d = 1 − cos θ. Three monotone transforms were compared empirically on this data with identical SMACOF runs (monotone ⇒ a nonmetric MDS would not distinguish them at all):

   transform	Stress-1	recall@10	fit ρ (d ↔ 2-D)	layout Δ vs 1−cos
   1 − cos	.347	.515	.709	—
   √(2(1−cos))	.392	.505	.687	.005
   arccos(cos)	.384	.511	.692	.003

   The textbook objection to 1 − cos is that it is only a semi-metric: it can violate the triangle inequality, while the chord √(2(1−cos)) and the angle arccos are true metrics (Borg & Groenen, 2005). On this dataset the objection is nearly moot: enumerating all C(124,3) = 310,124 triples, exactly 2 violate the triangle inequality, by a largest excess of just .023 — negligible for SMACOF. Meanwhile 1 − cos fits its target best, preserves the cosine ranking best, and the three layouts are nearly identical anyway (Procrustes disparity ≤ .005). Chord would be the safer default for datasets with near-duplicate items; here the simplest transform is also the best-behaved, so the map keeps d = 1 − cos.

Projection and evaluation

6. MDS. Metric MDS (Borg & Groenen, 2005; Kruskal, 1964) via SMACOF (scikit-learn 1.9) on the precomputed dissimilarities: 2 components, classical-MDS (Torgerson) initialization followed by stress-majorization iterations (max 3000, tolerance 10⁻⁹), which makes the solution deterministic. SMACOF minimizes raw stress Σ(d̂ₖₗ − dₖₗ)² between input dissimilarities d̂ and embedding distances d.
7. Fit diagnostics. Kruskal Stress-1, Stress-1 = √(Σ(d̂ − d)² / Σ d²) = .347 with Pearson correlation between input dissimilarities and 2-D distances r = .71 (Spearman ρ = .71). By Kruskal's rule of thumb (<.025 excellent, <.05 good, >.20 poor) this is a heavily compressed fit — unavoidable when flattening the high-rank similarity structure of 300-d embeddings into a plane. Read the map accordingly: broad groupings are informative, but exact nearest neighbours should be taken from the pinned similarity ranking, not from which dot looks closest. The hover overlay shows each city's true 300-d nearest neighbours precisely so the projection's compromises stay visible.
8. Alternative layouts (the switcher). The four algorithms, in historical order, answer progressively different questions, all on the same vectors with cosine distance. PCA (Pearson, 1901) projects the cloud onto the two directions of greatest variance — linear, so it cannot bend around curved structure and keeps the fewest true neighbourhoods (recall@10 = .41). MDS is the only one that explicitly optimizes all pairwise distances (Stress-1 = .347, recall@10 = .52), which is why it is the default. t-SNE (van der Maaten & Hinton, 2008) and UMAP (McInnes et al., 2018) are nonlinear methods prioritizing local neighbourhoods, both on cosine distance (t-SNE: perplexity 20, PCA init, learning-rate "auto"; UMAP: n_neighbors 15, min_dist 0.3): they preserve local structure far better (recall@10 = .71 / .70) at the price that distances between clusters stop being comparable. On the geography question (which map looks most like the world?), the alignment-free geo-fidelity ρ — rank correlation of on-screen distances with real great-circle distances — has a natural reference: the raw 300-d cosine distances, before any projection, which you might expect to be the best a 2-D map could hope to match. They score only ρ ≈ .50 (measured .503), and every 2-D layout beats that baseline on this metric (MDS .57, PCA .60, t-SNE .64, UMAP .67). Two mechanisms can produce that gain — denoising (the projection drops mostly non-geographic directions) and cluster quantization (the neighbourhood methods exaggerate between-region separation, which a metric dominated by between-region pairs rewards) — and the decomposition under the semantic map indicates the gain lives almost entirely in between-region structure: within regions the raw space ties the best layout, and collapsing any layout to nine region points raises its score. The tier ordering neighbourhood > global > raw is stable across reruns, and each layout's advantage over the raw space survives a city-level bootstrap; adjacent layout-vs-layout gaps do not.
   Stability details (seeds, initialization, bootstrap)

   PCA, classical-init MDS and PCA-init t-SNE are deterministic — the classical-init MDS sweep in compute_intervals.py returns ρ = .572 on all 12 seeds — and only UMAP varies with the random seed (ρ .60–.68, mean .65), so the t-SNE .64 / UMAP .67 gap is within that noise. Initialization matters more than seeds: SMACOF started from random configurations instead of the classical solution spans ρ .50–.58 over 12 starts — an init-sensitivity result reported separately in the same script. A 1,000-resample city-level bootstrap (robustness_study.py) gives 95% CIs excluding zero for every layout-minus-raw difference (e.g. PCA−raw [.04, .15]) and including zero for every adjacent layout difference (PCA−MDS [−.02, .08], UMAP−t-SNE [−.05, .10]).

   All four are scaled to a common size and rigidly rotated onto geography by the same Procrustes step (step 9); the reported metrics are invariant to that alignment.
9. Orientation (geographic). An MDS solution is unique only up to rotation, reflection and translation, so the raw axes are meaningless. Instead of an arbitrary convention, the configuration is aligned to real geography by orthogonal Procrustes — the single rotation (plus reflection) that best superimposes one set of points on another, solved in closed form from an SVD (Schönemann, 1966). Here it best maps the embedding onto each city's true (longitude, latitude), with the two geographic axes standardized so east–west and north–south count equally. This is a rigid transform — it preserves every pairwise distance and the stress exactly, and only decides which way is up. Afterwards east–west correlates with longitude at r = .70 and north–south with latitude at only r = .17; even the single best rotation cannot lift the MDS layout's latitude past r = .21 (a 3,600-angle rotation scan in robustness_study.py), so the MDS map recovers little north–south structure (a statement about this projection, not a claim that the 300-d space contains none — the same scan on t-SNE reaches north–south r = .41). These axis correlations are measured after an alignment that itself uses the real coordinates, so they describe residual correspondence rather than prove unsupervised recovery — for that, see the alignment-free geo-fidelity metric under the layout switcher.

Additional views and limitations

10. Geographic network view. The same similarity matrix drawn over real geography: cities at their true (longitude, latitude) on an equirectangular projection, with a simplified Natural Earth 110m land outline embedded in the page (no external tiles). A link joins every pair with cosine ≥ the cutoff; width and opacity grow with similarity. The slider spans 0.30–0.86; the default 0.50 keeps ≈450 of the 7,626 pairs. The link count falls off sharply with the cutoff (log scale):
11. Clustered similarity matrix. The heatmap reorders the full 124×124 matrix by agglomerative hierarchical clustering with average linkage (UPGMA). Concretely, scipy.cluster.hierarchy.linkage is run on the condensed upper triangle of d = 1 − cos θ with method="average" and optimal_ordering=True: starting from 124 singletons it repeatedly merges the two clusters with the smallest mean pairwise dissimilarity, recording the entire merge tree. No cluster count is imposed — the dendrogram keeps the full history, and the leaf order is refined by optimal leaf ordering (Bar-Joseph et al., 2001), which flips branches to place the most similar neighbours adjacent without changing the tree. That ordering alone makes the block structure legible: it is the exact matrix from card 4, complete and seriated — diagonal blocks = regions, off-block warm bands = cross-region bridges (find the Gulf ↔ South Asia patch), pale rows = cities similar to nothing in particular.
12. Beyond word2vec. The 2013 model is kept deliberately: it is the canonical teaching model, ships ready-made phrase tokens, and its quirks (Lisbon, St Petersburg) are instructive. Cleaner representations would fix those quirks. Wikipedia2Vec embeds Wikipedia entities, so "Santiago (Chile)" and "Santiago de Cuba" stop sharing one vector — the cleanest fix for name collisions (Yamada et al., 2020); fastText adds subword information and handles unseen spellings (Bojanowski et al., 2017); GloVe is a close word2vec alternative trained on global co-occurrence counts (Pennington et al., 2014); modern sentence or LLM embeddings could encode a whole city description, trading the elegant "meaning from raw co-occurrence" story for quality.

Software

The pipeline runs on Python 3.11. The GoogleNews-vectors-negative300 word2vec vectors (Mikolov, Sutskever, et al., 2013) are read straight from Google's published binary by a small NumPy parser (no gensim dependency); numerical work uses NumPy 2.4 and pandas 3.0. The projections use scikit-learn 1.9.0 (MDS, PCA, t-SNE) and umap-learn 0.5.12, and the pairwise distances, hierarchical clustering and orthogonal-Procrustes alignment use SciPy 1.17.1.

The figures are drawn in the browser with no charting library: the two maps, the geo-fidelity bar chart and the clustered matrix are hand-written inline SVG and HTML <canvas>, and the formulae are typeset with KaTeX 0.16. The non-overlapping city labels were pre-computed offline with matplotlib 3.11 and adjustText 1.4, then baked into the page as fixed coordinates. The faint continent outlines behind the network map are the land layer of Natural Earth — a free, public-domain world basemap — at its 1:110m (one-to-110-million) resolution, the coarsest it ships. Source, data and build scripts are in the GitHub repository.

Related work

Recovering geography from text alone has a long pedigree — word2vec itself famously encodes geographic relations as vector offsets, answering Paris − France + Italy ≈ Rome (Mikolov, Yih, & Zweig, 2013). Louwerse and Zwaan (2009) applied MDS to newspaper-text similarities for the 50 largest US cities and recovered their latitude and longitude, connecting it to Tobler's first law; the approach even extends to estimating the locations of unseen place names from their linguistic associations (Recchia & Louwerse, 2014) and reconstructs an invented world — the cities of Middle-earth from the co-occurrences in Tolkien (Louwerse & Benesh, 2012). The very reading this page relies on — screen distance as similarity — is the "first law of cognitive geography" that spatialization research has tested on human readers directly (Montello et al., 2003). The closest precedent to this page is Konkol et al. (2017), who fit city vectors from 16 embedding models to real coordinates as a benchmark — with a more rigorous setup (a spherical 3-D target, error reported in kilometres, noise-robustness and cross-validation) and exactly the failure mode this map shows: the worst errors come from a name overlapping a more famous entity (their Kobe → the basketball player, Bismarck → the chancellor; here Lisbon, St Petersburg, Santiago); supervised probes recover referential attributes of entities from distributional vectors more generally (Gupta et al., 2015). For contemporary language models the same signal appears in hidden states — probed early by Liétard et al. (2021) and mapped in detail by Gurnee and Tegmark (2024) and Godey et al. (2024) — linearly decodable, improving with scale, but geographically uneven, weakest where corpus coverage is thinnest; a different mechanism from this map's weak north–south recovery (a projection artefact), but the same lesson: geographic signal in text models is unevenly distributed. And Barenholtz (2026) showed plain word2vec/GloVe already carry it, riding on lexical directions (country and climate words) rather than on any built-in coordinate.

What this page adds is small and specific. The central finding is not new, and Konkol et al. (2017) established it more rigorously; the contribution here is an interactive, fully documented re-examination — deliberately both a teaching demonstration and a small standalone investigation — plus three minor empirical points I did not find in the geographic-embeddings literature (projection-quality comparison is of course a field of its own; Espadoto et al., 2021 — the geographic instantiation, not the comparison itself, is the new part). (1) The map is compared across projections rather than across embeddings, and all four 2-D layouts score higher geo-fidelity than the raw 300-d cosine distances they come from (ρ ≈ .57–.67 vs a raw-space baseline of ρ ≈ .50) — though the decomposition on the page shows the gain is almost entirely between-region structure, so this is a claim about the global rank metric on this curated sample, not about street-level geography surviving projection. (2) The semi-metric objection to d = 1 − cos θ is checked exhaustively (all 310,124 triples; 2 violations, max excess .023) rather than assumed — a check of this dataset, not a general result about 1 − cos. (3) The dateline-token artefact — MEXICO_CITY sitting among other agency datelines, so its cosines to ordinary words run uniformly low — is a concrete, checkable fact about the GoogleNews vectors. A caveat the literature predicts and this page inherits: a supervised probe trained to regress coordinates recovers geography far better (R² ≈ .8, reported in a preprint by Barenholtz, 2026) than these unsupervised layouts do (geo-fidelity ρ ≈ .57–.67) — the two figures are different metrics on different scales (variance-explained vs rank correlation), so read them as a direction, not a subtraction. Either way, what you see is a map of discourse more than of the Earth.