what do you mean, bert? assessing bert as a distributional
TRANSCRIPT
What do you mean, BERT?Assessing BERT as a Distributional Semantics Model
Timothee MickusUniversite de Lorraine
CNRS, [email protected]
Denis PapernoUtrecht University
Mathieu ConstantUniversite de Lorraine
CNRS, [email protected]
Kees van DeemterUtrecht University
Abstract
Contextualized word embeddings, i.e. vectorrepresentations for words in context, are nat-urally seen as an extension of previous non-contextual distributional semantic models. Inthis work, we focus on BERT, a deep neuralnetwork that produces contextualized embed-dings and has set the state-of-the-art in severalsemantic tasks, and study the semantic coher-ence of its embedding space. While showinga tendency towards coherence, BERT does notfully live up to the natural expectations for asemantic vector space. In particular, we findthat the position of the sentence in which aword occurs, while having no meaning corre-lates, leaves a noticeable trace on the word em-beddings and disturbs similarity relationships.
1 Introduction
A recent success story of NLP, BERT (Devlin et al.,2018) stands at the crossroad of two key innova-tions that have brought about significant improve-ments over previous state-of-the-art results. Onthe one hand, BERT models are an instance of con-textual embeddings (McCann et al., 2017; Peterset al., 2018), which have been shown to be subtleand accurate representations of words within sen-tences. On the other hand, BERT is a variant of theTransformer architecture (Vaswani et al., 2017)which has set a new state-of-the-art on a widevariety of tasks ranging from machine translation(Ott et al., 2018) to language modeling (Dai et al.,2019). BERT-based models have significantly in-creased state-of-the-art over the GLUE benchmarkfor natural language understanding (Wang et al.,2019b) and most of the best scoring models forthis benchmark include or elaborate on BERT. Us-ing BERT representations has become in manycases a new standard approach: for instance, allsubmissions at the recent shared task on genderedpronoun resolution (Webster et al., 2019) were
based on BERT. Furthermore, BERT serves bothas a strong baseline and as a basis for a fine-tuned state-of-the-art word sense disambiguationpipeline (Wang et al., 2019a).
Analyses aiming to understand the mechanicalbehavior of Transformers in general, and BERT inparticular, have suggested that they compute wordrepresentations through implicitly learned syntac-tic operations (Raganato and Tiedemann, 2018;Clark et al., 2019; Coenen et al., 2019; Jawa-har et al., 2019, a.o.): representations computedthrough the ‘attention’ mechanisms of Transform-ers can arguably be seen as weighted sums ofintermediary representations from the previouslayer, with many attention heads assigning higherweights to syntactically related tokens (however,contrast with Brunner et al., 2019; Serrano andSmith, 2019).
Complementing these previous studies, in thispaper we adopt a more theory-driven lexical se-mantic perspective. While a clear parallel was es-tablished between ‘traditional’ noncontextual em-beddings and the theory of distributional seman-tics (a.o. Lenci, 2018; Boleda, 2019), this link isnot automatically extended to contextual embed-dings: some authors (Westera and Boleda, 2019)even explicitly consider only “context-invariant”representations as distributional semantics. Hencewe study to what extent BERT, as a contextual em-bedding architecture, satisfies the properties ex-pected from a natural contextualized extension ofdistributional semantics models (DSMs).
DSMs assume that meaning is derived from usein context. DSMs are nowadays systematicallyrepresented using vector spaces (Lenci, 2018).They generally map each word in the domain ofthe model to a numeric vector on the basis ofdistributional criteria; vector components are in-ferred from text data. DSMs have also been com-puted for linguistic items other than words, e.g.,
word senses—based both on meaning inventories(Rothe and Schutze, 2015) and word sense induc-tion techniques (Bartunov et al., 2015)—or mean-ing exemplars (Reisinger and Mooney, 2010; Erkand Pado, 2010; Reddy et al., 2011). The defaultapproach has however been to produce represen-tations for word types. Word properties encodedby DSMs vary from morphological information(Marelli and Baroni, 2015; Bonami and Paperno,2018) to geographic information (Louwerse andZwaan, 2009), to social stereotypes (Bolukbasiet al., 2016) and to referential properties (Herbe-lot and Vecchi, 2015).
A reason why contextualized embeddings havenot been equated to distributional semantics maylie in that they are “functions of the entire inputsentence” (Peters et al., 2018). Whereas tradi-tional DSMs match word types with numeric vec-tors, contextualized embeddings produce distinctvectors per token. Ideally, the contextualized na-ture of these embeddings should reflect the seman-tic nuances that context induces in the meaning ofa word—with varying degrees of subtlety, rang-ing from broad word-sense disambiguation (e.g.‘bank’ as a river embankment or as a financialinstitution) to narrower subtypes of word usage(‘bank’ as a corporation or as a physical building)and to more context-specific nuances.
Regardless of how apt contextual embeddingssuch as BERT are at capturing increasingly finersemantic distinctions, we expect the contextualvariation to preserve the basic DSM properties.Namely, we expect that the space structure en-codes meaning similarity and that variation withinthe embedding space is semantic in nature. Simi-lar words should be represented with similar vec-tors, and only semantically pertinent distinctionsshould affect these representations. We connectour study with previous work in section 2 be-fore detailing the two approaches we followed.First, we verify in section 3 that BERT embeddingsform natural clusters when grouped by word types,which on any account should be groups of similarwords and thus be assigned similar vectors. Sec-ond, we test in sections 4 and 5 that contextualizedword vectors do not encode semantically irrelevantfeatures: in particular, leveraging some knowledgefrom the architectural design of BERT, we addresswhether there is no systematic difference betweenBERT representations in odd and even sentencesof running text—a property we refer to as cross-
sentence coherence. In section 4, we test whetherwe can observe cross-sentence coherence for sin-gle tokens, whereas in section 5 we study to whatextent incoherence of representations across sen-tences affects the similarity structure of the seman-tic space. We summarize our findings in section 6.
2 Theoretical background & connections
Word embeddings have been said to be ‘all-purpose’ representations, capable of unifying theotherwise heterogeneous domain that is NLP (Tur-ney and Pantel, 2010). To some extent this claimspearheaded the evolution of NLP: focus recentlyshifted from task-specific architectures with lim-ited applicability to universal architectures requir-ing little to no adaptation (Radford, 2018; Devlinet al., 2018; Radford et al., 2019; Yang et al., 2019;Liu et al., 2019, a.o.).
Word embeddings are linked to the distribu-tional hypothesis, according to which “you shallknow a word from the company it keeps” (Firth,1957). Accordingly, the meaning of a word can beinferred from the effects it has on its context (Har-ris, 1954); as this framework equates the meaningof a word to the set of its possible usage contexts,it corresponds more to holistic theories of meaning(Quine, 1960, a.o.) than to truth-value accounts(Frege, 1892, a.o.). In early works on word em-beddings (Bengio et al., 2003, e.g.), a straightfor-ward parallel between word embeddings and dis-tributional semantics could be made: the formerare distributed representations of word meaning,the latter a theory stating that a word’s meaning isdrawn from its distribution. In short, word embed-dings could be understood as a vector-based im-plementation of the distributional hypothesis. Thisparallel is much less obvious for contextual em-beddings: are constantly changing representationstruly an apt description of the meaning of a word?
More precisely, the literature on distributionalsemantics has put forth and discussed many math-ematical properties of embeddings: embeddingsare equivalent to count-based matrices (Levy andGoldberg, 2014b), expected to be linearly depen-dant (Arora et al., 2016), expressible as a unitarymatrix (Smith et al., 2017) or as a perturbationof an identity matrix (Yin and Shen, 2018). Allthese properties have however been formalized fornon-contextual embeddings: they were formulatedusing the tools of matrix algebra, under the as-sumption that embedding matrix rows correspond
to word types. Hence they cannot be applied assuch to contextual embeddings. This disconnectin the literature leaves unanswered the question ofwhat consequences there are to framing contextu-alized embeddings as DSMs.
The analyses that contextual embeddings havebeen subjected to differ from most analyses of dis-tributional semantics models. Peters et al. (2018)analyzed through an extensive ablation study ofELMo what information is captured by each layerof their architecture. Devlin et al. (2018) dis-cussed what part of their architecture is criti-cal to the performances of BERT, comparing pre-training objectives, number of layers and train-ing duration. Other works (Raganato and Tiede-mann, 2018; Hewitt and Manning, 2019; Clarket al., 2019; Voita et al., 2019; Michel et al., 2019)have introduced specific procedures to understandhow attention-based architectures function on amechanical level. Recent research has howeverquestioned the pertinence of these attention-basedanalyses (Serrano and Smith, 2019; Brunner et al.,2019); moreover these works have focused moreon the inner workings of the networks than on theiradequacy with theories of meaning.
One trait of DSMs that is very often encoun-tered, discussed and exploited in the literature isthe fact that the relative positions of embeddingsare not random. Early vector space models, by de-sign, required that word with similar meanings lienear one another (Salton et al., 1975); as a conse-quence, regions of the vectors space describe co-herent semantic fields.1 Despite the importanceof this characteristic, the question whether BERTcontextual embeddings depict a coherent semanticspace on their own has been left mostly untouchedby papers focusing on analyzing BERT or Trans-formers (with some exceptions, e.g. Coenen et al.,2019). Moreover, many analyses of how mean-ing is represented in attention-based networks orcontextual embeddings include “probes” (learnedmodels such as classifiers) as part of their evalu-ation setup to ‘extract’ information from the em-beddings (Peters et al., 2018; Tang et al., 2018;Coenen et al., 2019; Chang and Chen, 2019, e.g.).Yet this methodology has been criticized as po-tentially conflicting with the intended purpose ofstudying the representations themselves (Wietingand Kiela, 2019; Cover, 1965); cf. also Hewitt and
1Vectors encoding contrasts between words are also ex-pected to be coherent (Mikolov et al., 2013b), although thisassumption has been subjected to criticism (Linzen, 2016).
Liang (2019) for a discussion. We refrain fromusing learned probes in favor of a more direct as-sessment of the coherence of the semantic space.
3 Experiment 1: Word Type CohesionThe trait of distributional spaces that we focuson in this study is that similar words should liein similar regions of the semantic space. Thisshould hold all the more so for identical words,which ought to be be maximally similar. By de-sign, contextualized embeddings like BERT exhibitvariation within vectors corresponding to identicalword types. Thus, if BERT is a DSM, we expect thatword types form natural, distinctive clusters in theembedding space. Here, we assess the coherenceof word type clusters by means of their silhouettescores (Rousseeuw, 1987).
3.1 Data & Experimental setupThroughout our experiments, we used the Guten-berg corpus as provided by the NLTK platform,out of which we removed older texts (King John’sBible and Shakespeare). Sentences are enumer-ated two by two; each pair of sentences is thenused as a distinct input source for BERT. As wetreat the BERT algorithm as a black box, we re-trieve only the embeddings from the last layer,discarding all intermediary representations and at-tention weights. We used the bert-large-uncased model in all experiments2; thereforemost of our experiments are done on word-pieces.
To study the basic coherence of BERT’s seman-tic space, we can consider types as clusters oftokens—i.e. specific instances of contextualizedembeddings—and thus leverage the tools of clus-ter analysis. In particular, silhouette score is gen-erally used to assess whether a specific observa-tion ~v is well assigned to a given cluster Ci drawnfrom a set of possible clusters C. The silhouettescore is defined in eq. 1:
sep(~v, Ci) =min{mean~v02Cj
d(~v, ~v0)8 Cj 2 C � {Ci}}
coh(~v, Ci) = mean~v02Ci�{~v}
d(~v, ~v0)
silh(~v, Ci) =sep(~v, Ci)� coh(~v, Ci)
max{sep(~v, Ci), coh(~v, Ci)}(1)
We used Euclidean distance for d. In our case,observations ~v therefore correspond to tokens (thatis, word-piece tokens), and clusters Ci to types.
2Measurements were conducted before the release of thebert-large-uncased-whole-words model.
Silhouette scores consist in computing for eachvector observation ~v a cohesion score (viz. the av-erage distance to other observations in the clusterCi) and a separation score (viz. the minimal av-erage distance to other observations, i.e. the min-imal ‘cost’ of assigning ~v to any other clusterthan Ci). Optimally, cohesion is to be minimizedand separation is to be maximized, and this is re-flected in the silhouette score itself: scores are de-fined between -1 and 1; -1 denotes that the ob-servation ~v should be assigned to another clusterthan Ci, whereas 1 denotes that the observation ~vis entirely consistent with the cluster Ci. Keep-ing track of silhouette scores for a large numberof vectors quickly becomes intractable, hence weuse a slightly modified version of the above def-inition, and compute separation and cohesion us-ing the distance to the average vector for a clus-ter rather than the average distance to other vec-tors in a cluster, as suggested by Vendramin et al.(2013). Though results are not entirely equivalentas they ignore the inner structure of clusters, theystill present a gross view of the consistency of thevector space under study.
We do note two caveats with our proposedmethodology. Firstly, BERT uses subword repre-sentations, and thus BERT tokens do not necessar-ily correspond to words. However we may conjec-ture that some subwords exhibit coherent mean-ings, based on whether they tightly correspondto morphemes—e.g. ‘##s’, ‘##ing’ or ‘##ness’.Secondly, we group word types based on char-acter strings; yet only monosemous words shoulddescribe perfectly coherent clusters—whereas weexpect some degree of variation for polysemouswords and homonyms according to how widelytheir meanings may vary.
3.2 Results & Discussion
We compared cohesion to separation scores usinga paired Student’s t-test, and found a significanteffect (p-value < 2 · 2�16). This highlights thatcohesion scores are lower than separation scores.The effect size as measured by Cohen’s d (Cohen’sd = �0.121) is however rather small, suggestingthat cohesion scores are only 12% lower than sep-aration scores. More problematically, we can seein figure 1 that 25.9% of the tokens have a nega-tive silhouette score: one out of four tokens wouldbe better assigned to some other type than the onethey belong to. When aggregating scores by types,
Figure 1: Distribution of token silhouette scores
we found that 10% of types contained only tokenswith negative silhouette score.
The standards we expect of DSMs are not al-ways upheld strictly; the median and mean scoreare respectively at 0.08 and 0.06, indicating a gen-eral trend of low scores, even when they are posi-tive. We previously noted that both the use of sub-word representations in BERT as well as polysemyand homonymy might impact these results. Theamount of meaning variation induced by polysemyand homonymy can be estimated by using a dictio-nary as a sense inventory. The number of distinctentries for a type serves as a proxy measure of howmuch its meaning varies in use. We thus used alinear model to predict silhouette scores with log-scaled frequency and log-scaled definition counts,as listed in the Wiktionary, as predictors. We se-lected tokens for which we found at least one en-try in the Wiktionary, out of which we then ran-domly sampled 10000 observations. Both defini-tion counts and frequency were found to be signif-icant predictors, leading the silhouette score to de-crease. This suggests that polysemy degrades thecohesion score of the type cluster, which is com-patible with what one would expect from a DSM.We moreover observed that monosemous wordsyielded higher silhouette scores than polysemouswords (p < 2 · 2�16, Cohen’s d = 0.236), thoughthey still include a substantial number of tokenswith negative silhouette scores.
Similarity also includes related words, and notonly tokens of the same type. Other studies (Vialet al., 2019; Coenen et al., 2019, e.g.) alreadystressed that BERT embeddings perform well onword-level semantic tasks. To directly assesswhether BERT captures this broader notion of sim-ilarity, we used the MEN word similarity dataset
(Bruni et al., 2014), which lists pairs of Englishwords with human annotated similarity ratings.We removed pairs containing words for which wehad no representation, leaving us with 2290 pairs.We then computed the Spearman correlation be-tween similarity ratings and the cosine of the av-erage BERT embeddings of the two paired wordtypes, and found a correlation of 0.705, showingthat cosine similarity of average BERT embeddingsencodes semantic similarity. For comparison, aword2vec DSM (Mikolov et al., 2013a, henceforthW2V) trained on BooksCorpus (Zhu et al., 2015)using the same tokenization as BERT achieved acorrelation of 0.669.
4 Experiment 2: Cross-SentenceCoherence
As observed in the previous section, overall theword type coherence in BERT tends to match ourbasic expectations. In this section, we do furthertests, leveraging our knowledge of the design ofBERT. We detail the effects of jointly using seg-ment encodings to distinguish between paired in-put sentences and residual connections.
4.1 Formal approach
We begin by examining the architectural designof BERT. We give some elements relevant toour study here and refer the reader to the orig-inal papers by Vaswani et al. (2017) and Devlinet al. (2018), introducing Transformers and BERT,for a more complete description. On a formallevel, BERT is a deep neural network composedof superposed layers of computations. Each layeris composed of two “sub-layers”: the first per-forming “multi-head attention”, the second be-ing a simple feed-forward network. Throughoutall layers, after each sub-layer, residual connec-tions and layer normalization are applied, thus theintermediary output ~oL after sub-layer L can bewritten as a function of the input ~xL, as ~oL =LayerNorm(SubL( ~xL) + ~xL).
BERT is optimized on two training objectives.The first, called masked language model, is a vari-ation on the Cloze test for reading proficiency(Taylor, 1953). The second, called next sen-tence prediction (NSP), corresponds to predictingwhether two sentences are found one next to theother in the original corpus or not. Each examplepassed as input to BERT is comprised of two sen-tences, either contiguous sentences from a docu-
ment, or randomly selected sentences. A specialtoken [SEP] is used to indicate sentence bound-aries, and the full sentence is prepended with asecond special token [CLS] used to perform theactual prediction for NSP. Each token is trans-formed into an input vector using an input em-bedding matrix. To distinguish between tokensfrom the first and the second sentence, the modeladds a learned feature vector ~segA to all tokensfrom first sentences, and a distinct learned featurevector ~segB to all tokens from second sentences;these feature vectors are called ‘segment encod-ings’. Lastly, as Transformer models do not havean implicit representation of word-order, informa-tion regarding the index i of the token in the sen-tence is added using a positional encoding p(i).Therefore, if the initial training example was “Mydog barks. It is a pooch.”, the actual input wouldcorrespond to the following sequence of vectors:
~[CLS]+ ~p(0) + ~segA, ~My + ~p(1) + ~segA,
~dog + ~p(2) + ~segA, ~barks+ ~p(3) + ~segA,
~.+ ~p(4) + ~segA, ~[SEP]+ ~p(5) + ~segA,
~It+ ~p(6) + ~segB, ~is+ ~p(7) + ~segB,
~a+ ~p(8) + ~segB, ~pooch+ ~p(9) + ~segB,
~.+ ~p(10) + ~segB, ~[SEP]+ ~p(11) + ~segB
Due to the general use of residual connections,marking the sentences using the segment encod-ings ~segA and ~segB can introduce a systematicoffset within sentences. Consider that the firstlayer uses as input vectors corresponding to word,position, and sentence information: ~wi + ~p(i) +
~segi; for simplicity, let ~ii = ~wi + ~p(i); we alsoignore the rest of the input as it does not impactthis reformulation. The output from the first sub-layer ~o1i can be written:
~o1i = LayerNorm(Sub1(~ii + ~segi) + ~ii + ~segi)
= ~bl + ~g1 � 1
�1i
Sub1(~ii + ~segi) + ~g1 � 1
�1i
~ii
� ~g1 � 1
�1i
µ(Sub1(~ii + ~segi) + ~ii + ~segi)
+ ~g1 � 1
�1i
~segi
= ~o1i + ~g1 � 1
�1i
~segi (2)
This equation is obtain by simply injecting the
~dog
~My
~barks ~It’s
~pooch
~a
(a) without segment encoding bias
~dog
~My
~barks
~It’s
~pooch
~a
(b) with segment encoding bias
Figure 2: Segment encoding bias
definition for layer-normalization.3 Therefore, byrecurrence, the final output ~oLi for a given token~wi + ~p(i) + ~segi can be written as:
~oLi = ~oLi +
LK
l=1
~gl!
�
LY
l=1
1
�li
!⇥ ~segi (3)
This rewriting trick shows that segment encod-ings are partially preserved in the output. All em-beddings within a sentence contain a shift in aspecific direction, determined only by the initialsegment encoding and the learned gain parametersfor layer normalization. In figure 2, we illustratewhat this systematic shift might entail. Prior to theapplication of the segment encoding bias, the se-mantic space is structured by similarity (‘pooch’ isnear ‘dog’); with the bias, we find a different setof characteristics: in our toy example, tokens arelinearly separable by sentences.
3Layer normalization after sub-layer l is defined as:
LayerNorml(~x) = ~bl +~gl � (~x� µ(~x))
�
= ~bl + ~gl �1�~x� ~gl �
1�µ(~x)
where ~bl is a bias, � denotes element-wise multiplication, ~glis a “gain” parameter, � is the standard deviation of compo-nents of ~x and µ(~x) = hx, . . . , xi is a vector with all compo-nents defined as the mean of components of ~x.
4.2 Data & Experimental setupIf BERT properly describes a semantic vectorspace, we should, on average, observe no signifi-cant difference in token encoding imputable to thesegment the token belongs to. For a given wordtype w, we may constitute two groups: wsegA , theset of tokens for this type w belonging to first sen-tences in the inputs, and wsegB , the set of tokensof w belonging to second sentences. If BERT coun-terbalances the segment encodings, random differ-ences should cancel out, and therefore the mean ofall tokens wsegA should be equivalent to the meanof all tokens wsegB .
We used the same dataset as in section 3. Thissetting (where all paired input sentences are drawnfrom running text) allows us to focus on the effectsof the segment encodings. We retrieved the outputembeddings of the last BERT layer and groupedthem per word type. To assess the consistency ofa group of embeddings with respect to a purportedreference, we used a mean of squared error (MSE):given a group of embeddings E and a referencevector ~r, we computed how much each vector inE strayed from the reference ~r. It is formally de-fined as:
MSE(E,~r) =1
#E
X
~v2E
X
d
(~vd � ~rd)2 (4)
This MSE can also be understood as the averagesquared distance to the reference ~r. When ~r = E,i.e. ~r is set to be the average vector in E, theMSE measures variance of E via Euclidean dis-tance. We then used the MSE function to constructpairs of observations: for each word type w, andfor each segment encoding segi, we computedtwo scores: MSE(wsegi , wsegi)—which gives usan assessment of how coherent the set of embed-dings wsegi is with respect to the mean vectorin that set—and MSE(wsegi , wsegj )—which as-sesses how coherent the same group of embed-dings is with respect to the mean vector for theembeddings of the same type, but from the othersegment segj . If no significant contrast betweenthese two scores can be observed, then BERT coun-terbalances the segment encodings and is coherentacross sentences.
4.3 Results & DiscussionWe compared results using a paired Student’s t-test, which highlighted a significant differencebased on which segment types belonged to (p-value < 2 · 2�16); the effect size (Cohen’s d =
Figure 3: Log-scaled MSE per reference
�0.527) was found to be stronger than what wecomputed when assessing whether tokens clusteraccording to their types (cf. section 3). A vi-sual representation of these results, log-scaled, isshown in figure 3. For all sets wsegi , the averageembedding from the set itself was systematicallya better fit than the average embedding from thepaired set wsegj . We also noted that a small num-ber of items yielded a disproportionate differencein MSE scores and that frequent word types hadsmaller differences in MSE scores: roughly speak-ing, very frequent items—punctuation signs, stop-words, frequent word suffixes—received embed-dings that are almost coherent across sentences.
Although the observed positional effect of em-beddings’ inconsistency might be entirely due tosegment encodings, additional factors might beat play. In particular, BERT uses absolute posi-tional encoding vectors to order words within asequence: the first word w1 is marked with the po-sitional encoding p(1), the second word w2 withp(1), and so on until the last word, wn, markedwith p(n). As these positional encodings areadded to the word embeddings, the same remarkmade earlier on the impact of residual connectionsmay apply to these positional encodings as well.Lastly, we also note that many downstream appli-cations use a single segment encoding per input,and thus sidestep the caveat stressed here.
5 Experiment 3: Sentence-level structure
We have seen that BERT embeddings do not fullyrespect cross-sentence coherence; the same typereceives somewhat different representations for
occurrences in even and odd sentences. However,comparing tokens of the same type in consecutivesentences is not necessarily the main applicationof BERT and related models. Does the segment-based representational variance affect the structureof the semantic space, instantiated in similaritiesbetween tokens of different types? Here we inves-tigate how segment encodings impact the relationbetween any two tokens in a given sentence.
5.1 Data & Experimental setupConsistent with previous experiments, we usedthe same dataset (cf. section 3); in this experi-ment also mitigating the impact of the NSP objec-tive was crucial. Sentences were thus passed twoby two as input to the BERT model. As cosinehas been traditionally used to quantify semanticsimilarity between words (Mikolov et al., 2013b;Levy and Goldberg, 2014a, e.g.), we then com-puted pairwise cosine of the tokens in each sen-tence. This allows us to reframe our assessment ofwhether lexical contrasts are coherent across sen-tences as a comparison of semantic dissimilarityacross sentences. More formally, we compute thefollowing set of cosine scores CS for each sen-tence S:
CS = {cos(~v, ~u) | ~v 6= ~u ^ ~v, ~u 2 ES} (5)
with ES the set of embeddings for the sentence S.In this analysis, we compare the union of all sets ofcosine scores for first sentences against the unionof all sets of cosine scores for second sentences.To avoid asymmetry, we remove the [CLS] token(only present in first sentences), and as with pre-vious experiments we neutralize the effects of theNSP objective by using only consecutive sentencesas input.
5.2 Results & DiscussionWe compared cosine scores for first and secondsentences using a Wilcoxon rank sum test. Weobserved a significant effect, however small (Co-hen’s d = 0.011). This may perhaps be due to dataidiosyncrasies, and indeed when comparing with aW2V (Mikolov et al., 2013a) trained on BooksCor-pus (Zhu et al., 2015) using the same tokeniza-tion as BERT, we do observe a significant effect(p < 0.05). However the effect size is six timessmaller (d = 0.002) than what we found for BERTrepresentations; moreover, when varying the sam-ple size (cf. figure 4), p-values for BERT represen-tations drop much faster to statistical significance.
1K 10K 100K 1M Full
00.10.20.30.40.50.60.70.80.9
Sample size
p-va
lue
BERT W2V p = 0.05
Figure 4: Wilcoxon tests, 1st vs. 2nd sentences
A possible reason for the larger discrepancy ob-served in BERT representations might be that BERTuses absolute positional encodings, i.e. the kth
word of the input is encoded with p(k). There-fore, although all first sentences of a given length lwill be indexed with the same set of positional en-codings {p(1), . . . , p(l)}, only second sentencesof a given length l preceded by first sentences ofa given length j share the exact same set of posi-tional encodings {p(j + 1), . . . , p(j + l)}. Ashighlighted previously, the residual connectionsensure that the segment encodings were partiallypreserved in the output embedding: the same argu-ment can be made for positional encodings. In anyevent, the fact is that we do observe on BERT rep-resentations an effect of segment on sentence-levelstructure. This effect is greater than one can blameon data idiosyncrasies, as verified by the compari-son with a traditional DSM such as W2V. If we areto consider BERT as a DSM, we must do so at thecost of cross-sentence coherence.
The analysis above suggests that embeddingsfor tokens drawn from first sentences live in adifferent semantic space than tokens drawn fromsecond sentences, i.e. that BERT contains twoDSMs rather than one. If so, the comparison be-tween two sentence-representations from a singleinput would be meaningless, or at least less co-herent than the comparison of two sentence rep-resentations drawn from the same sentence posi-tion. To test this conjecture, we use two compo-sitional semantics benchmarks: STS (Cer et al.,2017) and SICK-R (Marelli et al., 2014). Thesedatasets are structured as triplets, grouping a pair
Model STS cor. SICK-R cor.Skip-Thought 0.255 60 0.487 62USE 0.666 86 0.689 97InferSent 0.676 46 0.709 03
BERT, 2 sent. ipt. 0.359 13 0.369 92BERT, 1 sent. ipt. 0.482 41 0.586 95W2V 0.370 17 0.533 56
Table 1: Correlation (Spearman ⇢) of cosine simi-larity and relatedness ratings on the STS and SICK-R benchmarks
of sentences with a human-annotated relatednessscore. The original presentation of BERT (Devlinet al., 2018) did include a downstream applicationto these datasets, but employed a learned classi-fier, which obfuscates results (Wieting and Kiela,2019; Cover, 1965; Hewitt and Liang, 2019).Hence we simply reduce the sequence of tokenswithin each sentence into a single vector by sum-ming them, a simplistic yet robust semantic com-position method. We then compute the Spearmancorrelation between the cosines of the two sumvectors and the sentence pair’s relatedness score.We compare two setups: a “two sentences input”scheme (or 2 sent. ipt. for short)—where we usethe sequences of vectors obtained by passing thetwo sentences as a single input—and a “one sen-tence input” scheme (1 sent. ipt.)—using two dis-tinct inputs of a single sentence each.
Results are reported in table 1; we also pro-vide comparisons with three different sentence en-coders and the aforementioned W2V model. Aswe had suspected, using sum vectors drawn froma two sentence input scheme single degrades per-formances below the W2V baseline. On the otherhand, a one sentence input scheme seems to pro-duce coherent sentence representations: in thatscenario, BERT performs better than W2V andthe older sentence encoder Skip-Thought (Kiroset al., 2015), but worse than the modern USE(Cer et al., 2018) and Infersent (Conneau et al.,2017). The comparison with W2V also showsthat BERT representations over a coherent inputare more likely to include some form of composi-tional knowledge than traditional DSMs; howeverit is difficult to decide whether some true form ofcompositionality is achieved by BERT or whetherthese performances are entirely a by-product ofthe positional encodings. In favor of the former,other research has suggested that Transformer-
based architectures perform syntactic operations(Raganato and Tiedemann, 2018; Hewitt and Man-ning, 2019; Clark et al., 2019; Jawahar et al., 2019;Voita et al., 2019; Michel et al., 2019). In all, theseresults suggest that the semantic space of tokenrepresentations from second sentences differ fromthat of embeddings from first sentences.
6 Conclusions
Our experiments have focused on testing to whatextent similar words lie in similar regions ofBERT’s latent semantic space. Although we sawthat type-level semantics seem to match our gen-eral expectations about DSMs, focusing on detailsleaves us with a much foggier picture.
The main issue stems from BERT’s “next sen-tence prediction objective”, which requires tokensto be marked according to which sentence they be-long. This introduces a distinction between firstand second sentence of the input that runs con-trary to our expectations in terms of cross-sentencecoherence. The validity of such a distinction forlexical semantics may be questioned, yet its ef-fects can be measured. The primary assessmentconducted in section 3 shows that token repre-sentations did tend to cluster naturally accordingto their types, yet a finer study detailed in sec-tion 4 highlights that tokens from distinct sen-tence positions (even vs. odd) tend to have dif-ferent representations. This can seen as a directconsequence of BERT’s architecture: residual con-nections, along with the use of specific vectors toencode sentence position, entail that tokens for agiven sentence position are ‘shifted’ with respectto tokens for the other position. Encodings have asubstantial effect on the structure of the semanticsubspaces of the two sentences in BERT input. Ourexperiments demonstrate that assuming samenessof the semantic space across the two input sen-tences can lead to a significant performance dropin semantic textual similarity.
One way to overcome this violation of cross-sentence coherence would be to consider first andsecond sentences representations as belonging todistinct distributional semantic spaces. The factthat first sentences were shown to have on aver-age higher pairwise cosines than second sentencescan be partially explained by the use of absolutepositional encodings in BERT representations. Al-though positional encodings are required so thatthe model does not devolve into a bag-of-word
system, absolute encodings are not: other workshave proposed alternative relative position encod-ings (Shaw et al., 2018; Dai et al., 2019, e.g.); re-placing the former with the latter may alleviate thegap in lexical contrasts. Other related questionsthat we must leave to future works encompasstesting on other BERT models such as the whole-words model, or that of Liu et al. (2019) whichdiffers only by its training objectives, as well asother contextual embeddings architectures.
Our findings suggest that the formulation of theNSP objective of BERT obfuscates its relation todistributional semantics, by introducing a system-atic distinction between first and second sentenceswhich impacts the output embeddings. Similarly,other works (Lample and Conneau, 2019; Yanget al., 2019; Joshi et al., 2019; Liu et al., 2019)stress that the usefulness and pertinence of the NSPtask were not obvious. These studies favored anempirical point of view; here, we have shown whatsorts of caveats came along with such artificial dis-tinctions from the perspective of a theory of lexi-cal semantics. We hope that future research willextend and refine these findings, and further ourunderstanding of the peculiarities of neural archi-tectures as models of linguistic structure.
Acknowledgments
We thank Quentin Gliosca whose remarks havebeen extremely helpful to this work. We alsothank Olivier Bonami as well as three anony-mous reviewers for their thoughtful criticism. Thework was supported by a public grant overseen bythe French National Research Agency (ANR) aspart of the “Investissements d’Avenir” program:Idex Lorraine Universite d’Excellence (reference:ANR-15-IDEX-0004).
ReferencesSanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma,
and Andrej Risteski. 2016. Linear algebraic struc-ture of word senses, with applications to polysemy.CoRR, abs/1601.03764.
Sergey Bartunov, Dmitry Kondrashkin, Anton Os-okin, and Dmitry P. Vetrov. 2015. Breaking sticksand ambiguities with adaptive skip-gram. CoRR,abs/1502.07257.
Yoshua Bengio, Rejean Ducharme, Pascal Vincent, andChristian Janvin. 2003. A neural probabilistic lan-guage model. J. Mach. Learn. Res., 3:1137–1155.
Gemma Boleda. 2019. Distributional semantics andlinguistic theory. CoRR, abs/1905.01896.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou,Venkatesh Saligrama, and Adam T Kalai. 2016.Man is to computer programmer as woman is tohomemaker? debiasing word embeddings. In D. D.Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, andR. Garnett, editors, Advances in Neural InformationProcessing Systems 29, pages 4349–4357. CurranAssociates, Inc.
Olivier Bonami and Denis Paperno. 2018. A charac-terisation of the inflection-derivation opposition in adistributional vector space. Lingua e Langaggio.
Elia Bruni, Nam-Khanh Tran, and Marco Baroni. 2014.Multimodal distributional semantics. J. Artif. Intell.Res., 49:1–47.
Gino Brunner, Yang Liu, Damian Pascual, OliverRichter, and Roger Wattenhofer. 2019. Onthe Validity of Self-Attention as Explanationin Transformer Models. arXiv e-prints, pagearXiv:1908.04211.
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua,Nicole Limtiaco, Rhomni St. John, Noah Constant,Mario Guajardo-Cespedes, Steve Yuan, Chris Tar,Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil.2018. Universal sentence encoder. CoRR,abs/1803.11175.
Daniel M. Cer, Mona T. Diab, Eneko Agirre, InigoLopez-Gazpio, and Lucia Specia. 2017. Semeval-2017 task 1: Semantic textual similarity multilin-gual and crosslingual focused evaluation. In Pro-ceedings of the 11th International Workshop on Se-mantic Evaluation, SemEval@ACL 2017, Vancou-ver, Canada, August 3-4, 2017, pages 1–14.
Ting-Yun Chang and Yun-Nung Chen. 2019. Whatdoes this word mean? explaining contextualized em-beddings with natural language definition.
Kevin Clark, Urvashi Khandelwal, Omer Levy, andChristopher D. Manning. 2019. What does BERTlook at? an analysis of BERT’s attention. In Pro-ceedings of the 2019 ACL Workshop BlackboxNLP:Analyzing and Interpreting Neural Networks forNLP, pages 276–286, Florence, Italy. Associationfor Computational Linguistics.
Andy Coenen, Emily Reif, Ann Yuan, Been Kim,Adam Pearce, Fernanda Viegas, and Martin Watten-berg. 2019. Visualizing and measuring the geometryof bert. arXiv e-prints, page arXiv:1906.02715.
Alexis Conneau, Douwe Kiela, Holger Schwenk, LoıcBarrault, and Antoine Bordes. 2017. Supervisedlearning of universal sentence representations fromnatural language inference data. In Proceedings ofthe 2017 Conference on Empirical Methods in Nat-ural Language Processing, pages 670–680, Copen-hagen, Denmark. Association for ComputationalLinguistics.
Thomas M. Cover. 1965. Geometrical and statisticalproperties of systems of linear inequalities with ap-plications in pattern recognition. IEEE Trans. Elec-tronic Computers, 14(3):326–334.
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G.Carbonell, Quoc V. Le, and Ruslan Salakhutdi-nov. 2019. Transformer-xl: Attentive languagemodels beyond a fixed-length context. CoRR,abs/1901.02860.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2018. BERT: pre-training ofdeep bidirectional transformers for language under-standing. CoRR, abs/1810.04805.
Katrin Erk and Sebastian Pado. 2010. Exemplar-basedmodels for word meaning in context. In ACL.
J. R. Firth. 1957. A synopsis of linguistic theory 1930-55. Studies in Linguistic Analysis (special volume ofthe Philological Society), 1952-59:1–32.
Gottlob Frege. 1892. Uber Sinn und Bedeutung.Zeitschrift fr Philosophie und philosophische Kritik,100:25–50.
Zellig Harris. 1954. Distributional structure. Word,10(23):146–162.
Aurelie Herbelot and Eva Maria Vecchi. 2015. Build-ing a shared world: mapping distributional to model-theoretic semantic spaces. In Proceedings of the2015 Conference on Empirical Methods in NaturalLanguage Processing, pages 22–32. Association forComputational Linguistics.
John Hewitt and Percy Liang. 2019. Designing andInterpreting Probes with Control Tasks. arXiv e-prints, page arXiv:1909.03368.
John Hewitt and Christopher D. Manning. 2019. Astructural probe for finding syntax in word represen-tations. In NAACL-HLT.
Ganesh Jawahar, Benoıt Sagot, and Djame Seddah.2019. What does BERT learn about the structureof language? In ACL 2019 - 57th Annual Meeting ofthe Association for Computational Linguistics, Flo-rence, Italy.
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S.Weld, Luke Zettlemoyer, and Omer Levy. 2019.Spanbert: Improving pre-training by representingand predicting spans. CoRR, abs/1907.10529.
Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov,Richard Zemel, Raquel Urtasun, Antonio Torralba,and Sanja Fidler. 2015. Skip-thought vectors. InC. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama,and R. Garnett, editors, Advances in Neural Infor-mation Processing Systems 28, pages 3294–3302.Curran Associates, Inc.
Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. CoRR,abs/1901.07291.
Alessandro Lenci. 2018. Distributional models of wordmeaning. Annual review of Linguistics, 4:151–171.
Omer Levy and Yoav Goldberg. 2014a. Linguisticregularities in sparse and explicit word representa-tions. In Proceedings of the Eighteenth Confer-ence on Computational Natural Language Learning,pages 171–180. Association for Computational Lin-guistics.
Omer Levy and Yoav Goldberg. 2014b. Neuralword embedding as implicit matrix factorization.In Z. Ghahramani, M. Welling, C. Cortes, N. D.Lawrence, and K. Q. Weinberger, editors, Advancesin Neural Information Processing Systems 27, pages2177–2185. Curran Associates, Inc.
Tal Linzen. 2016. Issues in evaluating semantic spacesusing word analogies. In Proceedings of the 1stWorkshop on Evaluating Vector-Space Representa-tions for NLP, pages 13–18, Berlin, Germany. Asso-ciation for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man-dar Joshi, Danqi Chen, Omer Levy, Mike Lewis,Luke Zettlemoyer, and Veselin Stoyanov. 2019.Roberta: A robustly optimized BERT pretrainingapproach. CoRR, abs/1907.11692.
Max M. Louwerse and Rolf A. Zwaan. 2009. Lan-guage encodes geographical information. CognitiveScience 33 (2009) 5173.
Marco Marelli and Marco Baroni. 2015. Affixationin semantic space: Modeling morpheme meaningswith compositional distributional semantics. Psy-chological review, 122 3:485–515.
Marco Marelli, Stefano Menini, Marco Baroni, LuisaBentivogli, Raffaella Bernardi, and Roberto Zam-parelli. 2014. A sick cure for the evaluation ofcompositional distributional semantic models. InProceedings of the Ninth International Conferenceon Language Resources and Evaluation (LREC’14),pages 216–223, Reykjavik, Iceland. European Lan-guage Resources Association (ELRA).
Bryan McCann, James Bradbury, Caiming Xiong, andRichard Socher. 2017. Learned in translation: Con-textualized word vectors. NIPS.
Paul Michel, Omer Levy, and Graham Neubig. 2019.Are Sixteen Heads Really Better than One? arXive-prints, page arXiv:1905.10650.
Tomas Mikolov, Kai Chen, Greg Corrado, and JeffreyDean. 2013a. Efficient estimation of word represen-tations in vector space. CoRR, abs/1301.3781.
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig.2013b. Linguistic regularities in continuous spaceword representations. In HLT-NAACL, pages 746–751.
Myle Ott, Sergey Edunov, David Grangier, andMichael Auli. 2018. Scaling neural machine trans-lation. In Proceedings of the Third Conference onMachine Translation: Research Papers, pages 1–9,Belgium, Brussels. Association for ComputationalLinguistics.
Matthew Peters, Mark Neumann, Mohit Iyyer, MattGardner, Christopher Clark, Kenton Lee, and LukeZettlemoyer. 2018. Deep contextualized word rep-resentations. In Proceedings of the 2018 Confer-ence of the North American Chapter of the Associ-ation for Computational Linguistics: Human Lan-guage Technologies, Volume 1 (Long Papers), pages2227–2237, New Orleans, Louisiana. Associationfor Computational Linguistics.
William Van Ormann Quine. 1960. Word And Object.MIT Press.
Alec Radford. 2018. Improving language understand-ing by generative pre-training.
Alec Radford, Jeff Wu, Rewon Child, David Luan,Dario Amodei, and Ilya Sutskever. 2019. Languagemodels are unsupervised multitask learners.
Alessandro Raganato and Jorg Tiedemann. 2018. Ananalysis of encoder representations in transformer-based machine translation. In Proceedings of the2018 EMNLP Workshop BlackboxNLP: Analyzingand Interpreting Neural Networks for NLP, pages287–297, Brussels, Belgium. Association for Com-putational Linguistics.
Siva Reddy, Ioannis P. Klapaftis, Diana McCarthy, andSuresh Manandhar. 2011. Dynamic and static proto-type vectors for semantic composition. In IJCNLP.
Joseph Reisinger and Raymond Mooney. 2010. Multi-prototype vector-space models of word meaning.pages 109–117.
Sascha Rothe and Hinrich Schutze. 2015. Autoex-tend: Extending word embeddings to embeddingsfor synsets and lexemes. CoRR, abs/1507.01127.
Peter Rousseeuw. 1987. Silhouettes: A graphical aid tothe interpretation and validation of cluster analysis.J. Comput. Appl. Math., 20(1):53–65.
G. Salton, A. Wong, and C. S. Yang. 1975. A vec-tor space model for automatic indexing. Commun.ACM, 18(11):613–620.
Sofia Serrano and Noah A. Smith. 2019. Is attentioninterpretable? In Proceedings of the 57th AnnualMeeting of the Association for Computational Lin-guistics, pages 2931–2951, Florence, Italy. Associa-tion for Computational Linguistics.
Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani.2018. Self-attention with relative position represen-tations. In Proceedings of the 2018 Conference ofthe North American Chapter of the Association for
Computational Linguistics: Human Language Tech-nologies, Volume 2 (Short Papers), pages 464–468,New Orleans, Louisiana. Association for Computa-tional Linguistics.
Samuel L. Smith, David H. P. Turban, Steven Hamblin,and Nils Y. Hammerla. 2017. Offline bilingual wordvectors, orthogonal transformations and the invertedsoftmax. In 5th International Conference on Learn-ing Representations, ICLR 2017, Toulon, France,April 24-26, 2017, Conference Track Proceedings.OpenReview.net.
Gongbo Tang, Rico Sennrich, and Joakim Nivre. 2018.An analysis of attention mechanisms: The case ofword sense disambiguation in neural machine trans-lation. In Proceedings of the Third Conference onMachine Translation: Research Papers, pages 26–35, Belgium, Brussels. Association for Computa-tional Linguistics.
Wilson Taylor. 1953. Cloze procedure: A new toolfor measuring readability. Journalism Quarterly,30:415–433.
Peter D. Turney and Patrick Pantel. 2010. From fre-quency and to meaning and vector space and modelsof semantics. In Journal of Artificial IntelligenceResearch 37 (2010) 141-188 Submitted 10/09; pub-lished.
Ashish Vaswani, Noam Shazeer, Niki Parmar, JakobUszkoreit, Llion Jones, Aidan N Gomez, Ł ukaszKaiser, and Illia Polosukhin. 2017. Attention is allyou need. In I. Guyon, U. V. Luxburg, S. Bengio,H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar-nett, editors, Advances in Neural Information Pro-cessing Systems 30, pages 5998–6008. Curran As-sociates, Inc.
Lucas Vendramin, Pablo A. Jaskowiak, and Ricardo J.G. B. Campello. 2013. On the combination of rel-ative clustering validity criteria. In Proceedings ofthe 25th International Conference on Scientific andStatistical Database Management, SSDBM, pages4:1–4:12, New York, NY, USA. ACM.
Loıc Vial, Benjamin Lecouteux, and Didier Schwab.2019. Sense Vocabulary Compression through theSemantic Knowledge of WordNet for Neural WordSense Disambiguation. In Global Wordnet Confer-ence, Wroclaw, Poland.
Elena Voita, David Talbot, Fedor Moiseev, Rico Sen-nrich, and Ivan Titov. 2019. Analyzing multi-headself-attention: Specialized heads do the heavy lift-ing, the rest can be pruned. In Proceedings of the57th Annual Meeting of the Association for Com-putational Linguistics, pages 5797–5808, Florence,Italy. Association for Computational Linguistics.
Alex Wang, Yada Pruksachatkun, Nikita Nangia,Amanpreet Singh, Julian Michael, Felix Hill, OmerLevy, and Samuel R Bowman. 2019a. Super-glue: A stickier benchmark for general-purpose
language understanding systems. arXiv preprintarXiv:1905.00537.
Alex Wang, Amanpreet Singh, Julian Michael, FelixHill, Omer Levy, and Samuel R. Bowman. 2019b.GLUE: A multi-task benchmark and analysis plat-form for natural language understanding. In the Pro-ceedings of ICLR.
Kellie Webster, Marta R. Costa-jussa, Christian Hard-meier, and Will Radford. 2019. Gendered ambigu-ous pronoun (GAP) shared task at the gender biasin NLP workshop 2019. In Proceedings of the FirstWorkshop on Gender Bias in Natural Language Pro-cessing, pages 1–7, Florence, Italy. Association forComputational Linguistics.
Matthijs Westera and Gemma Boleda. 2019. Don’tblame distributional semantics if it can’t do entail-ment. In Proceedings of the 13th International Con-ference on Computational Semantics - Long Papers,pages 120–133, Gothenburg, Sweden. Associationfor Computational Linguistics.
John Wieting and Douwe Kiela. 2019. No train-ing required: Exploring random encoders for sen-tence classification. In 7th International Conferenceon Learning Representations, ICLR 2019, New Or-leans, LA, USA, May 6-9, 2019. OpenReview.net.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G.Carbonell, Ruslan Salakhutdinov, and Quoc V.Le. 2019. Xlnet: Generalized autoregressivepretraining for language understanding. CoRR,abs/1906.08237.
Zi Yin and Yuanyuan Shen. 2018. On the dimension-ality of word embedding. In S. Bengio, H. Wallach,H. Larochelle, K. Grauman, N. Cesa-Bianchi, andR. Garnett, editors, Advances in Neural InformationProcessing Systems 31, pages 887–898. Curran As-sociates, Inc.
Yukun Zhu, Jamie Ryan Kiros, Richard S. Zemel, Rus-lan Salakhutdinov, Raquel Urtasun, Antonio Tor-ralba, and Sanja Fidler. 2015. Aligning books andmovies: Towards story-like visual explanations bywatching movies and reading books. 2015 IEEE In-ternational Conference on Computer Vision (ICCV),pages 19–27.