Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processingand the 9th International Joint Conference on Natural Language Processing, pages 31–42,

Hong Kong, China, November 3–7, 2019. c©2019 Association for Computational Linguistics

31

Transfer Learning Between Related TasksUsing Expected Label Proportions

Matan Ben Noach∗† and Yoav Goldberg∗‡∗Computer Science Department, Bar-Ilan University, Ramat-Gan Israel

†Intel AI Lab, Petah-Tikva Israel‡Allen Institute for Artificial Intelligence

matan.ben.noach@intel.com, yoav.goldberg@gmail.com

Abstract

Deep learning systems thrive on abundance oflabeled training data but such data is not al-ways available, calling for alternative methodsof supervision. One such method is expecta-tion regularization (XR) (Mann and McCal-lum, 2007), where models are trained basedon expected label proportions. We proposea novel application of the XR framework fortransfer learning between related tasks, whereknowing the labels of task A provides an es-timation of the label proportion of task B.We then use a model trained for A to labela large corpus, and use this corpus with anXR loss to train a model for task B. To makethe XR framework applicable to large-scaledeep-learning setups, we propose a stochasticbatched approximation procedure. We demon-strate the approach on the task of Aspect-based Sentiment classification, where we ef-fectively use a sentence-level sentiment pre-dictor to train accurate aspect-based predictor.The method improves upon fully supervisedneural system trained on aspect-level data, andis also cumulative with LM-based pretrain-ing, as we demonstrate by improving a BERT-based Aspect-based Sentiment model.

1 Introduction

Data annotation is a key bottleneck in many datadriven algorithms. Specifically, deep learningmodels, which became a prominent tool in manydata driven tasks in recent years, require largedatasets to work well. However, many tasks re-quire manual annotations which are relatively hardto obtain at scale. An attractive alternative islightly supervised learning (Schapire et al., 2002;Jin and Liu, 2005; Chang et al., 2007; Graca et al.,2007; Quadrianto et al., 2009a; Mann and Mc-Callum, 2010a; Ganchev et al., 2010; Hope andShahaf, 2016), in which the objective function issupplemented by a set of domain-specific soft-

constraints over the model’s predictions on unla-beled data. For example, in label regularization(Mann and McCallum, 2007) the model is trainedto fit the true label proportions of an unlabeleddataset. Label regularization is special case of ex-pectation regularization (XR) (Mann and McCal-lum, 2007), in which the model is trained to fit theconditional probabilities of labels given features.

In this work we consider the case of correlatedtasks, in the sense that knowing the labels fortask A provides information on the expected la-bel composition of task B. We demonstrate the ap-proach using sentence-level and aspect-level senti-ment analysis, which we use as a running example:knowing that a sentence has positive sentimentlabel (task A), we can expect that most aspectswithin this sentence (task B) will also have pos-itive label. While this expectation may be noisyon the individual example level, it holds well inaggregate: given a set of positively-labeled sen-tences, we can robustly estimate the proportion ofpositively-labeled aspects within this set. For ex-ample, in a random set of positive sentences, weexpect to find 90% positive aspects, while in a setof negative sentences, we expect to find 70% nega-tive aspects. These proportions can be easily eitherguessed or estimated from a small set.

We propose a novel application of the XRframework for transfer learning in this setup. Wepresent an algorithm (Sec 3.1) that, given a cor-pus labeled for task A (sentence-level sentiment),learns a classifier for performing task B (aspect-level sentiment) instead, without a direct supervi-sion signal for task B. We note that the label in-formation for task A is only used at training time.Furthermore, due to the stochastic nature of theestimation, the task A labels need not be fully ac-curate, allowing us to make use of noisy predic-tions which are assigned by an automatic classi-fier (Sections 3.1 and 4). In other words, given

32

a medium-sized sentiment corpus with sentence-level labels, and a large collection of un-annotatedtext from the same distribution, we can train an ac-curate aspect-level sentiment classifier.

The XR loss allows us to use task A labels fortraining task B predictors. This ability seamlesslyintegrates into other semi-supervised schemes: wecan use the XR loss on top of a pre-trained modelto fine-tune the pre-trained representation to thetarget task, and we can also take the model trainedusing XR loss and plentiful data and fine-tune it tothe target task using the available small-scale an-notated data. In Section 5.3 we explore these op-tions and show that our XR framework improvesthe results also when applied on top of a pre-trained BERT-based model (Devlin et al., 2018).

Finally, to make the XR framework applicableto large-scale deep-learning setups, we propose astochastic batched approximation procedure (Sec-tion 3.2). Source code is available at https://github.com/MatanBN/XRTransfer.

2 Background and Related Work

2.1 Lightly Supervised Learning

An effective way to supplement small annotateddatasets is to use lightly supervised learning, inwhich the objective function is supplemented bya set of domain-specific soft-constraints over themodel’s predictions on unlabeled data. Previ-ous work in lightly-supervised learning focusedon training classifiers by using prior knowledgeof label proportions (Jin and Liu, 2005; Changet al., 2007; Musicant et al., 2007; Mann and Mc-Callum, 2007; Quadrianto et al., 2009b; Lianget al., 2009; Ganchev et al., 2010; Mann and Mc-Callum, 2010b; Chang et al., 2012; Wang et al.,2012; Zhu et al., 2014; Hope and Shahaf, 2016)or prior knowledge of features label associations(Schapire et al., 2002; Haghighi and Klein, 2006;Druck et al., 2008; Melville et al., 2009; Moham-mady and Culotta, 2015). In the context of NLP,Haghighi and Klein (2006) suggested to use dis-tributional similarities of words to train sequencemodels for part-of-speech tagging and a classi-fied ads information extraction task. Melvilleet al. (2009) used background lexical informationin terms of word-class associations to train a sen-timent classifier. Ganchev and Das (2013); Wangand Manning (2014) suggested to exploit the bilin-gual correlations between a resource rich languageand a resource poor language to train a classifier

for the resource poor language in a lightly super-vised manner.

2.2 Expectation Regularization (XR)

Expectation Regularization (XR) (Mann and Mc-Callum, 2007) is a lightly supervised learningmethod, in which the model is trained to fit theconditional probabilities of labels given features.In the context of NLP, XR was used by Moham-mady and Culotta (2015) to train twitter-user at-tribute prediction using hundreds of noisy distri-butional expectations based on census demograph-ics. Here, we suggest using XR to train a targettask (aspect-level sentiment) based on the outputof a related source-task classifier (sentence-levelsentiment).

Learning Setup The main idea of XR is mov-ing from a fully supervised situation in which eachdata-point xi has an associated label yi, to a setupin which sets of data points Uj are associated withcorresponding label proportions pj over that set.

Formally, let X = {x1, x2, . . . , xn} ⊆ X be aset of data points, Y be a set of |Y| class labels,U = {U1, U2, . . . , Um} be a set of sets whereUj ⊆ X for every j ∈ {1, 2, . . . ,m}, and letpj ∈ R|Y| be the label distribution of set Uj .For example, pj = {.7, .2, .1} would indicate that70% of data points inUj are expected to have class0, 20% are expected to have class 1 and 10% areexpected to have class 2. Let pθ(x) be a param-eterized function with parameters θ from X to avector of conditional probabilities over labels inY . We write pθ(y|x) to denote the probability as-signed to the yth event (the conditional probabilityof y given x).

A typically objective when training on fully la-beled data of (xi, yi) pairs is to maximize likeli-hood of labeled data using the cross entropy loss,

Lcross(θ) = −n∑i

log pθ(yi|xi)

Instead, in XR our data comes in the form of pairs(Uj , pj) of sets and their corresponding expectedlabel proportions, and we aim to optimize θ to fitthe label distribution pj over Uj , for all j.

XR Loss As counting the number of pre-dicted class labels over a set U leads to a non-differentiable objective, Mann and McCallum(2007) suggest to relax it and use instead the

33

model’s posterior distribution pj over the set:

qj(y) =∑x∈Uj

pθ(y|x) (1)

pj(y) =qj(y)∑y′ qj(y′)

(2)

where q(y) indicates the yth entry in q. Then, wewould like to set θ such that pj and pj are close.Mann and McCallum (2007) suggest to use KL-divergence for this. KL-divergence is composedof two parts:

DKL(pj||pj) = −pj · log pj + pj · log pj

= H(pj, pj)−H(pj)

Since H(pj) is constant, we only need to min-imize H(pj, pj), therefore the loss function be-comes:1

LXR(θ) = −m∑j=1

pj · log pj (3)

Notice that computing qj requires summationover pθ(x) for the entire set Uj , which can be pro-hibitive. We present batched approximation (Sec-tion 3.2) to overcome this.

Temperature Parameter Mann and McCallum(2007) find that XR might find a degenerate solu-tion. For example, in a three class classificationtask, where pj = {.5, .35, .15}, it might find a so-lution such that pθ(y) = {.5, .35, .15} for everyinstance, as a result, every instance will be classi-fied the same. To avoid this, Mann and McCallum(2007) suggest to penalize flat distributions by us-ing a temperature coefficient T likewise:

pθ(y|x) =(

ezW+b∑k

e(zW+b)k

) 1T

(4)

Where z is a feature vector and W and b are thelinear classifier parameters.

2.3 Aspect-based Sentiment ClassificationIn the aspect-based sentiment classification(ABSC) task, we are given a sentence and anaspect, and need to determine the sentiment thatis expressed towards the aspect. For examplethe sentence “Excellent food, although the in-terior could use some help.“ has two aspects:

1Note also that ∀j |Uj | = 1 ⇐⇒ LXR(θ) = Lcross(θ)

Algorithm 1 Stochastic Batched XRInputs: A dataset (U1, ..., Um, p1, ..., pm), batchsize k, differentiable classifier pθ(y|x)

while not converged doj ← random(1, ...,m)U ′ ← random-choice(Uj ,k)q′u ←

∑x∈U ′ pθ(x)

p′u ← normalize(q′u)`← −pj log pu . Compute loss ` (eq (4))Compute gradients and update θ

end whilereturn θ

food and interior, a positive sentiment is ex-pressed about the food, but a negative sentimentis expressed about the interior. A sentenceα = (w1, w2, . . . , wn), may contain 0 or moreaspects ai, where each aspect corresponds to asub-sequence of the original sentence, and has anassociated sentiment label (NEG, POS, or NEU).Concretely, we follow the task definition in theSemEval-2015 and SemEval-2016 shared tasks(Pontiki et al., 2015, 2016), in which the relevantaspects are given and the task focuses on findingthe sentiment label of the aspects.

While sentence-level sentiment labels are rela-tively easy to obtain, aspect-level annotation aremuch more scarce, as demonstrated in the smalldatasets of the SemEval shared tasks.

3 Technical Contributions

3.1 Transfer-training between related taskswith XR

Consider two classification tasks over a shared in-put space, a source task s from X to Ys and a tar-get task t from X to Yt, which are related througha conditional distribution P (yt = i|ys = j). Inother words, a labeling decision for task s inducesan expected label distribution over the task t. Fora set of datapoints x1, ..., xn that share a source la-bel ys, we expect to see a target label distributionof P (yt|ys) = pys .

Given a large unlabeled dataset Du =(xu1 , ..., x

u|Du|), a small labeled dataset for the tar-

get task Dt = ((xt1, yt1), ..., (x

t|Dt|, y

t|Dt|)), classi-

fier Cs : X 7→ Ys (or sufficient training data totrain one) for the source task,2 we wish to use Cs

2Note that the classifier does not need to be trainable ordifferentiable. It can be a human, a rule based system, a non-parametric model, a probabilistic model, a deep learning net-

34

and Du to train a good classifier Ct : X 7→ Ytfor the target task. This can be achieved using thefollowing procedure.

• Apply Cs to Dt, resulting in a noisy source-side labels ysi = Cs(xti) for the target task.

• Estimate the conditional probability P (yt|ys)table using MLE estimates over Dt

pj(yt = i|ys = j) =

#(yt = i, ys = j)

#(ys = j)

where # is a counting function over Dt.3

• Apply Cs to the unlabeled data Du resultingin labels Cs(xui ). Split Du into |Ys| sets Ujaccording to the labeling induced by Cs:

Uj = {xui | xui ∈ Du ∧ Cs(xui ) = j}

• Use Algorithm 1 to train a classifier for thetarget task using input pairs (Uj , pj) and theXR loss.

In words, by using XR training, we use the ex-pected label proportions over the target task givenpredicted labels of the source task, to train a target-class classifier.

3.2 Stochastic Batched Training for Deep XR

Mann and McCallum (2007) and following worktake the base classifier pθ(y|x) to be a logistic re-gression classifier, for which they manually derivegradients for the XR loss and train with LBFGs(Byrd et al., 1995). However, nothing precludesus from using an arbitrary neural network instead,as long as it culminates in a softmax layer.

One complicating factor is that the computa-tion of qj in equation (1) requires a summationover pθ(x) for the entire set Uj , which in oursetup may contain hundreds of thousands of exam-ples, making gradient computation and optimiza-tion impractical. We instead proposed a stochasticbatched approximation in which, instead of requir-ing that the full constraint setUj will match the ex-pected label posterior distribution, we require thatsufficiently large random subsets of it will match

work, etc. In this work, we use a neural classification model.3In theory, we could estimate—or even “guess”—these

|Ys|× |Yt| values without usingDt at all. In practice, and inparticular because we care about the target label proportionsgiven noisy source labels ys assigned by Cs, we use MLEestimates over the tagged Dt.

the distribution. At each training step we com-pute the loss and update the gradient with respectto a different random subset. Specifically, in eachtraining step we sample a random pair (Uj , pj),sample a random subset U ′ of Uj of size k, andcompute the local XR loss of set U ′:

LXR(θ; j, U′) = −pj · log pu′ (5)

where pu′ is computed by summing over the ele-ments of U ′ rather than of Uj in equations (1–2).The stochastic batched XR training algorithm isgiven in Algorithm 1. For large enough k, the ex-pected label distribution of the subset is the sameas that of the complete set.

4 Application to Aspect-based Sentiment

We demonstrate the procedure given aboveby training Aspect-based Sentiment Classifier(ABSC) using sentence-level4 sentiment signals.

4.1 Relating the classification tasks

We observe that while the sentence-level senti-ment does not determine the sentiment of individ-ual aspects (a positive sentence may contain nega-tive remarks about some aspects), it is very pre-dictive of the proportion of sentiment labels ofthe fragments within a sentence. Positively la-beled sentences are likely to have more positive as-pects and fewer negative ones, and vice-versa fornegatively-labeled sentences. While these propor-tions may vary on the individual sentence level,we expect them to be stable when aggregatingfragments from several sentences: when consid-ering a large enough sample of fragments that allcome from positively labeled sentences, we expectthe different samples to have roughly similar labelproportions to each other. This situation is idealysuited for performing XR training, as described insection 3.1.

The application to ABSC is almost straight-forward, but is complicated a bit by the decom-position of sentences into fragments: each sen-tence level decision now corresponds to multi-ple fragment-level decisions. Thus, we apply thesentence-level (task A) classifier Cs on the aspect-level corpus Dt by applying it on the sentencelevel and then associating the predicted sentencelabels with each of the fragments, resulting in

4In practice, our “sentences” are in fact short documents,some of which are composed of two or more sentences.

35

Figure 1: Illustration of the algorithm. Cs is applied to Du resulting in y for each sentence, Uj is built accordingwith the fragments of the same labelled sentences, the probabilities for each fragment in Uj are summed andnormalized, the XR loss in equation (4) is calculated and the network is updated.

Figure 2: Illustration of the decomposition procedure,when given a1=“duck confit“ and a2= “foie gras terrinewith figs“ as the pivot phrases.

fragment-level labeling. Similarly, when we ap-ply Cs to the unlabeled data Du we again do it atthe sentence level, but the sets Uj are composed offragments, not sentences:

Uj = {fαi | α ∈ Du∧fαi ∈ frags(α)∧Cs(α) = j}

We then apply algorithm 1 as is: at eachstep of training we sample a source label j ∈{POS,NEG,NEU}, sample k fragments from Uj ,and use the XR loss to fit the expected fragment-label proportions over these k fragments to pj.Figure 1 illustrates the procedure.

4.2 Classification ArchitectureWe model the ABSC problem by associating each(sentence,aspect) pair with a sentence-fragment,and constructing a neural classifier from fragments

to sentiment labels. We heuristically decompose asentence into fragments. We use the same BiL-STM based neural architecture for both sentenceclassification and fragment classification.

Fragment-decomposition We now describe theprocedure we use to associate a sentence fragmentwith each (sentence,aspect) pairs. The sharedtasks data associates each aspect with a pivot-phrase a, where pivot phrase (w1, w2, ...wl) is de-fined as a pre-determined sequence of words thatis contained within the sentence. For a sentence α,a set of pivot phrases A = (a1, ..., am) and a spe-cific pivot phrase ai, we consult the constituencyparse tree of α and look for tree nodes that satisfythe following conditions:5

1. The node governs the desired pivot phrase ai.

2. The node governs either a verb (VB, VBD,VBN, VBG, VBP, VBZ) or an adjective (JJ,JJR, JJS), which is different than any aj ∈ A.

3. The node governs a minimal number of pivotphrases from (a1, ..., am), ideally only ai.

We then select the highest node in the tree thatsatisfies all conditions. The span governed by thisnode is taken as the fragment associated with as-

5Condition (2) coupled with selecting the highest nodepushes towards complete phrases that contain opinions(which are usually expressed with adjectives or verbs), whilethe other conditions focus the attention on the desired pivotphrase.

36

pect ai.6 The decomposition procedure is demon-strated in Figure 2.

When aspect-level information is given, wetake the pivot-phrases to be the requested aspects.When aspect-level information is not available,we take each noun in the sentence to be a pivot-phrase.

Neural Classifier Our classification model is asimple 1-layer BiLSTM encoder (a concatenationof the last states of a forward and a backward run-ning LSTMs) followed by a linear-predictor. Theencoder is fed either a complete sentence or a sen-tence fragment.

5 Experiments

Data Our target task is aspect-based fragment-classification, with small labeled datasets from theSemEval 2015 and 2016 shared tasks, each datasetcontaining aspect-level predictions for about 2000sentences in the restaurants reviews domain. Oursource classifier is based on training on up to10,000 sentences from the same domain and 2000sentences for validation, labeled for only forsentence-level sentiment. We additionally have anunlabeled dataset of up to 670,000 sentences fromthe same domain7. We tokenized all datasets us-ing the Tweet Tokenizer from NLTK package8 andparsed the tokenized sentences with AllenNLPparser.9

Training Details Both the sentence level classi-fication models and the models trained with XRhave a hidden state vector dimension of size 300,they use dropout (Hinton et al., 2012) on thesentence representation or fragment representa-tion vector (rate=0.5) and optimized using Adam(Kingma and Ba, 2014). The sentence classifica-tion is trained with a batch size of 30 and XR mod-els are trained with batch sizes k that each contain450 fragments10. We used a temperature param-

6On the rare occasions where we cannot find such a node,we take the root node of the tree (the entire sentence) as thefragment for the given aspect.

7All of the sentence-level sentiment data is obtained fromthe Yelp dataset challenge: https://www.yelp.com/dataset/challenge

8https://www.nltk.org/9https://allennlp.org/

10We also increased the batch sizes of the baselines tomatch those of the XR setups. This decreased the perfor-mance of the baselines, which is consistent with the folkknowledge in the community according to which smallerbatch sizes are more effective overall.

eter of 111. We use pre-trained 300-dimensionalGloVe embeddings12 (Pennington et al., 2014),and fine-tune them during training. The XR train-ing was validated with a validation set of 20% ofSemEval-2015 training set, the sentence level BiL-STM classifiers were validated with a validation of2000 sentences.13 When fine-tuning to the aspectbased task we used 20% of train in each dataset asvalidation and evaluated on this set. On each train-ing method the models were evaluated on the vali-dation set, after each epoch and the best model waschosen. The data is highly imbalanced, with onlyvery few sentences receiving a NEU label. We donot deal with this imbalance directly and train boththe sentence level and the XR aspect-based train-ing on the imbalanced data. However, when train-ing Cs, we trained five models and chose the bestmodel that predicts correctly at least 20% of theneutral sentences. The models are implementedusing DyNet14 (Neubig et al., 2017).

Baseline models In recent years many neuralnetwork architectures with increasing sophistica-tion were applied to the ABSC task (Nguyen andShirai, 2015; Vo and Zhang, 2015; Tang et al.,2016a,b; Wang et al., 2016; Zhang et al., 2016;Ruder et al., 2016; Ma et al., 2017; Liu andZhang, 2017; Chen et al., 2017; Liu et al., 2018;Yang et al., 2018; Wang et al., 2018b,a; Fanet al., 2018a,b; Li et al., 2018; Ouyang and Su,2018). We compare to a series of state-of-the-art ABSC neural classifiers that participated inthe shared tasks. TDLSTM-ATT (Tang et al.,2016a) encodes the information around an aspectusing forward and backward LSTMs, followedby an attention mechanism. ATAE-LSTM (Wanget al., 2016) is an attention based LSTM vari-ant. MM (Tang et al., 2016b) is a deep mem-ory network with multiple-hops of attention lay-ers. RAM (Chen et al., 2017) uses multiple atten-tion mechanisms combined with a recurrent neu-ral networks and a weighted memory mechanism.LSTM+SynATT+TarRep (He et al., 2018a) is anattention based LSTM which incorporates syn-

11Despite (Mann and McCallum, 2007) claim regardingthe temperature parameter, we observed lower performancewhen using it in our setup. However, in other setups this pa-rameter might be found to be beneficial.

12https://nlp.stanford.edu/projects/glove/

13We also tested the sentence BiLSTM baselines with aSemEval validation set, and received slightly lower resultswithout a significant statistical difference.

14https://github.com/clab/dynet

37

Data Method SemEval-15 SemEval-16Acc. Macro-F1 Acc. Macro-F1

A TDLSTM+ATT (Tang et al., 2016a) 77.10 59.46 83.11 57.53A ATAE-LSTM (Wang et al., 2016) 78.48 62.84 83.77 61.71A MM (Tang et al., 2016b) 77.89 59.52 83.04 57.91A RAM (Chen et al., 2017) 79.98 60.57 83.88 62.14A LSTM+SynATT+TarRep (He et al., 2018a) 81.67 66.05 84.61 67.45S+A Semisupervised (He et al., 2018b) 81.30 68.74 85.58 69.76S BiLSTM-104 Sentence Training 80.24 ±1.64 61.89 ±0.94 80.89 ±2.79 61.40 ±2.49S+A BiLSTM-104 Sentence Training →Aspect Based Finetuning 77.75 ±2.09 60.83 ±4.53 84.87±0.31 61.87 ±5.44N BiLSTM-XR-Dev Estimation 83.31∗ ± 0.62 62.24 ±0.66 87.68∗ ± 0.47 63.23 ±1.81N BiLSTM-XR 83.31∗ ± 0.77 64.42 ± 2.78 88.12∗ ± 0.24 68.60 ±1.79N+A BiLSTM-XR →Aspect Based Finetuning 83.44∗ ± 0.74 67.23 ± 1.42 87.66∗ ± 0.28 71.19†± 1.40

Table 1: Average accuracies and Macro-F1 scores over five runs with random initialization along with their stan-dard deviations. Bold: best results or within std of them. ∗ indicates that the method’s result is significantly betterthan all baseline methods, † indicates that the method’s result is significantly better than all baselines methods thatuse the aspect-based data only, with p < 0.05 according to a one-tailed unpaired t-test. The data annotations S, Nand A indicate training with Sentence-level, Noisy sentence-level and Aspect-level data respectively. Numbers forTDLSTM+Att,ATAE-LSTM,MM,RAM and LSTM+SynATT+TarRep are from (He et al., 2018a). Numbers forSemisupervised are from (He et al., 2018b).

tactic information into the attention mechanismand uses an auto-encoder structure to produce anaspect representations. All of these models aretrained only on the small, fully-supervised ABSCdatasets.

“Semisupervised” is the semi-supervised setupof (He et al., 2018b), it train an attention-based LSTM model on 30,000 documents addi-tional to an aspect-based train set, 10,000 doc-uments to each class. We consider additionaltwo simple but strong semi-supervised baselines.Sentence-BiLSTM is our BiLSTM model trainedon the 104 sentence-level annotations, and ap-plied as-is to the individual fragments. Sentence-BiLSTM+Finetuning is the same model, but fine-tuned on the aspect-based data as explained above.Finetuning is performed using our own implemen-tation of the attention-based model of He et al.(2018b).15 Both these models are on par with thefully-supervised ABSC models.

Empirical Proportions The proportion con-straint sets pj based on the SemEval-2015aspect-based train data are:pPOS = {POS : 0.93,NEG : 0.06,NEU : 0.01}pNEG = {POS : 0.27,NEG : 0.7,NEU : 0.03}pNEU = {POS : 0.45,NEG : 0.41,NEU : 0.14}

5.1 Main Results

Table 1 compares these baselines to three XR con-ditions.16

15We changed the LSTM component to a BiLSTM.16To be consistent with existing research (He et al., 2018b),

aspects with conflicted polarity are removed.

The first condition, BiLSTM-XR-Dev, per-forms XR training on the automatically-labeledsentence-level dataset. The only access it hasto aspect-level annotation is for estimating theproportions of labels for each sentence-level la-bel, which is done based on the validation set ofSemEval-2015 (i.e., 20% of the train set). The XRsetting is very effective: without using any in-taskdata, this model already surpasses all other mod-els, both supervised and semi-supervised, exceptfor the (He et al., 2018b,a) models which achievehigher F1 scores. We note that in contrast to XR,the competing models have complete access to thesupervised aspect-based labels. The second con-dition, BiLSTM-XR, is similar but now the modelis allowed to estimate the conditional label pro-portions based on the entire aspect-based trainingset (the classifier still does not have direct accessto the labels beyond the aggregate proportion in-formation). This improves results further, show-ing the importance of accurately estimating theproportions. Finally, in BiLSTM-XR+Finetuning,we follow the XR training with fully supervisedfine-tuning on the small labeled dataset, using theattention-based model of He et al. (2018b). Thisachieves the best results, and surpasses also thesemi-supervised He et al. (2018b) baseline on ac-curacy, and matching it on F1.17

We report significance tests for the robustness

17We note that their setup uses clean and more balancedannotations, i.e. they use 10,000 samples for each label,which helps predicting the infrequent neutral sentiment. Wehowever, use noisy sentence sentiment labels which are auto-matically obtained from a trained classifier, which trains on10,000 samples in their natural imbalanced distribution.

38

of the method under random parameter initializa-tion. Our reported numbers are averaged over fiverandom initialization. Since the datasets are un-balanced w.r.t the label distribution, we report bothaccuracy and macro-F1.

The XR training is also more stable than theother semi-supervised baselines, achieving sub-stantially lower standard deviations across differ-ent runs.

5.2 Further experiments

In each experiment in this section we estimate theproportions using the SemEval-2015 train set.

Effect of unlabeled data size How does the XRtraining scale with the amount of unlabeled data?Figure 3a shows the macro-F1 scores on the entireSemEval-2016 dataset, with different unlabeledcorpus sizes (measured in number of sentences).An unannotated corpus of 5×104 sentences is suf-ficient to surpass the results of the 104 sentence-level trained classifier, and more unannotated datafurther improves the results.

Effect of Base-classifier Quality Our methodrequires a sentence level classifier Cs to label boththe target-task corpus and the unlabeled corpus.How does the quality of this classifier affect theoverall XR training? We vary the amount of su-pervision used to train Cs from 0 sentences (as-signing the same label to all sentences), to 100,1000, 5000 and 10000 sentences. We again mea-sure macro-F1 on the entire SemEval 2016 corpus.The results in Figure 3b show that when using theprior distributions of aspects (0), the model strug-gles to learn from this signal, it learns mostly topredict the majority class, and hence reaches verylow F1 scores of 35.28. The more data given to thesentence level classifier, the better the potential re-sults will be when training with our method usingthe classifier labels, with a classifiers trained on100,1000,5000 and 10000 labeled sentences, weget a F1 scores of 53.81, 58.84, 61.81, 65.58 re-spectively. Improvements in the source task clas-sifier’s quality clearly contribute to the target taskaccuracy.

Effect of k The Stochastic Batched XR algo-rithm (Algorithm 1) samples a batch of k examplesat each step to estimate the posterior label distri-bution used in the loss computation. How does thesize of k affect the results? We use k = 450 frag-ments in our main experiments, but smaller values

of k reduce GPU memory load and may train bet-ter in practice. We tested our method with varyingvalues of k on a sample of 5× 104, using batchesthat are composed of fragments of 5, 25, 100, 450,1000 and 4500 sentences. The results are shownin Figure 3c. Setting k = 5 result in low scores.Setting k = 25 yields better F1 score but with highvariance across runs. For k = 100 fragments theresults begin to stabilize, we also see a slight de-crease in F1-scores with larger batch sizes. Weattribute this drop despite having better estimationof the gradients to the general trend of larger batchsizes being harder to train with stochastic gradientmethods.

5.3 Pre-training, BERT

The XR training can be performed also over pre-trained representations. We experiment with twopre-training methods: (1) pre-training by trainingthe BiLSTM model to predict the noisy sentence-level predictions. (2) Using the pre-trained BERT

representation (Devlin et al., 2018). For (1), wecompare the effect of pre-train on unlabeled cor-pora of sizes of 5 × 104, 105 and 6.7 × 105 sen-tences. Results in Figure 3d show that this form ofpre-training is effective for smaller unlabeled cor-pora but evens out for larger ones.

BERT For the BERT experiments, we experi-ment with the BERT-base model18 with k = 450sets, 30 epochs for XR training or sentence levelfine-tuning19 and 15 epochs for aspect based fine-tuning, on each training method we evaluated themodel on the dev set after each epoch and the bestmodel was chosen20. We compare the followingsetups:-BERT→Aspect Based Finetuning: pretrainedBERT model finetuned to the aspect based task.-BERT→ 104: A pretrained BERT model finetunedto the sentence level task on the 104 sentences, andtested by predicting fragment-level sentiment.-BERT→104→Aspect Based Finetuning: pre-trained BERT model finetuned to the sentence leveltask, and finetuned again to the aspect based one.-BERT→XR: pretrained BERT model followed by

18We could not fit k = 450 sets of BERT-large on ourGPU.

19When fine-tuning to the sentence level task, we providethe sentence as input. When fine-tuning to the aspect-leveltask, we provide the sentence, a seperator and then the aspect.

20The other configuration parameters were the de-fault ones in https://github.com/huggingface/pytorch-pretrained-BERT

39

(a) (b) (c) (d)

Figure 3: Macro-F1 scores for the entire SemEval-2016 dataset of the different analyses. (a) the contribution ofunlabeled data. (b) the effect of sentence classifier quality. (c) the effect of k. (d) the effect of sentence-levelpretraining vs. corpus size.

Data Training SemEval-15 SemEval-16Acc. Macro-F1 Acc. Macro-F1

N BiLSTM-XR 83.31 ± 0.77 64.42 ± 2.78 88.12 ± 0.24 68.60 ±1.79N+A BiLSTM-XR →Aspect Based Finetuning 83.44 ± 0.74 67.23 ± 1.42 87.66 ± 0.28 71.19 ± 1.40A BERT→Aspect Based Finetuning 81.87 ±1.12 59.24 ±4.94 85.81 ±1.07 62.46 ±6.76S BERT→104 Sent Finetuning 83.29 ±0.77 66.79 ±1.99 84.53 ±1.66 65.53 ±3.03S+A BERT→104 Sent Finetuning →Aspect Based Finetuning 82.54 ±1.21 64.13 ±5.05 85.67 ±1.14 64.13 ±7.07N BERT→XR 85.46∗ ±0.59 66.86 ±2.8 89.5∗ ±0.55 70.86†±2.96N+A BERT→XR →Aspect Based Finetuning 85.78∗ ± 0.65 68.74 ± 1.36 89.57∗ ± 1.4 73.89∗ ± 2.05

Table 2: BERT pre-training: average accuracies and Macro-F1 scores from five runs and their stdev. ∗ indicatesthat the method’s result is significantly better than all baseline methods, † indicates that the method’s result issignificantly better than all non XR baseline methods, with p < 0.05 according to a one-tailed unpaired t-test.The data annotations S, N and A indicate training with Sentence-level, Noisy sentence-level and Aspect-level datarespectively.

XR training using our method.-BERT→ XR → Aspect Based Finetuning: pre-trained BERT followed by XR training and thenfine-tuned to the aspect level task.

The results are presented in Table 2. As before,aspect-based fine-tuning is beneficial for bothSemEval-16 and SemEval-15. Training a BiL-STM with XR surpasses pre-trained BERT modelsand using XR training on top of the pre-trainedBERT models substantially increases the resultseven further.

6 Discussion

We presented a transfer learning method basedon expectation regularization (XR), and demon-strated its effectiveness for training aspect-basedsentiment classifiers using sentence-level supervi-sion. The method achieves state-of-the-art resultsfor the task, and is also effective for improving ontop of a strong pre-trained BERT model. The pro-posed method provides an additional data-efficienttool in the modeling arsenal, which can be ap-plied on its own or together with another trainingmethod, in situations where there is a conditional

relations between the labels of a source task forwhich we have supervision, and a target task forwhich we don’t.

While we demonstrated the approach on thesentiment domain, the required conditional depen-dence between task labels is present in many sit-uations. Other possible application of the methodincludes training language identification of tweetsgiven geo-location supervision (knowing the geo-graphical region gives a prior on languages spo-ken), training predictors for renal failure from tex-tual medical records given classifier for diabetes(there is a strong correlation between the two con-ditions), training a political affiliation classifierfrom social media tweets based on age-group clas-sifiers, zip-code information, or social-status clas-sifiers (there are known correlations between all ofthese to political affiliation), training hate-speechdetection based on emotion detection, and so on.

Acknowledgements

The work was supported in part by The Israeli Sci-ence Foundation (grant number 1555/15).

40

ReferencesRichard H. Byrd, Peihuang Lu, Jorge Nocedal, and

Ciyou Zhu. 1995. A limited memory algorithm forbound constrained optimization. SIAM J. ScientificComputing, 16(5):1190–1208.

Ming-Wei Chang, Lev Ratinov, and Dan Roth. 2007.Guiding semi-supervision with constraint-drivenlearning. In Proceedings of the 45th Annual Meet-ing of the Association of Computational Linguistics,pages 280–287. Association for Computational Lin-guistics.

Ming-Wei Chang, Lev-Arie Ratinov, and Dan Roth.2012. Structured learning with constrained condi-tional models. Machine Learning, 88(3):399–431.

Peng Chen, Zhongqian Sun, Lidong Bing, and WeiYang. 2017. Recurrent attention network on mem-ory for aspect sentiment analysis. In Proceedings ofthe 2017 Conference on Empirical Methods in Nat-ural Language Processing, pages 452–461. Associ-ation for Computational Linguistics.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2018. Bert: Pre-training of deepbidirectional transformers for language understand-ing. CoRR, abs/1810.04805.

Gregory Druck, Gideon S. Mann, and Andrew McCal-lum. 2008. Learning from labeled features usinggeneralized expectation criteria. In Proceedings ofthe 31st Annual International ACM SIGIR Confer-ence on Research and Development in InformationRetrieval, SIGIR 2008, Singapore, July 20-24, 2008,pages 595–602.

Chuang Fan, Qinghong Gao, Jiachen Du, LinGui, Ruifeng Xu, and Kam-Fai Wong. 2018a.Convolution-based memory network for aspect-based sentiment analysis. In The 41st InternationalACM SIGIR Conference on Research & Develop-ment in Information Retrieval, SIGIR 2018, Ann Ar-bor, MI, USA, July 08-12, 2018, pages 1161–1164.

Feifan Fan, Yansong Feng, and Dongyan Zhao. 2018b.Multi-grained attention network for aspect-levelsentiment classification. In Proceedings of the 2018Conference on Empirical Methods in Natural Lan-guage Processing, pages 3433–3442. Associationfor Computational Linguistics.

Kuzman Ganchev and Dipanjan Das. 2013. Cross-lingual discriminative learning of sequence modelswith posterior regularization. In Proceedings of the2013 Conference on Empirical Methods in NaturalLanguage Processing, pages 1996–2006. Associa-tion for Computational Linguistics.

Kuzman Ganchev, Joao Graca, Jennifer Gillenwater,and Ben Taskar. 2010. Posterior regularization forstructured latent variable models. Journal of Ma-chine Learning Research, 11:2001–2049.

Joao Graca, Kuzman Ganchev, and Ben Taskar. 2007.Expectation maximization and posterior constraints.In Advances in Neural Information Processing Sys-tems 20, Proceedings of the Twenty-First AnnualConference on Neural Information Processing Sys-tems, Vancouver, British Columbia, Canada, De-cember 3-6, 2007, pages 569–576.

Aria Haghighi and Dan Klein. 2006. Prototype-drivenlearning for sequence models. In Human Lan-guage Technology Conference of the North Amer-ican Chapter of the Association of ComputationalLinguistics, Proceedings, June 4-9, 2006, New York,New York, USA.

Ruidan He, Wee Sun Lee, Hwee Tou Ng, and DanielDahlmeier. 2018a. Effective attention modeling foraspect-level sentiment classification. In Proceed-ings of the 27th International Conference on Com-putational Linguistics, pages 1121–1131. Associa-tion for Computational Linguistics.

Ruidan He, Wee Sun Lee, Hwee Tou Ng, and DanielDahlmeier. 2018b. Exploiting document knowledgefor aspect-level sentiment classification. In Pro-ceedings of the 56th Annual Meeting of the Associa-tion for Computational Linguistics (Volume 2: ShortPapers), pages 579–585. Association for Computa-tional Linguistics.

Geoffrey E. Hinton, Nitish Srivastava, AlexKrizhevsky, Ilya Sutskever, and Ruslan Salakhut-dinov. 2012. Improving neural networks bypreventing co-adaptation of feature detectors.CoRR, abs/1207.0580.

Tom Hope and Dafna Shahaf. 2016. Ballpark learn-ing: Estimating labels from rough group compar-isons. In Machine Learning and Knowledge Dis-covery in Databases - European Conference, ECMLPKDD 2016, Riva del Garda, Italy, September 19-23, 2016, Proceedings, Part II, pages 299–314.

Rong Jin and Yi Liu. 2005. A framework for in-corporating class priors into discriminative classifi-cation. In Advances in Knowledge Discovery andData Mining, 9th Pacific-Asia Conference, PAKDD2005, Hanoi, Vietnam, May 18-20, 2005, Proceed-ings, pages 568–577.

Diederik P. Kingma and Jimmy Ba. 2014. Adam:A method for stochastic optimization. CoRR,abs/1412.6980.

Lishuang Li, Yang Liu, and AnQiao Zhou. 2018. Hi-erarchical attention based position-aware networkfor aspect-level sentiment analysis. In Proceedingsof the 22nd Conference on Computational NaturalLanguage Learning, pages 181–189. Association forComputational Linguistics.

Percy Liang, Michael I. Jordan, and Dan Klein. 2009.Learning from measurements in exponential fam-ilies. In Proceedings of the 26th Annual Inter-national Conference on Machine Learning, ICML

41

2009, Montreal, Quebec, Canada, June 14-18, 2009,pages 641–648.

Fei Liu, Trevor Cohn, and Timothy Baldwin. 2018. Re-current entity networks with delayed memory updatefor targeted aspect-based sentiment analysis. In Pro-ceedings of the 2018 Conference of the North Amer-ican Chapter of the Association for ComputationalLinguistics: Human Language Technologies, Vol-ume 2 (Short Papers), pages 278–283. Associationfor Computational Linguistics.

Jiangming Liu and Yue Zhang. 2017. Attention mod-eling for targeted sentiment. In Proceedings of the15th Conference of the European Chapter of the As-sociation for Computational Linguistics: Volume 2,Short Papers, pages 572–577. Association for Com-putational Linguistics.

Dehong Ma, Sujian Li, Xiaodong Zhang, and HoufengWang. 2017. Interactive attention networks foraspect-level sentiment classification. In Proceed-ings of the Twenty-Sixth International Joint Con-ference on Artificial Intelligence, IJCAI 2017, Mel-bourne, Australia, August 19-25, 2017, pages 4068–4074.

Gideon S. Mann and Andrew McCallum. 2007. Sim-ple, robust, scalable semi-supervised learning viaexpectation regularization. In Machine Learn-ing, Proceedings of the Twenty-Fourth InternationalConference (ICML 2007), Corvallis, Oregon, USA,June 20-24, 2007, pages 593–600.

Gideon S. Mann and Andrew McCallum. 2010a. Gen-eralized expectation criteria for semi-supervisedlearning with weakly labeled data. Journal of Ma-chine Learning Research, 11:955–984.

Gideon S. Mann and Andrew McCallum. 2010b. Gen-eralized expectation criteria for semi-supervisedlearning with weakly labeled data. Journal of Ma-chine Learning Research, 11:955–984.

Prem Melville, Wojciech Gryc, and Richard D.Lawrence. 2009. Sentiment analysis of blogs bycombining lexical knowledge with text classifica-tion. In Proceedings of the 15th ACM SIGKDDInternational Conference on Knowledge Discoveryand Data Mining, Paris, France, June 28 - July 1,2009, pages 1275–1284.

Ardehaly Ehsan Mohammady and Aron Culotta. 2015.Inferring latent attributes of twitter users with labelregularization. In Proceedings of the 2015 Confer-ence of the North American Chapter of the Associ-ation for Computational Linguistics: Human Lan-guage Technologies, pages 185–195. Association forComputational Linguistics.

David R. Musicant, Janara M. Christensen, andJamie F. Olson. 2007. Supervised learning by train-ing on aggregate outputs. In Proceedings of the7th IEEE International Conference on Data Mining(ICDM 2007), October 28-31, 2007, Omaha, Ne-braska, USA, pages 252–261.

Graham Neubig, Chris Dyer, Yoav Goldberg, AustinMatthews, Waleed Ammar, Antonios Anastasopou-los, Miguel Ballesteros, David Chiang, DanielClothiaux, Trevor Cohn, Kevin Duh, ManaalFaruqui, Cynthia Gan, Dan Garrette, Yangfeng Ji,Lingpeng Kong, Adhiguna Kuncoro, Gaurav Ku-mar, Chaitanya Malaviya, Paul Michel, YusukeOda, Matthew Richardson, Naomi Saphra, SwabhaSwayamdipta, and Pengcheng Yin. 2017. Dynet:The dynamic neural network toolkit. CoRR,abs/1701.03980.

Thien Hai Nguyen and Kiyoaki Shirai. 2015.Phrasernn: Phrase recursive neural network foraspect-based sentiment analysis. In Proceedings ofthe 2015 Conference on Empirical Methods in Nat-ural Language Processing, pages 2509–2514. Asso-ciation for Computational Linguistics.

Zhifan Ouyang and Jindian Su. 2018. Dependencyparsing and attention network for aspect-level sen-timent classification. In Natural Language Process-ing and Chinese Computing - 7th CCF InternationalConference, NLPCC 2018, Hohhot, China, August26-30, 2018, Proceedings, Part I, pages 391–403.

Jeffrey Pennington, Richard Socher, and ChristopherManning. 2014. Glove: Global vectors for wordrepresentation. In Proceedings of the 2014 Con-ference on Empirical Methods in Natural LanguageProcessing (EMNLP), pages 1532–1543. Associa-tion for Computational Linguistics.

Maria Pontiki, Dimitris Galanis, Haris Papageorgiou,Ion Androutsopoulos, Suresh Manandhar, Moham-mad AL-Smadi, Mahmoud Al-Ayyoub, YanyanZhao, Bing Qin, Orphee De Clercq, VeroniqueHoste, Marianna Apidianaki, Xavier Tannier, Na-talia Loukachevitch, Evgeniy Kotelnikov, Nuria Bel,Salud Marıa Jimenez-Zafra, and Gulsen Eryigit.2016. Semeval-2016 task 5: Aspect based senti-ment analysis. In Proceedings of the 10th Interna-tional Workshop on Semantic Evaluation (SemEval-2016), pages 19–30. Association for ComputationalLinguistics.

Maria Pontiki, Dimitris Galanis, Haris Papageorgiou,Suresh Manandhar, and Ion Androutsopoulos. 2015.Semeval-2015 task 12: Aspect based sentiment anal-ysis. In Proceedings of the 9th International Work-shop on Semantic Evaluation (SemEval 2015), pages486–495. Association for Computational Linguis-tics.

Novi Quadrianto, Alexander J. Smola, Tiberio S. Cae-tano, and Quoc V. Le. 2009a. Estimating labels fromlabel proportions. Journal of Machine Learning Re-search, 10:2349–2374.

Novi Quadrianto, Alexander J. Smola, Tiberio S. Cae-tano, and Quoc V. Le. 2009b. Estimating labels fromlabel proportions. Journal of Machine Learning Re-search, 10:2349–2374.

42

Sebastian Ruder, Parsa Ghaffari, and John G. Breslin.2016. A hierarchical model of reviews for aspect-based sentiment analysis. In Proceedings of the2016 Conference on Empirical Methods in NaturalLanguage Processing, pages 999–1005. Associationfor Computational Linguistics.

Robert E. Schapire, Marie Rochery, Mazin G. Rahim,and Narendra K. Gupta. 2002. Incorporating priorknowledge into boosting. In Machine Learning,Proceedings of the Nineteenth International Confer-ence (ICML 2002), University of New South Wales,Sydney, Australia, July 8-12, 2002, pages 538–545.

Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu.2016a. Effective lstms for target-dependent sen-timent classification. In Proceedings of COLING2016, the 26th International Conference on Compu-tational Linguistics: Technical Papers, pages 3298–3307. The COLING 2016 Organizing Committee.

Duyu Tang, Bing Qin, and Ting Liu. 2016b. Aspectlevel sentiment classification with deep memory net-work. In Proceedings of the 2016 Conference onEmpirical Methods in Natural Language Process-ing, pages 214–224. Association for ComputationalLinguistics.

Duy-Tin Vo and Yue Zhang. 2015. Target-dependenttwitter sentiment classification with rich automaticfeatures. In Proceedings of the Twenty-Fourth Inter-national Joint Conference on Artificial Intelligence,IJCAI 2015, Buenos Aires, Argentina, July 25-31,2015, pages 1347–1353.

Jingjing Wang, Jie Li, Shoushan Li, Yangyang Kang,Min Zhang, Luo Si, and Guodong Zhou. 2018a. As-pect sentiment classification with both word-leveland clause-level attention networks. In Proceed-ings of the Twenty-Seventh International Joint Con-ference on Artificial Intelligence, IJCAI 2018, July13-19, 2018, Stockholm, Sweden., pages 4439–4445.

Mengqiu Wang and Christopher D. Manning. 2014.Cross-lingual projected expectation regularizationfor weakly supervised learning. TACL, 2:55–66.

Shuai Wang, Sahisnu Mazumder, Bing Liu, MianweiZhou, and Yi Chang. 2018b. Target-sensitive mem-ory networks for aspect sentiment classification. InProceedings of the 56th Annual Meeting of the As-sociation for Computational Linguistics (Volume 1:Long Papers), pages 957–967. Association for Com-putational Linguistics.

Yequan Wang, Minlie Huang, xiaoyan zhu, andLi Zhao. 2016. Attention-based lstm for aspect-levelsentiment classification. In Proceedings of the 2016Conference on Empirical Methods in Natural Lan-guage Processing, pages 606–615. Association forComputational Linguistics.

Zuoguan Wang, Siwei Lyu, Gerwin Schalk, and QiangJi. 2012. Learning with target prior. In Advancesin Neural Information Processing Systems 25: 26th

Annual Conference on Neural Information Process-ing Systems 2012. Proceedings of a meeting heldDecember 3-6, 2012, Lake Tahoe, Nevada, UnitedStates., pages 2240–2248.

Min Yang, Qiang Qu, Xiaojun Chen, Chaoxue Guo,Ying Shen, and Kai Lei. 2018. Feature-enhanced at-tention network for target-dependent sentiment clas-sification. Neurocomputing, 307:91–97.

Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2016.Gated neural networks for targeted sentiment anal-ysis. In Proceedings of the Thirtieth AAAI Con-ference on Artificial Intelligence, February 12-17,2016, Phoenix, Arizona, USA., pages 3087–3093.

Jun Zhu, Ning Chen, and Eric P. Xing. 2014. Bayesianinference with posterior regularization and applica-tions to infinite latent svms. Journal of MachineLearning Research, 15(1):1799–1847.