s3-rec: self-supervised learning for sequential ...existing neural sequential recom-mendation models...

S3-Rec: Self-Supervised Learning for SequentialRecommendation with Mutual Information Maximization

Kun Zhou1†, Hui Wang1†, Wayne Xin Zhao2,3∗ , Yutao Zhu5, Sirui Wang4,Fuzheng Zhang4, Zhongyuan Wang4 and Ji-Rong Wen2,3

1School of Information, Renmin University of China2Gaoling School of Artificial Intelligence, Renmin University of China

3Beijing Key Laboratory of Big Data Management and Analysis Methods4Meituan-Dianping Group

5UniversitÃľ de MontrÃľal, MontrÃľal, QuÃľbec, [email protected], {hui.wang, batmanfly, jrwen}@ruc.edu.cn, [email protected],

[email protected], [email protected]

ABSTRACTRecently, significant progress has been made in sequential recom-mendation with deep learning. Existing neural sequential recom-mendation models usually rely on the item prediction loss to learnmodel parameters or data representations. However, the modeltrained with this loss is prone to suffer from data sparsity problem.Since it overemphasizes the final performance, the association orfusion between context data and sequence data has not been wellcaptured and utilized for sequential recommendation.

To tackle this problem, we propose the model S3-Rec, whichstands for Self-Supervised learning for Sequential Recommendation,based on the self-attentive neural architecture. The main idea ofour approach is to utilize the intrinsic data correlation to deriveself-supervision signals and enhance the data representations viapre-training methods for improving sequential recommendation.For our task, we devise four auxiliary self-supervised objectivesto learn the correlations among attribute, item, subsequence, andsequence by utilizing the mutual information maximization (MIM)principle. MIM provides a unified way to characterize the correla-tion between different types of data, which is particularly suitablein our scenario. Extensive experiments conducted on six real-worlddatasets demonstrate the superiority of our proposed method overexisting state-of-the-art methods, especially when only limitedtraining data is available. Besides, we extend our self-supervisedlearning method to other recommendation models, which also im-prove their performance.

CCS CONCEPTS• Information systems→ Recommender systems.

†Equal contribution.∗Corresponding author.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected] ’20, October 19–23, 2020, Virtual Event, Ireland© 2020 Association for Computing Machinery.ACM ISBN 978-1-4503-6859-9/20/10. . . $15.00https://doi.org/10.1145/3340531.3411954

KEYWORDSSelf-Supervised Learning, Sequential Recommendation, Mutual In-formation Maximization

ACM Reference Format:Kun Zhou1†, Hui Wang1†, Wayne Xin Zhao2,3∗ , Yutao Zhu5, Sirui Wang4,and Fuzheng Zhang4, Zhongyuan Wang4 and Ji-Rong Wen2,3. 2020. S3-Rec: Self-Supervised Learning for Sequential Recommendation with MutualInformation Maximization. In The 29th ACM International Conference onInformation and Knowledge Management (CIKM ’20), October 19–23, 2020,Virtual Event, Ireland. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3340531.3411954

1 INTRODUCTIONRecent years have witnessed the great success of many online plat-forms, such as Amazon and Taobao. Within online platforms, users’behaviors are dynamic and evolving over time. Thus it is criticalto capture the dynamics of sequential user behaviors for makingappropriate recommendations. In order to accurately character-ize user interests and provide high-quality recommendations, thetask of sequential recommendation has been widely studied in theliterature [3, 8, 20, 21, 24].

Typically, sequential recommendation methods [3, 8, 21, 24] cap-ture useful sequential patterns from users’ historical behaviors.Such motivation has been extensively explored with deep learning.Various methods using recurrent neural networks (RNNs) [3], con-volutional neural networks (CNNs) [24], and self-attention mecha-nisms [8] have been proposed to learn good representations of userpreference and characterize sequential user-item interactions.

Furthermore, researchers have incorporated rich contextual in-formation (such as item attributes) to neural sequential recom-menders [4, 6, 29]. It has been demonstrated that contextual infor-mation is important to consider for improving the performance ofsequential recommender systems.

Although existing methods have been shown effective to someextent, there are two major shortcomings that are likely to affectthe recommendation performance. First, they rely on the item pre-diction loss to learn the entire model. When context data is incor-porated, the involved parameters are also learned through the onlyoptimization objective. It has been found that such an optimiza-tion way is easy to suffer from issues such as data sparsity [21, 22].

arX

iv:2

008.

0787

3v1

[cs

.IR

] 1

8 A

ug 2

020

https://doi.org/10.1145/3340531.3411954

https://doi.org/10.1145/3340531.3411954

https://doi.org/10.1145/3340531.3411954

Second, they overemphasize the final performance, while the asso-ciation or fusion between context data and sequence data has notbeen well captured in data representations. As shown in increasingevidence from various fields [1, 5, 10], effective data representa-tion (e.g., pre-trained contextualized embedding) has been a keyfactor to improve the performance of existing models or architec-tures. Therefore, there is a need to rethink the learning paradigmto develop more effective sequential recommender systems.

To address the above issues, we borrow the idea of self-supervisedlearning for improving sequential recommendation. Self-supervisedlearning [1, 15] is a newly emerging paradigm, which aims to let themodel learn from the intrinsic structure of the raw data. A generalframework of self-supervised learning is to first construct trainingsignals directly from the raw data and then pre-train the modelparameters with additionally devised optimization objectives. Aspreviously discussed, limited supervision signals and ineffectivedata representations are the two major learning issues with existingneural sequential methods. Fortunately, self-supervised learningseems to provide a promising solution to both problems: it utilizesthe intrinsic data correlation to devise auxiliary training objectivesand enhances the data representations via pre-trained methodswith rich self-supervised signals. However, for sequential recom-mendation, the context information exists in different forms orwith varying intrinsics, including item, attribute, subsequence, orsequence. It is not easy to develop a unified approach to charac-terizing such data correlations. For this problem, we are inspiredby the recently proposed mutual information maximization (MIM)method [5, 10, 11, 30]. It has been shown to be particularly effectiveto capture the correlation between different views (or parts) of theoriginal input by maximizing the mutual information between theencoded representations of these views.

To this end, in this paper, we propose a novel Self-Supervisedlearning approach to improve Sequential Recommendation withMIM, which is called S3-Rec. Based on a self-attentive recom-mender architecture [8], we propose to first pre-train the sequen-tial recommender with self-supervised signals and then fine-tunethe model parameters according to the recommendation task. Themajor novelty lies in the pre-training stage. In particular, we care-fully devise four self-supervised optimization objectives for captur-ing item-attribute, sequence-item, sequence-attribute and sequence-subsequence correlations, respectively. These optimization objec-tives are developed in a unified form of MIM. As such, S3-Rec isable to characterize the correlation in varying levels of granularityor between different forms in a general way. It is also flexible toadapt to new data types or new correlation patterns. Via such apre-trained method, we can effectively fuse various kinds of contextdata, and learn attribute-aware contextualized data representations.Finally, the learned data representations are fed into the neuralrecommender, which will be optimized according to the recommen-dation performance.

To validate the effectiveness of our proposed S3-Rec method, weconduct extensive experiments on six real-world recommendationdatasets of different domains. Experimental results show that S3-Rec achieves state-of-the-art performance compared to a numberof competitive methods, especially when training data is limited.

We also show that our S3-Rec is effective to adapt to other classesof neural architectures, such as GRU and CNN.

Our main contributions are summarized as follows: (1) To thebest of our knowledge, it is the first time that self-supervised learn-ing with MIM has been applied to improve the sequential recom-mendation task; (2) We propose four self-supervised optimizationobjectives to maximize the mutual information of context informa-tion in different forms or granularities; (3) Extensive experimentsconducted on six real-world datasets demonstrate the effectivenessof our proposed approach.

2 RELATEDWORK2.1 Sequential RecommendationEarlyworks on sequential recommendation are based on theMarkovChain assumption. MC-based methods [20] estimated an item-itemtransition probability matrix and utilized it to predict the next itemgiven the last interaction of a user. A series of works follow thisline and extend it for high-order MCs [4, 8, 24]. With the devel-opment of the neural networks, Hidasi et al. [3] firstly introducedGated Recurrent Units (GRU) to the session-based recommendationand a surge of following variants modified this model by intro-ducing pair-wise loss functions [4], memory networks [6, 7], hier-archical structures [17], copy mechanism [18] and reinforcementlearning [27], etc. There are also studies that leverage other archi-tectures [8, 23, 24] for sequential recommendation. However, theseapproaches neglect the rich attribute information about items. Totackle this problem, TransFM [16] utilized Factorization Machinesto incorporate arbitrary real-valued features to the sequential rec-ommendation. FDSA [29] employed a feature-level self-attentionblock to leverage the attribute information about items in userhistory. Despite the remarkable success of these sequential rec-ommendation models, the correlations among attribute, item, andsequence are still not utilized and modeled sufficiently.

2.2 Self-supervised LearningSelf-supervised learning [1, 5, 15] aims at training a network on anauxiliary objective where the ground-truth samples are obtainedfrom the raw data automatically. The general framework is to con-struct training signals directly from the correlation within the rawdata and utilize them to train the model. The correlation informa-tion learned through self-supervised learning can then be easilyutilized to benefit other tasks. Several self-supervised objectiveshave been introduced to use non-visual but intrinsically correlatedfeatures to guide the visual feature learning [5]. As for languagemodeling [1, 15], it is a popular self-supervised objective for naturallanguage processing, where the model learns to predict the nextword or sentence given the previous sequences. The learned repre-sentations of words or sequences can improve the performance ofdownstream tasks such as machine reading comprehension [1] andnatural language understanding [10].

Mutual information maximization [5, 10, 11] is a special branchof the self-supervised learning. It is inspired by the InfoMax princi-ple [11] and has made important progress in several domains suchas computer vision [5], audio processing [25], and nature languageunderstanding [10]. This method splits the input data into multiple

(possibly overlapping) views and maximizes the mutual informa-tion between representations of these views. The views derivedfrom other inputs are used as negative samples.

Different from the above approaches, our work is the first toconsider the correlations within the contextual information as theself-supervised signals in sequential recommendation. We maxi-mize the mutual information among the views of the attribute, item,and sequence, which are in different levels of granularity of thecontextual information. The enhanced data representations canimprove recommendation performance.

3 PRELIMINARIESIn this section, we first formulate the sequential recommendationproblem and then introduce the technique of mutual informationmaximization.

3.1 Problem StatementAssume that we have a set of users and items, denoted byU andI, respectively, where u ∈ U denotes a user and i ∈ I denotesan item. The numbers of users and items are denoted as |U| and|I |, respectively. Generally, a user u has a chronologically-orderedinteraction sequencewith items: {i1, · · · , in }, wheren is the numberof interactions and it is the t-th item that the user u has interactedwith. For convenience, we use i j :k to denote the subsequence, i.e.,i j :k = {i j , · · · , ik } where 1 ≤ j < k ≤ n. Besides, each item i isassociated with several attributesAi = {a1, · · · ,am }. For example,a song is typical with auxiliary information such as artist, album,and popularity for music recommender. All attributes constitute anattribute set A, and the number of attributes is donated as |A|.

Based on the above notations, we now define the task of sequen-tial recommendation. Formally, given the historical behaviors of auser {i1, · · · , in } and the attributes Ai of each item i , the task ofsequential recommendation is to predict the next item that the useris likely to interact with at the (n + 1)-th step.

3.2 Mutual Information MaximizationAn important technique in our approach is the Mutual InformationMaximization (MIM). It is developed on the core concept of mutualinformation, which measures dependencies between random vari-ables. Given two random variables X and Y , it can be understood ashow much knowing X reduces the uncertainty in Y or vice versa.Formally, the mutual information between X and Y is:

I (X ,Y ) = H (X ) − H (X |Y ) = H (Y ) − H (Y |X ). (1)

Maximizing mutual information directly is usually intractable.Thus we resort to a lower bound on I (X ,Y ). One particular lowerbound that has been shown to work well in practice is InfoNCE [10,12, 25], which is based on Noise Contrastive Estimation (NCE) [2].InfoNCE is defined as:

Ep(X ,Y )[fθ (x ,y) − Eq(Y )[log∑y∈Y

exp fθ (x , y)]] + log |Y |, (2)

where x and y are different views of an input, and fθ is a functionparameterized by θ (e.g., a dot product between encoded represen-tations of a word and its context [10] or a dot product betweenencoded representations of an image and the local regions of the

image [5]), and Y is a set of samples drawn from a proposal distribu-tion q(Y ), which contains a positive sample y and |Y | − 1 negativesamples.

Note that InfoNCE is related to the cross-entropy. If Y alwaysincludes all possible values of the random variableY (i.e., Y = Y ) andthey are uniformly distributed, maximizing InfoNCE is analogousto maximize the standard cross-entropy loss:

Ep(X ,Y )[fθ (x ,y) − log∑y∈Y

exp fθ (x , y)]. (3)

This equation shows that InfoNCE is related to maximize pθ (y |x),and it approximates the summation over elements in Y (i.e.,, thepartition function) by negative sampling. Based on this formula,we can utilize specific X ,Y to maximize the mutual information be-tween different views of the raw data, e.g., an item and its attributes,or a sequence and the items that it contains.

4 APPROACH4.1 OverviewExisting studies [3, 4, 8, 24] mainly emphasize the effect of sequen-tial characteristics using an item-level optimization objective alone.Inspired by recent progress with MIM [5, 28], we take a differentperspective to develop neural sequential recommenders by maxi-mizing the mutual information among different views of the rawdata.

The basic idea of our approach is to incorporate several elabo-rately designed self-supervised learning objectives for enhancingthe original model. To develop such objectives, we leverage effec-tive correlation signals reflected in the intrinsic characteristics ofthe input. For our task, we consider the information in differentlevels of granularity, including attribute, item, segment (i.e., subse-quence), and sequence, which are considered as different views ofthe input. By capturing the multi-view correlation, we unify theseself-supervised learning objectives with the recently proposed pre-training framework in language modeling [1].

The overview of S3-Rec is presented in Fig. 1. In the followingsections, we first introduce the basemodel of our proposed approachthat is developed on the Transformer architecture [8]. Then, we willdescribe how we utilize the correlation signals among attributes,items, segments, and sequences to enhance the data representationsbased on the InfoNCE [10, 25] method. Finally, we present thediscussions on our approach.

4.2 Base ModelWe develop the basic framework for sequential recommendationmodel by stacking the embedding layer, self-attention blocks, andthe prediction layer.

4.2.1 Embedding Layer. In the embedding mapping stage, wemain-tains an item embedding matrix MI ∈ R |I |×d and an attributeembedding matrix MA ∈ R |A |×d . The two matrices project thehigh-dimensional one-hot representation of an item or attributeto low-dimensional dense representations. Given a n-length itemsequence, we apply a look-up operation from MI to form the inputembedding matrix E ∈ Rn×d . Besides, we incorporate a learnable

ItemsAttributes

ItemEmbedding

Bidirectional Self-Attenion

...

Sequence-Item MIM

ItemEmbedding

Sequence-Attribute MIM

Item-Attribute MIM

Attribute Embedding

ItemEmbedding

Sequence-Sequence MIM


(1) Associated Attribute Prediction (2) Masked Item Prediction (3) Masked Attribute Prediction (4) Segment Prediction

ItemEmbedding


...

ItemEmbedding


...

ItemEmbedding


...

Attribute Embedding

Masked ItemMasked Item

AttributesMasked Segment

𝒜i1

...

𝒜i2

...

ea, a ∈ 𝒜i2

i1 ...i2 i3 in i1 ...Mask i3 in i1 ...Mask i3 in i1 ...Mask Mask in

i2 i3i2

... ...AttributesEmbeddings

AttributesEmbeddings

ItemEmbedding ei2 ea, a ∈ 𝒜i1 ea, a ∈ 𝒜in... ...

𝒜i2𝒜i3 𝒜in

Figure 1: The overview of S3-Rec in the pre-training stage. We assume that the user sequence is {i1, · · · , in } and each item iis associated with several attributes Ai = {a1, · · · ,am }. We incorporate four self-supervised learning objectives: (1) Associ-ated Attribute Prediction (AAP), (2) Masked Item Prediction (MIP), (3) Masked Attribute Prediction (MAP), and (4) SegmentPrediction (SP). The embedding layers and bidirectional self-attention blocks are shared by the four pre-training objectives.

position encoding matrix P ∈ Rn×d to enhance the input repre-sentation of the item sequence. By this means, the sequence repre-sentation EI ∈ Rn×d can be obtained by summing two embeddingmatrices: EI = E + P. Since our task utilizes auxiliary context data,we also form an embedding matrix EA ∈ Rk×d for each item fromthe entire attribute embedding matrix MA, where k is the numberof item attributes.

4.2.2 Self-Attention Block. Based on the embedding layer, we de-velop the item encoder by stacking multiple self-attention blocks.A self-attention block generally consists of two sub-layers, i.e., amulti-head self-attention layer and a point-wise feed-forward net-work. The multi-head self-attention mechanism has been adoptedfor effectively extracting the information selectively from differentrepresentation subspaces. Specifically, the multi-head self-attentionis defined as:

MultiHeadAttn(Fl ) = [head1,head2, ...,headh ]WO , (4)

headi = Attention(FlWQi , F

lWKi , F

lWVi ), (5)

where the Fl is the input for the l-th layer. When l = 0, we setF0 = EI , and the projection matrixWQ

i ∈ Rd×d/h ,WKi ∈ Rd×d/h ,

WQV ∈ Rd×d/h and WO ∈ Rd×d are the corresponding learnable

parameters for each attention head. The attention function is im-plemented by scaled dot-product operation:

Attention(Q,K,V) = softmax( QK⊤√

d/h)V, (6)

where Q = FlWQi , K = FlWK

i , and V = FlWVi are the linear

transformations of the input embedding matrix, and√d/h is the

scale factor to avoid large values of the inner product.Since the multi-head attention function is mainly built on the

linear projections. We endow the non-linearity of the self-attention

block by applying a point-wise feed-forward network. The compu-tation is defined as:

Fl = [FFN(Fl1)⊤; · · · ; FFN(Fln )⊤], (7)

FFN(x) = (ReLU(xW1 + b1))W2 + b2, (8)

whereW1,b1,W2,b2 are trainable parameters.In sequential recommendation, only the information before the

current time step can be utilized, thus we apply the mask operationfor the output of the multi-head self-attention function to removeall connections between Qi and Ki . Inspired by BERT [1], at thepre-training stage, we remove the mask mechanism to acquire thebidirectional context-aware representation of each item in an itemsequence. It is beneficial to incorporate context from both directionsfor sequence representation learning [1, 23].

4.2.3 Prediction Layer. In the final layer of S3-Rec, we calculatethe user’s preference score for the item i in the step (t + 1) underthe context from user history as:

P(it+1 = i |i1:t ) = e⊤i · FLt , (9)

where ei is the representation of item i from item embedding matrixMI , FLt is the output of the L-layer self-attention block at step t andL is the number of self-attention blocks.

4.3 Self-supervised Learning with MIMBased on the above self-attention model, we further incorporateadditional self-supervised signals with MIM to enhance the repre-sentations of input data. We adopt a pre-training way to constructdifferent loss functions based on the multi-view correlation.

4.3.1 Modeling Item-Attribute Correlation. We first maximize themutual information between items and attributes. For each item, theattributes provide fine-grained information about it. Therefore, weaim to fuse item- and attribute-level information through modelingitem-attribute correlation. In this way, it is expected to inject usefulattribute information into item representations.

Given an item i and the attribute setAi = {a1, · · · ,ak }, we treatthe item itself and its associated attributes as two different views.Formally, let ei denote the item embedding obtained by the em-bedding layer, and eaj denote the embedding for the j-th attributeaj ∈ Ai . We design a loss function by the contrastive learningframework that maximizes the mutual information between thetwo views. Following Eq. 3, we minimize the Associated AttributePrediction (AAP) loss by:

LAAP (i,Ai ) = Eaj ∈Ai [f (i,aj ) − log∑

a∈A\Ai

exp(f (i, a))], (10)

where we sample negative attributes a that enhance the associationbetween the item i and the ground-truth attributes, “\” defines setsubtraction operation. The function f (·, ·) is implemented with asimple bilinear network:

f (i,aj ) = σ(e⊤i ·WAAP · eaj

), (11)

whereWAAP ∈ Rd×d is a parameter matrix to learn and σ (.) is thesigmoid function. Note that for clarity, we give the loss definitionLAAP for a single item. It will be easy to define this loss over theentire item set.

4.3.2 Modeling Sequence-Item Correlation. Conventional sequen-tial recommendation models are usually trained to predict the itemat the next step. This approach only considers the sequential charac-teristics in an item sequence from left to right. While it is noted thatthe entire interaction sequence is indeed observed by the modelin the training process. Inspired by the masked language modellike BERT [1], we propose to model the bidirectional informationin item sequence by a Cloze task. For our task, the Cloze settingis described as below: at each training step, we randomly mask aproportion of items in the input sequence (i.e., replace them withspecial tokens “[mask]”). Then we predict the masked items fromthe original sequence based on the surrounding context in bothdirections.

Therefore, the second loss we consider is to recover the ac-tual item with the bidirectional context from the input sequences.For this purpose, we prepare a pre-trained version of the basemodel in Section 4.2, which is a bidirectional Transformer archi-tecture. As illustration, let us mask the t-th item it in a sequence{i1, · · · , it , · · · , in }.We treat the rest sequence {i1, · · · ,mask, · · · , in }as the surrounding context for it , denoted by Cit . Given the sur-rounding context Cit and the masked item it , we treat them as twodifferent views to fuse for learning data representations. FollowingEq. 3, we minimize the Masked Item Prediction (MIP) loss by:

LMIP (Cit , it ) = f (Cit , it ) − log[∑

i ∈I\{it }f (Cit , it )], (12)

where i denotes an irrelevant item, and f (·, ·) is implemented ac-cording to the following formula:

f (Cit , it ) = σ(F⊤t ·WMIP · eit

), (13)

whereWMIP ∈ Rd×d is a parameter matrix to learn and Ft is thelearned representation for the t-th position using the bidirectionalTransformer architecture obtained in the same way as Eq. 7.

4.3.3 Modeling Sequence-Attribute Correlation. Having modeledboth item-attribute and sequence-item correlations, we further con-sider directly fusing attribute information with sequential contexts.Specifically, we adopt a similar way as in Section 4.3.2 to recoverthe attributes of a masked item based on surrounding contexts.Given a masked item it , we treat its surrounding context Cit andits attribute set Ait as two different views for MIM. As such, wecan develop the following Masked Attribute Prediction (MAP) lossby:

LMAP (Cit ,Ait )

=Ea∈Ait[f (Cit ,a) − log

∑a∈A\Ai

exp(f (Cit , a))], (14)

where f (·, ·) is implemented according to the following formula:

f (Cit ,a) = σ(F⊤t ·WMAP · ea

), (15)

where WMAP ∈ Rd×d is a parameter matrix to learn. Note thatexisting methods [4, 8, 24] seldom directly model the correlationbetween the sequential context and attribute information. While,we would like to explicitly model the correlation to derive moremeaningful supervision signals, which is useful to improve the datarepresentations for multi-granularity information.

4.3.4 Modeling Sequence-Segment Correlation. As shown above,the Cloze learning strategy plays a key role in our pre-trained ap-proach in fusing sequential contexts with target information. How-ever, a major difference between item sequence with word sequenceis that a single target item may not be highly related to surround-ing contexts. For example, a user has bought some products justbecause they were on sale. Based on this concern, we extend theCloze strategy from a single item to item subsequence (i.e., calledsegment). Apparently, an item segment reflects more clear, stableuser preference than a single item. Therefore, we follow a similarstrategy in Section 4.3.2 to recover an item subsequence from sur-rounding contexts. It is expected to enhance the self-supervisedlearning signal and improve the pre-trained performance.

Let i j1:j2 denote the subsequence from item i j1 to i j2 , and Ci j1:j2denote the context for i j1:j2 within the entire sequence. Similarto Eq. 12, we can recover the missing item segment with a MIMformulation, which is so called the Segment Prediction (SP) loss as:

LSP (Ci j1:j2 , i j1:j2 )

=f (Ci j1:j2 , i j1:j2 ) − log∑i j1:j2

exp(f (Ci j1:j2 , i j1:j2 )

), (16)

where i j1, j2 is the corrupted negative subsequence and f (·, ·) isimplemented according to the following formula:

f (Ci j1:j2 , i j1:j2 ) = σ(s⊤ ·WSP · s

), (17)

where WSP ∈ Rd×d is a parameter matrix to learn, and s and s arethe learned representations for the contexts Ci j1:j2 and subsequencei j1:j2 , respectively. In order to learn s and s, we apply the bidirec-tional Transformer to obtain the state representations of the lastposition in a sequence.

4.4 Learning and DiscussionIn this part, we present the learning and related discussions of ourS3-Rec for sequential recommendation.

4.4.1 Learning. The entire procedure of S3-Rec consists of twoimportant stages, namely pre-training and fine-tuning stages. Weadopt bidirectional and unidirectional Transformer [26] architec-tures for the two stages, respectively. At the pre-trained stage, weoptimize the self-supervised learning objectives by considering fourdifferent kinds of correlations (Eq. 10, Eq. 12, Eq. 14 and Eq. 16); atthe fine-tuning stage, we utilize the learned parameters from thepre-trained stage to initialize the parameters of the unidirectionalTransformer, and then utilize the left-to-right supervised signals totrain the network. We adopt the pairwise rank loss to optimize itsparameters as:

Lmain = −∑u ∈U

n∑t=1

logσ(P(it+1 |i1:t ) − P(i−t+1 |i1:t )

), (18)

where we pair each ground-truth item it+1 with a negative itemi−t+1 that is randomly sampled.

4.4.2 Discussion. Our work provides a novel self-supervised ap-proach to capturing the intrinsic data correlation from the input asan additional signal through the pre-trained models. This approachis quite general so that many existing methods can be included inthis framework. We make a brief discussion below.

Feature-based approaches such as Factorization Machine [20]and AutoInt [22] mainly learn data representations through theinteraction of context features. The final prediction is made ac-cording to the actual interaction results between the user and itemfeatures. In S3-Rec, the associated attribute prediction loss LAAPin Eq. 10 and the masked attribute prediction loss LMAP in Eq. 14have the similar effect in feature interaction. However, we do notexplicitly model the interaction between attributes. Instead, wefocus on capturing the association between attribute informationand item/sequential contexts. A major difference in our work isto utilize feature interaction as additional supervision signals toenhance data representations instead of making predictions.

Sequentialmodels such as GRU4Rec [21] and SASRec [8]mainlyfocus on modeling the sequential dependencies between contextualitems and the target item in a left-to-right order. S3-Rec additionallyincorporates a pre-trained stage that leverages four different kindsof self-supervised learning signals for enhancing data representa-tions. In particular, the masked item prediction loss LMIP in Eq. 12has a similar effect to capture sequential dependencies as in [8, 21]except that it can also utilize bidirectional sequential information.

Attribute-aware sequential models such as TransFM [16]and FDSA [29] leverage the contextual features to improve the se-quential recommender models, in which these features are treatedas auxiliary information to enhance the representation of itemsor sequences. In our S3-Rec, the LAAP loss and LMAP loss aim tofuse attribute with items or sequential contexts, which is able toachieve the same effect as previous methods [16, 29]. Besides, thepre-trained data representations can be also applied to improveexisting methods.

Table 1: Statistics of the datasets after preprocessing.

Dataset Meituan Beauty Sports Toys Yelp LastFM

# Users 13,622 22,363 25,598 19,412 30,431 1,090# Items 20,062 12,101 18,357 11,924 20,033 3,646# Avg. Actions / User 54.9 8.9 8.3 8.6 10.4 48.2# Avg. Actions / Item 37.3 16.4 16.1 14.1 15.8 14.4# Actions 747,827 198,502 296,337 167,597 316,354 52,551Sparsity 99.73% 99.93% 99.95% 99.93% 99.95% 98.68%# Attributes 331 1,221 2,277 1,027 1,001 388# Avg. Attribute / Item 8.8 5.1 6.0 4.3 4.8 31.5

5 EXPERIMENT5.1 Experimental Setup5.1.1 Dataset. We conduct experiments on six datasets collectedfrom four real-world platforms with varying domains and spar-sity levels. The statistics of these datasets after preprocessing aresummarized in Table 1.

(1)Meituan1: this dataset consists of six-year (from Jan. 2014 toJan. 2020) transaction records in Beijing on the Meituan platform.We select categories, locations, and the keywords extracted fromcustomer reviews as attributes.

(2)Amazon Beauty, Sports, and Toys: these three datasets areobtained from Amazon review datasets in [14]. In this work, weselect three subcategories: “Beauty”, “Sports and Outdoors”, and“Toys and Games”, and utilize the fine-grained categories and thebrands of the goods as attributes.

(3) Yelp2: this is a popular dataset for business recommendation.As it is very large, we only use the transaction records after January1st, 2019. We treat the categories of businesses as attributes.

(4) LastFM3: this is a music artist recommendation dataset andcontains user tagging behaviors for artists. In this dataset, the tagsof the artists given by the users are used as attributes.

For all datasets, we group the interaction records by users andsort them by the interaction timestamps ascendingly. Following [21,29], we only keep the 5-core datasets, and filter unpopular itemsand inactive users with fewer than five interaction records.

5.1.2 Evaluation Metrics. We employ top-k Hit Ratio (HR@k), top-k Normalized Discounted Cumulative Gain (NDCG@k), and MeanReciprocal Rank (MRR) to evaluate the performance, which arewidely used in related works [21, 29]. Since HR@1 is equal toNDCG@1, we report results on HR@{1, 5, 10}, NGCG@{5, 10}, andMRR. Following previous works [8, 19, 23], we apply the leave-one-out strategy for evaluation. Concretely, for each user interactionsequence, the last item is used as the test data, the item before thelast one is used as the validation data, and the remaining data isused for training. Since the item set is large, it is time-consumingto use all items as candidates for testing. Following the commonstrategy [7, 8], we pair the ground-truth item with 99 randomlysampled negative items that the user has not interacted with. Wecalculate all metrics according to the ranking of the items and reportthe average score over all test users.1https://www.meituan.com2https://www.yelp.com/dataset3https://grouplens.org/datasets/hetrec-2011/

https://www.meituan.com

https://www.yelp.com/dataset

https://grouplens.org/datasets/hetrec-2011/

5.1.3 Baseline Models. We compare our proposed approach withthe following eleven baseline methods:

(1) PopRec is a non-personalized method that ranks items ac-cording to popularity measured by the number of interactions.

(2) FM [20] characterizes the pairwise interactions between vari-ables using factorized model.

(3) AutoInt [22] utilizes the multi-head self-attentive neuralnetwork to learn the feature interaction.

(4) GRU4Rec [3] applies GRU to model user click sequencefor session-based recommendation. We represent the items usingembedding vectors rather than one-hot vectors.

(5) Caser [24] is a CNN-based method capturing high-orderMarkov Chains by applying horizontal and vertical convolutionaloperations for sequential recommendation.

(6) SASRec [8] is a self-attention based sequential recommen-dation model, which uses the multi-head attention mechanism torecommend the next item.

(7) BERT4Rec [23] uses a Cloze objective loss for sequentialrecommendation by the bidirectional self-attention mechanism.

(8)HGN [13] is recently proposed and adopts hierarchical gatingnetworks to capture long-term and short-term user interests.

(9) GRU4RecF [4] is an improved version of GRU4Rec, whichleverages attributes to improve the performance.

(10) SASRecF is our extension of SASRec, which concatenatesthe representations of item and attribute as the input to the model.

(11) FDSA [29] constructs a feature sequence and uses a feature-level self-attention block to model the feature transition patterns.This is the state-of-the-art model in sequential recommendation.

5.1.4 Implementation Details. For Caser and HGN, we use thesource code provided by their authors. For other methods, we im-plement them by PyTorch. All hyper-parameters are set followingthe suggestions from the original papers.

For our proposed S3-Rec, we set the number of the self-attentionblocks and the attention heads as 2. The dimension of the embeddingis 64, and the maximum sequence length is 50 (following [8]). Notethat our training phase contains two stages (i.e., pre-training andfine-tuning stage), the learned parameters in the pre-training stageare used to initialize the embedding layers and self-attention layersof our model in the fine-tuning stage.

In the pre-training stage, the mask proportion of item is set as0.2 and the weights for the four losses (i.e., AAP, MIP, MAP, and SP)are set as 0.2, 1.0, 1.0, and 0.5, respectively, based on our empiricalexperiments. We use the Adam optimizer [9] with a learning rate of0.001, where the batch size is set as 200 and 256 in the pre-trainingand the fine-tuning stage, respectively. We pre-train our model for100 epochs and fine-tune it on the recommendation task. The codeand data set are available at the link: https://github.com/RUCAIBox/CIKM2020-S3Rec 4.

5.2 Experimental ResultsThe results of different methods on all datasets are shown in Table 2.Based on the results, we can find:

4To further verify the effectiveness of our method, we have performed the experimentsthat rank the ground-truth item with all the items as candidates. The complete resultsare shown on our project website at this link.

For three non-sequential recommendation baselines, the perfor-mance order is consistent across all datasets, i.e., PopRec > AutoInt> FM. Due to the “rich-gets-richer” effect in product adoption,PopRec is a robust baseline. AutoInt performs better than FM onmost datasets because the multi-head self-attention mechanism hasa stronger capacity to model attributes. However, the performanceof AutoInt is worse than that of FM on Meituan dataset. A poten-tial reason is that the multi-head self-attention may incorporatemore noise from the attributes since they are keywords extractedfrom the reviews on Meituan platform. In general, non-sequentialrecommendation methods perform worse than sequential recom-mendation methods, since the sequential pattern is important toconsider in our task.

As for sequential recommendation baseline methods, SASRecand BERT4Rec utilize the unidirectional and bidirectional self-attention mechanism respectively, and achieve better performancethan GRU4Rec and Caser. It indicates that self-attentive architec-ture is particularly suitable for modeling sequential data. However,their improvements are not stable when training with the conven-tional next-item prediction loss. Besides, HGN achieves comparableperformance with SASRec and BERT4Rec. This indicates the hierar-chical gating network can well model the relations between closelyrelevant items. However, when directly injecting the attribute infor-mation into GRU4Rec and SASRec (i.e., GRU4RecF and SASRecF ),the performance improvement is not consistent. This method yieldsimprovement on Beauty, Sports, Toys, and Yelp datasets, but has anegative influence on other datasets. One possible reason is thatsimply concatenating item representations and its attributes repre-sentations cannot effectively fuse the two kinds of information. Inmost cases, FDSA achieves the best performance among all base-lines. This suggests that the feature-level self-attention blocks cancapture useful sequential feature interaction patterns.

Finally, by comparing our approach with all the baselines, it isclear to see that S3-Rec performs consistently better than them bya large margin on six datasets. Different from these baselines, weadopt the self-supervised learning to enhance the representationsof the attribute, item, and sequence for the recommendation task,which incorporates four pre-training objectives to model multipledata correlations by MIM. This result also shows that the self-supervised approach is effective to improve the performance of theself-attention architecture for sequential recommendation.

5.3 Further AnalysisNext, we continue to study whether S3-Rec works well in moredetailed analysis.

5.3.1 Ablation Study. Our proposed self-supervised approach S3-Rec designs four pre-training objectives based on MIM. To verifythe effectiveness of each objective, we conduct the ablation studyon Meituan, Beauty, Sports, and Toys datasets to analyze the contri-bution of each objective. NDCG@10 is adopted for this evaluation.The results from the best baseline FDSA are also provided for com-parison.

From the results in Fig. 2, we can observe that removing anyself-supervised objective would lead to the performance decrease.It indicates all the objectives are useful to improve the recommen-dation performance. Besides, the importance of these objectives

https://github.com/RUCAIBox/CIKM2020-S3Rec

https://github.com/RUCAIBox/CIKM2020-S3Rec

Table 2: Performance comparison of differentmethods on six datasets. The best performance and the second best performancemethods are denoted in bold and underlined fonts respectively. “∗” indicates the statistical significance for p < 0.01 comparedto the best baseline method.

Datasets Metric PopRec FM AutoInt GRU4Rec Caser SASRec BERT4Rec HGN GRU4RecF SASRecF FDSA S3-Rec Improv.

Meituan

HR@1 0.0946 0.1084 0.0804 0.1194 0.1368 0.1797 0.1381 0.1603 0.1436 0.1746 0.1778 0.2040∗ 13.52%HR@5 0.2660 0.3218 0.2662 0.3382 0.3812 0.4524 0.3985 0.4110 0.3799 0.4386 0.4595 0.4925∗ 7.18%NDCG@5 0.1813 0.2170 0.1739 0.2303 0.2619 0.3207 0.2713 0.2887 0.2639 0.3098 0.3236 0.3527∗ 8.99%HR@10 0.3863 0.4709 0.4077 0.4881 0.5267 0.6053 0.5514 0.5573 0.5378 0.5962 0.6164 0.6368∗ 3.31%NDCG@10 0.2200 0.2651 0.2194 0.2787 0.3090 0.3700 0.3208 0.3359 0.3149 0.3607 0.3743 0.3994∗ 6.71%MRR 0.1923 0.2242 0.1854 0.2359 0.2617 0.3146 0.2689 0.2863 0.2666 0.3064 0.3167 0.3421∗ 8.02%

Beauty


Sports


Toys


Yelp

HR@1 0.0801 0.0624 0.0731 0.2053 0.2188 0.2375 0.2405 0.2428 0.2293 0.2301 0.2198 0.2591∗ 6.71%HR@5 0.2415 0.2036 0.2249 0.5437 0.5111 0.5745 0.5976 0.5768 0.5858 0.5937 0.5728 0.6085∗ 1.82%NDCG@5 0.1622 0.1333 0.1501 0.3784 0.3696 0.4113 0.4252 0.4162 0.4137 0.4178 0.4014 0.4401∗ 3.50%HR@10 0.3609 0.3153 0.3367 0.7265 0.6661 0.7373 0.7597 0.7411 0.7574 0.7706 0.7555 0.7725 0.25%NDCG@10 0.2007 0.1692 0.1860 0.4375 0.4198 0.4642 0.4778 0.4695 0.4694 0.4751 0.4607 0.4934∗ 3.26%MRR 0.1740 0.1470 0.1616 0.3630 0.3595 0.3927 0.4026 0.3988 0.3929 0.3962 0.3834 0.4190∗ 4.07%

LastFM


FDSA¬ AAP

¬ MIP¬ MAP¬ SP

S3 -Rec0.33

0.35

0.37

0.39

0.41 Meituan

FDSA¬ AAP

¬ MIP¬ MAP¬ SP

S3 -Rec0.30

0.32

0.34

0.36

0.38 Beauty

FDSA¬ AAP

¬ MIP¬ MAP¬ SP

S3 -Rec0.260.280.300.320.340.36 Sports

FDSA¬ AAP

¬ MIP¬ MAP¬ SP

S3 -Rec0.30

0.32

0.34

0.36

0.38 Toys

Figure 2: Ablation study of our approach on four datasets (NDCG@10). “¬” indicates that the corresponding objective is re-moved in the pre-training stage, while the rest objectives are kept.

is varying on different datasets. Overall, the AAP (Associated At-tribute Prediction) and the MAP (Masked Attribute Prediction) aremore important than the other objectives. Removing each of them

yields a larger drop of performance on all datasets. One possiblereason is that these two objectives enhance the representations ofitem and sequence with the attributes information.

0.15

0.20

0.25

0.30

0.35

0.40 Beauty

AutoInt CaserGRU4Rec

GRU4RecFSASRec

SASRecF S3 -Rec

0.150.200.250.300.350.40 Toys

OriginalMIM

Figure 3: Performance (NDCG@10) comparison of differentmodels enhanced by our self-supervised learning approachon Beauty and Toys datasets.

100% 80% 60% 40% 20%0.100.150.200.250.300.350.40 SASRec

SASRecF

FDSAS3-Rec

(a) Sports

100% 80% 60% 40% 20%0.250.300.350.400.450.500.55 SASRec

SASRecF

FDSAS3-Rec

(b) Yelp

Figure 4: Performance (NDCG@10) comparison w.r.t. differ-ent sparsity levels on Sport and Yelp datasets.

It is clearly seen that all model variants are better than the bestbaseline FDSA, which is trained only with next-item predicationloss.

5.3.2 Applying Self-Supervised Learning to Other Models. Sinceself-supervised learning itself is a learning paradigm, it can gen-erally apply to various models. Thus, in this part, we conduct anexperiment to examine whether our method can bring improve-ments to other models. We use the self-supervised approach topre-training some baseline models on Beauty and Toys datasets.For GRU4Rec, GRU4RecF , SASRec, and SASRecF , we directly applyour pre-training objectives to improve them. It is worth noting thatGRU4Rec and SASRec are unidirectional models, so we maintainthe unidirectional encoder layer in the pre-training stage. For Au-toInt and Caser, since their architectures do not support some of thepre-training objectives5, we only utilize the pre-trained parametersto initialize the parameters of the embedding layers.

The results of NDCG@10 on Beauty and Toys datasets are shownin Fig. 3. First, after pre-training by our approach, all the baselinesachieve better performance. This shows that self-supervised learn-ing can also be applied to improve their performance. Second, S3-Rec outperforms all the baselines after pre-training. This is because5Because their base models do not support the mask operations.

30 60 90 120Epochs

0.30

0.32

0.34

0.36

0.38

0.40Pre-Trainingw/o Pre-Training

(a) Beauty

30 60 90 120Epochs

0.30

0.32

0.34

0.36

0.38

0.40Pre-Trainingw/o Pre-Training

(b) Toys

Figure 5: Performance (NDCG@10) comparison w.r.t. differ-ent numbers of pre-training epochs on Beauty and Toysdatasets.

our model adopts the bidirectional Transformer encoder in the pre-training stage, which is more suitable for our approach. Third, wecan see the GRU-based models achieve less improvement than theother models. One possible reason is that RNN-based architecturelimits the potential of self-supervised learning.

5.3.3 Performance Comparison w.r.t. the Amount of Training Data.Conventional recommendation systems require a considerable amo-unt of training data, thus they are likely to suffer from the coldstart problem in real-world applications. This problem can be allevi-ated by our method because the proposed self-supervised learningapproach can better utilize the data correlation from input. Wesimulate the data sparsity scenarios by using different proportionsof the full dataset, i.e., 20%, 40%, 60%, 80%, and 100%.

Fig. 4 shows the evaluation results on Sports and Yelp datasets. Aswe can see, the performance substantially drops when less trainingdata is used.While, S3-Rec is consistently better than baselines in allcases, especially in an extreme sparsity level (20%). This observationimplies that S3-Rec is able to make better use of the data withthe self-supervised method, which alleviates the influence of datasparsity problem for sequential recommendation to some extent.

5.3.4 Performance Comparison w.r.t. the Number of Pre-trainingEpochs. Our approach consists of a pre-training stage and a fine-tuning stage. In the pre-training stage, our model can learn theenhanced representations of the attribute, item, subsequence, andsequence for the recommendation task. The number of pre-trainingepochs affects the performance of the recommendation task. Toinvestigate this, we pre-train our model with a varying number ofepochs and fine-tune it on the recommendation task.

Fig. 5 presents the results on Beauty and Toys datasets. The hor-izontal dash lines represent the performance without pre-training.We can see that our model benefits mostly from the first 20 pre-training epochs. And after that, the performance improves slightly.Based on this observation, we can conclude that the correlationsamong different views (i.e., the attribute, item, subsequence, andsequence) can be well-captured by our self-supervised learningapproach through pre-training within a small number of epochs.So that the enhanced data representations can improve the perfor-mance of sequential recommendation.

5.3.5 Convergence Speed Comparison. After obtaining the enhancedrepresentations of the attribute, item, and sequence, we fine-tune

40 80 120 160Epochs

0.15

0.25

0.35

0.45 SASRecSASRecF

FDSAS3-Rec

(a) Beauty

40 80 120 160Epochs

0.15

0.25

0.35

0.45 SASRecSASRecF

FDSAS3-Rec

(b) Toys

Figure 6: Performance tuning (NDCG@10) of our approachand other baselineswith the increasing iterations in the fine-tuning stage.

our model on the recommendation task. To examine the conver-gence speed on the final recommendation task, we gradually in-crease the number of epochs for the fine-tuning stage and comparethe performance of our model and other baselines.

Fig. 6 shows the results on Beauty and Toys datasets. It can beobserved that our model converges quickly and achieves the bestperformance after about 40 epochs. In contrast to our model, thecomparison models need more epochs to achieve stable perfor-mance. This result shows that our approach can utilize pre-trainedparameters to help the model converge faster and achieve betterperformance.

6 CONCLUSIONIn this paper, we proposed a self-supervised sequential recommen-dation model S3-Rec based on the mutual information maximiza-tion (MIM) principle. In our approach, we adopted the self-attentiverecommender architecture as the base model and devised four self-supervised learning objectives to learn the correlations within theraw data. Based on MIM, the four objectives can learn the cor-relations among attribute, item, segment, and sequence, whichenhances the data representations for sequential recommendation.Experimental results have shown that our approach outperformsseveral competitive baselines.

In the future, we will investigate how to design other forms ofself-supervised optimization objectives. Wewill also consider apply-ing our approach to more complex recommendation tasks, such asconversational recommendation and multimedia recommendation.

ACKNOWLEDGEMENTThis work was partially supported by the National Natural Sci-ence Foundation of China under Grant No. 61872369 and 61832017,Beijing Academy of Artificial Intelligence (BAAI) under Grant No.BAAI2020ZJ0301, and Beijing Outstanding Young Scientist Pro-gram under Grant No. BJJWZYJH012019100020098, the Fundamen-tal Research Funds for the Central Universities, the Research Fundsof Renmin University of China under Grant No.18XNLG22 and19XNQ047. Xin Zhao is the corresponding author.

REFERENCES[1] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. 2019. BERT: Pre-training of

Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT

2019. 4171–4186.[2] M. Gutmann and A. Hyvärinen. 2012. Noise-Contrastive Estimation of Unnor-

malized Statistical Models, with Applications to Natural Image Statistics. J. Mach.Learn. Res. 13 (2012), 307–361.

[3] B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk. 2016. Session-based Recom-mendations with Recurrent Neural Networks. In ICLR 2016.

[4] B. Hidasi, M. Quadrana, A. Karatzoglou, and D. Tikk. 2016. Parallel RecurrentNeural Network Architectures for Feature-rich Session-based Recommendations.In RecSys 2016. 241–248.

[5] R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A.Trischler, and Y. Bengio. 2019. Learning deep representations by mutual in-formation estimation and maximization. In ICLR 2019.

[6] J. Huang, Z. Ren, W. X. Zhao, G. He, J.-R. Wen, and D. Dong. 2019. Taxonomy-Aware Multi-Hop Reasoning Networks for Sequential Recommendation. InWSDM 2019. 573–581.

[7] J. Huang, W. X. Zhao, H. Dou, J.-R. Wen, and E. Y. Chang. 2018. ImprovingSequential Recommendation with Knowledge-Enhanced Memory Networks. InSIGIR 2018. 505–514.

[8] W.-C. Kang and J. J. McAuley. 2018. Self-Attentive Sequential Recommendation.In ICDM 2018. 197–206.

[9] D. P. Kingma and J. Ba. 2015. Adam: A Method for Stochastic Optimization. InICLR 2015.

[10] L. Kong, C. de Masson d’Autume, L. Yu, W. Ling, Z. Dai, and D. Yogatama. 2020.A Mutual Information Maximization Perspective of Language RepresentationLearning. In ICLR 2020.

[11] R. Linsker. 1988. Self-Organization in a Perceptual Network. IEEE Computer 21,3 (1988), 105–117.

[12] L. Logeswaran and H. Lee. 2018. An efficient framework for learning sentencerepresentations. In ICLR 2018.

[13] C. Ma, P. Kang, and X. Liu. 2019. Hierarchical Gating Networks for SequentialRecommendation. In KDD 2019. 825–833.

[14] J. J. McAuley, C. Targett, Q. Shi, and A. van den Hengel. 2015. Image-BasedRecommendations on Styles and Substitutes. In SIGIR 2015. 43–52.

[15] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. DistributedRepresentations of Words and Phrases and their Compositionality. In NeurIPS2013. 3111–3119.

[16] R. Pasricha and J. J. McAuley. 2018. Translation-based factorization machines forsequential recommendation. In RecSys 2018. 63–71.

[17] M. Quadrana, A. Karatzoglou, B. Hidasi, and P. Cremonesi. 2017. PersonalizingSession-based Recommendations with Hierarchical Recurrent Neural Networks.In RecSys 2017. 130–137.

[18] Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten deRijke. 2019. RepeatNet: A Repeat Aware Neural Recommendation Machine forSession-Based Recommendation. In AAAI 2019. 4806–4813.

[19] R. Ren, Z. Liu, Y. Li, W. X. Zhao, H. Wang, B. Ding, and J.-R. Wen. 2020. SequentialRecommendation with Self-Attentive Multi-Adversarial Network. In SIGIR 2020.89–98.

[20] S. Rendle. 2010. Factorization Machines. In ICDM 2010. 995–1000.[21] S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme. 2010. Factorizing personal-

ized Markov chains for next-basket recommendation. In WWW 2010. 811–820.[22] W. Song, C. Shi, Z. Xiao, Z. Duan, Y. Xu, M. Zhang, and J. Tang. 2019. AutoInt:

Automatic Feature Interaction Learning via Self-Attentive Neural Networks. InCIKM 2019. 1161–1170.

[23] F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang. 2019. BERT4Rec: SequentialRecommendation with Bidirectional Encoder Representations from Transformer.In CIKM 2019. 1441–1450.

[24] J. Tang and K. Wang. 2018. Personalized Top-N Sequential Recommendation viaConvolutional Sequence Embedding. In WSDM 2018. 565–573.

[25] A. van den Oord, Y. Li, and O. Vinyals. 2018. Representation Learning withContrastive Predictive Coding. CoRR abs/1807.03748 (2018). arXiv:1807.03748

[26] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser,and I. Polosukhin. 2017. Attention is All you Need. In NeurIPS 2017. 5998–6008.

[27] Xin Xin, Alexandros Karatzoglou, Ioannis Arapakis, and Joemon M. Jose. 2020.Self-Supervised Reinforcement Learning for Recommender Systems. In SIGIR2020. 931–940.

[28] Y.-T. Yeh and Y.-N. Chen. 2019. QAInfomax: Learning Robust Question AnsweringSystem byMutual InformationMaximization. In EMNLP-IJCNLP 2019. 3368–3373.

[29] T. Zhang, P. Zhao, Y. Liu, V. S. Sheng, J. Xu, D. Wang, G. Liu, and X. Zhou. 2019.Feature-level Deeper Self-Attention Network for Sequential Recommendation.In IJCAI 2019. 4320–4326.

[30] Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, andJingsong Yu. 2020. Improving Conversational Recommender Systems via Knowl-edge Graph based Semantic Fusion. In KDD 2020.

http://arxiv.org/abs/1807.03748

s3-rec: self-supervised learning for sequential ...existing neural sequential recom-mendation models...

Documents