Upload File

multi page search with reinforcement learning to rankjunxu/publications/ictir2018... · multi page...

  • Home
  • Documents
  • Multi Page Search with Reinforcement Learning to Rankjunxu/publications/ICTIR2018... · Multi Page Search with Reinforcement Learning to Rank Wei Zeng, Jun Xu, Yanyan Lan, Jiafeng
4
Multi Page Search with Reinforcement Learning to Rank Wei Zeng, Jun Xu * , Yanyan Lan, Jiafeng Guo, Xueqi Cheng 1 University of Chinese Academy of Sciences, Beijing, China 2 CAS Key Lab of Network Data Science & Technology, Institute of Computing Technology, Chinese Academy of Sciences [email protected],{junxu,lanyanyan,guojiafeng,cxq}@ict.ac.cn ABSTRACT Web search engines are typically designed to involve multiple pages of search results, and the search users engaging in exploratory search with ad hoc queries are likely to access more than one result pages. The ranking of web pages for such queries should consider additional information other than the original query, e.g., the user clicks on previous result pages. Existing methods that utilize this kind of information usually involve relevance feedback, which uses the feedback information to explore the user’s intent. However, due to the limitation of the feedback mechanism, it is dicult to apply existing relevance feedback techniques to state-of-the-art learning to rank models. In this paper, we propose a novel learning to rank model for multi page search in which the user’s feedback can be naturally utilized for improving the ranking of next result page. The model, referred to as MDP-MPS, formalizes the ranking of documents in multi page search as a Markov decision process (MDP) in which the search engine corresponds to the agent for constructing the document rankings in the result pages, and the user corresponds to the environment for judging the rankings and providing rewards. The policy gradient algorithm of REINFORCE is adopted for learning the model parameters. Experimental results on OHSUMED dataset showed that our approach outperformed the baselines of traditional relevance ranking model of ListNet and relevance feedback method of Rocchio. KEYWORDS Multi Page Search; Reinforcement Learning ACM Reference Format: Wei Zeng, Jun Xu * , Yanyan Lan, Jiafeng Guo, Xueqi Cheng. 2018. Multi Page Search with Reinforcement Learning to Rank. In ICTIR ’18: 2018 ACM SIGIR International Conference on the Theory of Information Retrieval, September 14–17, 2018, Tianjin, China. ACM, New York, NY, USA, 4 pages. https://doi. org/10.1145/3234944.3234977 1 INTRODUCTION The modern information retrieval interface typically involves mul- tiple pages of search results and users are likely to access more than one page. In many common retrieval scenarios, the search results * Corresponding author: Jun Xu Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and the full citation on the rst page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specic permission and/or a fee. Request permissions from [email protected]. ICTIR ’18, September 14–17, 2018, Tianjin, China © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-5656-5/18/09. . . $15.00 https://doi.org/10.1145/3234944.3234977 are split into multiple pages that the user traverses across by click- ing ’next page’ button. The users examine a Search Engine Result Page(SERP) by browsing the rank list from top to down, click on relevant documents and return to the SERP in the same session. A good multi-page search system should begin by a static method and could continue to adapt the model based on the feedback from the user. Relevance feedback method [7] has been proved very eective for improving retrieval accuracy over the interactive information retrieval tasks. As for the Rocchio algorithm [11], the search engine gets the feedback information from the user, and then add weight of the terms from known relevant documents or minus weight of the terms from known irrelevant documents. Most relevant feedback methods balance the initial query and the feedback information based on a xed value, rather than apply an adaptive coecient for each query and each set of feedback. Learning to rank methods have been widely used for information retrieval, in which all the documents are represented by feature vectors to reect the relevance of the documents to the query [8]. The learning to rank methods aim to learn a score function for the candidate documents by minimizing a carefully designed loss function. Our work considers the multi page search scenario, and applies relevance feedback techniques to state-of-art learning to rank models. The multi-page search process is an interactive process between the user and the search engine. At each time step, the search engine selects M documents to construct a rank list. The user browses this rank list from top to down, clicks the relevant documents, skips the irrelevant documents, and then clicks the "next page" button for more results. In this paper, we mathematically formulate the multi page search process as an Markov Decision Process. The search engine is treated as the agent, which selects documents from remaining candidate document set to deliver to the user for satisfying the user’s information need. The state of the environment is consist of the query, remaining documents, rank position and user’s click information. We apply the soft-max policy to balance the exploration and exploitation during training and design the reward on the basis of IR measure metric. A classical policy gradient policy method based on the REINFORCE is applied to optimize the search policy. In this paper, we propose a technical schema to use user feedback from the top-ranked documents to generate re-ranking for the re- maining documents. Compared with existing methods, our method enjoys the following advantages: 1) Formulate the multi-page pro- cess as an Markov Decision Process and apply policy gradient to train the search policy which could optimize the search measure metric directly. 2) Apply the Recurrent Neural Network to process the feedback and improve the traditional learning to rank model with the feedback information based on Rocchio. We use traditional learning to rank method ListNet, RankNet and RankBoost as the initial model to construct the experiments

Upload: others

Post on 09-Jun-2020

4 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Multi Page Search with Reinforcement Learning to Rankjunxu/publications/ICTIR2018... · Multi Page Search with Reinforcement Learning to Rank Wei Zeng, Jun Xu, Yanyan Lan, Jiafeng

Multi Page Search with Reinforcement Learning to RankWei Zeng, Jun Xu∗, Yanyan Lan, Jiafeng Guo, Xueqi Cheng

1University of Chinese Academy of Sciences, Beijing, China2CAS Key Lab of Network Data Science & Technology, Institute of Computing Technology, Chinese Academy of Sciences

[email protected],{junxu,lanyanyan,guojiafeng,cxq}@ict.ac.cn

ABSTRACTWeb search engines are typically designed to involve multiple pagesof search results, and the search users engaging in exploratorysearch with ad hoc queries are likely to access more than one resultpages. The ranking of web pages for such queries should consideradditional information other than the original query, e.g., the userclicks on previous result pages. Existing methods that utilize thiskind of information usually involve relevance feedback, which usesthe feedback information to explore the user’s intent. However,due to the limitation of the feedback mechanism, it is dicult toapply existing relevance feedback techniques to state-of-the-artlearning to rank models. In this paper, we propose a novel learningto rank model for multi page search in which the user’s feedbackcan be naturally utilized for improving the ranking of next resultpage. The model, referred to as MDP-MPS, formalizes the rankingof documents in multi page search as a Markov decision process(MDP) in which the search engine corresponds to the agent forconstructing the document rankings in the result pages, and theuser corresponds to the environment for judging the rankings andproviding rewards. The policy gradient algorithm of REINFORCEis adopted for learning the model parameters. Experimental resultson OHSUMED dataset showed that our approach outperformedthe baselines of traditional relevance ranking model of ListNet andrelevance feedback method of Rocchio.

KEYWORDSMulti Page Search; Reinforcement LearningACM Reference Format:Wei Zeng, Jun Xu∗, Yanyan Lan, Jiafeng Guo, Xueqi Cheng. 2018. Multi PageSearch with Reinforcement Learning to Rank. In ICTIR ’18: 2018 ACM SIGIRInternational Conference on the Theory of Information Retrieval, September14–17, 2018, Tianjin, China. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3234944.3234977

1 INTRODUCTIONThe modern information retrieval interface typically involves mul-tiple pages of search results and users are likely to access more thanone page. In many common retrieval scenarios, the search results∗ Corresponding author: Jun Xu

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor prot or commercial advantage and that copies bear this notice and the full citationon the rst page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specic permission and/or afee. Request permissions from [email protected] ’18, September 14–17, 2018, Tianjin, China© 2018 Association for Computing Machinery.ACM ISBN 978-1-4503-5656-5/18/09. . . $15.00https://doi.org/10.1145/3234944.3234977

are split into multiple pages that the user traverses across by click-ing ’next page’ button. The users examine a Search Engine ResultPage(SERP) by browsing the rank list from top to down, click onrelevant documents and return to the SERP in the same session. Agood multi-page search system should begin by a static method andcould continue to adapt the model based on the feedback from theuser. Relevance feedback method [7] has been proved very eectivefor improving retrieval accuracy over the interactive informationretrieval tasks. As for the Rocchio algorithm [11], the search enginegets the feedback information from the user, and then add weight ofthe terms from known relevant documents or minus weight of theterms from known irrelevant documents. Most relevant feedbackmethods balance the initial query and the feedback informationbased on a xed value, rather than apply an adaptive coecient foreach query and each set of feedback.

Learning to rank methods have been widely used for informationretrieval, in which all the documents are represented by featurevectors to reect the relevance of the documents to the query [8].The learning to rank methods aim to learn a score function forthe candidate documents by minimizing a carefully designed lossfunction. Our work considers the multi page search scenario, andapplies relevance feedback techniques to state-of-art learning torankmodels. Themulti-page search process is an interactive processbetween the user and the search engine. At each time step, thesearch engine selects M documents to construct a rank list. Theuser browses this rank list from top to down, clicks the relevantdocuments, skips the irrelevant documents, and then clicks the"next page" button formore results. In this paper, wemathematicallyformulate the multi page search process as an Markov DecisionProcess. The search engine is treated as the agent, which selectsdocuments from remaining candidate document set to deliver tothe user for satisfying the user’s information need. The state of theenvironment is consist of the query, remaining documents, rankposition and user’s click information. We apply the soft-max policyto balance the exploration and exploitation during training anddesign the reward on the basis of IR measure metric. A classicalpolicy gradient policy method based on the REINFORCE is appliedto optimize the search policy.

In this paper, we propose a technical schema to use user feedbackfrom the top-ranked documents to generate re-ranking for the re-maining documents. Compared with existing methods, our methodenjoys the following advantages: 1) Formulate the multi-page pro-cess as an Markov Decision Process and apply policy gradient totrain the search policy which could optimize the search measuremetric directly. 2) Apply the Recurrent Neural Network to processthe feedback and improve the traditional learning to rank modelwith the feedback information based on Rocchio.

We use traditional learning to rank method ListNet, RankNetand RankBoost as the initial model to construct the experiments

https://doi.org/10.1145/3234944.3234977
https://doi.org/10.1145/3234944.3234977
https://doi.org/10.1145/3234944.3234977
Page 2: Multi Page Search with Reinforcement Learning to Rankjunxu/publications/ICTIR2018... · Multi Page Search with Reinforcement Learning to Rank Wei Zeng, Jun Xu, Yanyan Lan, Jiafeng

on OHSUMED dataset, simulate the interaction between the searchengine and user based on Dependent Click Model (DCM). Theexperimental result show that our model can prove the rankingaccuracy for traditional learning to rank method and has bettergeneralization ability.

2 RELATEDWORKIn interactive information retrial, several methods have been pro-posed to utilize the relevance feedback information to modied themodel. Relevance feedback has been extensively studied and mostof the methods take it as a learning problemwith s special treatmentof query. For example, the well known Rocchio algorithm [11, 12]uses explicit feedback to improve query. The balance parameter isusually set to be a xed value across all the queries and collections.However, due to the dierence in queries and feedback documents,it should be optimized for each query and each set of feedback.In [10], presented a learning approach to adaptively predict theoptimal balance coecient for each query and each collection andgiven three heuristics to characterize the balance between queryand feedback information.

A similar approach was explored by [6], where formulated thisproblem as a Bayesian sequential model and used a dynamic pro-gramming approach for optimizing. In [16], [17] and [15], hasadapted Markov Decision Process to model the diverse ranking,session search and relevance ranking process. Online learning torank methods aim to incrementally learn from user feedback in realtime and formulated as reinforcement learning problem in[5]. Dif-ferent these methods, we take the multi page search scenarios, usethe Recurrent Neural Network to repress the feedback informationand modied the initial static learning to rank model inspired bythe Rocchio algorithm.

3 PROBLEM FORMULATION3.1 Multi Page Search as MDPThe multi-page search process can be treated as an interactiveranking process between the search engine and the user, just asgure 1 shown. At each time step, the search engine selects Mdocuments from the candidate documents set to construct a searchresult page and delivers it to the user. The user receives thesedocuments, browses them from top to down, clicks the relevantdocuments and skips the irrelevant documents. Then, the user willclicks the ’next page’ button, and the search engine constructsanother search result page from the remaining candidate documentby considering the user’s click information. This process can bemathematically formulated as a Markov Decision Process(MDP),which can be represented as a tuple 〈S,A,T ,R,π 〉 composed bystates, actions, transition, reward, and policy, which are respectivelydened as follows:

State S describes the environment. In multi pages search process,the state can be the ranking position to calculate the reward of thereceived document, the relevant ranked documents, the irrelevantranked documents as well as the the remaining documents. A tupleis given to describe the state st :

[t ,X tr ,X

tir,X

tc ] (1)

Search engine

Query

User

page one

doc1doc3

doc4doc2

doc5

clicked docs skipped docs

doc1 doc2 doc5· · ·Feedback info

doc6 doc7 doc10· · ·page two

Figure 1: The Multi Page Search process.

where t is the time step, X tr and X t

ir are the relevant ranked docu-ments and irrelevant ranked documents lists respectively, X t

c is theremaining documents set.

Action A is taken by the agent to inuence the state of the envi-ronment. At each time step, the search engine selectsM documentsto construct the search result page. However, there are still an insur-mountable number of possible rank lists, even if restrict the searchagent to only select M documents. To lower the computationalcomplexity of generating the search result page, we consider thesearch engine select one document at a time. So, at the time stept , at ∈ A(st ) represents to select a document xm(at ) ∈ X

tc for the

ranking position t + 1, wherem(at ) is the index of the documentselected by at .

Transition T(S,A) is the dynamic of the environment, whichcan be described as a function T : S × A → S which maps astate st into a new state st+1 of the environment in response tothe selected action at . For multi page search, select a documentat means removing the document xm(at ) out of the candidate set.And once the user clicks the ’next page’ button, the search enginegains the judgements over the documents of current page. So thetransition function can be described by the following equation:

st+1 = T([t ,Xtr ,X

tir,X

tc ],at )

=

{[t + 1,X t

r + {xr},X tir + {xir},X

tc \ {xm(at )}] interaction

[t + 1,X tr ,X

tir,X

tc \ {xm(at )}] otherwise ,

(2)where interaction means the user clicks the ’next page’ button. And{xr} and {xir} is the rank list consisted by the relevant documentsand irrelevant documents of current page respectively.

Reward R(S,A) measures the inuence to the environment bythe selected action. For multi page search, the reward should beable to evaluate the quality of the chosen document. We design thereward function on the basis of the IR evaluation measures, thepromotion of the DCG [3]:

RDCG(st ,at ) =

{2ym(at ) − 1 t = 02ym(at )−1log2(t+1)

t > 0 , (3)

where ym(at ) is the relevance label of the selected document.Policy π (a |s) : A × S → [0, 1] describes the behaviors of the

agent, which is a probabilistic distribution over the possible actions.We apply a soft-max policy which based on the Gibbs distribution,which could be described as:

π (at |st ) =exp { f (at , st )}∑

a∈A(st ) exp { f (at , st )}, (4)

where f (at , st ) can be seen as a score function to each documents.The details of this score function f (at , st ) will be described in thenext section.

Page 3: Multi Page Search with Reinforcement Learning to Rankjunxu/publications/ICTIR2018... · Multi Page Search with Reinforcement Learning to Rank Wei Zeng, Jun Xu, Yanyan Lan, Jiafeng

query represent

relevant docs

irrelevant docs

doc1

doc3 doc5 doc6 doc10

doc9doc8doc4doc2 doc7 �<latexit sha1_base64="TeASWWecrkjItL2+A1nlGMAOjBY=">AAAB7HicbVBNT8JAEJ3iF+IX6tFLIzHxRFoveiTh4hETCyTQkO0yhQ3bbbM7NSGE3+DFg8Z49Qd589+4QA8KvmSSl/dmMjMvyqQw5HnfTmlre2d3r7xfOTg8Oj6pnp61TZprjgFPZaq7ETMohcKABEnsZhpZEknsRJPmwu88oTYiVY80zTBM2EiJWHBGVgr6ERIbVGte3VvC3SR+QWpQoDWofvWHKc8TVMQlM6bnexmFM6ZJcInzSj83mDE+YSPsWapYgiacLY+du1dWGbpxqm0pcpfq74kZS4yZJpHtTBiNzbq3EP/zejnFd+FMqCwnVHy1KM6lS6m7+NwdCo2c5NQSxrWwt7p8zDTjZPOp2BD89Zc3Sfum7nt1/8GrNZpFHGW4gEu4Bh9uoQH30IIAOAh4hld4c5Tz4rw7H6vWklPMnMMfOJ8/wuOOoQ==</latexit><latexit sha1_base64="TeASWWecrkjItL2+A1nlGMAOjBY=">AAAB7HicbVBNT8JAEJ3iF+IX6tFLIzHxRFoveiTh4hETCyTQkO0yhQ3bbbM7NSGE3+DFg8Z49Qd589+4QA8KvmSSl/dmMjMvyqQw5HnfTmlre2d3r7xfOTg8Oj6pnp61TZprjgFPZaq7ETMohcKABEnsZhpZEknsRJPmwu88oTYiVY80zTBM2EiJWHBGVgr6ERIbVGte3VvC3SR+QWpQoDWofvWHKc8TVMQlM6bnexmFM6ZJcInzSj83mDE+YSPsWapYgiacLY+du1dWGbpxqm0pcpfq74kZS4yZJpHtTBiNzbq3EP/zejnFd+FMqCwnVHy1KM6lS6m7+NwdCo2c5NQSxrWwt7p8zDTjZPOp2BD89Zc3Sfum7nt1/8GrNZpFHGW4gEu4Bh9uoQH30IIAOAh4hld4c5Tz4rw7H6vWklPMnMMfOJ8/wuOOoQ==</latexit><latexit sha1_base64="TeASWWecrkjItL2+A1nlGMAOjBY=">AAAB7HicbVBNT8JAEJ3iF+IX6tFLIzHxRFoveiTh4hETCyTQkO0yhQ3bbbM7NSGE3+DFg8Z49Qd589+4QA8KvmSSl/dmMjMvyqQw5HnfTmlre2d3r7xfOTg8Oj6pnp61TZprjgFPZaq7ETMohcKABEnsZhpZEknsRJPmwu88oTYiVY80zTBM2EiJWHBGVgr6ERIbVGte3VvC3SR+QWpQoDWofvWHKc8TVMQlM6bnexmFM6ZJcInzSj83mDE+YSPsWapYgiacLY+du1dWGbpxqm0pcpfq74kZS4yZJpHtTBiNzbq3EP/zejnFd+FMqCwnVHy1KM6lS6m7+NwdCo2c5NQSxrWwt7p8zDTjZPOp2BD89Zc3Sfum7nt1/8GrNZpFHGW4gEu4Bh9uoQH30IIAOAh4hld4c5Tz4rw7H6vWklPMnMMfOJ8/wuOOoQ==</latexit><latexit sha1_base64="TeASWWecrkjItL2+A1nlGMAOjBY=">AAAB7HicbVBNT8JAEJ3iF+IX6tFLIzHxRFoveiTh4hETCyTQkO0yhQ3bbbM7NSGE3+DFg8Z49Qd589+4QA8KvmSSl/dmMjMvyqQw5HnfTmlre2d3r7xfOTg8Oj6pnp61TZprjgFPZaq7ETMohcKABEnsZhpZEknsRJPmwu88oTYiVY80zTBM2EiJWHBGVgr6ERIbVGte3VvC3SR+QWpQoDWofvWHKc8TVMQlM6bnexmFM6ZJcInzSj83mDE+YSPsWapYgiacLY+du1dWGbpxqm0pcpfq74kZS4yZJpHtTBiNzbq3EP/zejnFd+FMqCwnVHy1KM6lS6m7+NwdCo2c5NQSxrWwt7p8zDTjZPOp2BD89Zc3Sfum7nt1/8GrNZpFHGW4gEu4Bh9uoQH30IIAOAh4hld4c5Tz4rw7H6vWklPMnMMfOJ8/wuOOoQ==</latexit>

RNN represent

RNN represent

�<latexit sha1_base64="ps4okQhgqARfAR4hcXrXVmWLKXk=">AAAB7XicbVA9SwNBEJ3zM8avqKXNYhCswp2NloE0lhHMByRHmNvsJWt2947dPSEc+Q82ForY+n/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9or7x8cHh1XTk7bJsk0ZS2aiER3IzRMcMVallvBuqlmKCPBOtGkMfc7T0wbnqgHO01ZKHGkeMwpWie1+yOUEgeVql/zFyDrJChIFQo0B5Wv/jChmWTKUoHG9AI/tWGO2nIq2KzczwxLkU5wxHqOKpTMhPni2hm5dMqQxIl2pSxZqL8ncpTGTGXkOiXasVn15uJ/Xi+z8W2Yc5Vmlim6XBRngtiEzF8nQ64ZtWLqCFLN3a2EjlEjtS6gsgshWH15nbSva4FfC+79ar1RxFGCc7iAKwjgBupwB01oAYVHeIZXePMS78V79z6WrRteMXMGf+B9/gCF+I8S</latexit><latexit sha1_base64="ps4okQhgqARfAR4hcXrXVmWLKXk=">AAAB7XicbVA9SwNBEJ3zM8avqKXNYhCswp2NloE0lhHMByRHmNvsJWt2947dPSEc+Q82ForY+n/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9or7x8cHh1XTk7bJsk0ZS2aiER3IzRMcMVallvBuqlmKCPBOtGkMfc7T0wbnqgHO01ZKHGkeMwpWie1+yOUEgeVql/zFyDrJChIFQo0B5Wv/jChmWTKUoHG9AI/tWGO2nIq2KzczwxLkU5wxHqOKpTMhPni2hm5dMqQxIl2pSxZqL8ncpTGTGXkOiXasVn15uJ/Xi+z8W2Yc5Vmlim6XBRngtiEzF8nQ64ZtWLqCFLN3a2EjlEjtS6gsgshWH15nbSva4FfC+79ar1RxFGCc7iAKwjgBupwB01oAYVHeIZXePMS78V79z6WrRteMXMGf+B9/gCF+I8S</latexit><latexit sha1_base64="ps4okQhgqARfAR4hcXrXVmWLKXk=">AAAB7XicbVA9SwNBEJ3zM8avqKXNYhCswp2NloE0lhHMByRHmNvsJWt2947dPSEc+Q82ForY+n/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9or7x8cHh1XTk7bJsk0ZS2aiER3IzRMcMVallvBuqlmKCPBOtGkMfc7T0wbnqgHO01ZKHGkeMwpWie1+yOUEgeVql/zFyDrJChIFQo0B5Wv/jChmWTKUoHG9AI/tWGO2nIq2KzczwxLkU5wxHqOKpTMhPni2hm5dMqQxIl2pSxZqL8ncpTGTGXkOiXasVn15uJ/Xi+z8W2Yc5Vmlim6XBRngtiEzF8nQ64ZtWLqCFLN3a2EjlEjtS6gsgshWH15nbSva4FfC+79ar1RxFGCc7iAKwjgBupwB01oAYVHeIZXePMS78V79z6WrRteMXMGf+B9/gCF+I8S</latexit><latexit sha1_base64="ps4okQhgqARfAR4hcXrXVmWLKXk=">AAAB7XicbVA9SwNBEJ3zM8avqKXNYhCswp2NloE0lhHMByRHmNvsJWt2947dPSEc+Q82ForY+n/s/Ddukis08cHA470ZZuZFqeDG+v63t7G5tb2zW9or7x8cHh1XTk7bJsk0ZS2aiER3IzRMcMVallvBuqlmKCPBOtGkMfc7T0wbnqgHO01ZKHGkeMwpWie1+yOUEgeVql/zFyDrJChIFQo0B5Wv/jChmWTKUoHG9AI/tWGO2nIq2KzczwxLkU5wxHqOKpTMhPni2hm5dMqQxIl2pSxZqL8ncpTGTGXkOiXasVn15uJ/Xi+z8W2Yc5Vmlim6XBRngtiEzF8nQ64ZtWLqCFLN3a2EjlEjtS6gsgshWH15nbSva4FfC+79ar1RxFGCc7iAKwjgBupwB01oAYVHeIZXePMS78V79z6WrRteMXMGf+B9/gCF+I8S</latexit>

·<latexit sha1_base64="TLIdTs0AzYQ0FV877TCsMY98R+I=">AAAB7XicbVBNS8NAEJ34WetX1aOXxSJ4KokXPRZ68VjBfkAbymazaddusmF3IpTQ/+DFgyJe/T/e/Ddu2hy09cHA470ZZuYFqRQGXffb2djc2t7ZrexV9w8Oj45rJ6ddozLNeIcpqXQ/oIZLkfAOCpS8n2pO40DyXjBtFX7viWsjVPKAs5T7MR0nIhKMopW6QxYqrI5qdbfhLkDWiVeSOpRoj2pfw1CxLOYJMkmNGXhuin5ONQom+bw6zAxPKZvSMR9YmtCYGz9fXDsnl1YJSaS0rQTJQv09kdPYmFkc2M6Y4sSseoX4nzfIMLr1c5GkGfKELRdFmSSoSPE6CYXmDOXMEsq0sLcSNqGaMrQBFSF4qy+vk+51w3Mb3r1bb7bKOCpwDhdwBR7cQBPuoA0dYPAIz/AKb45yXpx352PZuuGUM2fwB87nDw5AjsM=</latexit><latexit sha1_base64="TLIdTs0AzYQ0FV877TCsMY98R+I=">AAAB7XicbVBNS8NAEJ34WetX1aOXxSJ4KokXPRZ68VjBfkAbymazaddusmF3IpTQ/+DFgyJe/T/e/Ddu2hy09cHA470ZZuYFqRQGXffb2djc2t7ZrexV9w8Oj45rJ6ddozLNeIcpqXQ/oIZLkfAOCpS8n2pO40DyXjBtFX7viWsjVPKAs5T7MR0nIhKMopW6QxYqrI5qdbfhLkDWiVeSOpRoj2pfw1CxLOYJMkmNGXhuin5ONQom+bw6zAxPKZvSMR9YmtCYGz9fXDsnl1YJSaS0rQTJQv09kdPYmFkc2M6Y4sSseoX4nzfIMLr1c5GkGfKELRdFmSSoSPE6CYXmDOXMEsq0sLcSNqGaMrQBFSF4qy+vk+51w3Mb3r1bb7bKOCpwDhdwBR7cQBPuoA0dYPAIz/AKb45yXpx352PZuuGUM2fwB87nDw5AjsM=</latexit><latexit sha1_base64="TLIdTs0AzYQ0FV877TCsMY98R+I=">AAAB7XicbVBNS8NAEJ34WetX1aOXxSJ4KokXPRZ68VjBfkAbymazaddusmF3IpTQ/+DFgyJe/T/e/Ddu2hy09cHA470ZZuYFqRQGXffb2djc2t7ZrexV9w8Oj45rJ6ddozLNeIcpqXQ/oIZLkfAOCpS8n2pO40DyXjBtFX7viWsjVPKAs5T7MR0nIhKMopW6QxYqrI5qdbfhLkDWiVeSOpRoj2pfw1CxLOYJMkmNGXhuin5ONQom+bw6zAxPKZvSMR9YmtCYGz9fXDsnl1YJSaS0rQTJQv09kdPYmFkc2M6Y4sSseoX4nzfIMLr1c5GkGfKELRdFmSSoSPE6CYXmDOXMEsq0sLcSNqGaMrQBFSF4qy+vk+51w3Mb3r1bb7bKOCpwDhdwBR7cQBPuoA0dYPAIz/AKb45yXpx352PZuuGUM2fwB87nDw5AjsM=</latexit><latexit sha1_base64="TLIdTs0AzYQ0FV877TCsMY98R+I=">AAAB7XicbVBNS8NAEJ34WetX1aOXxSJ4KokXPRZ68VjBfkAbymazaddusmF3IpTQ/+DFgyJe/T/e/Ddu2hy09cHA470ZZuYFqRQGXffb2djc2t7ZrexV9w8Oj45rJ6ddozLNeIcpqXQ/oIZLkfAOCpS8n2pO40DyXjBtFX7viWsjVPKAs5T7MR0nIhKMopW6QxYqrI5qdbfhLkDWiVeSOpRoj2pfw1CxLOYJMkmNGXhuin5ONQom+bw6zAxPKZvSMR9YmtCYGz9fXDsnl1YJSaS0rQTJQv09kdPYmFkc2M6Y4sSseoX4nzfIMLr1c5GkGfKELRdFmSSoSPE6CYXmDOXMEsq0sLcSNqGaMrQBFSF4qy+vk+51w3Mb3r1bb7bKOCpwDhdwBR7cQBPuoA0dYPAIz/AKb45yXpx352PZuuGUM2fwB87nDw5AjsM=</latexit>

·<latexit sha1_base64="TLIdTs0AzYQ0FV877TCsMY98R+I=">AAAB7XicbVBNS8NAEJ34WetX1aOXxSJ4KokXPRZ68VjBfkAbymazaddusmF3IpTQ/+DFgyJe/T/e/Ddu2hy09cHA470ZZuYFqRQGXffb2djc2t7ZrexV9w8Oj45rJ6ddozLNeIcpqXQ/oIZLkfAOCpS8n2pO40DyXjBtFX7viWsjVPKAs5T7MR0nIhKMopW6QxYqrI5qdbfhLkDWiVeSOpRoj2pfw1CxLOYJMkmNGXhuin5ONQom+bw6zAxPKZvSMR9YmtCYGz9fXDsnl1YJSaS0rQTJQv09kdPYmFkc2M6Y4sSseoX4nzfIMLr1c5GkGfKELRdFmSSoSPE6CYXmDOXMEsq0sLcSNqGaMrQBFSF4qy+vk+51w3Mb3r1bb7bKOCpwDhdwBR7cQBPuoA0dYPAIz/AKb45yXpx352PZuuGUM2fwB87nDw5AjsM=</latexit><latexit sha1_base64="TLIdTs0AzYQ0FV877TCsMY98R+I=">AAAB7XicbVBNS8NAEJ34WetX1aOXxSJ4KokXPRZ68VjBfkAbymazaddusmF3IpTQ/+DFgyJe/T/e/Ddu2hy09cHA470ZZuYFqRQGXffb2djc2t7ZrexV9w8Oj45rJ6ddozLNeIcpqXQ/oIZLkfAOCpS8n2pO40DyXjBtFX7viWsjVPKAs5T7MR0nIhKMopW6QxYqrI5qdbfhLkDWiVeSOpRoj2pfw1CxLOYJMkmNGXhuin5ONQom+bw6zAxPKZvSMR9YmtCYGz9fXDsnl1YJSaS0rQTJQv09kdPYmFkc2M6Y4sSseoX4nzfIMLr1c5GkGfKELRdFmSSoSPE6CYXmDOXMEsq0sLcSNqGaMrQBFSF4qy+vk+51w3Mb3r1bb7bKOCpwDhdwBR7cQBPuoA0dYPAIz/AKb45yXpx352PZuuGUM2fwB87nDw5AjsM=</latexit><latexit sha1_base64="TLIdTs0AzYQ0FV877TCsMY98R+I=">AAAB7XicbVBNS8NAEJ34WetX1aOXxSJ4KokXPRZ68VjBfkAbymazaddusmF3IpTQ/+DFgyJe/T/e/Ddu2hy09cHA470ZZuYFqRQGXffb2djc2t7ZrexV9w8Oj45rJ6ddozLNeIcpqXQ/oIZLkfAOCpS8n2pO40DyXjBtFX7viWsjVPKAs5T7MR0nIhKMopW6QxYqrI5qdbfhLkDWiVeSOpRoj2pfw1CxLOYJMkmNGXhuin5ONQom+bw6zAxPKZvSMR9YmtCYGz9fXDsnl1YJSaS0rQTJQv09kdPYmFkc2M6Y4sSseoX4nzfIMLr1c5GkGfKELRdFmSSoSPE6CYXmDOXMEsq0sLcSNqGaMrQBFSF4qy+vk+51w3Mb3r1bb7bKOCpwDhdwBR7cQBPuoA0dYPAIz/AKb45yXpx352PZuuGUM2fwB87nDw5AjsM=</latexit><latexit sha1_base64="TLIdTs0AzYQ0FV877TCsMY98R+I=">AAAB7XicbVBNS8NAEJ34WetX1aOXxSJ4KokXPRZ68VjBfkAbymazaddusmF3IpTQ/+DFgyJe/T/e/Ddu2hy09cHA470ZZuYFqRQGXffb2djc2t7ZrexV9w8Oj45rJ6ddozLNeIcpqXQ/oIZLkfAOCpS8n2pO40DyXjBtFX7viWsjVPKAs5T7MR0nIhKMopW6QxYqrI5qdbfhLkDWiVeSOpRoj2pfw1CxLOYJMkmNGXhuin5ONQom+bw6zAxPKZvSMR9YmtCYGz9fXDsnl1YJSaS0rQTJQv09kdPYmFkc2M6Y4sSseoX4nzfIMLr1c5GkGfKELRdFmSSoSPE6CYXmDOXMEsq0sLcSNqGaMrQBFSF4qy+vk+51w3Mb3r1bb7bKOCpwDhdwBR7cQBPuoA0dYPAIz/AKb45yXpx352PZuuGUM2fwB87nDw5AjsM=</latexit>

xT

PNr

k xrk

|PNr

k xrk|

<latexit sha1_base64="OFRUekaXquJDk2xqDxZJwXtEcZ4=">AAACXXiclVFNTwIxFOyuX7iioh48eGkkJp7Irhc9knDxZDABIWFx0y1dbeh+pH1rIGX/pDe9+FfsAgkKXpykyWRmXvo6DTPBFbjuh2Vvbe/s7lX2nYPq4dFx7eT0SaW5pKxLU5HKfkgUEzxhXeAgWD+TjMShYL1w3Cr93huTiqdJB6YZG8bkJeERpwSMFNTAjwm8hpGeFM8d7EeSUO2rPA7Gz/oh0D6wCWhZFAVeBUtvZejZv/KzwnGCWt1tuHPgTeItSR0t0Q5q7/4opXnMEqCCKDXw3AyGmkjgVLDC8XPFMkLH5IUNDE1IzNRQz9sp8JVRRjhKpTkJ4Ln6c0KTWKlpHJpkubJa90rxL2+QQ3Q31DzJcmAJXVwU5QJDisuq8YhLRkFMDSFUcrMrpq/ENAzmQ8oSvPUnb5Knm4bnNrxHt95sLeuooAt0ia6Rh25RE92jNuoiij4tZO1bjvVl79hV+2gRta3lzBn6Bfv8G0JWuk8=</latexit><latexit sha1_base64="OFRUekaXquJDk2xqDxZJwXtEcZ4=">AAACXXiclVFNTwIxFOyuX7iioh48eGkkJp7Irhc9knDxZDABIWFx0y1dbeh+pH1rIGX/pDe9+FfsAgkKXpykyWRmXvo6DTPBFbjuh2Vvbe/s7lX2nYPq4dFx7eT0SaW5pKxLU5HKfkgUEzxhXeAgWD+TjMShYL1w3Cr93huTiqdJB6YZG8bkJeERpwSMFNTAjwm8hpGeFM8d7EeSUO2rPA7Gz/oh0D6wCWhZFAVeBUtvZejZv/KzwnGCWt1tuHPgTeItSR0t0Q5q7/4opXnMEqCCKDXw3AyGmkjgVLDC8XPFMkLH5IUNDE1IzNRQz9sp8JVRRjhKpTkJ4Ln6c0KTWKlpHJpkubJa90rxL2+QQ3Q31DzJcmAJXVwU5QJDisuq8YhLRkFMDSFUcrMrpq/ENAzmQ8oSvPUnb5Knm4bnNrxHt95sLeuooAt0ia6Rh25RE92jNuoiij4tZO1bjvVl79hV+2gRta3lzBn6Bfv8G0JWuk8=</latexit><latexit sha1_base64="OFRUekaXquJDk2xqDxZJwXtEcZ4=">AAACXXiclVFNTwIxFOyuX7iioh48eGkkJp7Irhc9knDxZDABIWFx0y1dbeh+pH1rIGX/pDe9+FfsAgkKXpykyWRmXvo6DTPBFbjuh2Vvbe/s7lX2nYPq4dFx7eT0SaW5pKxLU5HKfkgUEzxhXeAgWD+TjMShYL1w3Cr93huTiqdJB6YZG8bkJeERpwSMFNTAjwm8hpGeFM8d7EeSUO2rPA7Gz/oh0D6wCWhZFAVeBUtvZejZv/KzwnGCWt1tuHPgTeItSR0t0Q5q7/4opXnMEqCCKDXw3AyGmkjgVLDC8XPFMkLH5IUNDE1IzNRQz9sp8JVRRjhKpTkJ4Ln6c0KTWKlpHJpkubJa90rxL2+QQ3Q31DzJcmAJXVwU5QJDisuq8YhLRkFMDSFUcrMrpq/ENAzmQ8oSvPUnb5Knm4bnNrxHt95sLeuooAt0ia6Rh25RE92jNuoiij4tZO1bjvVl79hV+2gRta3lzBn6Bfv8G0JWuk8=</latexit><latexit sha1_base64="OFRUekaXquJDk2xqDxZJwXtEcZ4=">AAACXXiclVFNTwIxFOyuX7iioh48eGkkJp7Irhc9knDxZDABIWFx0y1dbeh+pH1rIGX/pDe9+FfsAgkKXpykyWRmXvo6DTPBFbjuh2Vvbe/s7lX2nYPq4dFx7eT0SaW5pKxLU5HKfkgUEzxhXeAgWD+TjMShYL1w3Cr93huTiqdJB6YZG8bkJeERpwSMFNTAjwm8hpGeFM8d7EeSUO2rPA7Gz/oh0D6wCWhZFAVeBUtvZejZv/KzwnGCWt1tuHPgTeItSR0t0Q5q7/4opXnMEqCCKDXw3AyGmkjgVLDC8XPFMkLH5IUNDE1IzNRQz9sp8JVRRjhKpTkJ4Ln6c0KTWKlpHJpkubJa90rxL2+QQ3Q31DzJcmAJXVwU5QJDisuq8YhLRkFMDSFUcrMrpq/ENAzmQ8oSvPUnb5Knm4bnNrxHt95sLeuooAt0ia6Rh25RE92jNuoiij4tZO1bjvVl79hV+2gRta3lzBn6Bfv8G0JWuk8=</latexit>

xTPNir

k xirk

|PNirk xir

k |<latexit sha1_base64="J211U5ZncQ3ASmUyahrJl8HYT38=">AAACYXiclVFNS8MwGE7r1G3qrHrcJTgET6P1okfBiydRcB+wbiXN0hmWfpC8lY3YP+nNixf/iOk2cG5efCHw8HyQN0/CTHAFrvth2TuV3b39aq1+cHjUOHZOTrsqzSVlHZqKVPZDopjgCesAB8H6mWQkDgXrhdO7Uu+9Mql4mjzDPGPDmEwSHnFKwFCBM/NjAi9hpGfF6Bn7kSRU+yqPg+lIPwTaBzYDzWVRFPjHWYprin77Z+KtqNcDp+W23cXgbeCtQAut5jFw3v1xSvOYJUAFUWrguRkMNZHAqWBF3c8VywidkgkbGJiQmKmhXjRU4AvDjHGUSnMSwAt2PaFJrNQ8Do2z3FltaiX5lzbIIboZap5kObCELi+KcoEhxWXdeMwloyDmBhAqudkV0xdiWgbzKWUJ3uaTt0H3qu25be/Ja93ereqooiY6R5fIQ9foFt2jR9RBFH1aFevIalhfds127NOl1bZWmTP0a+zmNxlqvBw=</latexit><latexit sha1_base64="J211U5ZncQ3ASmUyahrJl8HYT38=">AAACYXiclVFNS8MwGE7r1G3qrHrcJTgET6P1okfBiydRcB+wbiXN0hmWfpC8lY3YP+nNixf/iOk2cG5efCHw8HyQN0/CTHAFrvth2TuV3b39aq1+cHjUOHZOTrsqzSVlHZqKVPZDopjgCesAB8H6mWQkDgXrhdO7Uu+9Mql4mjzDPGPDmEwSHnFKwFCBM/NjAi9hpGfF6Bn7kSRU+yqPg+lIPwTaBzYDzWVRFPjHWYprin77Z+KtqNcDp+W23cXgbeCtQAut5jFw3v1xSvOYJUAFUWrguRkMNZHAqWBF3c8VywidkgkbGJiQmKmhXjRU4AvDjHGUSnMSwAt2PaFJrNQ8Do2z3FltaiX5lzbIIboZap5kObCELi+KcoEhxWXdeMwloyDmBhAqudkV0xdiWgbzKWUJ3uaTt0H3qu25be/Ja93ereqooiY6R5fIQ9foFt2jR9RBFH1aFevIalhfds127NOl1bZWmTP0a+zmNxlqvBw=</latexit><latexit sha1_base64="J211U5ZncQ3ASmUyahrJl8HYT38=">AAACYXiclVFNS8MwGE7r1G3qrHrcJTgET6P1okfBiydRcB+wbiXN0hmWfpC8lY3YP+nNixf/iOk2cG5efCHw8HyQN0/CTHAFrvth2TuV3b39aq1+cHjUOHZOTrsqzSVlHZqKVPZDopjgCesAB8H6mWQkDgXrhdO7Uu+9Mql4mjzDPGPDmEwSHnFKwFCBM/NjAi9hpGfF6Bn7kSRU+yqPg+lIPwTaBzYDzWVRFPjHWYprin77Z+KtqNcDp+W23cXgbeCtQAut5jFw3v1xSvOYJUAFUWrguRkMNZHAqWBF3c8VywidkgkbGJiQmKmhXjRU4AvDjHGUSnMSwAt2PaFJrNQ8Do2z3FltaiX5lzbIIboZap5kObCELi+KcoEhxWXdeMwloyDmBhAqudkV0xdiWgbzKWUJ3uaTt0H3qu25be/Ja93ereqooiY6R5fIQ9foFt2jR9RBFH1aFevIalhfds127NOl1bZWmTP0a+zmNxlqvBw=</latexit><latexit sha1_base64="J211U5ZncQ3ASmUyahrJl8HYT38=">AAACYXiclVFNS8MwGE7r1G3qrHrcJTgET6P1okfBiydRcB+wbiXN0hmWfpC8lY3YP+nNixf/iOk2cG5efCHw8HyQN0/CTHAFrvth2TuV3b39aq1+cHjUOHZOTrsqzSVlHZqKVPZDopjgCesAB8H6mWQkDgXrhdO7Uu+9Mql4mjzDPGPDmEwSHnFKwFCBM/NjAi9hpGfF6Bn7kSRU+yqPg+lIPwTaBzYDzWVRFPjHWYprin77Z+KtqNcDp+W23cXgbeCtQAut5jFw3v1xSvOYJUAFUWrguRkMNZHAqWBF3c8VywidkgkbGJiQmKmhXjRU4AvDjHGUSnMSwAt2PaFJrNQ8Do2z3FltaiX5lzbIIboZap5kObCELi+KcoEhxWXdeMwloyDmBhAqudkV0xdiWgbzKWUJ3uaTt0H3qu25be/Ja93ereqooiY6R5fIQ9foFt2jR9RBFH1aFevIalhfds127NOl1bZWmTP0a+zmNxlqvBw=</latexit>

+<latexit sha1_base64="13uYDxgX3EBX4RalW3Z1t7ESa7A=">AAAB6nicbVBNS8NAEJ3Urxq/qh69LBZBEEriRY+FXjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQc/7dkobm1vbO+Vdd2//4PCocnzSNkmmGW+xRCa6G1LDpVC8hQIl76aa0ziUvBNOGnO/88S1EYl6xGnKg5iOlIgEo2ilhyvXHVSqXs1bgKwTvyBVKNAcVL76w4RlMVfIJDWm53spBjnVKJjkM7efGZ5SNqEj3rNU0ZibIF+cOiMXVhmSKNG2FJKF+nsip7Ex0zi0nTHFsVn15uJ/Xi/D6DbIhUoz5IotF0WZJJiQ+d9kKDRnKKeWUKaFvZWwMdWUoU1nHoK/+vI6aV/XfK/m33vVeqOIowxncA6X4MMN1OEOmtACBiN4hld4c6Tz4rw7H8vWklPMnMIfOJ8/2QyM1A==</latexit><latexit sha1_base64="13uYDxgX3EBX4RalW3Z1t7ESa7A=">AAAB6nicbVBNS8NAEJ3Urxq/qh69LBZBEEriRY+FXjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQc/7dkobm1vbO+Vdd2//4PCocnzSNkmmGW+xRCa6G1LDpVC8hQIl76aa0ziUvBNOGnO/88S1EYl6xGnKg5iOlIgEo2ilhyvXHVSqXs1bgKwTvyBVKNAcVL76w4RlMVfIJDWm53spBjnVKJjkM7efGZ5SNqEj3rNU0ZibIF+cOiMXVhmSKNG2FJKF+nsip7Ex0zi0nTHFsVn15uJ/Xi/D6DbIhUoz5IotF0WZJJiQ+d9kKDRnKKeWUKaFvZWwMdWUoU1nHoK/+vI6aV/XfK/m33vVeqOIowxncA6X4MMN1OEOmtACBiN4hld4c6Tz4rw7H8vWklPMnMIfOJ8/2QyM1A==</latexit><latexit sha1_base64="13uYDxgX3EBX4RalW3Z1t7ESa7A=">AAAB6nicbVBNS8NAEJ3Urxq/qh69LBZBEEriRY+FXjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQc/7dkobm1vbO+Vdd2//4PCocnzSNkmmGW+xRCa6G1LDpVC8hQIl76aa0ziUvBNOGnO/88S1EYl6xGnKg5iOlIgEo2ilhyvXHVSqXs1bgKwTvyBVKNAcVL76w4RlMVfIJDWm53spBjnVKJjkM7efGZ5SNqEj3rNU0ZibIF+cOiMXVhmSKNG2FJKF+nsip7Ex0zi0nTHFsVn15uJ/Xi/D6DbIhUoz5IotF0WZJJiQ+d9kKDRnKKeWUKaFvZWwMdWUoU1nHoK/+vI6aV/XfK/m33vVeqOIowxncA6X4MMN1OEOmtACBiN4hld4c6Tz4rw7H8vWklPMnMIfOJ8/2QyM1A==</latexit><latexit sha1_base64="13uYDxgX3EBX4RalW3Z1t7ESa7A=">AAAB6nicbVBNS8NAEJ3Urxq/qh69LBZBEEriRY+FXjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQc/7dkobm1vbO+Vdd2//4PCocnzSNkmmGW+xRCa6G1LDpVC8hQIl76aa0ziUvBNOGnO/88S1EYl6xGnKg5iOlIgEo2ilhyvXHVSqXs1bgKwTvyBVKNAcVL76w4RlMVfIJDWm53spBjnVKJjkM7efGZ5SNqEj3rNU0ZibIF+cOiMXVhmSKNG2FJKF+nsip7Ex0zi0nTHFsVn15uJ/Xi/D6DbIhUoz5IotF0WZJJiQ+d9kKDRnKKeWUKaFvZWwMdWUoU1nHoK/+vI6aV/XfK/m33vVeqOIowxncA6X4MMN1OEOmtACBiN4hld4c6Tz4rw7H8vWklPMnMIfOJ8/2QyM1A==</latexit>

+<latexit sha1_base64="13uYDxgX3EBX4RalW3Z1t7ESa7A=">AAAB6nicbVBNS8NAEJ3Urxq/qh69LBZBEEriRY+FXjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQc/7dkobm1vbO+Vdd2//4PCocnzSNkmmGW+xRCa6G1LDpVC8hQIl76aa0ziUvBNOGnO/88S1EYl6xGnKg5iOlIgEo2ilhyvXHVSqXs1bgKwTvyBVKNAcVL76w4RlMVfIJDWm53spBjnVKJjkM7efGZ5SNqEj3rNU0ZibIF+cOiMXVhmSKNG2FJKF+nsip7Ex0zi0nTHFsVn15uJ/Xi/D6DbIhUoz5IotF0WZJJiQ+d9kKDRnKKeWUKaFvZWwMdWUoU1nHoK/+vI6aV/XfK/m33vVeqOIowxncA6X4MMN1OEOmtACBiN4hld4c6Tz4rw7H8vWklPMnMIfOJ8/2QyM1A==</latexit><latexit sha1_base64="13uYDxgX3EBX4RalW3Z1t7ESa7A=">AAAB6nicbVBNS8NAEJ3Urxq/qh69LBZBEEriRY+FXjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQc/7dkobm1vbO+Vdd2//4PCocnzSNkmmGW+xRCa6G1LDpVC8hQIl76aa0ziUvBNOGnO/88S1EYl6xGnKg5iOlIgEo2ilhyvXHVSqXs1bgKwTvyBVKNAcVL76w4RlMVfIJDWm53spBjnVKJjkM7efGZ5SNqEj3rNU0ZibIF+cOiMXVhmSKNG2FJKF+nsip7Ex0zi0nTHFsVn15uJ/Xi/D6DbIhUoz5IotF0WZJJiQ+d9kKDRnKKeWUKaFvZWwMdWUoU1nHoK/+vI6aV/XfK/m33vVeqOIowxncA6X4MMN1OEOmtACBiN4hld4c6Tz4rw7H8vWklPMnMIfOJ8/2QyM1A==</latexit><latexit sha1_base64="13uYDxgX3EBX4RalW3Z1t7ESa7A=">AAAB6nicbVBNS8NAEJ3Urxq/qh69LBZBEEriRY+FXjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQc/7dkobm1vbO+Vdd2//4PCocnzSNkmmGW+xRCa6G1LDpVC8hQIl76aa0ziUvBNOGnO/88S1EYl6xGnKg5iOlIgEo2ilhyvXHVSqXs1bgKwTvyBVKNAcVL76w4RlMVfIJDWm53spBjnVKJjkM7efGZ5SNqEj3rNU0ZibIF+cOiMXVhmSKNG2FJKF+nsip7Ex0zi0nTHFsVn15uJ/Xi/D6DbIhUoz5IotF0WZJJiQ+d9kKDRnKKeWUKaFvZWwMdWUoU1nHoK/+vI6aV/XfK/m33vVeqOIowxncA6X4MMN1OEOmtACBiN4hld4c6Tz4rw7H8vWklPMnMIfOJ8/2QyM1A==</latexit><latexit sha1_base64="13uYDxgX3EBX4RalW3Z1t7ESa7A=">AAAB6nicbVBNS8NAEJ3Urxq/qh69LBZBEEriRY+FXjxWtB/QhrLZbtqlm03YnQgl9Cd48aCIV3+RN/+N2zYHbX0w8Hhvhpl5YSqFQc/7dkobm1vbO+Vdd2//4PCocnzSNkmmGW+xRCa6G1LDpVC8hQIl76aa0ziUvBNOGnO/88S1EYl6xGnKg5iOlIgEo2ilhyvXHVSqXs1bgKwTvyBVKNAcVL76w4RlMVfIJDWm53spBjnVKJjkM7efGZ5SNqEj3rNU0ZibIF+cOiMXVhmSKNG2FJKF+nsip7Ex0zi0nTHFsVn15uJ/Xi/D6DbIhUoz5IotF0WZJJiQ+d9kKDRnKKeWUKaFvZWwMdWUoU1nHoK/+vI6aV/XfK/m33vVeqOIowxncA6X4MMN1OEOmtACBiN4hld4c6Tz4rw7H8vWklPMnMIfOJ8/2QyM1A==</latexit>

xT q<latexit sha1_base64="F4Wrmkprqx9NX+RjtBuc6IxuG20=">AAAB63icbVA9T8MwEL2Ur1K+AowsFhUSU5WwwFipC2OR+iW1oXJcp7VqO8F2EFXUv8DCAEKs/CE2/g1OmwFannTS03t3ursXJpxp43nfTmljc2t7p7xb2ds/ODxyj086Ok4VoW0S81j1QqwpZ5K2DTOc9hJFsQg57YbTRu53H6nSLJYtM0toIPBYsogRbHLp6b71MHSrXs1bAK0TvyBVKNAcul+DUUxSQaUhHGvd973EBBlWhhFO55VBqmmCyRSPad9SiQXVQba4dY4urDJCUaxsSYMW6u+JDAutZyK0nQKbiV71cvE/r5+a6CbImExSQyVZLopSjkyM8sfRiClKDJ9Zgoli9lZEJlhhYmw8FRuCv/ryOulc1Xyv5t951XqjiKMMZ3AOl+DDNdThFprQBgITeIZXeHOE8+K8Ox/L1pJTzJzCHzifPw+pjjo=</latexit><latexit sha1_base64="F4Wrmkprqx9NX+RjtBuc6IxuG20=">AAAB63icbVA9T8MwEL2Ur1K+AowsFhUSU5WwwFipC2OR+iW1oXJcp7VqO8F2EFXUv8DCAEKs/CE2/g1OmwFannTS03t3ursXJpxp43nfTmljc2t7p7xb2ds/ODxyj086Ok4VoW0S81j1QqwpZ5K2DTOc9hJFsQg57YbTRu53H6nSLJYtM0toIPBYsogRbHLp6b71MHSrXs1bAK0TvyBVKNAcul+DUUxSQaUhHGvd973EBBlWhhFO55VBqmmCyRSPad9SiQXVQba4dY4urDJCUaxsSYMW6u+JDAutZyK0nQKbiV71cvE/r5+a6CbImExSQyVZLopSjkyM8sfRiClKDJ9Zgoli9lZEJlhhYmw8FRuCv/ryOulc1Xyv5t951XqjiKMMZ3AOl+DDNdThFprQBgITeIZXeHOE8+K8Ox/L1pJTzJzCHzifPw+pjjo=</latexit><latexit sha1_base64="F4Wrmkprqx9NX+RjtBuc6IxuG20=">AAAB63icbVA9T8MwEL2Ur1K+AowsFhUSU5WwwFipC2OR+iW1oXJcp7VqO8F2EFXUv8DCAEKs/CE2/g1OmwFannTS03t3ursXJpxp43nfTmljc2t7p7xb2ds/ODxyj086Ok4VoW0S81j1QqwpZ5K2DTOc9hJFsQg57YbTRu53H6nSLJYtM0toIPBYsogRbHLp6b71MHSrXs1bAK0TvyBVKNAcul+DUUxSQaUhHGvd973EBBlWhhFO55VBqmmCyRSPad9SiQXVQba4dY4urDJCUaxsSYMW6u+JDAutZyK0nQKbiV71cvE/r5+a6CbImExSQyVZLopSjkyM8sfRiClKDJ9Zgoli9lZEJlhhYmw8FRuCv/ryOulc1Xyv5t951XqjiKMMZ3AOl+DDNdThFprQBgITeIZXeHOE8+K8Ox/L1pJTzJzCHzifPw+pjjo=</latexit><latexit sha1_base64="F4Wrmkprqx9NX+RjtBuc6IxuG20=">AAAB63icbVA9T8MwEL2Ur1K+AowsFhUSU5WwwFipC2OR+iW1oXJcp7VqO8F2EFXUv8DCAEKs/CE2/g1OmwFannTS03t3ursXJpxp43nfTmljc2t7p7xb2ds/ODxyj086Ok4VoW0S81j1QqwpZ5K2DTOc9hJFsQg57YbTRu53H6nSLJYtM0toIPBYsogRbHLp6b71MHSrXs1bAK0TvyBVKNAcul+DUUxSQaUhHGvd973EBBlWhhFO55VBqmmCyRSPad9SiQXVQba4dY4urDJCUaxsSYMW6u+JDAutZyK0nQKbiV71cvE/r5+a6CbImExSQyVZLopSjkyM8sfRiClKDJ9Zgoli9lZEJlhhYmw8FRuCv/ryOulc1Xyv5t951XqjiKMMZ3AOl+DDNdThFprQBgITeIZXeHOE8+K8Ox/L1pJTzJzCHzifPw+pjjo=</latexit>

Figure 2: The framework for Multi-Page Search.

doc1 doc9doc8doc4doc2 doc7

h1 h2 h3 h4 h5 h6 �WRNN WRNN WRNN WRNN WRNN wout

Winput<latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="vs9ab4kWCFb03AIBSzY8OTf+Fpo=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw7116s3zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB91XmEg=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit>

Winput<latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="vs9ab4kWCFb03AIBSzY8OTf+Fpo=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw7116s3zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB91XmEg=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit>

Winput<latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="vs9ab4kWCFb03AIBSzY8OTf+Fpo=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw7116s3zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB91XmEg=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit>

Winput<latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="vs9ab4kWCFb03AIBSzY8OTf+Fpo=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw7116s3zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB91XmEg=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit>

Winput<latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="vs9ab4kWCFb03AIBSzY8OTf+Fpo=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw7116s3zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB91XmEg=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit>

Winput<latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="BDCnKKKTXV97L23WeUAaX1ghILY=">AAAB93icbVBNS8NAFHzxs9aq0Zt4CRbBU0m86FHw4rGC/YAmhM120y7dbMLui1hC8OJf8eJBEf+JN/+Nm7YHbR1YGGbeY99MlAmu0XW/rbX1jc2t7dpOfbext39gHza6Os0VZR2ailT1I6KZ4JJ1kKNg/UwxkkSC9aLJTeX3HpjSPJX3OM1YkJCR5DGnBI0U2sd+QnAcxUWvDAsf2SMWXGY5lmVoN92WO4OzSrwFacIC7dD+8ocpzRMmkQqi9cBzMwwKopBTwcq6n2uWETohIzYwVJKE6aCYRSidM6MMnThV5kl0ZurvjYIkWk+TyExWB+tlrxL/8wY5xlfBPBOTdP5RnAsHU6fqwxlyxSiKqSGEKm5udeiYKELRtFY3JXjLkVdJ96LluS3vzoUanMApnIMHl3ANt9CGDlB4ghd4g3fr2Xq1PuZ1rVmL3o7gD6zPH1fZlvU=</latexit><latexit sha1_base64="vs9ab4kWCFb03AIBSzY8OTf+Fpo=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw7116s3zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB91XmEg=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit><latexit sha1_base64="Z5xDOmDs9sdm8WMpGoig2XdRG1M=">AAACAnicbVDLSsNAFJ3UV62vqCtxM1gEF1ISN7osuHFZwT6gDWEynbRDJ5MwcyOWENz4K25cKOLWr3Dn3zhps9DWAwOHc+5l7jlBIrgGx/m2Kiura+sb1c3a1vbO7p69f9DRcaooa9NYxKoXEM0El6wNHATrJYqRKBCsG0yuC797z5TmsbyDacK8iIwkDzklYCTfPhpEBMZBmHVzPxsAe4CMyySFPPftutNwZsDLxC1JHZVo+fbXYBjTNGISqCBa910nAS8jCjgVLK8NUs0SQidkxPqGShIx7WWzCDk+NcoQh7EyTwKeqb83MhJpPY0CM1kcrBe9QvzP66cQXnnzTEzS+UdhKjDEuOgDD7liFMTUEEIVN7diOiaKUDCt1UwJ7mLkZdK5aLhOw711683zso4qOkYn6Ay56BI10Q1qoTai6BE9o1f0Zj1ZL9a79TEfrVjlziH6A+vzB92nmEk=</latexit>

Figure 3: Calculate the coecient β .

3.2 Involve the feedbackRelevant feedback technology has been proved can eectively im-prove the rank accuracy. For example, the well known Rocchioalgorithm [11, 12] uses explicit feedback to improve the user’squery. The formula and variable denitions dened as:

Qi+1 = αQi + β∑rel

Di|Di |− γ

∑irrel

Di|Di |

(5)

where, Qi is the i-th iteration queries, and Di represents the docu-ment which has been delivered to the user.

Inspired by Rocchio algorithm, we try to improve the tradi-tional learning to rank method based on the relevance feedbacktechnology. In traditional learning to rank settings [9] , the datais represented as {qn,Xn,Y n }Nn=1 , where X (n)=

{x(n)1 , · · · ,x(n)Mn

}is the

query-document feature set for query qn , Y (n)={y (n)1 , · · · ,y (n)Mn

}is the

relevance label set, and Mn is the number of the candidate docu-ments. The learning to rank methods aim to learn a score functionf (x), which is used to compute the relevant score for each candi-date document. The search result rank list is generated by sortingthe candidate documents by these scores. We try to adopt Rocchiostyle algorithm to update the score of the remaining documentsbased on the user click feedback information from previous pages.

fnew(x) = f (x) + β · xT∑xrel

|∑xrel |

+ γ · xT∑xirrel

|∑xirrel |

(6)

where xrel and xirrel represent the query-document feature of therelevant ranked document and irrelevant document respectively.

As equation 6 shown, the score of a remaining document isformed by three part. The rst part computes the score based onthe traditional learning to rank model, which learned by traditionallearning to rank method. The second part, measures the inuencefrom the relevant ranked documents, and the third part, measuresthe inuence from the irrelevant ranked documents. Note that thenormalization procedure ensures the equal emphasis of the impactfrom relevant ranked documents and irrelevant ranked documents.

The parameters, β and γ , are set to describe the inuence fromthe relevant ranked documents and irrelevant documents. In thispaper, we apply an recurrent neural network (RNN) to representthe ranked documents and calculate the parameters based on these

Algorithm 1 MDP for Multi Page RankInput: Labeled training set D = {(q(n), X (n), Y (n))}Nn=1 , learning rate η, discount

factor γ̂ , reward function R ,Output: The model parameters Θ =

{Wrelinput,W

relRNN,wrel

out,Wirrelinput,W

irrelRNN,wirrel

out

}1: Initialize the parameters Θ← random values2: for epoch = 1, 2, 3, · · · do3: ∆Θ = 04: for each query q do5: Select topM documents from the candidate document set XC to construct

a rank list {x}.6: The user bowsers the rank list, click the relevant documents {xrel } and

skips the irrelevant documents {xirrel }.7: Initialize s0 ← [0, {xrel }, {xirrel }, XC \ {x}], and episode E ← ∅8: for i = 0 to T do9: for t = i ∗M + 1 to (i + 1) ∗M do10: Sample an action at ∈ A(st ) ∼ π (at |st ;w)11: Calculate the reward rt+1 ← R(st , at )12: Append (st , at , rt+1) at the end of the episode13: if mod(t, M ) == 0 then14: Transit state by the relevant documents {xrel } and the irrelevant

documents {xirrel } on current page.st+1 ←

[t+1,XMr +{xir},XM

ir +{xirrel},Xtc \{xm(at ) }

]15: else16: State transition st+1 ←

[t+1,XMr ,XM

ir ,Xtc \{xm(at ) }]

17: end if18: end for19: end for20: for t = 0 to (t + 1) ∗M do21: Gt ←

∑(t+1)∗Mk=t γ̂ k−t rt

22: ∆Θ← ∆Θ + ηγ̂ t−1Gt ∇Θ log π (at |st ;Θ)23: end for24: end for25: Update the model parameter Θ← Θ + η∆Θ26: end for27: return w

represents. Figure 3 shows the details about how to calculate theparameter β based on the relevant ranked documents.

ht+1 = σ(Wrel

imputxm(at ) +WrelRNNht

)β =Wrel

outhNrel

(7)

where Nrel is the total number of the relevant ranked documents.The parameter γ is calculated in a similar way.

3.3 Learning with Policy GradientIn this paper, we calculate the parameter β and γ dynamically on

the basis of the relevant ranked documents and irrelevant rankeddocuments. The parameters of our model can be represented as:

Θ ={Wrel

input,WrelRNN,w

relout,W

irrelinput,W

irrelRNN,w

irrelout

}(8)

The policy gradient method [13, 14] is used to learn the pa-rameters with the objective to maximize the expected discountcumulative reward, which dened as:

L(Θ) = EE∼π [M ·T∑k=1

γ̂k−1rk ]. (9)

where E is the ranking list sampled by the search engine based oncurrent policy π ,M is the number of the documents per search page,T is the number of interaction between the user the search engine,γ̂ is the discount rate which set as 1 to guarantee the consistencyover the objective and IR measure metric.

According to REINFORCE algorithm, the gradient can be∇ΘL(Θ) = γ̂

tGt∇Θ logπΘ(at |st ;Θ), (10)

Page 4: Multi Page Search with Reinforcement Learning to Rankjunxu/publications/ICTIR2018... · Multi Page Search with Reinforcement Learning to Rank Wei Zeng, Jun Xu, Yanyan Lan, Jiafeng

Table 1: Ranking accuracies based on ListNet.Method NDCG@1 NDCG@10 NDCG@15 NDCG@20ListNet 0.4943 0.4438 0.4383 0.4401Rocchio 0.4943 0.4438 0.4419 0.4405

MDP-MPS 0.4943 0.4438 0.4436 0.4453

Table 2: Ranking accuracies based on RankBoost.Method NDCG@1 NDCG@10 NDCG@15 NDCG@20

RankBoost 0.5199 0.4521 0.4440 0.4449Rocchio 0.5199 0.4521 0.4438 0.4359

MDP-MPS 0.5199 0.4521 0.4459 0.4452

Table 3: Ranking accuracies based on RankNet.Method NDCG@1 NDCG@10 NDCG@15 NDCG@20RankNet 0.5039 0.4347 0.4305 0.4335Rocchio 0.5039 0.4347 0.4345 0.4335

MDP-MPS 0.5039 0.4347 0.4365 0.4358

which is further be estimated with Monte-Carlo sampling. Algo-rithm 1 shows the procedure of optimize the parameters. At eachiteration, an episode is sampled according to current policy. Then,at each time step t of the sampled episode, the model parame-ters are adjusted according to the gradients of the parameters∇Θ logπ (at |st ;Θ), scaled by the step size η, the discount rate γ̂ t ,and the long-term return of the sampled episode starting fromposition t , denoted as Gt : Gt =

∑M ·Tk=t γ̂k−t rt .

4 EXPERIMENTSWe simulate the user’s interaction by an evaluation setup proposedin [5]. This setup combine datasets with explicit relevance judge-ments that are typically used for supervised learning to rank withrecently developed click models. This click model is based on theDependent Click Model (DCM), a generalization of the cascademodel. The model posits that users travers result lists from top tobutton, examining each document as it is encountered. We use theperfect model, where all relevant documents are clicked and nonon-relevant documents are clicked.

The experiments are constructed over the learning to rank dataset"OHSUMED". We take the classical learning to rank methods List-Net [2] RankNet [1] and RankBoost [4] as the initial methods andsetM = 10 as it provides reasonable and meaningful results withinthe scope of our test data set and is reective of actual search sys-tem. The grid search strategy is applied to nd the best parametersβ ∈ [−5, 5] and γ ∈ [−5, 5] based on the validation data set.

The table 1 and 2 give us the experiment results of our methodas take the ListNet and RankBoost as the initial learning to rankmodel representatively. And the boldface indicates the highest scoreamong all runs. From the results, we can see that our methods canoutperform the Rocchio algorithm in both ListNet and RankBoost.It should note that, Rocchio algorithm would decrease the perfor-mance of the initial RankBoost method, as for dierent queriessatisfy dierent balance coecient. However, our method MDP-MPS gain a commensurate performance, which mean our methodhas better generalization ability.

5 CONCLUSIONThe paper formulates the ranking process in multi page search as anMarkov Decision Process and optimizes the model parameters withpolicy gradient. Specically, on the basis of a traditional learning

to rank model, the traditional relevance feedback model Rocchiois used to adapt it for the task of multi page search in which aRecurrent Neural Network is used to represent the user feedbackinformation. Experimental results based on the OHSUMED datasetshowed that the proposed MDP-MPS model, initialized with thedocument ranking generated by ListNet and RankBoost, can outper-formed the corresponding baselines, respectively. The experimentalresults showed the eectiveness of MDP-MPS for multi page search.The results also showed that MDP-MPS has better generalizationability in the multi page search task.

6 ACKNOWLEDGMENTSThis work was funded by the National Key R&D Program of Chinaunder Grants No. 2016QY02D0405, the 973 Program of China underGrant No. 2014CB340401, the National Natural Science Foundationof China (NSFC) under Grants No. 61773362, 61425016, 61472401,61722211, and 20180290, and the Youth Innovation Promotion As-sociation CAS under Grants No. 20144310, and 2016102.

REFERENCES[1] Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton,

and Greg Hullender. 2005. Learning to rank using gradient descent. In Proceedingsof the 22nd international conference on Machine learning. 89–96.

[2] Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learningto rank: from pairwise approach to listwise approach. In Proceedings of the 24thinternational conference on Machine learning. 129–136.

[3] Charles LA Clarke, Maheedhar Kolla, Gordon V Cormack, Olga Vechtomova,Azin Ashkan, Stefan Büttcher, and Ian MacKinnon. 2008. Novelty and diversityin information retrieval evaluation. In Proceedings of the 31st annual internationalACM SIGIR conference. ACM, 659–666.

[4] Yoav Freund, Raj Iyer, Robert E Schapire, and Yoram Singer. 2003. An ecientboosting algorithm for combining preferences. Journal of machine learningresearch 4, Nov (2003), 933–969.

[5] Katja Hofmann, Shimon Whiteson, and Maarten de Rijke. 2011. Balancing ex-ploration and exploitation in learning to rank online. In European Conference onInformation Retrieval. 251–263.

[6] Xiaoran Jin, Marc Sloan, and Jun Wang. 2013. Interactive exploratory search formulti page search results. In Proceedings of the 22nd international conference onWorld Wide Web. ACM, 655–666.

[7] Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski,and Geri Gay. 2007. Evaluating the accuracy of implicit feedback from clicks andquery reformulations in web search. ACM Transactions on Information Systems(TOIS) 25, 2 (2007), 7.

[8] Tie-Yan Liu et al. 2009. Learning to rank for information retrieval. Foundationsand Trends® in Information Retrieval 3, 3 (2009), 225–331.

[9] Tie-Yan Liu, Jun Xu, Tao Qin, Wenying Xiong, and Hang Li. 2007. Letor: Bench-mark dataset for research on learning to rank for information retrieval. In Pro-ceedings of SIGIR 2007 workshop, Vol. 310.

[10] Yuanhua Lv and ChengXiang Zhai. 2009. Adaptive relevance feedback in infor-mation retrieval. In Proceedings of the 18th ACM conference on Information andknowledge management. 255–264.

[11] Joseph John Rocchio. 1971. Relevance feedback in information retrieval. THESMART RETRIEVAL SYSTEM: Experiments in Automatic Document Processing(1971), 313–323.

[12] Gerard Salton and Chris Buckley. 1990. Improving retrieval performance byrelevance feedback. Vol. 41. Wiley Online Library, 288–297.

[13] Richard S Sutton and Andrew G Barto. 1998. Reinforcement learning: An intro-duction. Vol. 1. MIT press Cambridge.

[14] Richard S Sutton, David AMcAllester, Satinder P Singh, and YishayMansour. 2000.Policy gradient methods for reinforcement learning with function approximation.In Advances in neural information processing systems. 1057–1063.

[15] Zeng Wei, Jun Xu, Yanyan Lan, Jiafeng Guo, and Xueqi Cheng. 2017. Reinforce-ment learning to rank with Markov decision process. In Proceedings of the 40thInternational ACM SIGIR Conference. 945–948.

[16] Long Xia, Jun Xu, Yanyan Lan, Jiafeng Guo, Wei Zeng, and Xueqi Cheng. 2017.Adapting Markov decision process for search result diversication. In Proceedingsof the 40th International ACM SIGIR Conference. 535–544.

[17] Hui Yang, Dongyi Guan, and Sicong Zhang. 2015. The query change model:Modeling session search as a markov decision process. ACM Transactions onInformation Systems (TOIS) 33, 4 (2015), 20.


Transformation Driven Visual Reasoning · 2021. 6. 11. · Transformation Driven Visual Reasoning Xin Hong1,2 Yanyan Lan3,* Liang Pang1,2 Jiafeng Guo1,2 Xueqi Cheng1,2 1 CAS Key Laboratory

Inverse Reinforcement Learning - Peoplecbfinn/_files/bootcamp_inverserl.pdf · Apprenticeship Learning via Inverse Reinforcement Learning. Good introduction to inverse reinforcement

Reinforcement Learning or Active Inference?karl/Reinforcement Learning or Active... · Reinforcement Learning or Active Inference? ... From the point of view of reinforcement learning

MalagosGardenResort - Yanyan

Schedules of reinforcement. Schedules of Reinforcement Continuous reinforcement refers to reinforcement being administered to each instance of a response

Guide to Historical Reinforcement - SRIA Concrete 2017 Historical Reinforcement... · Guide to Historical Reinforcement ... reinforcement material properties to use when checking

Reinforcement Learning Lecture Inverse Reinforcement Learningipvs.informatik.uni-stuttgart.de/mlr/wp-content/uploads/2017/07/09... · Reinforcement Learning Inverse Reinforcement

Teacher: Wu Yanyan Students: Pre-Junior Class Four Discussing what to eat

Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning Applications Summary

1 A Biterm Topic Model for Short Texts Xiaohui Yan, Jiafeng Guo, Yanyan Lan, Xueqi Cheng Institute of Computing Technology, Chinese Academy of Sciences

Reinforcement Learning - Multi-Agent Reinforcement

Reinforcement and deep reinforcement learning for wireless

Multi-Objective Reinforcement Learning using Sets of Pareto … · 2020. 10. 19. · learning and multi-objective reinforcement learning. 2.1 Reinforcement Learning A reinforcement

McAfee Suyang “ sean ” hong , yanyan xu

Bayesian Reinforcement Learning in Continuous …MotivationBayesian Reinforcement LearningExperiments and ApplicationsConclusion Bayesian Reinforcement Learning in Continuous POMDPs

Unit 1 Whos younger? (Part A) Fun with English 6B Gewu Primary School Zhao Yanyan

Yue Guo-an Chen Hao Zhang Yanyan

Jiafeng Cui , Tengfei Huang

Jiafeng Guo, Gu Xu, Xueqi Cheng, Hang Li Presentation by Gonçalo Simões Course: Recuperação de Informação SIGIR 2009

More Than Relevance: High Utility Query Recommendation By Mining Users' Search Behaviors Xiaofei Zhu, Jiafeng Guo, Xueqi Cheng, Yanyan Lan Institute of

Is Top-k Sufficient for Ranking? Yanyan Lan, Shuzi Niu, Jiafeng Guo, Xueqi Cheng Institute of Computing Technology, Chinese Academy of Sciences

Reinforcement Learning - uni-freiburg.degki.informatik.uni-freiburg.de/.../recordings/reinforcement.pdf · Reinforcement Learning 3 What is Reinforcement Learning? Learning from interaction

From Reinforcement Learning to Deep Reinforcement …fagostin/assets/files/...Keywords: Machine learning · Reinforcement learning Deep learning · Deep reinforcement learning 1 Introduction

Eick: Reinforcement Learning. Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning

1 A Probabilistic Model for Bursty Topic Discovery in Microblogs Xiaohui Yan, Jiafeng Guo, Yanyan Lan, Jun Xu, Xueqi Cheng CAS Key Laboratory of Web Data

Reinforcement Learning and Deep Reinforcement Learningcse.ucdenver.edu/.../Class-22-Reinforcement-learning-DL.pdf · 2018. 11. 28. · Outlines 1 Principles of Reinforcement Learning

Yanyan Shen Department of Computer Science and Engineeringshen-yy/cs359/slides/1_fundamentals_A.pdf · Yanyan Shen Department of Computer Science and Engineering. 2 ... 5.3 80186

Jidong Chen Xiaofeng Meng Yanyan Guo S.Grumbach Hui Sun

FLOLAB – FLAT PLATE Anoop Samant Yanyan Zhang Saptarshi Basu Andres Chaparro

Inverse Reinforcement Learning CS885 Reinforcement

Discrete Structures Lecture 13: Trees Ji Yanyan United International College Thanks to Professor Michael Hvidsten

Continuous hoops for transverse reinforcement of ... · Continuous hoops for transverse reinforcement of ... transverse reinforcement details for the ... Fig. 1 Transverse reinforcement

Discrete Structures Lecture 11: Algorithms Miss, Yanyan,Ji United International College Thanks to Professor Michael Hvidsten

Skinner’s Behavioral Reinforcement Theory Positive Reinforcement Negative Reinforcement PunishmentExtinction Person repeats desired behaviors to gain a

Modeling Diverse Relevance Patterns in Ad-hoc Retrieval › pdf › 1805.05737.pdfModeling Diverse Relevance Patterns in Ad-hoc Retrieval Yixing Fan†,‡, Jiafeng Guo†,‡, Yanyan