blank language modelspeople.csail.mit.edu/tianxiao/papers/emnlp20_blm_slides.pdf · blm - overview...

Blank Language ModelsTianxiao Shen* Victor Quach* Regina Barzilay Tommi Jaakkola (*: equal contribution)

[email protected]

mailto:[email protected]

Left-to-Right Language Model

They also have ice

They also have ice

cream

…

Generate from scratch

Start with partially specified text

• text editing

• template filling

• text restoration

• …

Blank Language Model (BLM)

Fine-grained control over generation location

Respect preceding and following context

Variable number of missing tokens

Output: They also have ice cream which is really good .Input: They also have which .

BLM - Overview

• Dynamic canvas where “ ” controls where tokens can be placed

• At each step,

1. select a “ ”

2. predict a word w

3. replace that blank with “w”, “ w”, “w ”, or “ w ”

• Stop when there is no “ ”

BLM - Overview

They also have which .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

really

They also have which .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

They also have which really .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

ice

They also have which really .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

They also have ice which really .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

is

They also have ice which really .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

They also have ice which is really .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

cream

They also have ice which is really .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

good

They also have ice cream which is really .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview

They also have ice cream which is really good .


• At each step,

1. select a “ ”

2. predict a word w



BLM - Overview


• At each step,

1. select a “ ”

2. predict a word w


• Stop when there is no “ ”Grammar

• Nonterminal:

• Terminals: w V

• Production rules: ? w ?

(dist. depends on model and context)

∈

→

BLM - Architecture

alsoThey have ____ which ____

Transformer

.

Linear & Softmax1) Choose a blank 2) Predict a word

3) Create new blanks

Linear & Softmax

really

MLP

Fill and repeat

really

really

really____

really

____

________

✓

Trajectory: generating process from an initial “ ” to complete text

BLM - Likelihood

BLM - Likelihood

A sentence with words can be realized by trajectories, each trajectory corresponds to a different word insertion order

x

AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKQL0FBPGYgHlAsoTZSW8yZnZ2mZkVwxLw7sWDIl79JG/+jZPHQaMFDUVVN91dQSK4Nq775eRWVtfWN/Kbha3tnd294v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWjq6nfukeleSxvzThBP6IDyUPOqLFS/aFXLLlldwbyl3gLUoIFar3iZ7cfszRCaZigWnc8NzF+RpXhTOCk0E01JpSN6AA7lkoaofaz2aETcmKVPgljZUsaMlN/TmQ00nocBbYzomaol72p+J/XSU144WdcJqlByeaLwlQQE5Pp16TPFTIjxpZQpri9lbAhVZQZm03BhuAtv/yXNM/KXqV8Wa+UqteP8zjycATHcAoenEMVbqAGDWCA8AQv8OrcOc/Om/M+b805iwgP4Recj28RXI2M

n

AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgKexKQL0FBPGYgHlAsoTZSW8yZnZ2mZkVwhLw7sWDIl79JG/+jZPHQRMLGoqqbrq7gkRwbVz328mtrW9sbuW3Czu7e/sHxcOjpo5TxbDBYhGrdkA1Ci6xYbgR2E4U0igQ2ApGN1O/9YhK81jem3GCfkQHkoecUWOluuwVS27ZnYGsEm9BSrBArVf86vZjlkYoDRNU647nJsbPqDKcCZwUuqnGhLIRHWDHUkkj1H42O3RCzqzSJ2GsbElDZurviYxGWo+jwHZG1Az1sjcV//M6qQmv/IzLJDUo2XxRmApiYjL9mvS5QmbE2BLKFLe3EjakijJjsynYELzll1dJ86LsVcrX9Uqpevs0jyMPJ3AK5+DBJVThDmrQAAYIz/AKb86D8+K8Ox/z1pyziPAY/sD5/AECNI2C

n!

AAAB6XicbVDLSgNBEOyNrxhfUY9eRoPgKexKQL0FBPEYxTwgWcLsZDYZMju7zPQKYQn4AV48KOLVP/Lm3zh5HDSxoKGo6qa7K0ikMOi6305uZXVtfSO/Wdja3tndK+4fNEycasbrLJaxbgXUcCkUr6NAyVuJ5jQKJG8Gw+uJ33zk2ohYPeAo4X5E+0qEglG00r067hZLbtmdgiwTb05KMEetW/zq9GKWRlwhk9SYtucm6GdUo2CSjwud1PCEsiHt87alikbc+Nn00jE5tUqPhLG2pZBM1d8TGY2MGUWB7YwoDsyiNxH/89ophpd+JlSSIldstihMJcGYTN4mPaE5QzmyhDIt7K2EDaimDG04BRuCt/jyMmmcl71K+equUqrePM3iyMMRnMAZeHABVbiFGtSBQQjP8ApvztB5cd6dj1lrzplHeAh/4Hz+AFmKja0=

p(x; ✓) =X

�2Sn

p(x,�; ✓) =X

�2Sn

n�1Y

t=0

p(ax,�t |cx,�t ; ✓)

AAACcHichVFdaxNBFJ1dtda01mhehCqdGoQqGnYl0IoUCiL4mNImDWST5e5kkgydmV1m7oph3Wf/n2/+iL74C5x8QZsWemHgzLnn3Jk5k2RSWAyCv57/4OGjjcebTypb2093nlWfv+jYNDeMt1kqU9NNwHIpNG+jQMm7meGgEskvksuvs/7FD26sSPU5TjPeVzDWYiQYoKPi6u/s4OeXCCcc4R09ppHNVVxEVowVRELTs1iX1Ek+LKj7lFFm0mFc4HFQDgr9MZx5YVCs7GWMv9iNLV1NjKv1oBHMi94G4RLUybJacfVPNExZrrhGJsHaXhhk2C/AoGCSl5UotzwDdglj3nNQg+K2X8wDK+lbxwzpKDVuaaRz9rqjAGXtVCVOqQAndr03I+/q9XIcHfULobMcuWaLg0a5pJjSWfp0KAxnKKcOADPC3ZWyCRhg6P6o4kII1598G3Q+NcJm4/Nps37ybRnHJtklb8gBCckhOSHfSYu0CSNXXs175b32/vkv/T1/fyH1vaWnRm6U//4/vr+8Qw==

order action, canvas at step t

AAADF3icjVJLbxMxEPYurxJeKRy5uESVEkGj3RIeFQqq4MKxCNJWipOV1/FurNrelT2LGi37L7jwV7hwACGucOPf4GwS0VIqMZKl8TffzHwzdpxLYSEIfnn+hYuXLl9Zu9q4dv3GzVvN9dv7NisM4wOWycwcxtRyKTQfgADJD3PDqYolP4iPXs7jB++4sSLTb2GW85GiqRaJYBQcFK17W5tEZinO28fPCEw50A7u4xoitlBRSaxIFSVC4zeRrjDJTTaJSugH1bjUW2HlMum4PH6w4FURvGenrnhVtlE3IpIn0CaJoawMq1JViy6iH1ZjhWkkiBHpFDr4uVNxDqtW56iuYsrrW1tvdPD9Pwl6ozpHfg2eUL8c/j9HgKjZCrpBbTjoPn74aHunh8MVsnJaaGl7UfMnmWSsUFwDk9TaYRjkMCqpAcEkrxqksDyn7IimfOhcTRW3o7J+1wpvOmSCk8y4owHX6MmMkiprZyp2TEVhav+OzcF/xYYFJE9HpdB5AVyzRaOkkBgyPP8keCIMZyBnzqHMCKcVsyl1ywX3lRpuCWdGPuvsb3fDXnfnda+1+2K5jjV0F91DbRSiJ2gXvUJ7aICY98H75H3xvvof/c/+N//7gup7y5w76JT5P34DRPL7Zw==

1 2 3 4 5 6 7 8 9 10

3

1

10

6

2

8

4

7

5

9

BLM - Training

log p(x; ✓) = logX

�2Sn

n�1Y

t=0

p(ax,�t |cx,�t ; ✓)

AAACUHicbVFdSxtBFL0bbavpV2of+zIYChbasFuEtoggFEoflTYqZONydzLZDM7HMnO3GNaF/sG++Nbf4YsPljpJFKr2wjBnzjl3Ps7kpZKe4vh31FpafvDw0cpq+/GTp8+ed16s7XtbOS763CrrDnP0Qkkj+iRJicPSCdS5Egf58eeZfvBDOC+t+U7TUgw1FkaOJUcKVNYpUmULVm6cbKU0EYRv2DabU6mvdFanXhYaU2nYt8w0LC2dHWU1bcfNUW3eJU3oxKP65O3C12R0ym8t2c22Wacb9+J5sfsguQZduK7drHOWjiyvtDDEFXo/SOKShjU6klyJpp1WXpTIj7EQgwANauGH9TyQhr0OzIiNrQvDEJuz/3bUqL2f6jw4NdLE39Vm5P+0QUXjj8NamrIiYfjioHGlGFk2S5eNpBOc1DQA5E6GuzI+QYecwh+0QwjJ3SffB/vve8lm79PeZnfny89FHCvwCtZhAxL4ADvwFXahDxx+wTlcwp/oLLqI/raihfVmhpdwq1rtKyMXtPc=

intractable

log

1

m

mX

i=1

ai

!>=

1

m

mX

i=1

log ai

AAADFHicjVJLbxMxEPYurxJeKRy5uESREqDRbgmPCgVF4sKxCNJWipOV43g3Vm3vyp5FjZb9EVz4K1w4gBBXDtz4NzibRLSUSoxkafzNN55vxjPJpLAQBL88/8LFS5evbFytXbt+4+at+ubtfZvmhvEBS2VqDifUcik0H4AAyQ8zw6maSH4wOXq5iB+848aKVL+FecZHiiZaxIJRcFC06d1vEpkmOGsdPycw40DbuIcriNhcRQWxIlGUCI3fRLrEJDPpNCqgF5TjQm+Hpcuk4+L44ZJXRvCenbri9bO1xaNE8hhaJDaUFWFZqHJZRPTCcqwwjQQxIplBG79wIs5hVeIctdYkCa9uLb3Vxg/+JOit8hz1FXhC/Kr3/+sgqjeCTlAZDjpPHj3e2e3icI2snQZa2V5U/0mmKcsV18AktXYYBhmMCmpAMMnLGsktzyg7ogkfOldTxe2oqD61xE2HTHGcGnc04Ao9mVFQZe1cTRxTUZjZv2ML8F+xYQ7xs1EhdJYD12xZKM4lhhQvNgRPheEM5Nw5lBnhtGI2o2604Pao5oZwpuWzzv5OJ+x2dl93G/3+ahwb6C66h1ooRE9RH71Ce2iAmPfB++R98b76H/3P/jf/+5Lqe6ucO+iU+T9+A7/G+qQ=

� log(n!) + 1n!

X

�2Sn

n�1X

t=0

log p(ax,�t |cx,�t ; ✓)

AAADFHicjVLLihQxFE2Vr7F89ejSTcZmoNtHUyUDKtIyIIjLEe2ZgU53kUqnqsMkqSK5JdOUBf6CG3/FjQtF3Lpw59+YrmpxHg54IXBy7knuuTdJCikshOEvzz93/sLFS2uXgytXr12/0Vm/uWvz0jA+YrnMzX5CLZdC8xEIkHy/MJyqRPK95OD5Mr/3lhsrcv0GFgWfKJppkQpGwVHxund3k8g8w0Xv8CmBOQfax0PcUMSWKq6IFZmiRGj8OtY1JoXJZ3EFw7CeVvpBVLuTdFod3m91dQzv2LEt/nNt0BQikqfQI6mhrIrqStVtFTGM6qnCNBbEiGwOffzMuThD1bhz0oBkvNn09EYf3/ur1xv1Ge4b8oj5Ve//10Hc6YaDsAl8GkQr0EWr2Ik7P8ksZ6XiGpik1o6jsIBJRQ0IJnkdkNLygrIDmvGxg5oqbidV86g13nTMDKe5cUsDbtijJyqqrF2oxCkVhbk9mVuS/8qNS0gfTyqhixK4Zm2htJQYcrz8IXgmDGcgFw5QZoTzitmcutGC+0eBG0J0suXTYPfhINoaPHm11d1+8b4dxxq6je6gHorQI7SNXqIdNELM++B98r54X/2P/mf/m/+9lfreaoS30LHwf/wGqgD7DA==

BLM - Training

log p(x; ✓) = logX

�2Sn

n�1Y

t=0

p(ax,�t |cx,�t ; ✓) intractable

log

1

m

mX

i=1

ai

!>=

1

m

mX

i=1

log ai

� log(n!) + 1n!

X

�2Sn

n�1X

t=0



1. Uniformly sample from

2. Uniformly sample from to

3. Construct canvas

4. Compute estimated loss

cx,�t

AAADbXiclVJbixMxFE6nXtbx1l3xwQuStRRb3JYZWVCRyoIIPq5odxeadkjTzDRskhmSM7JlnCf/oW/+BV/8C6bTqttdF/RA4Mt3bt9JziSTwkIQfKt59UuXr1zduOZfv3Hz1u3G5taBTXPD+IClMjVHE2q5FJoPQIDkR5nhVE0kP5wcv1n4Dz9xY0WqP8I84yNFEy1iwSg4KtqsfWkRmSY4a5+8IjDjQDu4jyuK2FxFBbEiUZQIjT9EusQkM+k0KqAflONCd8PSZdJxcbKzjCsj+MzWrvhXWb9qRCSPoU1iQ1kRloUql11EPyzHCtNIECOSGXTwa6figqhKnQt1FRNe3dp6u4Of/knQ2+UF8ivylPrV8P88QsX5636/1f0toos1YdMU/qtu1GgGvaAyfB6EK9BEK9uPGl/JNGW54hqYpNYOwyCDUUENCCZ56ZPc8oyyY5rwoYOaKm5HRbUtJW45Zorj1LijAVfs6YyCKmvnauIiFYWZPetbkH/zDXOIX4wKobMcuGbLRnEuMaR4sXp4KgxnIOcOUGaE04rZjLovA7egvnuE8OzI58HBs16423v5fre593b1HBvoAXqM2ihEz9Eeeof20QCx2nev4d3z7ns/6nfrD+uPlqFebZVzB61Z/clP4VEWPg==

� log(n!)� n · log p(ax,�t |cx,�t ; ✓)

AAADbXiclVJtixMxEE63vpzrW+/ED74gOUuxxWvZlQMVqRyI4McT7d1B0y5pmt2GS7JLMitX1v3kP/Sbf8Ev/gXTbdXrnQc6EJg888zMM8lMMiksBMG3mle/dPnK1Y1r/vUbN2/dbmxuHdg0N4wPWCpTczShlkuh+QAESH6UGU7VRPLDyfGbRfzwEzdWpPojzDM+UjTRIhaMgoOizdqXFpFpgrP2ySsCMw60g/u4gojNVVQQKxJFidD4Q6RLTDKTTqMC+kE5LnQ3LF0mHRcnO0teGcFntnbFv8r6VSMieQxtEhvKirAsVLnsIvphOVaYRoIYkcygg187FRewKnWO6iomvLq19XYHP/2ToLfLC+RX4Cn1q+H/eYQK81vrBL/7W0QXa8KmKfxX3ajRDHpBZfi8E66cJlrZftT4SqYpyxXXwCS1dhgGGYwKakAwyUuf5JZnlB3ThA+dq6nidlRU21LilkOmOE6NOxpwhZ7OKKiydq4mjqkozOzZ2AL8W2yYQ/xiVAid5cA1WzaKc4khxYvVw1NhOAM5dw5lRjitmM2o+zJwC+q7RwjPjnzeOXjWC3d7L9/vNvferp5jAz1Aj1Ebheg52kPv0D4aIFb77jW8e95970f9bv1h/dGS6tVWOXfQmtWf/ATd1RY+

t

AAADcHiclVJtixMxEE63vpzrW8/7IqiYWgqt2rIrBypSORDBjyfau4OmXdI0uw2XZJdkVq6s+9Uf6Dd/hF/8Babbqtc7D3QgMHnmmZlnkplmUlgIgm81r37p8pWrW9f86zdu3rrd2L5zYNPcMD5kqUzN0ZRaLoXmQxAg+VFmOFVTyQ+nx2+W8cNP3FiR6o+wyPhY0USLWDAKDoq2a1/aRKYJzjonrwjMOdAuHuAKIjZXUUGsSBQlQuMPkS4xyUw6iwoYBOWk0L2wdJl0Upw8XfHKCD6zjSv+VdavGhHJY+iQ2FBWhGWhylUXMQjLicI0EsSIZA5d/NqpuIBVqXNUVzHh1a2jm1385E+CbpYXyK/AU+rXw//zCBXmg9/epPjt3m8dPawJm6XwX6WjRivoB5Xh8064dlpobftR4yuZpSxXXAOT1NpRGGQwLqgBwSQvfZJbnlF2TBM+cq6mittxUS1MidsOmeE4Ne5owBV6OqOgytqFmjqmojC3Z2NL8G+xUQ7xi3EhdJYD12zVKM4lhhQvtw/PhOEM5MI5lBnhtGI2p+7XwO2o7x4hPDvyeefgWT/c7b98v9vae7t+ji10Dz1CHRSi52gPvUP7aIhY7bu34933Hng/6nfrD+vNFdWrrXN20IbVH/8EzZYW/w==

0

AAADcHiclVJtixMxEE63vpzrW8/7IqiYWgqt2rIrBypSORDBjyfau4OmXdI0uw2XZJdkVq6s+9Uf6Dd/hF/8Babbqtc7D3QgMHnmmZlnkplmUlgIgm81r37p8pWrW9f86zdu3rrd2L5zYNPcMD5kqUzN0ZRaLoXmQxAg+VFmOFVTyQ+nx2+W8cNP3FiR6o+wyPhY0USLWDAKDoq2a1/aRKYJzjonrwjMOdAuHuAKIjZXUUGsSBQlQuMPkS4xyUw6iwoYBOWk0L2wdJl0Upw8XfHKCD6zjSv+VdavGhHJY+iQ2FBWhGWhylUXMQjLicI0EsSIZA5d/NqpuIBVqXNUVzHh1a2jm1385E+CbpYXyK/AU+rXw//zCBXmB357k+K3e7919LAmbJbCf5WOGq2gH1SGzzvh2mmhte1Hja9klrJccQ1MUmtHYZDBuKAGBJO89ElueUbZMU34yLmaKm7HRbUwJW47ZIbj1LijAVfo6YyCKmsXauqYisLcno0twb/FRjnEL8aF0FkOXLNVoziXGFK83D48E4YzkAvnUGaE04rZnLpfA7ejvnuE8OzI552DZ/1wt//y/W5r7+36ObbQPfQIdVCInqM99A7toyFite/ejnffe+D9qN+tP6w3V1Svts7ZQRtWf/wTUQIWuw==

n� 1

AAADcniclVJti9NAEN6mvpzxraf4RUG3lkLLXUsiBypSORDBjyfau4NuGzbbTbpcdhN2J3Il5ru/z2/+Cr/4A9ymUa93HuhAYPaZeWaemUyYJcKA531rOM0rV69d37rh3rx1+87d1va9Q5PmmvExS5NUH4fU8EQoPgYBCT/ONKcyTPhRePJmFT/6xLURqfoIy4xPJY2ViASjYKFgu/GlS5I0xlnv9BWBBQfaxyNcQcTkMiiIEbGkRCj8IVAlJplO50EBI6+cFWrgl5ZJZ8Xp7jqvDOAz23jiX2XdqhFJeAQ9EmnKCr8sZLnuIkZ+OZOYBoJoES+gj19bFZdkVepsqq0Y8+rVU+0+3vlDUO3yEvkVeEZ9Pfw/j1BhrqW63c0ktzv4rWSAFWHzFP6reNDqeEOvMnzR8Wung2o7CFpfyTxlueQKWEKNmfheBtOCahAs4aVLcsMzyk5ozCfWVVRyMy2qkylx1yJzHKXafgpwhZ5lFFQas5ShzZQUFuZ8bAX+LTbJIXoxLYTKcuCKrRtFeYIhxav7w3OhOYNkaR3KtLBaMVtQ+9/AXqlrl+CfH/mic/hs6O8NX77f6+y/rdexhR6hp6iHfPQc7aN36ACNEWt8dx44j50nzo/mw2a7We/OadSc+2jDmrs/AZvdF2s=

�

AAADcniclVJti9NAEN6mvpzxraf4RUG3lkLLXUsiBypSORDBjyfau4NuGzbbTbpcdhN2J3Il5ru/z2/+Cr/4A9ymUa93HuhAYPaZeWaemUyYJcKA531rOM0rV69d37rh3rx1+87d1va9Q5PmmvExS5NUH4fU8EQoPgYBCT/ONKcyTPhRePJmFT/6xLURqfoIy4xPJY2ViASjYKFgu/GlS5I0xlnv9BWBBQfaxyNcQcTkMiiIEbGkRCj8IVAlJplO50EBI6+cFWrgl5ZJZ8Xp7jqvDOAz23jiX2XdqhFJeAQ9EmnKCr8sZLnuIkZ+OZOYBoJoES+gj19bFZdkVepsqq0Y8+rVU+0+3vlDUO3yEvkVeEZ9Pfw/jrCG3K6lut3NJLc7+K1kgBVh8xT+q3jQ6nhDrzJ80fFrp4NqOwhaX8k8ZbnkClhCjZn4XgbTgmoQLOGlS3LDM8pOaMwn1lVUcjMtqpMpcdcicxyl2n4KcIWeZRRUGrOUoc2UFBbmfGwF/i02ySF6MS2EynLgiq0bRXmCIcWr+8NzoTmDZGkdyrSwWjFbUPvfwF6pa5fgnx/5onP4bOjvDV++3+vsv63XsYUeoaeoh3z0HO2jd+gAjRFrfHceOI+dJ86P5sNmu1nvzmnUnPtow5q7PwGdURdr

Sn

AAADd3iclVJNbxMxEHU2fJTlK4UbHHAJQYkg0S6qBAgFVUJIHIsgbaU4WXkd78aq7V3Zs6jRsj+BP8eN/8GFG84mQNNSCUayNH7zZuaNPXEuhYUg+NbwmpcuX7m6dc2/fuPmrdut7TsHNisM4yOWycwcxdRyKTQfgQDJj3LDqYolP4yP3yzjh5+4sSLTH2GR84miqRaJYBQcFG03vnSIzFKcd09eEZhzoD08xDVEbKGikliRKkqExh8iXWGSm2wWlTAMqmmp+2HlMum0PHm64lURfGYbV/yrrF83IpIn0CWJoawMq1JVqy5iGFZThWkkiBHpHHr4tVNxAatW56iuYsrrW1fv9PCTPwl6p7pAfg2eUr8e/p9HqDHf1fI7Lt/vbDL9Tv+3nD7WhM0y+K8OUasdDILa8HknXDtttLb9qPWVzDJWKK6BSWrtOAxymJTUgGCSVz4pLM8pO6YpHztXU8XtpKz3psIdh8xwkhl3NOAaPZ1RUmXtQsWOqSjM7dnYEvxbbFxA8mJSCp0XwDVbNUoKiSHDyyXEM2E4A7lwDmVGOK2Yzan7PHCr6rtHCM+OfN45eDYIdwcv3++2996un2ML3UcPUReF6DnaQ+/QPhoh1vju3fPa3iPvR/NB83Gzu6J6jXXOXbRhzfAn9wYY7A==

one action loss per pass :(

BLM - Training

� log(n!) + 1n!

X

�2Sn

n�1X

t=0



depends only on the first elements of

—> combine into one pass loss calculations of trajectories that are the samecx,�t

AAAFp3icrVTbbtNAEHXbGIq5pfDIy5YQKVFJZFeVgKKgSgiEeKGoTRopTqz1ZuOsaq8t7xg1MivxY/wEb/wN60tp0rSiAVayNHvmcs7MWOtGPhNgmj/X1jcq+q3bm3eMu/fuP3hY3XrUE2ESE9oloR/GfRcL6jNOu8DAp/0opjhwfXrinr7N/CdfaCxYyI9hFtFhgD3OJoxgUJCztfG9bvuhh6LG2WsbphRwE3VQDtkiCZzUFswLsM04OnK4RHYUh2MnhY4pRylvWVJl4lF69ryIkw58JQtXdF7WyIlsn06gYU9iTFJLpoEsWFjHkqMAYYfZMfOm0ERvlIpronJ1KlRV9Gh+a/DtJtq5SODb8hr5OTinvmz+xi3kmFFXxYy6KmAsRhr11m85LcRtMg5hRYbOfD9Lav/c4N+znesNMExdF71z5sySxkmtfZCKZdkBO5aUV+K7+zxLyRY/WEle8SsMlcr5Jf+7yv+n5UYqcrJydVytrgWLu7tQtboep1oz22Z+0LJhlUZNK8+hU/1hj0OSBJQD8bEQA8uMYJjiGBjxqTTsRNAIk1Ps0YEyOQ6oGKb5OyNRXSFjNAlj9XFAOTqfkeJAiFngqshsGuKyLwOv8g0SmLwcpoxHCVBOCqJJ4iMIUfZooTGLKQF/pgxMYqa0IjLFaqCgnjZDDcG63PKy0dttW3vtV5/3agfvvxXj2NSeaE+1hmZpL7QD7YN2qHU1UnlW+Vg5qhzrTf2T3tP7Rej6WjnCx9rC0fEvyQXkHg==

t

AAAFqnicrVRbb9MwFM62Bka4dfDIi0dV0Wpr1UyTgKGiSQjEC9K4tB2q28hx3NRa4kTxCVoVLPHT+A288W9wk4616yZWwFKk4+9cvu+cE9mNAy6h1fq5tr5RMm/c3Lxl3b5z99798taDrozShLIOjYIoOXaJZAEXrAMcAnYcJ4yEbsB67smrqb/3hSWSR+ITTGI2CIkv+IhTAhpytja+V3EQ+Siunb7AMGZA6qiNcgjLNHQyLLkfEswF+ugIhXCcRJ6TQbulhplo2EpnkmF2ulvEKQe+0oUrOitr5UQ4YCOo4VFCaGarLFQFC2/bahgi4nCccH8MdfRSq7giKlenQ3VFn+W3mtiuo53zBLGtrpCfg3PqZ81fu4Ucs6q6mFXVBSywqovBVrXxW1EDCUy9CFYkac+3tCT4zz3+PduZ3pDA2HXRa2fOnNE4mX0ASrMsO2DHVupSfO9ATFOmu++vJK/4GwZa5fye/13l/9NyLRU52Wx1Qq+uAYu7O1e1uh6nXGk1W/lBy4Y9MyrG7Bw55R/Yi2gaMgE0IFL27VYMg4wkwGnAlIVTyWJCT4jP+toUJGRykOVPjUJVjXhoFCX6E4BydD4jI6GUk9DVkdNpyIu+KXiZr5/C6Nkg4yJOgQlaEI3SAEGEpu8W8njCKAQTbRCacK0V0THRAwX9ull6CPbFlpeN7l7T3m8+f79fOXzzrRjHpvHIeGzUDNt4ahwab40jo2PQ0pPSu1K31DN3zQ/mZ7NfhK6vzUb40Fg4pvcLKDjk3w==

�

AAAFqnicrVRtb9MwEM62FkZ46+AjXzyqiFZbq2SaBAwVTUIgviCNl7ZDdRs5rttaS5wovqBVwRI/jd/AN/4NTtKxdt3ECliKdH7uzs9zd9F5kc8l2PbPtfWNUvnGzc1b5u07d+/dr2w96MgwiSlr09AP42OPSOZzwdrAwWfHUcxI4Pms6528yvzdLyyWPBSfYBqxfkDGgo84JaAhd2vju4X9cIyi2ukLDBMGpI5aKIewTAI3xZKPA4K5QB9doRCO4nDoptCy1SAVDUfpTDJIT3eLOOXCV7pwRWfPmjkR9tkIangUE5o6Kg1UwcJbjhoEiLgcx3w8gTp6qVVcEZWr06H6xTHLbzWxXUc75wliW10hPwfn1M+Kv2YJBWRa+i3T0vmmBaa1GGxajd+KGkhgOgxhNRKrNV/SkuA/1/j3bGd6AwITz0Ov3TlzRuOmzgEozbLsgB1HqUvxvQORpWSz760kr/gb+lrl/Jz/XeX/03ItFTnZbHRCj64Bi7M7V7W6HrdStZt2ftCy4cyMqjE7R27lBx6GNAmYAOoTKXuOHUE/JTFw6jNl4kSyiNATMmY9bQoSMNlP81WjkKWRIRqFsf4EoBydz0hJIOU08HRk1g150ZeBl/l6CYye9VMuogSYoAXRKPERhCjbW2jIY0bBn2qD0JhrrYhOiG4o6O1m6iY4F0teNjp7TWe/+fz9fvXwzbeiHZvGI+OxUTMc46lxaLw1joy2QUtPSu9KnVK3vFv+UP5c7hWh62uzFj40Fk55+AsqhOTf

t


t+ 1

AAAFrHicrVRdb9MwFPW2Bkb46uCRF4+qUqvRKpkmAUNFkxCIN4ZYt6KmjRzXSa0lThTfoFXBEr+Nn8Ab/wYn7Vi7bmIFLEW6PvfjnHtvZC8JuQTL+rm2vlExbt3evGPevXf/wcPq1qNjGWcpZV0ah3Ha84hkIResCxxC1ktSRiIvZCfe6ZvCf/KFpZLH4ggmCRtEJBDc55SAhtytje91J4wDnDTOXjkwZkCauINLyJFZ5OaO5EFEHC7wJ1co7CRpPHJz6FhqmIuWrXQmGeZnz6ZxyoWvdOGKz8uaJZETMh8ajp8Smtsqj9SUhXdsNYwwcbmT8mAMTfxaq7gmqlSnQ3XFgJW3hthu4p2LBLGtrpFfgnPqZ83fuIUSM+u6mFnXBUzYsc36YrhZb/3W1MLCoaMYVqTpzDe1JPnPXf4927neiMDY8/Bbd86c0bi5vQ9Ksyw79DSUuhLf3RdFSrH9/krypv/DQKuc3/S/q/x/Wm6koiSbrU7o1bVgcXcXqlbX41ZrVtsqD1427JlRQ7Nz6FZ/OKOYZhETQEMiZd+2EhjkJAVOQ6ZMJ5MsIfSUBKyvTUEiJgd5+dgoXNfICPtxqj8BuETnM3ISSTmJPB1ZTENe9hXgVb5+Bv6LQc5FkgETdErkZyGGGBcvFx7xlFEIJ9ogNOVaK6ZjogcK+n0z9RDsyy0vG8e7bXuv/fLjXu3g3bfpODbRE/QUNZCNnqMD9B4doi6ilWblQ6VX+Wy0jSOjbwymoetrsxE+RgvH8H8B0J3lTw==

in the first steps but different at the step

BLM - Training depends only on the first elements of

—> combine into one pass loss calculations of trajectories that are the samecx,�t

AAAFp3icrVTbbtNAEHXbGIq5pfDIy5YQKVFJZFeVgKKgSgiEeKGoTRopTqz1ZuOsaq8t7xg1MivxY/wEb/wN60tp0rSiAVayNHvmcs7MWOtGPhNgmj/X1jcq+q3bm3eMu/fuP3hY3XrUE2ESE9oloR/GfRcL6jNOu8DAp/0opjhwfXrinr7N/CdfaCxYyI9hFtFhgD3OJoxgUJCztfG9bvuhh6LG2WsbphRwE3VQDtkiCZzUFswLsM04OnK4RHYUh2MnhY4pRylvWVJl4lF69ryIkw58JQtXdF7WyIlsn06gYU9iTFJLpoEsWFjHkqMAYYfZMfOm0ERvlIpronJ1KlRV9Gh+a/DtJtq5SODb8hr5OTinvmz+xi3kmFFXxYy6KmAsRhr11m85LcRtMg5hRYbOfD9Lav/c4N+znesNMExdF71z5sySxkmtfZCKZdkBO5aUV+K7+zxLyRY/WEle8SsMlcr5Jf+7yv+n5UYqcrJydVytrgWLu7tQtboep1oz22Z+0LJhlUZNK8+hU/1hj0OSBJQD8bEQA8uMYJjiGBjxqTTsRNAIk1Ps0YEyOQ6oGKb5OyNRXSFjNAlj9XFAOTqfkeJAiFngqshsGuKyLwOv8g0SmLwcpoxHCVBOCqJJ4iMIUfZooTGLKQF/pgxMYqa0IjLFaqCgnjZDDcG63PKy0dttW3vtV5/3agfvvxXj2NSeaE+1hmZpL7QD7YN2qHU1UnlW+Vg5qhzrTf2T3tP7Rej6WjnCx9rC0fEvyQXkHg==

t


�

AAAFqnicrVRtb9MwEM62FkZ46+AjXzyqiFZbq2SaBAwVTUIgviCNl7ZDdRs5rttaS5wovqBVwRI/jd/AN/4NTtKxdt3ECliKdH7uzs9zd9F5kc8l2PbPtfWNUvnGzc1b5u07d+/dr2w96MgwiSlr09AP42OPSOZzwdrAwWfHUcxI4Pms6528yvzdLyyWPBSfYBqxfkDGgo84JaAhd2vju4X9cIyi2ukLDBMGpI5aKIewTAI3xZKPA4K5QB9doRCO4nDoptCy1SAVDUfpTDJIT3eLOOXCV7pwRWfPmjkR9tkIangUE5o6Kg1UwcJbjhoEiLgcx3w8gTp6qVVcEZWr06H6xTHLbzWxXUc75wliW10hPwfn1M+Kv2YJBWRa+i3T0vmmBaa1GGxajd+KGkhgOgxhNRKrNV/SkuA/1/j3bGd6AwITz0Ov3TlzRuOmzgEozbLsgB1HqUvxvQORpWSz760kr/gb+lrl/Jz/XeX/03ItFTnZbHRCj64Bi7M7V7W6HrdStZt2ftCy4cyMqjE7R27lBx6GNAmYAOoTKXuOHUE/JTFw6jNl4kSyiNATMmY9bQoSMNlP81WjkKWRIRqFsf4EoBydz0hJIOU08HRk1g150ZeBl/l6CYye9VMuogSYoAXRKPERhCjbW2jIY0bBn2qD0JhrrYhOiG4o6O1m6iY4F0teNjp7TWe/+fz9fvXwzbeiHZvGI+OxUTMc46lxaLw1joy2QUtPSu9KnVK3vFv+UP5c7hWh62uzFj40Fk55+AsqhOTf

t


t+ 1

AAAFrHicrVRdb9MwFPW2Bkb46uCRF4+qUqvRKpkmAUNFkxCIN4ZYt6KmjRzXSa0lThTfoFXBEr+Nn8Ab/wYn7Vi7bmIFLEW6PvfjnHtvZC8JuQTL+rm2vlExbt3evGPevXf/wcPq1qNjGWcpZV0ah3Ha84hkIResCxxC1ktSRiIvZCfe6ZvCf/KFpZLH4ggmCRtEJBDc55SAhtytje91J4wDnDTOXjkwZkCauINLyJFZ5OaO5EFEHC7wJ1co7CRpPHJz6FhqmIuWrXQmGeZnz6ZxyoWvdOGKz8uaJZETMh8ajp8Smtsqj9SUhXdsNYwwcbmT8mAMTfxaq7gmqlSnQ3XFgJW3hthu4p2LBLGtrpFfgnPqZ83fuIUSM+u6mFnXBUzYsc36YrhZb/3W1MLCoaMYVqTpzDe1JPnPXf4927neiMDY8/Bbd86c0bi5vQ9Ksyw79DSUuhLf3RdFSrH9/krypv/DQKuc3/S/q/x/Wm6koiSbrU7o1bVgcXcXqlbX41ZrVtsqD1427JlRQ7Nz6FZ/OKOYZhETQEMiZd+2EhjkJAVOQ6ZMJ5MsIfSUBKyvTUEiJgd5+dgoXNfICPtxqj8BuETnM3ISSTmJPB1ZTENe9hXgVb5+Bv6LQc5FkgETdErkZyGGGBcvFx7xlFEIJ9ogNOVaK6ZjogcK+n0z9RDsyy0vG8e7bXuv/fLjXu3g3bfpODbRE/QUNZCNnqMD9B4doi6ilWblQ6VX+Wy0jSOjbwymoetrsxE+RgvH8H8B0J3lTw==

in the first steps but different at the step

� log(n!) +n�1X

t=0

1

n!

X

�2Sn


AAAFrnicrVRdb9MwFPW2Fkb46uCRF48qUqvSKpkmAUNFkxCIx03QbqJpI8d1U2uJE8U3aFWwxI/jF/DGv8FNMtaum1gFlhLdnPtxzr03shcHXIJl/drY3KpU79zdvmfcf/Dw0ePazpO+jNKEsh6Ngig59YhkAResBxwCdhonjIRewE68s3dz/8lXlkgeic8wi9kwJL7gE04JaMjd2fphOkHk47hx/saBKQPSxF2cQ45MQzdzJPdD4nCBP7lCYSdOorGbQddSo0y0baUzySg7f1HEKRe+0aVPfFHWyImcgE2g4UwSQjNbZaEqWHjXVqMQE5c7Cfen0MRvtYobonJ1OlRX9Fn+1RC7Tdy6TBC76gb5Obigvmz+1i3kmGHqYoapCxgmtPRrOd4w239EtbFw6DiC9Xh0X4ttrYj+e5/rtdVdYLsQHBKYeh5+7y6YJY2b2QegNMuqQ49DqWvxvQMxT5nvf7CWvOKPGGqVi7v+d5X/T8utVORk5eqEXl0blnd3qWp9PW6tbnWs/OBVwy6NOirPkVv76YwjmoZMAA2IlAPbimGYkQQ4DZgynFSymNAz4rOBNgUJmRxm+XWjsKmRMZ5EiX4E4BxdzMhIKOUs9HTkfBryqm8OXucbpDB5Ncy4iFNgghZEkzTAEOH53YXHPGEUgpk2CE241orplOiBgr7hDD0E+2rLq0Z/r2Pvd14f79cPP3wvxrGNnqHnqIFs9BIdoo/oCPUQrbQqx5UvlUHVqvarw6pbhG5ulCN8ipZOdfoblObmTg==

= log(n!) + n · EtE�1:tE�t+1E�t+2:n [log p(ax,�t |c

x,�t ; ✓)]

AAAFpXicrVRdb9MwFM22Bkb46uCRF29VodPWKpkmAUNFkxBoL0gbrN2kuo0c102tJU4U36BVwYj/xb/gjX+Dm2asXTexCixFuj7345x7b2QvDrgE2/61tLxSMu/cXb1n3X/w8NHj8tqTtozShLIWjYIoOfWIZAEXrAUcAnYaJ4yEXsBOvLN3Y//JF5ZIHoljGMWsGxJf8AGnBDTkrq38qOIg8lFcO3+DYciAbKImyiEs09DNsOR+SDAX6LMrFMJxEvXdDJq26mWi7iidSXrZ+fYkTrnwlc5c0UVZKyfCARtADQ8SQjNHZaGasPCmo3ohIi7HCfeHsIneahU3ROXqdKiu6LP8VhPrm2jrMkGsqxvk5+CU+qL5W7eQY1ZVF7OquoBVnQ21qvU/eupIYNqPYEGK5nRDc3L/3uFCbNNkF3JDAkPPQ+/dKbNgcTNnD5QmmXfAlqPUtfjOnhinjBffWUjd5Ffo6pF8Q8//h7x/EIGwiEQaeixBGBeKbiUlZyz2JvTe6jC7uEtpi4tyyxW7YecHzRtOYVSM4hy65Z+4H9E0ZAJoQKTsOHYM3YwkwGnAlIVTyWJCz4jPOtoUJGSym+WvjEJVjfTRIEr0JwDl6HRGRkIpR6GnI8fTkFd9Y/A6XyeFwatuxkWcAhN0QjRIAwQRGj9ZqM8TRiEYaYPQhGutiA6JHijoh83SQ3CutjxvtHcazm7j9dFuZf/D98k4Vo1nxoZRMxzjpbFvHBiHRsugpY3SQemo9Ml8YX40j832JHR5qRjhU2PmmO5vexTkmg==

= log(n!) + EtE�1:t

2

4 nn� t

X

�t+1


3

5

AAAFp3icpVTbbtNAEHXbGIq5pfDIy5ZgKVFJFFeVgKKgSgiEeKGoTRopTqz1Zp2saq8t7xg1MivxY/wEb/wN60tp0rSioStZmj1zOWdmrHUjnwlot3+vrW9U9Dt3N+8Z9x88fPS4uvWkJ8IkJrRLQj+M+y4W1GecdoGBT/tRTHHg+vTEPX2f+U++0ViwkB/DLKLDAE848xjBoCBna+OnafvhBEX1s7c2TCngBuqgHLJFEjipLdgkwDbj6MjhEtlRHI6dFDptOUp505IqE4/Ss5dFnHTgO1m4ovOyRk5k+9SDuu3FmKSWTANZsLCOJUcBwg6zYzaZQgO9UyquicrVqVBVcULzW51vN9DORQLfltfIz8E59WXzN24hxwxTFTNMVcAwF0MNs/lXTxNxm4xDWJGiM9/Qktx/d/j/bOd6AwxT10UfnDmzpHFSax+kYll2wI4l5ZX47j7PUrLND1aSV/wLQ6Vyfsu3V3kbLQtSbiQi5yo3x9XmmrC4ugtRq8txqrV2q50ftGxYpVHTynPoVH/Z45AkAeVAfCzEwGpHMExxDIz4VBp2ImiEySme0IEyOQ6oGKb5OyORqZAx8sJYfRxQjs5npDgQYha4KjKbhrjsy8CrfIMEvNfDlPEoAcpJQeQlPoIQZY8WGrOYEvBnysAkZkorIlOsBgrqaTPUEKzLLS8bvd2Wtdd683WvdvDxRzGOTe2Z9lyra5b2SjvQPmmHWlcjlReVz5WjyrHe0L/oPb1fhK6vlSN8qi0cHf8BXfLkHg==

BLM - Training

= log(n!) + EtE�1:t

2

4 nn� t

X

�t+1


3

5

AAAFp3icpVTbbtNAEHXbGIq5pfDIy5ZgKVFJFFeVgKKgSgiEeKGoTRopTqz1Zp2saq8t7xg1MivxY/wEb/wN60tp0rSioStZmj1zOWdmrHUjnwlot3+vrW9U9Dt3N+8Z9x88fPS4uvWkJ8IkJrRLQj+M+y4W1GecdoGBT/tRTHHg+vTEPX2f+U++0ViwkB/DLKLDAE848xjBoCBna+OnafvhBEX1s7c2TCngBuqgHLJFEjipLdgkwDbj6MjhEtlRHI6dFDptOUp505IqE4/Ss5dFnHTgO1m4ovOyRk5k+9SDuu3FmKSWTANZsLCOJUcBwg6zYzaZQgO9UyquicrVqVBVcULzW51vN9DORQLfltfIz8E59WXzN24hxwxTFTNMVcAwF0MNs/lXTxNxm4xDWJGiM9/Qktx/d/j/bOd6AwxT10UfnDmzpHFSax+kYll2wI4l5ZX47j7PUrLND1aSV/wLQ6Vyfsu3V3kbLQtSbiQi5yo3x9XmmrC4ugtRq8txqrV2q50ftGxYpVHTynPoVH/Z45AkAeVAfCzEwGpHMExxDIz4VBp2ImiEySme0IEyOQ6oGKb5OyORqZAx8sJYfRxQjs5npDgQYha4KjKbhrjsy8CrfIMEvNfDlPEoAcpJQeQlPoIQZY8WGrOYEvBnysAkZkorIlOsBgrqaTPUEKzLLS8bvd2Wtdd683WvdvDxRzGOTe2Z9lyra5b2SjvQPmmHWlcjlReVz5WjyrHe0L/oPb1fhK6vlSN8qi0cHf8BXfLkHg==


2. Uniformly sample

3. Construct canvas


cx,�t


t


0


n� 1


�1:t

AAAFtHicrVRbb9MwFM62Bka4dfDIi0dVqdVolUwTl6GiSQjE4xB0m1S3wXGd1lriRPEJWhUs8ft45I1/g5N0LF03sQosJTr+zuX7zrFlLw64BNv+tba+UTNv3d68Y929d//Bw/rWoyMZpQllfRoFUXLiEckCLlgfOATsJE4YCb2AHXunb3P/8VeWSB6JzzCL2TAkE8F9TgloyN3a+NHEQTRBcevsNYYpA9JGPVRAWKahm2HJJyHBXKBPrlAIx0k0djPo2WqUiY6jdCYZZWfPyjjlwje6sEXnZa2CCAfMhxb2E0IzR2WhKll4z1GjEBGX44RPptBGb7SKa6IKdTpUV5ywYtcS2220c5EgttU18guwon7e/A1bKCE3c/ZBWU1d0WrqKlYTdvRvMclqdv4o6yCB6TiC1cjy7qrNLUn/e7er8fUqbOeKQwJTz0Pv3IqZVcegWZYdeh5KXYnv7os8Jb8Fg5XklfdiqFVWT/zfVf4/LTdSUZDNj07oo+vA4tldqFpdj1tv2F27WGjZcOZGw5ivQ7f+E48jmoZMAA2IlAPHjmGYkQQ4DZiycCpZTOgpmbCBNgUJmRxmxaOjUFMjY+RHif4EoAKtZmQklHIWejoyn4a87MvBq3yDFPyXw4yLOAUmaEnkpwGCCOUvGBrzhFEIZtogNOFaK6JTogcK+p2z9BCcyy0vG0e7XWev++rjXuPg/fdyHJvGE+Op0TIc44VxYHwwDo2+QWtO7bj2pUbM5yY2qcnK0PW1+QgfGwvLFL8BSJ3owA==

� log(n!)� nn� t

X

�t+1


AAAGEXicrVRNb9NAEHVLDMV8pXDksiWySFQSxVUloCioEgJxLIK0leLEWm82zqr22vKOUSOzEr+AC3+FCwcQ4sqNG/+GtZ1Sp2lFQ1nJ1uzbmXlvZlbrRj4T0G7/Wlq+VNEvX1m5aly7fuPmrerq7V0RJjGhXRL6YbzvYkF9xmkXGPh0P4opDlyf7rkHz7Lzvbc0Fizkb2AS0X6APc5GjGBQkLNauW/afuihqH74xIYxBdxAHZRDtkgCJ7UF8wJsM45eO1wiO4rDoZNCpy0HKW9aUkXiQXr4oPCTDrwjM1t0lNbIiWyfjqBuj2JMUkumgSxYWMeSgwBhh9kx88bQQE+VijO8cnXKVWX0aL6r87UGWj8O4GvyDPk5WFI/Lf7cJeSYk1pbIA1TpTRMlcYwYV39ZqMMs/lHWhNxmwxDWJTNo+Xq5rT/vdzF+DoltiPFAYax66LnTslMy21QLPMHqh9SnopvbPEsJLsGvYXkFRejr1SWR35xlf9Py7lU5GTT0XE1uibMzu5Y1T/oaaLSlZvhuDCFU621W+18oXnDmho1bbp2nOpPexiSJKAciI+F6FntCPopjoERn0rDTgSNMDnAHu0pk+OAin6av2gSmQoZolEYq48DytFyRIoDISaBqzyzTouTZxl42lkvgdGjfsp4lADlpCAaJT6CEGXPIxqymBLwJ8rAJGZKKyJjrBoJ6hE1VBOskyXPG7sbLWuz9fjVZm37xfuiHSvaXe2eVtcs7aG2rb3UdrSuRiofKp8qXypf9Y/6Z/2b/r1wXV6atvCONrP0H78Bf9EM+g==

n/2 action losses per pass :)

BLM - Training


1 2 3 4 5 6 7 8 9 10x =

AAAGO3icrVTLbtNAFHVLAsU8msKSzZQoUqImkR3Kq8ioAiGxbAVpK8WpNZ5MnFHtseW5RonM/BcbfoIdGzYsQIgteyZOSp2mFQ1lJFvX93XOfXjcyGcCDOPz0vKVQvHqtZXr+o2bt26vltbu7IkwiQltk9AP4wMXC+ozTtvAwKcHUUxx4Pp03z16Obbvv6OxYCF/C6OIdgPscdZnBINSOWuF3Yrthx6KqsNnNgwo4BqyUKayRRI4qS2YF2CbcfTG4RLZURz2nBQsQx6mvGFKFYkP02F94icdeE9mPtFxWj0Dsn3ah6rdjzFJTZkGcoLCLFMeBgg7zI6ZN4Aaeq5YnOOVsVOuKqNHs68qX6+hjZMAvi7PoZ8pc+ynxV+4hEznpOYWSL2iUuoVlUavwIZ6zUbplcYfag3EbdILYVE0j+arm+P+93IXw7NyaMeMAwwD10WvnJyY5tugUOYNqh9SnqlvbfFxyHgNOgvRmyxGV7HMj/zyLP8flwuxyMCmo+NqdA2Ynd0Jq3/h00C5nZsBuTSG2nTLNNSqWw9n/wOr+qjeqpv1B3XTqOlDyymVjaaRHTQvmFOhrE3PjlP6ZPdCkgSUA/GxEB3TiKCb4hgY8anU7UTQCJMj7NGOEjkOqOim2d0nUUVpeqgfxurhgDJtPiLFgRCjwFWe45mI07ax8ixbJ4H+k27KeJQA5WQC1E98BCEaX6Sox2JKwB8pAZOYKa6IDLDqOKjrVldNME+XPC/stZrmZvPp7mZ5+8W0HSvaPe2+VtVM7bG2rb3WdrS2RgofCl8K3wrfix+LX4s/ij8nrstL05i72swp/voNaU0Yiw==

AAAB63icbVBNSwMxEJ2tX7V+VT16CRbBU8mKoheh6MVjBWsL7VKyabYNTbJLkhXK0r/gxYMiXv1D3vw3Zts9aOuDgcd7M8zMCxPBjcX42yutrK6tb5Q3K1vbO7t71f2DRxOnmrIWjUWsOyExTHDFWpZbwTqJZkSGgrXD8W3ut5+YNjxWD3aSsECSoeIRp8Tmkrr2cb9aw3U8A1omfkFqUKDZr371BjFNJVOWCmJM18eJDTKiLaeCTSu91LCE0DEZsq6jikhmgmx26xSdOGWAoli7UhbN1N8TGZHGTGToOiWxI7Po5eJ/Xje10VWQcZWklik6XxSlAtkY5Y+jAdeMWjFxhFDN3a2Ijogm1Lp4Ki4Ef/HlZfJ4Vvcv6vj+vNa4KeIowxEcwyn4cAkNuIMmtIDCCJ7hFd486b14797HvLXkFTOH8Afe5w84oI2y

n = 10


2. Uniformly sample

3. Construct canvas


cx,�t


t


0


n� 1


�1:t



X

�t+1




BLM - Training

t = 5

AAAGHHicrVTLbtNAFHVLAsU8msKSzZQoUqKSyI6KgKKgCoTEsgjSVopTazwZO6PaY8tzjRqZ+RA2/AobFiDEhgUSf8PYcanTtKKhjGTrzn2dcx8aJ/KZAMP4tbR8pVK9em3lun7j5q3bq7W1O7siTGJC+yT0w3jfwYL6jNM+MPDpfhRTHDg+3XMOX2T2vXc0Fizkb2ES0WGAPc5cRjAolb1W6TYsP/RQ1Dx6asGYAm6hHspVlkgCO7UE8wJsMY7e2FwiK4rDkZ1Cz5AHKW+bUkXig/TowdRP2vCezFzRcVo9B7J86kLTcmNMUlOmgZyisJ4pDwKEbWbFzBtDCz1TLM7xytkpV5XRo/mtyddbaOMkgK/Lc+jnyhL7ovgLl5Dr7NTcAqk3VEq9odLoDdhQv9kovdH+Q62NuEVGISyK5tFydXPc/17uYni9Etox4wDD2HHQS7skpuU2KJR5g+qHlGfqu1s8C8nWYLAQveliDBXL8sgvz/L/cbkQixysGB1Xo2vD7OxOWP0LnzYq7dwMyKUx1Kb3TEOH3kO7Vjc6Rn7QvGAWQl0rzo5d+2GNQpIElAPxsRAD04hgmOIYGPGp1K1E0AiTQ+zRgRI5DqgYpvnjJlFDaUbIDWP1cUC5thyR4kCISeAoz6zp4rQtU55lGyTgPh6mjEcJUE6mQG7iIwhR9lKiEYspAX+iBExiprgiMsaqpaDeU101wTxd8ryw2+2Ym50nrzfr28+Ldqxo97T7WlMztUfatvZK29H6Gql8qHyqfKl8rX6sfq5+q36fui4vFTF3tZlT/fkbAV4PNA==


1 2 3 4 5 6 7 8 9 10x =



n = 10


2. Uniformly sample

3. Construct canvas


cx,�t


t


0


n� 1


�1:t



X

�t+1




BLM - Training

t = 5



1 2 3 4 5 6 7 8 9 10x =


�1:t = (6, 2, 1, 3, 10)

AAAGN3icrVTLbtNAFHVLAsU8msKSzZQoUqImkR3Kq8ioAiGxQSqCtJXixBpPJs6o9tjyXKNGZv6KDb/BDjYsQIgtf8DESanTtKKhjGTrzn2dcx8aN/KZAMP4vLR8qVC8fGXlqn7t+o2bq6W1W7siTGJC2yT0w3jfxYL6jNM2MPDpfhRTHLg+3XMPno/te+9oLFjI38Ioot0Ae5wNGMGgVM5a4VXF9kMPRdXDJzYMKeAaslCmskUSOKktmBdgm3H0xuES2VEc9p0ULEP2Ut4wpYrEvfSwPvGTDrwnM1d0lFbPgGyfDqBqD2JMUlOmgZygMMuUvQBhh9kx84ZQQ08VizO8MnbKVWX0aHar8vUa2jgO4OvyDPqZMsd+Wvy5S8h0TmpugdQrKqVeUWn0Cmyo32yUXmn8odZA3Cb9EBZF82i+ujnufy93MTwrh3bEOMAwdF30wsmJab4NCmXeoPoh5an61hYfh4zXoLMQvclidBXL/MgvzvL/cTkXiwxsOjquRteA2dkds/oXPg2U27kZkAtjqE23TEOtunVfz9dkVR/UW3Wzfq9uGjWnVDaaRnbQvGBOhbI2PTtO6ZPdD0kSUA7Ex0J0TCOCbopjYMSnUrcTQSNMDrBHO0rkOKCim2bvnkQVpemjQRirjwPKtPmIFAdCjAJXeY7nIU7axsrTbJ0EBo+6KeNRApSTCdAg8RGEaPyIoj6LKQF/pARMYqa4IjLEqtugnlpdNcE8WfK8sNtqmpvNx683y9vPpu1Y0e5od7WqZmoPtW3tpbajtTVS+FD4UvhW+F78WPxa/FH8OXFdXprG3NZmTvHXb01XF38=


n = 10


2. Uniformly sample

3. Construct canvas


cx,�t


t


0


n� 1


�1:t



X

�t+1




BLM - Training

t = 5


alsoThey have ____ which ____ .cx,�t

AAAGT3icrVRbb9MwFM5GO7Zw2+CRF4+pUqv1Eo+OXVDQBELicQi6TWrayHHd1FriRLGDNgX/Q17gjb/BCw8ghJNmLF03sTIsRTo5t+87F9sJPcqFYXydm79VKi/cXlzS79y9d//B8srDAx7EESYdHHhBdOQgTjzKSEdQ4ZGjMCLIdzxy6By/Su2HH0jEacDei9OQ9HzkMjqkGAmlsldKpGJ5gQvC6slzS4yIQDVggkxl8di3E4tT10cWZeCdzSSwwigY2IkwDdlPWANKFYn6yUl97Cdt8RFP/IKztHoGZHlkKKrWMEI4gTLx5RiFmlD2fYBsakXUHYkaeKFYXOGVsVOuKqNLsr8qW62B9fMAtiqvoJ8pC+zz4q9dQqazE7grpF5RKfWKSqNXxDrUJ4P0SuMPswZgFh4EYlYwlxSLm6L+92pnwzMLaGeMfSRGjgNe2wUxKXZBoUwbVDukvFS/scvSkHQLujPRG+9FT7EsTvzmLP8fl2uxyMDy0TE1uoaYnN05q3/h0wCFnZsAuTGGWnQTGmrTzc3Ja2BWn9U36rD+tA6Nwg1JIcx2a7O11dpu7djLa0bTyA6YFmAurGn52beXv1iDAMc+YQJ7iPMuNELRS1AkKPaI1K2YkxDhY+SSrhIZ8gnvJdl7KEFFaQZgGETqYwJk2mJEgnzOT31HeaaD4hdtqfIyWzcWw+1eQlkYC8LwGGgYe0AEIH1cwYBGBAvvVAkIR1RxBXiE1BiEeoJ11QR4seRp4WCjCdvNnbfttb2XeTsWtcfaE62qQW1L29PeaPtaR8OlT6VvpR+ln+XP5e/lXwu56/xcLjzSJs7C0m8O4h0+


1 2 3 4 5 6 7 8 9 10x =


�1:t = (6, 2, 1, 3, 10)


(6, 2, 1, 3, 10)

AAAB+XicbVDLSgMxFL1TX7W+Rl26CRahQhlm2lrtrujGZQX7gHYomTRtQzMPkkyhDP0TNy4UceufuPNvTF+g1gP3cjjnXnJzvIgzqWz7y0htbG5t76R3M3v7B4dH5vFJQ4axILROQh6Klocl5SygdcUUp61IUOx7nDa90d3Mb46pkCwMHtUkoq6PBwHrM4KVlrqmmSvnUSGPnDwq6m5fds2sbdlzINsqF68KlRJyVsqKZGGJWtf87PRCEvs0UIRjKduOHSk3wUIxwuk004kljTAZ4QFtaxpgn0o3mV8+RRda6aF+KHQFCs3VnxsJ9qWc+J6e9LEayr/eTPzPa8eqf+MmLIhiRQOyeKgfc6RCNIsB9ZigRPGJJpgIpm9FZIgFJkqHldEhrH15nTQKllOyKg+lbPV2GUcazuAccuDANVThHmpQBwJjeIIXeDUS49l4M94XoyljuXMKv2B8fAMl45Aj


n = 10


2. Uniformly sample

3. Construct canvas


cx,�t


t


0


n� 1


�1:t



X

�t+1




BLM - Training

t = 5


Transformer

alsoThey have ____ which ____ .cx,�t

AAAGT3icrVRbb9MwFM5GO7Zw2+CRF4+pUqv1Eo+OXVDQBELicQi6TWrayHHd1FriRLGDNgX/Q17gjb/BCw8ghJNmLF03sTIsRTo5t+87F9sJPcqFYXydm79VKi/cXlzS79y9d//B8srDAx7EESYdHHhBdOQgTjzKSEdQ4ZGjMCLIdzxy6By/Su2HH0jEacDei9OQ9HzkMjqkGAmlsldKpGJ5gQvC6slzS4yIQDVggkxl8di3E4tT10cWZeCdzSSwwigY2IkwDdlPWANKFYn6yUl97Cdt8RFP/IKztHoGZHlkKKrWMEI4gTLx5RiFmlD2fYBsakXUHYkaeKFYXOGVsVOuKqNLsr8qW62B9fMAtiqvoJ8pC+zz4q9dQqazE7grpF5RKfWKSqNXxDrUJ4P0SuMPswZgFh4EYlYwlxSLm6L+92pnwzMLaGeMfSRGjgNe2wUxKXZBoUwbVDukvFS/scvSkHQLujPRG+9FT7EsTvzmLP8fl2uxyMDy0TE1uoaYnN05q3/h0wCFnZsAuTGGWnQTGmrTzc3Ja2BWn9U36rD+tA6Nwg1JIcx2a7O11dpu7djLa0bTyA6YFmAurGn52beXv1iDAMc+YQJ7iPMuNELRS1AkKPaI1K2YkxDhY+SSrhIZ8gnvJdl7KEFFaQZgGETqYwJk2mJEgnzOT31HeaaD4hdtqfIyWzcWw+1eQlkYC8LwGGgYe0AEIH1cwYBGBAvvVAkIR1RxBXiE1BiEeoJ11QR4seRp4WCjCdvNnbfttb2XeTsWtcfaE62qQW1L29PeaPtaR8OlT6VvpR+ln+XP5e/lXwu56/xcLjzSJs7C0m8O4h0+

ice ____ ____ cream

is ____ ____ really ____

____ goodax,�t

AAAGT3icrVRbb9MwFM5GO7Zw2+CRF4+pUqv1Eo+OXVDQBELicQi6TWrayHHd1FriRLGDNgX/Q17gjb/BCw8ghJNmLF03sTIsRTo5t+87F9sJPcqFYXydm79VKi/cXlzS79y9d//B8srDAx7EESYdHHhBdOQgTjzKSEdQ4ZGjMCLIdzxy6By/Su2HH0jEacDei9OQ9HzkMjqkGAmlsldKpGJ5gQvC6slzS4yIQDVggkxl8di3E4tT10cWZeCdzSSwwigY2IkwDdlPWANKFYn6yUl97Cdt8RFP/IKztHoGZHlkKKrWMEI4gTLx5RiFmlD2fYBsakXUHYkaeKFYXOGVsVOuKqNLsr8qW62B9fMAtiqvoJ8pC+zz4q9dQqazE7grpF5RKfWKSqNXxDrUJ3PolcYfZg3ALDwIxKxgLikWN0X979XOhmcW0M4Y+0iMHAe8tgtiUuyCQpk2qHZIeal+Y5elIekWdGeiN96LnmJZnPjNWf4/LtdikYHlo2NqdA0xObtzVv/CpwEKOzcBcmMMtegmNNSmm5uT18CsPqtv1GH9aR0ahRuSQpjt1mZrq7Xd2rGX14ymkR0wLcBcWNPys28vf7EGAY59wgT2EOddaISil6BIUOwRqVsxJyHCx8glXSUy5BPeS7L3UIKK0gzAMIjUxwTItMWIBPmcn/qO8kwHxS/aUuVltm4shtu9hLIwFoThMdAw9oAIQPq4ggGNCBbeqRIQjqjiCvAIqTEI9QTrqgnwYsnTwsFGE7abO2/ba3sv83Ysao+1J1pVg9qWtqe90fa1joZLn0rfSj9KP8ufy9/LvxZy1/m5XHikTZyFpd8FpB08


1 2 3 4 5 6 7 8 9 10x =


�1:t = (6, 2, 1, 3, 10)


(6, 2, 1, 3, 10)

AAAB+XicbVDLSgMxFL1TX7W+Rl26CRahQhlm2lrtrujGZQX7gHYomTRtQzMPkkyhDP0TNy4UceufuPNvTF+g1gP3cjjnXnJzvIgzqWz7y0htbG5t76R3M3v7B4dH5vFJQ4axILROQh6Klocl5SygdcUUp61IUOx7nDa90d3Mb46pkCwMHtUkoq6PBwHrM4KVlrqmmSvnUSGPnDwq6m5fds2sbdlzINsqF68KlRJyVsqKZGGJWtf87PRCEvs0UIRjKduOHSk3wUIxwuk004kljTAZ4QFtaxpgn0o3mV8+RRda6aF+KHQFCs3VnxsJ9qWc+J6e9LEayr/eTPzPa8eqf+MmLIhiRQOyeKgfc6RCNIsB9ZigRPGJJpgIpm9FZIgFJkqHldEhrH15nTQKllOyKg+lbPV2GUcazuAccuDANVThHmpQBwJjeIIXeDUS49l4M94XoyljuXMKv2B8fAMl45Aj


n = 10


2. Uniformly sample

3. Construct canvas


cx,�t


t


0


n� 1


�1:t



X

�t+1




BLM - Inference

Simple greedy decoding or beam search to fill in the blanks in input

not search for sentence with the maximum marginal likelihood ,

but for sentence and trajectory that have the maximum joint likelihood

p(x; ✓)

AAAGXHicrVRbb9MwFM5GW7qMwQYSL7x4TJU6rV2TsrELCpqEQDwOwS5S00aO66bWEieKT9CmYInfyNte+CvgpB1L101sDEuJjo/POd93LrYb+UyAYZzPzD4olSsPq3P6/KOFx08Wl54eijCJCT0goR/Gxy4W1GecHgADnx5HMcWB69Mj9+Rddn70lcaChfwLnEW0G2CPswEjGJTKWSrFNdsPPRTVT9/YMKSAV5GFcpUtksBJbcG8ANuMo88Ol8iO4rDvpGAZspfypimVJ+6lp42RnXTgG5nYoouweg5k+3QAdXsQY5KaMg3kCIVZpuwFCDvMjpk3hFX0VrG4wSpnp0xVRI/muzpfXkVrlw58Wd5AP1cW2I+Tv3UKuc5JzV2Qek2F1GsqjF6DNfWbDKLXmn+oNRG3ST+Eu6J5tJjdFPe/p3s3PKuAdsE4wDB0XfTeKYhpsQwKZfpA1UPKa/XtXZ65ZGPQuRO90WB0Fctiy+/P8v9xuRWLHGzcOq5a14TJ3l2y+hc+TVSYuQmQe2OoSbdMQ426tTl5D6z660a7YTZeNUyjcEUyCGujtdnaam23dvTCA+MsrhjrRr7QtGCOhRVtvPadxR92PyRJQDkQHwvRMY0IuimOgRGfSt1OBI0wOcEe7SiR44CKbpo/jhLVlKaPBmGsPg4o1xY9UhwIcRa4yjJrmrh6limvO+skMNjupoxHCVBORkCDxEcQouylRX0WUwL+mRIwiZniisgQq5aAeo91VQTzasrTwmF73dxY3/m0sbL34fuoHFXthfZSq2umtqXtaR+1fe1AI6Xz0q9ytTxX/lkpVeYrCyPT2ZlxCZ9pE6vy/DeLWx5b

p(x,�; ✓)

AAAGY3icrVRbb9MwFM7GWkYGrBu8ISSPqVKn9ZKMjV1Q0CQE4nEIdpGaNnJcN7WWOFF8glYFS/xG3njjhf+Bk3YsXTexMSwlOj4+53zfudhu5DMBhvFjZvbeXKl8f/6BvvDw0ePFytLykQiTmNBDEvphfOJiQX3G6SEw8OlJFFMcuD49dk/fZufHX2gsWMg/wzCinQB7nPUZwaBUztLcsGr7oYei2tlrGwYU8BqyUK6yRRI4qS2YF2CbcfTJ4RLZURz2nBQsQ3ZT3jCl8sTd9Kw+spMOfCUTW3QeVs+BbJ/2oWb3Y0xSU6aBHKEwy5TdAGGH2THzBrCG3igW11jl7JSpiujRfFfjK2to/cKBr8hr6OfKAvtx8jdOIdc5qbkHUq+qkHpVhdGrsK5+k0H0auMPtQbiNumFcFs0jxazm+L+93Rvh2cV0M4ZBxgGroveOQUxLZZBoUwfqHpIeaV+Y49nLtkYtG9FbzQYHcWy2PK7s/x/XG7EIgcbt46r1jVgsncXrP6FTwMVZm4C5M4YatIt01Cjbm1N3gOr9qq+UTfrL+umUbgiGYS12dpqbbd2Wru6emDGQc8DOpVVo2nkC00L5lhY1cbrwKl8t3shSQLKgfhYiLZpRNBJcQyM+FTqdiJohMkp9mhbiRwHVHTS/I2UqKo0PdQPY/VxQLm26JHiQIhh4CrLrHfi8lmmvOqsnUB/p5MyHiVAORkB9RMfQYiyBxf1WEwJ+EMlYBIzxRWRAVadAfUs66oI5uWUp4Wjjaa52dz9uLm6//7bqBzz2jPthVbTTG1b29c+aAfaoUbmfpbKpcVSpfSrvFBeLj8dmc7OjEv4RJtY5ee/AX1LIDs=

x

AAAGZnicrVRbT9RAFC7IrlCVa4wPvgySJkvYZVsEuZgaEqPxEaNcku3STGdnuxPaadM5NZAyib/RN5998Wc47S7QZSGCOEmbM2fOOd93LjNeHDABpvlzbPzRRKX6eHJKf/L02fTM7Nz8gYjShNB9EgVRcuRhQQPG6T4wCOhRnFAcegE99E7e5+eH32giWMS/wllM2yH2OesygkGp3LmJc8MJIh/FtdO3DvQo4GVko0LliDR0M0cwP8QO4+iLyyVy4iTquBnYpjzOeMOSyhMfZ6f1vp104ZwMbdFFWL0AcgLahZrTTTDJLJmFso/CbEsehwi7zEmY34Nl9E6xuMWqYKdMVUSfFrsaX1xGK1cOfFHeQr9QltgPkr9zCoXOzawdkLqhQuqGCqMbsKJ+w0F0o3FJrYG4QzoR3BfNp+XsRrj/Pd374dkltAvGIYae56EPbknMymVQKKMHqh5S3qhf2+G5Sz4GrXvR6w9GW7Est/zhLP8flzuxKMAGreOqdQ0Y7t0Vq3/h00ClmRsCeTCGmnTbMtWo2xvD98Cuvamv1a3667pllq5IDmGvNzeam82t5rZuqBdmEPUy4qk7u2SumsVCo4I1EJa0wdpzZ384nYikIeVAAixEyzJjaGc4AUYCKnUnFTTG5AT7tKVEjkMq2lnxTEpkKE0HdaNEfRxQoS17ZDgU4iz0lGXePnH9LFfedNZKobvVzhiPU6Cc9IG6aYAgQvmbizosoQSCMyVgkjDFFZEeVs0B9TLrqgjW9ZRHhYO1VWt9dfvz+tLux+/9ckxqL7VXWk2ztE1tV/uk7Wn7Gpn4VZmqzFcWKr+r09Xn1Rd90/GxQQkXtKFVRX8As+shAA==

x

AAAGZnicrVRbT9RAFC7IrlCVa4wPvgySJkvYZVsEuZgaEqPxEaNcku3STGdnuxPaadM5NZAyib/RN5998Wc47S7QZSGCOEmbM2fOOd93LjNeHDABpvlzbPzRRKX6eHJKf/L02fTM7Nz8gYjShNB9EgVRcuRhQQPG6T4wCOhRnFAcegE99E7e5+eH32giWMS/wllM2yH2OesygkGp3LmJc8MJIh/FtdO3DvQo4GVko0LliDR0M0cwP8QO4+iLyyVy4iTquBnYpjzOeMOSyhMfZ6f1vp104ZwMbdFFWL0AcgLahZrTTTDJLJmFso/CbEsehwi7zEmY34Nl9E6xuMWqYKdMVUSfFrsaX1xGK1cOfFHeQr9QltgPkr9zCoXOzawdkLqhQuqGCqMbsKJ+w0F0o3FJrYG4QzoR3BfNp+XsRrj/Pd374dkltAvGIYae56EPbknMymVQKKMHqh5S3qhf2+G5Sz4GrXvR6w9GW7Est/zhLP8flzuxKMAGreOqdQ0Y7t0Vq3/h00ClmRsCeTCGmnTbMtWo2xvD98Cuvamv1a3667pllq5IDmGvNzeam82t5rZuqBdmEPUy4qk7u2SumsVCo4I1EJa0wdpzZ384nYikIeVAAixEyzJjaGc4AUYCKnUnFTTG5AT7tKVEjkMq2lnxTEpkKE0HdaNEfRxQoS17ZDgU4iz0lGXePnH9LFfedNZKobvVzhiPU6Cc9IG6aYAgQvmbizosoQSCMyVgkjDFFZEeVs0B9TLrqgjW9ZRHhYO1VWt9dfvz+tLux+/9ckxqL7VXWk2ztE1tV/uk7Wn7Gpn4VZmqzFcWKr+r09Xn1Rd90/GxQQkXtKFVRX8As+shAA==

�

AAAGa3icrVRbb9MwFM5GW0aAXdgDCHjwmCK1WrsmpWMXFDQJgXgcgl2kpo0c102tJU4UO2hVsAR/kTf+AS/8B5w029J1ExvDUqLj43PO952L7YQeYVzXf87M3imVK3fn7qn3HzycX1hcenTAgjhCeB8FXhAdOZBhj1C8zwn38FEYYeg7Hj50jt+m54dfcMRIQD/zUYi7PnQpGRAEuVTZS6XvmuUFLgirJ68tPsQc1oAJMpXFYt9OLEZcH1qEgk82FcAKo6BvJ9zURS+hDUNIT9hLTupjO2Hzr2hiC07DqhmQ5eEBr1qDCKLEEIkvxijENETPB9AmVkTcIa+BN5LFFVYZO2kqI7o421XpSg2snTvQFXEF/UxZYJ8nf+0UMp2dGDtcqJoMqWoyjKrxNfmbDKJqjTNqDUAt1A/4TdFcXMxuivvf070ZnllAO2XsQz50HPDOLohJsQwSZfpA1kOIS/WtHZq6pGPQuRG98WB0Jctiy2/P8v9xuRaLDCxvHZWta/DJ3p2z+hc+DVCYuQmQW2PISTcNXY66uTF5D8zqq3qrbtRf1g29cEVSCLPd3GhuNrea26omX5g86lnE3HRxVV/XswWmBSMXVpV87dmLP6x+gGIfU448yFjH0EPeTWDECfKwUK2Y4RCiY+jijhQp9DHrJtlbKYAmNX0wCCL5UQ4ybdEjgT5jI9+RlmkP2cWzVHnZWSfmg61uQmgYc0zRGGgQe4AHIH14QZ9EGHFvJAWIIiK5AjSEskNcPs+qLIJxMeVp4aC1brTXtz+2V3fffxuXY055prxQqoqhbCq7ygdlT9lXUOlXeb78uPyk/LuyXHlaeT42nZ3JS7isTKyK9gdtoSMn

Note:

Experiments

Input: They also have which . Output: They also have ice cream which is really good .

Input: τε εγγονον εισαι???????σοφιαι Output: τε εγγονον εισαιου του σοφιαι

Input: The employees were super nice and efficient ! Output: The employees were rude and unprofessional !

Output: They also have ice cream which is really good.

Text infilling

Ancient text restoration

Sentiment transfer

Language modeling

Text Infilling

Dataset: Yahoo Answers (100K documents, max length 200 words)

Test data: randomly mask tokens with ratio , contiguous masked tokens —> “ ”

Metrics:

• Accuracy:

• Fluency:

r

AAAGbnicrVRbT9RAFC7IrlgvgCY+SIyDpMlu2GVbBLmYGhKj8RGjXJLt0kxnZ7sT2mnTOTWQOokP/kLf/A2++BOcdhfpshBBnKTNmTPnnO87lxkvDpgA0/wxMXlrqlK9PX1Hv3vv/oOZ2bmHeyJKE0J3SRREyYGHBQ0Yp7vAIKAHcUJx6AV03zt6k5/vf6aJYBH/BCcx7YTY56zHCAalcuemvhlOEPkorh2/cqBPAdeRjQqVI9LQzRzB/BA7jKOPLpfIiZOo62Zgm/Iw401LKk98mB03BnbShS9kZItOw+oFkBPQHtScXoJJZskslAMUZlvyMETYZU7C/D7U0WvF4hKrgp0yVRF9WuxqfKGOls4c+IK8hH6hLLEfJn/lFAqdm1lbIHVDhdQNFUY3YEn9RoPoRvMPtSbiDulGcF00n5azG+P+93Svh2eX0E4Zhxj6nofeuiUxK5dBoYwfqHpIeaF+ZYvnLvkYtK9FbzAYHcWy3PKbs/x/XK7EogAbto6r1jVhtHdnrP6FTxOVZm4E5MYYatJty1Sjbq+N3gO79rKx0rAaLxqWWboiOYS92lprrbc2Wpu6oV6YYdTz10lP3NlFc9ksFhoXrKGwqA3Xjjv73elGJA0pBxJgIdqWGUMnwwkwElCpO6mgMSZH2KdtJXIcUtHJiudSIkNpuqgXJerjgApt2SPDoRAnoacs8zaK82e58qKzdgq9jU7GeJwC5WQA1EsDBBHK317UZQklEJwoAZOEKa6I9LFqEqgXWldFsM6nPC7srSxbq8ubH1YXt999HZRjWpvXnms1zdLWtW3tvbaj7Wpk6mdlrvKkMl/5VX1cfVp9NjCdnBiW8JE2sqq136WDI+Y=

BLEU score against original document

perplexity evaluated by a pre-trained left-to-right LM

Text Infilling

Baselines:

• BERT+LM

- use BERT representation of each blank as seed for a left-to-right LM that learns to

generate tokens in the corresponding blank

- at test time, multiple blanks are filled in one after another

Text Infilling

Baselines:

• BERT+LM

• Masked Language Model (MLM) with oracle length

- replace blanks with the target number of ⟨mask⟩ tokens

- fill masks autoregressively by most-confident-first heuristic

Text Infilling

Baselines:

• BERT+LM


• Insertion Transformer (Stern et al., 2019)

- not allow users to specify where to insert

- force it to generate at valid locations, disable ⟨eos⟩ unless all blanks are filled,

prioritize slots that haven’t been filled yet; otherwise failure rate > 88%

Text Infilling

Baselines:

• BERT+LM



• Seq2seq-full (Donahue et al., 2020)

- output the full document from input

- may miss existing tokens in input or generate tokens in incorrect locations

• Seq2seq-fill (Donahue et al., 2020)

- output tokens to be placed in blanks, using ‘|’ to indicate separation

- may not generate the correct number of ‘|’

Text Infilling

Baselines:

• BERT+LM



• Seq2seq-full (Donahue et al., 2020)

- output the full document from input

- may miss existing tokens in input or generate tokens in incorrect locations

• Seq2seq-fill (Donahue et al., 2020)

- output tokens to be placed in blanks, using ‘|’ to indicate separation

- may not generate the correct number of ‘|’

failure rate from 15% to 47%

Text Infilling

BLEU

20

30

40

50

60

70

80

90

Mask ratio10% 20% 30% 40% 50%

BERT+LM MLM (oracle length) InsT BLM

Text Infilling

Perp

lexi

ty

0

10

20

30

40

50

60

Mask ratio10% 20% 30% 40% 50%

BERT+LM MLM (oracle length) InsT BLM

original data

Text Infilling

when time flies , where does it go ? to the center of the earth to be recycled came made into new time .

when time flies , where does it go ? from the center of the earth to be recycled converted made into new time .

when time flies , where does it go ? for the center of the earth has to be recycled and made into new time .

when time flies , where does it go ? for the center of the earth to be recycled and made into new time .

Original

Blanked

BERT+LM

MLM (oracle len)

InsT

BLM

when time flies , where does it go ? to the center of the universe to be recycled and made into new time .

when time flies , does it go ? the center of the to be recycled made into new time .

Mask ratio 10%

Text Infilling

when time flies , where does it go ? to the center of the universe to be recycled and made into new time .

when time , where ? the of universe to recycled made into .

when time is , where to ? i need to find the way of the universe to be recycled and made into a lot .

when time is , where is the universe ? from the creation of the universe to be recycled and made into the universe .

when time was created , where was it ? what was the name of the universe to be recycled and made into space .

when time was created , where did it come from ? it was the first part of the universe to be recycled and made into space .

Original

Blanked

BERT+LM

MLM (oracle len)

InsT

BLM

Mask ratio 50%

Ancient Text Restoration

Dataset (Assael et al., 2019): ancient Greek inscriptions (18M characters/3M words)

• number of characters to recover is assumed to be known

l 2 {0, · · · , t� 1}

AAAGgXicrVRbb9MwFM4uLSPcNnjkxWNEarWkTcrGbgqahEA8DsEuUt1Fjuum1hInih20KVjiX/C/eOPPIJy0Y+m6iY1hKdHx8Tnn+87F9pOQcmHbP2dm5+Zr9XsL9/UHDx89frK49PSAx1mKyT6Owzg98hEnIWVkX1ARkqMkJSjyQ3Lon7wtzg+/kJTTmH0WZwnpRShgdEAxEkrlLc1/N2AYByBpnO5AMSQCNYELShXkWeTlkNMgQpAy8MljEsAkjfteLlxbHufMcqTyRMf5qTmyk574iie24DysXgLBkAxEAw5ShHNH5pEcoVDXkccRQB6FKQ2GogneKBbXWJXslKmKGJBy12DLTbB64cCW5TX0S2WF/Tj5G6dQ6rzc2RZSN1RI3VBhdEOsqt9kEN2w/lCzAIO4H4vbogWkmt0U97+nezs8t4J2zjhCYuj74J1XEfNqGRTK9IGqh5RX6jvbrHApxqB7K3qjwegpltWW353l/+NyIxYl2Lh1TLXOEpO9u2D1L3wsUJm5CZA7Y6hJdx1bjbq7PnkP3MZrs2M65ivTsStXpIBw19rr7Y32ZntLN9QLM456+TrpoRpYmNtm2UtuCsuB0ltcsVt2ucC04IyFFW289rzFH7Af4ywiTOAQcd517ET0cpQKikMidZhxkiB8ggLSVSJDEeG9vHxBJTCUpg8Gcao+JkCprXrkKOL8LPKVZdFZfvmsUF511s3EYLOXU5ZkgjA8AhpkIRAxKJ5j0KcpwSI8UwLCKVVcAR4i1TehHm1dFcG5nPK0cNBpOWutrY9rK7vvv43KsaA9115oDc3RNrRd7YO2p+1reP5X7WXNqrXqc/Vm3a53RqazM+MSPtMmVn3nN0y9KlU=

t� 1� l

AAAGiHicrVRbb9MwFM7GWka4bfDIi8cUqdOSNRkdu6CgCQTicQi2IdVd5Lhuas1xothBm4Ilfgk/ijf+DU7asXTdxMawlOj4+JzzfedihymjQrrur5nZO3ON5t35e+b9Bw8fPV5YfHIgkjzDZB8nLMm+hEgQRjnZl1Qy8iXNCIpDRg7D47fl+eFXkgma8M/yNCW9GEWcDihGUquCxbkfFmRJBNLWySsoh0SiFeCDSgVFHgcFFDSKEaQcfAq4AjDNkn5QSN9VRwV3PKU90VFxYo/sVCC/4YktOAtrVkCQkYFswUGGcOGpIlYjFOp76igGKKAwo9FQroDXmsUVVhU7baojRqTatfjSClg9d+BL6gr6lbLGfpz8tVOodEHh7UhlWjqkaekwpiVX9W8yiGk5f6g5gEPcT+RN0SJSz26K+9/TvRmeX0M7YxwjOQxD8C6oiUW9DBpl+kDXQ6lL9es7vHQpx6B7I3qjwehplvWW357l/+NyLRYV2Lh1XLfOkZO9O2f1L3wcUJu5CZBbY+hJ9z1Xj7q/MXkP/NZLe9327Be259auSAnhd9ob7c32VnvbtPQLM4568TqZFtMTCwvXrpopbOl4UJn677BgYdldc6sFpgVvLCwb47UXLPyE/QTnMeESMyRE13NT2StQJilmRJkwFyRF+BhFpKtFjmIiekX1kCpgaU0fDJJMf1yCSlv3KFAsxGkcasuyweLiWam87Kyby8FWr6A8zSXheAQ0yBmQCShfZdCnGcGSnWoB4YxqrgAPkW6f1G+3qYvgXUx5WjhYX/M6a9sfO8u777+PyjFvPDOeGy3DMzaNXeODsWfsG7jRaKw2Oo2Nptl0m5vN7ZHp7My4hE+NidV88xsLiCs2

Length-aware BLM (L-BLM)

• [t] denotes blank of length t

• predict length of the new left blank,

new right blank has length

Baselines (Assael et al., 2019):

• Pythia: character-level seq2seq model to fill in one slot at a time

• Pythia-word: use both character and word representations

Ancient Text Restoration

Cha

ract

er e

rror r

ate

20

25

30

35

40

45

50

55

60

1% 25% 40% 50%

Human Pythia Pythia-word L-BLM

Single-slot Multi-slot

Mask ratio

Sentiment Transfer

Two-step approach:

1. Remove expressions of high polarity

- train a sentiment classifier and mask words with attention weight above average

2. Complete the partial sentence with expressions of the target sentiment

- train two instances of BLM, one for each sentiment

everyone that i spoke with was very helpful and kind .

everyone that i spoke with was rude and unprofessional .

there is definitely not enough room in that part of the venue .

there is always enough parking in that part of the venue .

it is n’t terrible , but it is n’t very good either .

it is n’t fancy , but it is still very good either .

Input

BLM

Input

BLM

Input

BLM

Sentiment Transfer

30

40

50

60

70

80

90

100

Accuracy

MLM (Wu et al. 2019) BLM

10

12

14

16

18

20

22

BLEU

Yelp reviews

Language Modeling

To compute , use Monte-Carlo estimate

• estimated perplexity is likely to be higher than actual perplexity

• as increases, it converges to actual perplexity

p(x; ✓) =X

�2Sn

p(x,�; ✓)

AAAGvXicrVRbb9MwFM7GWka4bfDIi8cUqdOSNSkdu6DAJATicQh2keo2OK6bmjlOFDuwKUTiN8IT/wYn7Vi6bmJjWEpycnx8vu9cfPyYUSFt+9fM7K25Wv32/B397r37Dx4uLD7aF1GaYLKHIxYlhz4ShFFO9iSVjBzGCUG

blank language modelspeople.csail.mit.edu/tianxiao/papers/emnlp20_blm_slides.pdf · blm - overview...

Documents