machine learning and finance ii
TRANSCRIPT
-
8/9/2019 Machine Learning and Finance II
1/103
Machine learning and portfolio selections. II.
Laszlo (Laci) Gyorfi1
1Department of Computer Science and Information TheoryBudapest University of Technology and Economics
Budapest, Hungary
August 8, 2007
e-mail: [email protected]/gyorfiwww.szit.bme.hu/oti/portfolio
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
2/103
Dynamic portfolio selection: general case
xi = (x(1)i , . . . x
(d)i ) the return vector on day i
b = b1 is the portfolio vector for the first day
initial capital S0
S1 = S0 b1 , x1
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
3/103
Dynamic portfolio selection: general case
xi = (x(1)i , . . . x
(d)i ) the return vector on day i
b = b1 is the portfolio vector for the first day
initial capital S0
S1 = S0 b1 , x1
for the second day, S1 new initial capital, the portfolio vectorb2 = b(x1)
S2 = S0 b1 , x1 b(x1) , x2 .
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
4/103
Dynamic portfolio selection: general case
xi = (x(1)i , . . . x
(d)i ) the return vector on day i
b = b1 is the portfolio vector for the first day
initial capital S0
S1 = S0 b1 , x1
for the second day, S1 new initial capital, the portfolio vectorb2 = b(x1)
S2 = S0 b1 , x1 b(x1) , x2 .
nth day a portfolio strategy bn = b(x1, . . . , xn1) = b(xn11 )
Sn = S0
n
i=1b(xi11 ) , xi = S0enWn(B)
with the average growth rate
Wn(B) =1
n
n
i=1ln
b(xi11 ) , xi
.
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
5/103
log-optimum portfolio
X1, X2, . . . drawn from the vector valued stationary and ergodicprocesslog-optimum portfolio B = {b()}
E{ln
b(Xn11 ) , Xn
| Xn11 } = maxb()
E{ln
b(Xn11 ) , Xn
| Xn11 }
Xn11 = X1, . . . , Xn1
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
6/103
Optimality
Algoet and Cover (1988): If Sn = Sn(B) denotes the capital after
day n achieved by a log-optimum portfolio strategy B, then for
any portfolio strategy B with capital Sn = Sn(B) and for anyprocess {Xn}
,
lim supn
1
nln Sn
1
nln Sn 0 almost surely
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
7/103
Optimality
Algoet and Cover (1988): If Sn = Sn(B) denotes the capital after
day n achieved by a log-optimum portfolio strategy B, then for
any portfolio strategy B with capital Sn = Sn(B) and for anyprocess {Xn}
,
lim supn
1
nln Sn
1
nln Sn 0 almost surely
for stationary ergodic process {Xn},
limn
1
nln Sn = W
almost surely,
where
W = E
maxb()
E{ln
b(X1) , X0
| X1}
is the maximal growth rate of any portfolio.
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
8/103
Martingale difference sequences
for the proof of optimality we use the concept of martingale
differences:
Definition
there are two sequences of random variables:
{Zn} {Xn}
Zn is a function of X1, . . . , Xn,
E{Zn | X1, . . . , Xn1} = 0 almost surely.
Then {Zn} is called martingale difference sequence with respect to{Xn}.
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
9/103
A strong law of large numbers
Chow Theorem: If {Zn} is a martingale difference sequence withrespect to {Xn} and
n=1
E{Z2n }
n2
<
then
limn
1
n
ni=1
Zi = 0 a.s.
Gyorfi Machine learning and portfolio selections. II.
A k l f l b
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
10/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Gyorfi Machine learning and portfolio selections. II.
A k l f l b
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
11/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj}
Gyorfi Machine learning and portfolio selections. II.
A k l f l b
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
12/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj} = E{E{ZiZj | X1, . . . , Xj1}}
Gyorfi Machine learning and portfolio selections. II.
A k l f l b
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
13/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj} = E{E{ZiZj | X1, . . . , Xj1}}
= E{ZiE{Zj | X1, . . . , Xj1}}
Gyorfi Machine learning and portfolio selections. II.
A k l f l b s
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
14/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj} = E{E{ZiZj | X1, . . . , Xj1}}
= E{ZiE{Zj | X1, . . . , Xj1}}
= E{Zi 0}
Gyorfi Machine learning and portfolio selections. II.
A weak law of large numbers
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
15/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj} = E{E{ZiZj | X1, . . . , Xj1}}
= E{ZiE{Zj | X1, . . . , Xj1}}
= E{Zi 0} = 0
Gyorfi Machine learning and portfolio selections. II.
A weak law of large numbers
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
16/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj} = E{E{ZiZj | X1, . . . , Xj1}}
= E{ZiE{Zj | X1, . . . , Xj1}}
= E{Zi 0} = 0
Corollary
E
1
n
ni=1
Zi
2
Gyorfi Machine learning and portfolio selections. II.
A weak law of large numbers
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
17/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj} = E{E{ZiZj | X1, . . . , Xj1}}
= E{ZiE{Zj | X1, . . . , Xj1}}
= E{Zi 0} = 0
Corollary
E
1
n
ni=1
Zi
2
=1
n2
ni=1
nj=1
E{ZiZj}
Gyorfi Machine learning and portfolio selections. II.
A weak law of large numbers
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
18/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj} = E{E{ZiZj | X1, . . . , Xj1}}
= E{ZiE{Zj | X1, . . . , Xj1}}
= E{Zi 0} = 0
Corollary
E
1
n
ni=1
Zi
2
=1
n2
ni=1
nj=1
E{ZiZj}
=1
n2
ni=1
E{Z2i }
Gyorfi Machine learning and portfolio selections. II.
A weak law of large numbers
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
19/103
A weak law of large numbers
Lemma: If {Zn} is a martingale difference sequence with respectto {Xn} then {Zn} are uncorrelated.
Proof. Put i < j.E{ZiZj} = E{E{ZiZj | X1, . . . , Xj1}}
= E{ZiE{Zj | X1, . . . , Xj1}}
= E{Zi 0} = 0
Corollary
E
1
n
ni=1
Zi
2
=1
n2
ni=1
nj=1
E{ZiZj}
=1
n2
ni=1
E{Z2i }
0
if, for example,E
{Z2i } is a bounded sequence.
Gyorfi Machine learning and portfolio selections. II.
Constructing martingale difference sequence
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
20/103
Constructing martingale difference sequence
{Yn} is an arbitrary sequence such that Yn is a function of
X1, . . . , Xn
Gyorfi Machine learning and portfolio selections. II.
Constructing martingale difference sequence
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
21/103
Constructing martingale difference sequence
{Yn} is an arbitrary sequence such that Yn is a function of
X1, . . . , XnPut
Zn = Yn E{Yn | X1, . . . , Xn1}
Then {Zn} is a martingale difference sequence:
Gyorfi Machine learning and portfolio selections. II.
Constructing martingale difference sequence
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
22/103
Constructing martingale difference sequence
{Yn} is an arbitrary sequence such that Yn is a function of
X1, . . . , XnPut
Zn = Yn E{Yn | X1, . . . , Xn1}
Then {Zn} is a martingale difference sequence:
Zn is a function of X1, . . . , Xn,
Gyorfi Machine learning and portfolio selections. II.
Constructing martingale difference sequence
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
23/103
Constructing martingale difference sequence
{Yn} is an arbitrary sequence such that Yn is a function of
X1, . . . , XnPut
Zn = Yn E{Yn | X1, . . . , Xn1}
Then {Zn} is a martingale difference sequence:
Zn is a function of X1, . . . , Xn,
E{Zn | X1, . . . , Xn1}
Gyorfi Machine learning and portfolio selections. II.
Constructing martingale difference sequence
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
24/103
C g g q
{Yn} is an arbitrary sequence such that Yn is a function of
X1, . . . , XnPut
Zn = Yn E{Yn | X1, . . . , Xn1}
Then {Zn} is a martingale difference sequence:
Zn is a function of X1, . . . , Xn,
E{Zn | X1, . . . , Xn1}
= E{Yn E{Yn | X1, . . . , Xn1} | X1, . . . , Xn1}
= 0
almost surely.
Gyorfi Machine learning and portfolio selections. II.
Optimality
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
25/103
p y
log-optimum portfolio B = {b()}
E{ln
b(Xn11 ) , Xn
| Xn11 } = maxb()
E{ln
b(Xn11 ) , Xn
| Xn11 }
Gyorfi Machine learning and portfolio selections. II.
Optimality
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
26/103
p y
log-optimum portfolio B = {b()}
E{ln
b(Xn11 ) , Xn
| Xn11 } = maxb()
E{ln
b(Xn11 ) , Xn
| Xn11 }
If Sn = Sn(B) denotes the capital after day n achieved by a
log-optimum portfolio strategy B, then for any portfolio strategyB with capital Sn = Sn(B) and for any process {Xn}
,
lim supn
1n ln S
n 1n ln Sn
0 almost surely
Gyorfi Machine learning and portfolio selections. II.
Proof of optimality
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
27/103
p y
1
n
ln Sn =1
n
n
i=1
lnb(Xi11
) , Xi
Gyorfi Machine learning and portfolio selections. II.
Proof of optimality
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
28/103
1
n
ln Sn =1
n
n
i=1
lnb(Xi11 ) , Xi=
1
n
ni=1
E{ln
b(Xi11 ) , Xi
| Xi11 }
+1
n
ni=1
ln
b(Xi11 ) , Xi
E{ln
b(Xi11 ) , Xi
| Xi11 }
Gyorfi Machine learning and portfolio selections. II.
Proof of optimality
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
29/103
1
n
ln Sn =1
n
n
i=1
lnb(Xi11 ) , Xi=
1
n
ni=1
E{ln
b(Xi11 ) , Xi
| Xi11 }
+1
n
ni=1
ln
b(Xi11 ) , Xi
E{ln
b(Xi11 ) , Xi
| Xi11 }
and
1
n ln Sn =
1
n
ni=1
E{ln
b
(Xi11 ) , Xi
| X
i11 }
+1
n
ni=1
ln
b(Xi11 ) , Xi
E{ln
b(Xi11 ) , Xi
| Xi11 }
Gyorfi Machine learning and portfolio selections. II.
Universally consistent portfolio
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
30/103
These limit relations give rise to the following definition:
Definition
An empirical (data driven) portfolio strategy B is called
universally consistent with respect to a class C of stationaryand ergodic processes {Xn}
, if for each process in the class,
limn
1
nln Sn(B) = W
almost surely.
Gyorfi Machine learning and portfolio selections. II.
Empirical portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
31/103
E{ln b(Xn11 ) , Xn | X
n11 } = max
b()E{ln b(X
n11 ) , Xn | X
n11 }
Gyorfi Machine learning and portfolio selections. II.
Empirical portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
32/103
E{ln b(Xn11 ) , Xn | X
n11 } = max
b()E{ln b(X
n11 ) , Xn | X
n11 }
fixed integer k > 0
E{ln
b(Xn11 ) , Xn
| Xn11 } E{ln
b(Xn1nk) , Xn
| Xn1nk}
Gyorfi Machine learning and portfolio selections. II.
Empirical portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
33/103
E{ln b(Xn11 ) , Xn | X
n11 } = max
b()E{ln b(X
n11 ) , Xn | X
n11 }
fixed integer k > 0
E{ln
b(Xn11 ) , Xn
| Xn11 } E{ln
b(Xn1nk) , Xn
| Xn1nk}
and
b(Xn11 ) bk(Xn1nk) = argmaxb()
E{ln
b(Xn1nk) , Xn
| Xn1nk}
Gyorfi Machine learning and portfolio selections. II.
Empirical portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
34/103
E{ln b(Xn11 ) , Xn | X
n11 } = max
b()E{ln b(X
n11 ) , Xn | X
n11 }
fixed integer k > 0
E{ln
b(Xn11 ) , Xn
| Xn11 } E{ln
b(Xn1nk) , Xn
| Xn1nk}
and
b(Xn11 ) bk(Xn1nk) = argmaxb()
E{ln
b(Xn1nk) , Xn
| Xn1nk}
because of stationarity
bk(xk1 ) = argmax
b()E{lnb(x
k1 ) , Xk+1 | X
k1 = x
k1 }
= argmaxb
E{ln b , Xk+1 | Xk1 = x
k1 },
Gyorfi Machine learning and portfolio selections. II.
Empirical portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
35/103
E{ln b(Xn11 ) , Xn | X
n11 } = max
b()E{ln b(X
n11 ) , Xn | X
n11 }
fixed integer k > 0
E{ln
b(Xn11 ) , Xn
| Xn11 } E{ln
b(Xn1nk) , Xn
| Xn1nk}
and
b(Xn11 ) bk(Xn1nk) = argmaxb()
E{ln
b(Xn1nk) , Xn
| Xn1nk}
because of stationarity
bk(xk1 ) = argmax
b()E{lnb(x
k1 ) , Xk+1 | X
k1 = x
k1 }
= argmaxb
E{ln b , Xk+1 | Xk1 = x
k1 },
which is the maximization of the regression function
mb(xk1 ) = E{ln b , Xk+1 | X
k1 = x
k1 }
Gyorfi Machine learning and portfolio selections. II.
Regression function
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
36/103
Y real valuedX observation vector
Gyorfi Machine learning and portfolio selections. II.
Regression function
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
37/103
Y real valuedX observation vectorRegression function
m(x) = E{Y | X = x}
i.i.d. data: Dn = {(X1, Y1), . . . , (Xn, Yn)}
Gyorfi Machine learning and portfolio selections. II.
Regression function
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
38/103
Y real valuedX observation vectorRegression function
m(x) = E{Y | X = x}
i.i.d. data: Dn = {(X1, Y1), . . . , (Xn, Yn)}Regression function estimate
mn(x) = mn(x, Dn)
Gyorfi Machine learning and portfolio selections. II.
Regression function
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
39/103
Y real valuedX observation vectorRegression function
m(x) = E{Y | X = x}
i.i.d. data: Dn = {(X1, Y1), . . . , (Xn, Yn)}Regression function estimate
mn(x) = mn(x, Dn)
local averaging estimates
mn(x) =
ni=1
Wni(x; X1, . . . , Xn)Yi
Gyorfi Machine learning and portfolio selections. II.
Regression function
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
40/103
Y real valuedX observation vectorRegression function
m(x) = E{Y | X = x}
i.i.d. data: Dn = {(X1, Y1), . . . , (Xn, Yn)}Regression function estimate
mn(x) = mn(x, Dn)
local averaging estimates
mn(x) =
ni=1
Wni(x; X1, . . . , Xn)Yi
L. Gyorfi, M. Kohler, A. Krzyzak, H. Walk (2002) ADistribution-Free Theory of Nonparametric Regression,Springer-Verlag, New York.
Gyorfi Machine learning and portfolio selections. II.
Correspondence
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
41/103
X Xk1
Gyorfi Machine learning and portfolio selections. II.
Correspondence
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
42/103
X Xk1
Y ln b , Xk+1
Gyorfi Machine learning and portfolio selections. II.
Correspondence
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
43/103
X Xk1
Y ln b , Xk+1m(x) = E{Y | X = x} mb(x
k1 ) = E{ln b , Xk+1 | X
k1 = x
k1 }
Gyorfi Machine learning and portfolio selections. II.
Partitioning regression estimate
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
44/103
Partition Pn = {An,1, An,2 . . . }
Gyorfi Machine learning and portfolio selections. II.
Partitioning regression estimate
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
45/103
Partition Pn = {An,1, An,2 . . . }An(x) is the cell of the partition Pn into which x falls
Gyorfi Machine learning and portfolio selections. II.
Partitioning regression estimate
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
46/103
Partition Pn = {An,1, An,2 . . . }An(x) is the cell of the partition Pn into which x falls
mn(x) =
ni=1 YiI[XiAn(x)]ni=1 I[XiAn(x)]
Gyorfi Machine learning and portfolio selections. II.
Partitioning regression estimate
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
47/103
Partition Pn = {An,1, An,2 . . . }An(x) is the cell of the partition Pn into which x falls
mn(x) =
ni=1 YiI[XiAn(x)]ni=1 I[XiAn(x)]
Let Gn be the quantizer corresponding to the partition Pn:Gn(x) = j if x An,j.
Gyorfi Machine learning and portfolio selections. II.
Partitioning regression estimate
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
48/103
Partition Pn = {An,1, An,2 . . . }An(x) is the cell of the partition Pn into which x falls
mn(x) =
ni=1 YiI[XiAn(x)]ni=1 I[XiAn(x)]
Let Gn be the quantizer corresponding to the partition Pn:Gn(x) = j if x An,j.the set of matches
In = {i n : Gn(x) = Gn(Xi)}
Then
mn(x) =
iIn
Yi
|In|.
Gyorfi Machine learning and portfolio selections. II.
Partitioning-based portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
49/103
fix k, = 1, 2, . . .P = {A,j,j = 1, 2, . . . , m} finite partitions ofRd,
Gyorfi Machine learning and portfolio selections. II.
Partitioning-based portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
50/103
fix k, = 1, 2, . . .P = {A,j,j = 1, 2, . . . , m} finite partitions ofRd,G be the corresponding quantizer: G(x) = j, if x A,j.
Gyorfi Machine learning and portfolio selections. II.
Partitioning-based portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
51/103
fix k, = 1, 2, . . .
P = {A,j,j = 1, 2, . . . , m} finite partitions ofRd,G be the corresponding quantizer: G(x) = j, if x A,j.G(x
n1) = G(x1), . . . , G(xn),
Gyorfi Machine learning and portfolio selections. II.
Partitioning-based portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
52/103
fix k, = 1, 2, . . .
P = {A,j,j = 1, 2, . . . , m} finite partitions ofRd,G be the corresponding quantizer: G(x) = j, if x A,j.G(x
n1) = G(x1), . . . , G(xn),
the set of matches:
Jn = {k < i < n : G(xi1ik) = G(xn1nk)}
Gyorfi Machine learning and portfolio selections. II.
Partitioning-based portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
53/103
fix k, = 1, 2, . . .
P = {A,j,j = 1, 2, . . . , m} finite partitions ofRd,G be the corresponding quantizer: G(x) = j, if x A,j.G(x
n1) = G(x1), . . . , G(xn),
the set of matches:
Jn = {k < i < n : G(xi1ik) = G(xn1nk)}
b(k,)(xn11 ) = argmaxb iJn
ln b , xi
if the set In is non-void, and b0 = (1/d, . . . , 1/d) otherwise.
Gyorfi Machine learning and portfolio selections. II.
Elementary portfolios
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
54/103
for fixed k, = 1, 2, . . .,B(k,) = {b(k,)()}, are called elementary portfolios
Gyorfi Machine learning and portfolio selections. II.
Elementary portfolios
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
55/103
for fixed k, = 1, 2, . . .,B(k,) = {b(k,)()}, are called elementary portfolios
That is, b(k,)n quantizes the sequence x
n11 according to the
partition P, and browses through all past appearances of the lastseen quantized string G(x
n1nk) of length k.
Then it designs a fixed portfolio vector according to the returns onthe days following the occurence of the string.
Gyorfi Machine learning and portfolio selections. II.
Combining elementary portfolios
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
56/103
How to choose k,
small k or small : large bias
large k and large : few matching, large variance
Gyorfi Machine learning and portfolio selections. II.
Combining elementary portfolios
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
57/103
How to choose k,
small k or small : large bias
large k and large : few matching, large variance
Machine learning: combination of experts
N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games.Cambridge University Press, 2006.
Gyorfi Machine learning and portfolio selections. II.
Exponential weighing
combine the elementary portfolio strategies B(k,) = {b(k,)n }
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
58/103
y p g { n }
Gyorfi Machine learning and portfolio selections. II.
Exponential weighing
combine the elementary portfolio strategies B(k,) = {b(k,)n }
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
59/103
y p g { n }let {qk,} be a probability distribution on the set of all pairs (k, )such that for all k, , qk, > 0.
Gyorfi Machine learning and portfolio selections. II.
Exponential weighing
combine the elementary portfolio strategies B(k,) = {b(k,)n }
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
60/103
y p g { }let {qk,} be a probability distribution on the set of all pairs (k, )such that for all k, , qk, > 0.
for > 0 putwn,k, = qk,e
lnSn1(B(k,))
Gyorfi Machine learning and portfolio selections. II.
Exponential weighing
combine the elementary portfolio strategies B(k,) = {b(k,)n }
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
61/103
g { }let {qk,} be a probability distribution on the set of all pairs (k, )such that for all k, , qk, > 0.
for > 0 putwn,k, = qk,e
lnSn1(B(k,))
for = 1,
wn,k, = qk,elnSn
1(B(k,)
) = qk,Sn1(B(k,
))
Gyorfi Machine learning and portfolio selections. II.
Exponential weighingcombine the elementary portfolio strategies B(k,) = {b
(k,)n }
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
62/103
let {qk,} be a probability distribution on the set of all pairs (k, )such that for all k, , qk, > 0.
for > 0 putwn,k, = qk,e
lnSn1(B(k,))
for = 1,
wn,k, = qk,elnSn
1(B(k,)
) = qk,Sn1(B(k,
))
andvn,k, =
wn,k,
i,jwn,i,j
.
Gyorfi Machine learning and portfolio selections. II.
Exponential weighingcombine the elementary portfolio strategies B(k,) = {b
(k,)n }
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
63/103
let {qk,} be a probability distribution on the set of all pairs (k, )such that for all k, , qk, > 0.
for > 0 putwn,k, = qk,e
lnSn1(B(k,))
for = 1,
wn,k, = qk,elnS
n
1(B(k,))
= qk,Sn1(B(k,)
)
andvn,k, =
wn,k,
i,jwn,i,j
.
the combined portfolio b:
bn(xn11 ) =
k,
vn,k,b(k,)n (x
n11 ).
Gyorfi Machine learning and portfolio selections. II.
n
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
64/103
Sn(B) =n
i=1bi(x
i11 ) , xi
Gyorfi Machine learning and portfolio selections. II.
n
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
65/103
Sn(B) =n
i=1bi(x
i11 ) , xi
=n
i=1
k, wi,k,
b
(k,)i (x
i11 ) , xi
k, wi,k,
Gyorfi Machine learning and portfolio selections. II.
n
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
66/103
Sn(B) =n
i=1bi(x
i11 ) , xi
=n
i=1
k, wi,k,
b
(k,)i (x
i11 ) , xi
k, wi,k,
=
ni=1
k, qk,Si1(B(k,))b(k,)i (xi11 ) , xik, qk,Si1(B
(k,))
Gyorfi Machine learning and portfolio selections. II.
n
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
67/103
Sn(B) =n
i=1bi(x
i11 ) , xi
=n
i=1
k, wi,k,
b
(k,)i (x
i11 ) , xi
k, wi,k,
=
ni=1
k, qk,Si1(B(k,))b(k,)i (xi11 ) , xik, qk,Si1(B
(k,))
=n
i=1
k, qk,Si(B(k,))
k, qk,Si1(B(k,))
Gyorfi Machine learning and portfolio selections. II.
n
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
68/103
Sn(B) =n
i=1bi(x
i11 ) , xi
=n
i=1
k, wi,k,
b
(k,)i (x
i11 ) , xi
k, wi,k,
=
ni=1
k, qk,Si1(B(k,))b(k,)i (xi11 ) , xik, qk,Si1(B
(k,))
=n
i=1
k, qk,Si(B(k,))
k, qk,Si1(B(k,))
=k,
qk,Sn(B(k,)),
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
69/103
The strategy B = BH then arises from weighing the elementary
portfolio strategies B(k,) = {b(k,)n } such that the investors capital
becomesSn(B) =
k,
qk,Sn(B(k,)).
Gyorfi Machine learning and portfolio selections. II.
Theorem
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
70/103
Assume that
(a) the sequence of partitions is nested, that is, any cell of P+1 isa subset of a cell of P, = 1, 2, . . .;
(b) ifdiam(A) = supx,yA x y denotes the diameter of a set,then for any sphere S centered at the origin
lim
maxj:A,jS=
diam(A,j) = 0 .
Then the portfolio scheme BH defined above is universallyconsistent with respect to the class of all ergodic processes suchthat E{| ln X(j)| < , for j = 1, 2, . . . , d.
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
71/103
L. Gyorfi, D. Schafer (2003) Nonparametric prediction, inAdvances in Learning Theory: Methods, Models and Applications,J. A. K. Suykens, G. Horvath, S. Basu, C. Micchelli, J. Vandevalle(Eds.), IOS Press, NATO Science Series, pp. 341-356.
www.szit.bme.hu/gyorfi/histog.ps
Gyorfi Machine learning and portfolio selections. II.
ProofWe have to prove that
1
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
72/103
lim infn
Wn(B) = lim infn
1
nln Sn(B) W
a.s.
W.l.o.g. we may assume S0 = 1, so that
Wn(B) =1
nln Sn(B)
Gyorfi Machine learning and portfolio selections. II.
ProofWe have to prove that
1
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
73/103
lim infn
Wn(B) = lim infn
1
nln Sn(B) W
a.s.
W.l.o.g. we may assume S0 = 1, so that
Wn(B) =1
nln Sn(B)
=
1
n ln
k,qk,Sn(B
(k,)
)
Gyorfi Machine learning and portfolio selections. II.
ProofWe have to prove that
1
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
74/103
lim infn
Wn(B) = lim infn
1
nln Sn(B) W
a.s.
W.l.o.g. we may assume S0 = 1, so that
Wn(B) =1
nln Sn(B)
=
1
n ln
k,qk,Sn(B
(k,)
)
1
nln
supk,
qk,Sn(B(k,))
Gyorfi Machine learning and portfolio selections. II.
ProofWe have to prove that
1
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
75/103
lim infn
Wn(B) = lim infn
1
nln Sn(B) W
a.s.
W.l.o.g. we may assume S0 = 1, so that
Wn(B) =1
nln Sn(B)
=
1
n ln
k,qk,Sn(B
(k,)
)
1
nln
supk,
qk,Sn(B(k,))
= 1n
supk,
ln qk, + ln Sn(B(k,))
Gyorfi Machine learning and portfolio selections. II.
ProofWe have to prove that
1
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
76/103
lim infn
Wn(B) = lim infn
1
nln Sn(B) W
a.s.
W.l.o.g. we may assume S0 = 1, so that
Wn(B) =1
nln Sn(B)
=
1
n ln
k,qk,Sn(B
(k,)
)
1
nln
supk,
qk,Sn(B(k,))
= 1n
supk,
ln qk, + ln Sn(B(k,))
= sup
k,
Wn(B
(k,)) +ln qk,
n
.
Gyorfi Machine learning and portfolio selections. II.
Thus
l
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
77/103
lim infn
Wn(B) lim infn
sup
k,Wn(B(k,)) +
ln qk,n
Gyorfi Machine learning and portfolio selections. II.
Thus
l
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
78/103
lim infn
Wn(B) lim infn
sup
k,Wn(B(k,)) +
ln qk,n
supk,
lim infn
Wn(B
(k,)) +ln qk,
n
Gyorfi Machine learning and portfolio selections. II.
Thus
ln q
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
79/103
lim infn
Wn(B) lim infn
sup
k,Wn(B(k,)) +
ln qk,n
supk,
lim infn
Wn(B
(k,)) +ln qk,
n
= supk,
lim infn
Wn(B(k,))
Gyorfi Machine learning and portfolio selections. II.
Thus
ln q
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
80/103
lim infn
Wn(B) lim infn
supk,Wn(B(k,)) +
ln qk,n
supk,
lim infn
Wn(B
(k,)) +ln qk,
n
= supk,
lim infn
Wn(B(k,))
= supk,
k,
Since the partitions P are nested, we have that
supk,
k,
= limk
liml
k,
= W.
Gyorfi Machine learning and portfolio selections. II.
Kernel regression estimate
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
81/103
Kernel function K(x) 0Bandwidth h > 0
mn(x) =
ni=1 YiK
xXih
ni
=1
K xXih
Naive (window) kernel function K(x) = I{x1}
mn(x) = ni=1 YiI{xXih}
ni=1 I{xXih}
Gyorfi Machine learning and portfolio selections. II.
Kernel-based portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
82/103
choose the radius rk, > 0 such that for any fixed k,
lim
rk, = 0.
Gyorfi Machine learning and portfolio selections. II.
Kernel-based portfolio selection
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
83/103
choose the radius rk, > 0 such that for any fixed k,
lim
rk, = 0.
for n > k + 1, define the expert b(k,) by
b(k,)(xn11 ) = argmaxb
{k
-
8/9/2019 Machine Learning and Finance II
84/103
The kernel-based portfolio scheme is universally consistent withrespect to the class of all ergodic processes such thatE{| ln X(j)| < , for j = 1, 2, . . . , d.
L. Gyorfi, G. Lugosi, F. Udina (2006) Nonparametric kernel-basedsequential investment strategies, Mathematical Finance, 16, pp.337-357www.szit.bme.hu/gyorfi/kernel.pdf
Gyorfi Machine learning and portfolio selections. II.
k-nearest neighbor (NN) regression estimate
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
85/103
mn(x) =n
i=1Wni(x; X1, . . . , Xn)Yi.
Wni is 1/k if Xi is one of the k nearest neighbors of x amongX1, . . . , Xn, and Wni is 0 otherwise.
Gyorfi Machine learning and portfolio selections. II.
Nearest-neighbor-based portfolio selection
choose p (0 1) such that lim p = 0
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
86/103
choose p (0, 1) such that lim p 0for fixed positive integers k, (n > k + + 1) introduce the set ofthe = pn nearest neighbor matches:
J(k,)n = {i; k + 1 i n such that x
i1ik is among the NNs of x
n1nk
in xk1 , . . . , xn1nk}.
Gyorfi Machine learning and portfolio selections. II.
Nearest-neighbor-based portfolio selection
choose p (0, 1) such that lim p = 0
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
87/103
choose p (0, 1) such that lim p 0for fixed positive integers k, (n > k + + 1) introduce the set ofthe = pn nearest neighbor matches:
J(k,)n = {i; k + 1 i n such that x
i1ik is among the NNs of x
n1nk
in xk1 , . . . , xn1nk}.
Define the portfolio vector by
b(k,)(xn11 ) = argmaxb
iJ
(k,)n
ln b , xi
if the sum is non-void, and b0 = (1/d, . . . , 1/d) otherwise.
Gyorfi Machine learning and portfolio selections II
Theorem
If for any vector s = sk1 the random variable
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
88/103
y 1
Xk1 s
has continuous distribution function, then the nearest-neighborportfolio scheme is universally consistent with respect to the classof all ergodic processes such that E{| ln X(j)|} < , for
j = 1, 2, . . . d.
NN is robust, there is no scaling problem
Gyorfi Machine learning and portfolio selections II
Theorem
If for any vector s = sk1 the random variable
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
89/103
y 1
Xk1 s
has continuous distribution function, then the nearest-neighborportfolio scheme is universally consistent with respect to the classof all ergodic processes such that E{| ln X(j)|} < , for
j = 1, 2, . . . d.
NN is robust, there is no scaling problem
L. Gyorfi, F. Udina, H. Walk (2006) Nonparametric nearest
neighbor based empirical portfolio selection strategies,(submitted), www.szit.bme.hu/gyorfi/NN.pdf
Gyorfi Machine learning and portfolio selections II
Semi-log-optimal portfolio
empirical log-optimal:
(k ) 1
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
90/103
h(k,)(xn11 ) = argmax
biJn
ln b , xi
Gyorfi Machine learning and portfolio selections II
Semi-log-optimal portfolio
empirical log-optimal:
(k ) 1
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
91/103
h(k,)(xn11 ) = argmax
biJn
ln b , xi
Taylor expansion: ln z h(z) = z 1 12 (z 1)2
Gyorfi Machine learning and portfolio selections II
Semi-log-optimal portfolio
empirical log-optimal:
(k )( n 1)
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
92/103
h(k,)(xn11 ) = argmax
biJn
ln b , xi
Taylor expansion: ln z h(z) = z 1 12 (z 1)2 empirical
semi-log-optimal:
h(k,)(xn11 ) = argmaxb
iJn
h(b , xi) = argmaxb
{b , mb , Cb}
Gyorfi Machine learning and portfolio selections II
Semi-log-optimal portfolio
empirical log-optimal:
h(k )( n 1) l b
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
93/103
h(k,)(xn11 ) = argmax
biJn
ln b , xi
Taylor expansion: ln z h(z) = z 1 12 (z 1)2 empirical
semi-log-optimal:
h(k,)(xn11 ) = argmaxb
iJn
h(b , xi) = argmaxb
{b , mb , Cb}
smaller computational complexity: quadratic programming
Gyorfi Machine learning and portfolio selections II
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
94/103
Conditions of the model:
-
8/9/2019 Machine Learning and Finance II
95/103
Assume that
the assets are arbitrarily divisible,
the assets are available in unbounded quantities at the currentprice at any given trading period,
there are no transaction costs,
the behavior of the market is not affected by the actions ofthe investor using the strategy under investigation.
Gyorfi Machine learning and portfolio selections II
NYSE data sets
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
96/103
At www.szit.bme.hu/~oti/portfolio there are two benchmarkdata set from NYSE:
The first data set consists of daily data of 36 stocks withlength 22 years.
The second data set contains 23 stocks and has length 44years.
Gyorfi Machine learning and portfolio selections II
NYSEdata sets
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
97/103
At www.szit.bme.hu/~oti/portfolio there are two benchmarkdata set from NYSE:
The first data set consists of daily data of 36 stocks withlength 22 years.
The second data set contains 23 stocks and has length 44years.
Our experiment is on the second data set.
Gyorfi Machine learning and portfolio selections II
Experiments on average annual yields (AAY)
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
98/103
Kernel based semi-log-optimal portfolio selection withk = 1, . . . , 5 and l = 1, . . . , 10
r2k,l = 0.0001 d k ,
Gyorfi Machine learning and portfolio selections. II.
Experiments on average annual yields (AAY)
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
99/103
Kernel based semi-log-optimal portfolio selection withk = 1, . . . , 5 and l = 1, . . . , 10
r2k,l = 0.0001 d k ,
AAY of kernel based semi-log-optimal portfolio is 222.7%
Gyorfi Machine learning and portfolio selections. II.
Experiments on average annual yields (AAY)
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
100/103
Kernel based semi-log-optimal portfolio selection withk = 1, . . . , 5 and l = 1, . . . , 10
r2k,l = 0.0001 d k ,
AAY of kernel based semi-log-optimal portfolio is 222.7%double the capital
Gyorfi Machine learning and portfolio selections. II.
Experiments on average annual yields (AAY)
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
101/103
Kernel based semi-log-optimal portfolio selection withk = 1, . . . , 5 and l = 1, . . . , 10
r2k,l = 0.0001 d k ,
AAY of kernel based semi-log-optimal portfolio is 222.7%double the capitalMORRIS had the best AAY, 30.1%
Gyorfi Machine learning and portfolio selections. II.
Experiments on average annual yields (AAY)
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
102/103
Kernel based semi-log-optimal portfolio selection withk = 1, . . . , 5 and l = 1, . . . , 10
r2k,l = 0.0001 d k ,
AAY of kernel based semi-log-optimal portfolio is 222.7%double the capitalMORRIS had the best AAY, 30.1%the BCRP had average AAY 35.2%
Gyorfi Machine learning and portfolio selections. II.
The average annual yields of the individual experts.
k 1 2 3 4 5
http://find/http://goback/ -
8/9/2019 Machine Learning and Finance II
103/103
1 29.4% 27.8% 22.9% 23.9% 23.7%
2 201.0% 124.6% 98.3% 36.1% 90.8%
3 114.2% 62.9% 38.8% 91.4% 31.6%
4 172.5% 155.0% 100.6% 162.3% 50.8%
5 233.5% 170.2% 166.6% 171.1% 107.7%6 245.4% 216.2% 176.9% 182.0% 143.0%
7 261.8% 211.0% 181.2% 165.6% 158.7%
8 229.4% 189.2% 171.0% 138.8% 131.3%
9 219.3% 172.8% 162.4% 118.7% 116.0%
10 210.6% 151.5% 131.8% 103.4% 110.9%
Gyorfi Machine learning and portfolio selections. II.
http://find/http://goback/