f 2 2 F to o the best t possible measurable function, , but t it has nothingto dowith h the
geometricstructureofF. Themarginisdeterminedforeveryfseparately,because
f
doesnotdependonthechoiceofF atall.
Inthesecond,\learningtheory"setup,wedonotassumethatthetargetfunction
f
belongs to F. The e aimis to o construct a function
b
f whose e risk is s as close as
possibletothatofthebestelementf
F
2F. And,assumingthattheexcesslossclass
L
F
satises theBernsteincondition(BC for short) one canimprove the errorrate
(see,e.g.,[20,5]).
Atarstglance,MAandBC(for=1)shareverystrongsimilarities. Indeed,
sayingthatL
F
isa(1;B)-Bernsteinclassmeansthatforeveryf2F
E
(Q(Z;f) Q(Z;f
F
))
2
B(R(f) R(f
F
));
butnevertheless,theyaredierent.Indeed,aswementioned,MAisonlyamatterof
concentration(andclassicalstatisticsquestionsaremostlyaquestionofthetradeo
between concentrationandcomplexity). On n theother r hand, , BC involves alot t of
geometryofthefunctionclassF,becausef
F
mightchangesignicantlybyaddinga
singlefunctiontoF orbyremovingone. Infact,thedicultyof\learningtheory"
problemsis determinedby thetradeo betweenconcentrationandcomplexity, , and
thegeometryofthegivenclass,sinceonemeasurestheperformanceofthelearning
algorithmrelativetothebestintheclass. Assumingthatf
2F,asisusuallydone
inclassicalstatistics,exemptsonefromtheneedtoconsiderthegeometryofF,but
wedonothavethatfreedomintheaggregationframework.Indeed,sinceintheAEW
algorithmtheestimatorisdeterminedbytheempiricalmeansR
n
(f) R
n
(f
F
),itis
alearningproblemratherthanaprobleminclassicalstatistics(despitethefactthat
ithasbeenusedinstatisticalframeworkstoconstructadaptiveestimators,see,for
example,[4,11,14,23,6,18,25,2,28]).Therefore,becauseofitsnature,aggregation
proceduresliketheAEWaremorenaturalunderaBCassumptionandnottheMA
one(aby-product of TheoremAis thatthe MAcannot improve the performance
of AEWsince inTheoremA’ssetupMAis satisedwiththebestpossiblemargin
parameter=1).
3 Preliminaryresultsongaussianapproximation
Our starting point is the e Berry-Esseen Theorem on gaussian approximation. Let
(W
n
)
n2N
beasequenceofi.i.d.,meanzerorandomvariableswithvariance1,setgto
beastandardGaussianvariableandput
X
n
=
1
p
n
Xn
i=1
W
i
:
11
Convert pdf to html code online - SDK application service:C# PDF Convert to HTML SDK: Convert PDF to html files in C#.net, ASP.NET MVC, WinForms, WPF application
How to Convert PDF to HTML Webpage with C# PDF Conversion SDK
www.rasteredge.com
Convert pdf to html code online - SDK application service:VB.NET PDF Convert to HTML SDK: Convert PDF to html files in vb.net, ASP.NET MVC, WinForms, WPF application
PDF to HTML Webpage Converter SDK for VB.NET PDF to HTML Conversion
www.rasteredge.com
Theorem3.1 ([21])There exists s an absolute constant A > 0 0 such that for every
integern,
sup
x2R
P[
X
n
x] P[gx]
AEjW
1
j
3
p
n
:
FromhereonwewilldenotebyAtheconstantappearinginTheorem3.1.
Whenthetailbehavior oftheW
i
hasasub-exponentialdecay,thegaussianap-
proximationcanbeimproved. Indeed,recallthatareal-valuedrandomvariableW
belongstoL
forsome1ifthereexists0<c<1suchthat
Eexp
jWj
=c
2:
(3.1)
Theinmumoverallconstantscforwhich(3.1)holdsdenesanOrlicznorm,which
iscalledthe 
normandisdenotedbykk
.FormorefactsonOrlicznormssee,
forinstance,[26]and[22].
Proposition3.2(Chapter5in[21]) ForeveryL>0thereexistconstantsB
0
;c
1
andc
2
thatdependonlyonLforwhichthefollowingholds. IfkWk
1
Lthenfor
anyx0suchthatxB
0
n
1=6
,
P[
X
n
x]=P[gx]exp
x
3
EW
3
6
p
n

1+O
x+1
p
n

and
P[
X
n
 x]=P[g x]exp
x
3
EW
3
6
p
n

1+O
x+1
p
n

;
wherebyv=O(u)wemeanthat c
1
uvc
1
u.
Inparticular,ifjxjB
0
n
1=6
andEW
3
=0then
jP[
X
n
x] P[gx]jc
2
(n
1=2
exp( x
2
=2)):
FromhereonwewilldenotebyB
0
theconstantappearinginProposition3.2.
4 Proof ofTheoremA
Before presenting the proof ofTheoremA,letus introduce the followingnotation.
Givenaprobabilitymeasureand(Z
i
)
n
i=1
selectedindependentlyaccordingto,we
setP
n
=n
1
P
n
i=1
Z
i
theempiricalmeasuresupportedon(Z
i
)
n
i=1
. WedenotebyP
theexpectationE
.Fromhereon,wewillassumethatT1andrecallthatnisan
oddinteger.
LetY =0anddeneXbyP[X=1]=1=2 n
1=2
andP[X= 1]=1=2+n
1=2
.
Letf
1
=1I
[0;1]
andf
2
=1I
[ 1;0]
,andconsiderthedictionary F=ff
1
;f
2
g. Itiseasy
12
SDK application service:Online Convert PDF to HTML5 files. Best free online PDF html
it extremely easy for C# developers to convert and transform a save each PDF page as a separate HTML file in Download and try RasterEdge.XDoc.PDF for .NET with
www.rasteredge.com
SDK application service:VB.NET PDF Convert to Tiff SDK: Convert PDF to tiff images in vb.
Resize converted Tiff image using VB.NET. Convert PDF file to Tiff and jpeg in ASPX webpage online. Online source code for VB.NET class.
www.rasteredge.com
toverifythatthebestfunctioninF (theoracle)withrespecttothequadraticrisk
isf
1
andthattheexcesslossfunctionoff
2
,L
2
=f
2
2
f
2
1
=f
2
f
1
,satisesthat
L
2
(X)= X; EL
2
(X)=2n
1=2
and
2
=E
L
2
(X) EL
2
(X)
2
=1 4=n:
Toshortennotation,wedenePL
2
=EL
2
(X)andP
n
L
2
=n
1
P
n
i=1
L
2
(X
i
).
An important parameter r which h is at the heart of this s counter-example is the
Bernsteinconstant(whichisverybadinthiscase):
=
E(f
1
f
2
)
2
PL
2
=
p
n
2
(4.1)
AstraightforwardcomputationshowsthatAEWonF withtemperatureT isgiven
by
~
f
AEW
=
b
1
f
1
+(1 
b
1
)f
2
;
b
1
=
1
1+exp
n
T
P
n
L
2
;
andthat,forh()=+(1 )denedforall2[0;1],wehave
E[R(
~
f
AEW
) R(f
1
)]=E
h
b
1
b
1
(1 
b
1
)
i
PL
2
=E
1 h(
b
1
)
PL
2
=E
h
Z
1
0
h
0
(t)P[
b
1
t]dt
i
PL
2
=
1+
Z
1
0
(2t (1+))P[
b
1
t]dt
PL
2
=
1+
Z
1
0
(2t (1+))P[P
n
L
2
 (t)]dt
PL
2
;
(4.2)
where (t)isanincreasingfunctiondenedforanyt2(0;1)by
(t)=
T
n
log
t
1 t
:
Inparticular,
E
h
R(
~
f
AEW
) R(f
1
)
i
=[I
1
+I
2
]PL
2
;
for
I
1
=
Z
1
0
(2t (1+))P[P
n
L
2
 (t)]dt+1
and
I
2
=
Z
1
 1
(2t (1+))P[P
n
L
2
 (t)]dt:
First,letus boundI
1
frombelow. Tothatendoneshouldnoticethefollowing
facts.First,thatforevery0t
1
,1+ 2t0and
Z
1
0
(2t (1+))dt= 1:
13
SDK application service:VB.NET PDF Convert to Word SDK: Convert PDF to Word library in vb.
VB.NET class source code for .NET framework. This VB.NET PDF to Word converter control is a and mature .NET solution which aims to convert PDF document to Word
www.rasteredge.com
SDK application service:C# PDF Convert to Jpeg SDK: Convert PDF to JPEG images in C#.net
in .NET developing platforms using simple C# programming code. Using this C#.NET PDF to JPEG conversion C# developers can easily and quickly convert a large
www.rasteredge.com
Second,ifwesetE=exp(nPL
2
=T),thenforT .
p
n=logn,0<(1+E)
1

1
.In
particular,thisholdsunderourassumptionthatT1.Also,because isincreasing
thenfor(1+E)
1
t
1
, (t)
(1+E)
1
= PL
2
.Therefore,
I
1
=
Z
1
0
(2t (1+))P[P
n
L
2
 (t)]dt+1
=
Z
1
0
(2t (1+))(P[P
n
L
2
 (t)] 1)dt
Z
1
(1+E) 1
(1+ 2t)P[P
n
L
2
< (t)]dt
Z
 1
(1+E) 1
(1+ 2t)dtP
(
p
n=)(P
n
L
2
PL
2
)<(
p
n=)( 2PL
2
)
Z
1
(1+E) 1
(1+ 2t)dt
P[g 8] A=
p
n
c
0
>0;
whereinthe laststepwe usedtheBerry-EsseenTheorem,that jL
2
j 1andthat
n8_(2A=P[g 8])
2
,implyingthat0<c
0
<1=2.
LetusturntoalowerboundforI
2
.Applyingachangeofvariablest7!1+
1
u
inthesecondtermofI
2
,itisevidentthat
I
2
=
Z
+1
2
 1
(2t (1+))P[P
n
L
2
 (t)]dt+
Z
1
+1
2
(2t (1+))P[P
n
L
2
 (t)]dt
=
Z
+1
2
 1
(2t (1+))P
(t)P
n
L
2
<
1+
1
t

dt=I
3
+I
4
for
I
3
=
Z
(1+c
0
=4) 1
 1
(2t (1+))P
(t)P
n
L
2
<
1+
1
t

dt
and
I
4
=
Z
+1
2
(1+c
0
=4)
1
(2t (1+))P
(t)P
n
L
2
<
1+
1
t

dt:
ToestimateI
3
,notethat2t (1+)0fort2[
1
;(+1)=(2)]andthus
I
3
Z
(1+c
0
=4)
1
 1
(2t (1+))dt
c
0
4
1+
1
 
c
0
3
;
14
SDK application service:C# PDF Convert to SVG SDK: Convert PDF to SVG files in C#.net, ASP
PDFDocument pdf = new PDFDocument(@"C:\input.pdf"); pdf.ConvertToVectorImages( ContextType.SVG, @"C:\demoOutput Description: Convert to html/svg files and
www.rasteredge.com
SDK application service:C# PDF Convert to Images SDK: Convert PDF to png, gif images in C#
Online C# Demo Codes for Converting PDF to Raster Images, .NET Graphics, and REImage in C#.NET Project. You can use this sample code to convert PDF file to
www.rasteredge.com
forourchoiceof.
ThenalstepoftheproofistoboundI
4
,andinparticulartoshowthatforsmall
valuesofT,I
4
 c
0
=3.
Forany0<t(+1)=(2),considertheintervalsI
T
(t)=
n (t);n (1+
1
t)
,
and set N
T
(t) = jfI
T
(t)\Zgj, which h is the number of integers in I
T
(t). Since
L
2
(X)= Xthen
P
(t)P
n
L
2
<
1+
1
t

=P
"
Xn
i=1
X
i
2I
T
(t)
#
=P
T
(t):
Recall that X 2 f 1;1g and thus P[
P
i
X
i
2I
T
(t)] = P[
P
i
X
i
2I
T
(t)\Z].
Sincen (t)isincreasingandnonnegativefort>1=2thenif1=2<t(+1)=(2)
it follows that0 <n (t)< n (1+1= t) ) <1, , providedthatT 1. Thus,for
suchvaluesoft,N
T
(t)=0,implyingthatP
T
(t)=0.Ontheotherhand,ift1=2,
thenf0gI
T
(t)\Z. Inparticular,ifN
T
(t)=1thenI
T
(t)\Z=f0gandsincen
isoddthenP
T
(t)=P[
P
n
i=1
X
i
=0]=0. Otherwise,N
T
(t)2whichimpliesthat
N
T
(t)2
T
(t)where
T
(t)isthelengthofI
T
(t),givenby
T
(t)=n( (1+
1
t)  (t))=Tlog
(1 t)(+1 t)
t(t 1)
:
Therefore,foreverytinourrange,
P
T
(t)N
T
(t) max
k2I
T
(t)
P
"
Xn
i=1
X
i
=k
#
2
T
(t)max
k2Z
P
"
Xn
i=1
X
i
=k
#
:
Since2t (1+)0forevery0<t(+1)=(2)itisevidentthat
I
4
2Tmax
k2Z
P
"
n
X
i=1
X
i
=k
#
Z
+1
2
(1+c
0
=4) 1
(2t (1+))log
(1 t)(+1 t)
t(t 1)
dt:
Onemayshowthatmax
k2Z
P[
P
n
i=1
X
i
=k]isoftheorderofn
1=2
eitherbyadirect
computationor by the Berry-EsseenTheorem. Moreover,forany y (1+c
0
=4)
1
t(+1)=(2),onehast 1c
0
(4+c
0
)
1
t,andthus,
log
(1 t)(+1 t)
t(t 1)
log
2(4+c
0
)
c
0
t2
:
Therefore, combining the two observations s with h a change of variables u = Ctfor
C = (c
0
=(2(4+c
0
)))
1=2
, it is evident that there e are absolute constants c
1
;c
2
for
which
I
4
c
1
T
p
n
Z
C(+1)
2
C(1+c
0
=4) 1
1+ 2u=C
(logu)du c
2
T
p
n
:
15
SDK application service:VB.NET PDF File Split Library: Split, seperate PDF into multiple
This online VB tutorial aims to illustrate the process of PDF document splitting. VB.NET PDF Splitting & Disassembling DLLs. In order to run the sample code,
www.rasteredge.com
SDK application service:VB.NET PDF Convert to Images SDK: Convert PDF to png, gif images
VB.NET Sample Code for PDF to Png Conversion. This is a VB.NET programming sample code to convert PDF file to Png image. ' Load a PDF file.
www.rasteredge.com
Hence,thereisanabsoluteconstantc
3
suchthatifTc
3
thenI
4
 c
0
=3,implying
that
E
h
R(
~
f
AEW
) R(f
1
)
i
c
0
3
p
n
andprovingtherstpartofTheoremA.
Toprovethesecondpartoftheclaim,notethatbytheBerry-EsseenTheorem,
foreveryx2R,withprobabilitygreaterthanP[gx] 2A=
p
n
p
n
(L
2
)
(P
n
L
2
PL
2
)x:
Thus,ifn islargeenoughtoensurethatP[g 4] 2A=
p
nP[g 4]=2=c
4
andtakingx= 4,thenwithprobabilityatleastc
4
,P
n
L
2
 n
1=2
.Onthatevent
b
1
exp( 
p
n=T),whichyieldsthat
R(
~
f
AEW
) R(f
1
)=
b
1
b
1
(1 
b
1
)
PL
2
PL
2
=4=n
1=2
=2;
providedthatT.
p
n=logn.
5 Proof ofTheoremB
Therst stepintheproofofTheoremBis ageneralstatementaboutamonotone
rearrangementofindependentrandomvariablesthatareclosetobeinggaussian.
LetW beameanzero,varianceonerandomvariable,thatisabsolutelycontinuous
withrespecttotheLebesguemeasure. Assumefurtherthat t jWjhas anitethird
moment(infact,therandomvariableswewillbeinterestedinwillbebounded)and
set(W)=AEjWj
3
,whereAistheconstantappearingintheBerry-EsseenTheorem
(Theorem3.1). Let t W
1
;:::;W
n
beindependentrandomvariablesdistributedas W
andset
X =n
1=2
P
n
i=1
W
i
. Let t (
X
j
)
j=1
be ‘independent copies of
X,andput
1
=
1
(‘)2Rtosatisfythat
P
min
1j‘
X
j
1
(‘)
=1 
1
n
:
Note thatsuch a
1
existsbecause W has s adensity withrespect tothe Lebesgue
measure.
ThroughouttheproofofTheoremBwewillrequirethefollowingsimpleestimates
on
1
.
Lemma5.1 Thereexistabsoluteconstantsc
0
;:::;c
3
forwhichthefollowingholds.
16
1. If‘c
0
lognthen
1 c
1
logn
P[
X>
1
]1 
logn
:
2. If‘andnaresuchthat((W)=
p
n+(logn)=‘)<P[g< 2],then
1
 2.
3. If
1
 2andc
0
logn‘c
2
1
(W)
p
nlognthen
j
1
jlog
1=2
c
3
logn
and exp( 
2
1
=2)
logn
log
1=2
c
3
logn
:
BeforepresentingtheproofofLemma5.1,recallthatforeveryx2,
3
4
p
2
exp
x
2
=2
x
P[gx]
1
p
2
exp
x
2
=2
x
:
(5.1)
ProofofLemma5.1. Toprovetherstpart,notethatbyindependenceandsince
exp( x)1 x,
P[
X>
1
]=P[min
1j‘
X
j
>
1
]
1
=
1
n
1=‘
1 
logn
:
(5.2)
The reverse inequality follows in an identical fashion,since exp( x) )  1 x=3if
0x1.
Turningtothesecondpart,if
1
> 2then
1
n
=P[min
1j‘
X
j
 
1
]P[min
1j‘
X
j
 2]=1 (P[
X> 2])
;
implying that P[
X    2] ]  (logn)=‘. On n the other r hand, , by the Berry-Esseen
Theorem, P[
X   2]  P[g   2] (W)=
p
n, which h is impossible under the
assumptionsof(2).
Finally,toprove(3),oneusestheBerry-EsseenTheoremcombinedwiththelower
andupperestimatesontheGaussiantail(5.1)and(5.2). Thus,
3
4
p
2
1
j
1
j
exp
j
1
j
2
2
P[g<
1
]P[
X<
1
]+
(W)
p
n
(W)
p
n
+c
1
logn
;
and
1
p
2
1
j
1
j
exp
j
1
j
2
2
logn
(W)
p
n
:
fromwhichbothpartsofthethirdclaimfollow.
17
Proposition5.2 There existsconstantsc
1
;c
2
;c
3
andc
4
dependingonlyonkWk
2
forwhichthefollowingholds. Let2M
2
exp( c
1
n
1=3
)<1,assumethatEW
3
=0
andthat
1
=
1
(M 1) 2. . Then,
P
9j2f2;:::;Mg:
X
j
1
andforeveryk2f2;:::;Mgnfjg;
X
k
X
j

1 
1
n
c
2
1
p
n
+
(logn)
2
p
logM;
providedthatc
3
lognMc
4
p
n(logn).
Proof. Forevery2jM,let
j
=
X
j
1
and
X
k
X
j
foreveryk2f2;:::;Mgnfjg
 
:
Theevents
j
for2jMaredisjointsandthus
P
9j2f2;:::;Mg:
X
j
1
and
X
k
X
j
foreveryk2f2;:::;Mgnfjg
=P[[
M
j=2
j
]=(M 1)P[
2
]:
Sincethevariables(
X
j
)
M
j=2
areindependent,then
P[
2
]=
Z
1
1
f
X
(z)
Z
1
z+
f
X
(t)d(t)
M 2
d(z);
wheref
X
isadensityfunctionof
X withrespecttotheLebesguemeasure.
Ontheother hand,for any z 
1
,P[
X z]>0becauseof(5.2). Hence,for
everyz
1
,
Z
1
z+
f
X
(t)d(t)=
R
z+
z
f
X
(t)d(t)
R
1
z
f
X
(t)d(t)
!
Z
1
z
f
X
(t)d(t):
(5.3)
Notethatforevery0x1,(1 x)
M 2
1 (M 2)x,andappliedto(5.3),
P[
2
]
Z
1
1
f
X
(z)
Z
1
z
f
X
(t)d(t)
M 2
d(z)
(M 2)
Z
1
1
f
X
(z)
Z
1
z
f
X
(t)d(t)
M 3
Z
z+
z
f
X
(t)d(t)
d(z)
P
X
2
1
and
X
k
X
2
; foreveryk3
T
2
=
1
M 1
P
min
2jM
X
j
1
T
2
;
18
where
T
2
=(M 2)
Z
1
1
f
X
(z)
Z
z+
z
f
X
(t)d(t)
d(z):
Recalltheif(W
i
)areindependent,meanzerorandomvariablesthenk
P
a
i
W
i
k
2
c(
P
a
2
i
kW
i
k
2
2
)
1=2
where cis anabsolute constant[26]. Hence, , k
Xk
2
ckWk
2
,
andforanyt<0,
Z
t
1
f
X
(z)
Z
z+
z
f
X
(t)d(t)
d(z)P[
Xt]2exp( t
2
=c
2
kWk
2
2
):
Lett
0
<0besuchthat
2exp( t
2
0
=c
2
kWk
2
2
)=
p
log(M 1)
(M 1)(M 2)
:
Hence,
(M 2)
Z
t
0
1
f
X
(z)
Z
z+
z
f
X
(t)d(t)
d(z)
p
log(M 1)
M 1
:
Notethatift
0
1
thenourclaimfollows. Indeed,sinceP
min
2jM
X
j
1
1 n
1
,then
P[
0
]
1
M 1
1
n
p
log(M 1)
M 1
:
Otherwise,wesplittheinterval( 1;
1
]=( 1;t
0
)[[t
0
;
1
],andtoupperbound
T
2
itremainstocontroltheintegralonthesecondinterval[t
0
;
1
].
Recallthat W 2 2 L
1
andthat EW
3
= 0. Therefore, , by y Proposition 3.2, it t is
evidentthatifzandsatisfythatzz+0andjzj;jz+jB
0
n
1=6
,then
Z
z+
z
f
X
(t)d(t)=P[z
Xz+]
P[zgz+]+
B
1
p
n
exp
z
2
=2
;
(5.4)
whereB
0
andB
1
areconstantsthatdependonlyonkWk
1
.Also,foreveryz0,
P[zgz+]
1
p
2
exp
z
2
=2
Z
0
exp( zt)dt
p
2
exp
z
2
=2
: (5.5)
19
If2M
2
exp( B
2
0
n
1=3
=kWk
2
2
)< 1thenjt
0
jB
0
n
1=6
. Combining(5.4)and
(5.5)withthedenitionofT
2
,
(M 2)
Z
1
t
0
f
X
(z)
Z
z+
z
f
X
(t)d(t)
d(z)
(M 2)
B
1
p
n
+
p
2
Z
1
t
0
f
X
(z)exp
z
2
=2
d(z)
(M 2)
B
1
p
n
+
p
2
exp( 
2
1
=2)P[
X
1
]
(M 2)
B
1
p
n
+
p
2
exp( 
2
1
=2)
logn
M 1
;
wherethelastinequalityfollowsfrom(5.2).ByLemma5.1andsinceM.
p
nlogn,
(M 2)
Z
1
t
0
f
X
(z)
Z
z+
z
f
X
(t)d(t)
d(z)
c
1
p
n
+

logn
M
(logn)
p
logM
forsomeconstantc=c(),fromwhichourclaimfollows.
Next,letusdescribetheconstructionwe needfortheproofofTheoremB.Let
(X;Y)andF =ff
1
;:::;f
M
gbedenedby
Y =0;
f
1
(X)=(12)
1=4
U
1
;
f
j
(X)=(12)
1=4
(U
j
+) forevery2jM;
whereU
1
;:::;U
M
areMindependentrandomvariableswiththedensityu7 !2(u+
)1I
[ ;1 ]
(u)for0<<1=2tobexedlater. Notethatforthischoiceofdensity
function,(U
1
+)
2
isuniformlydistributedon[0;1]andthatthebestelementinF
withrespecttothequadraticriskisf
1
.
Let(U
(i)
j
:j=1;:::;M;i=1;:::;n)beafamilyofindependentrandomvariables
distributedasU
1
. Thus,for r every 1in, , f
j
(X
i
)=(12)
1=4
(U
(i)
j
+)forevery
2jM andf
1
(X
i
)=(12)
1=4
U
(i)
1
.Forevery1jMset
R
j
=
r
12
n
Xn
i=1
(U
(i)
j
+)
2
E(U
(i)
j
+)
2
!
;
andobservethatifW =
p
12
(U+)
2
E(U+)
2
thenWisameanzero,variance
1randomvariablethatisabsolutelycontinuouswithrespecttotheLebesguemeasure;
20
Documents you may be interested
Documents you may be interested