DeepLearningTutorial,Release0.1
Ast!1,samples(v
(t)
;h
(t)
)areguaranteedtobeaccuratesamplesofp(v;h).
Intheory,eachparameterupdateinthelearningprocesswouldrequirerunningonesuchchaintoconver-
gence.Itisneedlesstosaythatdoingsowouldbeprohibitivelyexpensive.Assuch,severalalgorithmshave
beendevisedforRBMs,inordertoefficientlysamplefromp(v;h)duringthelearningprocess.
9.3.1 ContrastiveDivergence(CD-k)
ContrastiveDivergenceusestwotrickstospeedupthesamplingprocess:
• sinceweeventuallywantp(v)p
train
(v)(thetrue,underlyingdistributionofthedata),weinitialize
theMarkovchainwithatrainingexample(i.e.,fromadistributionthatisexpectedtobeclosetop,so
thatthechainwillbealreadyclosetohavingconvergedtoitsfinaldistributionp).
• CDdoesnotwaitforthechaintoconverge.Samplesareobtainedafteronlyk-stepsofGibbssampling.
Inpratice,k=1hasbeenshowntoworksurprisinglywell.
9.3.2 PersistentCD
PersistentCD[Tieleman08]usesanotherapproximationforsamplingfromp(v;h). Itreliesonasingle
Markovchain,whichhasapersistentstate(i.e.,notrestartingachainforeachobservedexample).Foreach
parameterupdate,weextractnewsamplesbysimplyrunningthechainfork-steps.Thestateofthechainis
thenpreservedforsubsequentupdates.
Thegeneralintuitionisthatifparameterupdatesaresmallenoughcomparedtothemixingrateofthechain,
theMarkovchainshouldbeableto“catchup”tochangesinthemodel.
9.4 Implementation
WeconstructanRBMclass.Theparametersofthenetworkcaneitherbeinitializedbytheconstructororcan
bepassedasarguments.ThisoptionisusefulwhenanRBMisusedasthebuildingblockofadeepnetwork,
inwhichcasetheweightmatrixandthehiddenlayerbiasissharedwiththecorrespondingsigmoidallayer
ofanMLPnetwork.
class RBM(object):
"""Restricted Boltzmann n Machine (RBM)
"""
def __init__(
self,
input=None,
n_visible=784,
n_hidden=500,
W=None,
hbias=None,
vbias=None,
numpy_rng=None,
theano_rng=None
):
"""
RBM constructor. Defines the parameters of f the e model along with
9.4. Implementation
95
.Pdf to .jpg converter online - Convert PDF to JPEG images in C#.net, ASP.NET MVC, WinForms, WPF project
How to convert PDF to JPEG using C#.NET PDF to JPEG conversion / converter library control SDK
convert pdf to 300 dpi jpg; batch pdf to jpg converter
.Pdf to .jpg converter online - VB.NET PDF Convert to Jpeg SDK: Convert PDF to JPEG images in vb.net, ASP.NET MVC, WinForms, WPF project
Online Tutorial for PDF to JPEG (JPG) Conversion in VB.NET Image Application
best pdf to jpg converter for; convert pdf file to jpg on
DeepLearningTutorial,Release0.1
basic operations for inferring hidden from m visible e (and vice-versa),
as well as for performing CD updates.
:param input: None for standalone RBMs or symbolic variable if RBM is
part of a a larger r graph.
:param n_visible: number of visible units
:param n_hidden: number of hidden units
:param W: : None e for standalone RBMs or symbolic variable pointing to a
shared weight matrix in case RBM is part of f a a DBN network; in a DBN,
the weights are shared between RBMs and layers of a MLP
:param hbias: None for standalone RBMs or symbolic variable pointing
to a shared hidden units bias vector in case RBM is part of a
different network
:param vbias: None for standalone RBMs or a a symbolic c variable
pointing to o a shared visible units bias
"""
self.n_visible n_visible
self.n_hidden n_hidden
if numpy_rng is None:
# create a number generator
numpy_rng numpy.random.RandomState(1234)
if theano_rng is None:
theano_rng RandomStreams(numpy_rng.randint(2
**
30))
if is None:
# W is initialized with ‘initial_W‘ which is uniformely
# sampled from -4
*
sqrt(6./(n_visible+n_hidden)) and
# 4
*
sqrt(6./(n_hidden+n_visible)) the output of uniform if
# converted using asarray to dtype theano.config.floatX so
# that the code is runable on GPU
initial_W numpy.asarray(
numpy_rng.uniform(
low=-4
*
numpy.sqrt(6. (n_hidden n_visible)),
high=4
*
numpy.sqrt(6. (n_hidden n_visible)),
size=(n_visible, n_hidden)
),
dtype=theano.config.floatX
)
# theano shared variables for weights and biases
theano.shared(value=initial_W, name=’W’, borrow=True)
if hbias is None:
# create shared variable for hidden units bias
hbias theano.shared(
value=numpy.zeros(
96
Chapter9. RestrictedBoltzmannMachines(RBM)
Online Convert PDF to Jpeg images. Best free online PDF JPEG
Online PDF to JPEG Converter. Download Free Trial. Convert a PDF File to JPG. Drag and drop your PDF in the box above and we'll convert the files for you.
convert pdf file to jpg format; change pdf to jpg
Online Convert Jpeg to PDF file. Best free online export Jpg image
Online JPEG to PDF Converter. Download Free Trial. Convert a JPG to PDF. You can drag and drop your JPG file in the box, and then start
convert pdf to jpg; bulk pdf to jpg
DeepLearningTutorial,Release0.1
n_hidden,
dtype=theano.config.floatX
),
name=’hbias’,
borrow=True
)
if vbias is None:
# create shared variable for visible units bias
vbias theano.shared(
value=numpy.zeros(
n_visible,
dtype=theano.config.floatX
),
name=’vbias’,
borrow=True
)
# initialize input layer for standalone RBM M or r layer0 of DBN
self.input input
if not input:
self.input T.matrix(’input’)
self.W
self.hbias hbias
self.vbias vbias
self.theano_rng theano_rng
#
****
WARNING: It is not a good idea to put things in this list
# other than shared variables created in this function.
self.params [self.W, self.hbias, self.vbias]
NextstepistodefinefunctionswhichconstructthesymbolicgraphassociatedwithEqs. (9.7)-(9.8).The
codeisasfollows:
def propup(self, vis):
’’’This function propagates the visible units activation upwards to
the hidden units
Note that t we e return also the pre-sigmoid activation of the
layer. As s it t will turn out later, due to how Theano deals with
optimizations, this symbolic variable will l be e needed to write
down a more stable computational graph (see e details s in the
reconstruction cost function)
’’’
pre_sigmoid_activation T.dot(vis, self.W) self.hbias
return [pre_sigmoid_activation, T.nnet.sigmoid(pre_sigmoid_activation)]
def sample_h_given_v(self, v0_sample):
’’’ This s function n infers state of hidden units given visible units ’’’
# compute e the e activation of the hidden units given a sample of
# the visibles
pre_sigmoid_h1, h1_mean self.propup(v0_sample)
9.4. Implementation
97
C# Image Convert: How to Convert Adobe PDF to Jpeg, Png, Bmp, &
using RasterEdge.XDoc.Converter; String inputFilePath = @"C:\input.pdf"; String outputFilePath = @"C:\output.jpg"; // Convert PDF to jpg.
convert pdf pages to jpg; convert pdf into jpg format
C# Image Convert: How to Convert Dicom Image File to Raster Images
RasterEdge.XDoc.PDF.dll. using RasterEdge.XDoc.Converter; String inputFilePath = @"C:\input.dcm"; String outputFilePath = @"C:\output.jpg"; // Convert dicom to
change pdf to jpg image; .pdf to jpg
DeepLearningTutorial,Release0.1
# get a sample of the hiddens given their activation
# Note that theano_rng.binomial returns a symbolic sample of dtype
# int64 by default. If we want to keep our r computations s in floatX
# for the e GPU U we need to specify to return n the e dtype floatX
h1_sample self.theano_rng.binomial(size=h1_mean.shape,
n=1, p=h1_mean,
dtype=theano.config.floatX)
return [pre_sigmoid_h1, h1_mean, h1_sample]
def propdown(self, hid):
’’’This function propagates the hidden units activation downwards to
the visible units
Note that t we e return also the pre_sigmoid_activation of the
layer. As s it t will turn out later, due to how Theano deals with
optimizations, this symbolic variable will l be e needed to write
down a more stable computational graph (see e details s in the
reconstruction cost function)
’’’
pre_sigmoid_activation T.dot(hid, self.W.T) self.vbias
return [pre_sigmoid_activation, T.nnet.sigmoid(pre_sigmoid_activation)]
def sample_v_given_h(self, h0_sample):
’’’ This s function n infers state of visible units given hidden units ’’’
# compute e the e activation of the visible given the hidden sample
pre_sigmoid_v1, v1_mean self.propdown(h0_sample)
# get a sample of the visible given their activation
# Note that theano_rng.binomial returns a symbolic sample of dtype
# int64 by default. If we want to keep our r computations s in floatX
# for the e GPU U we need to specify to return n the e dtype floatX
v1_sample self.theano_rng.binomial(size=v1_mean.shape,
n=1, p=v1_mean,
dtype=theano.config.floatX)
return [pre_sigmoid_v1, v1_mean, v1_sample]
WecanthenusethesefunctionstodefinethesymbolicgraphforaGibbssamplingstep. Wedefinetwo
functions:
• gibbs_vhvwhichperformsastepofGibbssamplingstartingfromthevisibleunits. . Asweshall
see,thiswillbeusefulforsamplingfromtheRBM.
• gibbs_hvhwhichperformsastepofGibbssamplingstartingfromthehiddenunits.Thisfunction
willbeusefulforperformingCDandPCDupdates.
Thecodeisasfollows:
def gibbs_hvh(self, h0_sample):
’’’ This s function n implements one step of Gibbs sampling,
starting from the hidden state’’’
pre_sigmoid_v1, v1_mean, v1_sample self.sample_v_given_h(h0_sample)
pre_sigmoid_h1, h1_mean, h1_sample self.sample_h_given_v(v1_sample)
return [pre_sigmoid_v1, v1_mean, v1_sample,
pre_sigmoid_h1, h1_mean, h1_sample]
98
Chapter9. RestrictedBoltzmannMachines(RBM)
JPEG to PDF Converter | Convert JPEG to PDF, Convert PDF to JPEG
Features and Benefits. Powerful image converter to convert images of JPG Support a batch conversion of JPG to PDF with high speed; Get a compressed PDF file after
convert pdf image to jpg; conversion of pdf to jpg
VB.NET PDF - Convert PDF with VB.NET WPF PDF Viewer
An advanced PDF converter tool, which supports to be integrated in .NET to image file formats with high quality, support converting PDF to PNG, JPG, BMP and
pdf to jpg converter; convert pdf pictures to jpg
DeepLearningTutorial,Release0.1
def gibbs_vhv(self, v0_sample):
’’’ This s function n implements one step of Gibbs sampling,
starting from the visible state’’’
pre_sigmoid_h1, h1_mean, h1_sample self.sample_h_given_v(v0_sample)
pre_sigmoid_v1, v1_mean, v1_sample self.sample_v_given_h(h1_sample)
return [pre_sigmoid_h1, h1_mean, h1_sample,
pre_sigmoid_v1, v1_mean, v1_sample]
# start-snippet-2
Notethatwealsoreturnthepre-sigmoidactivation.Tounderstandwhythisissoyouneedtounderstanda
bitabouthowTheanoworks.WheneveryoucompileaTheanofunction,thecomputationalgraphthatyou
passasinputgetsoptimizedforspeedandstability.Thisisdonebychangingseveralpartsofthesubgraphs
withothers. Onesuchoptimizationexpressestermsoftheformlog(sigmoid(x))intermsofsoftplus. . We
needthisoptimizationforthecross-entropysincesigmoidofnumberslargerthan30. (orevenlessthen
that)turnto1.andnumberssmallerthan-30.turnto0whichintermswillforcetheanotocomputelog(0)
andthereforewewillgeteither-inforNaNascost.Ifthevalueisexpressedintermsofsoftpluswedonot
getthisundesirablebehaviour. Thisoptimizationusuallyworksfine,butherewehaveaspecialcase.The
sigmoidisappliedinsidethescanop,whilethelogisoutside.ThereforeTheanowillonlyseelog(scan(..))
insteadoflog(sigmoid(..)) andwillnotapplythewantedoptimization. . Wecannotgoandreplacethe
sigmoidinscanwithsomethingelsealso,becausethisonlyneedstobedoneonthelaststep.Thereforethe
easiestandmoreefficientwayistogetalsothepre-sigmoidactivationasanoutputofscan,andapplyboth
thelogandsigmoidoutsidescansuchthatTheanocancatchandoptimizetheexpression.
Theclassalsohasafunctionthatcomputesthefreeenergyofthemodel,neededforcomputingthegradient
oftheparameters(seeEq.(9.4)).Notethatwealsoreturnthepre-sigmoid
def free_energy(self, v_sample):
’’’ Function to compute the free energy ’’’
wx_b T.dot(v_sample, self.W) self.hbias
vbias_term T.dot(v_sample, self.vbias)
hidden_term T.sum(T.log(T.exp(wx_b)), , axis=1)
return -hidden_term vbias_term
Wethenaddaget_cost_updatesmethod,whosepurposeistogeneratethesymbolicgradientsfor
CD-kandPCD-kupdates.
def get_cost_updates(self, lr=0.1, persistent=None, k=1):
"""This functions implements one step of CD-k or PCD-k
:param lr: learning rate used to train the e RBM
:param persistent: None for CD. For PCD, shared variable
containing old state of Gibbs chain. This must be a shared
variable of size (batch size, number of f hidden n units).
:param k: : number r of Gibbs steps to do in CD-k/PCD-k
Returns a a proxy y for the cost and the updates dictionary. The
dictionary contains the update rules for weights and biases but
also an update of the shared variable used d to o store the persistent
chain, if f one e is used.
9.4. Implementation
99
JPG to GIF Converter | Convert JPEG to GIF, Convert GIF to JPG
Features and Benefits. High speed JPEG to GIF Converter, faster than other JPG Converters; Standalone software, so the user who is not online still can use
convert pdf to high quality jpg; convert pdf to jpg file
VB.NET Create PDF from images Library to convert Jpeg, png images
Support create PDF from multiple image formats in VB.NET, including Jpg, Png, Bmp, Gif, Tiff, Bitmap, .NET Graphics, and REImage. .NET converter control for
.net convert pdf to jpg; batch pdf to jpg converter online
DeepLearningTutorial,Release0.1
"""
# compute e positive e phase
pre_sigmoid_ph, ph_mean, ph_sample self.sample_h_given_v(self.input)
# decide e how w to initialize persistent chain:
# for CD, , we e use the newly generate hidden n sample
# for PCD, we initialize from the old state e of f the chain
if persistent is None:
chain_start ph_sample
else:
chain_start persistent
Notethatget_cost_updatestakesasargumentavariablecalledpersistent.Thisallowsustouse
thesamecodetoimplementbothCDandPCD.TousePCD,persistentshouldrefertoasharedvariable
whichcontainsthestateoftheGibbschainfromthepreviousiteration.
IfpersistentisNone,weinitializetheGibbschainwiththehiddensamplegeneratedduringthepositive
phase,thereforeimplementingCD.Oncewehaveestablishedthestartingpointofthechain,wecanthen
computethesampleattheendoftheGibbschain,samplethatweneedforgettingthegradient(seeEq.
(9.4)).Todoso,wewillusethescanopprovidedbyTheano,thereforeweurgethereadertolookitupby
followingthislink.
# perform m actual l negative phase
# in order to implement CD-k/PCD-k we need d to o scan over the
# function that implements one gibbs step k k times.
# Read Theano tutorial on scan for more information :
# http://deeplearning.net/software/theano/library/scan.html
# the scan will return the entire Gibbs chain
(
[
pre_sigmoid_nvs,
nv_means,
nv_samples,
pre_sigmoid_nhs,
nh_means,
nh_samples
],
updates
theano.scan(
self.gibbs_hvh,
# the e None e are place holders, saying that
# chain_start is the initial state corresponding to the
# 6th h output
outputs_info=[NoneNoneNoneNoneNone, chain_start],
n_steps=k
)
Oncewehavethegeneratedthechainwetakethesampleattheendofthechaintogetthefreeenergyofthe
negativephase.Notethatthechain_endisasymbolicalTheanovariableexpressedintermsofthemodel
parameters,andifwewouldapplyT.gradnaively,thefunctionwilltrytogothroughtheGibbschainto
getthegradients.Thisisnotwhatwewant(itwillmessupourgradients)andthereforeweneedtoindicate
100
Chapter9. RestrictedBoltzmannMachines(RBM)
DeepLearningTutorial,Release0.1
toT.gradthatchain_endisaconstant.Wedothisbyusingtheargumentconsider_constantof
T.grad.
# determine gradients on RBM parameters
# note that we only need the sample at the e end d of the chain
chain_end nv_samples[-1]
cost T.mean(self.free_energy(self.input)) T.mean(
self.free_energy(chain_end))
# We must t not t compute the gradient through h the e gibbs sampling
gparams T.grad(cost, self.params, consider_constant=[chain_end])
Finally,weaddtotheupdatesdictionaryreturnedbyscan(whichcontainsupdatesrulesforrandomstates
oftheano_rng)tocontaintheparameterupdates.InthecaseofPCD,theseshouldalsoupdatetheshared
variablecontainingthestateoftheGibbschain.
# constructs the update dictionary
for gparam, param in zip(gparams, self.params):
# make sure that the learning rate is of the right dtype
updates[param] param gparam
*
T.cast(
lr,
dtype=theano.config.floatX
)
if persistent:
# Note that this works only if persistent is a shared variable
updates[persistent] nh_samples[-1]
# pseudo-likelihood is a better proxy for PCD
monitoring_cost self.get_pseudo_likelihood_cost(updates)
else:
# reconstruction cross-entropy is a better proxy for CD
monitoring_cost self.get_reconstruction_cost(updates,
pre_sigmoid_nvs[-1])
return monitoring_cost, updates
9.4.1 TrackingProgress
RBMsareparticularlytrickytotrain. BecauseofthepartitionfunctionZofEq.(9.1),wecannotestimate
thelog-likelihoodlog(P(x))duringtraining. Wethereforehavenodirectusefulmetricforchoosingthe
optimalhyperparameters.
Severaloptionsareavailabletotheuser.
InspectionofNegativeSamples
Negativesamplesobtainedduringtrainingcanbevisualized. Astrainingprogresses,weknowthatthe
modeldefinedbytheRBMbecomesclosertothetrueunderlyingdistribution,p
train
(x).Negativesamples
shouldthuslooklikesamplesfromthetrainingset.Obviouslybadhyperparameterscanbediscardedinthis
fashion.
VisualInspectionofFilters
Thefilterslearntbythemodelcanbevisualized. Thisamountstoplottingtheweightsofeachunitasa
9.4. Implementation
101
DeepLearningTutorial,Release0.1
gray-scaleimage(afterreshapingtoasquarematrix). Filtersshouldpickoutstrongfeaturesinthedata.
Whileitisnotclearforanarbitrarydataset,whatthesefeaturesshouldlooklike,trainingonMNISTusually
resultsinfilterswhichactasstrokedetectors,whiletrainingonnaturalimagesleadtoGaborlikefiltersif
trainedinconjunctionwithasparsitycriteria.
ProxiestoLikelihood
Other,moretractablefunctionscanbeusedasaproxytothelikelihood.WhentraininganRBMwithPCD,
onecanusepseudo-likelihoodastheproxy.Pseudo-likelihood(PL)ismuchlessexpensivetocompute,as
itassumesthatallbitsareindependent.Therefore,
PL(x)=
Y
i
P(x
i
jx
i
)and
logPL(x)=
X
i
logP(x
i
jx
i
)
Herex
i
denotesthesetofallbitsofxexceptbiti.Thelog-PListhereforethesumofthelog-probabilities
ofeachbitx
i
,conditionedonthestateofallotherbits.ForMNIST,thiswouldinvolvesummingoverthe
784inputdimensions,whichremainsratherexpensive. Forthisreason,weusethefollowingstochastic
approximationtolog-PL:
g=NlogP(x
i
jx
i
),whereiU(0;N);,and
E[g]=logPL(x)
wheretheexpectationistakenovertheuniformrandomchoiceofindexi,andNisthenumberofvisible
units. Inordertoworkwithbinaryunits,wefurtherintroducethenotation~x
i
torefertoxwithbit-ibeing
flipped(1->0,0->1).Thelog-PLforanRBMwithbinaryunitsisthenwrittenas:
logPL(x)Nlog
e
FE(x)
FE(x) +e FE(~x
i
)
Nlog[sigm(FE(~x
i
) FE(x))]
WethereforereturnthiscostaswellastheRBMupdatesintheget_cost_updatesfunctionoftheRBM
class. Noticethatwemodifytheupdatesdictionarytoincrementtheindexofbiti.Thiswillresultinbiti
cyclingoverallpossiblevaluesf0;1;:::;Ng,fromoneupdatetoanother.
NotethatforCDtrainingthecross-entropycostbetweentheinputandthereconstruction(thesameasthe
oneusedforthede-noisingautoencoder)ismorereliablethenthepseudo-loglikelihood. Hereisthecode
weusetocomputethepseudo-likelihood:
def get_pseudo_likelihood_cost(self, updates):
"""Stochastic approximation to the pseudo-likelihood"""
# index of bit i in expression p(x_i | x_{\i})
bit_i_idx theano.shared(value=0, name=’bit_i_idx’)
# binarize the input image by rounding to nearest integer
xi T.round(self.input)
102
Chapter9. RestrictedBoltzmannMachines(RBM)
DeepLearningTutorial,Release0.1
# calculate free energy for the given bit configuration
fe_xi self.free_energy(xi)
# flip bit x_i of matrix xi and preserve all other bits x_{\i}
# Equivalent to xi[:,bit_i_idx] = 1-xi[:, bit_i_idx], but assigns
# the result to xi_flip, instead of working g in n place on xi.
xi_flip T.set_subtensor(xi[:, bit_i_idx], xi[:, , bit_i_idx])
# calculate free energy with bit flipped
fe_xi_flip self.free_energy(xi_flip)
# equivalent to e^(-FE(x_i)) / (e^(-FE(x_i)) + e^(-FE(x_{\i})))
cost T.mean(self.n_visible
*
T.log(T.nnet.sigmoid(fe_xi_flip -
fe_xi)))
# increment bit_i_idx % number as part of updates
updates[bit_i_idx] (bit_i_idx 1self.n_visible
return cost
9.4.2 MainLoop
Wenowhaveallthenecessaryingredientstostarttrainingournetwork.
Before going overthe training loop however, , the e reader r should familiarize e himself f with the function
tile_raster_images (seePlottingSamplesand Filters). . SinceRBMs s aregenerativemodels, we
areinterestedinsamplingfromthemandplotting/visualizingthesesamples. Wealsowanttovisualizethe
filters(weights)learntbytheRBM,togaininsightsintowhattheRBMisactuallydoing. Bearinmind
however,thatthisdoesnotprovidetheentirestory,sinceweneglectthebiasesandplottheweightsuptoa
multiplicativeconstant(weightsareconvertedtovaluesbetween0and1).
Havingtheseutilityfunctions,wecanstarttrainingtheRBMandplot/savethefiltersaftereachtraining
epoch. WetraintheRBMusingPCD,asithasbeenshowntoleadtoabettergenerativemodel([Tiele-
man08]).
# it is ok for a theano function to have no output
# the purpose e of f train_rbm is solely to update e the e RBM parameters
train_rbm theano.function(
[index],
cost,
updates=updates,
givens={
x: train_set_x[index
*
batch_size: (index 1)
*
batch_size]
},
name=’train_rbm’
)
plotting_time 0.
start_time timeit.default_timer()
# go through h training g epochs
9.4. Implementation
103
DeepLearningTutorial,Release0.1
for epoch in xrange(training_epochs):
# go through the training set
mean_cost []
for batch_index in xrange(n_train_batches):
mean_cost += [train_rbm(batch_index)]
print ’Training epoch %d, cost is ’ epoch, numpy.mean(mean_cost)
# Plot filters after each training epoch
plotting_start timeit.default_timer()
# Construct image from the weight matrix
image Image.fromarray(
tile_raster_images(
X=rbm.W.get_value(borrow=True).T,
img_shape=(2828),
tile_shape=(1010),
tile_spacing=(11)
)
)
image.save(’filters_at_epoch_%i.png’ epoch)
plotting_stop timeit.default_timer()
plotting_time += (plotting_stop plotting_start)
end_time timeit.default_timer()
pretraining_time (end_time start_time) plotting_time
print (’Training took %f minutes’ (pretraining_time 60.))
OncetheRBMistrained,wecanthenusethegibbs_vhvfunctiontoimplementtheGibbschainrequired
forsampling.WeinitializetheGibbschainstartingfromtestexamples(althoughwecouldaswellpickit
fromthetrainingset)inordertospeedupconvergenceandavoidproblemswithrandominitialization.We
againuseTheano’sscanoptodo1000stepsbeforeeachplotting.
#################################
#
Sampling from the RBM
#
#################################
# find out the number of test samples
number_of_test_samples test_set_x.get_value(borrow=True).shape[0]
# pick random m test t examples, with which to initialize the persistent chain
test_idx rng.randint(number_of_test_samples n_chains)
persistent_vis_chain theano.shared(
numpy.asarray(
test_set_x.get_value(borrow=True)[test_idx:test_idx n_chains],
dtype=theano.config.floatX
)
)
Nextwecreatethe20persistentchainsinparalleltogetoursamples.Todoso,wecompileatheanofunction
whichperformsoneGibbsstepandupdatesthestateofthepersistentchainwiththenewvisiblesample.We
applythisfunctioniterativelyforalargenumberofsteps,plottingthesamplesatevery1000steps.
104
Chapter9. RestrictedBoltzmannMachines(RBM)
Documents you may be interested
Documents you may be interested