122
mingbotnet.Byinfiltratingitscommandandcontrolinfrastructure
parasitically,weconvincedittomodifyasubsetofthespamital-
readysends,therebydirectinganyinterestedrecipientstoservers
underourcontrol,ratherthanthosebelongingtothespammer. In
turn, our servers presented Web sites mimicking those actually
hosted by thespammer, but“defanged”to removefunctionality
thatwouldcompromisethevictim’ssystemorreceivesensitiveper-
sonalinformationsuchasname,addressorcreditcardinformation.
Usingthismethodology,wehavedocumentedthreespamcam-
paigns comprising over 469millione-mails. Weidentifiedhow
muchofthisspamissuccessfullydelivered,howmuchisfiltered
bypopularanti-spamsolutions,and,mostimportantly,howmany
users“click-through”tothesitebeingadvertised(responserate)
andhowmanyofthoseprogresstoa“sale”or“infection”(conver-
sionrate).
Theremainderofthispaperisstructuredasfollows. Section2
describestheeconomicbasisforspamandreviewspriorresearch
inthisarea. Section3describestheStormbotnet,andSection4
describesour experimentalmethodology using Storm. Section5
describesourspamfilteringandconversionresults,Section6an-
alyzestheeffectsofblacklistingonspamdelivery, andSection7
analyzesthepossibleinfluencesonspamresponses.Wesynthesize
ourfindingsinSection8andconclude.
2. BACKGROUND
Directmarketinghasarichhistory,datingbacktothe19thcen-
turydistributionofthefirstmail-ordercatalogs.Whatmakesdirect
marketingsoappealingisthatonecandirectlymeasureitsreturn
oninvestment. Forexample, theDirectMailAssociationreports
thatdirectmailsalescampaignsproducearesponserateof 2.15
percentonaverage[5]. Meanwhile,roughestimatesofdirectmail
costpermille(CPM)– thecosttoaddress, produceand deliver
materialstoathousandtargets–rangebetween$250and$1000.
Thus,followingtheseestimatesitmightcost$250,000tosendout
amillionsolicitations,whichmightthenproduce21,500responses.
Thecostofdevelopingtheseprospects(roughly$12each)canbe
directlycomputedand,assumingeachprospectcompletesasaleof
anaveragevalue,onecanbalancethisrevenuedirectlyagainstthe
marketingcoststodeterminetheprofitabilityofthecampaign.As
longastheproductoftheconversionrateandthemarginalprofit
persaleexceedsthemarginaldeliverycost,thecampaignisprof-
itable.
Giventhisunderlyingvalueproposition,itisnotatallsurpris-
ingthatbulkdirecte-mailmarketingemergedveryquickly after
e-mailitself.Themarginalcosttosendane-mailistinyand,thus,
ane-mail-basedcampaigncanbeprofitableevenwhentheconver-
sionrateisnegligible. Unfortunately,aperversebyproductofthis
dynamicisthatsendingasmuchspamaspossibleislikelytomax-
imizeprofit.
Theresultingsocialnuisancebegatavibrantanti-spamcommu-
nity,eventually producingamulti-billiondollarindustryfocused
onthesameproblem. However, witheachanti-spaminnovation
spammersadaptedinkindand,whiletheresultingco-evolutionhas
notsignificantly changedthespamproblem, ithaschangedhow
spamispurveyed.Forexample,theadventofreal-timeIPblacklist-
ingdeployedinMailTransferAgents(MTAs)forcedspammersto
relaytheirmessagesthrough“untainted”third-partyhosts—driv-
ingthecreationofmodernlarge-scalebotnets. Similarly,content-
basedanti-spamfiltersinturnforcedspammerstocreatesophisti-
catedpolymorphismengines,modifyingeachspammessagetobe
tivities,whilestrictlyreducingharmforthosesituationsinwhich
userpropertywasatrisk.
distinct. Aswell,itforcedthemtosendevenmorespam. Thus,
ithasbeenestimatedthatover120billionspammessagesarenow
senteachday[11].
However, whilespamhas longbeenunderstood tobean eco-
nomicproblem, itisonlyrecently thattherehasbeensignificant
effortinmodeling spameconomicsandunderstandingthevalue
propositionfromthespammer’spointofview.Rarelydospammers
talk aboutfinancialaspectsoftheiractivitiesthemselves, though
suchaccountsdoexist[14,21].Judgeetal.describeaprototypical
modelofspamprofitability,includingboththebasicvaluepropo-
sitionaswellastheimpactofanti-spamfilteringandlawenforce-
ment. Theyspeculatethatresponseratesaslowas0.000001are
sufficienttomaintainprofitability[17]. Khong[13]likewiseem-
ploysaneconomiccostmodelofspam,comparingthesuccessof
severalanti-spamstrategies.GoodmanandRounthwaiteconstruct
amorecomplexmodel,aimedatderivingthecostfactorsforsend-
ingspam,andconcludedepressinglythattheoptimalstrategyfor
sendingspamistosendasfastaspossible[9].SerjantovandClay-
tonexploretheseissuesfromthestandpointofanISPandtryto
understandhowtoplaceappropriateincentivesaroundtheuseof
anti-spamblacklists[19].
However,theworkthatismostcloselyrelatedtoourown are
theseveralpapersconcerning“StockSpam”[7,8,10].Stockspam
referstothepracticeofsendingpositive“touts”foralow-volume
securityinordertomanipulateitspriceand thereby profitonan
existing position inthestock. Whatdistinguishesstock spamis
thatitismonetizedthroughpricemanipulationandnotviaasale.
Consequently, itisnotnecessary to measuretheconversionrate
tounderstandprofitability. Instead,profitabilitycanbeinferredby
correlatingstockspammessagevolumewithchangesinthetrading
volumeandpricefortheassociatedstocks.
TheworkofMaandChenissimilartooursinthatitanalyzesin
detailthestructureofaspammingoperation. However,theirfocus
isonredirectionchainsemployedbyspammersasasearchengine
optimizationstrategy[20].
3. THESTORMBOTNET
ThemeasurementsinthispaperarecarriedoutusingtheStorm
botnetanditsspammingagents. Whileacompletetechnicalde-
scription ofStormisoutsidethescopeof this paper, wereview
keymechanismsinStorm’scommunicationprotocolsandorgani-
zationalhierarchy.
Stormisapeer-to-peer botnetthatpropagatesviaspam(usu-
allybydirectingrecipientstodownloadanexecutablefromaWeb
site). Stormcommunicatesusingtwoseparateprotocols:thefirst
isanencryptedversionoftheUDP-basedOvernetprotocol(inturn
basedontheKademliaDHT[16])and isusedprimarilyasadi-
rectoryservicetofindothernodes. Aswell,Stormusesacustom
TCP-basedprotocolformanagingcommandandcontrol—thedi-
rectionsinforming eachbotwhatactionsitshouldtake. Wede-
scribeeachofthesebelow.
3.1 Overnetprotocol
Therearefourbasicmessagestofacilitatethebasicfunctioning
ofOvernet:Connect,Search,Publicize,andPublish. Duringthe
bootstrapphase,aStormnodeonlyhastheinitiallistofpeersthat
itwasshippedwith. TogathermorepeersStormchoosesaOID
pseudo-randomlyfromthe128-bitOvernetaddressspaceandpro-
ceedstoConnecttoallthepeersinitsbootstraplist.Eachavailable
peercontactedreturnsalistofupto20peers. Stormdoesthisfor
afewroundsuntilithasgatheredenoughpeerstobeadequately
connectedinOvernet.Onceanewnodehaslearnedaboutenough
peersitswitchesto Publicizing itspresencetonearbypeersand
109
Figure1: TheStormbotnethierarchy.
periodicallysearchingforitsownOIDtostayconnectedandlearn
aboutnewclose-bypeerstokeepupwithchurn.
Overnetalsoprovidestwomessagesforstoringandfindingcon-
tentinthenetwork: PublishandSearchwhichexportastandard
DHT(key,value)pairinterface. However,Stormusesthisinter-
faceinan unusualway. Inparticular,thekeysencodeadynam-
ically changing rendezvouscodethatallowStormnodesto find
eachotherondemand.
AStormnodegeneratesandusesthreerendezvouskeyssimulta-
neously:onebasedonthecurrentdate,onebasedontheprevious
date,andonebasedonthenextdate.Todeterminethecorrectdate,
StormfirstsetsthesystemclockusingNTP.
Inparticular, eachkey is basedon acombinationof thetime
(with24-hourresolution)mixedwitharandomintegerbetween0
and31. Thusthereare32uniqueStormkeysinuseperdaybut
asingleStormbotwillonlyuse1of the32. Becausekeysare
basedontime,StormusesNTPtosyncabot’sclockandattempts
to normalizethetimezone. Even so, tomakesurebotsaround
theworldcanstayinsync,Stormuses3daysofkeysatonce,the
previous,current,andnextday.
Inturn,thesekeysareusedtorendezvouswithStormnodesthat
implementthecommand andcontrol(C&C) channel. AStorm
nodethatwishestooffertheC&Cservicewillusethetime-based
hashingalgorithmtogenerateakeyandencodeitsownIPaddress
andTCPportintothevalue.Itwillthensearchfortheappropriate
peersclosetothekeyandpublishits(key,value)pairtothem. A
peerwishingtolocateaC&Cchannelcangenerateatime-based
keyandsearchforpreviouslypublishedvaluestodecodeandcon-
necttotheTCPnetwork.
3.2 Stormhierarchy
TherearethreeprimaryclassesofStormnodesinvolvedinsend-
ingspam(showninFigure1).Workerbotsmakerequestsforwork
and, uponreceivingorders, sendspamasrequested. Proxy bots
actasconduitsbetweenworkersandmasterservers. Finally,the
masterserversprovidecommandstotheworkersandreceivetheir
statusreports. Inourexperiencethereareaverysmallnumberof
masterservers(typicallyhostedatso-called“bullet-proof”hosting
centers)andthesearelikelymanagedbythebotmasterdirectly.
However,thedistinctionbetweenworkerandproxyisonethat
isdeterminedautomatically.WhenStormfirstinfectsahostittests
ifitcanbereachedexternally. Ifso,thenitiseligibletobecomea
proxy.Ifnot,thenitbecomesaworker.
3.3 Spamengine
Having decided to become a worker, a new bot first checks
whetheritcanreachtheSMTPserverofapopularWeb-basedmail
provideronTCPport25.Ifthischeckfailstheworkerwillremain
activebutnotparticipateinspammingcampaigns.
4
Figure2outlinesthebroadstepsforlaunchingspamcampaigns
whentheportcheckissuccessful.Theworkerfindsaproxy(using
thetime-varyingprotocoldescribedearlier)andthensendsanup-
daterequest(viatheproxy)toanassociatedmasterserver(Step1),
whichwillrespondwithaspamworkloadtask(Step2). Aspam
workload consistsof threecomponents: oneor morespamtem-
plates,adeliverylistofe-mailaddresses,andasetofnamed“dic-
tionaries”.Spamtemplatesarewritteninacustommacrolanguage
forgeneratingpolymorphicmessages[15]. Themacrosinsertele-
mentsfromthedictionaries(e.g.,targete-mailaddresses,message
subjectlines),randomidentifiers(e.g.,SMTPmessageidentifiers,
IPaddresses),thedateandtime,etc.,intomessagefieldsandtext.
GeneratedmessagesappearasiftheyoriginatefromavalidMTA,
andusepolymorphiccontentforevadingspamfilters.
Upon receiving a spam workload, a worker bot generates a
uniquemessageforeachoftheaddressesonthedeliverylistand
attemptstosendthemessagetotheMXoftherecipientviaSMTP
(Step3). Whentheworkerbothasexhausteditsdeliverylist, it
requeststwoadditionalspamworkloadsandexecutesthem.Itthen
sendsadeliveryreportbacktoitsproxy(Step4). Thereportin-
cludesaresultcodeforeachattempteddelivery.Ifanattemptwas
successful,itincludesthefulle-mailaddressoftherecipient;oth-
erwise,itreportsanerrorcodecorrespondingtothefailure. The
proxy, in turn, relaysthesestatus reportsback to theassociated
masterserver.
Tosummarize,Stormusesathree-levelself-organizinghierarchy
comprisedof worker bots, proxybotsandmasterservers. Com-
mandandcontrolis“pull-based”,drivenbyrequestsfromindivid-
ualworkerbots. Theserequestsaresenttoproxieswho,inturn,
automaticallyrelaytheserequeststomasterserversandsimilarly
forwardanyattendantresponsesbacktototheworkers.
4. METHODOLOGY
Ourmeasurementapproachisbasedonbotnetinfiltration—that
is, insinuating ourselvesinto abotnet’s “command and control”
(C&C)network,passivelyobservingthespam-relatedcommands
anddataitdistributesand, whereappropriate,activelychanging
individualelementsof thesemessagesin transit. Storm’sarchi-
tecturelendsitselfparticularlywelltoinfiltrationsincetheproxy
bots,bydesign,interposeonthecommunicationsbetweenindivid-
ualworkerbotsandthemasterserverswhodirectthem.Moreover,
sinceStormcompromiseshostsindiscriminately(normallyusing
malwaredistributedviasocialengineeringWebsites)itisstraight-
forwardtocreateaproxybotondemandbyinfecting aglobally
reachablehostunderourcontrolwiththeStormmalware.
Figure2alsoillustratesourbasicmeasurementinfrastructure.At
thecore,weinstantiateeightunmodifiedStormproxybotswithina
controlledvirtualmachineenvironmenthostedonVMWareESX3
servers.Thenetworktrafficforthesebotsisthenroutedthrougha
centralizedgateway,providingameansforblockingunanticipated
behaviors(e.g.,participationinDDoSattacks)andaninterposition
pointforparsingC&Cmessagesand“rewriting”themastheypass
fromproxiestoworkers.Mostcritically,bycarefullyrewritingthe
spamtemplateanddictionaryentriessentbymasterservers,wear-
rangeforworkerbotstoreplacetheintendedsitelinksintheirspam
withURLsofourchoosing.Fromthisbasiccapabilitywesynthe-
sizeexperimentstomeasuretheclick-throughandconversionrates
forseverallargespamcampaigns.
4
Suchbotsarestill“useful”forothertaskssuchasmountingcoor-
dinatedDDoSattacksthatStormperpetratesfromtimetotime.
96
Figure2: TheStorm spamcampaigndataflow(Section 3.3)
andourmeasurementandrewritinginfrastructure(Section4).
(1)Workersrequest spam tasksthrough proxies, (2) proxies
forward spam workload responses from master servers, (3)
workerssendthespamand(4)returndelivery reports. Our
infrastructureinfiltratestheC&Cchannelsbetweenworkers
andproxies.
Intheremainderofthissectionweprovideadetaileddescription
ofourStormC&Crewritingengine,discusshowweusethistool
toobtainempiricalestimatesforspamdelivery,click-throughand
conversionratesanddescribetheheuristicsusedfordifferentiating
realuservisitsfromthosedrivenbyautomated crawlers, honey-
clients, etc. With thiscontext, wethen review theethicalbasis
uponwhichthesemeasurementswereconducted.
4.1 C&Cprotocolrewriting
OurruntimeC&Cprotocolrewriterconsistsoftwocomponents.
AcustomClick-basednetwork elementredirects potentialC&C
trafficto afixed IPaddress and port, wherea user-space proxy
serverimplemented inPythonacceptsincomingconnectionsand
impersonatestheproxybots. Thisserverinturnforwardsconnec-
tionsbackintotheClickelement,whichredirectsthetraffictothe
intendedproxybot. Toassociateconnectionstotheproxyserver
withthoseforwardedbytheproxyserver,theClickelementinjects
aSOCKS-styledestinationheaderintotheflows.Theproxyserver
usesthisheadertoforwardaconnectiontoaparticularaddressand
port,allowingtheClickelementtomaketheassociation.Fromthat
pointon,trafficflowstransparentlythroughtheproxyserverwhere
C&Ctrafficisparsedandrewrittenasrequired.Rulesforrewriting
can beinstalledindependentlyfortemplates,dictionaries, ande-
mailaddresstargetlists.TherewriterlogsallC&Ctrafficbetween
workerandourproxybots,betweentheproxybotsandthemaster
servers,andallrewritingactionsonthetraffic.
SinceC&C trafficarriveson arbitrary ports, wedesigned the
proxyserversothatitinitiallyhandlesanytypeofconnectionand
fallsbacktopassivepass-throughforanynon-C&Ctraffic. Since
theproxyserver needs to maintain aconnection for each of the
(many)workers,weuseapreforked,multithreadeddesign.Apool
of30processesallowedustohandlethefullworkerloadforthe
eightStormproxybotsatalltimes.
4.2 Measuringspamdelivery
Toevaluatetheeffectofspamfilteringalongthee-maildelivery
pathtouserinboxes,weestablishedacollectionofteste-mailac-
countsandarrangedtohaveStormworkerbotssendspamtothose
accounts.Wecreatedmultipleaccountsatthreepopularfreee-mail
providers(Gmail,Yahoo!,andHotmail),accountsfilteredthrough
ourdepartmentcommercialspamfilteringappliance(aBarracuda
SpamFirewallModel300withslightlymorepermissivespamtag-
gingthanthedefaultsetting),andmultipleSMTP“sinks”atdis-
tinctinstitutionsthatacceptanymessagesenttothem(theseserved
as“controls”toensurethatspame-mailswerebeingsuccessfully
delivered, absentanyreceiver-sidespamfiltering). Whenworker
botsrequestspamworkloads, our rewriterappendsthesee-mail
addressestotheendofeachdeliverylist. Whenaworkerbotre-
portssuccessorfailurebacktothemasterservers,weremoveany
successreportsforoure-mailaddressestohideourmodifications
fromthebotmaster.
We periodically poll each e-mail account (both inbox and
“junk/spam”folders)forthemessagesthatitreceived,andwelog
themwiththeirtimestamps. However,someofthemessageswe
receive have nothing to do with our study and mustbefiltered
out. Thesemessagesoccurforarangeofreasons,includingspam
generatedby“dictionarybots”thatexhaustivelytargetpotentiale-
mailaddresses,orbecausetheaddressesweuseareunintentionally
“leaked”(thiscanhappenwhen aStormworkerbotconnectsto
ourproxyandthenleavesbeforeithasfinishedsendingitsspam;
whenitreconnectsviaanewproxythedeliveryreporttothemas-
terserverswillincludeour addresses). Tofiltersuche-mail,we
validatethateachmessageincludesbothasubjectlineusedbyour
selectedcampaignsandcontainsalinktooneoftheWebsitesun-
derourcontrol.
4.3 Measuringclick-throughandconversion
Toevaluatehowoftenuserswhoreceivespamactuallyvisitthe
sitesadvertisedrequiresmonitoringtheadvertisedsitesthemselves.
Sinceitisgenerallyimpracticaltomonitorsitesnotunderourcon-
trol,wehavearrangedtohaveafractionofStorm’sspamadvertise
sitesofourcreationinstead.
Inparticular,wehavefocusedontwotypesofStormspamcam-
paigns,aself-propagationcampaigndesignedtospreadtheStorm
malware (typically under the guise of advertising an electronic
postcardsite)andtheotheradvertisingapharmacysite. Theseare
thetwo mostpopularStormspamcampaignsandrepresentover
40%ofrecentStormactivity[15].
Foreachofthesecampaigns,theStormmasterserversdistribute
aspecific“dictionary”thatcontainsthesetoftargetURLstobein-
sertedintospame-mailsastheyaregeneratedbyworkerbots. To
divertuservisitstooursitesinstead,therewriterreplacesanydic-
tionariesthatpassthroughourproxieswithentriesonlycontaining
URLstoourWebservers.
Ingeneral,westriveforverisimilitudewiththeactualStormop-
eration. Thus,wearecarefultoconstructtheseURLsinthesame
mannerastherealStormsites(whetherthisisrawIPaddresses,as
used in theself-propagation campaigns, ortheparticular “noun-
noun.com”naming schema usedby thepharmacy campaign) to
ensurethegeneratedspamisqualitativelyindistinguishablefrom
the“realthing”. Animportantexception,uniquetothepharmacy
campaign,isanidentifierweaddtotheendofeachURLbymodi-
95
(a)Pharmaceuticalsite
(b)Postcard-themedself-propagationsite
Figure3: Screenshotsof theWebsitesoperatedto measure
userclick-throughandconversion.
fyingtheassociatedspamtemplate.Thisidentifierallowsustoun-
ambiguouslyassociateindividualspammessageswithsubsequent
accessesto thesite. We did notadd this identifier to theself-
propagationcampaignssincetheirURLstypicallyconsistentirely
ofrawIPaddresses. Theadditionofatextidentifiersuffixmight
thusappearoutofplace,reducingverisimilitude,andperhapsbias
userclickbehavior.
Finally, wecreatedtwo Web sitesto mimicthoseused in the
associated campaigns (screenshots of these sites are shown in
Figure3). Thepharmaceuticalsite, primarily marketing “male-
enhancement”drugssuchasViagra,isanearly-precisereplicaof
thesitenormallyadvertisedbyStormdowntousingthesamenam-
ingconventionforthedomainsthemselves. Oursitemirrorsthe
originalsite’suserinterface,theadditionofproductsadvertisedfor
saletoa“shoppingcart”,andnavigationupto,butnotincluding,
theinputofpersonalandpaymentinformation(therearearange
ofcomplexregulatory, legalandethicalissuesin accepting such
information).Instead,whenauserclickson“Checkout”wereturn
a404errormessage. Welogallaccessestothesite,allowingus
todeterminewhenavisitorattemptstomakeapurchaseandwhat
thecontentoftheirshoppingcartisatthetime. Weassumethata
purchaseattemptisaconversion,whichwespeculateisareason-
ableassumption, although our methodologydoesnotallowusto
validatethattheuserwouldhaveactuallycompletedthepurchase
orthattheircreditcardinformationwouldhavebeenvalid.
Theself-propagation campaign is Storm’s keymechanismfor
growth. ThecampaignenticesuserstodownloadtheStormmal-
wareviadeception;forexamplebytellingthemitispostcardsoft-
wareessentialfor viewing amessageor jokesentto thembya
friend. Unlikethepharmacyexample,wewerenotabletomirror
thegraphicalcontentofthepostcardsite,sinceitwasitselfstolen
fromalegitimateInternetpostcardsite.Instead,wecreatedaclose
analogdesignedtomimictheoveralllookandfeel. Wealso“de-
fanged”oursitebyreplacingitslinktotheStormmalwarewiththat
ofabenignexecutable. Ifrun, ourexecutableisdesignedtoper-
formsasimpleHTTPPOSTwithaharmlesspayload(“data=1”)
toaserver underourcontrol,andthen exit. Asaroughtimeout
mechanism, theexecutablewillnotsend themessageifthesys-
temdateis2009orlater. Sincethepostcardsiteweimpersonated
servedthreedifferentexecutablesunderdifferentnames,weserved
threeexecutableswithdifferenttargetfilenamesinthePOSTcom-
mandaswell.Again,allaccessestothesiteareloggedandweare
abletoidentifywhenourbinaryhasbeendownloaded. Moreover,
bycorrelatingwiththePOSTsignal,weareabletodetermineifa
particulardownloadisultimatelyexecutedonthevisitor’smachine
(andhenceisaconversion). Downloadsandexecutionscandiffer
becausetheuserhassecondthoughtsaboutallowinganexecution
orbecausetheuser’ssecuritysoftwarepreventsitfromexecuting
(indeed,weobservedthatseveralanti-virusvendorsdevelopedsig-
naturesforourbenignexecutablewithinafewdaysofourintro-
ducingit).
4.4 Separatingusersfromcrawlers
Aswithoure-mailaccounts,notallvisitstoourWebsiteare
prospectiveconversions. Thereisarangeofautomatedandsemi-
automated processesthatvisitour sites, ranging frompureWeb
crawlers,to“honeyclient”systemsdesignedtogatherintelligence
onspamadvertisedsites,tosecurityresearcherstryingtoidentify
newmalware.
Tofilter outsuchvisits(whichwegenericallycall“crawlers”)
fromintentfulones, wehavedeveloped aseries ofheuristicsto
identifycrawlersandusethisdatatopopulateaglobalIPblacklist
acrossallofourWebsites.Weoutlinetheseheuristicsbelow.
First, weconsider allhosts thataccessthepharmacysitethat
do not use aURLcontaining theunique identifier discussed in
Section4.3 to be crawlers. Second, weblacklisthosts that ac-
cessrobots.txt(site-specificinstructionsmeantonlyforWeb
crawlers)andhoststhatmakemalformedrequests(mostoftenex-
ploitattempts). Third,weblacklistallhoststhatdisablejavascript
anddonotloadembeddedimages. Weassumethattypicalusers
do notbrowseundertheseconditions, whereassomelarge-scale
anti-spamhoneypotsthatfollowembeddedlinksinsuspectedspam
exhibitthisbehaviortoreduceload.
In additiontoblacklistingbased onthebehavior ofindividual
sitevisits,anothercommonpatternweobservedwasthesameIP
addressaccessingthepharmacysiteusingseveraldifferentunique
identifiers,presumablyaspartofaspamdefenseormeasurement
mechanism.Consequently,weblacklistanIPaddressseenaccess-
ingthepharmacy sitewithmorethan oneuniqueidentifierwith
thesameUser-Agentfield. Thisheuristicdoesnotfilterusers
browsingbehindlargerWebproxyservices,butdoesfiltertheho-
mogeneousaccessesseenfromspamhoneyclients. Similarly,we
also blacklist any host thatrequeststhedownloaded executable
fromthepostcardsitetenormoretimes,undertheassumptionthat
suchhostsareusedbyresearchersorotherobserversinterestedin
trackingupdatestotheStormmalware.
Finally,ithasbecomecommonforanti-malwareresearchersto
findnewversionsoftheStormmalwarebydirectlyaccessingthe
self-propagationdictionary entries. To detectsuch userswein-
jectednewIPaddresses(neveradvertisedinspammessages)into
theself-propagationdictionary duringaperiodof inactivity (i.e.,
when noself-propagationspamwasbeingsent). Anyvisitorsto
157
Mar 07
Mar 12
Mar 17
Mar 22
Mar 27
Apr 01
Apr 06
Apr 11
Apr 16
0
0.5
1
1.5
2
2.5
3
Date
Emails assigned per hour (millions)
Postcard
Pharmacy
April Fool
Figure4: Numberofe-mailmessagesassignedperhourfor
eachcampaign.
C
AMPAIGN
D
ATES
W
ORKERS
E-
MAILS
Pharmacy
Mar21–Apr15
31,348
347,590,389
Postcard
Mar9–Mar15
17,639
83,665,479
AprilFool
Mar31–Apr2
3,678
38,651,124
Total 469,906,992
Table1: Campaignsusedintheexperiment.
theseIPaddressescouldnothaveresultedfromspam,andwethere-
forealsoaddedthemtoourcrawlerblacklist.
Itisstillpossiblethatsomeoftheaccesseswereviafull-featured,
low-volumehoneyclients,buteveniftheseexistwebelievetheyare
unlikelytosignificantlyimpactthedata.
4.5 Measurementethics
Wehavebeencarefultodesignexperimentsthatwebelieveare
bothconsistentwithcurrentU.S.legaldoctrineandarefundamen-
tallyethicalaswell. Whileitisbeyondthescopeofthispaperto
fullydescribethecomplexlegallandscapeinwhichactivesecurity
measurementsoperate,webelievetheethicalbasis for ourwork
isfareasiertoexplain:westrictlyreduceharm. First,ourinstru-
mentedproxy botsdonotcreateanynewharm. Thatis, absent
ourinvolvement,thesamesetofuserswouldreceivethesameset
ofspame-mailssentbythesameworker bots. Stormisalarge
self-organizingsystemandwhenaproxyfailsitsworkerbotsau-
tomaticallyswitchtootheridleproxies(indeed,whenourproxies
failweseeworkersquicklyswitchaway).Second,ourproxiesare
passiveactorsanddonotthemselvesengageinanybehaviorthat
isintrinsicallyobjectionable;theydonotsendspame-mail,they
donotcompromisehosts, nor do they evencontactworker bots
asynchronously.Indeed,theironlyfunctionistoprovideaconduit
betweenworkerbotsmakingrequestsandmasterserversproviding
responses. Finally,wherewedomodifyC&Cmessagesintransit,
theseactionsthemselvesstrictlyreduceharm.Userswhoclickon
spamalteredbythesechangeswillbedirectedtooneofourinnocu-
ousdoppelgangerWebsites. Unlikethesitesnormallyadvertised
byStorm,oursitesdonotinfectuserswithmalwareanddonotcol-
lectusercreditcardinformation.Thus,nousershouldreceivemore
spamduetoourinvolvement,butsomeuserswillreceivespamthat
islessdangerousthatitwouldotherwisebe.
Mar 24
Mar 29
Apr 02
Apr 06
Apr 10
Apr 14
0
100
200
300
400
500
600
Time
Number of connected workers
Proxy 1
Proxy 2
Proxy 3
Proxy 4
Proxy 5
Proxy 6
Proxy 7
Proxy 8
Figure5: Timelineofproxybotworkload.
D
OMAIN
F
REQ
.
hotmail.com
8.47%
yahoo.com
5.05%
gmail.com
3.17%
aol.com
2.37%
yahoo.co.in
1.13%
sbcglobal.net
0.93%
mail.ru
0.86%
shaw.ca
0.61%
wanadoo.fr
0.61%
msn.com
0.58%
Total 23.79%
Table 2: The 10 most-targeted e-mailaddress domains and
theirfrequencyinthecombinedlistsoftargetedaddressesover
allthreecampaigns.
5. EXPERIMENTALRESULTS
Wenowpresenttheoverallresultsofourrewritingexperiment.
WefirstdescribethespamworkloadobservedbyourC&Crewrit-
ingproxy.Wethencharacterizetheeffectsoffilteringonthespam
workloadalongthedeliverypathfromworkerbotstouserinboxes,
aswellasthenumberofuserswhobrowsetheadvertisedWebsites
andactonthecontentthere.
5.1 Campaigndatasets
OurstudycoversthreespamcampaignssummarizedinTable1.
The“Pharmacy”campaignisa26-daysample(19activedays)of
anon-goingStormcampaignadvertisinganon-linepharmacy.The
“Postcard”and“AprilFool”campaignsaretwodistinctandserial
instancesofself-propagationcampaigns,whichattempttoinstall
anexecutableontheuser’smachineundertheguiseofbeingpost-
cardsoftware. Foreachcampaign,Figure4showsthenumberof
messagesperhourassignedtobotsformailing.
Storm’sauthorshaveshowngreatcunninginexploitingthecul-
turalandsocialexpectationsofusers—hencetheAprilFoolcam-
paignwasrolledoutforalimitedrunaroundApril1st. OurWeb
sitewasdesignedtomimictheearlierPostcardcampaignandthus
ourdataprobablydoesnotperfectlyreflectuserbehaviorforthis
campaign,butthetwoaresimilarenoughinnaturethatwesurmise
thatanyimpactissmall.
Webegantheexperimentwith8proxybots,ofwhich7survived
untiltheend.OneproxycrashedlateonMarch31.Thetotalnum-
berofworkerbotsconnectedtoourproxieswas75,869.
Figure5showsatimelineoftheproxybotworkload.Thenum-
berofworkersconnectedtoeachproxyisroughlyuniformacross
152
A
B
C
D
E
targeted
addresses
email not
delivered
blocked by
spam filter
ignored
by user
user left site
crawler
converter
Figure6: Thespamconversionpipeline.
S
TAGE
P
HARMACY
P
OSTCARD
A
PRIL
F
OOL
A–SpamTargets
347,590,389
100%
83,655,479
100%
40,135,487
100%
B–MTADelivery(est.)
82,700,000
23.8%
21,100,000
25.2%
10,100,000
25.2%
C–InboxDelivery
—
—
—
—
—
—
D–UserSiteVisits
10,522
0.00303%
3,827
0.00457%
2,721
0.00680%
E–UserConversions
28
0.0000081%
316
0.000378%
225
0.000561%
Table3:Filteringateachstageofthespamconversionpipelinefortheself-propagationandpharmacycampaigns.Percentagesrefer
totheconversionraterelativetoStageA.
allproxies(23workerbotsonaverage), butshowsstrongspikes
correspondingtonewself-propagationcampaigns. Atpeak, 539
workerbotswereconnectedtoourproxiesatthesametime.
Mostworkersonlyconnectedtoourproxiesonce: 78%ofthe
workersonlyconnectedtoourproxiesasingletime,92%atmost
twice,and99%atmostfivetimes. ThemostprolificworkerIP
address,ahostinanacademicnetworkinNorthCarolina, USA,
contactedourproxies269times;furtherinspectionidentifiedthis
asaNATegresspointfor 19 individualinfections. Conversely,
mostworkersdonotconnecttomorethanoneproxy:81%ofthe
workersonlyconnectedtoasingleproxy,12%totwo,3%tofour,
4%connectedtofiveormore,and90workerbotsconnectedtoall
ofourproxies. Onaverage, workerbotsremainedconnectedfor
40minutes,althoughover40%workersconnectedforlessthana
minute.Thelongestconnectionlastedalmost81hours.
The workers were instructed to send postcard spam to a to-
tal of 83,665,479 addresses, of which 74,901,820 (89.53%) are
unique. TheAprilFoolcampaigntargeted38,651,124addresses,
ofwhich36,909,792 (95.49%)areunique. Pharmacyspamtar-
geted347,590,389addresses,ofwhich213,761,147(61.50%)are
unique. Table2shows the15mostfrequentlytargeteddomains
ofthethreecampaigns.Theindividualcampaigndistributionsare
identicalinorderingandtoaprecisionofonetenthofapercentage,
thereforeweonlyshowtheaggregatebreakdown.
5.2 Spamconversionpipeline
Conceptually,webreakdownspamconversion intoapipeline
withfive“filtering”stagesinamannersimilartothatdescribedby
AycockandFriess[6]. Figure6illustratesthispipelineandshows
thetypeoffilteringateachstage.Thepipelinestartswithdelivery
listsoftargete-mailaddressessenttoworkerbots(StageA).For
awiderangeofreasons(e.g.,thetargetaddressisinvalid,MTAs
refusedeliverybecauseofblacklists, etc.),workerswillsuccess-
fullydeliveronlyasubsetoftheirmessagestoanMTA(StageB).
S
PAM
F
ILTER
P
HARMACY
P
OSTCARD
A
PRIL
F
OOL
Gmail
0.00683%
0.00176%
0.00226%
Yahoo
0.00173%
0.000542%
none
Hotmail
none
none
none
Barracuda
0.131%
N/A
0.00826%
Table4: Numberofmessages deliveredto auser’sinbox as
a fraction of those injected for test accounts at free e-mail
providersandacommercialspamfilteringappliance.Thetest
account fortheBarracudaappliancewasnotincludedinthe
Postcardcampaign.
Atthispoint,spamfiltersatthesitecorrectlyidentifymanymes-
sagesasspam,anddropthemorplacethemasideinaspamfolder.
Theremainingmessageshavesurvivedthegauntletandappearin
auser’sinboxasvalidmessages(StageC).Usersmaydeleteor
otherwiseignorethem,butsomeuserswillactonthespam,click
ontheURLinthemessage,andvisittheadvertisedsite(StageD).
Theseusersmaybrowsethesite,butonlyafraction“convert”on
thespam(StageE)byattemptingtopurchaseproducts(pharmacy)
orbydownloadingandrunninganexecutable(self-propagation).
Weshowthespamflowintwoparts,“crawler”and“converter”,
todifferentiatebetweenrealandmasqueradingusers(Section4.4).
Forexample,thedeliverylistsgiventoworkerscontainhoneypot
e-mailaddresses.Workersdeliverspamtothesehoneypots,which
thenusecrawlerstoaccessthesitesreferencedbytheURLinthe
messages(e.g.,ourownSpamscatterproject[3]). Sincewewant
tomeasurethespamconversionrateforactualusers,weseparate
outtheeffectsofautomatedprocesseslikecrawlers—anecessary
aspectofstudyinganartifactthatisalsobeingactivelystudiedby
othergroups[12].
Table3showstheeffectsoffilteringateachstageofthecon-
versionpipelineforboththeself-propagationandpharmaceutical
campaigns.Thenumberoftargetedaddresses(A)issimplytheto-
123
talnumberofaddressesonthedeliverylistsreceivedbytheworker
botsduringthemeasurementperiod,excludingthetestaddresses
weinjected.
Weobtain the number of messages delivered to an MTA (B)
byrelyingon delivery reportsgeneratedbytheworkers. Unfor-
tunately,anexactcountofsuccessfullydeliveredmessagesisnot
possiblebecauseworkersfrequentlychangeproxiesorgooffline,
causingbothextraneous(resultingfromaprevious,non-interposed
proxysession)andmissingdeliveryreports.Wecan,however,es-
timatetheaggregatedeliveryratio(B/A)foreachcampaignusing
thesuccessratioofallobserveddeliveryreports.Thisratioallows
ustothenestimatethenumberofmessagesdeliveredtotheMTA
andeventodosoonaper-domainbasis.
Thenumber ofmessagesdelivered toa user’sinbox (C)isa
much harder valuetoestimate. Wedo notknowwhatspamfil-
tering,ifany,isusedbyeachmailprovider,andthenbyeachuser
individually,andthereforecannotreasonablyestimatethisnumber
intotal. Itispossible,however, todeterminethisnumberforin-
dividualmailprovidersorspamfilters. Thethreemailproviders
andthespamfilteringapplianceweusedinthisexperimenthada
methodforseparatingdeliveredmailsinto“junk”andinboxcat-
egories. Table4givesthenumberofmessagesdeliveredauser’s
inboxforthefreee-mailproviders,whichtogetheraccountedfor
about16.5%ofaddressestargetedbyStorm(Table2),aswellas
ourdepartment’scommercialspamfilteringappliance.Itisimpor-
tanttonotethattheseareresultsfromonespamcampaignovera
shortperiodoftimeandshouldnotbeusedasmeasuresoftherel-
ativeeffectivenessforeachservice.Thatsaid,weobservethatthe
popularWebmailprovidersalldoaveryagoodjobatfilteringthe
campaignsweobserved,althoughitiscleartheyusedifferentmeth-
odstogetthere(forexample,HotmailrejectsmostStormspamat
theMTA-level,whileGmailacceptsasignificantfractiononlyto
filteritlaterasjunk).
Thenumberofvisits(D)isthenumberofaccessestoourem-
ulatedpharmacyandpostcardsites,excludinganycrawlersasde-
terminedusingthemethodsoutlinedinSection4.2. Wenotethat
crawlerrequestscamefromasmallfractionofhostsbutaccounted
forthemajorityofallrequeststoourWebsites.Forthepharmacy
site,forinstance,ofthe11,720uniqueIPaddressesseenaccessing
thesitewithavaliduniqueidentifier,only10.2%wereblacklisted
ascrawlers. Incontrast,55.3%ofalluniqueidentifiersusedinre-
questsoriginatedfromthesecrawlers. Forallnon-imagerequests
madetothesite,87.43%weremadebyblacklistedIPaddresses.
Thenumberof conversions(E) is thenumberof visitsto the
purchasepageofthepharmacysite,orthenumberofexecutionsof
thefakeself-propagationprogram.
OurresultsforStormspamcampaignsshowthatthespamcon-
versionrateisquitelow.Forexample,outof350millionpharmacy
campaigne-mailsonly28conversionsresulted(andnocrawlerever
completedapurchasesoerrorsincrawlerfilteringplaysnorole).
However,averylowconversionratedoesnotnecessaryimplylow
revenueorprofitability.Wediscusstheimplicationsoftheconver-
sionrateonthespamconversionpropositionfurtherinSection8.
5.3 Timetoclick
Theconversionpipelineshowswhatfractionofspamultimately
resulted visits to theadvertised sites. However, itdoes notre-
flectthelatencybetweenwhenthespamwassentandwhenauser
clickedonit. Thelongerittakesuserstoact,thelongerthescam
hostinginfrastructurewillneedtoremainavailabletoextractrev-
enuefromthespam[3]. Putanotherway,howlongdoesaspam-
advertisedsiteneedtobeavailabletocollectitspotentialrevenue?
1s
10s
1min
10min
1h
6h
1d
1w
1m
0
0.2
0.4
0.6
0.8
1
Time to click
Fraction of clicks
Crawlers
Users
Converters
Figure7: Time-to-clickdistributionsforaccessestothephar-
macysite.
Figure7showsthecumulativedistributionofthe“time-to-click”
for accessesto thepharmacy site. Thetime-to-clickisthetime
fromwhenspamissent(whenaproxyforwardsaspamworkload
toaworkerbot)towhenauser“clicks”ontheURLinthespam
(whenahostfirstaccessestheWebsite). Thegraphshowsthree
distributionsfortheaccessesbyallusers,theuserswhovisitedthe
purchasepage(“converters”),andtheautomatedcrawlers(14,716
such accesses). Notethatwefocusonthepharmacysitesince,
absentauniqueidentifier,wedonothaveamechanismtolinkvisits
totheself-propagationsitetospecificspammessagesandtheirtime
ofdelivery.
Theuserandcrawlerdistributionsshowdistinctlydifferentbe-
havior. Almost30% ofthecrawleraccesses arewithin20 sec-
onds ofworker botssending spam. Thisbehavior suggeststhat
thesecrawlersareconfiguredtoscansitesadvertisedinspamim-
mediatelyupondelivery. Another10%ofcrawleraccesseshave
atime-to-clickof1day,suggestingcrawlersconfiguredtoaccess
spam-advertisedsitesperiodicallyinbatches.Incontrast,only10%
oftheuserpopulationaccessesspamURLsimmediately,andthe
remainingdistributionissmoothwithoutanydistinctmodes. The
distributionsforallusersanduserswho“convert”areroughlysimi-
lar,suggestinglittlecorrelationbetweentime-to-clickandwhether
auser visitingasitewillconvert. Whilemostuser visits occur
within thefirst24 hours, 10% oftimes-to-clickareaweek to a
month,indicatingthatadvertisedsitesneedtobeavailableforlong
durationstocapturefullrevenuepotential.
6. EFFECTSOFBLACKLISTING
Amajoreffectontheefficacyofspamdeliveryistheemploy-
mentbynumerousISPsofaddress-basedblacklistingtorejecte-
mailfromhostspreviouslyreportedassourcingspam. Toassess
theimpactofblacklisting, during thecourseofour experiments
wemonitoredtheCompositeBlockingList(CBL)[1],ablacklist
sourceusedbytheoperatorsofsomeofourinstitutions. Atany
giventimetheCBLlistsontheorderof4–6millionIPaddresses
thathavesente-mailtovariousspamtraps.Wewereabletomonitor
theCBLfromMarch21–April2,2008,fromthestartofthePhar-
macycampaignuntiltheendoftheAprilFoolcampaign.Although
themonitoringdoesnotcoverthefullextentofallcampaigns,we
believeourresultstoberepresentativeoftheeffectsofCBLduring
thetimeframeofourexperiments.
10834
Figure9: Geographiclocationsof thehoststhat“convert” onspam: the541 hoststhatexecutetheemulatedself-propagation
program(lightgrey),andthe28hoststhatvisitthepurchasepageoftheemulatedpharmacysite(black).
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Delivery Rate Prior to Blacklisting
Delivery Rate Post Blacklisting
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
lll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
lll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
lll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
lll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
lll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
lll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
lll
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
ll
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l