A Task Allocation Method for Stream Processing with Recovery Latency Constraint

在线阅读 下载PDF 导出详情
摘要 Streamprocessingapplicationscontinuouslyprocesslargeamountsofonlinestreamingdatainrealtimeornearrealtime.Theyhavestrictlatencyconstraints.However,thecontinuousprocessingmakesthemvulnerabletoanyfailures,andtherecoveriesmayslowdowntheentireprocessingpipelineandbreaklatencyconstraints.Theupstreambackupschemeisoneofthemostwidelyappliedfault-tolerantschemesforstreamprocessingsystems.Itintroducescomplexbackupdependenciestotasks,whichincreasesthedifficultyofcontrollingrecoverylatencies.Moreover,whendependenttasksarelocatedonthesameprocessor,theyfailatthesametimeinprocessor-levelfailures,bringingextrarecoverylatenciesthatincreasetheimpactsoffailures.Thispaperstudiestherelationshipbetweenthetaskallocationandtherecoverylatencyofastreamprocessingapplication.Wepresentacorrelatedfailureeffectmodeltodescribetherecoverylatencyofastreamtopologyinprocessor-levelfailuresunderataskallocationplan.Weintroducearecovery-latencyawaretaskallocationproblem(RTAP)thatseekstaskallocationplansforstreamtopologiesthatwillachieveguaranteedrecoverylatencies.WediscussthedifferencebetweenRTAPandclassictaskallocationproblemsandpresentaheuristicalgorithmwithacomputationalcomplexityofO(nlog2n)tosolvetheproblem.Extensiveexperimentswereconductedtoverifythecorrectnessandeffectivenessofourapproach.Itimprovestheresourceusageby15%-20%onaverage.
机构地区 不详
出版日期 2018年06月16日(中国期刊网平台首次上网日期,不代表论文的发表时间)
  • 相关文献