论文（设计）基于蚁群算法的副本选择策略的研究.doc

基于蚁群算法的副本选择策略的研究王韶君(运河高等师范学校 计算机组)摘 要: 副本技术是数据网格中的关键技术。它能大大减少传输延迟，提高数据访问和处理的效率。分布着大量数据和计算能力的数据网格环境中,研究副本选择策略尤为重要。本文研究蚁群算法的原理同时分析了影响副本选择性能的主要因素，利用这些参考因素设计了基于蚁群算法的副本选择策略，并对这个新算法进行了分析和实现，经仿真平台实验，结果表明该算法可以减少数据访问延迟及带宽消耗, 并有效做到网格中存储节点间的负载平衡，提高数据的访问速度。关键词: 副本选择；蚁群算法；p2p；OptorSim中图分类号：TP393 文献标志码：DResearch of ant algorithm to replica selection in p2pWANG Shaojun （Team of computer science&application, Yunhe Higher Nomal School）Abstract: Replica is the one of the most important key technique in data grid. They are able to reduce the delay of data transfer and improve the efficiency of data access and processing. Replication of data is an important method to improve the availability of applications in distributed dataintensive Grid environment. how to choose the replicas is a key factor to that influence the performance of replica selection greatly. Within this paper, the strategy is analyzed and implemented. Through using simulator, test results that this new ant algorithm can reduce data access latency, decrease bandwidth consumption and distribute storage site load, improve data access speed。Key words: replica selection; ant algorithm; p2p; OptorSim0 引言网格系统是一种无缝、集成的资源共享和协作环境。在网格系统中,需要将需求量大的资源复制到多个站点上提供服务。副本选择是数据网格中非常重要的基础，即是从分布在网格中的众多副本中选择一个副本的过程。选择依靠很多因素，如数据的放置、数据的大小、网络的带宽和延迟、用户和服务器间的网络状态、副本所在节点的负载情况及磁盘I/O读取速度等。蚁群算法12的正反馈性和协同性、隐含的并行性使其适用于分布式系统，而其具有的可扩展性使其很适合于网络结构和副本动态改变的数据网格环境。本文提出的基于蚁群的副本优化选择算法针对大规模数据密集型网格环境, 既可以做到根据历史记录进行副本选择的预测, 又可以有效做到副本存储节点的负载动态平衡。1 基于蚁群算法的副本选择策略蚁群算法是利用与环境的动态交互获得反馈信息调整自我, 以期逐步获得最佳解。蚁群算法已被广泛应用到许多最优化问题中, 如TSP 分配问题、网络路由、任务高度及着色问题。在数据网格中, 选择一个最佳副本同样是最优化问题, 因此基于蚁群算法的副本选择策略3在理论上具有其可行性。蚁群算法是一种智能优化仿生算法4，其显著特点为：其原理是一种正反馈机制或称增强型学习系统，它通过信息素的不断更新达到最终收敛于最优路径上。它是一种分布式的优化方法，不仅适合目前的串行计算机，而且适合未来的并行计算机。它是一种全局优化的方法，不仅可用于求解单目标优化问题，而且可用于求解多目标优化问题。它是一种启发式算法。1.1 数据副本选择的因素副本选择算法的优劣很大程度上取决于对影响副本选择因素参数的选取。这些因素成为副本选择的主要依据, 主要包括以下几个方面:l 磁盘I/O 传输。 针对副本选择策略而言, 主要指磁盘读取时间。低的读取时间能降低数据副本的远程访问时间。l SE 的读取速度。针对副本选择策略而言，主要指磁盘读取时间。我们忽略存储方式不同所引起的时间上的差异，认为所有的文件存储格式都是相同的。除此以外，在存储格式相同的情况下，能影响数据读取速度的还有硬盘的转数，硬盘的转数不同，读取的速度自然不同。l CE的计算速度。它反映了CE处理任务的快慢程度。CE的处理速度对读取的影响不会很大，可以忽略这个特征值。l 网络状态。在进行副本选择时, 通用的作法是选择最小延迟的链路进行数据访问。传输带宽决定了数据的传输速率, 因此网络中最大可用传输带宽可成为副本选择的一个依据, 并且平均传输带宽可以帮助预测对某一特定副本的访问情况。l 存储副本节点的负载情况: 如果许多个任务需要访问某一网格节点上的相同数据副本, 则该节点访问负载也是影响数据远程访问时间的重要因素之一。1.2 基于蚁群算法的副本选择方法当在数据网格中产生新的数据副本时，需要提供其网络带宽w、存储副本节点的负载完成率y、磁盘I/O读取速度n，以及副本响应时间d等基本参数，据此可以初始化该副本的各种信息素(每种信息素对应于一个基本参数)。其中，在初始化各种信息素之前，先定义副本中各基本参数的阈值wmax = b0、smax = f0、jmin = r0、dmin = d0 并规定：如果副本的各参数值超过阈值，则统一以阈值进行后面的计算。然后应用以下公式对该副本进行初始化： 网络带宽w的信息素 存储副本节点的负载完成率s的信息素 磁盘I/O读取速度的r信息素 (1)副本响应预测时间的d信息素则副本参数包j的初始化总信息素为 (2)，其中、分别表示网络带宽信息素、存储副本节点的负载完成率信息素、磁盘I/O读取速度信息素以及副本响应时间信息素在该副本总信息素中所占的比重。在蚁群算法的基础上对蚁群算法进行改进, 设计出基于蚁群算法的副本选择策略。通过信息素Pheromone对副本进行选择，信息素决定了数据副本的好坏。应用以下公式对该副本的信息素进行初始化: (3) 其中r代表对数据副本的读取速度；f表示副本文件的大小；Sj表示参数包的大小。当有新副本被创建, 某副本被删除, 部分副本被成功选取或失败返回时, 副本信息素都会随之改变: (4)其中是信息素改变量；表示信息素持久性(取0.8) , 表示信息素的挥发性。当一个副本被选择且对它进行远程访问时, 它的信息素会减小, 减小量 ( 在后文中以k代表) , 它意味着当副本已被某一任务所选择时,其信息素降低, 其它任务选择该副本的概率会减小。当对副本j进行远程访问成功时, , 其中是奖励因子(取值0.8)；当副本j访问失败时, ,其中为惩罚因子( 取值1.1)。从这两个公式中可以看出, 某一副本的访问成功返回, 其信息素会增加。副本信息素改变时, 该副本被选择的概率随之改变, 选择概率公式定义如下: (5) 其中代表副本j的信息素浓度；表示该副本的固有属性,即副本j的初始信息素；和分别表示信息素和副本固有属性的重要性, 以这两个参数对副本动态变化的信息素和其固有属性的重要程度进行控制，两个参数分别取值0.5。因此, 选择概率由信息素和副本固有属性共同决定,是两者的权衡折衷。副本的选择过程依据以上概率公式进行, 但并不是概率最大的副本一定被选中, 原因在于还需要考虑副本所在节点的负载情况。因此采用随机概率与选择概率进行匹配的方法。其匹配过程如下:假设符合要求且需要在其中进行选择的副本共有k个,每个副本被选择的概率分别为p(i) (iZ+,1ik) , 则令p(0)=0, ps(j) =p(0)+p(1) + +p(j) (jZ,0jk)。利用随机函数产生一个随机数rdm, 若ps(j-1)rdmps(j) , 则选择第j个副本，如：图1 图1 概率分配过程说明图 图2 Optorsim体系结构Figure 1 Probability distribution process description diagram Figure 2 OptorSim Architecture1.3 算法流程基于上一节所讨论的公式及思想, 基于蚁群方法的副本选择算法过程如下:(1) 初始化过程: 利用信息素初始化公式, 即公式(3) 初始化各副本的信息素。(2) 根据客户需求从副本目录中查找出对应数据集的所有可用副本, 依据每个副本的选择概率来选择副本。具体做法如(3)-(6)所描述。(3) 利用选择概率公式, 即公式(5)算出选择每个副本的概率。(4) 概率匹配过程: 利用随机概率与上述概率进行匹配, 选择概率相匹配的副本。(5) 信息素更新过程: 利用副本被选择后副本信息素更新公式, 即公式(4) 对该副本信息素更新。根据选择结果, 与对应副本所在节点建立连接, 进行副本传输, 同时进行信息素更新。若副本传输成功, 则利用奖励公式将该副本的信息素更新；若副本传输失败, 则利用惩罚公式将该副本的信息素更新。(6) 满足程度终止条件, 则算法结束, 否则转(4) 。基于蚁群算法的副本选择策略表述为：InitialiseLoop /* An iteration */Each ant is positioned on a starting node.Loop /* A step */Each ant applies a state transition rule to incrementallybuild a solution and a local pheromone updating ruleUntil all ants have built a complete solutionA global pheromone updating rule is appliedUntil End condition.2 仿真实验及结果分析2.1 仿真实验由于数据网格是一个高度动态的环境，因此在进行数据副本创建的时候，中间件应该处理数据副本的状态变化。特别地，在任务调度时，在选择用哪个副本来为任务服务的时候，即在进行副本选择的时候，网格优化服务应该避免做不可挽回的决定。因此在触发副本的创建和删除时，副本优化器非常重要，副本优化器被嵌入在副本管理器中，用来对副本的创建、删除以及选择进行优化控制。算法在网格仿真模拟器OptorSim（如图2）中具体实现。该模拟器旨在研究动态数据副本策略。它提供了一系列副本创建及选择方法并就算法的性能给出仿真数据。其目的在于研究典型数据网格中的复杂动态特性及在数据网格环境中对副本优化算法的性能做出评价。在实验过程中，请求副本访问的客户端被随机地分布在网格中。来自各个独立客户端的请求以并行的方式执行。由于同一副本在某一特定时刻只能接受一个客户请求，如果多个客户同时请求相同副本，则必然存在一些客户端出现等待直到该副本可用为止。本文设计了三种情景就算法的运行结果进行评估。情景一：在数据网格中存在少量的副本；情景二：网格环境中存在许多副本，但几乎没有客户端访问相同的副本；情景三：网格环境中存在大量副本，同时许多客户端请求相同的数据副本。在这三种情景下，均以副本的平均访问时间作为算法性能的评价指标。22 结果分析与讨论为得到基于蚁群方法的副本选择算法的执行效率, 将该算法的执行结果与OptorSim 仿真器中的SimpleOptimiser 算法进行对比分析。之所以与SimpleOptimiser 算法进行比较, 因为SimpleOptimiser 算法不进行副本的动态创建, 可以排除因副本创建所产生的对副本选择策略的影响；并且SimpleOptimiser 算法始终动态选择数据网格中访问代价最小的副本, 每个客户端请求均对数据网格中副本代价进行评估, 选择总体访问代价最小者作为需要进行访问的副本。这种策略在多种其它副本选择算法中具有典型性。三种不同算法的副本平均访问时间情况如图所示: 图3 情景一中副本访问时间 图4 情景二中副本访问时间Figure 3 replica access time in 1 case Figure 4 replica access time in 2 case 图5 情景三中副本访问时间 Figure 5 replica access time in 3 case 从如上的结果图中可以发现：在情景一中两种算法的执行效果并没有太大差异，由于网格中副本数量较少，两种算法在每次选择时基本上选择相同的副本；但随着网格中副本数量的增加，基于蚁群方法的选择算法逐渐显示其优越性，尤其在多个客户端请求相同副本的情况下。结果同时也证实了前文的理论分析，情景三中新算法性能最佳，因有效做到副本所在节点的负载平衡，从而使副本的平均访问时间减小。3 结论本文针对数据密集型的数据网格中副本选择的优化问题,提出了基于蚁群方法的副本选择算法。在网格仿真器OptorSim中对该算法进行了评估。通过扩展仿真器的相应模块进行仿真实验。实验数据显示该算法在数据密集型数据网格中具有良好性能, 能减少副本的平均时间, 降低网络的带宽消耗以及有效做到副本所在节点间的负载平衡。参考文献：1 DorigoM,Gambardells LM. Ant colonied for the traveling salesman problemJ。Biosystems, 1997,43(2):73-81.2 Naor Z, Levy H. LATS:a load- adaptive threshold scheme for tracking mobile users J. IEEE/ACM Transactions on Networking, 1999,7(6):808-817.3 Nan DS, YounC H. An efficient replication scheme for data grids C. Proceedings of the IEEE International Conference on Cluster Computing, 2004.4 HOMNAN, WATTANACHAI, BENJAPOLAKUL. The performance analysis for fuzzy inference system- based adaptive soft handoff thresholdsC/2004 IEEE Region 10 Conference. TENCON: IEEE Computer Society, 2004.5 DU Zhihui. Grid computing M. Beijing: Tsinghua University Press,2002.6 Krishnamurthy S, Sanders W H, Cukier M. Performance evaluation of a probabilistic replica selection algorithmC/Proceedings of the 5th international workshop on Software and performance, Spain, 2005.Editor's note: Judson Jones is a meteorologist, journalist and photographer. He has freelanced with CNN for four years, covering severe weather from tornadoes to typhoons. Follow him on Twitter: jnjonesjr (CNN) - I will always wonder what it was like to huddle around a shortwave radio and through the crackling static from space hear the faint beeps of the world's first satellite - Sputnik. I also missed watching Neil Armstrong step foot on the moon and the first space shuttle take off for the stars. Those events were way before my time.As a kid, I was fascinated with what goes on in the sky, and when NASA pulled the plug on the shuttle program I was heartbroken. Yet the privatized space race has renewed my childhood dreams to reach for the stars.As a meteorologist, I've still seen many important weather and space events, but right now, if you were sitting next to me, you'd hear my foot tapping rapidly under my desk. I'm anxious for the next one: a space capsule hanging from a crane in the New Mexico desert.It's like the set for a George Lucas movie floating to the edge of space.You and I will have the chance to watch a man take a leap into an unimaginable free fall from the edge of space - live.The (lack of) air up there Watch man jump from 96,000 feet Tuesday, I sat at work glued to the live stream of the Red Bull Stratos Mission. I watched the balloons positioned at different altitudes in the sky to test the winds, knowing that if they would just line up in a vertical straight line "we" would be go for launch.I feel this mission was created for me because I am also a journalist and a photographer, but above all I live for taking a leap of faith - the feeling of pushing the envelope into uncharted territory.The guy who is going to do this, Felix Baumgartner, must have that same feeling, at a level I will never reach. However, it did not stop me from feeling his pain when a gust of swirling wind kicked up and twisted the partially filled balloon that would take him to the upper end of our atmosphere. As soon as the 40-acre balloon, with skin no thicker than a dry cleaning bag, scraped the ground I knew it was over.How claustrophobia almost grounded supersonic skydiverWith each twist, you could see the wrinkles of disappointment on the face of the current record holder and "capcom" (capsule communications), Col. Joe Kittinger. He hung his head low in mission control as he told Baumgartner the disappointing news: Mission aborted.The supersonic descent could happen as early as Sunday.The weather plays an important role in this mission. Starting at the ground, conditions have to be very calm - winds less than 2 mph, with no precipitation or humidity and limited cloud cover. The balloon, with capsule attached, will move through the lower level of the atmosphere (the troposphere) where our day-to-day weather lives. It will climb higher than the tip of Mount Everest (5.5 miles/8.85 kilometers), drifting even higher than the cruising altitude of commercial airliners (5.6 miles/9.17 kilometers) and into the stratosphere. As he crosses the boundary layer (called the tropopause), he can expect a lot of turbulence.The balloon will slowly drift to the edge of space at 120,000 feet (22.7 miles/36.53 kilometers). Here, "Fearless Felix" will unclip. He will roll back the door.Then, I would assume, he will slowly step out onto something resembling an Olympic diving platform.Below, the Earth becomes the concrete bottom of a swimming pool that he wants to land on, but not too hard. Still, he'll be traveling fast, so despite the distance, it will not be like diving into the deep end of a pool. It will be like he is diving into the shallow end.Skydiver preps for the big jumpWhen he jumps, he is expected to reach the speed of sound - 690 mph (1,110 kph) - in less than 40 seconds. Like hitting the top of the water, he will begin to slow as he approaches the more dense air closer to Earth. But this will not be enough to stop him completely.If he goes too fast or spins out of control, he has a stabilization parachute that can be deployed to slow him down. His team hopes it's not needed. Instead, he plans to deploy his 270-square-foot (25-square-meter) main chute at an altitude of around 5,000 feet (1,524 meters).In order to deploy this chute successfully, he will have to slow to 172 mph (277 kph). He will have a reserve parachute that will open automatically if he loses consciousness at mach speeds.Even if everything goes as planned, it won't. Baumgartner still will free fall at a speed that would cause you and me to pass out, and no parachute is guaranteed to work higher than 25,000 feet (7,620 meters).It might not be the moon, but Kittinger free fell from 102,800 feet in 1960 - at the dawn of an infamous space race that captured the hearts of many. Baumgartner will attempt to break that record, a feat that boggles the mind. This is one of those monumental moments I will always remember, because there is no way I'd miss this.