集群計算機系統(tǒng)設(shè)計.doc
約60頁DOC格式手機打開展開
集群計算機系統(tǒng)設(shè)計,碩士論文頁數(shù)：60字?jǐn)?shù)：27993摘要通信性能是影響集群系統(tǒng)實際并行處理效率的關(guān)鍵，因此高性能通信系統(tǒng)的研究在整個集群系統(tǒng)中占有非常重要的地位，并且一直是近幾年國內(nèi)外研究的熱點。myrinet是集群系統(tǒng)普遍采用的互連網(wǎng)絡(luò)，具有高帶寬、低延遲、擴展性好等優(yōu)點?；谙鬟f的用戶態(tài)通信協(xié)議gm充分利用了m...
內(nèi)容介紹
此文檔由會員 915 發(fā)布
集群計算機系統(tǒng)設(shè)計
碩士論文
頁數(shù)：60 字?jǐn)?shù)：27993
摘 要
通信性能是影響集群系統(tǒng)實際并行處理效率的關(guān)鍵，因此高性能通信系統(tǒng)的研究在整個集群系統(tǒng)中占有非常重要的地位，并且一直是近幾年國內(nèi)外研究的熱點。Myrinet是集群系統(tǒng)普遍采用的互連網(wǎng)絡(luò)，具有高帶寬、低延遲、擴展性好等優(yōu)點?；谙鬟f的用戶態(tài)通信協(xié)議GM充分利用了Myrinet硬件結(jié)構(gòu)的特點，為上層通信系統(tǒng)或用戶應(yīng)用程序提供高性能高可靠的通信服務(wù)。
本論文針對集群通信的需要，主要貢獻(xiàn)如下：
1．綜述了集群通信的研究目標(biāo)以及已有的實現(xiàn)技術(shù)?；趯σ环N成熟的集群用戶態(tài)通信協(xié)議GM的分析，針對GM的限制以及可以改進(jìn)的空間，提出了集群通信面臨的新問題。
2．設(shè)計并實現(xiàn)了一種基于共享內(nèi)存的集群系統(tǒng)本地通信協(xié)議FMP_Shmem，優(yōu)化了CLUMPs（CLUsters of MultiProcessors）的本地通信性能。CLUMPs由SMP結(jié)點互連而成。由于SMP機器的性能價格比較高，CLUMPs成為超級計算的潮流。FMP_Shmem充分利用了SMP結(jié)點的硬件特點，采用共享內(nèi)存機制在結(jié)點內(nèi)部交換數(shù)據(jù)。它為每個進(jìn)程分配獨立的發(fā)送隊列和接收隊列，避免了進(jìn)程間的互斥操作。相比較不采用共享內(nèi)存機制的GM本地通信性能（延遲26μs，帶寬105MB/s），F(xiàn)MP_Shmem的本地通信測試結(jié)果可達(dá)到延遲2.84μs，帶寬272MB/s，性能優(yōu)化的幅度相當(dāng)大。
3．提出了MPICH-MNC——基于多通道網(wǎng)絡(luò)通信的高性能MPI實現(xiàn)方案。MNC協(xié)議是GM協(xié)議的擴展，針對單通道網(wǎng)絡(luò)通信性能的提高空間已經(jīng)不大，提出了多通道網(wǎng)絡(luò)通信的思想。MNC協(xié)議能夠控制物理層的多條網(wǎng)絡(luò)通道并行傳輸數(shù)據(jù)。MPICH-MNC以MNC協(xié)議為基礎(chǔ)，向用戶提供規(guī)范的MPI編程接口，讓底層通信協(xié)議MNC的實現(xiàn)細(xì)節(jié)對用戶完全透明，原有的MPI應(yīng)用程序不必進(jìn)行任何修改即可在MPICH-MNC平臺上運行。MPICH-MNC的設(shè)計目標(biāo)是提高數(shù)據(jù)傳輸?shù)膸捯约皽p少長消息的傳輸延遲。
關(guān)鍵詞：集群，用戶層通信，CLUMPs，共享內(nèi)存，多通道網(wǎng)絡(luò)通信
Abstract
The communication performance is the critical factor which significantly affects the actual efficiency of parallel processing on clusters, therefore how to achieve high-performance communication systems plays an important role in the research of clusters and has been a hot research topic in recent years. Myrinet is a kind of SAN with high bandwidth, low latency and scalability. It is widely used in cluster systems. GM, a user-level message passing protocol, exploits characteristics of Myrinet, provides data transfer of high performance and reliance for the upper communication layer or applications.
This paper contributes a lot to clusters communication as follows:
1. The research goal and existing technology of clusters communication are presented. Based on the analysis of GM, a well-rounded user-level cluster communication protocol, several new challenges to clusters communication are posed.
2. Designs and implements FMP_Shmem, a local communication protocol based on shared memory for CLUMPs(CLUsters of MultiProcessors), improves the local communication performance of CLUMPs. CLUMPs are constructed with SMPs. CLUMPs have become the trend of super computing due to their cost-performance rates. Based on shared memory, FMP_Shmem allocates one receiving queue and one transmitting queue independently for each process. Such scheme avoids the mutually exclusive operations among processes and achieves considerable performance (the latency is 2.84μs and the bandwidth is as high as 272MB/s).
3. A high performance MPI implementation scheme based on multi-networking communication is presented, known as MPICH-MNC. The MNC protocol extends GM with the support to multi-networking. MNC controls several physical links to perform parallel communication. MPICH-MNC is based on the MNC protocol, provides uniform MPI programming interfaces to users, totally shields the implementation details of low-layer protocols. The existing MPI applications can be ported to MPICH-MNC without any modifications. The design goal of MPICH-MNC is improving data transfer rate and reducing the transfer latency of long message.
Keywords: Clusters, User-level communication, CLUMPs, Shared memory, Multi-networking communication
目 錄
摘 要 I
ABSTRACT II
第一章 引言 1
1.1 課題研究的目的與意義 1
1.2 國內(nèi)外研究現(xiàn)狀 1
1.3 論文各部分的內(nèi)容安排 3
第二章 集群通信與GM協(xié)議 4
2.1 集群通信系統(tǒng)概況 4
2.1.1高性能可擴展集群及其通信系統(tǒng) 4
2.1.2集群通信系統(tǒng)體系結(jié)構(gòu) 6
2.1.3集群通信系統(tǒng)研究目標(biāo) 7
2.1.4常用集群系統(tǒng)通信技術(shù) 8
2.1.5幾個有代表性的集群通信系統(tǒng) 11
2.2 基于Myrinet的GM協(xié)議 14
2.2.1 Myrinet簡介 14
2.2.2 GM協(xié)議分析 16
2.2.3 Myrinet的性能 18
第三章 基于共享內(nèi)存的CLUMPS結(jié)點本地通信優(yōu)化 20
3.1 CLUMPs的結(jié)構(gòu) 20
3.2 CLUMPs要解決的兩個問題 21
3.2.1 互連網(wǎng)絡(luò)仍是影響集群通信性能的瓶頸 21
3.2.2基于消息傳遞的本地通信性能受限 22
3.3 CLUMPs結(jié)點內(nèi)部的共享內(nèi)存通信 23
3.3.1 AM-II的本地通信機制 23
3.3.2 FMP_Shmem概述 24
3.3.3 FMP_Shmem的數(shù)據(jù)結(jié)構(gòu)與算法描述 25
3.3.4 FMP_Shmem的性能 26
第四章 支持多通道網(wǎng)絡(luò)通信的高性能MPI 30
4.1 支持多通道網(wǎng)絡(luò)通信的集群底層通信協(xié)議MNC 30
4.1.1 多通道網(wǎng)絡(luò)技術(shù) 30
4.1.2 支持多通道網(wǎng)絡(luò)通信的MNC協(xié)議 31
4.2 MPI標(biāo)準(zhǔn)與MPICH 33
4.2.1 MPI與MPICH概述 33
4.2.2 MPICH的體系結(jié)構(gòu) 35
4.2.3 抽象設(shè)備接口（ADI） 36
4.2.4 Channel接口 37
4.3 MPICH-MNC的設(shè)計方案 40
4.3.1 配置文件 40
4.3.2 主要函數(shù)的實現(xiàn) 41
4.4 MPI與PVM的比較 43
4.4.1 PVM概述 43
4.4.2 PVM-GM的實現(xiàn)及其與MPI的性能對比 44
第五章 結(jié)論與展望 47
參考文獻(xiàn) 49
碩士論文
頁數(shù)：60 字?jǐn)?shù)：27993
摘 要
通信性能是影響集群系統(tǒng)實際并行處理效率的關(guān)鍵，因此高性能通信系統(tǒng)的研究在整個集群系統(tǒng)中占有非常重要的地位，并且一直是近幾年國內(nèi)外研究的熱點。Myrinet是集群系統(tǒng)普遍采用的互連網(wǎng)絡(luò)，具有高帶寬、低延遲、擴展性好等優(yōu)點?；谙鬟f的用戶態(tài)通信協(xié)議GM充分利用了Myrinet硬件結(jié)構(gòu)的特點，為上層通信系統(tǒng)或用戶應(yīng)用程序提供高性能高可靠的通信服務(wù)。
本論文針對集群通信的需要，主要貢獻(xiàn)如下：
1．綜述了集群通信的研究目標(biāo)以及已有的實現(xiàn)技術(shù)?；趯σ环N成熟的集群用戶態(tài)通信協(xié)議GM的分析，針對GM的限制以及可以改進(jìn)的空間，提出了集群通信面臨的新問題。
2．設(shè)計并實現(xiàn)了一種基于共享內(nèi)存的集群系統(tǒng)本地通信協(xié)議FMP_Shmem，優(yōu)化了CLUMPs（CLUsters of MultiProcessors）的本地通信性能。CLUMPs由SMP結(jié)點互連而成。由于SMP機器的性能價格比較高，CLUMPs成為超級計算的潮流。FMP_Shmem充分利用了SMP結(jié)點的硬件特點，采用共享內(nèi)存機制在結(jié)點內(nèi)部交換數(shù)據(jù)。它為每個進(jìn)程分配獨立的發(fā)送隊列和接收隊列，避免了進(jìn)程間的互斥操作。相比較不采用共享內(nèi)存機制的GM本地通信性能（延遲26μs，帶寬105MB/s），F(xiàn)MP_Shmem的本地通信測試結(jié)果可達(dá)到延遲2.84μs，帶寬272MB/s，性能優(yōu)化的幅度相當(dāng)大。
3．提出了MPICH-MNC——基于多通道網(wǎng)絡(luò)通信的高性能MPI實現(xiàn)方案。MNC協(xié)議是GM協(xié)議的擴展，針對單通道網(wǎng)絡(luò)通信性能的提高空間已經(jīng)不大，提出了多通道網(wǎng)絡(luò)通信的思想。MNC協(xié)議能夠控制物理層的多條網(wǎng)絡(luò)通道并行傳輸數(shù)據(jù)。MPICH-MNC以MNC協(xié)議為基礎(chǔ)，向用戶提供規(guī)范的MPI編程接口，讓底層通信協(xié)議MNC的實現(xiàn)細(xì)節(jié)對用戶完全透明，原有的MPI應(yīng)用程序不必進(jìn)行任何修改即可在MPICH-MNC平臺上運行。MPICH-MNC的設(shè)計目標(biāo)是提高數(shù)據(jù)傳輸?shù)膸捯约皽p少長消息的傳輸延遲。
關(guān)鍵詞：集群，用戶層通信，CLUMPs，共享內(nèi)存，多通道網(wǎng)絡(luò)通信
Abstract
The communication performance is the critical factor which significantly affects the actual efficiency of parallel processing on clusters, therefore how to achieve high-performance communication systems plays an important role in the research of clusters and has been a hot research topic in recent years. Myrinet is a kind of SAN with high bandwidth, low latency and scalability. It is widely used in cluster systems. GM, a user-level message passing protocol, exploits characteristics of Myrinet, provides data transfer of high performance and reliance for the upper communication layer or applications.
This paper contributes a lot to clusters communication as follows:
1. The research goal and existing technology of clusters communication are presented. Based on the analysis of GM, a well-rounded user-level cluster communication protocol, several new challenges to clusters communication are posed.
2. Designs and implements FMP_Shmem, a local communication protocol based on shared memory for CLUMPs(CLUsters of MultiProcessors), improves the local communication performance of CLUMPs. CLUMPs are constructed with SMPs. CLUMPs have become the trend of super computing due to their cost-performance rates. Based on shared memory, FMP_Shmem allocates one receiving queue and one transmitting queue independently for each process. Such scheme avoids the mutually exclusive operations among processes and achieves considerable performance (the latency is 2.84μs and the bandwidth is as high as 272MB/s).
3. A high performance MPI implementation scheme based on multi-networking communication is presented, known as MPICH-MNC. The MNC protocol extends GM with the support to multi-networking. MNC controls several physical links to perform parallel communication. MPICH-MNC is based on the MNC protocol, provides uniform MPI programming interfaces to users, totally shields the implementation details of low-layer protocols. The existing MPI applications can be ported to MPICH-MNC without any modifications. The design goal of MPICH-MNC is improving data transfer rate and reducing the transfer latency of long message.
Keywords: Clusters, User-level communication, CLUMPs, Shared memory, Multi-networking communication
目 錄
摘 要 I
ABSTRACT II
第一章 引言 1
1.1 課題研究的目的與意義 1
1.2 國內(nèi)外研究現(xiàn)狀 1
1.3 論文各部分的內(nèi)容安排 3
第二章 集群通信與GM協(xié)議 4
2.1 集群通信系統(tǒng)概況 4
2.1.1高性能可擴展集群及其通信系統(tǒng) 4
2.1.2集群通信系統(tǒng)體系結(jié)構(gòu) 6
2.1.3集群通信系統(tǒng)研究目標(biāo) 7
2.1.4常用集群系統(tǒng)通信技術(shù) 8
2.1.5幾個有代表性的集群通信系統(tǒng) 11
2.2 基于Myrinet的GM協(xié)議 14
2.2.1 Myrinet簡介 14
2.2.2 GM協(xié)議分析 16
2.2.3 Myrinet的性能 18
第三章 基于共享內(nèi)存的CLUMPS結(jié)點本地通信優(yōu)化 20
3.1 CLUMPs的結(jié)構(gòu) 20
3.2 CLUMPs要解決的兩個問題 21
3.2.1 互連網(wǎng)絡(luò)仍是影響集群通信性能的瓶頸 21
3.2.2基于消息傳遞的本地通信性能受限 22
3.3 CLUMPs結(jié)點內(nèi)部的共享內(nèi)存通信 23
3.3.1 AM-II的本地通信機制 23
3.3.2 FMP_Shmem概述 24
3.3.3 FMP_Shmem的數(shù)據(jù)結(jié)構(gòu)與算法描述 25
3.3.4 FMP_Shmem的性能 26
第四章 支持多通道網(wǎng)絡(luò)通信的高性能MPI 30
4.1 支持多通道網(wǎng)絡(luò)通信的集群底層通信協(xié)議MNC 30
4.1.1 多通道網(wǎng)絡(luò)技術(shù) 30
4.1.2 支持多通道網(wǎng)絡(luò)通信的MNC協(xié)議 31
4.2 MPI標(biāo)準(zhǔn)與MPICH 33
4.2.1 MPI與MPICH概述 33
4.2.2 MPICH的體系結(jié)構(gòu) 35
4.2.3 抽象設(shè)備接口（ADI） 36
4.2.4 Channel接口 37
4.3 MPICH-MNC的設(shè)計方案 40
4.3.1 配置文件 40
4.3.2 主要函數(shù)的實現(xiàn) 41
4.4 MPI與PVM的比較 43
4.4.1 PVM概述 43
4.4.2 PVM-GM的實現(xiàn)及其與MPI的性能對比 44
第五章 結(jié)論與展望 47
參考文獻(xiàn) 49
