550 High performance computing and parallelism
In this program, the full vectorsAandARightare declared – if we are dealing
with a distributed memory machine, it is up to the compiler to distribute these over
the nodes. The commandCSHIFTreturns the vectorA, left-shifted in a cyclical
fashion over one position. In the last statement, the result is calculated in vector
notation (Fortran 90 allows for such vector commands). This program would run
equally well on a conventional SISD machine – the compiler must find out how it
distributes the data over the memory and how theCSHIFTis carried out.
In a message-passing model, the program looks quite different:
1 INTEGER, PARAMETER :: N =100! Declaration of
! array size
2 INTEGER :: VecElem, RVecElem! Elements of
! the vector
3 INTEGER :: MyNode, RightNode, &! Variables to
LeftNode! contain node
! addresses
4 MyNode = whoami()! Determine node
! address
5 VecElem = MyNode! Initialise
! vector element
6 RightNode = MOD(MyNode+1, N)+1! Address of
! right neighbour
7 LeftNode = MOD(Mynode-1+N, N)+1! Address of left
! neighbour
8 CALL send&get(VecElem, LeftNode, &! Circular
RVecElem, RightNode)! left-shift
9 VecElem = VecElem + RVecElem! Calculate sum
This program is more difficult to understand. In the program it is assumed that there
are at least 100 free processors. They are numbered 1 through 100. The function
whoami()returns the number of the processor – at each processor,MyNodewill
have a different value. The variableVecElemis anINTEGERwhich is allocated
locally on each processor: the same name,VecElemis used to denote a collection
of different numbers. Referring to this variable on processor 11 gives in general a
different result than on processor 37. The right and left neighbours are calculated in
statements 6 and 7. Statement 8 contains the actual message passing:VecElem, the
element stored at the present processor, is sent to its left neighbour and the element
of the right hand neighbour,RVecElem, is received at the present processor. In
statement 9 this is then added toVecElem. A popular system for parallel commu-
nication in the message-passing paradigm is themessage-passing systemMPI-2,
which is highly portable and offers extensive functionality.
Another classification of programming paradigms is connected with the organisa-
tion of the program. A simple but efficient way of parallelisation is themaster–slave