Hardware parallelism deals with the CPU of an individual system and how you can squeeze performance out of sub-components of the CPU (Including caches, pipelines, and multiple execution units) that can be speed up your code. At another level, there is parallelism that is gained by having multiple systems working on a computational problem in a distributed function. These systems are called “fine grained” for parallelism inside the CPU or having multiple CPU’s within the same system, or “coarse grained” for parallelism of a collection of separate systems processing together.
4.1.1 CPU-level parallelism
Many CPU’s operate on the principle of operating one instruction after another in a straight line, always completing one step or instruction before another one is started. This serial execution continues until the CPU is forced for some reason to follow the code as it branches someplace else to start executing another set of instructions.
In older machines, such as the DEC VAX, and the Intel 8088, this is exactly what happens the follow of instructions are very deterministic and predictable, while in newer CPU’s such as the Sun UltraSPARC, the DEC alpha and the Intel Pentium II and III, and so have the ability to more than one process at a time. The logic on the CPU chip divides the CPU into multiple execution units.
4.1.2 System Level parallelism
System parallelism is what clustering is about. It is the parallelism of multiple nodes co-coordinating to work on a problem in parallel that gives the cluster its power. More levels of parallelism can be introduced to a cluster by using Multi-processor systems, as these will be running tasks in parallel. Using more than one network interface can also increase the performance as each machine will have a greater bandwidth to communicate over.
In order for these clustered machines to communicate a network of some kind is going to need to be used;
Protocols are sets of standards that describe a common information format that computers use when communicating a network .A network protocol is integral to how information is packaged and sent to another computer. Mostly commonly the TCP/IP protocol will be used for these to communicate, as this is the protocol used for the Internet and has the most available software for it.
The network is a hardware device that takes the information packaged by a network protocol and puts it into a format that can be transmitted over a cable such as Ethernet or Fibre-optical cable.
The transmission medium is the mechanism which the information bundled together by the networking protocol and transmitted by the network interface is delivered from one computer to another. This can be something simple as a twisted pair of wires (such as use in Ethernet 10-baseT and 100-base TX networks)
Bandwidth is the amount of information that can be transmitted over a given transmission medium over a given amount of time. It is usually expressed in some form of "bits per second". Most local area networks run at 10Mbits/second or 100Mbits/second.