granularity, n.
In general, the quality of being composed of large particles.
In parallel computing, the quality of being composed of large chunks of code that can run in parallel. Also defined as the ratio of computation to communication.
Fine-grain parallelism:
- Low computation/communication ratio
- Small amounts of work done between communication
- Easier to load-balance
Coarse-grain parallelism:
- High computation/communication ratio
- Large amounts of work done between communication
- Harder to load-balance
Usually it is better create a coarse program because the overheads of communication and
synchronization are relatively high (
cpus sit
idle while waiting), although it depends on the
algorithm and
hardware environment. For example, if you have 500 identical nodes and simply want to split up a large array of independent elements to be processed (once), program coarsely. If you have a
heterogeneous environment with constantly changing requests, program finely to allow for load-balancing (e.g., a large
database search).