Algorithmic Framework: Algorithmic frameworks are application agnostic, algorithm dependent and platform agnostic (they help build Application Frameworks efficiently)
Application Framework: Application frameworks are application dependent, algorithm dependent and platform agnostic (they help build Applications efficiently)
Concurrency: The state of a system in which a set of tasks can remain active and make progress at the same time.
Concurrent algorithm: An algorithm that fundamentally depends on concurrency to correctly execute. For example, a print-spool program is explicitly designed to support concurrent access by multiple agents. The algorithsm supporting the program must manage conconcurrency in order to carry out its purpose. For a contrasting use of concurrency, see “parallel algorithm”.
Data parallel: A type of parallel algorithm in which the concurrency is expressed by applying a single stream of instructions simultaneously to the elements of a data structure.
Fork/join: A programming model used in multithreaded APIs such as Pthreads and OpenMP. A thread executes a fork, which creates or uses additional threads. The collection of threads (called a team in OpenMP) execute concurrently. When the members of the team complete their concurrent tasks, they suspend until every member of the team has arrived at the join. At that point, the members of the team may be destroyed or recycled and the original thread continues.
Framework: A framework is a software environment organized around a software architecture that allows for programmer customizations only in harmony with the software architecture
Parallel algorithm: An algorithm that uses concurrency in order to solve a problem in less time or to address a larger problem. The problem addressed by a parallel algorithm conceptually allows for a seriel implementation. For example, a parallel algorithm for matrix multiplication will hopefully execute in less time as more processing elements are included, but the problem can be correctly posed for execution on computere consisting of a single processing element. For a contrasting use of concurrency, see “concurrent algorithm”.
Parallelism: the state of a program or algorithm in which concurrency is exploited to support simultaneous execution on multiple processing elements in order to (1) decrease runtime, or (2) address larger problems.
Patterns: Time-tested solutions to recurring problems within a well-defined context
Processing element: A generic term used to reference a hardware element that executes a stream of instructions. The context defines what unit of hardware is considered a processing element (e.g., core, processor, computer). Consider a cluster of SMP workstations. In some programming environments, each workstation is viewed as executing a single instruction stream; in this case, a processing element is a workstation. A different programming environment running on the same hardware, however, may view each processor or core of the individual workstations as executing an individual instruction stream; in this case, the processing element is the processor or core rather than the workstation.
Programming Framework: Programming frameworks are application agnostic, algorithm agnostic and platform agnostic (they help you Program efficiently)
Race condition: An error condition peculiar to parallel programs in which the outcome of a program changes as the relative scheduling of units of execution (UEs) varies. This is often the result of modifications to a shared or global variable by more than one UE without an appropriate synchronization mechanism (such as a barrier, mutex, or semaphore).
Software Architecture: The structure of a piece of software, comprised of a hierarchy of patterns
Software Pattern Language: An organized way of tackling the process of creating a software architecture using a hierarchy of patterns
Synchronization: Enforcing constraints on the ordering of events occurring in different units of execution (UEs). This is primarily used to ensure that a collection of UEs access shared resources in such a way that the program is correct regardless of how the UEs are scheduled.
Task: A logically related sequence of operations.
Task Parallelism: A class of parallelism where the concurrency is expressed directly in terms of the tasks that execute concurrently. This is a very broad class of techniques and includes master-worker algorithms, systems such as OpenMP that creates both explicit tasks and tasks from loop iterations, SPMD programs with explicit message passing, and many more approaches to parallelism. Note that any data parallel algorithm can be cast in terms of tasks, but the converse is not true. Hence task parallelism is the more general model.
Task queue: A queue that holds tasks for execution by one or more units of execution (UEs). Task queues are commonly used to implement dynamic scheduling algorithms in task parallel algorithms. Thread: A fundamental unit of execution (UE) on certain computers. In a UNIX context, threads are associated with a process and share the process's environment. This makes the threads relatively lightweight compared to processes (i.e., a context switch between threads is cheap vs. a context switch between processes). A more high-level view is that a thread is a “lightweight” UE that shares an address space with other threads.
Thread pool: A collection of threads that are managed over the course of a program to minimize the overhead of thread creation and destruction. Threads not in use are maintained by the pool in an idle state. Thread pools are implicit in some APIs and explicit in others.
Unit of Execution: A software entity that advances a program counter. Use of the term “Unit of exeuction” (or “UE”) is a short hand notation for “thread, process, fiber, work-item, etc”.