Concurrency and Parallelism in C

Explore techniques and libraries for achieving concurrency and parallelism in C programming, including multi-threading, synchronization, and parallel processing.

Concurrency and parallelism are essential concepts in modern programming to harness the full potential of multi-core processors and improve the efficiency of software applications. In this explanation, we will delve into the techniques and libraries available in C programming for achieving concurrency and parallelism. We’ll cover multi-threading, synchronization, and parallel processing, along with syntax, examples, outputs, and explanations.

Multi-Threading in C:

Multi-threading in C refers to the concurrent execution of multiple threads within a single process. Each thread operates independently and shares the same memory space, allowing for better resource utilization and improved program efficiency. Multi-threading is commonly used to perform tasks concurrently, take advantage of multi-core processors, and enhance the responsiveness of applications.

Syntax:

#include <pthread.h>

void* thread_function(void* arg) {
    // Thread code here
}

int main() {
    pthread_t thread_id;
    pthread_create(&thread_id, NULL, thread_function, NULL);
    pthread_join(thread_id, NULL);
    return 0;
}

Explanation:

• #include <pthread.h>: This header file is used for multi-threading support in C.
• void* thread_function(void* arg): This is the function that will be executed by the new thread. The arg parameter can be used to pass arguments to the thread function.
• pthread_t thread_id;: This is the identifier for the thread.
• pthread_create(&thread_id, NULL, thread_function, NULL);: This function call creates a new thread and associates it with the specified thread function.
• pthread_join(thread_id, NULL);: This function call waits for the created thread to finish its execution before continuing in the main thread.
• return 0;: This is the end of the main program.

//Program that uses two threads to print numbers concurrently

#include <stdio.h>
#include <pthread.h>

void* print_numbers(void* arg) {
    for (int i = 1; i <= 5; ++i) {
        printf("Thread %d: %d\n", (int)arg, i);
    }
    return NULL;
}

int main() {
    pthread_t thread1, thread2;
    pthread_create(&thread1, NULL, print_numbers, (void*)1);
    pthread_create(&thread2, NULL, print_numbers, (void*)2);

    pthread_join(thread1, NULL);
    pthread_join(thread2, NULL);

    return 0;
}

Output:
Thread 1: 1
Thread 2: 1
Thread 1: 2
Thread 2: 2
Thread 1: 3
Thread 2: 3
Thread 1: 4
Thread 2: 4
Thread 1: 5
Thread 2: 5

Explanation:

In this example, two threads (thread1 and thread2) are created using pthread_create. Each thread executes the print_numbers function, which prints numbers from 1 to 5 with a thread identifier. The pthread_join function is used to wait for the completion of both threads before the program exits.

2. Synchronization in Multi-Threading:

Synchronization is a critical aspect of multi-threading that ensures proper coordination and control between threads to avoid data races, maintain consistency, and achieve thread safety. In multi-threaded programs, threads may access shared resources concurrently, potentially leading to conflicts and incorrect results. Synchronization mechanisms provide ways to protect shared resources and ensure that threads work together effectively. Let’s explore synchronization in multi-threading using mutexes (mutual exclusion) as an example.

Syntax (Mutex):

#include <pthread.h>

pthread_mutex_t mutex;

void* thread_function(void* arg) {
    pthread_mutex_lock(&mutex);
    // Critical section
    pthread_mutex_unlock(&mutex);
    return NULL;
}

Explanation:

• #include <pthread.h>: This header file is used for multi-threading support in C.
• pthread_mutex_t mutex;: This is the declaration of a mutex variable that will be used for synchronization.
• pthread_mutex_lock(&mutex);: This function call locks the mutex, allowing the thread to enter the critical section. If the mutex is already locked by another thread, the calling thread will be blocked until the mutex is available.
• // Critical section: This is the portion of code that needs to be protected from concurrent access by multiple threads.
• pthread_mutex_unlock(&mutex);: This function call releases the mutex, allowing other threads to access the critical section.

Example:

Let’s modify the previous multi-threading example to use mutexes for thread synchronization.

#include <stdio.h>
#include <pthread.h>

pthread_mutex_t mutex;

void* print_numbers(void* arg) {
    pthread_mutex_lock(&mutex);
    for (int i = 1; i <= 5; ++i) {
        printf("Thread %d: %d\n", (int)arg, i);
    }
    pthread_mutex_unlock(&mutex);
    return NULL;
}

int main() {
    pthread_t thread1, thread2;

    pthread_mutex_init(&mutex, NULL);

    pthread_create(&thread1, NULL, print_numbers, (void*)1);
    pthread_create(&thread2, NULL, print_numbers, (void*)2);

    pthread_join(thread1, NULL);
    pthread_join(thread2, NULL);

    pthread_mutex_destroy(&mutex);

    return 0;
}

Output:

Thread 1: 1
Thread 1: 2
Thread 1: 3
Thread 1: 4
Thread 1: 5
Thread 2: 1
Thread 2: 2
Thread 2: 3
Thread 2: 4
Thread 2: 5

Explanation:

In this example, a mutex (pthread_mutex_t mutex) is used to ensure that only one thread can access the critical section (printing numbers) at a time. The pthread_mutex_lock function locks the mutex before entering the critical section, and pthread_mutex_unlock unlocks it after the critical section. This prevents the threads from interleaving and ensures synchronized output.

3. Parallel Processing with OpenMP

Parallel Processing with OpenMP refers to the use of the Open Multi-Processing (OpenMP) API to harness the power of multi-core processors and execute code in parallel. OpenMP provides compiler directives and runtime library routines that allow developers to easily parallelize sections of code, making it a valuable tool for improving the performance of computationally intensive tasks. Let’s explore parallel processing with OpenMP using an example.

Syntax:

#include <omp.h>

int main() {
    #pragma omp parallel
    {
        // Parallel code here
    }
    return 0;
}

Explanation:

• #include <omp.h>: This header file provides the necessary functions and directives for OpenMP.
• #pragma omp parallel: This compiler directive creates a parallel region where multiple threads will execute the enclosed code concurrently.
• { // Parallel code here }: This is the section of code that will be executed by each thread in parallel.

Example:

Let’s create a program that calculates the sum of an array in parallel using OpenMP.

#include <stdio.h>
#include <omp.h>

#define SIZE 100000

int main() {
    int arr[SIZE];
    for (int i = 0; i < SIZE; ++i) {
        arr[i] = i + 1;
    }

    int sum = 0;
    #pragma omp parallel for reduction(+:sum)
    for (int i = 0; i < SIZE; ++i) {
        sum += arr[i];
    }

    printf("Sum of array elements: %d\n", sum);

    return 0;
}

Output:

Sum of array elements: 5000050000

Explanation:

In this example, the #pragma omp parallel for directive instructs the compiler to parallelize the loop using multiple threads. The reduction(+:sum) clause ensures that the partial sums from different threads are properly combined to calculate the final sum. OpenMP automatically divides the loop iterations among the available threads, allowing for efficient parallel execution.

Benefits of Parallel Processing with OpenMP:

1. Performance Improvement: Parallel processing with OpenMP utilizes multiple cores to perform tasks in parallel, resulting in faster execution and improved performance for computationally intensive operations.
2. Ease of Use: OpenMP directives are easy to understand and implement, making it relatively straightforward to parallelize sections of code without significant code modifications.
3. Automatic Load Balancing: OpenMP automatically divides work among threads, ensuring balanced workload distribution and efficient resource utilization.
4. Portability: OpenMP is supported by various compilers and platforms, making it a portable solution for parallel processing across different systems.
5. Incremental Parallelism: Developers can choose specific sections of code to parallelize, allowing for incremental parallelization and optimization of critical parts of an application.

Comparison between Concurrency and Parallelism in C

Aspect	Concurrency	Parallelism
Definition	Concurrency refers to the ability of a system to handle multiple tasks concurrently, often using a single processor. Threads can be interleaved, allowing apparent simultaneous execution.	Parallelism involves executing multiple tasks or processes simultaneously using multiple processors or cores to achieve faster execution. Each task has its own dedicated execution unit.
Focus	Focuses on managing multiple tasks that may start, run, and complete in overlapping time periods.	Focuses on executing multiple tasks simultaneously to achieve faster results.
Purpose	Enhances the responsiveness and efficiency of an application by managing and scheduling tasks efficiently.	Improves the performance and speed of execution by utilizing multiple processors or cores to execute tasks in parallel.
Resource Usage	Shares resources, such as memory and CPU time, among threads. Threads may be preemptively switched.	Utilizes separate resources for each task, such as dedicated cores or processors.
Dependency	Concurrency is suitable for tasks with high latency or tasks that can be interleaved, such as I/O-bound operations.	Parallelism is effective for tasks that can be divided into smaller independent subtasks, such as CPU-bound operations.
Communication	Threads may need synchronization mechanisms (e.g., mutexes) to coordinate access to shared resources and avoid data races.	Parallel tasks often require less communication as they work independently and may only need synchronization occasionally.
Example	Multiple threads in a web server handling incoming requests concurrently, allowing for efficient use of resources.	Performing matrix multiplication using multiple processors, each responsible for different portions of the matrix.
Code Indication	Often indicated by interleaved execution of threads, with context switching and task switching occurring.	Often indicated by distinct tasks or processes executing simultaneously on different processors or cores.
Scalability	May not always lead to linear performance improvement as overhead from context switching and synchronization may limit scalability.	Can lead to linear or near-linear performance improvement as tasks are independent and can be executed in parallel without much overhead.

In summary, concurrency focuses on managing multiple tasks efficiently, allowing them to overlap and run concurrently, while parallelism leverages multiple processors or cores to execute tasks simultaneously, leading to faster execution. Both concurrency and parallelism have their applications and advantages, and the choice between them depends on the nature of the tasks and the available hardware resources.