Mastering
AlgorithmsA systematic approach to data structures
and problem-solving techniques

Prof. (Dr.) Rahul Bhandari

Prof. Om Prakash Suthar

www.bpbonline.com

First Edition 2025

Copyright © BPB Publications, India

eISBN: 978-93-65892-994

All Rights Reserved. No part of this publication may be reproduced, distributed or transmitted in any form or by any means or stored in a database or retrieval system, without the prior written permission of the publisher with the exception to the program listings which may be entered, stored and executed in a computer system, but they can not be reproduced by the means of publication, photocopy, recording, or by any electronic and mechanical means.

LIMITS OF LIABILITY AND DISCLAIMER OF WARRANTY

The information contained in this book is true and correct to the best of author’s and publisher’s knowledge. The author has made every effort to ensure the accuracy of these publications, but the publisher cannot be held responsible for any loss or damage arising from any information in this book.

All trademarks referred to in the book are acknowledged as properties of their respective owners but BPB Publications cannot guarantee the accuracy of this information.

www.bpbonline.com

Dedicated to

My wife, Dr. Sunita, my lovely daughter Rudrakshi,
my parents, Veena and Ramesh Chand Bhandari,
my younger brother’s family and my maternal, Karnawat family
                 -Prof. (Dr.) Rahul Bhandari

My parents, Mr. and Mrs. G.R. Suthar, my wife Renu,
my son Rudra, my brother Lokesh and his family, and my sisters
              -Prof. Om Prakash Suthar

About the Authors

Prof. (Dr.) Rahul Bhandari is a senior IEEE member. He is an accomplished academic and researcher in computer science and engineering with over a decade of experience. Currently a professor at Chandigarh University, Punjab, he holds a PhD, M.Tech, and B.E. in the field and has previously served at Chitkara University and JIET Group of Institutions. He also worked as research associate in the project, sponsored by ICMR and entitled Validation of the Jodhpur Instrumented Kursi (JiK) against DXA to diagnose Sarcopenia under the guidance of Dr. Vivek Vijay. His expertise spans data science, artificial intelligence, video compression, and machine learning, with numerous SCI/scopus-indexed publications and several patents, including innovations in wireless networks and healthcare technologies. A dedicated mentor, he has guided PhD and M.E. students in advanced areas like NLP and AI-driven disease detection, while excelling in teaching subjects such as algorithms and machine learning. He is also the organizing member of international conferences: Joint International Scientific Conference Information Technologies for Intelligent Decision Support 2024, SCMAI 2025.

Prof. Om Prakash Suthar is an IBM Bluemix certified trainer for cloud and IoT environment. Since 2022, he has been pursuing his Ph.D. computer vision, ML-DL domain in computer engineering at Marwadi University in Rajkot, India. He completed his M.Tech CSE from JNU Jodhpur in 2015 and his B.Tech in CSE from JIET Jodhpur in 2011. He has held academic positions at several reputed Indian institutions, including LPU. Currently, he is employed as an assistant professor (sr. scale) at the department of computer engineering at Marwadi University in Rajkot, Gujarat, India. AI, machine learning, deep learning, computer vision, and IoT comprise his research interests. With over 13 years of experience, he has authored over 14 research articles in reputed international conferences, journals and professional book chapters, and 8 patents that have been published. Various papers are in pipeline for publishing. He has reviewed various conferences and reputed journal papers.

About the Reviewer

Prof. (Dr.) Suresh, IEEE-senior member, ISTE advisor, professor and HoD-CSE, Chandigarh University, India. He is PhD, M.Tech, B.Tech in computer science and engineering, having more than 20 years of experience in technology teaching, administration, education management and accreditation evaluation process, NAAC, NBA, NIRF, coordinator-IQAC, convener and organizer of national and international conferences: ITIDS2024, SCMAI2025, FDPs, curriculum development with industry association, innovate and implement job readiness program, industry-institution partnership delivery, industry-academia collaborated research, consultation and training. He is the organizer of five international conferences with Russia, Springer, CSI, T&F, etc. He has delivered many invited and keynote talks at international and national conferences, workshops, and seminars. He has published more than 45 papers in SCI, Scopus, WOS, peer-reviewed international and national journals, and conferences. He has also published a number of books and book chapters in reputed publications like Springer, Bentham, and IGI.

Professional engagements and memberships:

Senior member IEEE

AI scholar expert reviewer, AEIC publishers

ACM member

Fellowship member, London Press Journal

Life member of ISROSET (IJCSE- UGC Approved)

Life member of I.S.T.E

Member of Computer Society of India (C.S.I.)

International Association of Computer Sc. and IT (IACSIT)

International Association of Engineers (IAENG)

All India Council for Technical Skill Development

Acknowledgements

Our sincere appreciation goes out to each and every person who has made a contribution to the publication of this book.

We would like to express our gratitude to our families and friends who have been there for us during the process of authoring this book. Without their words of encouragement and love, we would not have been able to overcome this obstacle.

In addition, we would like to express our deepest gratitude to our colleagues at Marwadi University and Chandigarh University, who are constantly contributing to the development of our technical capabilities. We are grateful to you for all of the efforts that you put in each day to push the limits of algorithmic thinking even farther.

We would like to express our sincere appreciation to BPB Publications for entrusting us with this book. Thank you for providing us with direction and assistance during the entire process of authoring this book.

Additionally, we would like to express our gratitude to the reviewers, technical specialists, and editors who contributed to the production of the book in its final form. There is no doubt that the quality of this book has been improved as a result of your ideas and skills.

Preface

In an era where technology drives innovation, mastering algorithms is essential for solving complex problems efficiently. Mastering Algorithms, an extended exploration of algorithmic foundations, equips developers and students with the tools to design, analyze, and implement high-performance solutions. From optimizing code to tackling real-world challenges, algorithms form the backbone of modern computing.

This book comprises 12 comprehensive units, guiding readers through the core of algorithmic thinking. We begin with Algorithm and Program Performance, laying the groundwork for evaluating efficiency. Review of Data Structures ensures a solid foundation, followed by in-depth explorations of Sorting Algorithms and Searching Algorithms for data manipulation. Divide and Conquer algorithms introduce elegant problem-solving strategies, while Greedy Algorithms and Dynamic Programming offer optimization techniques. Backtracking and Branch and Bound provide systematic approaches to complex problems, and Graph Algorithms address networked systems. Computational Complexity demystifies performance analysis, and Other Advanced Algorithms push the boundaries of innovation. Finally, Most Frequently Asked Questions prepares readers for real-world applications and interviews.

Designed for technically inclined students and professionals, this book caters to all programming levels. No prior algorithmic expertise is required, though those with experience will find advanced insights to elevate their skills. Our hands-on approach ensures you can apply concepts practically, from coding efficient solutions to mastering interview challenges.

This book is your guide to unlocking the power of algorithms, transforming ideas into impactful solutions.

Chapter 1: Algorithm and Program Performance - Algorithm and program performance refer to the efficiency and effectiveness of algorithms and the software programs that implement them. Efficient algorithms and well-optimized programs are crucial for achieving fast execution times, minimizing resource usage, and providing a responsive user experience. Topics to be covered: designing and analyzing algorithms, time and space complexity, average and worst-case analysis, asymptotic notations, and recurrence equations and their solution.

Chapter 2: Review of Data Structures - This chapter involves revisiting fundamental concepts in computer science that deal with organizing and storing data in efficient and accessible ways. Data structures provide a framework for managing data effectively, allowing for optimized operations such as insertion, deletion, retrieval, and manipulation. Topics to be covered: arrays, stacks, queues, pointers, linked lists (one-way, two-way and circular two-way), hashing, trees (BST, B-tree, balanced trees (AVL, red-black trees)), and heaps: Binomial and Fibonacci heap, etc.

Chapter 3: Sorting Algorithms - Sorting algorithms are essential techniques in computer science used to arrange elements in a specific order, typically ascending or descending. Sorting plays a fundamental role in data manipulation, enabling efficient searching, grouping, and analyzing of information.

Chapter 4: Searching Algorithms - Searching algorithms are essential techniques in computer science used to arrange elements in a specific order, typically ascending or descending. Sorting plays a fundamental role in data manipulation, enabling efficient searching, grouping, and analyzing of information.

Chapter 5: Divide and Conquer - Divide and conquer is an algorithmic strategy that breaks down a complex problem into smaller, more manageable subproblems. It solves each subproblem independently and then combines their solutions to obtain the solution to the original problem. This technique is particularly effective for problems with overlapping subproblems and optimal substructure.

Chapter 6: Greedy Algorithms - Greedy algorithms make locally optimal choices at each step to construct a globally optimal solution. While they do not guarantee the absolute best solution, they are often used when finding an optimal solution is either computationally expensive or unnecessary.

Chapter 7: Dynamic Programming - Dynamic programming (DP) is a powerful algorithmic technique used to solve optimization problems by breaking them down into smaller subproblems and solving each subproblem only once, storing their solutions to avoid redundant computations. DP is particularly effective when a problem has overlapping subproblems and optimal substructure, meaning the solution to a larger problem can be constructed from solutions to smaller subproblems.

Chapter 8: Backtracking - Backtracking is a systematic algorithmic technique used to solve problems by exploring all possible solutions through a trial-and-error approach. It is particularly effective for problems where you need to find one or more solutions among a large set of possibilities. Backtracking is commonly used for solving puzzles, combinatorial optimization problems, and constraint satisfaction problems.

Chapter 9: Branch and Bound - Branch and bound is an algorithmic technique used to solve optimization problems by systematically exploring the solution space and pruning branches that cannot lead to an optimal solution. It divides the problem into smaller subproblems and maintains bounds on the possible solutions within each subproblem.

Chapter 10: Graph Algorithms - Graph algorithms are a fundamental subset of algorithms in computer science that focus on solving problems related to graphs. Graphs provide a versatile way to model relationships and connections between entities, making graph algorithms essential for tasks across diverse fields.

Chapter 11: Computational Complexity - Computational complexity theory is a branch of computer science that studies the resources (such as time and space) required to solve computational problems. It categorizes problems based on their inherent difficulty and helps us understand the efficiency of algorithms.

Chapter 12: Other Advanced Algorithms - This chapter provides advanced algorithms, which are categorized into various algorithms used in cryptosystem, computational geometry, and complexity classes for randomized algorithm, NP-completeness problems, and algorithms for parallel computers.

Appendix: Most Frequently Asked Questions - This chapter provides objective-type questions asked in various examinations such as GATE, NET, etc. with solutions.

Code Bundle and Coloured Images

Please follow the link to download the

Code Bundle and the Coloured Images of the book:

https://rebrand.ly/96ae80

The code bundle for the book is also hosted on GitHub at https://github.com/bpbpublications/Mastering-Algorithms. In case there’s an update to the code, it will be updated on the existing GitHub repository.

We have code bundles from our rich catalogue of books and videos available at https://github.com/bpbpublications. Check them out!

Errata

We take immense pride in our work at BPB Publications and follow best practices to ensure the accuracy of our content to provide with an indulging reading experience to our subscribers. Our readers are our mirrors, and we use their inputs to reflect and improve upon human errors, if any, that may have occurred during the publishing processes involved. To let us maintain the quality and help us reach out to any readers who might be having difficulties due to any unforeseen errors, please write to us at :

errata@bpbonline.com

Your support, suggestions and feedbacks are highly appreciated by the BPB Publications’ Family.

Did you know that BPB offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.bpbonline.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at :

business@bpbonline.com for more details.

At www.bpbonline.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on BPB books and eBooks.

Piracy

If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at business@bpbonline.com with a link to the material.

If you are interested in becoming an author

If there is a topic that you have expertise in, and you are interested in either writing or contributing to a book, please visit www.bpbonline.com. We have worked with thousands of developers and tech professionals, just like you, to help them share their insights with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions. We at BPB can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about BPB, please visit www.bpbonline.com.

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 1Algorithm and Program Performance

Introduction

In this chapter of the book, we will present an introduction to algorithms and their role in contemporary computing systems. Furthermore, this chapter introduces the concept of asymptotic notation, which we will use to analyze the growth of functions, particularly those describing algorithm runtime. We commence by briefly explaining the commonly used asymptotic notations and present an illustrative example of their application. We subsequently establish formal definitions for various asymptotic notations and outline conventions for their proper utilization.

Structure

The chapter covers the following topics:

• Introduction to algorithms

• Algorithm complexity

• Asymptotic notations

• General rules for the analysis of programs

• Recurrence relation

Objectives

The objectives of algorithm and program performance encompass a range of essential goals aimed at enhancing the efficiency and effectiveness of computational processes. One key objective is to optimize computational effectiveness, leading to quicker processing speeds. This involves the development of algorithms and programs that can execute tasks with greater speed and efficiency. Additionally, streamlining memory and hardware utilization to minimize resource usage is a crucial aim, ensuring that computational resources are utilized judiciously. Accelerating program tasks is another priority, enabling swift and flexible execution of operations, which in turn improves overall system responsiveness. Moreover, there is a focus on ensuring the ability to manage larger datasets and increased computational requirements, accommodating the growing demands of modern applications and systems. Ultimately, these objectives collectively contribute to elevating user satisfaction by providing a more responsive and efficient computing experience.

Introduction to algorithms

An algorithm is a structured sequence of steps or methods to solve problems or accomplish tasks. It consists of a precise set of rules or instructions establishing a series of actions, enabling a computer to perform specific tasks or calculations. Algorithms are foundational in computer science and have widespread applications in various domains, including data processing, software development, artificial intelligence, and solving computational problems.

Essentially, an algorithm serves as a roadmap for task execution, offering an unambiguous set of guidelines that can be followed to achieve a particular objective. The development and assessment of algorithms are pivotal in comprehending computational procedures and complexity theory, while also guaranteeing the effective utilization of computational resources. Competent algorithm design can significantly influence the performance, efficiency, and reliability of software systems and computational tasks.

Algorithm

An algorithm is defined as a sequence of unambiguous instructions for solving a problem, that is, for obtaining a required output for any legitimate input in a finite amount of time.

This definition can be demonstrated through Figure 1.1:

Figure 1.1: Pictorial definition of an algorithm

Algorithms play a crucial role in various fields and find extensive application in different contexts, listed as follows:

• In computer science, they serve as the fundamental building blocks for tasks ranging from simple sorting and searching to complex functions such as artificial intelligence (AI) and machine learning (ML).

• In mathematics, they are crucial for solving problems, such as determining optimal solutions for systems of linear equations or finding the shortest path in a graph.

• In operations research, algorithms contribute to streamlining and making decisions for transportation, logistics, and resource allocation.

• In AI, they form the backbone of AI and ML, enabling the creation of intelligent systems capable of tasks such as image recognition, natural language processing, and decision-making.

• In data science, algorithms are indispensable for analyzing, processing, and extracting insights from extensive datasets in fields such as marketing, finance, and healthcare.

• In cryptocurrency and blockchain, algorithms play a vital role in ensuring secure transactions and maintaining the integrity of blockchain networks.

• In network routing, algorithms are pivotal in determining the most efficient routes for data packets through networks, ensuring fast and reliable data transmission.

• In security systems, algorithms are essential for encrypting sensitive data, securing communication, and protecting systems from cyber threats and unauthorized access.

• In recommendation systems, algorithms are utilized to analyze user preferences, providing personalized recommendations for products, services, and content.

These examples underscore the diverse applications of algorithms. As technology and fields continue to evolve, the role of algorithms remains indispensable, highlighting their significance in contemporary society.

Let us take the example of preparing a meal. This simple algorithm offers a basic structure for cooking a range of dishes. Additional steps may be required, depending on the intricacy of the recipe and specific cooking methods.

Algorithm 1.1: Algorithm for cooking food

The steps are as follows:

1. Collect all the necessary ingredients and cooking utensils.

2. Clean and arrange the cooking area.

3. Measure and prepare the required ingredients as per the recipe.

4. Ensure that the ingredients are properly washed, peeled, and chopped if needed.

5. Heat the cooking surface (for example, stovetop, oven) to a suitable temperature.

6. Combine the ingredients in the correct sequence, following the recipe instructions.

7. Regularly monitor the food while cooking to prevent overcooking or burning.

8. Adjust the heat levels if necessary to maintain the desired cooking temperature.

9. Incorporate seasonings and spices to enrich the flavor of the dish.

10. Fine-tune the flavors as required by tasting.

11. Transfer the cooked food to a serving dish or plate.

12. Present the food neatly for an appealing appearance.

13. Serve the dish while it is still hot and fresh.

14. Enjoy the meal with friends, family, or alone.

Likewise, algorithms aid in executing tasks in programming to achieve the desired output. The algorithm created is not reliant on a specific programming language; rather, it comprises straightforward instructions that can be applied in any language, ensuring the anticipated outcome remains consistent.

Need for algorithms

Let us consider two children, Richard and Henry, attempting to solve the Rubik's cube. Richard has a method to solve it in a specific number of steps. In contrast, Henry is confident he can solve it but does not know the procedure. Richard successfully solves the cube in just 2 minutes, while Henry struggles throughout the day, finally managing to solve it (although he might have used an unauthorized method). This illustrates that having a procedure or algorithm significantly speeds up the solving process, making it more efficient than attempting to solve it without one. Therefore, the importance of having an algorithm is evident.

Characteristics of an algorithm

Just as one might not follow written instructions for cooking a recipe but instead rely on the standard procedure, not all written instructions in programming qualify as algorithms. To be considered an algorithm, instructions must exhibit the following characteristics:

• Clarity and precision: An algorithm must be entirely unambiguous. Each of its steps should be precise and lead to a singular interpretation.

• Well-defined inputs: When an algorithm requires inputs, those inputs must be well-defined. The algorithm may accept zero or more inputs.

• Well-defined outputs: The algorithm should distinctly specify the output it will produce. This output should also be well-defined and must include at least one result.

• Finiteness: An algorithm must have a finite nature, meaning it will eventually terminate after a finite number of steps.

• Feasibility: An algorithm must be practical, generic, and straightforward, making it executable with existing resources. It should not rely on futuristic technologies or other elements.

• Language independence: The designed algorithm must be independent of any specific programming language. It should consist of straightforward instructions that can be implemented in any language, resulting in the expected output.

• Input handling: An algorithm can have zero or more inputs. Each instruction containing a fundamental operator may accept zero or more inputs.

• Output production: An algorithm should generate at least one output. Every instruction featuring a fundamental operator can accept zero or more inputs.

• Definiteness: All instructions within an algorithm must be unambiguous, precise, and easily understandable. Consulting any instruction in the algorithm should provide a clear understanding of what needs to be done.

• Finiteness: An algorithm must terminate within a finite number of steps in all test cases. Every instruction featuring a fundamental operator should conclude within a finite timeframe. Infinite loops or recursive functions lacking base conditions do not meet the criterion of finiteness.

• Effectiveness: An algorithm should be built using basic, simple, and practical operations, allowing it to be traced using only paper and pencil.

Designing an algorithm

As the algorithm is not tied to a specific programming language, it serves as a means to outline the logical steps for solving a problem. However, before drafting an algorithm, it is important to consider the following:

• Clarity and precision: The algorithm must offer unambiguous instructions.

• Input flexibility: An algorithm may accept zero or more well-defined inputs.

• Output consistency: It must yield one or more well-defined outputs that align with the desired result.

• Termination: Algorithms are required to conclude after a specific number of steps.

• Finiteness: They must reach an endpoint within a finite number of steps.

• Language neutrality: Step-by-step instructions in an algorithm should be independent of any specific computer code.

Let us consider some examples of algorithms which are given as follows.

Algorithm 1.2: Write an algorithm to multiply two numbers and display the result obtained from the multiplication.

1. Start

2. Get the knowledge of input. Here, we need 3 variables: a and b will be the user input, and c will hold the result.

3. Define variables a, b, and c. Variables a and b will receive input from the user, while c will store the result.

4. Acquire input values for variables a and b from the user.

5. Determine the solution using operators, data structures, and logic: c = a * b.

6. Ultimately, display the result c.

7. End

Algorithm 1.3: Write an algorithm to identify the maximum value among all the elements contained within the array.

1. Start

2. Define a variable called max and assign it the value of the first element in the array.

3. Check the variable max against the remaining elements of the array using a loop.

4. If max < array element, then the value in max will change and store array element.

5. If no element is left, return max otherwise go to Step 3.

6. End

Algorithm 1.4: Write an algorithm to find the average of 3 subjects.

1. Start

2. Read the marks of three subjects, for example, S1, S2, S3.

3. Compute the total of the marks for all three subjects and save the outcome in a variable called Sum (Sum = S1+S2+S3).

4. Divide the Sum by 3 and assign it to another variable Average (Average = Sum/3).

5. Print the value of the Average of three subjects.

6. End

Algorithm complexity

Algorithm complexity refers to the volume of resources, including time and memory, needed to solve a problem or execute a task. Time complexity, a widely used metric, signifies the duration an algorithm takes to generate a result relative to the input's size. Memory complexity, on the other hand, indicates the memory consumption of an algorithm. Designers aim to create algorithms with minimal time and memory complexities to enhance efficiency and scalability.

Complexity of an algorithm

The complexity of an algorithm is defining a function that describes the algorithm's efficiency based on the quantity of data it processes.

Analyzing an algorithm involves evaluating its time complexity and space complexity. Crafting an efficient algorithm ensures minimal time consumption for processing the logic.

Time complexity

Determining the time required for an algorithm to solve a problem involves evaluating loop iterations, comparisons, and related factors. Time complexity represents the duration an algorithm requires relative to its input size. The term time can denote the count of memory accesses performed the number of integer comparisons, the frequency of inner loop operations, or any other pertinent unit linked to the actual time taken by the algorithm.

Space complexity

Space complexity pertains to the amount of memory an algorithm uses during problem-solving. This incorporates the memory taken up by essential input variables, in addition to any extra space (excluding the input space) that the algorithm uses. For instance, if a data structure like a hash table is employed, an array is necessary to store values, thus contributing to the algorithm's space complexity as an auxiliary space. This extra space is known as auxiliary space. Space complexity is a metric that determines the quantity of memory (space) an algorithm requires in relation to its input size. While sometimes overlooked due to minimal or evident space usage, space complexity can become a concern akin to time complexity. The memory requirement of a program includes the following elements:

• Instruction space: This denotes the space required to store the compiled version of the program's instructions.

• Data space: Data space encompasses the memory needed for storing both constant and variable values. Data space is comprised of two components:

o Space utilized by constants and simple variables within the program.

o Space occupied by dynamically allocated objects like arrays and class instances.

• Environment stack space: The environment stack serves the purpose of preserving information essential for resuming the execution of partially completed functions.

The extent of instruction space necessary is contingent on various factors, including:

• The compiler employed for converting the program into machine code

• The compiler options that are in effect during the compilation process

• The specifications of the target computer

The primary goals in the design of algorithms include maximizing efficiency in time, preserving space, and upholding dependability. The excellence of a program is often indicated by its superior speed, highlighting the importance of prioritizing time-saving as a fundamental objective. Likewise, giving precedence to space efficiency over competing programs is greatly valued. Additionally, securing the program's stability and averting potential problems, such as system freezes or data distortion, is a critical aspect of maintaining a positive reputation.

The running time of an algorithm, represented by the function f(n), relies not solely on the size n of the input data but also on the specific data itself. The complexity function f(n) takes various forms in particular scenarios:

• Best case: The best-case scenario corresponds to the lowest achievable value of f(n).

• Average case: The average case situation pertains to the expected value of f(n).

• Worst case: The worst-case scenario refers to the highest possible value of f(n) for any conceivable input.

Analysis of algorithms

The field of computer science, which studies efficiency of algorithms, is known as analysis of algorithms.

Algorithms can be evaluated using different benchmarks. Typically, this chapter concentrates on comprehending how the time or space required for solving larger instances of a problem escalates. We assign an integer, referred to as the problem's size, which functions as a measure for the volume of input data.

Asymptotic notations

The subsequent notations are frequently utilized in performance analysis to define the complexity of an algorithm:

• Big–OH (O)

• Big–OMEGA (Ω)

• THETA (θ)

Big–OH (O) notation

This notation provides the precise upper limit of the given function. Typically, it can be expressed as f(n) = O(g(n)). In other words, when dealing with larger values of n, the upper boundary of f(n) is g(n). For instance, if the algorithm is defined as f(n) = n3 + 10n2 + 5n + 10, then n3 serves as g(n). This signifies that g(n) represents the highest rate of growth for f(n) as n becomes larger.

It is defined as O(g(n)) = {f(n): there exist positive constants c and no such that 0 <= f(n) <= cg(n) for all n >= no}.

Where g(n) is an asymptotic tight upper bound for f(n).

In Figure 1.2, n0 represents the point from which we evaluate the rates of growth for an algorithm. Below this point, the rates of growth may vary:

Figure 1.2: Rate growth of Big–OH (O) notation

Big–OMEGA (Ω) notation

This notation provides a more stringent lower limit for the given algorithm, denoted as f(n) = Ω(g(n)). This implies that for larger values of n, the tighter lower bound of f(n) is g(n), as illustrated in Figure 1.3. For instance, if f(n) is a function defined as 10n2 + 8n + 5, then g(n) is Ω(n2).

It is defined as Ω (g (n)) = {f (n): there exist positive constants c and no such that 0 <= cg (n) <= f (n) for all n >= no}.

Where g(n) is an asymptotic tight lower bound for f(n).

Figure 1.3: Rate growth of Big–OMEGA (Ω) notation

THETA (θ) notation

This notation determines whether the upper and lower bounds of a given function are identical. The average running time of an algorithm consistently falls between the lower and upper bounds. When the upper bound (O) and lower bound (Ω) produce equivalent outcomes, the θ notation will likewise exhibit the same rate of growth. For instance, considering the expression f(n) = 5n + 4n, its tight upper bound, g(n), is O(n). In this scenario, the rate of growth in the best-case is g(n) = O(n), which aligns with the rate of growth in the worst case. Consequently, the average case will also yield the same rate. The θ notation is depicted in Figure 1.4.

It is defined as θ (g(n)) = {f(n): there exist positive constants C1, C2 and no such that
5 C1g(n) <= f(n) <= C2g(n) for all n >= no}.

g(n) is an asymptotic tight bound for f(n).

Figure 1.4: Rate growth of THETA (θ) notation

The time efficiencies of numerous algorithms can be categorized into a limited number of classes. Table 1.1 presents these classes in ascending order of their growth orders, accompanied by their respective names and descriptions:

Class

Name

Description

1

constant

In the majority of programs, the instructions that follow are executed either once or, at the most, a few times. When all the program's instructions exhibit this trait, we define its running time as constant.

log n

logarithmic

If a program's running time is logarithmic, the program's speed diminishes slightly as n increases. This type of running time is typical in programs that tackle a substantial issue by dividing it into a more manageable problem, reducing the size by a fixed fraction. For instance, when n equals a million, log n doubles. With each doubling of n, log n increases by a constant amount, but log n does not double until n reaches n2.

N

linear

If a program's running time is linear, typically, a minor computation is performed on each input element. This scenario is considered optimal for an algorithm that needs to handle n inputs.

n log n

linearithmic

This type of running time is observed in algorithms that resolve a problem by subdividing it into smaller sub-problems, solving them independently, and subsequently amalgamating the solutions. As 'n' doubles, the running time increases more than twofold.

n2

quadratic

If the running time of an algorithm is quadratic, it is generally suitable for handling relatively small problems. Quadratic running times commonly emerge in algorithms that handle all pairs of data items (perhaps within a nested loop). As n doubles, the running time increases fourfold.

n3

cubic

Likewise, an algorithm that handles triples of data items (possibly within a triple-nested loop) exhibits cubic running time and is feasible for managing only small problems. When n doubles, the running time increases eightfold.

2n

exponential

Only a limited number of algorithms with exponential running time are viable for practical applications; these algorithms commonly emerge as brute-force solutions to problems. As n doubles, the running time quadruples.

n!

factorial

Commonly observed in algorithms that produce all permutations of a set with n elements.

Table 1.1: Basic asymptotic efficiency classes

Example 1.1: Let us consider a short piece of source code:

x = 3*y + 2; z = z + 1;

For scalars y and z, this code snippet consumes a fixed duration of time, denoted as O(1). In terms of concrete computer instructions or clock cycles, pinpointing the exact time it takes is challenging. However, whatever the duration, it remains consistent each time this code segment is executed. O(1) implies a certain constant, which could be 5, 1, or 1000, for instance.

Example 1.2: If a program 1 takes 100n2 milliseconds and program 2 takes 5n3 milliseconds, then might not 5n3 program be better than program 1?

Since programs can be assessed by comparing functions with respect to their running time, disregarding proportional constants, a program like 5n3 would be superior to a program like 100n2.

5 n3/100 n2 = n/20

Given inputs where n < 20, a running time of the program is 5n3 will outperform the one with a running time of 100n2. Consequently, if the program is primarily intended for small-sized inputs, opting for the program with a running time of O(n3) would be preferable.

Nonetheless, as n increases, the ratio of the running times, denoted as n/20, becomes progressively larger. Consequently, with an increase in the input size, the O(n3) program will require substantially more time compared to the O(n2) program. Therefore, it is consistently advisable to select a program with a lower growth rate in its running time. Functions with lower growth rates, such as O(n) or O(n log n), always offer superior performance.

Example 1.3: Let us consider the analysis of a simple for loop:

for (i = 1; i<=n; i++)

v[i] = v[i] + 1;

This loop iterates precisely n times, and since constant time is taken by inner loop, the overall running time is directly related to n. This is denoted as O(n). The specific number of instructions might be 50n, with the running time measured at 17n microseconds. It could even be 17n + 3 microseconds, accounting for the startup time of the loop. The big-O notation includes provisions for both a multiplication factor (such as 17) and an additive factor (such as 3). As long as it forms a linear function directly proportional to n, the appropriate notation remains O(n), and the code is classified as having linear running time.

Example 1.4: Let us look at a more complicated example where we will analyze a nested for loop:

for (i = 1; i<=n; i++)

for (j = 1; j<=n; j++)

a[i,j] = b[i,j] * x;

The outer for loop runs n times, and within each iteration of the outer loop, the inner loop executes n times. Thus, the inner loop runs n * n = n2 times. Since the assignment statement within the inner loop takes constant time, the overall running time of the code amounts to O(n2) steps. This code is classified as having quadratic running time.

Example 1.5: Let us consider an example of the analysis of matrix multiply.

The multiplication of two n × n matrices is denoted as A and B. The following code computes the matrix product C = A * B:

for (i = 1; i<=n; i++)

for (j = 1; j<=n; j++)

C[i, j] = 0;

for (k = 1; k<=n; k++)

C[i, j] = C[i, j] + A[i, k] * B[k, j];

With 3 nested for loops, each of them iterating n times, the innermost loop consequently runs n * n * n = n3 times. The innermost statement, involving a scalar sum and product, requires constant O(1) time. Consequently, the algorithm, on the whole, operates in O(n3) time.

General rules for the analysis of programs

In general, the runtime of a statement or group of statements can be described by the input size and/or one or more variables. The sole permissible parameter for the overall program's runtime is n, representing size of the input.

The runtime of each assignment read and write statement can typically be considered O(1):

• The runtime of a sequence of statements is determined by the sum rule, meaning the runtime of the sequence is, within a constant factor, the largest runtime among any statement within the sequence.

• The runtime of an if-statement comprises the cost of conditionally executed statements, along with the time for evaluating the condition. The time to evaluate the condition is usually O(1). For an if-then-else construct, the runtime is the time to evaluate the condition plus the greater of the time needed for the statements executed when the condition is true and the time for the statements executed when the condition is false.

• The time to execute a loop is the sum over all iterations of the loop, involving the time to execute the body and the time to evaluate the termination condition (usually the latter is O(1)). Frequently, this time, neglecting constant factors, is the product of the number of loop iterations and the maximum possible time for a single execution of the body. However, it is necessary to consider each loop separately to ensure accuracy.

Recurrence relation

A recurrence relation for a sequence of numbers S is an equation that links all terms of S, excluding a finite number of them, to preceding terms of the sequence, specifically {a0, a1, a2, …… , an-1}, for all integers n with n ≥ n0, where n0 is a non-negative integer. Recurrence relations are also known as difference equations.

Recurrence

A recurrence is a mathematical statement or inequality that characterizes a function by its values on smaller inputs.

Sequences are frequently defined in the most straightforward manner using a recurrence relation; nonetheless, directly applying a recurrence relation to compute terms can be a time-intensive task. The method of deriving a mathematical expression for the elements of a sequence from its recurring connection is recognized as solving the relation. Several trial-and-error methods for solving recurrence relations are as follows:

• Formulate simplifying assumptions regarding the inputs.

• Create a table of the initial values of the recurrence.

• Identify patterns and propose a solution.

• Generalize the outcome to eliminate the assumptions.

For example, factorial, Fibonacci, quick sort, binary search, etc.

A recurrence relation is an equation that is defined in reference to itself. No single technique or algorithm can be universally applied to solve all recurrence relations. In reality, some recurrence relations are unsolvable. The majority of the recurrence relations encountered are typically linear recurrence relations with constant coefficients. Various techniques, such as substitution, induction, characteristic roots, and generating function, are accessible for solving recurrence relations.

There are four approaches exist for resolving recurrence relations:

• Substitution method

• Iteration method

• Recursion tree method

• Master method

Substitution method

The substitution method involves two steps:

1. Predict the structure of the solution using symbolic constants.

2. Employ mathematical induction to verify the validity of the solution and determine the constants.

To utilize the inductive hypothesis, you substitute the anticipated solution for the function on smaller values, which is why the method is called the substitution method. While this approach is robust, it necessitates guessing the form of the solution. Despite the apparent challenge in generating a suitable guess, honing this skill through practice can swiftly enhance your intuition.

Example 1.6: Consider the recurrence .

We have to show that it is asymptotically bound by O (log n).

Solution: For T (n) = O (log n)

We have to show that for some constant c,

T(n) ≤ c log log n

Put this in the given recurrence equation:

Thus, T(n)=O(log n)

Example 1.7: Consider the recurrence

We have to show that it is asymptotically bound by O(log n).

Solution: For T (n) = O (n log n)

We have to show that for some constant c,

T(n) ≤ cn log log n

Put this in the given recurrence equation,

Thus, T(n)=O(n log n)

Example 1.8: Solve the recurrence: by making a change of variable.

Solution:

Suppose

Thus,

S(m)= T(2m)

We have,

We know the solution to above recurrence:

S(m)=O(m)

Substituting for m we get:

T(n)=S(m)=O(m)=O(logn)

Iteration method

The iterative method involves the fundamental concept of expanding the recurrence and representing it as a sum of terms that are solely dependent on n and initial conditions.

Example 1.9: Solve the recurrence: T(n)=T(n-1)+1 and T(1)=θ(1)

Solution:

T(n)=T(n-1)+1

=(T(n-2)+1)+1

=(T(n-3)+1)+1+1

=T(n-4)+4

=T(n-5)+5

=T(n-k)+k

Where k=n-1

That is T(n-k)=T(1)=θ(1)

That is T(n)=θ(1)+(n-1)=1+n-1=n= θ(n)

Example 1.10: Using the recurrence for the binary search given below, find the time complexity of binary search:

Solution:

Thus,

Now we choose k such that = 1

Thus, n=2k ⇒ k=n

T(n)=T(1)+n

Since, T(1)=1 we get T(n)=n +1

Thus the time complexity of the binary search algorithm is n.

Recursion tree method

The method of recursion tree analysis is employed to solve repetitive relationships. It entails converting a repetitive relationship into a series of recursive trees, where every node represents the costs accumulated at different stages of recursion. By adding up the expenses at each stage, the total cost can be ascertained. The steps involved in addressing a repetitive relationship using the recursion tree technique are as follows:

1. Draw a recursive tree that corresponds to the provided repetitive relationship.

2. Calculate the expense at each stage and determine the total number of stages in the recursion tree.

3. Count the total number of nodes at the final stage and evaluate the expenses associated with the last stage.

4. Combine the costs across all stages in the recursive tree.

Example 1.11: Solve the recurrence relation: T(n)=2T+c using recursion tree method.

Solution:

1. Generate a tree through recursion as shown in Figure 1.5:

Figure 1.5:Recursion tree

2. Determine the work accomplished or cost at each level of a tree and count the total levels in the recursion tree as shown in Figure 1.6:

Figure 1.6: Recursion tree

Calculate the entire number of levels.

Select the most extended path from the starting node to the endpoint node:

Size of problem at last level =. At the last level, size of the problem becomes 1:

Total number of levels in recursive tree =k+1=n+1

3. Count the total number of nodes in the last level and calculate the cost of the last level.

No. of nodes at level 0 = 20 = 1

No. of nodes at level 1 = 21 = 2

No. of nodes at level n =2n=nn =n

Cost of sub-problems at level n (last level) = n*T(1) = n*1 = n

4. Sum up the cost of all the levels in the recursive tree.

T (n) = c + 2c + 4c + —- + (no. of levels-1) times + last level cost

= c + 2c + 4c + —- +n times + θ (n)

= c (1 + 2 + 4 + —- +n times) + θ (n)

1 + 2 + 4 +… +n times → 20 + 21 + 22 +… +n times

The aforementioned series is a geometric progression (GP), so:

= c (n) + θ (n)

Thus, T (n) = θ (n)

Master’s method

The master method functions as a methodical strategy for identifying the asymptotic patterns of a broad spectrum of recurring equations. Its application is tailored for the resolution of recurring equations structured as:

For an integer constant d > 1, positive constants a > 0, c > 0, and b > 1, and a positive function f(n) for n ≥ d, the master method's approach to solving these recurring equations relies on directly presenting the solution based on the application of one of three distinct scenarios. These cases are determined by contrasting f(n) with a particular function, na.

Theorem 1.1: Let f(n) and T(n) be defined as previously mentioned.

We have three cases:

• Case 1: If there is a small constant ϵ> 0, such that f(n) is θ(na-ϵ), then T(n) is θ(na).

• Case 2: If there is a constant K≥0, such that f(n) is θ(na logkn), then T(n) is
θ(na logk+1n).

• Case 3: If there are small constant ϵ>0 and δ<1, such that f(n) is Ω(na+ϵ) and a f(n/b)<δ f(n), for n≥d, then T(n) is θ(f(n)).

Case 1 identifies situations where f(n) diminishes polynomially compared to the special function, na.

Case 2 distinguishes situations in which f(n) approaches the special function asymptotically.

Case 3 outlines scenarios in which f(n) grows polynomially as compared to the special function.

The application of the master method is demonstrated through a few examples (with each taking the assumption that T(n) = c for n < d, for constants c > 1 and d > 1).

Example 1.12: Solve the recurrence relation:

Solution: Here a = 9, b = 3, f(n) = n, and na = n9 = θ(n2).

Since f(n) = O(n9-ϵ) for ϵ = 1, case 1 of the master theorem applies.

The solution is T(n) = θ(n2).

Example 1.13: Solve the recurrence relation:

Solution: Here a = 1, b = 3/2, f(n) = 1, and na =n0 = 1.

Since, f(n) = θ(na), Case 2 of the master theorem applies.

So, the solution is T(n) = θ(log n).

Conclusion

This chapter highlights the critical importance of optimizing computational efficiency, minimizing resource usage, and enhancing overall system responsiveness. By giving priority to these factors, it becomes feasible to attain accelerated processing speeds, decrease execution time, and ensure the seamless management of extensive data and increased computational requirements. These endeavors contribute to a more efficient and effective utilization of computational resources, ultimately resulting in enhanced program performance and user satisfaction.

The next chapter will provide an overview of essential data structures, frequently employed in the field of computer science and programming.

Exercise

1. Obtain the running time for the following sample snippets:

a) for(i←2 to k-1)

{

for(j←3 to k-2)

{

A←A+2;

}

}

b) max←A[0]

for(i←1 to n-1)

do if max<A[i]

then max←A[i]

return max

2. Arrange the following growth rates in the increasing order:

O(n4), O(1), O(n3), O(n2logn), Ω(n1.5), θ(n1.5), θ(nlogn)

3. Can the master method be applied to the recurrence relation, (n)=4T+n? Give the reason of your answer. Also give an asymptotic upper bound for this recurrence.

4. Solve the following recurrence relations:

(a)

(b)

(c)

(d)

5. Use a recursion tree to give an asymptotic tight solution to the recurrence:

T(n)=T(n-a)+T(a)+cn Where a ≥ 1 and c > 0 are constants.

6. Solve the following recurrence relation.

CHAPTER 2Review of Data Structures

Introduction

In this chapter, we will present a review of data structures that involves revisiting fundamental concepts in computer science that deal with organizing and storing data in efficient and accessible ways. Data structures provide a framework for managing data effectively, allowing for optimized operations such as insertion, deletion, retrieval, and manipulation. The chapter examines the primary classifications of data structures, including arrays, linked list, queues, stacks, and trees, with an emphasis on their respective advantages and limitations. The discourse encompasses the significance of carefully choosing an appropriate data format for particular jobs in order to enhance efficiency and optimize performance. In this chapter, we have used different linear and non-linear data structures with their algorithms and solved examples. It subsequently indicates the complexity of all the algorithms.

Structure

The chapter covers the following topics:

• Introduction to data structures

• Array

• Stack

• Queue

• Pointers

• Linked lists

• Hashing

• Trees

• Heap

Objectives

The focus of this chapter is to provide a comprehensive overview of essential data structures, frequently employed in the field of computer science and programming, with their strengths, weaknesses, and applications of various data structures. The chapter also aims to facilitate a deeper understanding of how the choice of data structure impacts algorithm design and computational efficiency.

Introduction to data structures

Data structures are a fundamental concept in computer science that involves the organization, storage, and manipulation of data in a way that enables efficient access and modification. They are essential for handling various computational issues and form the building blocks of software and algorithms. It is a form of storage utilized to store and arrange data. Data organization on a computer is a systematic method employed to facilitate quick access and updating of information.

Purpose of data structures

Data structures are designed to achieve specific objectives, such as:

• Enhancing efficiency: The main purpose of data structures is to make it easier to view and change data. This efficiency is very important for making algorithms and software systems work better. Developers can greatly cut down on the time and resources needed to process and handle data by picking the right data structure for each job.

• Abstraction: Data structures provide a level of abstraction that makes it easier to work with and handle large amounts of complex data in software development. They protect coders from the details of managing memory and storing data by letting them work with data in a more abstract and high-level way. This level of abstraction makes it easier to see how data can be used in a problem or application.

• Organization: When it comes to solving a problem or accomplishing a task, data structures are crucial for arranging information in a way that makes sense. They allow for information to be arranged in a way that is both readable and easily accessible.

The following figure represents the classification of data structure into two categories, i.e., linear and non-linear data structure:

Figure 2.1: Classification of data structures

When selecting data structures, you must perform the following steps:

1. Analysis of the problem to determine basic operations like insert, delete, and search data in data structures.

2. Determine the exact number of resources that can be used for each task.

3. Figure out which data structure is ideal for these needs:

a. Data and operations that are to be performed on them.

b. Representation of data.

c. Implementation of that representation.

Array

An array is a fixed-size, sequence grouping of items with the same kind of data. So, it is a linear data structure, i.e., the collection of homogeneous data elements at contiguous memory locations.

Refer to the following example, int Age[10], showing 10 data items indexed from 0 to 9:

Age 0

Age 1

Age 2

Age 3

Age 4

Age 5

Age 6

Age 7

Age 8

Age 9

40

45

32

26

29

23

34

30

12

11

Table 2.1: Example of an array

Declaring and storing an array

An array declaration can be explained in three different ways:

1. int Ar[3] = {11, 12, 25} [Compile Time]

2. int i, Ar[3]; [Run time]

for (i=0 ; i< 3; i++)

Ar[i] = i;

3. int i, Ar[3]; [Run time]

for (i=0 ; i< 3 ; i++)

scanf(“%d”, &Ar[i]);

Some applications of array data structures are mentioned as follows:

• Contacts on a cell phone indicate the array structure and maintain dictionary order.

• Book titles in a library management system represent an array formation.

• Speech processing involves representing each speech signal as an array.

• The display screen is also a multidimensional array of pixels.

Address calculation of array elements

In this section, we will be able to pinpoint memory precisely, where a single array element is stored. The initial byte of the array is represented by the address of the array's name. By referring to the array by its name, we are referring to the array's initial byte. The index or subscript specifies the position in the array relative to the array's initial position. Since an array's data elements are kept in successive memory locations, it is sufficient to keep track of only the base address, or the location of the first member in the array. Using the starting point's address, we can easily determine the location of any further data components. The calculating formula is as follows:

Address of array data element, Arr[k] = Base_Address[Arr] + w(k – lower_bound).

Example 2.1: An array int S_marks[ ] = {98,65,77,54,89,91,32,84} is shown. Determine the address of S_marks[4] if the base address is 1000:

Figure 2.2: Array S_marks structure

Solution: Since we know that two bytes are needed to store an integer value. Therefore, its size is two bytes, shown as follows:

S_marks[4] = 1000 + 2(4 – 0)

= 1000 + 2(4) = 1008

Declaration of 2D array

The most basic type of multi-dimensional array in C is the 2D array, or two-dimensional array, which can be thought of as a table with X rows and Y columns. The row numbers fall within the range of 0 to (X-1), and the column numbers fall within the range of 0 to (Y-1).

data_type array_name[rowsize] [colsize];

This statement shows the declaration syntax of 2D array.

int marks[3][3]; represents 2D array name marks having 3 rows and 3 columns. The memory representation is as follows:

00

01

02

10

11

12

20

21

22

marks

Table 2.2: 2D array structure of 3X3

These elements are stored sequentially in two ways, row major order (RMO) and column major order (CMO), described as follows:

• RMO:

(0,0)
(0,1)
(0,2)
(1,0)
(1,1)
(1,2)
(2,0)
(2,1)
(2,2)

Table 2.3: 2D array structure using RMO

The preceding structure shows the sequential RMO structure that reads 2D array row-wise.

A[M][N];

The formula to calculate address of any data element in 2D array using RMO is as follows:

Address A[i, j] = Base_Address+ w*{ N(i-1) + (j-1)}

The equation demonstrates the following:

• w is the number of words stored per memory location.

• N is the number of elements in one row (number of columns).

• i and j are the subscripts of the array element.

Example 2.2: An array int A[20][5] is shown with Base_Address=1000, and number of words per memory location=2. Compute the address of element A[18,4]. Assume the elements are stored in RMO.

Solution: Address of A[i, j] = Base_Address+ w * [ N*(i-1) + (j-1) ]

Address of A[18,4] = 1000 + 2 * [ 5 * (18-1) + (4-1) ]

= 1000 + 2 * [85 + 3]

= 1176

• CMO:

2

1

3

4

3

6

6

5

9

(0,0)
(1,0)
(2,0)
(0,1)
(1,1)
(2,1)
(0,2)
(1,2)
(2,2)

Table 2.4: 2D array structure using CMO

The preceding structure shows the sequential CMO structure that reads the 2D array column-wise:

2 4 6

1 3 5

3 6 9

A[M][N];

The formula to calculate the address of any data element in 2D array using CMO is as follows:

Address A[i, j] = Base_Address+ w*{ M(j-1) + (i-1)}

The equation demonstrates the following:

• w is the number of words stored per memory location.

• ī is the number of elements in one column (number of rows).

• i and j are the subscripts of the array element.

Example 2.3: The array int A[10][10], Base_Address=1000, and number of words per memory location=2. Compute the address of element A[8,5]. Assume the elements are stored in CMO.

Solution: Address of A[i, j] = Base_Address+ w * [ M*(j-1) + (i-1) ]

Address of A[8,5] = 1000 + 2 * [ 10 * (5-1) + (8-1) ]

= 1000 + 2 * [40 + 7]

= 1094

Array operations

Several operations can be executed on arrays, encompassing the following tasks:

• Traversing an array

• Inserting an item into an array

• Searching for an item in an array

• Deleting an item from an array

• Merging two arrays

• Sorting an array in an ascending or descending order

Let us discuss the topics in detail:

• Traversal: In order to perform a traversal operation in an array, or just traverse an array, you must access or print every array element precisely once so, that you can verify or use the data items (values) of the array in some other operation or process (sometimes referred to as visiting the array). The following shows the initialization of an array for traversal:

A[] is the array.

The lower bound is the starting index.

The upper bound is the last index.

Algorithm 2.1: Array traversal

1. Set i = lower_bound

2. Repeat Steps 2 to 4 while i < = upper_bound

3. Print (A[i])

4. Set i = i + 1

5. Exit

First, we set the index to the array's minimum value. The second step is a while loop. The third step involves handling each array element according to its index value and name. The index is increased in Step 4 so that the next element in the array may be accessed and worked with. Step 2's while loop runs until the array's elements have been processed, or until i is greater than the upper bound of the array.

Program 2.1: Using an array to read and show N numbers

#include <stdio.h>

#include <conio.h>

int main()

{

int i, n, arr[20];

clrscr();

printf("\n Enter the number of elements in the array : ");

scanf(“%d”, &n);

for(i=0;i<n;i++)

{

printf(“\n arr[%d] = “, i);

scanf("%d",&arr[i]);

}

printf("\n The array elements are ");

for(i=0;i<n;i++)

printf("\t %d", arr[i]);

return 0;

}

Output:

Enter the number of elements in the array: 5

arr[0] = 10

arr[1] = 20

arr[2] = 30

arr[3] = 40

arr[4] = 50

The array elements are 10 20 30 40 50

Adding an item to the end of an existing array is a straightforward operation. We can easily set the value by adding one to the current upper_bound. Here, the array's allotted memory area is still accessible. For example, if an array is stated to have ten components, but now it has only eight elements, then there is space to accommodate two additional elements. We cannot add anything else if there are already ten elements in there.

Algorithm 2.2: Array insertion at the end

The algorithm is as follows:

1. Set upper_bound = upper_bound + 1

2. Set A[upper_bound] = VALUE

3. EXIT

The process of adding an item to the end of an array is done by using an algorithm. We increase the upper_bound by one in the first step. In Step 2, the upper_bound is used to determine where to put the updated value:

• Insert an element in the middle of an array: The algorithm INSERT is declared as INSERT (Arr, Num, POS, VAL).

The arguments for the insertion of an element into an array are as follows:

o Arr, represents the array in which the element is to be inserted.

o Num, indicating the total number of elements currently present in the array.

o POS, denoting the desired spot, where the component will be placed.

o VAL represents the specific value that is to be inserted into the array.

Algorithm 2.3: Demonstrating the algorithm for insertion in the array in the middle

The algorithm is as follows:

1. Set i = Num.

2. Repeat Steps 3 and 4 while i > = POS .

3. Set Arr[i + 1] = A[i].

4. Set i = i – 1.

5. [End of the Loop]

6. Set Num = Num + 1.

7. Set Arr[POS] = VAL.

8. EXIT

In the first step, the variable i is initialized with the entire number of elements in the array. During Step 2, a while loop is conducted in order to shift all elements with an index larger than POS, one place to the right, creating room for the insertion of a new element. In Step 5, the total number of elements in the array is incremented by 1. At the end, in Step 6, the new value is placed at the desired position. Figure 2.3 demonstrates the insertion of a data element at the middle in a 1D array:

Figure 2.3: Insertion at middle

Program 2.2: Put a value into an array at a specific spot

#include <stdio.h>

#include <conio.h>

int main()

{

int i, n, num, pos, arr[10];

clrscr();

printf("\n Enter the number of elements in the array : ");

scanf(“%d”, &n);

for(i=0;i<n;i++)

{

printf(“\n arr[%d] = “, i);

scanf("%d", &arr[i]);

}

printf("\n Enter the number to be inserted : ");

scanf("%d", &num);

printf("\n Enter the position at which the number has to be added : ");

scanf(“%d”, &pos);

for(i=n–1;i>=pos;i––)

arr[i+1] = arr[i];

arr[pos] = num;

n = n+1;

printf("\n The array after insertion of %d is : ", num);

for(i=0;i<n;i++)

printf("\n arr[%d] = %d", i, arr[i]);

getch();

return 0;

}

Output:

Enter the number of elements in the array: 5

arr[0] = 1

arr[1] = 2

arr[2] = 3

arr[3] = 4

arr[4] = 5

Enter the number to be inserted: 0

Enter the position at which the number has to be added: 3

The array after the insertion of 0 is:

arr[0] = 1

arr[1] = 2

arr[2] = 3

arr[3] = 0

arr[4] = 4

arr[5] = 5

• Deletion: The act of deleting an element from an array involves the removal of a specific data element from an array that already exists. If the removal of an element is required from the end of the given array, then the process of deletion can be considered straightforward. It is necessary to do a subtraction operation by decrementing the value of the upper bound by one.

Algorithm 2.4: Array deletion at the end

The algorithm is as follows:

1. Set upper_bound = upper_bound – 1

2. EXIT

The process of removing an item at the end of an array is done by using an algorithm. We decrease the upper_bound by one in the first step, then return. Delete an element from the middle of an array. The algorithm deletion is stated as DELETE (Arr, Num, POS).

The arguments for the deletion of an element from an array are as follows:

• Arr, represents the array in which the element is to be inserted.

• Num, indicating the total number of elements currently present in the array.

• POS, denoting the desired position at which the element is to be deleted.

Algorithm 2.5: Insertion in the array at the middle:

1. Set i = POS

2. Repeat step 3 & 4 while i < = Num – 1

3. Set Arr[i] = Arr[i + 1]

4. Set i = i + 1

5. [End of the Loop]

6. Set Num = Num - 1

7. EXIT

The algorithm commences by initializing the variable I, with the position from which the element is to be eliminated. During Step 2, a while loop is implemented to shift any elements with an index larger than POS one position to the left, in order to fill the gap left by the removed element. When referring to the act of deleting an element, it is more accurate to state that the element is overwritten with the value of its subsequent element. In the fifth step, the total number of elements in the array is reduced by one through the process of decrementing. The complete process of deletion of a data element at a given location in 1D array is explained in Figure 2.4:

Figure 2.4: Deletion of an element at given position

Program 2.3: Delete a number from a given location in an array

#include <stdio.h>

#include <conio.h>

int main()

{

int i, n, pos, arr[10];

clrscr();

printf("\n Enter the number of elements in the array : ");

scanf(“%d”, &n);

for(i=0;i<n;i++)

{

printf(“\n arr[%d] = “, i);

scanf("%d", &arr[i]);

}

printf("\nEnter the position from which the number has to be deleted : ");

scanf(“%d”, &pos);

for(i=pos; i<n–1;i++)

arr[i] = arr[i+1];

n––;

printf("\n The array after deletion is : ");

for(i=0;i<n;i++)

printf("\n arr[%d] = %d", i, arr[i]);

getch();

return 0;

}

Output:

Enter the number of elements in the array : 5

arr[0] = 1

arr[1] = 2

arr[2] = 3

arr[3] = 4

arr[4] = 5

Enter the position from which the number has to be deleted : 3

The array after deletion is :

arr[0] = 1

arr[1] = 2

arr[2] = 3

arr[3] = 5

• Merging two arrays: The process of merging two arrays into a third array involves initially duplicating the first array's elements into the third array, followed by duplicating the second array's elements into the third array. Consequently, the merged array comprises the first array's elements followed by the second array's elements. In the case of unsorted arrays, the process of merging them is straightforward, as it involves merely duplicating one array's elements into another. However, the process of merging becomes more complex when dealing with sorted arrays, as the resulting merged array must likewise maintain a sorted order. Firstly, let us initiate a discussion regarding the merge operation applied to arrays that have not been sorted. The following structure shows the merging of two arrays:

55

45

57

41

Table 2.5: Array 1

22

43

84

92

73

Table 2.6: Array 2

The merged array is named array 3 as follows:

55

45

57

41

22

43

84

92

73

Table 2.7: Array 3

Program 2.4: Merge two unsorted arrays

#include <stdio.h>

#include <conio.h>

int main()

{

int arr1[10], arr2[10], arr3[20];

int i, n1, n2, m, index=0;

clrscr();

printf("\n Enter the number of elements in array1 : ");

scanf("%d", &n1);

printf("\n\n Enter the elements of the first array");

for(i=0;i<n1;i++)

{

printf(“\n arr1[%d] = “, i);

scanf("%d", &arr1[i]);

}

printf("\n Enter the number of elements in array2 : ");

scanf("%d", &n2);

printf("\n\n Enter the elements of the second array");

for(i=0;i<n2;i++)

{

printf(“\n arr2[%d] = “, i);

scanf(“%d”, &arr2[i]);

}

m = n1+n2;

for(i=0;i<n1;i++)

{

arr3[index] = arr1[i];

index++;

}

for(i=0;i<n2;i++)

{

arr3[index] = arr2[i];

index++;

}

printf("\n\n The merged array is");

for(i=0;i<m;i++)

printf("\n arr[%d] = %d", i, arr3[i]);

getch();

return 0;

}

Output:

Enter the number of elements in array1 : 3

Enter the elements of the first array

arr1[0] = 1

arr1[1] = 2

arr1[2] = 3

Enter the number of elements in array2 : 3

Enter the elements of the second array

arr2[0] = 4

arr2[1] = 5

arr2[2] = 6

The merged array is

arr[0] = 1

arr[1] = 2

arr[2] = 3

arr[3] = 4

arr[4] = 5

arr[5] = 6

Merging two sorted arrays

When confronted with the requirement of merging two sorted arrays into a resultant array that must likewise be sorted, the process of merging becomes more challenging. The process of merging can be elucidated through the utilization of the following tables:

41

45

55

57

Table 2.8: Array 1

22

43

73

84

92

Table 2.9: Array 2

The merged table is as follows:

22

41

43

45

55

57

73

84

92

Table 2.10: Array 3

The example demonstrates the process of creating a merged array by combining two arrays that are already sorted. Here, we first compare the first element of array1 with the first element of array 2 and then put the smaller element in the merged array. Since 41 > 22, we put 22 as the first element in the merged array. We then compare the second element of the second array with the first element of the first array. Since 41 < 43, now 41 is stored as the second element of the merged array. Next, the second element of the first array is compared with the second element of the second array. Since 45 > 43, we store 43 as the third element of the merged array. Now, we will compare the second element of the first array with the third element of the second array. Because 45 < 73, we store 45 as the fourth element of the merged array. This procedure will be repeated until elements of both arrays are placed in the right location in the merged array.

Program 2.5: Merge two sorted arrays

#include <stdio.h>

#include <conio.h>

int main()

{

int arr1[10], arr2[10], arr3[20];

int i, n1, n2, m, index=0;

int index_first = 0, index_second = 0;

clrscr();

printf("\n Enter the number of elements in array1 : ");

scanf("%d", &n1);

printf("\n\n Enter the elements of the first array");

for(i=0;i<n1;i++)

{

printf(“\n arr1[%d] = “, i);

scanf("%d", &arr1[i]);

}

printf("\n Enter the number of elements in array2 : ");

scanf("%d", &n2);

printf("\n\n Enter the elements of the second array");

for(i=0;i<n2;i++)

{

printf(“\n arr2[%d] = “, i);

scanf(“%d”, &arr2[i]);

}

m = n1+n2;

while(index_first < n1 && index_second < n2)

{

if(arr1[index_first]<arr2[index_second])

{

arr3[index] = arr1[index_first];

index_first++;

}

else

{

arr3[index] = arr2[index_second];

index_second++;

}

index++;

}

// if elements of the first array are over and the second array has some elements

if(index_first == n1)

{

while(index_second<n2)

{

arr3[index] = arr2[index_second];

index_second++;

index++;

}

}

// if elements of the second array are over and the first array has some elements

else if(index_second == n2)

{

while(index_first<n1)

{

arr3[index] = arr1[index_first];

index_first++;

index++;

}

}

printf("\n\n The merged array is");

for(i=0;i<m;i++)

printf("\n arr[%d] = %d", i, arr3[i]);

getch();

return 0;

}

Output:

Enter the number of elements in array1 : 3

Enter the elements of the first array

arr1[0] = 1

arr1[1] = 3

arr1[2] = 5

Enter the number of elements in array2 : 3

Enter the elements of the second array

arr2[0] = 2

arr2[1] = 4

arr2[2] = 6

The merged array is

arr[0] = 1

arr[1] = 2

arr[2] = 3

arr[3] = 4

arr[4] = 5

arr[5] = 6

Stack

A stack is a type of linear data structure that adheres to the notion of adding and removing elements exclusively from one end, known as the top. Therefore, a stack is sometimes referred to as a last-in-first-out (LIFO) data structure, as the piece that was most recently put, is the first one to be removed.

The basic operations that can be performed on the stacks are push, pop, and peek. Stacks are widely used in computer science for a variety of applications with the inclusion of function calls, evaluation of expression, and memory management.

Figure 2.5 illustrates a stack of plates arranged one over the other. This visual metaphor is commonly used to explain the LIFO principle in data structures.

Figure 2.5: Stack of plates

The other plate will be placed over the TOP plate, and this new inserted plate will be removed first.

The concept of stacks can be elucidated through the use of an analogy. Consider a stack of plates, as depicted in the figure, where each plate is positioned atop another. When desiring to remove a plate, the topmost plate is the first to be taken away. Consequently, the addition and removal of an element, represented by a plate, can solely occur at the highest position within the stack.

Some applications of stack data structures are mentioned as follows:

• Reversing a list

• Parentheses checker

• Conversion of an infix expression into a postfix expression

• Evaluation of a postfix expression

• Conversion of an infix expression into a prefix expression

• Evaluation of a prefix expression

• Tower of Hanoi using recursion

• Undo-redo operation, etc.

Stacks can be implemented using either arrays or linked lists. In the following sections, we will discuss the array implementation of stacks.

Stack operations

The stack data structure facilitates three fundamental operations, namely push, pop, and peek. The push action is responsible for appending an element to the TOP of the stack, while the pop operation is responsible for removing the element from the TOP of the stack. The peek operation retrieves the value of the element located at the TOP of the stack.

Push operation

The push action is employed to insert an element onto the stack. The addition of the new element occurs at the highest place inside the stack. Prior to inserting the value, it is imperative to verify whether TOP is equal to MAX-1. This is necessary since it indicates that the stack has reached its maximum capacity, rendering any further insertions impossible. If an endeavor is made to insert a value into a stack that has already reached its maximum capacity, a notification indicating an overflow condition is displayed:

6

TOP

6

6

5

5

5

4

4

TOP

4

50

TOP

3

23

3

23

3

23

2

17

2

17

2

17

1

25

1

25

1

25

0

15

0

15

0

15

Index

Stack, S

Index

Stack, S

MAX=7

MAX=7

Table 2.11: Stack push operation

The diagram illustrates the algorithm for inserting an element into a stack. In the initial step, the first task is to examine the presence of the OVERFLOW condition. During Step 2, the value of TOP is incremented in order to indicate the subsequent location inside the array. During the third step, the value is put into the stack at the specific location indicated by the TOP pointer.

Algorithm 2.6: Insert an element in the stack

1. If TOP = MAX – 1, print OVERFLOW. Go to Step 4. [End of IF].

2. Set TOP = TOP + 1.

3. Set Stack[TOP] = Value.

4. END

Pop operation

The pop action is utilized to remove the uppermost element from the stack. Prior to deleting the value, it is imperative to verify if TOP=NULL, as this condition indicates that the stack is devoid of elements and further deletions are not feasible. If an endeavor is made to remove a value from a stack that is now devoid of elements, an underflow notification is generated. The following table illustrates the algorithm for inserting an element into a stack:

6

TOP

6

6

5

5

5

4

4

4

3

23

3

23

TOP

3

23

2

17

2

17

2

17

TOP 

1

25

1

25

1

25

0

15

0

15

0

15

Index

Stack, S

Index

Stack, S

MAX=7

MAX=7

Table 2.12: Stack pop operation

In the initial step, the first task is to examine the presence of the OVERFLOW condition. During Step 2, the value of TOP is incremented in order to indicate the subsequent location inside the array. During the third step, the value is put into the stack at the specific location indicated by the TOP pointer.

Algorithm 2.7: Delete an element from a stack

1. If TOP = NULL, print UNDERFLOW. Go to Step 4. [End of IF]

2. Set Value = Stack[TOP].

3. Set TOP = TOP – 1.

4. END

In order to remove the element at the top of the stack, it is necessary to first verify if the value of the TOP pointer is equal to NULL. If the condition is determined to be false, the value referenced by the variable TOP is subsequently decreased.

Peek operation

The peek operation is a function that retrieves the value of the topmost element in a stack, without removing it from the stack. The peek action involves an initial check to determine if the stack is empty. This check is performed by verifying if the TOP pointer is NULL. If the stack is indeed empty, an appropriate message is written. However, if the stack is not empty, the value is returned:

6

5

4

3

23

TOP 

2

17

1

25

0

15

Index

Stack, S

MAX=7

Table 2.13: Stack peek operation

The peek action will yield the number 23, as it corresponds to the element situated at the top of the stack.

Algorithm 2.8: Peek operation in the stack

1. If TOP = NULL, Print STACK IS EMPTY, then go to Step 3.

2. Return Stack[TOP]

3. END

Program 2.6: Push, pop, and peek operations on a stack

#include <stdio.h>

#include <stdlib.h>

#include <conio.h>

#define MAX 3 // Altering this value changes size of stack created

int st[MAX], top=-1;

int main(int argc, char *argv[]) {

int val, option;

do

{

printf (“\n *****MAIN MENU*****\n 1. PUSH \n 2. POP \n 3. PEEK \n 4. DISPLAY\n 5. EXIT “);

printf("\n Enter your option: ");

scanf("%d", &option);

switch(option)

{

case 1: printf("\n Enter the number to be pushed on stack: ");

scanf("%d", &val);

push(st, val);

break;

case 2: val = pop(st);

if(val != -1)

printf("\n The value deleted from stack is: %d", val);

break;

case 3: val = peek(st);

if(val != -1)

printf("\n Value stored at top of stack is: %d", val);

break;

case 4: display(st);

break;

} while(option != 5);

return 0;

}

void push(int st[], int val){

if(top == MAX-1)

printf("\n STACK OVERFLOW");

else{

top++;

st[top] = val;

}

}

int pop(int st[]){

int val;

if(top == -1){

printf("\n STACK UNDERFLOW");

return -1;

}

else{

val = st[top];

top--;

return val;

}

}

void display(int st[]){

int i;

if(top == -1)

printf("\n STACK IS EMPTY");

else

{

for(i=top;i>=0;i--)

printf("\n %d",st[i]);

printf("\n"); // Added for formatting purposes

}

}

int peek(int st[])

{

if(top == -1)

{

printf("\n STACK IS EMPTY");

return -1;

}

else

return (st[top]);

}

Output:

*****MAIN MENU*****

1. PUSH

2. POP

3. PEEK

4. DISPLAY

5. EXIT

Enter your option : 1

Enter the number to be pushed on stack : 555

Linked list representation of stack

The process of creating a stack by using an array has been observed. The process of constructing a stack by using this method is quite straightforward; nevertheless, it is important to note that a limitation of this approach is the requirement to declare the array with a pre-determined size. If the stack is either relatively tiny or its maximum size is pre-determined, the array implementation is typically chosen over the linked representation.

The linked representation of a stack with n members has a storage need of O(n), and the operations typically have a time requirement of O(1).

Queue

When an element is added to a queue, it will be withdrawn in the same order that it was added. This data structure is known as a first-in, first-out (FIFO) queue. Elements are added to a queue at one end, called the REAR, and taken out at the other end, called the FRONT.

Both arrays and linked lists are viable implementations of queues. This section will demonstrate the implementation of queues by using each of these data types.

Queue can handle multiple data items. We can access the data elements in the queue from both ends and they are flexible and fast to access. Figure 2.6 shows the basic structure of queue with two pointer FRONT and REAR:

Figure 2.6: Structure of queue

The applications of queue data structures are mentioned as follows:

• Printers and single shared resources utilize the queue structure.

• Both slow and fast devices synchronize with the queue.

• In networking, queue is used in switch or router and mail queue.

• Within a single source program, there may be multiple queue requests.

Queue operations

Linear arrays provide a simple and efficient way to represent queues. As previously mentioned, each queue is equipped with FRONT and REAR variables that indicate the positions where deletions and insertions can be performed, respectively. The following table displays the array representation of the queue:

0

1

2

3

4

5

11

16

21

31

41

Initial queue

0

1

2

3

4

5

11

16

21

31

41

46

Queue after insertion of new element 46

0

1

2

3

4

5

16

21

31

41

46

Queue after deletion of an element 11

Table 2.14: Queue insertion and deletion operation

Let us discuss insertion and deletion in the queue.

In the example, we will consider FRONT = 0 and REAR = 4. We would save the value in the position pointed by REAR and increment REAR by 1 if we wanted to add another element with a value of 46. Following addition, the queue would look like this: FRONT = 0 and REAR = 5. We always use the same method when adding new elements. To remove an item from a queue, we increment the FRONT value. After the deletion, FRONT becomes 1 and REAR becomes 5.

Prior to entering an element into a queue, it is necessary to verify if there are any overflow circumstances. An overflow will arise if we attempt to enter an element into a queue that is already at its maximum capacity. An overflow condition occurs when the value of REAR is equal to MAX - 1, where MAX represents the size of the queue. Please note that we have used MAX - 1 in our expression to account for the fact that the index begins at 0.

Prior to removing an element from a queue, it is necessary to verify if there are any underflow circumstances. An underflow problem arises while attempting to remove an element from a queue that is already devoid of any elements. If the value of FRONT is -1 and the value of REAR is -1, it indicates that the queue is empty and there are no elements present.

Algorithm 2.9: Insert an element in a queue

1. IF variable REAR = MAX – 1, print OVERFLOW message. Go to Step 4. [END of IF]

2. IF FRONT =-1 and REAR = -1, SET FRONT = 0, REAR = 0. Else, SET REAR++. [END of IF].

3. Set Queue [REAR] = Value

4. END

Algorithm 2.10: Remove an item from a given queue

1. If FRONT == -1 or FRONT > REAR, print OVERFLOW. Else, set value = Queue[FRONT], and set FRONT = FRONT + 1. [End of IF]

2. END

Program 2.7: Insert, delete, and peek operations in a queue

#include <stdio.h>

#include <stdlib.h>

#include <conio.h>

#define MAX 10

int queue[MAX], int front = -1, rear = -1;

int main(){

int option, val;

do{

printf(“\n\n ***** MAIN MENU *****\n1. Insert \n2. Delete \n3. Peek \n3. Display \n5. Exit”);

printf(“\n Enter your option : “);

scanf(“%d”, &option);

switch(option)

{ case 1: insert();

break;

case 2: val = delete_element();

if (val != -1)

printf(“\n The number deleted is : %d”, val);

break;

case 3: val = peek();

if (val != -1)

printf(“\n The first value in queue is : %d”, val);

break;

case 4: display();

break;

}

}while(option != 5);

return 0;

}

void insert(){

int num;

printf(“\n Enter the number to be inserted in the queue: “);

scanf(“%d”, &num);

if(rear == MAX-1)

printf(“\n OVERFLOW”);

else if(front == -1 && rear == -1)

front = rear = 0;

else

rear++;

queue[rear] = num;

}

int delete_element(){

int val;

if(front == -1 || front>rear){

printf(“\n UNDERFLOW”);

return -1;

}

else{

val = queue[front];

front++;

if(front > rear)

front = rear = -1;

return val;

}

}

int peek(){

if(front==-1 || front>rear){

printf(“\n QUEUE IS EMPTY”);

return -1;

}

else

return queue[front];

}

void display(){

int i;

printf(“\n”);

if(front == -1 || front > rear)

printf(“\n QUEUE IS EMPTY”);

else{

for(i = front;i <= rear;i++)

printf(“\t %d”, queue[i]);

}

}

Output:

***** MAIN MENU *****"

1. Insert

2. Delete

3. Peek

4. Display

5. EXIT

Enter your option : 1

Enter the number to be inserted in the queue : 50

If the queue is either relatively tiny or its maximum size is predetermined, then implementing the queue by using an array is an efficient approach. However, if the size of the array cannot be pre-determined, the alternative option, namely the linked form, is employed.

The storage complexity of a queue implemented by using a linked representation with n members is O(n), but the time complexity for operations is typically O(1).

The different types of queues are mentioned as follows:

• Circular queue: Assuming we wish to add a new element to the queue, despite having adequate space, the overflow situation persists if the condition REAR = MAX -1 remains true. A significant limitation of a linear queue is evident. In order to address this issue, we have two potential resolutions. Initially, perform a leftward shift of the elements to create a vacant space that can be effectively occupied and exploited. However, this process can be exceedingly time-consuming, particularly when the backlog is of considerable size.

An alternative is to employ a circular queue. In a circular queue, the first index immediately follows the last index. It is filled only when the FRONT pointer is at position zero and the REAR pointer is at position MAX - 1. A circular queue is implemented by using the same approach as a linear queue. The sole distinction will be in the code that executes insertion and deletion operations. When it comes to insertion, it is now necessary to verify the following three conditions:

o If the value of FRONT is 0, and the value of REAR is MAX -1, at that time the circular queue is considered to be full.

o If the value of REAR! is MAX - 1, then the REAR will be increased and the value will be entered.

o If the value of the FRONT is not equal to 0 and the value of the REAR is equal to MAX - 1, then it can be concluded that the queue is not full. To insert the new element, assign the value of the REAR as zero.

• Deque: The data structure that permits the addition or removal of components at both ends is called a deque, which is also called a double-ended queue. The data structure is sometimes referred to as a head-tail linked list due to its ability to assemble or disassemble by removing members from the head or tail end.

Nevertheless, it is not possible to insert or remove elements from the middle. Two types of circular memory structures are circular arrays and circular doubly linked lists. They are used to implement a deque. A deque is a data structure that utilizes two pointers, LEFT and RIGHT, to indicate the positions at each end of the deque. The items in a deque span from the leftmost end to the rightmost end, and due to their circular nature, dequeue[N–1] is immediately followed by dequeue [0].

There exist two variations of a double-ended queue. The items encompass the following:

o Restricted input deque: In this deque, insertions are limited to one end, while deletions can occur from either end.

o Output-restricted deque: In this deque, deletions are limited to one of the ends, while insertions can be performed on both ends.

• Priority queue: In a priority queue, each element has a specific position in the hierarchy. The element's priority will define the sequence in which the elements will be processed. The fundamental principles governing the processing of components in a priority queue are as follows:

o Components having a higher priority are processed prior to those with a lower priority.

o Elements of equal priority are handled in the order they were received, following a first-come-first-served approach.

A priority queue can be conceptualized as an altered queue where the element with the highest priority is retrieved first when an element needs to be removed from the queue. The element's priority can be determined by considering multiple aspects. Priority queues are extensively utilized in operating systems to prioritize the execution of the most important processes. The process's priority can be determined by the amount of CPU time it needs to complete execution. In this scenario, with three processes, the second process, which requires 4 nanoseconds to complete, will be assigned the greatest priority and will therefore be run first. Nevertheless, CPU time is merely one of several factors that determine the priority. Another factor to consider is the prioritization of one process over another. If we need to simultaneously execute two processes, where one process involves online order booking and the other involves printing stock details, it is evident that the online booking process takes precedence and should be completed first.

• Multiple queue: When utilizing an array to create a queue, it is necessary to have prior knowledge of the array's size. If the authorized space for the queue is reduced, it will result in frequent occurrences of overflow circumstances. In order to address this issue, it will be necessary to make modifications to the code in order to allocate additional space for the array.

If we give a substantial amount of space for the queue, it will lead to excessive memory wastage. Hence, there exists a compromise between the occurrence rate of overflows and the allocated space.

An optimal approach to address this issue is to implement multiple queues or incorporate various queues into a single array of ample capacity. A collection of elements of the data structure Queue[n] is utilized to denote two separate queues, namely Queue A and Queue B. The choice of n ensures that the total size of both queues will never surpass n. When working with these queues, it is crucial to observe that queue A will expand from left to right, whereas queue B will expand from right to left simultaneously.

Pointers

A pointer, in the context of computer science and data structures, is a variable that saves the location in memory of another variable. Pointers are extensively utilized in programming languages such as C and C++, serving as a potent tool for manipulating memory and data structures. Comprehending pointers is essential for optimizing memory allocation and handling of intricate data structures.

Some key concepts related to pointers in data structures are mentioned as follows:

• Memory address: A pointer stores the memory location of a variable or an object. The memory address corresponds to the specific region in the computer's memory where the data is stored.

• Declaration and initialization: In order to declare a pointer, one must first identify the data type they are pointing to, followed by an asterisk (*). For instance, the declaration int *ptr; signifies the creation of a pointer that points to an integer. Prior to utilization, it is imperative to initialize pointers in order to prevent them from referencing an indeterminate memory location.

• Pointer arithmetic: Pointers are capable of being incremented or decremented, and arithmetic operations can be executed on them. This is especially advantageous when dealing with arrays and traversing memory.

• Dereferencing: Dereferencing a pointer involves retrieving the value stored at the specific memory address to which it is pointing. This is accomplished by using the dereference operator (*). For instance, the expression int x = *ptr; retrieves the value stored at the memory address indicated by ptr and assigns it to the variable x.

• Dynamic memory allocation: Pointers are frequently employed for dynamic memory allocation, a process in which memory is allocated or deallocated while the program is running. The C programming language utilizes functions such as malloc() and free() to manage dynamic memory.

• Pointers and data structures: Pointers are essential for constructing diverse data structures, such as linked list, trees, and graphs. They facilitate efficient manipulation and traversal of these structures.

• Pointer to pointer (double pointer): Pointers can reference other pointers as well. This is particularly advantageous in situations where there is a need to alter the value of a pointer itself, such as via function parameters or when allocating dynamic memory.

• Null pointers: Pointers can have a special value called NULL (or nullptr in C++), which indicates that they do not indicate a legitimate location in memory.

Pointers offer a degree of indirection, enabling greater flexibility and efficiency in memory and data structure management. However, they also require careful handling to stay away from typical problems like memory leaks, dangling pointers, and segmentation faults.

Program 2.8: Dynamic memory allocation using single pointer

#include <stdio.h>

#include <stdlib.h>

int main() {

// Declare a single pointer to an integer

int *ptr;

// Allocate memory for a single integer dynamically

ptr = (int *)malloc(sizeof(int));

// Check if memory allocation was successful

if (ptr == NULL) {

printf("Memory allocation failed. Exiting program.\n");

return 1; // Exit with an error code

}

// Input a value into the allocated memory

printf("Enter an integer: ");

scanf("%d", ptr);

// Display the value using the pointer

printf("Value entered: %d\n", *ptr);

// Deallocate the allocated memory

free(ptr);

return 0; // Exit program successfully

}

This program does the following:

• It declares a single pointer to an integer (int *ptr).

• It allocates memory for an integer using malloc.

• It inputs an integer from the user and stores it in the allocated memory.

• It displays the entered value by using the pointer.

• It deallocates the allocated memory using free.

Remember that using malloc and free requires careful handling to avoid memory leaks. Additionally, always check if memory allocation was successful before using the allocated memory, as demonstrated in the program.

Program 2.9: Double pointer to dynamically allocate a 2D array

#include <stdio.h>

#include <stdlib.h>

int main() {

int rows, cols;

printf("Enter the number of rows: ");

scanf("%d", &rows);

printf("Enter the number of columns: ");

scanf("%d", &cols);

int **matrix;

matrix = (int **)malloc(rows * sizeof(int *));

if (matrix == NULL) {

printf("Memory allocation failed. Exiting program.\n");

return 1; // Exit with an error code

}

for (int i = 0; i < rows; i++) {

matrix[i] = (int *)malloc(cols * sizeof(int));

if (matrix[i] == NULL) {

printf("Memory allocation failed. Exiting program.\n");

for (int j = 0; j < i; j++) {

free(matrix[j]);

}

free(matrix);

return 1; // Exit with an error code

}

}

printf("Enter elements of the matrix:\n");

for (int i = 0; i < rows; i++) {

for (int j = 0; j < cols; j++) {

printf("Element at [%d][%d]: ", i, j);

scanf("%d", &matrix[i][j]);

}

}

printf("\nMatrix entered:\n");

for (int i = 0; i < rows; i++) {

for (int j = 0; j < cols; j++) {

printf("%d\t", matrix[i][j]);

}

printf("\n");

}

for (int i = 0; i < rows; i++) {

free(matrix[i]);

}

free(matrix);

return 0; // Exit program successfully

}

This program does the following:

• Takes user input for the number of rows and columns of a 2D array.

• Allocates memory for the 2D array using a double pointer.

• Inputs values into the 2D array.

• Displays the entered 2D array.

• Deallocates the allocated memory.

Make sure to handle memory deallocation properly, especially when working with double pointers, to avoid memory leaks. The program includes error handling for failed memory allocation to ensure proper cleanup before exiting.

Linked lists

A linked list is a data structure consisting of nodes, where each node is connected to the next node through links, creating a linear representation. So, a linked list is a linear data structure that consists of a collection of data pieces. These bits of data are referred to as nodes. It is a data structure that can be utilized to construct additional data structures. Therefore, it serves as a fundamental component for implementing data structures such as stacks, queues, and their variations. It can be conceptualized as a series of nodes, resembling a train, where each node holds one or more data fields and a reference to the next node. The following figure represents the linked list structure:

Figure 2.7: Linked list representation

A linked list consists of nodes, each containing an integer and a pointer to the next node. The left part of the node holds the data, which can be a simple data type, an array, or a structure. The right part of the node holds a pointer to the next node in the sequence. The last node does not have a connected next node and is indicated by a special value called NULL, represented as X. In programming, NULL is typically defined as -1. Therefore, a NULL pointer signifies the end of the list. Since each node in a linked list contains a pointer to another node of the same type, it is also referred to as a self-referential data type.

We can implement a linked list by using the following code:

struct node

{

int data;

struct node *next;

};

Singly linked list

A singly linked list (SLL) is a basic form of a linked list where each node holds data and a pointer to the next node of the same data type. The pointer in each node stores the address of the next node in the sequence, enabling the traversal of data in only one direction.

Traversing a linked list

Let us look at the algorithm.

Algorithm 2.11: Traverse singly linked list

1. [Initialize], Set ptr = Start.

2. Repeat steps 3 and 4 while ptr != NULL.

3. Apply process to ptr | Data.

4. Update ptr = ptr | next. [End of Loop]

5. End

For a single linked list, the time complexity of traversal is O(n), where n is the number of nodes.

As you move through a uniquely linked list, you typically start from the head (the first node) and move through the list one node at a time until you reach the end (where the final node's next pointer is empty). Since you visit each node once, the time complexity is directly proportional to the number of nodes in the linked list.

In Big O Notation, O(n) represents linear time complexity, meaning the time required for the operation grows linearly with the size of the input (in this case, the number of nodes in the linked list).

Searching an element in a linked list

This operation searches for an element in the given linked list and checks whether the input element exists in the list or not.

Algorithm 2.12: Search an element in a singly linked list

1. [Initialization]. Assign the value of Start to ptr.

2. Repeat Step 3 as long as ptr is not equal to NULL.

3. If VAL is equal to ptr | data. Assign the value of ptr to pos and jump to Step 5. Else, move ptr to ptr | Next. [End of If] and [End of the Loop].

4. Assign NULL to pos.

5. Exit

In a singly linked list, searching for an element involves traversing the list from the beginning until you find the desired element or reach the end. If the list contains n elements, then this operation will take O(n) time to complete. This is because, in the worst case, you may need to visit each node in the list once to find the target element.

Insertion in singly linked list

Here, we will examine the process of adding a new node to an existing linked list. We will analyze four examples to determine the insertion process:

• First scenario: The new node is added at the very beginning.

• Second scenario: The new node is appended to the end in this case.

• Third scenario: A specific node is placed before the new one.

• Fourth scenario: The new node is appended to an existing one.

The first scenario shows the insertion at the beginning of the list.

Algorithm 2.13: Insert at the beginning of SLL

1. If Avail is NULL. Write OVERFLOW message. Go to Step 7 and [End of If].

2. Set NEW_NODE = Avail

3. Set Avail to Avail | NEXT

4. Set DATA to VAL

5. Set NEW_NODE | NEXT to START

6. Set START to NEW_NODE

7. Exit

During Step 1, our initial task is to verify the availability of memory for the new node. If the available memory has been completely used, an OVERFLOW notice is displayed. Alternatively, if there is an unoccupied memory cell, we reserve space for the new node. Assign the value VAL to the DATA portion of the node and assign the next part to the address location of the first node in the list, which is held in the variable START. Since the new node is added to the list's starting point, it will be designated as the START node. A new variable called START will now hold the address of the NEW_NODE. Remember these two procedures:

• Assign the value of Avail to the variable NEW_NODE.

• Assign the value of Avail to the next node in the linked list.

These steps reserve memory for the newly created node. In the C programming language, there exist functions such as malloc(), alloc, and calloc() that handle the task of memory allocation on behalf of the user.

The second scenario shows the insertion at the end.

Algorithm 2.14: Insert at the end in SLL

1. If AVAIL is NULL. Write OVERFLOW message. Go to Step 7 and [END of If statement].

2. Set NEW_NODE to AVAIL

3. Set AVAIL to AVAIL | NEXT

4. Set NEW_NODE | DATA to VAL

5. Set NEW_NODE | NEXT to NULL

6. Set PTR to START

7. Repeat the Step 8 While variable PTR | NEXT is != NULL.

8. Set PTR to PTR | NEXT and [END of while Loop].

9. Set PTR | NEXT to NEW_NODE

10. EXIT

The third scenario shows the insertion after a given node.

Algorithm 2.15: Insert after a given node in SLL

1. If AVAIL is NULL. Write OVERFLOW message and jump to the Step 12.

2. Set NEW_NODE to AVAIL

3. Set AVAIL to AVAIL | NEXT

4. Set NEW_NODE | DATA to VAL

5. Set PTR to START

6. Set PREPTR to PTR

7. Repeat Step 8 and 9 While PREPTR | DATA is != NUM

8. Set PREPTR to PTR

9. Set PTR to PTR | NEXT. [END of the while Loop] and [END of the If statement].

10. Update PREPTR | NEXT to NEW_NODE

11. Set NEW_NODE | NEXT to PTR

12. Exit

The fourth scenario shows the insertion before a given node.

Algorithm 2.16: Insert a node before a given node in SLL

1. If AVAIL is NULL. Write OVERFLOW message and jump to Step 12. Then, [END of If statement]

2. Set NEW_NODE to AVAIL

3. Set AVAIL to AVAIL | NEXT

4. Set NEW_NODE | DATA to VAL

5. Set PTR to START

6. Set PREPTR to PTR

7. Repeat Steps 8 and 9, while PTR | DATA is != NUM.

8. Set PREPTR to PTR

9. Set PTR to PTR | NEXT. [END of the while Loop]

10. Update PREPTR | NEXT to NEW_NODE

11. Set NEW_NODE | NEXT to PTR

12. Exit

So, the time complexity for insertion in a singly linked list is generally O(1) for insertion at the beginning and O(n) for insertion at the end or a specific position in the middle.

Deletion in singly linked list

Here, we will examine the process of removing a node from an existing linked list. We will analyze three examples to determine the deletion process:

• Situation 1: Remove an initial node.

• Situation 2: Remove the last node.

• Situation 3: Remove a node after a specified node.

The first case shows the deletion at the beginning of the list.

Algorithm 2.17: Delete the first node in SLL

1. If START is NULL

Write “UNDERFLOW” message

Goto Step 5

[END of the If statement]

2. Set PTR to START

3. Set START to START | NEXT

4. FREE the PTR pointer

5. Exit

Initially, we verify the presence or absence of the connected list. If the value of START is NULL, it indicates that there are no nodes in the list and control is passed to the final step of the algorithm. Nevertheless, if the linked list contains nodes, we employ a pointer variable PTR that is assigned to reference the initial node in the list.

In order to accomplish this, we begin by initializing PTR with START, which holds the memory address of the first node in the list. During Step 3, the START pointer is updated to refer to the next node in the sequence. Then, the memory location engaged by the node pointed to by the PTR pointer (which is initially the first node of the list) is deallocated and returned to the free pool.

The second case shows the deletion at the end of the list.

Algorithm 2.18: Delete the last node in SLL

1. If START is NULL then

Write “UNDERFLOW” message

Goto Step 8

[END of If statement]

2. Set PTR to START

3. Repeat Step 4 and 5 While PTR| NEXT is not equal to NULL

4. Set PREPTR to PTR

5. Set PTR to PTR | NEXT

[END of the while Loop]

6. Set PREPTR | NEXT to NULL

7. FREE PTR pointer

8. Exit

The third case shows deletion after a given node in the list.

Algorithm 2.19: Delete a node after a node in SLL

1. If START is equals to NULL

Write “UNDERFLOW” message

Goto the Step 10

[END of If statement]

2. Set PTR to START

3. Set PREPTR to PTR

4. Repeat Step 5 and 6 While PREPTR | DATA is not equal to NUM

5. Set PREPTR to PTR

6. Set PTR to PTR | NEXT

[END of the while Loop]

7. Set TEMP to PTR

8. Set PREPTR | NEXT to PTR | NEXT

9. FREE PTR pointer

10. Exit

So, the deletion at the beginning of a singly linked list involves updating the head to the next node. This operation holds constant time O(1), irrespective of the size of the list.

In a single linked list, removing the last node requires traversing the entire list to reach the second-to-last node. This makes the time complexity linear and proportional to the number of elements in the list. The time complexity will be O(n). Deleting a node from a specific position in the middle of a singly linked list requires traversing the list to find the node to be deleted and updating the pointers of the adjacent nodes. The time complexity is again linear, dependent on the size of the list, i.e., O(n).

Two way or doubly linked list

A doubly linked list (DLL), also termed a 2-way linked list, is a more intricate variant of a linked list that contains a pointer to both the next and prior nodes in the sequence. Thus, it comprises three components: data, a pointer variable to the next node, and another pointer variable to the prior node.

By using the C language structure, a doubly linked list can be written as:

struct node1

{

struct node1 *previous;

int Data;

struct node1 *Next;

};

The initial node's PREV field and the final node's NEXT field will both be assigned the value NULL. The PREV field stores the address of the previous node, allowing for backward traversal of the list.

Therefore, it is evident that a doubly linked list requires a greater amount of memory per node and more costly fundamental operations. On the other hand, a doubly linked list offers the convenience of manipulating list elements by keeping references to nodes in both the forward and backward directions. Figure 2.8 shows the representation of the two-way linked list:

Figure 2.8: Two-way linked list representation

The primary benefit of utilizing a doubly linked list is that it enhances search efficiency by a factor of two. The doubly linked list can be used to denote other data structures like trees and graphs.

Insertion in two-way linked list

Adding a new node to an existing doubly linked list is demonstrated in this section. The process of inserting is going to be examined in four separate cases:

• Instance 1: The initial position of the new node is placed.

• Instance 2: The new node is appended to the end in this case.

• Instance 3: A specific node is placed before the new one.

• Instance 4: A specific node is preceded by the new node.

Algorithm 2.20: Insertion at the beginning of the DLL

1. If AVAIL is equal to NULL

Write “OVERFLOW” message

Goto the Step 9

[END of the If statement]

2. Set variable NEW_NODE to AVAIL

3. Set AVAIL to AVAIL | NEXT

4. Set NEW_NODE | DATA to Value

5. Set NEW_NODE | PREV to NULL

6. Set NEW_NODE | NEXT to START

7. Set START | PREV to NEW_NODE

8. Set START to NEW_NODE

9. Exit

Algorithm 2.20: Add a new node at the start of a doubly linked list

During Step 1, our initial task is to verify the availability of memory for the new node. If the available memory has been completely used, then an OVERFLOW notification is displayed.

Alternatively, if there is an unoccupied memory cell, we assign space for the new node. Assign the DATA part of the node with the provided VALUE, then initialize the NEXT part with the memory location of the first node in the list, which is saved in the START variable. Now, as the new node is inserted at the beginning of the list, it will be designated as the START node. Consequently, the address of the NEW_NODE will now be stored in the START pointer variable.

Instance two demonstrates Algorithm 2.21, it shows the insertion at the end in DLL:

1. If AVAIL is equal to NULL

Write “OVERFLOW” message

Goto the Step 11

[END of the If statement]

2. Set NEW_NODE to AVAIL

3. Set AVAIL to AVAIL | NEXT

4. Set NEW_NODE | DATA to VALUE

5. Set NEW_NODE | NEXT to NULL

6. Set PTR to START

7. Repeat Step 8 while PTR | NEXT is not equal to NULL

8. Set PTR to PTR | NEXT

9. [END of the while Loop]

10. Set PTR | NEXT to NEW_NODE

11. Set NEW_NODE | PREV to PTR

12. Exit

Algorithm 2.22: Insert after a given node in DLL

1. If AVAIL is NULL

Write “OVERFLOW” message

Goto Step 12

2. Set NEW_NODE to AVAIL

3. Set AVAIL to AVAIL | NEXT

4. Set NEW_NODE | DATA to VALUE

5. Set PTR to START

6. Repeat Step 7 While PTR | DATA is not equal to NUM

7. Set PTR to PTR | NEXT

8. [END of the while Loop]

9. [END of the If statement]

10. Update NEW_NODE | NEXT to PTR | NEXT

11. Update NEW_NODE | PREV to PTR

12. Update PTR | NEXT to NEW_NODE

13. Update PTR | NEXT | PREV to NEW_NODE

14. Exit

Insertion at the beginning or end of a doubly linked list has a O(1) constant time complexity.

In the worst-case scenario, inserting data into a given location in the list takes O(n) time, where n is the total number of nodes in the list.

Deletion in doubly linked list

In this scenario, we will check how a node is removed from an already existing two-way linked list. We will take three situations and then find the deletion process in each case:

• Case 1: Delete a node at the beginning.

• Case 2: Delete a node at the end.

• Case 3: Delete a node after a given node.

Algorithm 2.23: Deletion of the first node in DLL

1. If START is equal to NULL

Write “UNDERFLOW” message

Goto Step 6

[END of If statement]

2. Set PTR pointer to START

3. Set START to START | NEXT

4. Set START | PREV to NULL

5. FREE PTR pointer

6. Exit

Algorithm 2.24: Deletion at the end of the list

1. If START is equal to NULL

Write “UNDERFLOW” message

Goto Step 7

[END of If statement]

2. Set PTR to START

3. Repeat Step 4 While PTR | NEXT is not equal to NULL

4. Set PTR to PTR - > NEXT

[END of the while Loop]

5. Set PTR | PREV | NEXT to NULL

6. FREE PTR pointer

7. Exit

Algorithm 2.25: Delete a node after a node in DLL

1. If START is equal to NULL

Write “UNDERFLOW” message

Goto Step 9

[END of If statement]

2. Initialize PTR to START

Repeat Step 4 While PTR | DATA is not equal to NUM

Set PTR to PTR | NEXT

[END of the while Loop]

3. Set TEMP to PTR - > NEXT

4. Set PTR | NEXT to TEMP | NEXT

5. Set TEMP | NEXT | PREV to PTR

6. Free TEMP

7. Exit

Deletion at the beginning or end of a doubly linked list has a constant time complexity of O(1).

When deleting a node at a specific position in a doubly linked list, the time complexity is O(n/2) on average, where n is the number of nodes in the list. In the worst case, it is still O(n). This is because, on average, you need to traverse half of the list to find the node to delete. In the worst case, you may need to traverse the entire list.

Circular doubly linked list

A circular doubly linked list is a discrepancy of a doubly linked list in which the end node of the list is linked to the first node, forming a circle or loop. In a standard doubly linked list, the end node points to null to signify the end of the list. A circular doubly linked list, on the other hand, is always in a loop since the last node's pointer always points back to the first. Figure 2.9 shows the representation of a circular doubly linked list:

Figure 2.9: Circular doubly linked list representation

Due to its structure consisting of three parts, a circular DLL requires additional space per node and incurs higher costs for fundamental operations. On the other hand, a circular doubly linked list offers the convenience of manipulating list elements by keeping references to nodes in both the forward and backward directions. A significant advantage of utilizing a circular DLL is that it enhances the efficiency of search operations by a factor of two.

Insertion in circular doubly linked list

Adding a new node to an existing circular DLL is demonstrated in this section. We shall split the instances into two parts:

• Instance 1: The initial position of the new node is placed.

• Instance 2: The new node is appended to the end in this case.

Algorithm 2.26: Insertion at the beginning of the circular DLL

1. If AVAIL is equal to NULL

Write “OVERFLOW” message

Goto Step 13

[END of the If statement]

2. Set NEW_NODE to AVAIL

3. Set AVAIL to AVAIL - > NEXT

4. Set NEW_NODE | DATA to VALUE

5. Set PTR pointer to START

6. Repeat the Step 7 while PTR | NEXT is not equals to START

7. Set PTR to PTR | NEXT

8. [END of the while loop]

9. Set PTR | NEXT to NEW_NODE

10. Set NEW_NODE | PREV to PTR

11. Set NEW_NODE | NEXT to START

12. Set START | PREV to NEW_NODE

13. Set START to NEW_NODE

14. then Exit

Case 2 insertion at the end.

Algorithm 2.27: Insert at the end in CDLL

Step 1: If AVAIL is equals to NULL

Write “OVERFLOW” message

Jump to Step 12

[END of If statement]

Step 2: Set NEW_NODE to AVAIL

Step 3: Set AVAIL to AVAIL | NEXT

Step 4: Set NEW_NODE | DATA to VALUE

Step 5: Set NEW_NODE | NEXT to START

Step 6: Set PTR to START

Step 7: Repeat the Step 8 While PTR | NEXT is not equals to START

Step 8: Set PTR to PTR | NEXT

[END of the while loop]

Step 9: Set PTR | NEXT to NEW_NODE

Step 10.  Set NEW_NODE | PREV to PTR

Step 11.  Set START | PREV to NEW_NODE

Step 12: Exit

The constant-time complexity of inserting at the start or end of a circular doubly linked list is O(1). At worst, inserting data at a given point in the list takes O(n/2) time, where n is the total number of nodes.

Deletion in circular doubly linked list

Here, we will examine the process of removing a node from an existing circular doubly linked list. We shall examine two instances.

In Case 1, the new node is deleted at the beginning of the list, while in case 2, the new node is deleted at the end.

Let us look at Case 1, which involves the deletion at the beginning of the list.

Algorithm 2.28: Delete first node in CDLL

Step 1: If START is equals to NULL

Write “UNDERFLOW” message

Then Goto Step 8

[END of the If statement]

Step 2: Initialize PTR to START

Step 3: Repeat the Step 4 while PTR | NEXT is != START

Step 4: Move PTR to PTR | NEXT

[END of the while Loop]

Step 5: Set PTR | NEXT to START | NEXT

Step 6: Update START | NEXT | PREV to PTR

Step 7: Free the START pointer

Step 8: Finally set START to PTR | NEXT

Case 2 involves the deletion at the end of the list

Algorithm 2.29: Delete the last node in CDLL

Step 1: If START is equals to NULL

Write “UNDERFLOW” message

Then Goto Step 8

[END of the If statement]

Step 2: Initialize PTR pointer to START

Step 3: Repeat the Step 4 While PTR | NEXT is != START

Step 4: Set PTR to PTR - > NEXT

[END of the while Loop]

Step 5: Update PTR | PREV | NEXT to START

Step 6: Set START | PREV to PTR | PREV

Step 7: Free PTR pointer

Step 8: Then Exit

The operation deletion in a circular DLL is generally O(1) if you have a reference to the node and O(n) if you need to search for the node.

Hashing

Hashing is a method employed in the examination of algorithms and data structures, specifically in the field of searching and retrieving information from extensive datasets. The main objective of hashing is to efficiently map data of any size to fixed-size values, usually integers, while preventing collisions, which occur when two distinct bits of data are mapped to the same spot.

The following is a fundamental explanation of how hashing operates within the framework of algorithm analysis:

• Hash function: A hash function is a mathematical function that takes an input, sometimes known as a key, and generates a fixed-size output, typically referred to as a hash code. The hash code functions as the index or location inside the data structure where the appropriate data should be stored or retrieved.

o The general idea of using the key to determine the address of records is an excellent idea. But it must be modified to prevent the wastage of space.

o The modification takes the form of a function H from the set K of keys into the set L of memory addresses.

H: K  L

o Two principal criteria used in selecting a hash function H are:

 H should be very easy and quick to compute.

 H should be uniformly distributing the hash address throughout the set L so that the number of collisions is minimized.

• Hash table: It is a data structure that uses a hash function to map keys to specific places in an array. We utilize a hash function that retrieves the final two digits of the key. Consequently, we assign the keys to specific positions in an array, known as array locations or array indices. By utilising a hash function, which creates an address from the key and determines the index of the array where the value is kept, a value stored in a hash table can be found in constant time, denoted as O(1).

Different hash functions:

o The division method: This is the most basic approach to hashing an integer x. This approach involves dividing the value of x by M and subsequently utilising the resulting remainder. The hash function in this scenario can be expressed as follows:

H(x) = x mod M

The division method is highly efficient for any value of M, as it only takes a single division operation, resulting in quick performance. Nevertheless, it is imperative to use caution while choosing an appropriate value for M.

For instance, let us assume that M is an even number. In this case, H(x) will also be even if x is even, whereas H(x) will be odd if x is odd. If all potential keys have an equal probability of occurring, then this situation does not pose an issue. However, if the probability of even keys is higher than that of odd keys, the division algorithm will not distribute the hashed values evenly.

It is often advisable to select M as a prime integer because doing so enhances the probability of achieving a uniform mapping of keys to a range of output values. M should also be sufficiently distant from the precise powers of two. Given H(x)=x mod 2k function, the binary form of x will only have its lowest k bits extracted.

Example 2.4: Find the matching hash values for keys 5462 and 1234.

Solution: Consider the value of M = 97, hash values can be evaluated as:

H(5642) = 5642 % 97 = 16

H(1234) = 1234 % 97 = 70

• Multiplication method: The procedure for the multiplication method consists of the following steps:

1. Select a constant A within the range of 0 to 1,

2. Compute the product of the key k and A.

3. Obtain the decimal portion of the product of k and A.

4. Compute the product of the outcome from Step 3 and the magnitude of the hash table (m).

Therefore, the hash function can be expressed like: H(k) = [M (KA mod 1)]

Where (KA mod 1) represents the fractional part of KA, and the total number of indices in the given hash table is denoted by M.

Example 2.5: Find the correct location in a 1000-by-key hash table for the key 12345.

Solution: We will consider A = 0.619034, M= 1000, and K = 12345

H(12345) = [ 1000 ( 12345 X 0.619034 mod 1 ) ]

H(12345) = [ 1000 ( 7641.97473 mod 1 ) ]

H(12345) = [ 1000 ( 0.97473 ) ]

H(12345) = [ 974.73 ]

H(12345) = 974

• Mid-square method: This method is an effective hashing function that operates in two distinct steps:

1. Calculate the square of the key value. That is, determine the value of k2.

2. Retrieve the middle r digits from the outcome received in Step 1.

The algorithm is effective because the majority or all of the digits in the key value have an impact on the outcome. This occurs because each individual digit in the original key value collectively contributes to generating the middle digits of the squared value. Consequently, the outcome is not influenced by the prevalence of either the lowest or highest digit in the initial key value.

In the mid-square approach, the selection of r digits must be consistent across all keys. Thus, the hash function can be expressed as follows: h(k) = s, where s is derived by picking r digits from the square of k.

Example 2.6: Use the mid-square method to determine the hash value of keys 5642 and 1234. The hash table occupies one hundred memory spaces.

Solution: Be aware that the hash table uses 100 memory regions with indices ranging from zero to 99. This implies that r= 2 is the minimum number of digits required to map the key to a place in the hash table.

When K = 5642, K2 = 31832164, H (5642) = 21

When K = 1234, K2 = 1522756, H (1234) = 27

Keeping with the right-hand side of the equation, the third and fourth digits are selected.

• Folding method: The folding technique operates through the sequential execution of two distinct steps:

1. Partition the key value into multiple segments. That is, partition k into k1, k2, ..., kn, where the exception of the final partition, every partition has an identical number of digits, which may have fewer digits than the others.

2. Incorporate the different components. i.e., acquire the total of K1, K2, ..., Kn. The hash-value is generated by disregarding the final carry, if one exists.

It should be noted that the number of digits in each section of the key will differ based on the size of the hash table. For instance, if the hash table has a size of 1000, it means that there are 1000 slots or positions available in the hash table. To address these 1000 places, we require at least three digits; consequently, every section of the key, with the exception of the last part, needs to have three digits.

Example 2.7: Utilizing the folding approach, determine the hash value for keys 5678, 322, and 34567 in a hash table with 100 places.

Solution: Since there are 100 memory locations to address, we will break the key into parts where each part (except the last) will contain two digits.

Key 5679 Parts are 56 and 79 Sum 135 Hash value 35 (ignore the last carry)

Key 322 Parts are 32 and 2 Sum 34 Hash value 34

Key 35568 Parts are 35, 56 and 8 Sum 99 Hash value 99

• Chaining: In the hash table, each bucket holds a linked list (or another data structure) to store numerous items that have the same hash code. Thus, different keys that generate the identical hash value can reside together within a single bucket.

• Open addressing: In the event of a collision, this technique entails locating another position inside the hash table to accommodate the conflicting item. This may entail looking for the subsequent accessible position using techniques like linear probing or quadratic probing.

Hashing is frequently employed in algorithms that necessitate efficient data retrievals, such as hash-based search structures, caches, and diverse database indexing strategies. When evaluating methods that utilize hashing, it is crucial to take into account the effectiveness of the hash function, the dispersion of hash codes, and the approach to resolving collisions. A well-constructed hash function can result in a more evenly distributed allocation of data, hence decreasing the probability of collisions and enhancing overall efficiency.

Trees

Trees are hierarchical non-linear data structures that are made out of nodes linked together by edges. Every tree starts at the root node and branches outward from there with one or more child nodes at each level, except for the root node, which has zero or more children.

There are various types of trees used in algorithm design and analysis, and they serve different purposes. Some common types of trees include:

• Binary tree: In a binary tree, the number of children per node is strictly limited to two: the left child and the right child.

• Binary search tree (BST): The BST is a kind of binary tree where the left subtree of a node consists exclusively of nodes with keys that are less than the key of the node, and the right subtree consists exclusively of nodes with keys that are higher than the key of the node. The ordering aspect of this data structure enhances its efficiency for searching, insertion, and deletion operations.

• AVL tree: One kind of binary search tree is the AVL tree that automatically maintains balance. The balance of the tree is maintained by ensuring that the difference in heights between the two child subtrees of each node is no more than one.

• Heap: A heap is a data structure that is built on a tree and fulfils the heap attribute. In a maximum heap, the value of each node is always larger than or equal to the values of its child nodes. A min-heap is a data structure where the value of each node is smaller or equal to the values of its child nodes.

Trees are utilised in algorithms for several objectives, including optimizing data organization, facilitating element search, and illustrating hierarchical connections. Algorithmic design and analysis frequently entail the utilisation or construction of tree structures to enhance specific operations or represent connections between elements in a problem.

The application of trees is mentioned as follows:

• Trees serve as a means of storing both basic and intricate information. In this context, simple refers to an integer or character value, while complex refers to a structure or record.

• Trees are commonly employed for the implementation of various data structures such as hash tables, sets, and maps.

• Red-black tree, one kind of self-balancing tree, is utilized in kernel scheduling for preempting massively multiprocessor computer operating systems.

• B-trees are commonly employed for storing tree structures on disc, serving as an alternative form of trees. They are utilised for the purpose of cataloguing a substantial quantity of records.

• B-trees are utilised as supplementary indexes in databases, enabling efficient select operations to satisfy certain range conditions.

• There is another usage for trees in file system directories, database design etc.

Binary search tree

The BST, sometimes termed as an, is an ordered binary tree type of binary tree where nodes are organized in a specific order. Within a binary search tree, every node in the left subtree has a value lower than the root node. Similarly, all the nodes in the right subtree possess a value that is either equivalent to or larger than the value of the root node. Each subtree in the tree is subject to the same rule.

The search operation typically takes O(log2n) time to complete because we remove half of the subtree at each stage.

We do not need to traverse the entire tree for searching an element due to its efficiency in searching, widely used in dictionary problems. We can get all values in sorted order by just doing the inorder traversal of BST.

Example 2.8: Make a binary search tree with these pieces of information:

45, 39, 56, 12, 34, 78, 32, 10, 89, 54, 67, 81

Solution: The solution is given in the following figure:

Figure 2.10: Finding the final BST

So, in the preceding over tree structure, we first pick up value 45 and make it the root node. Now, the next value, 39 is compared with the root value, 39 < 45, so place 39 as the left child of the root. The next value is 56, 56 > 45, place 56 as the right child of the root. The next value is 12, which is compared with the root node first and then compared with the left child 39. Since 12 < 39, put this to the left of 39. Now another value is 34, which will be compared in the left subtree, placed as the right child of the 12. The next value is 78, 78 > 45, so we will process it in the right subtree. It is also greater than 56, so it is a right child to 56. Next is 32, compare it in the left subtree, and next place it as the left child of 34. Now another value is 10, which is compared with 45, then compared with 39, then compared with 12. So, 10 < 12, thus, make it the left child of 12. The next value is 89, which will be compared in the right subtree and placed as the right child of 78. The next three values are 54, 67, and 81, all these values are greater than the root node, so we will place these values in the right subtree one by one at their correct positions.

B-tree

B-tree is a data structure that automatically adjusts its balance and keeps data sorted. It enables efficient operations such as searching, accessing data in order, inserting new data, and deleting data, all of which can be done in logarithmic time. It is an abstract representation of the binary search tree. B-trees are highly suitable for storage systems with slow, cumbersome data access, like hard drives, flash memory, and CD-ROMs. The value of 't' is contingent upon the size of the disc block.

A B-tree node can accommodate many keys, enabling the tree to have a higher branching factor and hence a shorter height. The limited height of this structure reduces the amount of disc input/output, hence enhancing the speed of search and insertion operations. A B-trees achieves balance by enforcing a minimum number of keys in each node, hence assuring constant balance throughout the tree. The equilibrium ensures that the Time Complexity for operations like searching, insertion, and deletion is consistently O(log n), irrespective of the starting structure of the tree.

Features of B-tree are mentioned as follows:

• All the leaves are situated at the same level.

• A B-tree is characterised by the parameter 't', which represents the minimal degree. The value of 't' depends upon the size of the disc block.

• Each node, excluding the root, can have a maximum of t-1 keys. A minimum of one key is required for the root.

• Each node, including the root, can have a maximum of the (2t – 1) keys.

• A node's total number of children is equivalent to the number of keys it contains, plus one.

• The keys of a node are arranged in ascending order. The child located b/w 2 keys, K1 and K2, encompasses all keys inside the range defined by K1 and K2.

• The B-tree expands and contracts starting from the root, which is in contrast to the BST. BST trees expand downwards and contracts above.

• Each operation on a balanced BST takes O(log n) time, just like any other balanced BST.

• Node insertion in a B-tree occurs exclusively at leaf nodes.

Example 2.9: Use the numbers 11–18 to construct a three-order B-tree.

Solution: The first element is 11, inserted into a new node, i.e., root node:

Figure 2.11: Add first element to root node

The second element is 12, added to the same leaf node, value 12 can be inserted at an empty position:

Figure 2.12: Add second element to leaf node

The third element, 13, is added to the same leaf, but we do not have free space in the leaf. So, we split the node by transferring the middle value 12 to its parent node. But there is no parent, so we make the middle-value node the parent node:

Figure 2.13: Splitting the node

The fourth value is 14, 14 > 12, so we move to the right of 12. We reach the leaf node having value 13, it has empty space, and we place value 14 there:

Figure 2.14: Adding the fourth element

The fifth value is 15, 15 > 12(root node), it is not a leaf node, we proceed to the leaf node, which is already full. We divide that node by transferring the middle value 14 to its parent node 12. So, there is an empty space in its parent, so 14 is added to the parent and 15 is added to a new leaf node:

Figure 2.15: Add fifth element to new leaf node

The sixth number is 16, i.e., greater than root values 12 and 14, and it is not a leaf node, so we move to the right of 14 and reach to the leaf node (15). It has vacant space, add 16 here:

Figure 2.16: Adding element to leaf node

The seventh number is 17, larger than 12 and 14, and not a leaf node. So, we move right of 14, we reach at the leaf node which is already full. We split the node by transferring the middle value 16 to its ancestor (12 and 14), but the parent is also full. So, again, split by sending the middle value 14 to its parent, but the parent does not exist. So, 14 becomes the new root node:

Figure 2.17: 14 becomes new root node

The eighth value is 18, larger than the root node 14, and it is not a leaf node. So, we proceed to the right subtree, reach at 16 again. As 18 > 16 is also not a leaf node, we move to a leaf node 17. There is empty space, place 18 with 17 at a leaf node position:

Figure 2.18: Final tree

AVL tree

The AVL tree, developed by Adelson, Velsky, and Landis, is a binary search tree that can balance itself. There can be no more than a one-degree difference between a node's two child trees in an AVL tree. The AVL tree is referred to as a height balanced tree because of this characteristic.

The advantage of the AVL tree is that it takes O(log2n) time to execute search, insertion, and deletion operations in both the worst and average-case scenarios.

It keeps the balance factor variable, which is an extra one in an AVL tree structure.

It is possible to determine a node's balance factor by deducting the heights of its right and left subtrees.

bf = Height of Left subtree - Height of Right subtree

bf = hL - hR

The term height balanced describes a binary search tree where each node's balancing factor is either 1, 0, or -1.

It may be necessary to rebalance the tree if AVL tree insertions or deletions upset the balance factor of nodes. By rotating the tree at the essential node, the equilibrium is restored.

Four kinds of rotations are applied, which are left-left (LL), right-right (RR), left-right (LR), and right-left (RL).

Insertion operation of AVL tree consists of the following steps:

1. Insertion of the new node occurs at the leaf node.

2. Nodes that are on a path between the tree's root and the newly inserted leaf are the only ones whose balance factor can be altered by this technique of insertion.

3. Every node along this route has the potential to undergo the following transformations:

a. After insertion, the node becomes balanced, even though it was previously heavy on one side or the other.

b. After insertion, the node becomes unbalanced and favors one side over the other.

c. The new node has been added to the heavy subtree, making it an unbalanced subtree because the node was either left or right-heavy before. A crucial node is one such node.

Figure 2.19 illustrates the balanced AVL tree structure after insertion:

Figure 2.19: Balanced tree

The tree is rebalanced by performing rotations at the critical node.

The four types of rotations are mentioned as follows:

• The new node is inserted into the critical node's left subtree as part of the LL rotation.

• When performing RR rotation, a new node is added to the critical node's right subtree.

• The new node is inserted into the right subtree of the critical node's left subtree during the LR rotation.

• The new node is inserted into the critical node's left subtree of the right subtree via RL rotation.

So, in the AVL tree:

• Search operation: The time complexity is O(log2 n).

The search operation in AVL trees is the same as binary search tree operations. The tree is balanced, so the height is guaranteed to be logarithmic with respect to the number of nodes.

• Insertion operation: The time complexity is O(log2 n).

During insertion, the AVL tree may need to perform rotations to maintain balance. However, since the height of the tree is logarithmic, the overall complexity remains O(log2 n).

• Deletion operation: The time complexity is O(log2 n).

Similar to insertion, deletion in AVL trees may require rotations to restore balance. The height of the tree remains logarithmic, resulting in O(log2 n) time complexity.

• Rotations: Time complexity is O(1) per rotation.

Rotations (single or double) are constant time operations, as they involve rearranging pointers without traversing the entire tree.

Red-black tree

A red-black (RB) tree is a type of BST that automatically maintains balance by assigning each node an additional attribute indicating its color, which may be either red or black.

During the process of adding a new node, the new node is consistently added as a red node. When we insert a new node, if the tree is in violation of the properties of a red-black tree, we do the following activities:

• Change the color: This step involves color changing of the node from red to black or vice versa.

• Rotate: Denote the tree rotation in a particular direction.

Recolouring refers to the alteration of the colour of a node, such that if it is initially red, it is changed to black, and vice versa. It is important to mention that the color of the NULL node is consistently black. In addition, our initial approach is to attempt recolouring. If recolouring proves unsuccessful, we then proceed with rotation. The algorithms primarily consist of two scenarios, which are determined by the color of the uncle node. If the uncle is red in color, we proceed with the process of altering the color. If the uncle node has a black color, we perform rotations and recoloring.

Example 2.10: Construct a red black tree having elements 3, 22, 33, 18 in a blank tree structure.

Solution: The first element is 3, inserted into the tree, i.e., the root node:

Figure 2.20: RB tree step 1

The second element is 22, inserted right to the root node:

Figure 2.21: RB tree step 2

The third element is 33, which is greater than 3 and 21. So, we will add it in the right sub tree. Here, two nodes have the color red. So, we rotate RR and recolour the balanced tree as follows:

Figure 2.22: RB tree step 3

The fourth element is 18. Since 18 < 22, we move towards the left subtree, but 18 > 3, so we put it as the right child of 3. After adding the new node, we rearrange the colour of the tree:

Figure 2.23: RB tree step 4

The time complexity of various operations in a red-black tree is generally expressed in terms of big O notation. Here are the typical time complexities for red-black tree operations:

• Search: O(log n), the height of an RB tree is guaranteed to be logarithmic, which guarantees efficient search operations.

• Insertion: O(log n), the insertion procedure may need several recoloring and rotations operations, but the overall time complexity remains O(log n) because of the balanced structure of red-black trees.

• Deletion: O(log n), deletion, like insertion, may necessitate rotations and recoloring, but the time complexity remains O(log n) because of the balanced features of the tree.

Heap

A heap is a tree-based data structure in computer science that adheres to the heap property. Heaps are frequently employed for the implementation of priority queues, which are data structures that store elements along with their corresponding priorities and enable efficient retrieval of the element with the highest (or lowest) priority.

There are two main types of heaps, described as follows:

• Max heap: A max heap is a data structure where the value of each node is larger than or equal to the values of its descendants. Consequently, the element with the greatest value is consistently located at the root. In a max heap, each node i, except for the root, has a value that is less than or equal to its parent's value. This is the main characteristic of a max heap.

• Min heap: In a minimum heap, the value of each node is smaller than or equal to the values of its offspring. Consequently, the element with the minimum value is consistently located at the root. A fundamental characteristic of a min heap is that, for each node i that is not the root, the value of i is larger than or equal to the value of its parent.

Heap operations typically include the following:

• Insertion operation: To add a new element to the heap. the time complexity of insertion in a heap is O(log n).

• Deletion: Removing the element with the highest (for max heap) or lowest (for min heap) priority deletion in the worst-case scenario, the time complexity is O(log n).

• Heapify: Ensuring that the heap property is maintained after an insertion or deletion operation. Building a heap from an array of n elements can be done in O(n) time using a bottom-up approach called heapify.

Binomial heap

A group of binomial trees is called a binomial heap. A binomial tree Bk is a tree that is defined recursively in a specific order. A binomial tree B0 consists of a solitary node.

A binomial tree Bk is composed of two binomial trees Bk-1 that are interconnected. The root of one is the leftmost offspring of the root of the other.

Properties of a binomial tree are as follows:

• The height of the tree is k.

• Binomial tree with Bk has 2k nodes.

• There are exactly (k j) nodes at depth i for all the i having range 0 - k.

A binomial heap, H denotes the collection of binomial trees. Several qualities exist, shown as follows:

• Every binomial tree in H is arranged in a heap order. The key value of a node is always greater than or equal to the key value of its parent node.

• There exists a maximum of 1 binomial tree in H, in which the root of the tree has a certain degree.

Figure 2.24: Binomial heap

The binomial heap H comprises binomial trees B0, B2, and B3, which contain nodes 1, 4, and 8, respectively. The total number of nodes, n, is 13. The roots of the binomial trees are connected by a linked list, arranged in ascending order of degree.

Fibonacci heap

A data structure known as a Fibonacci heap offers efficient time complexity for multiple operations, including insert, merge, decrease key, and extract minimum. It outperforms other heap data structures, such as binary heaps or binomial heaps, in terms of amortised time complexity. Michael L. Fredman and Robert E. Tarjan introduced it in 1984.

The effectiveness of the Fibonacci heap is attributed to several inherent properties:

• Amortized time: The amortized time complexity of key operations such as insert, reduce key, and delete is superior to that of other heap structures, rendering it appropriate for specific algorithms and applications.

• Merging mechanism: Fibonacci heap distinguishes itself from other heap structures by employing a slow merging mechanism. Consequently, the two heaps are not instantaneously merged during the merge procedure. Alternatively, a circular doubly linked list is generated to connect the root nodes of the two heaps. This methodical methodology enables the attainment of amortized constant time for merging procedures.

• Potential approach: The evaluation of Fibonacci heap operations is commonly conducted with the potential approach. This approach takes into account not just the current work performed by an operation, but also the possibility of saving future work. This results in more advantageous amortized time complexities.

The reduce key action in a Fibonacci heap can be executed in constant time without the need to consolidate the heap. This is an additional component that contributes to the effectiveness of the data structure.

Fibonacci heaps consist of a collection of trees. They have a loose foundation derived from binomial heaps. In contrast to trees in binomial heaps, in Fibonacci heaps, trees are rooted but not arranged in a specific way. Every node x in Fibonacci heaps possesses a pointer p[x] that points to its parent, as well as a pointer child[x] that denotes to its children. The children of x are interconnected in a circular, doubly linked list referred to as the child list of x. Each kid y in a child list possesses pointers left[y] and right[y] that shows the left and right siblings of y, respectively. If the node y is the lone child, then the values of left[y] and right[y] are both set to y. The placement of siblings in a child list is random. Fibonacci heap is shown in the following figure:

Figure 2.25: Fibonacci heap

In the Fibonacci heap, insertion, decrease key, and merge operation time complexity is amortized O(1), and in the deletion operation, the time complexity is amortized O(log n), where 'n' represents the number of items in the heap.

Conclusion

This chapter provided a comprehensive overview of fundamental data structures, emphasizing their roles in efficiently organizing and managing data. By exploring both linear structures like arrays, stacks, queues, and linked lists, as well as non-linear ones such as trees and heaps, we gained insight into their unique use cases, advantages, and trade-offs. Additionally, concepts like hashing and pointers were discussed to highlight their importance in memory management and fast data access. Through algorithms and examples, we analyzed how the right choice of data structure can significantly impact an application's performance. Understanding these core structures lays the groundwork for mastering more advanced computational techniques.

In the next chapter, Sorting Algorithms, we will study a variety of sorting methods ranging from basic techniques like bubble and insertion sort to more advanced ones such as radix and bucket sort, focusing on their logic, implementation, and efficiency.

Exercise

1. Let us pretend we have a 2-dimensional array Arr1[10][10] with a base address of 2000 and 2 bytes each entry. Presuming the items are stored in column major order, we can now calculate the address of the element Arr1[7][4].

2. Write a program using pointers to interchange the second largest and the second smallest number in the given array.

3. Write a program to implement Tower of Hanoi using recursion and find the complexity of the recursive algorithm and also justify your answer.

4. Design code that creates an array-based queue that accepts insertions at both ends.

5. Write a program to perform followings:

a. Remove the first occurrence of a specific character in a linked list.

b. Remove the last occurrence of a specific character.

c. Remove all instances of a certain character.

6. Write a program to form a linked list of floating-point numbers. Display the sum and mean of these numbers.

7. 10, 3, 15, 22, 6, 45, 65, 23, 78, 34, 5. Create BST and find its inorder, preorder and postorder traversal.

8. Construct the AVL tree (height balanced tree) for the following data: 42, 06, 54, 62, 88, 50, 22, 32, 12, 33. After creating AVL tree structure delete node 50 and 80 from the AVL tree.

9. Create a B-tree for following data:

a. Insert 1, 6, 8, 12, 14, 16, 19, and 21 in the tree.

b. Delete 19 and 21 from the tree.

10. Compare the binomial heap and Fibonacci heap.

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 3Sorting Algorithms

Introduction

In this chapter of the book, we will present sorting algorithms and their concept in the field of computer science. Sorting algorithms serve as essential tools in computer science, facilitating the arrangement of data in particular orders, usually in ascending or descending sequences. Their significance extends to various applications, such as data analysis, database management, and information retrieval systems. Through effective data organization, sorting algorithms expedite search and retrieval operations, streamline data analysis, and bolster the overall performance of computing systems. Comprehending these algorithms is crucial for maximizing the efficiency and efficacy of data handling across various fields of research and practical implementation.

Structure

The chapter covers the following topics:

• Sorting

• Bubble sort

• Selection sort

• Insertion sort

• Shell sort

• Counting sort

• Radix sort

• Bucket sort

Objectives

This chapter aims to arrange a set of data in a specific order, either ascending or descending. The goal is to efficiently restructure elements, reducing the need for extensive comparisons and exchanges, all in an effort to improve the effectiveness of search and retrieval operations, simplify data analysis, and boost the overall performance of systems. These algorithms hold significant importance across diverse disciplines, such as computer science, data analysis, and information management, facilitating the efficient management and manipulation of large data.

Sorting

Sorting is a fundamental operation in computer science and is used to arrange elements in a specific order. The order could be numerical (ascending or descending) or lexicographical (dictionary order). Sorting algorithms play a crucial role in optimizing data retrieval and making information more accessible. There are various sorting algorithms, each with its own set of characteristics and best-use scenarios.

Sorting is essential for organizing and retrieving information efficiently. In many applications, data is unsorted and needs to be arranged in a meaningful way for analysis or presentation. Here are some common scenarios where sorting is crucial:

• Searching: Sorted data allows for more efficient search operations, such as binary search, which relies on the order of elements.

• Data presentation: For better readability and user experience, it is often desirable to present data in a sorted form.

• Algorithmic operations: Many algorithms and data structures, such as binary search trees, rely on sorted data to perform efficiently.

Key concepts

Let us discuss comparison-based vs. non-comparison-based sorting:

• Comparison-based sorting: These algorithms sort elements by comparing them using a comparison operator (<, >, =). Examples include bubble sort, selection sort, merge sort, and quick sort.

• Non-comparison-based sorting: These algorithms exploit specific properties of the data and do not rely solely on element comparisons. Examples include counting sort and radix sort.

Let us discuss stable vs. unstable sorting:

• If a sorting algorithm, after sorting the contents, does not change the sequence of similar content in which they appear, it is called stable sorting. This is shown in the following figure:

Figure 3.1: Stable sorting algorithms

• If a sorting algorithm, after sorting the contents, changes the sequence of similar content in which they appear, it is called unstable sorting. This is shown in the following figure:

Figure 3.2: Unstable sorting algorithms

Refer to the following list for an overview of the general types of sorting:

• In-place sorting: Sorting algorithms may require some extra space for comparison and temporary storage of a few data elements. These algorithms do not require any extra space, and sorting is said to happen in-place or, for example, within the array itself. This is called in-place sorting. Bubble sort is an example of in-place sorting.

• Adaptive sorting: A sorting algorithm is said to be adaptive if it takes advantage of already sorted elements in the list that is to be sorted. That is, while sorting if the source list has some element already sorted, adaptive algorithms will take this into account and will try not to re-order them.

• Time and space complexity: Sorting algorithms are analyzed based on their time and space complexity. Time complexity describes the computational time required, while space complexity refers to the additional memory needed. Some of the popular sorting algorithms are as follows:

o Bubble sort

o Selection sort

o Insertion sort

o Shell sort

o Counting sort

o Radix sort

o Bucket sort

Understanding these concepts and algorithms provides a foundation for choosing the most appropriate sorting algorithm based on the specific requirements of a given task.

Bubble sort

The fundamental principle of the bubble sort algorithm involves iteratively traversing a list, checking adjacent elements, and swapping them if they are not in the correct sequence. This procedure continues until the entire list is arranged in the correct order.

Algorithm 3.1: Algorithm of bubble sort

bubbleSort(arr: list)

n = length(arr)

for i = 0 to n-1 do:

swapped = False

for j = 0 to n-i-1 do:

if arr[j] > arr[j+1] then:

swap(arr[j], arr[j+1])

swapped = True

if swapped = False then:

break

Following is a concise overview highlighting the main principles of the bubble sort algorithm:

• Comparison of adjacent elements:

o Initially, the algorithm compares the initial two elements within the list.

o When the first element surpasses the second, a swap occurs; otherwise, they retain their positions.

o This comparison and potential swap routine continues for each adjacent pair of elements in the list.

• Passes through the list:

o With each iteration through the list, the largest unsorted element gradually moves to its proper place at the list's end, resembling a bubble-up motion.

o Following the initial pass, the largest element invariably settles at the list's conclusion.

o Subsequent passes ensure the placement of the second-largest element, and this pattern continues until all elements find their correct positions.

• Repeated passes: The iteration continues for the remaining unsorted elements until no further swaps are necessary, signaling that the entire list has been sorted.

• Optimization and early exit:

o An optimization for bubble sort involves using a flag to track if any swaps occurred during a pass.

o If no swaps happen during a pass, it means the list is already sorted, allowing the algorithm to terminate early.

In comparison to more advanced sorting techniques like quick sort or merge sort, bubble sort is deemed inefficient for handling large datasets. Nonetheless, its straightforward nature and ease of comprehension make it a popular choice for educational purposes.

Program 3.1: Implementation of bubble sort

#include <iostream>

using namespace std;

// Function to perform bubble sort

void bubbleSort(int arr[], int n) {

for (int i = 0; i < n - 1; ++i) {

// Last i elements are already in place, so no need to check them

for (int j = 0; j < n - i - 1; ++j) {

// Swap if the element found is greater than the next element

if (arr[j] > arr[j + 1]) {

// Swap arr[j] and arr[j+1]

int temp = arr[j];

arr[j] = arr[j + 1];

arr[j + 1] = temp;

}

}

}

}

// Function to print an array

void printArray(int arr[], int size) {

for (int i = 0; i < size; ++i) {

cout << arr[i] << " ";

}

cout << endl;

}

// Example usage

int main() {

int arr[] = {64, 34, 25, 12, 22, 11, 90};

int n = sizeof(arr) / sizeof(arr[0]);

cout << "Original array: ";

printArray(arr, n);

// Sorting the array using bubble sort

bubbleSort(arr, n);

cout << "Sorted array: ";

printArray(arr, n);

return 0;

}

Following is the explanation of code:

• The bubbleSort function executes the bubble sort algorithm on an array of integers.

• It involves two nested loops: the outer loop (i) manages passes through the array, while the inner loop (j) navigates the unsorted segment of the array.

• Within the inner loop, elements are compared in pairs, and if they are in the incorrect order, they are swapped.

• printArray serves as a utility function to display the elements within an array.

• Inside the main() function, an array example is created, sorted using bubbleSort, and subsequently printed to exhibit the sorting process.

Complexity analysis

An algorithm's time complexity quantifies the time it requires to execute relative to the input size. In the case of bubble sort, its time complexity is commonly denoted in big O notation, indicating the maximum rate of growth for the algorithm.

In the case of bubble sort:

• Worst case time complexity (O(n2)): Bubble sort exhibits its worst-case scenario when the input list is arranged in reverse order. Here, the algorithm necessitates the highest count of comparisons and swaps to position the largest element correctly in each pass through the list. The quantity of comparisons and swaps escalates quadratically concerning the input list's size, leading to a time complexity of O(n2).

• Best case time complexity (O(n)): Bubble sort demonstrates its best-case scenario when the input list is already in sorted order. Despite needing passes through the list to verify that no swaps are required, the algorithm terminates early in each pass as no swaps are necessary. The count of comparisons remains proportional to n, while the number of swaps remains minimal, leading to a best-case time complexity of O(n).

• Average case time complexity (O(n2)): The average-case time complexity of bubble sort is O(n2). This estimation arises from the fact that, on average, it performs roughly n2/4 comparisons and n2/4 swaps when sorting a random arrangement of elements.

Selection sort

The selection sort algorithm divides an input list into two segments: the sorted and the unsorted. Initially, the sorted segment is empty, and the unsorted segment contains the complete list.

Throughout each iteration, the algorithm examines the unsorted segment, identifies the smallest element, and exchanges it with the leftmost unsorted element. This process progressively expands the sorted segment by integrating one element and reduces the unsorted segment by one element.

Algorithm 3.2: Algorithm of selection sort

selection_sort (arr)

n = length of arr

for i from 0 to n - 2 do

min_index = i

// Find the index of the minimum element in the unsorted part

for j from i + 1 to n - 1 do

if arr[j] < arr[min_index] then

min_index = j

// Swap the found minimum element with the first unsorted element

swap arr[i] with arr[min_index]

end for

end SelectionSort

Program 3.2: Implementation of selection sort

#include <iostream>

#include <vector>

// Function to perform Selection Sort

void selectionSort(std::vector<int> &arr) {

int n = arr.size();

// Iterate through the array

for (int i = 0; i < n - 1; ++i) {

// Assume the current index has the minimum value

int min_index = i;

// Find the index of the minimum element in the unsorted segment

for (int j = i + 1; j < n; ++j) {

// Compare current element with the assumed minimum

if (arr[j] < arr[min_index]) {

// If a smaller element is found, update the minimum index

min_index = j;

}

}

// Swap the found minimum element with the first unsorted element

std::swap(arr[min_index], arr[i]);

}

}

int main() {

// Example vector of integers

std::vector<int> elements = {64, 25, 12, 22, 11};

// Display the original array

std::cout << "Original Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

// Sort the array using Selection Sort

selectionSort(elements);

// Display the sorted array

std::cout << "Sorted Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

return 0;

}

Following is the explanation of code:

• The selectionSort function arranges the given vector arr utilizing the selection sort algorithm.

• The outer loop (for (int i = 0; i < n - 1; ++i)) cycles through the array.

• Within this loop, min_index is initially designated as i, presuming that the current index contains the minimum value.

• The inner loop (for (int j = i + 1; j < n; ++j)) identifies the index of the minimum element in the unsorted segment by comparing each element with the assumed minimum.

• If a smaller element is discovered, min_index is updated.

• Ultimately, the smallest element found in the unsorted segment gets swapped with the leftmost unsorted element.

In the main function:

• Elements initialized with vector.

• The original array is displayed.

• The selectionSort function is executed to sort the array.

• The sorted array is displayed after the sorting process.

Complexity analysis

The complexity analysis of selection sort involves evaluating its time and space complexities:

• Time complexity: In both the worst-case and average-case scenarios, selection sort requires roughly n2/2 comparisons and n swaps, where n represents the number of elements in the array. Consequently, the time complexity of selection sort remains O(n^2) across all cases.

Best-case: The best-case scenario arises when the input array is already sorted. However, even in the best-case, selection sort performs roughly n2/2 comparisons as it still iterates through the array to locate the minimum element in each iteration.

Consequently, the best-case time complexity of selection sort remains O(n2).

• Space complexity: Selection sort operates in-place, meaning it does not require additional memory proportional to the input size. It only uses a constant amount of extra space for variables like indices and temporary variables.

Therefore, the space complexity of selection sort is O(1) or constant space.

Insertion sort

Insertion sort is a straightforward sorting algorithm that constructs the final sorted array by gradually incorporating one element at a time. It traverses the input array, extracting each element and positioning it in its appropriate sorted position within the array.

Refer to the following list for a description of the insertion sort algorithm:

1. Commence from the second element (index 1) in the array.

2. Compare this element with the elements to its left (within the sorted section).

3. Shift elements larger than the current element one position forward to generate space for the current element.

4. Position the current element into its suitable sorted position among the elements to its left.

5. Carry out this procedure for each remaining element in the array until the entire array is sorted.

Algorithm 3.3: Algorithm of insertion sort

insertionSort(arr):

n = length of arr

for i = 1 to n - 1 do

key = arr[i]

j = i - 1

// Move elements greater than key to one position ahead

while j >= 0 and arr[j] > key do

arr[j + 1] = arr[j]

j = j - 1

arr[j + 1] = key

Program 3.3: Insertion sort

#include <iostream>

#include <vector>

// Function to perform Insertion Sort

void insertionSort(std::vector<int> &arr) {

int n = arr.size();

for (int i = 1; i < n; ++i) {

int key = arr[i];

int j = i - 1;

// Move elements greater than key to one position ahead

while (j >= 0 && arr[j] > key) {

arr[j + 1] = arr[j];

j = j - 1;

}

arr[j + 1] = key;

}

}

int main() {

std::vector<int> elements = {64, 25, 12, 22, 11};

std::cout << "Original Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

insertionSort(elements);

std::cout << "Sorted Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

return 0;

}

The following is the explanation of the preceding code:

• The insertionSort function organizes the given vector arr using the insertion sort algorithm.

• The outer loop (for (int i = 1; i < n; ++i)) navigates through the array, commencing from the second element.

• For each element at index i, it undergoes comparison with the elements to its left and is inserted into the appropriate position among the sorted elements.

• The inner loop (while (j >= 0 && arr[j] > key)) shifts elements greater than the key (the current element being sorted) one position ahead to make room for the key.

• Subsequently, the key element is placed in its correct sorted location.

• The main function initializes an example vector with elements, displays the original array, performs sorting using insertionSort, and showcases the resulting sorted array.

Complexity analysis

The complexity of insertion sort can be understood as follows:

• Time complexity: The time complexity of the insertion sort algorithm is as follows:

o In the worst-case scenario, where the array is in reverse order, insertion sort takes the maximum time to sort the elements.

o The outer loop executes n-1 times (where n represents the number of elements in the array).

o In the worst-case, each iteration of the inner loop might compare and shift nearly all elements to the right, resulting in approximately n/2 comparisons on average.

o As a result, in the worst-case, insertion sort performs approximately O(n2) comparisons and shifts to sort an array comprising n elements.

o In the best-case scenario, where the array is already sorted, insertion sort performs O(n) comparisons since each element is already in its correct position. However, it still executes O(n) shifts.

• Space complexity: Insertion sort maintains a space complexity of O(1) as it conducts an in-place sorting of the array, utilizing a constant amount of extra space for temporary variables, regardless of the input size.

While insertion sort exhibits efficiency for small datasets or partially sorted arrays due to its simplicity and linear-time performance in the best-case, it becomes less efficient for larger datasets because of its quadratic time complexity in the worst-case.

Shell sort

Shell sort, also known as Shell's method, is an efficient variation of insertion sort. It enhances insertion sort by enabling elements to be compared and moved over longer distances, thereby reducing the number of shifting operations.

Gap sequence

Shell sort utilizes a sequence of decreasing gaps (referred to as h-sorted), starting with a substantial gap and reducing it to a size of 1. Commonly used sequences include the Knuth sequence (3x + 1) or other predefined sequences.

Gap-sorted insertion can be described as follows:

1. The algorithm applies insertion sort for each gap.

2. Elements separated by a distance of gap are compared and moved to their appropriate positions within the subarrays.

3. The sequence of gaps determines the number of subarrays and their individual sizes.

4. As the algorithm progresses, it reduces the gap size.

5. Eventually, the gap diminishes to 1, resulting in a final pass of the insertion sort that effectively resembles a nearly sorted array.

Algorithm 3.4: Algorithm of shell sort

ShellSort (arr):

n = length of arr

// Start with a large gap

for gap = n/2; gap > 0; gap = gap / 2 do

// Perform insertion sort for each gap

for i = gap to n - 1 do

temp = arr[i]

j = i

// Compare elements at a distance of gap and swap if necessary

while j >= gap and arr[j - gap] > temp do

arr[j] = arr[j - gap]

j = j - gap

arr[j] = temp

Program 3.4: Implementation of shell sort

#include <iostream>

#include <vector>

// Function to perform Shell Sort

void shellSort(std::vector<int>& arr) {

int n = arr.size();

// Generate gap sequence (Knuth sequence)

int gap = 1;

while (gap <= n / 3) {

gap = gap * 3 + 1;

}

while (gap > 0) {

for (int i = gap; i < n; ++i) {

int temp = arr[i];

int j = i;

// Perform Insertion Sort on subarrays

while (j >= gap && arr[j - gap] > temp) {

arr[j] = arr[j - gap];

j -= gap;

}

arr[j] = temp;

}

gap /= 3; // Reduce the gap

}

}

int main() {

std::vector<int> elements = {64, 25, 12, 22, 11};

std::cout << "Original Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

shellSort(elements);

std::cout << "Sorted Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

return 0;

}

Complexity analysis

The complexity of the shell sort algorithm can be understood as follows:

• Time complexity: The time complexity of the shell sort algorithm is as follows:

o Shell sort's time complexity relies on the gap sequence used.

o Typically, the average time complexity ranges between O(n log2 n) and O(n{4/3}).

o In specific scenarios with particular gap sequences, the worst-case time complexity can reach O(n2), resembling insertion sort's worst-case scenario.

o However, for many practical cases and specific gap sequences, shell sort surpasses O(n2) algorithms like insertion sort or bubble sort.

• Space complexity: Having a space complexity of O(1), shell sort arranges the array in place, utilizing a set amount of additional space for temporary variables.

Shell sort improves upon insertion sort by allowing elements to move more than one position at a time towards their sorted positions, thereby reducing the number of comparisons and swaps. The efficiency of shell sort heavily depends on the chosen gap sequence.

Counting sort

Counting sort, an integer sorting technique, does not rely on comparisons between elements. Instead, it tallies the occurrences of unique elements within an input array. Using this tally, it arranges the elements into their rightful sorted sequence. It involves the following:

1. Counting occurrences: Let us talk about counting occurrences:

a. Determine the range of elements within the input array and create a count array (commonly known as a frequency array) to store the occurrence count of each element.

b. Iterate through the input array, incrementing the count of each element in the count array.

2. Cumulative sum: Let us talk about cumulative sum:

a. Adjust the count array to hold the cumulative sum of counts.

b. This cumulative sum indicates the index positions where elements should be placed within the sorted array.

3. Sorting: Sorting involves the following:

a. Traverse the input array once more.

b. For each element, determine its position using the count array and place it correctly in the sorted array.

c. Decrease the count in the count array to manage duplicate elements.

Algorithm 3.5: Algorithm of counting sort

countingSort(arr):

n = length of arr

max_element = maximum element in arr

min_element = minimum element in arr (if negative numbers are present)

range = max_element - min_element + 1

// Create count array and initialize with zeros

count[range] = {0}

// Count occurrences of each element

for i = 0 to n - 1 do

count[arr[i] - min_element]++

// Modify count array to store cumulative sum

for i = 1 to range - 1 do

count[i] += count[i - 1]

// Create the sorted array using count array

sorted[n] = {0}

for i = n - 1 to 0 do

sorted[count[arr[i] - min_element] - 1] = arr[i]

count[arr[i] - min_element]--

// Copy the sorted array back to the original array

for i = 0 to n - 1 do

arr[i] = sorted[i]

Program 3.5: Implementation of counting sort

#include <iostream>

#include <vector>

void countingSort(std::vector<int>& arr) {

int n = arr.size();

// Find the maximum and minimum elements in the array

int max_element = *std::max_element(arr.begin(), arr.end());

int min_element = *std::min_element(arr.begin(), arr.end());

int range = max_element - min_element + 1;

// Create a count array to store the count of each element

std::vector<int> count(range, 0);

// Count occurrences of each element

for (int i = 0; i < n; ++i) {

count[arr[i] - min_element]++;

}

// Modify count array to store cumulative sum

for (int i = 1; i < range; ++i) {

count[i] += count[i - 1];

}

// Create the sorted array using count array

std::vector<int> sorted(n);

for (int i = n - 1; i >= 0; --i) {

sorted[count[arr[i] - min_element] - 1] = arr[i];

count[arr[i] - min_element]--;

}

// Copy the sorted array back to the original array

for (int i = 0; i < n; ++i) {

arr[i] = sorted[i];

}

}

int main() {

std::vector<int> elements = {64, 25, 12, 22, 11};

std::cout << "Original Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

countingSort(elements);

std::cout << "Sorted Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

return 0;

}

The following is the explanation of code:

• The countingSort function arranges the given vector, arr, using the counting sort algorithm.

• It identifies the range of elements by determining the maximum and minimum values within the array.

• A count array is created to monitor the occurrence count of each element.

• It tallies and stores the occurrences of each element from the input array into the count array.

• The count array is modified to represent the cumulative count of elements.

• A sorted array is created by accurately positioning elements based on information from the count array.

• Finally, the sorted array is copied back into the original array.

The counting sort algorithm efficiently sorts integers within a specific range by counting the occurrences of each element to determine their positions in the sorted array. This implementation works well for positive integers. However, to handle a broader range of integers, including negative values, adjustments might be necessary.

Complexity analysis

The complexity analysis of counting sort is as follows:

• Time complexity: The time complexity of the counting sort algorithm can be explained as follows:

o Counting sort demonstrates a time complexity of O(n + k), where n stands for the quantity of elements in the input array, and k denotes the input range.

o The algorithm consists of two primary phases: counting occurrences and generating the sorted array based on the count data.

o Counting the occurrences of each element in the input array takes O(n) time.

o Generating the sorted array using the count information requires O(n + k) time.

• Space complexity: The space complexity of the counting sort algorithm is as follows:

o Counting sort demonstrates a space complexity of O(n + k), where n represents the number of elements in the input array and k signifies the range of input values.

o Additional space is required for both the count array and the sorted array.

Counting sort showcases notable efficiency in sorting a range of integers. However, its effectiveness may diminish if the range of elements substantially exceeds the number of elements to be sorted, primarily due to increased space requirements.

Radix sort

Radix sort is a non-comparative integer sorting algorithm that operates by sorting elements digit by digit, either from the least significant digit (LSD) to the most significant digit (MSD) or vice versa. This algorithm functions based on the concept of categorizing elements into buckets according to their digit values. The steps involved are as follows:

1. LSD radix sort: This involves the following procedure:

a. Sorting initiates from the rightmost (least significant) digit and progresses towards the leftmost (most significant) digit.

b. Each digit column undergoes stable sorting methods such as counting sort or bucket sort.

2. Digit extraction: In digit extraction, the following is done:

a. Traverse through the elements, extracting the current digit (from LSD to MSD) to categorize elements into their respective buckets.

b. Elements with the same digit value are grouped into the same bucket.

3. Bucketing and combining: Once the elements are sorted according to the current digit, reassemble them into a single array.

4. Repeat for all digits: Continue the process for each digit position until the entire array is sorted.

a. When sorting decimal numbers, the typical procedure involves executing radix sort on each digit (0-9) from right to left.

Algorithm 3.6: Algorithm of radix sort

LSD Radix sort (sorting from least significant digit):

radixSort(arr):

n = length of arr

max_element = maximum element in arr

// Perform sorting for each digit place, starting from the least significant digit

for exp = 1 to max_element do

// Initialize count array for each digit (0-9)

count[10] = {0}

// Count occurrences of each digit in the current place

for i = 0 to n - 1 do

digit = (arr[i] / exp) % 10

count[digit]++

// Modify count array to store cumulative sum

for i = 1 to 9 do

count[i] += count[i - 1]

// Create the sorted array using count array

sorted[n] = {0}

for i = n - 1 to 0 do

digit = (arr[i] / exp) % 10

sorted[count[digit] - 1] = arr[i]

count[digit]--

// Copy the sorted array back to the original array

for i = 0 to n - 1 do

arr[i] = sorted[i]

Program 3.6: Implementation of radix sort

#include <iostream>

#include <vector>

void countingSort(std::vector<int>& arr, int exp) {

int n = arr.size();

std::vector<int> output(n);

std::vector<int> count(10, 0);

for (int i = 0; i < n; ++i)

count[(arr[i] / exp) % 10]++;

for (int i = 1; i < 10; ++i)

count[i] += count[i - 1];

for (int i = n - 1; i >= 0; --i) {

output[count[(arr[i] / exp) % 10] - 1] = arr[i];

count[(arr[i] / exp) % 10]--;

}

for (int i = 0; i < n; ++i)

arr[i] = output[i];

}

void radixSort(std::vector<int>& arr) {

int max_element = *std::max_element(arr.begin(), arr.end());

for (int exp = 1; max_element / exp > 0; exp *= 10)

countingSort(arr, exp);

}

int main() {

std::vector<int> elements = {170, 45, 75, 90, 802, 24, 2, 66};

std::cout << "Original Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

radixSort(elements);

std::cout << "Sorted Array: ";

for (int elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

return 0;

}

Let us look at the explanation of the code:

o The radixSort function arranges the given vector arr using LSD radix sort.

o It determines the maximum element in the array to establish the number of digits in the largest number.

o The countingSort function is a modified counting sort explicitly crafted to sort digits at each place value.

o The main function initializes an example vector, sorts it using radixSort, and displays the sorted array.

Radix sort operates by sorting individual digits at each place value, effectively organizing the entire array based on these digits, either from the least significant to the most significant or vice versa. This iterative process ultimately results in a fully sorted array.

Complexity analysis

The complexity analysis of radix sort is as follows:

• Time complexity: The time complexity of the radix sort algorithm can be explained as follows:

o The time complexity of radix sort is O(n*k), where n denotes the number of elements in the input array and k represents the number of digits or characters in the largest number or key.

o The algorithm involves iterating through each digit of every element and performing a stable sort (such as counting sort or bucket sort) for each digit.

o The number of iterations through the digits is determined by the quantity of digits present in the largest element.

o Since each iteration processes the entire array, the time complexity grows linearly with the number of digits.

• Space complexity: The space complexity of the radix sort algorithm can be explained in the following manner:

o The space complexity of radix sort is O(n + k), where n represents the number of elements in the input array, while k denotes the range of input values.

o Additional space is required for creating count arrays and auxiliary arrays during the sorting process.

o If counting sort is used as a subroutine, additional space is needed for the count array, which increases in size proportionally to the range of elements.

Radix sort demonstrates efficiency in sorting integers or fixed-length strings when the number of digits is confined and predetermined. Its time complexity shows linearity in relation to both the number of elements and the number of digits in the largest element. This characteristic makes it suitable for specific scenarios that adhere to the constraints of the input data.

Bucket sort

Bucket sort is a distribution-based sorting algorithm that segments the input into several buckets, each capable of accommodating a specific range of values. After distributing the elements across these buckets, each bucket undergoes individual sorting, often utilizing another sorting algorithm or recursively employing the bucket sort algorithm itself. Ultimately, the sorted elements from all buckets are merged together to create the final sorted array.

The algorithm's description is as follows:

1. Creating buckets: The first step involves:

a. Determine the number of buckets needed.

b. Traverse through the input array and assign elements to their respective buckets using a mapping function that allocates elements to specific buckets.

2. Sorting individual buckets: This can be done as follows:

a. Conduct separate sorting for each bucket, employing either another sorting algorithm or employing bucket sort recursively when required.

b. Each bucket can undergo sorting using a suitable sorting algorithm, typically insertion sort or quick sort, acknowledged for their efficiency on smaller datasets.

3. Concatenating buckets: Combine all the sorted buckets to form the final sorted array.

The key points of bucket sorting are as follows:

• Bucket distribution: Elements are assigned to buckets using a function that evenly distributes elements across buckets, ensuring a reasonably uniform distribution.

• Bucket sort variation: When the range of input values is known and comfortably fits within the available memory, bucket sort can operate independently as a sorting algorithm.

• Stability: The stability of bucket sort relies on the stability of the sorting algorithm employed for sorting individual buckets. For instance, when a stable sorting algorithm like insertion sort is utilized, it maintains the overall stability of bucket sort.

Bucket sort operates efficiently with uniformly distributed data and proves especially valuable when the input range is well-defined and limited. However, its efficiency may decrease when dealing with non-uniform data distributions or if the number of elements in a single bucket significantly exceeds expectations.

Algorithm 3.7: Algorithm of bucket sort

bucketSort(arr):

n = length of arr

create n empty buckets

// Distribution of elements into buckets

for i = 0 to n - 1 do

index = n * arr[i] // Determine which bucket the element should go to

insert arr[i] into bucket[index]

// Sorting individual buckets

for i = 0 to n - 1 do

sort bucket[i] using a suitable sorting algorithm (e.g., Insertion Sort, Quick Sort)

// Concatenate sorted buckets to get the final sorted array

concatenate all elements in bucket[i] to the output array

Program 3.7: Implementation of bucket sort

#include <iostream>

#include <vector>

#include <algorithm>

// Function to perform Bucket Sort

void bucketSort(std::vector<float>& arr) {

int n = arr.size();

// Create n empty buckets

std::vector<std::vector<float>> buckets(n);

// Distribution of elements into buckets

for (int i = 0; i < n; ++i) {

int index = n * arr[i];

buckets[index].push_back(arr[i]);

}

// Sorting individual buckets (using a suitable sorting algorithm)

for (int i = 0; i < n; ++i) {

std::sort(buckets[i].begin(), buckets[i].end());

}

// Concatenate sorted buckets to get the final sorted array

int index = 0;

for (int i = 0; i < n; ++i) {

for (float num : buckets[i]) {

arr[index++] = num;

}

}

}

int main() {

std::vector<float> elements = {0.42, 0.32, 0.33, 0.52, 0.37, 0.47, 0.51};

std::cout << "Original Array: ";

for (float elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

bucketSort(elements);

std::cout << "Sorted Array: ";

for (float elem : elements) {

std::cout << elem << " ";

}

std::cout << std::endl;

return 0;

}

Let us look at the explanation of code:

• The bucketSort function sorts the provided vector arr using bucket sort.

• It creates n empty buckets and allocates elements into the respective buckets based on a function that determines the bucket index for each element.

• Each bucket undergoes individual sorting using std::sort, a sorting algorithm.

• Finally, the sorted elements from all buckets are merged to generate the final sorted array.

Complexity analysis

The complexity analysis of bucket sort is as follows:

• Time complexity: The time complexity of bucket sort relies on various factors:

o n: The number of elements to be sorted.

o k: The number of buckets created.

o m: The average number of elements in each bucket.

In the average-case, when elements are uniformly distributed across buckets:

o Distributing elements into buckets takes O(n) time.

o Sorting each individual bucket using an efficient algorithm (like insertion sort, quick sort, etc.) requires O(m*log m) time per bucket on average.

The total time complexity can be approximated as O(n + k * m * log m).

When m is close to n and elements are evenly distributed across buckets, bucket sort approaches O(n * log n) time complexity. However, uneven distribution of elements leading to a few buckets holding a large number of elements (resulting in large **m**) can degrade the time complexity, approaching O(n2) due to the sorting of such large buckets.

• Space complexity: The space complexity of bucket sort is O(n + k), where:

o n represents the number of elements in the input array.

o k denotes the number of buckets created.

Bucket sort exhibits efficiency when the elements are evenly distributed across buckets. However, its performance can vary based on the data distribution and the sorting algorithm chosen for sorting individual buckets. Adjustments in bucket creation or the selection of sorting algorithms for buckets can significantly influence its practical efficiency.

Conclusion

In summary, this chapter covered a variety of sorting algorithms, such as bubble sort, selection sort, and insertion sort. Understanding these methods equips you to select and implement the most appropriate algorithm for different scenarios, ensuring efficiency and optimal performance. In the next chapter, we will focus on searching algorithms, such as linear search, binary search, interpolation search, jump search, exponential search, Fibonacci search, and sublist search.

Exercise

1. Implement the bubble sort algorithm in your preferred programming language. Analyze its time complexity in best, average, and worst cases.

2. Given the following unsorted list: [45, 12, 89, 33, 22, 10], manually trace and show the steps of selection sort to sort the list in ascending order.

3. Compare insertion sort and selection sort. Which performs better on nearly sorted data? Justify your answer with an example.

4. Design a program that implements shell sort. Use a gap sequence of your choice and discuss its impact on performance.

5. Use counting sort to sort the array: [4, 2, 2, 8, 3, 3, 1]. Show the intermediate counting array and explain how the algorithm works step by step.

6. Explain how radix sort works and apply it to sort the list: [170, 45, 75, 90, 802, 24, 2, 66]. Use the LSD approach.

7. Design and implement bucket sort for sorting a list of floating-point numbers in the range [0,1). Use at least five buckets and show step-by-step bucket assignment and sorting.

8. Write a function that takes a list of integers and a sorting algorithm choice (bubble, insertion, or selection) and returns the sorted list. Demonstrate how modular design can be applied to sorting algorithms.

9. Suppose you have a dataset with millions of integers in a small range (e.g., 1–100). Which sorting algorithm would you choose and why? Analyze the efficiency compared to other algorithms.

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 4Searching Algorithms

Introduction

In this chapter of the book, we will discuss searching algorithms and their utility in the field of computer science engineering. The act of searching is a fundamental and essential operation in computer science, involving the precise identification of a specific element within a collection of data. This exploration spans various data structures, such as lists, arrays, trees, and others, emphasizing the critical nature of efficient searching across diverse applications and problem-solving scenarios. Playing a pivotal role in this endeavor are searching algorithms, acting as the cornerstone in determining the most optimal and effective approaches to locate the desired item. These algorithms serve as the backbone of the search process, supplying the computational process necessary for navigating and extracting pertinent information from intricate data structures.

Structure

This searching algorithm chapter covers the following topics:

• Searching

• Linear search

• Binary search

• Interpolation search

• Jump search

• Exponential search

• Fibonacci search

• Sublist search

Objectives

This chapter provides readers with the knowledge of searching algorithms by means of usage and implementation. The fundamental ideas are time and space efficiency, accuracy, and adaptation for various data structures and scalability for growing datasets. Readers will be shown the selection process of best algorithms coupled with actual implementation and maintenance techniques depending on problem needs.

Searching

In the context of data structures, searching refers to the process of locating a particular element within a collection of data. The goal is to determine whether a specific item exists in the data structure and, if so, to find its location or retrieve it. The search operation is fundamental and common in various applications, and it is crucial for efficiently accessing and manipulating data.

The importance of searching

Searching is a fundamental operation in computer science and data processing that serves several essential purposes, which are as follows:

• Retrieval of information: One of the primary reasons for searching is to locate specific pieces of information within a dataset. This could be anything from finding a particular record in a database, locating a file on a computer, or searching for a specific value in an array.

• Data analysis: In many applications, searching is a crucial step in data analysis. Researchers, analysts, and scientists often need to search through large datasets to identify patterns, trends, or specific data points relevant to their study.

• Information retrieval systems: Search is a key component in information retrieval systems, such as search engines. Users rely on search functionality to find relevant documents, websites, or information on the internet.

• Database operations: Databases store vast amounts of information, and searching is essential for retrieving specific records or subsets of data. Efficient searching algorithms are crucial for optimizing the performance of database queries.

• Sorting and filtering: Searching is often used in conjunction with sorting and filtering operations. For example, in a list or database, users may want to search for specific items based on certain criteria while also sorting the results in a particular order.

• Algorithmic problem-solving: Many algorithmic problems involve searching for a solution within a given space. Efficient search algorithms are critical for solving problems in areas such as artificial intelligence, optimization, and computational biology.

• Navigation: In various applications, users navigate through data structures by searching. This is common in file systems, directories, and other hierarchical structures.

• Decision-making: In decision-making processes, searching is often employed to find the best solution or optimal choice among a set of alternatives.

Linear search

Linear search, also called sequential search, is a very simple method used for searching an array for a particular value. It works by comparing the value to be searched with every element of the array one by one in a sequence until a match is found. Linear search is mostly used to search an unordered list of elements (an array in which data elements are not sorted). For example, consider an array, Array1[], which is declared and initialized as follows:

int Array1[] = {11, 18, 12, 17, 13, 14, 19, 10, 16, 15};

In this array, the value to be searched is VAL = 17, then searching means to find whether the value 17 is present in the array or not. If yes, then it returns the position of its occurrence. Here, POS = 3 (index starting from 0).

Algorithm 4.1: Algorithm of linear search

LINEAR_SEARCH(A, N, VAL)

Step 1: [INITIALIZE] SET POS = -1

Step 2: [INITIALIZE] SET I = 1

Step 3: Repeat Step 4 while I<=N

Step 4: IF A[I] = VAL

SET POS = I

PRINT POS

Go to Step 6

[END OF IF]

[END OF LOOP]

Step 6: EXIT

SET I = I + 1

Step 5: IF POS = –1

PRINT VALUE IS NOT PRESENT

IN THE ARRAY

[END OF IF]

In Step 1 and Step 2 of the algorithm, we initialize the value of POS and I. In Step 3, a while loop is executed that would be executed till I is less than N (total number of elements in the array). In Step 4, a check is made to see if a match is found between the current array element and VAL. If a match is found, then the position of the array element is printed; else, the value of I is incremented to match the next element with VAL. However, if all the array elements have been compared with VAL and no match is found, then it means that VAL is not present in the array.

Complexity of linear search algorithm

The linear search algorithm, also known as sequential search, has a time complexity of O(n), where n is the number of elements in the array. The complexity analysis is as follows:

• Time complexity: The time complexity analysis is different based on the scenario, shown as follows:

o Worst-case scenario: In the worst-case scenario, the element being searched for is at the last position in the array or is not present at all. In this case, the algorithm needs to iterate through the entire array to determine that the element is not present. The maximum number of comparisons is n. Therefore, the worst-case time complexity is O(n).

o Best-case scenario: The best-case scenario occurs when the element being searched is found at the beginning of the array. In this case, the algorithm makes only one comparison. The best-case time complexity is O(1). However, best-case time complexity is not commonly used for representing the efficiency of an algorithm because it does not provide a meaningful average case.

o Average-case scenario: In the average case, the linear search algorithm, on average, needs to inspect half of the array elements before finding the target element (if it exists). Therefore, the average-case time complexity is O(n/2), which is still expressed as O(n) in big O notation.

• Space complexity: The space complexity of the linear search algorithm is O(1) because it uses a constant amount of extra space for variables (like loop counters and temporary variables) that do not depend on the input size.

Program 4.1: Implementation of linear search

#include <stdio.h>

#include <stdlib.h>

#include <conio.h>

#define size 20 // Added so the size of the array can be altered more easily

int main(int argc, char *argv[]) {

int arr[size], num, i, n, found = 0, pos = -1;

printf("\n Enter the number of elements in the array : ");

scanf("%d", &n);

printf("\n Enter the elements: ");

for(i=0;i<n;i++)

{

scanf("%d", &arr[i]);

}

printf("\n Enter the number that has to be searched : ");

scanf("%d", &num);

for(i=0;i<n;i++)

{

if(arr[i] == num)

{

found =1;

pos=i;

printf("\n %d is found in the array at position= %d", num,i+1);

*+1 added in line 23 so that it would display the number in

the first place in the array as in position 1 instead of 0 */

break;

}

}

if (found == 0)

printf("\n %d does not exist in the array", num);

return 0;

}

Binary search

Binary search is a searching algorithm that works efficiently with a sorted list. The mechanism of binary search can be better understood by an analogy of a telephone directory. When we are searching for a particular name in a directory, we first open the directory from the middle and then decide whether to look for the name in the first part of the directory or in the second part of the directory. Again, we open some page in the middle, and the whole process is repeated until we finally find the right name.

Let us consider another analogy. To find words in a dictionary, we first open the dictionary somewhere in the middle. Then, we compare the first word on that page with the desired word whose meaning we are looking for. If the desired word comes before the word on the page, we look in the first half of the dictionary; otherwise, we look in the second half. Again, we open a page in the first half of the dictionary, compare the first word on that page with the desired word, and repeat the same procedure until we finally get the word. The same mechanism is applied in binary search.

Now, let us consider how this mechanism is applied to search for a value in a sorted array. Consider an array, A[], that is declared and initialized as follows:

int A[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

Here, the value to be searched is VAL = 9. The algorithm will proceed in the following manner:

• BEG = 0, END = 10, MID = (0 + 10)/2 = 5

• Now, VAL = 9 and A[MID] = A[5] = 5

• A[5] is less than VAL, therefore, we now search for the value in the second half of the array. So, we change the values of BEG and MID.

• Now, BEG = MID + 1 = 6, END = 10, MID = (6 + 10)/2 =16/2 = 8

• VAL = 9 and A[MID] = A[8] = 8

• [8] is less than VAL, therefore, we now search for the value in the second half of the segment.

• So, again, we change the values of BEG and MID.

• Now, BEG = MID + 1 = 9, END = 10, MID = (9 + 10)/2 = 9

• Now, VAL = 9 and A[MID] = 9

In this algorithm, we see that BEG and END are the beginning and ending positions of the segment that we are looking to search for the element. MID is calculated as (BEG + END)/2. Initially, BEG = lower_bound and END = upper_bound. The algorithm will terminate when A[MID] = VAL. When the algorithm ends, we will set POS = MID. POS is the position at which the value is present in the array.

However, if VAL is not equal to A[MID], then the values of BEG, END, and MID will be changed depending on whether VAL is smaller or greater than A[MID] in the following manner:

• If VAL < A[MID], then VAL will be present in the left segment of the array. So, the value of END will be changed as END = MID – 1.

• If VAL > A[MID], then VAL will be present in the right segment of the array. So, the value of BEG will be changed as BEG = MID + 1.

Finally, if VAL is not present in the array, then eventually, END will be less than BEG. When this happens, the algorithm will terminate, and the search will be unsuccessful.

Algorithm 4.2: Algorithm of binary search

BINARY_SEARCH(A, lower_bound, upper_bound, VAL)

Step 1: [INITIALIZE] SET BEG = lower_bound

END = upper_bound, POS = - 1

Step 2: Repeat Steps 3 and 4 while BEG <= END

Step 3: SET MID = (BEG + END)/2

Step 4: IF A[MID] = VAL

SET POS = MID

PRINT POS

Go to Step 6

ELSE IF A[MID] > VAL

SET END = MID - 1

ELSE

SET BEG = MID + 1

[END OF IF]

[END OF LOOP]

Step 5: IF POS = -1

PRINT “VALUE IS NOT PRESENT IN THE ARRAY.”

[END OF IF]

Step 6: EXIT

In Step 1, we initialize the value of variables BEG, END, and POS. In Step 2, a while loop is executed until BEG is less than or equal to END. In Step 3, the value of MID is calculated. In Step 4, we check if the array value at MID is equal to VAL (the item to be searched in the array). If a match is found, then the value of POS is printed, and the algorithm exits. However, if a match is not found, and if the value of A[MID] is greater than VAL, the value of END is modified. Otherwise, if A[MID] is greater than VAL, then the value of BEG is altered. In Step 5, if the value of POS = –1, then VAL is not present in the array, and an appropriate message is printed on the screen before the algorithm exits.

Program 4.2: Implementation of binary search

#include <stdio.h>

#include <stdlib.h>

#include <conio.h>

#define size 10 // Added to make changing size of array easier

int smallest(int arr[], int k, int n); // Added to sort array

void selection_sort(int arr[], int n); // Added to sort array

int main(int argc, char *argv[]) {

int arr[size], num, i, n, beg, end, mid, found=0;

printf("\n Enter the number of elements in the array: ");

scanf("%d", &n);

printf("\n Enter the elements: ");

for(i=0;i<n;i++)

{

scanf("%d", &arr[i]);

}

selection_sort(arr, n); // Added to sort the array

printf("\n The sorted array is: \n");

for(i=0;i<n;i++)

printf(" %d\t", arr[i]);

printf("\n\n Enter the number that has to be searched: ");

scanf(“%d”, &num);

beg = 0, end = n-1;

while(beg<=end)

{

mid = (beg + end)/2;

if (arr[mid] == num)

{

printf("\n %d is present in the array at position %d", num, mid+1);

found =1;

break;

}

else if (arr[mid]>num)

end = mid-1;

else

beg = mid+1;

}

if (beg > end && found == 0)

printf("\n %d does not exist in the array", num);

return 0;

}

int smallest(int arr[], int k, int n)

{

int pos = k, small=arr[k], i;

for(i=k+1;i<n;i++)

{

if(arr[i]< small)

{

small = arr[i];

pos = i;

}

}

return pos;

}

void selection_sort(int arr[],int n)

{

int k, pos, temp;

for(k=0;k<n;k++)

{

pos = smallest(arr, k, n);

temp = arr[k];

arr[k] = arr[pos];

arr[pos] = temp;

}

}

Complexity of binary search algorithm

The binary search algorithm is more efficient compared to linear search, especially for sorted arrays. Let us analyze the complexity of the binary search algorithm:

• Time complexity: The time complexity analysis is as follows:

o Worst-case scenario: In the worst-case scenario, the element being searched is either not present in the array or is at the last position. The binary search algorithm divides the array in half at each step, and the search space is reduced by half in each iteration. The maximum number of comparisons needed to find the element is logarithmic with base 2 of the number of elements in the array. Therefore, the worst-case time complexity is O(log n), where n is the number of elements in the array.

o Best-case scenario: The best-case scenario occurs when the element being searched is exactly in the middle of the array. In this case, the algorithm finds the element in the first comparison. The best-case time complexity is O(1). However, similar to linear search, the best-case is not commonly used for representing the efficiency of an algorithm because it does not provide a meaningful average case.

o Average-case scenario: On average, binary search also requires logarithmic time. The average-case time complexity is O(log n), which is considered very efficient.

• Space complexity: The space complexity of binary search is O(1) because it uses a constant amount of extra space for variables (like low, high, and mid) that do not depend on the input size.

Interpolation search

Interpolation search, also known as extrapolation search, is a searching technique that finds a specified value in a sorted array. The concept of interpolation search is similar to how we search for names in a telephone book or for keys by which a book’s entries are ordered. For example, when looking for a name, such as Bharat, in a telephone directory, we know that it will be near the extreme left, so applying a binary search technique by dividing the list in two halves each time is not a good idea. We must start scanning the extreme left in the first pass itself.

In each step of the interpolation search, the remaining search space for the value to be found is calculated. The calculation is done based on the values at the bounds of the search space and the value to be searched. The value found at this estimated position is then compared with the value being searched for. If the two values are equal, then the search is complete.

However, in case the values are not equal, then depending on the comparison, the remaining search space is reduced to the part before or after the estimated position. Thus, we see that interpolation search is similar to the binary search technique. However, the important difference between the two techniques is that binary search always selects the middle value of the remaining search space. It discards half of the values based on the comparison between the value found at the estimated position and the value to be searched. But in interpolation search, interpolation is used to find an item near the one being searched for, and then linear search is used to find the exact item.

Algorithm 4.3: Algorithm of interpolation search

INTERPOLATION_SEARCH (A, lower_bound, upper_bound, VAL)

Step 1: [INITIALIZE] SET LOW = lower_bound,

HIGH = upper_bound, POS = –1

Step 2: Repeat Steps 3 to 4 while LOW <= HIGH

Step 3: SET MID = LOW + (HIGH – LOW) ×((VAL – A[LOW]) / (A[HIGH] – A[LOW]))

Step 4: IF VAL = A[MID]

POS = MID

PRINT POS

Go to Step 6

ELSE IF VAL < A[MID]

SET HIGH = MID – 1

ELSE

SET LOW = MID + 1

[END OF IF]

[END OF LOOP]

Step 5: IF POS = –1

PRINT "VALUE IS NOT PRESENT IN THE ARRAY"

[END OF IF]

Step 6: EXIT

Example 4.1: Given a list of numbers arr[] = {11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31}. Search for value 19 using interpolation search technique.

Solution: The solution is as follows:

Low = 0, High = 10, VAL = 29, a[Low] = 11, a[High] = 31

Middle = Low + (High – Low)×((VAL – a[Low]) /(a[High] – a[Low] ))

= 0 +(10 – 0) × ((29 – 11) / (31 – 11) )

= 0 + 10 × 0.9 = 9

arr[middle] = arr[9] = 29 which is equal to value to be searched.

Middle = (Low + High) / 2

(Binary search approach for division of given array list in 2 halves)

Middle = Low + (High - Low) X ((key – Arr[Low]) / (Arr[High] – Arr[Low]))

(Interpolation search approach for division of given array list in 2 halves)

Program 4.3: Implementation of interpolation search

#include <stdio.h>

#include <conio.h>

#define MAX 20

int interpolation_search(int a[], int low, int high, int val)

{

int mid;

while(low <= high)

{

mid = low + (high – low)*((val – a[low]) / (a[high] – a[low]));

if(val == a[mid])

return mid;

if(val < a[mid])

high = mid – 1;

else

low = mid + 1;

}

return –1;

}

int main()

{

int arr[MAX], i, n, val, pos;

clrscr();

printf("\n Enter the number of elements in the array : ");

scanf("%d", &n);

printf("\n Enter the elements : ");

for(i = 0; i <n; i++)

scanf("%d", &arr[i]);

printf("\n Enter the value to be searched : ");

scanf("%d", &val);

pos = interpolation_search(arr, 0, n–1, val);

if(pos == –1)

printf("\n %d is not found in the array", val);

else

printf("\n %d is found at position %d", val, pos);

getche();

return 0;

}

Complexity of interpolation search algorithm

Interpolation search is an algorithm for searching for a target value within a sorted array of elements. It improves upon binary search by making a more informed guess about the position of the target element based on the distribution of values in the array. Let us analyze the complexity of the interpolation search algorithm:

• Time complexity: The time complexity analysis is as follows:

o Worst-case scenario: The worst-case scenario for interpolation search occurs when the target element is either the smallest or largest element in the array or when the values in the array are not uniformly distributed. In such cases, the interpolation search may degrade to perform a linear search, leading to a worst-case time complexity of O(n). This happens because the interpolation formula may not provide a good estimate of the target's position, causing the algorithm to traverse through a significant portion of the array.

o Best-case scenario: The best-case scenario for interpolation search occurs when the target element is found at the middle of the array in the first iteration. In this case, the time complexity is O(1), as the algorithm directly finds the target without any further comparisons.

o Average-case scenario: The average-case time complexity of interpolation search is typically expressed as O(log(log(n))). This means that, on average, the algorithm performs logarithmic time operations. However, it is important to note that this average-case complexity assumes a uniform distribution of elements in the array. The interpolation search algorithm calculates an approximate position for the target element based on the values at the ends of the array and uses this estimate to guide the search towards the target.

• Space complexity: The space complexity of the interpolation search algorithm is O(1), meaning it requires constant space regardless of the size of the input array.

Jump search

When we have an already sorted list, then another efficient algorithm to search for a value is jump search or block search. In jump search, it is not necessary to scan all the elements in the list to find the desired value. We just check an element, and if it is less than the desired value, then some of the elements following it are skipped by jumping ahead. After moving a little forward, the element is checked again. If the checked element is greater than the desired value, then we have a boundary, and we are sure that the desired value lies between the previously checked element and the currently checked element. However, if the checked element is less than the value being searched for, then we again make a small jump and repeat the process.

Once the boundary of the value is determined, a linear search is done to find the value and its position in the array. For example, consider an array arr[] = {11,12,13,14,15,16,17,18,19}. The length of the array is 9. If we have to find value 18, then the following steps are performed using the jump search technique:

1. The first three elements are checked. Since 3 is smaller than 8, we will have to make a jump ahead, shown as follows:

11

12

13

14

15

16

17

18

19

2. The next three elements are checked. Since 6 is smaller than 8, we will have to make a jump ahead.

11

12

13

14

15

16

17

18

19

3. The next three elements are checked. Since 9 is greater than 8, the desired value lies within the current boundary, illustrated as follows:

11

12

13

14

15

16

17

18

19

4. A linear search is now done to find the value in the array.

Algorithm 4.4: Algorithm of jump search

JUMP_SEARCH (A, lower_bound, upper_bound, VAL, N)

Step 1: [INITIALIZE] SET STEP = sqrt(N), I = 0, LOW = lower_bound, HIGH = upper_bound, POS = –1

Step 2: Repeat Step 3 while I < STEP

Step 3: IF VAL < A[STEP]

SET HIGH = STEP – 1

ELSE

SET LOW = STEP + 1

[END OF IF]

SET I = I + 1

[END OF LOOP]

Step 4: SET I = LOW

Step 5: Repeat Step 6 while I <= HIGH

Step 6: IF A[I] = Val

POS = I

PRINT POS

Go to Step 8

[END OF IF]

SET I = I + 1

[END OF LOOP]

Step 7: IF POS = –1

PRINT "VALUE IS NOT PRESENT IN THE ARRAY"

[END OF IF]

Step 8: EXIT

Advantages of jump search over linear search

Suppose we have a sorted list of 1000 elements where the elements have values 0, 1, 2, 3, 4, …, 999, then a sequential search will find the value 674 in exactly 674 iterations. However, with a jump search, the same value can be found in 44 iterations. Hence, a jump search performs far better than a linear search on a sorted list of elements.

Advantages of jump search over binary search

Binary search is very easy to implement and has a complexity of O(log n), but in the case of a list having a very large number of elements, jumping to the middle of the list to make comparisons is not a good idea because if the value being searched is at the beginning of the list, then one (or even more) large step(s) in the backward direction would have to be taken. In such cases, jump search performs better as we have to move a little backward, that too only once. Hence, when jumping back is slower than jumping forward, the jump search algorithm always performs better.

Choosing the step length

For the jump search algorithm to work efficiently, we must define a fixed size for the step. If the step size is 1, then the algorithm is the same as a linear search. Now, in order to find an appropriate step size, we must first try to figure out the relation between the size of the list (n) and the size of the step (k). Usually, k is calculated as √n.

Further optimization of jump search

Till now, we were dealing with lists having a small number of elements. But in real-world applications, lists can be very large. In such large lists, searching for the value from the beginning of the list may not be a good idea. A better option is to start the search from the kth element.

We can also improve the performance of the jump search algorithm by repeatedly applying jump search. For example, if the size of the list is 10,00,000 (n). The jump interval would then be √n = √1000000 = 1000. Now, even the identified interval has 1000 elements and is again a large list. So, jump search can be applied again with a new step size of √1000 ≈ 31. Thus, every time we have a desired interval with a large number of values, the jump search algorithm can be applied again but with a smaller step. However, in this case, the complexity of the algorithm will no longer be O(√n) but will approach a logarithmic value.

Program 4.4: Implementation of jump search

#include <stdio.h>

#include<math.h>

#include <conio.h>

#define MAX 20

int jump_search(int a[], int low, int high, int val, int n)

{

int step, i;

step = sqrt(n);

for(i=0;i<step;i++)

{

if(val < a[step])

high = step – 1;

else

low = step + 1;

}

for(i=low;i<=high;i++)

{

if(a[i]==val)

return i;

}

return –1;

}

int main()

{

int arr[MAX], i, n, val, pos;

clrscr();

printf("\n Enter the number of elements in the array : ");

scanf("%d", &n);

printf("\n Enter the elements : ");

for(i = 0; i <n; i++)

scanf("%d", &arr[i]);

printf("\n Enter the value to be searched : ");

scanf("%d", &val);

pos = jump_search(arr, 0, n–1, val, n);

if(pos == –1)

printf("\n %d is not found in the array", val);

else

printf("\n %d is found at position %d", val, pos);

getche();

return 0;

}

Complexity of jump search algorithm

Jump search is another searching algorithm that operates on sorted arrays. Its complexity is as follows:

• Time complexity: The following is the complexity analysis:

o Worst-case scenario: The worst-case scenario for jump search also occurs when the target element is not present in the array. In this case, the algorithm will perform a linear search through each block until it reaches the end of the array or finds a block where the target element could potentially exist. The number of comparisons in the worst-case scenario is also approximately √n, resulting in a time complexity of O(√n).

o Best-case scenario: In the best-case scenario, the target element is found at the first position itself. This results in a time complexity of O(1) as the algorithm only requires one comparison.

o Average-case scenario: In an average-case scenario, the jump search algorithm divides the array into blocks of size √n, where n is the number of elements in the array. It then performs a linear search within each block to find the block containing the target element. Once the block is found, it performs a linear search within that block. On average, this results in approximately √n comparisons. Hence, the average-case time complexity is O(√n).

• Space complexity: The space complexity of the jump search algorithm is O(1), meaning it requires constant space regardless of the size of the input array.

Exponential search

Exponential search is a searching algorithm that aims to find the position of a target value within a sorted array. It works by first finding a range where the target element may lie using exponential increments, and then performing a binary search within that range. Exponential search is particularly useful when the array size is unknown or unbounded because it quickly narrows down the search space to a manageable size using exponentially increasing indices.

Phases of exponential search

The first phase finds the portion of x that contains z if it is present in the array. More specifically, the first phase finds the number j such that x[2^j] <= z < x[2^{j+1}].

The idea is to partition x into sub-arrays of incremental sizes and check them consecutively until we find the one that can contain z. The boundary indices of those sub-arrays are powers of 2, hence the algorithm’s name.

In the second phase, the algorithm looks for {z} using binary search in the following array:

{x[2^j : (2^{j+1}-1)] = [x[2^j], x[2^j + 1], x[2^j + 2], . . ...., x[2^{j+1} - 1]]}

We first check if x[1] <= z < x[2], which simplifies to testing if x[1] = z. If the test fails, we check if x[2] <= z < x[4]. If that is not the case, we test if x[4] <= z < x[8], then x[8] <= z < x[16], x[16] <= z < x[32], and so on until we find the boundaries that capture z: x[2^j] <= z < x[2^{j+1}].

Afterward, we run binary search on x[2^j : (2^{j+1}-1)]. If it does not find z, then it is not an element of x. If it does, it will return the index of z in the sub-array as an integer from 1 to 2^j. Adjust it to get the index of z in the entire array by adding the offset 2^j (the starting index of the sub-array) and taking away 1. The following figure shows the starting index of the sub-array:

Figure 4.1: Initialization of array with sub-array starting index

Algorithm 4.5: Algorithm of exponential search

int Exponential_Search (int arr[], int n, int x) {

if (arr[0] == x) return 0;

int i = 1;

while (i < n && arr[i] <= x) i = i * 2;

return Binary_Search (arr, i / 2, min(i, n - 1), x);

}

int Binary_Search (int arr[], int l, int r, int x)

{

if (r >= l)

{

int mid = l + (r - l)/2;

if (arr[mid] == x)

return mid;

if (arr[mid] > x)

return Binary_Search (arr, l, mid-1, x);

return Binary_Search (arr, mid+1, r, x);

}

return -1;

}

Program 4.5: Implementation of exponential search

#include <stdio.h>

#include <conio.h>

// Function to perform binary search within a sorted array

int binarySearch(int arr[], int left, int right, int target) {

while (left <= right) {

int mid = left + (right - left) / 2;

if (arr[mid] == target)

return mid;

else if (arr[mid] < target)

left = mid + 1;

else

right = mid - 1;

}

return -1; // Return -1 if the target is not found

}

// Function to perform exponential search

int exponentialSearch(int arr[], int size, int target) {

if (arr[0] == target)

return 0; // If the target is found at index 0

// Find the range where the target may lie

int i = 1;

while (i < size && arr[i] <= target)

i *= 2;

// Perform binary search within the identified range

return binarySearch(arr, i / 2, (i < size) ? i : size - 1, target);

}

int main() {

int arr[] = {2, 3, 5, 8, 10, 15, 18};

int size = sizeof(arr) / sizeof(arr[0]);

int target = 10;

int index = exponentialSearch(arr, size, target);

if (index != -1)

printf("Element %d found at index %d\n", target, index);

else

printf("Element %d not found in the array\n", target);

return 0;

}

Complexity of exponential search algorithm

The time complexity of an exponential search can be analyzed in two steps, which are as follows:

1. Finding the range where the target may lie: This step involves iterating through indices exponentially until either the element at the current index exceeds the target value or the index exceeds the size of the array. Since the index is multiplied by 2 in each iteration, the number of iterations required to find the range is O(log n), where n is the size of the array.

2. Performing binary search within the identified range: After finding the range where the target may lie, a binary search is performed within that range. The time complexity of binary search is O(log m), where m is the size of the range (which can be, at most, n elements).

Overall, the time complexity of exponential search is the sum of the time complexities of the preceding two steps, which is as follows:

O(log n) (for finding the range) + O(log m) (for binary search within the range)

Since m can be at most n, the overall time complexity of exponential search can be expressed as O(log n) + O(log n), which simplifies to O(log n).

Fibonacci search

In the Fibonacci series, the first two terms are 0 and 1, and then each successive term is the sum of the previous two terms. In the following Fibonacci series, each number is called a Fibonacci number:

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, …

The same series and concept can be used to search for a given value in a list of numbers. Such a search algorithm, which is based on Fibonacci numbers, is called Fibonacci search and was developed by Kiefer in 1953. The search follows a divide and conquer technique and narrows down possible locations with the help of Fibonacci numbers. Similar to jump search and exponential search, this algorithm does search by skipping through the indices of the input array.

Algorithm 4.6: Algorithm of Fibonacci search

1. Find the smallest Fibonacci numbers, i.e., greater than or equal to the length of the array.

2. Initialize two pointers: left and right.

a. Set left to 0.

b. Set right to the smallest Fibonacci number greater than or equal to the length of the array.

3. While the target value is not found and the left is less than the length of the array:

a. If the target value is greater than the value at index right, update left and right to move fib_(m-1) steps ahead.

b. If the target value is less than the value at index right, update left and right to move fib_(m-2) steps ahead.

c. If the target value is equal to the value at index right, return right.

d. Update m to m - 1 and recalculate the Fibonacci numbers accordingly.

4. If the target value is found, return the index of the target value.

5. If the target value is not found, return -1.

Example 4.2: Imagine we need to find element 24 in a sorted array of {12, 14, 16, 17, 20, 24, 31, 43, 50, 62}, shown in Figure 4.2, using Fibonacci search:

Figure 4.2: Initialization of array in Fibonacci search

1. The size of the input array is 10. The smallest Fibonacci number greater than 10 is 13. Therefore, Fm = 13, Fm-1 = 8, Fm-2 = 5.

We initialize offset = -1.

2. In the first iteration, compare it with the element at index = minimum (offset + Fm-2, n – 1) = minimum (-1 + 5, 9) = minimum (4, 9) = 4.

The fourth element in the array is 20, which is not a match and is less than the key element 24, shown as follows:

Figure 4.3: First iteration compares at index 4

3. In the second iteration, update the offset value and the Fibonacci numbers.

Since the key is greater, the offset value will become the index of the element, i.e., 4. Fibonacci numbers are updated as Fm = Fm-1 = 8.

Fm-1 = 5, Fm-2 = 3.

Now, compare it with the element at index = minimum (offset + Fm-2, n – 1) = minimum (4 + 3, 9) = minimum (7, 9) = 7.

The element at seventh index of the array is 43, which is not a match and is also more than the key element 24, shown as follows:

Figure 4.4: First iteration compares at index 7

4. We discard the elements after the seventh index, so n = 7, and the offset value remains 4.

Fibonacci numbers are pushed two steps backward, i.e., Fm = Fm-2 = 3.

Fm-1 = 2, Fm-2 = 1.

Now, compare it with the element at index = minimum (offset + Fm-2, n – 1) = minimum (4 + 1, 6) = minimum (5, 7) = 5.

The element at the fifth index in the array is 24, which is our key element. The fifth index is returned as the output for this example array, shown as follows:

Figure 4.5: Output returned at index 5

The output is index 5.

Program 4.6: Implementation of Fibonacci search

#include <stdio.h>

int min(int, int);

int fibonacci_search(int[], int, int);

int min(int a, int b){

return (a > b) ? b : a;

}

int fibonacci_search(int arr[], int n, int key){

int offset = -1;

int Fm2 = 0;

int Fm1 = 1;

int Fm = Fm2 + Fm1;

while (Fm < n) {

Fm2 = Fm1;

Fm1 = Fm;

Fm = Fm2 + Fm1;

}

while (Fm > 1) {

int i = min(offset + Fm2, n - 1);

if (arr[i] < key) {

Fm = Fm1;

Fm1 = Fm2;

Fm2 = Fm - Fm1;

offset = i;

} else if (arr[i] > key) {

Fm = Fm2;

Fm1 = Fm1 - Fm2;

Fm2 = Fm - Fm1;

} else

return i;

}

if (Fm1 && arr[offset + 1] == key)

return offset + 1;

return -1;

}

int main(){

int i, n, key, pos;

int arr[10] = {6, 11, 19, 24, 33, 54, 67, 81, 94, 99};

printf("Array elements are: ");

int len = sizeof(arr) / sizeof(arr[0]);

for(int j = 0; j<len; j++){

printf(“%d “, arr[j]);

}

n = 10;

key = 67;

printf("\nThe element to be searched: %d", key);

pos = fibonacci_search(arr, n, key);

if(pos >= 0)

printf("\nThe element is found at index %d", pos);

else

printf("\nUnsuccessful Search");

}

Complexity of Fibonacci search algorithm

The time complexity of the Fibonacci search algorithm remains O(log n) for all cases, including the best, worst, and average cases. This is because the algorithm's efficiency is determined by the logarithmic reduction of the search space at each step, which is consistent across different scenarios.

The space complexity of the Fibonacci search algorithm is O(1), which means it requires constant amount of additional space, regardless of the size of the input array.

Sublist search

One way to find a linked list within another linked list is through the sublist search algorithm. The goal is to find out if one list is in the other list, just like any basic pattern matching method. The algorithm iterates through the linked list, comparing the first element of one list with the first element of the second list. If a match is not found, it then compares the second element of the first list with the first element of the second list. This process persists until a match is encountered or it reaches the conclusion of a list.

As an illustration, let us examine two linked lists containing the following values: {14, 16, 17, 13, 18, 12} and {13, 18, 12}. Sublist search determines if the elements of the second list exist within the first linked list. The output is obtained as a binary result, represented by the Boolean values of either true or false. Due to the fact that a linked list is not an ordered data structure, it is unable to provide the position of the sub-list.

The output is true only if the second linked list is present in the first list in the exact same order, shown as follows:

Figure 4.6: Output of sub-list 13, 18, 12

Algorithm 4.7: Algorithm of sublist search

The primary objective of this approach is to demonstrate that one linked list is a subset of another list. The search procedure in this algorithm is performed sequentially, examining each element of the linked list individually. If the result is true, it confirms that the second list is a sublist of the first linked list.

1. Utilize two pointers, with each pointer indicating a separate list. These pointers are utilized for navigating around the linked lists.

2. Examine the base cases of the linked lists as follows:

a. If both linked lists are empty, then return TRUE.

b. If the second list is not empty but the first list is empty, return FALSE.

c. If the first list is not empty but the second list is empty, return FALSE.

3. After confirming that both lists are not empty, employ the pointers to sequentially navigate through the elements in the lists.

4. Compare the initial element of the first linked list with the initial element of the second linked list. If they are identical, update the pointers to point to the subsequent values in both lists.

5. If there is no match, maintain the pointer in the second list at the first entry, but advance the pointer in the first list. Reevaluate the components once more.

6. Repeat Steps 4 and 5 until we reach the end of the lists.

7. If the final status is found, TRUE is returned and if not, FALSE.

Program 4.7: Implementation of sublist search

#include <stdio.h>

#include <stdlib.h>

#include <stdbool.h>

struct Node {

int data;

struct Node* next;

};

struct Node *newNode(int key){

struct Node *val = (struct Node*)malloc(sizeof(struct Node));;

val-> data= key;

val->next = NULL;

return val;

}

bool sublist_search(struct Node* list_ptr, struct Node* sub_ptr){

struct Node* ptr1 = list_ptr, *ptr2 = sub_ptr;

if (list_ptr == NULL && sub_ptr == NULL)

return true;

if ( sub_ptr == NULL || (sub_ptr != NULL && list_ptr == NULL))

return false;

while (list_ptr != NULL) {

ptr1 = list_ptr;

while (ptr2 != NULL) {

if (ptr1 == NULL)

return false;

else if (ptr2->data == ptr1->data) {

ptr2 = ptr2->next;

ptr1 = ptr1->next;

} else

break;

}

if (ptr2 == NULL)

return true;

ptr2 = sub_ptr;

list_ptr = list_ptr->next;

}

return false;

}

int main(){

struct Node *list = newNode(2);

list->next = newNode(5);

list->next->next = newNode(3);

list->next->next->next = newNode(3);

list->next->next->next->next = newNode(6);

list->next->next->next->next->next = newNode(7);

list->next->next->next->next->next->next = newNode(0);

struct Node *sub_list = newNode(3);

sub_list->next = newNode(6);

sub_list->next->next = newNode(7);

bool res = sublist_search(list, sub_list);

if (res)

printf("Is the sublist present in the list? %d" , res);

}

Complexity of sublist search algorithm

The time complexity of the sublist search is contingent upon the quantity of elements contained in the linked lists being compared. The algorithm has a worst-case time complexity of O(m*n), where m represents the number of entries in the first linked list and n represents the number of elements in the second linked list.

Conclusion

In this chapter, we discussed various searching algorithms. We also identified the time complexities of different cases and found the space complexity. So, the programmer can use any algorithm to find the element in the given array list. The list may be either in sorted or unsorted order. According to the requirement, the user can apply a suitable algorithm to find the element with its exact location.

In the next chapter, readers will explore the divide and conquer method, which represents a strong algorithmic framework that resolves difficulties using small modular problems. The algorithm operates through three fundamental steps, which include splitting the problem into areas, along with the recursive execution of these parts to deliver the outcome. Studying merge sort alongside quick sort and binary search within this chapter will offer students comprehensive skills to create efficient recursive solutions to boost system performance scalability for many computational issues.

Exercise

1. We are given an array list having elements {22, 45, 34, 67, 54, 88, 73, 12, 33}. Search element 88 in the array list using linear search method and return the index value where 88 is found.

2. In the previous question, consider data elements, sort the elements, and find the index of value 54 using the binary search method.

3. We are given a list of numbers, Arr[] = {15, 18, 21, 27, 39, 41, 43, 55, 57, 59}. Search for value 41 using the interpolation search technique.

4. Consider the data element from the previous question and search element 43 using jump search and exponential search and return the status.

5. What do you mean by Fibonacci search? How do we search and element in Fibonacci search?

6. Write a short note on sublist search and also mention the time complexity of this algorithm.

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 5Divide and Conquer

Introduction

This chapter will introduce the divide and conquer concept within the field of computer science. Divide and conquer is an algorithmic paradigm in which a problem is dissected into smaller, more easily handled sub-problems. These sub-problems are then solved recursively, and their solutions are amalgamated to resolve the initial problem. Widely applied in computer science and mathematics, this approach aids in crafting efficient algorithms for diverse problem sets. The typical divide and conquer strategy involves three primary steps: divide, conquer, and combine. The initial problem is partitioned into more manageable subproblems, a critical step in simplifying the overall complexity by breaking it down into simpler components. In the next step, every subproblem is addressed recursively, employing the divide and conquer strategy until they reach a level of simplicity where direct solutions are feasible. Recursive solutions are commonly implemented through function calls in this process. In the final step, the resolutions of the subproblems are subsequently consolidated to derive the solution to the original problem. This step entails combining or merging the outcomes from the smaller subproblems into a comprehensive solution for the overarching problem.

The divide and conquer methodology proves highly efficient in addressing problems characterized by overlapping subproblems and optimal substructure. Overlapping subproblems refer to the recurrence of solving the same subproblems multiple times, while optimal substructure indicates that the optimal solution to the original problem can be constructed from the optimal solutions of its subproblems.

Structure

The chapter covers the following topics:

• The general method

• Binary search

• Find maximum and minimum in number sequence

• Merge sort

• Quick sort

• Strassen’s matrix multiplication

• Karatsuba multiplication

• Closest pair of points

• Maximum subarray sum

• Merge-based inversion count

Objectives

The chapter provides readers with extensive knowledge about divide and conquer, along with its vital role in designing algorithms. Students will master the process of reducing complex issues into smaller parts that they solve recursively in order to achieve efficient computation. The section presents several advantages, which include superior performance along with interchangeable solutions, as well as well-maintained code and simultaneous execution capabilities. Throughout the chapter, readers will examine practical algorithm implementations in different subject domains while developing more effective problem-solving abilities. Students will develop the ability to create efficient, modular, and scalable algorithms through the divide and conquer methodology.

In essence, the aim of the divide and conquer approach is to effectively address intricate problems by dissecting them into smaller, more manageable components. These components are addressed independently, and their solutions are then integrated to attain the resolution for the original problem.

The general method

The divide and conquer method adhere to a standard pattern, encompassing three fundamental steps: divide, conquer, and combine. A more comprehensive delineation of the overarching divide and conquer algorithmic approach is as follows:

• Divide

o Problem segmentation: Fragment the primary problem into smaller, more digestible subproblems, entailing the division of the input data or problem instance into two or more smaller components.

o Recursion application: Employ the divide and conquer strategy iteratively on each of the subproblems. This entails resolving the subproblems by reiterating the division step until they attain a level of simplicity suitable for direct solutions.

• Conquer

o Subproblem resolution: Address the subproblems produced during the divide step. Frequently, this involves employing the same divide and conquer approach, iteratively applying the algorithm to the subproblems until reaching base cases.

o Base case definition: Establish base cases that denote the smallest instances of the problem capable of being solved without additional division. These base cases act as termination conditions for the recursive process.

• Combine

o Solution integration: Merge the solutions of the subproblems to derive the solution for the original problem. This encompasses consolidating or combining the outcomes obtained from the conquer step.

o Final result output: Present the ultimate solution to the original problem derived from the amalgamated solutions of the subproblems.

This general method is systematically applied until the base cases are attained. Upon reaching the base cases, the algorithm computes and directly returns the solution without further subdivision. The efficacy of divide and conquer algorithms often hinges on minimizing redundant computations and leveraging the solutions of subproblems to prevent unnecessary recalculations.

It is crucial to emphasize that the success of the divide and conquer approach is contingent on the problem manifesting two key properties: overlapping subproblems and optimal substructure. Overlapping subproblems indicate the recurrence of solving the same subproblems multiple times, while optimal substructure signifies that the optimal solution to the original problem can be constructed from the optimal solutions of its subproblems. These properties facilitate the efficient reuse of solutions and contribute to the effectiveness of the divide and conquer strategy.

The process of the divide and conquer approach is shown in the following figure:

Figure 5.1: Process of divide and conquer approach

Binary search

Binary search stands as a timeless algorithm employed for locating an element in a sorted array or list. Employing the divide and conquer strategy, it exhibits high efficiency, boasting a time complexity of O(log n). The fundamental concept involves iteratively halving the search space until either the target element is discovered or the search space becomes empty.

Algorithm 5.1: Binary search

binarySearch(arr, target)

low = 0

high = length(arr) - 1

while low <= high

mid = (low + high) / 2

if arr[mid] equals target

return mid // Target found at index mid

else if arr[mid] < target

low = mid + 1 // Search the right half

else

high = mid - 1 // Search the left half

return -1 // Target not found

Program 5.1: Implementation in C++

#include <iostream>

#include <vector>

int binarySearch(const std::vector<int>& arr, int target) {

int low = 0;

int high = arr.size() - 1;

while (low <= high) {

int mid = low + (high - low) / 2;

if (arr[mid] == target) {

return mid; // Target found

} else if (arr[mid] < target) {

low = mid + 1; // Search the right half

} else {

high = mid - 1; // Search the left half

}

}

return -1; // Target not found

}

int main() {

std::vector<int> arr = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

int target = 5;

int result = binarySearch(arr, target);

if (result != -1) {

std::cout << "Element found at index " << result << std::endl;

} else {

std::cout << "Element not found" << std::endl;

}

return 0;

}

Example 5.1: Consider the sorted array {1, 3, 5, 7, 9, 11, 13, 15}. Find the index of the target element 9.

Solution: Let us find the index of the target element 9 using binary search:

• Initial: low = 0, high = 7

• Iteration 1: mid = 3, arr[mid] = 7 < 9, so update low = mid + 1

• Iteration 2: mid = 5, arr[mid] = 11 > 9, so update high = mid - 1

• Iteration 3: mid = 4, arr[mid] = 9, target found at index 4

Complexity analysis

The complexity analysis is as follows:

• Time complexity: It is calculated as O(log n). Binary search reduces the search space by half in each iteration.

• Space complexity: It is calculated as O(1). Binary search utilizes a constant amount of additional space.

Applications

Binary search, renowned for its efficiency in locating elements within sorted datasets, finds application across various fields and domains. Some prevalent areas where binary search is employed encompass the following:

• Computer science and programming: Binary search is extensively used in computer science and programming for searching elements in arrays, lists, or other sorted data structures. It serves as a fundamental algorithm taught in introductory computer science courses.

• Databases: In databases, binary search is a common technique employed to search for records based on indexed columns, facilitating swift location and retrieval of specific data from extensive datasets.

• Information retrieval: Information retrieval systems utilize binary search to efficiently search and locate documents or records that match specific criteria.

• Sorting algorithms: Binary search is integral to various sorting algorithms. For instance, in merge sort or quick sort, it is employed to divide the dataset into smaller parts.

• Networking: Applied in networking protocols and routing algorithms, binary search efficiently locates and routes information through a network.

• Game development: In game development, binary search is employed for tasks such as searching for items in a sorted list, finding positions, or optimizing decision-making processes.

• Genomics and bioinformatics: In genomics and bioinformatics, binary search is utilized for tasks like searching and comparing DNA sequences or identifying patterns in extensive datasets.

• Mathematics: Binary search finds applications in mathematical tasks, including finding roots of equations or optimizing mathematical functions.

• Economics and finance: In finance and economics, binary search can be employed for tasks such as searching for specific financial data or optimizing investment portfolios.

• Robotics: In robotics, binary search is applied for efficient path planning and navigation algorithms.

Binary search stands as a foundational algorithm, laying the groundwork for more advanced algorithms and data structures.

Find maximum and minimum in number sequence

The task of finding the maximum and minimum elements in a sequence of numbers can be efficiently solved using a divide and conquer approach. This technique involves breaking down the problem into smaller subproblems, solving them independently, and then combining the results to obtain the final solution.

Algorithm 5.2: Find maximum and minimum

findMinMax(arr, low, high)

if low equals high

// Only one element in the array

return {arr[low], arr[low]}

else if high equals low + 1

// Two elements in the array

return {min(arr[low], arr[high]), max(arr[low], arr[high])}

else

mid = (low + high) / 2

// Recursively find min and max in the left half

leftResult = findMinMax(arr, low, mid)

// Recursively find min and max in the right half

rightResult = findMinMax(arr, mid + 1, high)

// Combine the results

return {min(leftResult.min, rightResult.min), max(leftResult.max, rightResult.max)}

Program 5.2: Implementation in C++

#include <iostream>

#include <vector>

struct MinMax {

int min;

int max;

};

MinMax findMinMax(const std::vector<int>& arr, int low, int high) {

MinMax result;

// Base case: If the sequence has only one element

if (low == high) {

result.min = result.max = arr[low];

return result;

}

// If the sequence has more than one element, divide and conquer

int mid = (low + high) / 2;

MinMax leftResult = findMinMax(arr, low, mid);

MinMax rightResult = findMinMax(arr, mid + 1, high);

// Combine the results

result.min = std::min(leftResult.min, rightResult.min);

result.max = std::max(leftResult.max, rightResult.max);

return result;

}

int main() {

std::vector<int> arr = {3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5};

int n = arr.size();

MinMax result = findMinMax(arr, 0, n - 1);

std::cout << "Minimum element: " << result.min << std::endl;

std::cout << "Maximum element: " << result.max << std::endl;

return 0;

}

Example 5.2: Consider the sequence {3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5}.

Solution: The algorithm divides the sequence into two halves, recursively finds the minimum and maximum in each half, and then combines the results. The final output will be the minimum element (1) and the maximum element (9) in the sequence.

Complexity analysis

The complexity analysis is performed as follows:

• Time complexity: It is depicted as O(n), where n is the number of elements in the sequence. Each element is examined once during the divide and conquer process.

• Space complexity: It is depicted as O(log n) due to the recursive call stack.

Applications

This divide and conquer approach for finding the maximum and minimum is often used in algorithms where the goal is to efficiently search for both the maximum and minimum values simultaneously. It is employed in various applications, such as computational geometry, image processing, and optimization algorithms where identifying extreme values is a common requirement. Additionally, it serves as a building block in more complex algorithms and data structures.

Merge sort

Merge sort is a popular sorting algorithm that uses the divide and conquer paradigm. It divides the input array into two halves, recursively sorts each half, and then merges the sorted halves to produce a fully sorted array. Merge sort is known for its stability and guarantees O(n log n) time complexity, making it efficient for large datasets.

Algorithm 5.3: Merge sort

mergeSort(arr, low, high)

if low < high

mid = (low + high) / 2

mergeSort(arr, low, mid)

mergeSort(arr, mid + 1, high)

merge(arr, low, mid, high)

merge(arr, low, mid, high)

n1 = mid - low + 1

n2 = high - mid

// Create temporary arrays

leftArray[1...n1], rightArray[1...n2]

// Copy data to temporary arrays leftArray[] and rightArray[]

for i = 1 to n1

leftArray[i] = arr[low + i - 1]

for j = 1 to n2

rightArray[j] = arr[mid + j]

// Merge the temporary arrays back into arr[low...high]

i = 1

j = 1

k = low

while i <= n1 and j <= n2

if leftArray[i] <= rightArray[j]

arr[k] = leftArray[i]

i++

else

arr[k] = rightArray[j]

j++

k++

// Copy the remaining elements of leftArray[], if there are any

while i <= n1

arr[k] = leftArray[i]

i++

k++

// Copy the remaining elements of rightArray[], if there are any

while j <= n2

arr[k] = rightArray[j]

j++

k++

Program 5.3: Implementation in C++

#include <iostream>

#include <vector>

void merge(std::vector<int>& arr, int left, int mid, int right) {

int n1 = mid - left + 1;

int n2 = right - mid;

// Create temporary arrays

std::vector<int> L(n1), R(n2);

// Copy data to temporary arrays L[] and R[]

for (int i = 0; i < n1; i++)

L[i] = arr[left + i];

for (int j = 0; j < n2; j++)

R[j] = arr[mid + 1 + j];

// Merge the temporary arrays back into arr[left..right]

int i = 0, j = 0, k = left;

while (i < n1 && j < n2) {

if (L[i] <= R[j]) {

arr[k] = L[i];

i++;

} else {

arr[k] = R[j];

j++;

}

k++;

}

// Copy the remaining elements of L[], if there are any

while (i < n1) {

arr[k] = L[i];

i++;

k++;

}

// Copy the remaining elements of R[], if there are any

while (j < n2) {

arr[k] = R[j];

j++;

k++;

}

}

void mergeSort(std::vector<int>& arr, int left, int right) {

if (left < right) {

// Same as (left+right)/2, but avoids overflow

int mid = left + (right - left) / 2;

// Sort first and second halves

mergeSort(arr, left, mid);

mergeSort(arr, mid + 1, right);

// Merge the sorted halves

merge(arr, left, mid, right);

}

}

int main() {

std::vector<int> arr = {38, 27, 43, 3, 9, 82, 10};

int n = arr.size();

std::cout << "Original array: ";

for (int num : arr) {

std::cout << num << " ";

}

std::cout << std::endl;

mergeSort(arr, 0, n - 1);

std::cout << "Sorted array: ";

for (int num : arr) {

std::cout << num << " ";

}

std::cout << std::endl;

return 0;

}

Example 5.3: Consider the array {38, 27, 43, 3, 9, 82, 10}.

Solution: The solution includes the following steps:

1. Initial state: The array is divided into two halves, and the mergeSort function is called recursively on each half.

2. Recursive calls: The array is further divided until individual elements are reached.

3. Merge steps: The merging process begins. The subarrays {38} and {27} are merged to form {27, 38}. Similarly, {43} and {3} are merged to form {3, 43}, and so on.

4. Final merge: The two halves {3, 27, 38, 43} and {9, 10, 82} are merged to produce the fully sorted array {3, 9, 10, 27, 38, 43, 82}.

Complexity analysis

The complexity analysis is as follows:

• Time complexity: It is calculated as O(n log n). The array is repeatedly divided into halves log n times, and each level requires linear time for merging.

• Space complexity: It is calculated as O(n). Additional space is required for creating temporary arrays during the merging process.

Applications

Merge sort is widely used in scenarios where stability and predictable performance are important. The following list includes some common use cases:

• External sorting: Merge sort is efficient for sorting large datasets that do not fit into memory because it minimizes the number of comparisons needed.

• Sorting linked lists: Merge sort's divide and conquer approach is suitable for sorting linked lists, where random access is not as efficient as with arrays.

• Parallel computing: Merge sort can be parallelized, making it suitable for parallel computing environments.

• Inversion counting: Merge sort can be modified to count the number of inversions in an array, which has applications in areas like finance and data analysis.

Quick sort

Quick sort is a widely used sorting algorithm that employs the divide and conquer paradigm. It works by selecting a pivot element from the array and partitioning the other elements into two sub-arrays according to whether they are less than or greater than the pivot. The sub-arrays are then recursively sorted. Quick sort is known for its efficiency and often outperforms other sorting algorithms, especially for large datasets.

Algorithm 5.4: Quick sort

quickSort(arr, low, high)

if low < high

pi = partition(arr, low, high)

quickSort(arr, low, pi - 1)

quickSort(arr, pi + 1, high)

partition(arr, low, high)

pivot = arr[high]

i = low - 1

for j = low to high - 1

if arr[j] < pivot

i++

swap(arr[i], arr[j])

swap(arr[i + 1], arr[high])

return i + 1

Program 5.4: Implementation in C++

#include <iostream>

#include <vector>

// Function to partition the array and return the pivot index

int partition(std::vector<int>& arr, int low, int high) {

int pivot = arr[high];

int i = low - 1;

for (int j = low; j < high; j++) {

if (arr[j] < pivot) {

i++;

std::swap(arr[i], arr[j]);

}

}

std::swap(arr[i + 1], arr[high]);

return i + 1;

}

// Function to perform Quick Sort

void quickSort(std::vector<int>& arr, int low, int high) {

if (low < high) {

int pi = partition(arr, low, high);

// Recursively sort the sub-arrays

quickSort(arr, low, pi - 1);

quickSort(arr, pi + 1, high);

}

}

int main() {

std::vector<int> arr = {38, 27, 43, 3, 9, 82, 10};

int n = arr.size();

std::cout << "Original array: ";

for (int num : arr) {

std::cout << num << " ";

}

std::cout << std::endl;

quickSort(arr, 0, n - 1);

std::cout << "Sorted array: ";

for (int num : arr) {

std::cout << num << " ";

}

std::cout << std::endl;

return 0;

}

Example 5.4: Consider the array {38, 27, 43, 3, 9, 82, 10}.

Solution: The following steps are involved:

1. Initial state: The function quickSort is called with the entire array. A pivot is selected, and the partition function is called to rearrange elements such that elements smaller than the pivot are on the left, and elements greater than the pivot are on the right.

2. Recursive calls: The array is divided into two sub-arrays based on the pivot. The quickSort function is recursively called on the left and right sub-arrays.

3. Final result: The process continues until each sub-array has one or zero elements. The final result is a fully sorted array.

Complexity analysis

The complexity analysis is as follows:

1. Time complexity: Time complexity is calculated based on the following cases:

o Best case: O(n log n)

o Average case: O(n log n)

o Worst case: O(n^2)

The pivot selection and partitioning steps dominate the time complexity. In the average case, the pivot is often close to the median, leading to efficient partitioning.

• Space complexity: It is calculated as O(log n) and is the space required for the recursive call stack.

Applications

Quick sort is widely used in practice due to its efficiency and is the default sorting algorithm in many programming languages and libraries. Some common use cases include:

• General-purpose sorting: Quick sort is used for sorting large datasets efficiently and is often preferred over other sorting algorithms.

• System libraries: Many system libraries and programming languages use quick sort as their default sorting algorithm.

• Data deduplication: Quick sort's in-place nature makes it suitable for tasks like data deduplication, where rearranging elements is acceptable.

• Sorting in database systems: Quick sort is used in sorting records and indexes in database systems.

• Parallel computing: Quick sort can be parallelized, making it suitable for parallel computing environments.

Despite the worst-case time complexity, quick sort is often faster in practice than other algorithms, making it a popular choice for sorting applications.

Strassen’s matrix multiplication

Strassen's algorithm is an efficient divide and conquer method for matrix multiplication. It reduces the number of multiplications from 8 to 7 by using a set of recursive formulas. This reduction in the number of multiplications leads to a more efficient algorithm, especially for large matrices. Strassen's algorithm is based on the idea of dividing each matrix into submatrices and applying a set of mathematical formulas to compute the product.

Algorithm 5.5: Strassen’s matrix multiplication

strassenMatrixMultiply(A, B)

if size of A is 1

return A * B // Standard matrix multiplication for 1x1 matrices

// Divide matrices into submatrices

A11, A12, A21, A22 = submatrices of A

B11, B12, B21, B22 = submatrices of B

// Compute products of submatrices using Strassen's formulas

P1 = strassenMatrixMultiply(A11 + A22, B11 + B22)

P2 = strassenMatrixMultiply(A21 + A22, B11)

P3 = strassenMatrixMultiply(A11, B12 - B22)

P4 = strassenMatrixMultiply(A22, B21 - B11)

P5 = strassenMatrixMultiply(A11 + A12, B22)

P6 = strassenMatrixMultiply(A21 - A11, B11 + B12)

P7 = strassenMatrixMultiply(A12 - A22, B21 + B22)

// Compute submatrices of the result

C11 = P1 + P4 - P5 + P7

C12 = P3 + P5

C21 = P2 + P4

C22 = P1 - P2 + P3 + P6

// Construct the result matrix

result = constructMatrix(C11, C12, C21, C22)

return result

Program 5.5: Implementation

#include <iostream>

#include <vector>

using namespace std;

vector<vector<int>> addMatrices(const vector<vector<int>>& A, const vector<vector<int>>& B) {

int n = A.size();

vector<vector<int>> result(n, vector<int>(n, 0));

for (int i = 0; i < n; i++) {

for (int j = 0; j < n; j++) {

result[i][j] = A[i][j] + B[i][j];

}

}

return result;

}

vector<vector<int>> subtractMatrices(const vector<vector<int>>& A, const vector<vector<int>>& B) {

int n = A.size();

vector<vector<int>> result(n, vector<int>(n, 0));

for (int i = 0; i < n; i++) {

for (int j = 0; j < n; j++) {

result[i][j] = A[i][j] - B[i][j];

}

}

return result;

}

vector<vector<int>> strassenMatrixMultiply(const vector<vector<int>>& A, const vector<vector<int>>& B) {

int n = A.size();

if (n == 1) {

// Base case: standard matrix multiplication for 1x1 matrices

vector<vector<int>> result(1, vector<int>(1, 0));

result[0][0] = A[0][0] * B[0][0];

return result;

}

// Divide matrices into submatrices

int mid = n / 2;

vector<vector<int>> A11(mid, vector<int>(mid)), A12(mid, vector<int>(mid)), A21(mid, vector<int>(mid)), A22(mid, vector<int>(mid));

vector<vector<int>> B11(mid, vector<int>(mid)), B12(mid, vector<int>(mid)), B21(mid, vector<int>(mid)), B22(mid, vector<int>(mid));

for (int i = 0; i < mid; i++) {

for (int j = 0; j < mid; j++) {

A11[i][j] = A[i][j];

A12[i][j] = A[i][j + mid];

A21[i][j] = A[i + mid][j];

A22[i][j] = A[i + mid][j + mid];

B11[i][j] = B[i][j];

B12[i][j] = B[i][j + mid];

B21[i][j] = B[i + mid][j];

B22[i][j] = B[i + mid][j + mid];

}

}

// Compute products of submatrices using Strassen's formulas

vector<vector<int>> P1 = strassenMatrixMultiply(addMatrices(A11, A22), addMatrices(B11, B22));

vector<vector<int>> P2 = strassenMatrixMultiply(addMatrices(A21, A22), B11);

vector<vector<int>> P3 = strassenMatrixMultiply(A11, subtractMatrices(B12, B22));

vector<vector<int>> P4 = strassenMatrixMultiply(A22, subtractMatrices(B21, B11));

vector<vector<int>> P5 = strassenMatrixMultiply(addMatrices(A11, A12), B22);

vector<vector<int>> P6 = strassenMatrixMultiply(subtractMatrices(A21, A11), addMatrices(B11, B12));

vector<vector<int>> P7 = strassenMatrixMultiply(subtractMatrices(A12, A22), addMatrices(B21, B22));

// Compute submatrices of the result

vector<vector<int>> C11 = subtractMatrices(addMatrices(P1, P4), addMatrices(P5, P7));

vector<vector<int>> C12 = addMatrices(P3, P5);

vector<vector<int>> C21 = addMatrices(P2, P4);

vector<vector<int>> C22 = addMatrices(subtractMatrices(P1, P2), addMatrices(P3, P6));

// Construct the result matrix

vector<vector<int>> result(n, vector<int>(n, 0));

for (int i = 0; i < mid; i++) {

for (int j = 0; j < mid; j++) {

result[i][j] = C11[i][j];

result[i][j + mid] = C12[i][j];

result[i + mid][j] = C21[i][j];

result[i + mid][j + mid] = C22[i][j];

}

}

return result;

}

int main() {

// Example usage

vector<vector<int>> A = {{1, 2}, {3, 4}};

vector<vector<int>> B = {{5, 6}, {7, 8}};

cout << "Matrix A:\n";

for (const auto& row : A) {

for (int elem : row) {

cout << elem << " ";

}

cout << endl;

}

cout << "Matrix B:\n";

for (const auto& row : B) {

for (int elem : row) {

cout << elem << " ";

}

cout << endl;

}

vector<vector<int>> result = strassenMatrixMultiply(A, B);

cout << "Resultant Matrix:\n";

for (const auto& row : result) {

for (int elem : row) {

cout << elem << " ";

}

cout << endl;

}

return 0;

}

Example 5.5: Consider two matrices:

Solution: Perform the following steps:

1. Divide: The matrices are divided into submatrices as follows:

2. Conquer: The recursive calls are made to compute products of submatrices using Strassen's formulas.

3. Combine: The submatrices of the result are computed using the products obtained from the recursive calls.

The final result is shown as follows:

Complexity analysis

The time complexity and space complexity are calculated as follows:

• Time complexity:

• Space complexity:

Strassen's algorithm is more efficient than the standard matrix multiplication algorithm for large matrices due to the reduction in the number of multiplications. However, its practical advantage is limited because of the overhead associated with the extra additions and subtractions.

Applications

Strassen's algorithm is primarily used in scenarios where large matrix multiplications need to be performed efficiently, such as in numerical simulations, scientific computing, and certain machine learning algorithms. Despite its theoretical efficiency, in practice, more optimized algorithms like the Coppersmith–Winograd algorithm or specialized linear algebra libraries (e.g., Basic Linear Algebra Subprograms (BLAS), Linear Algebra Package (LAPACK )) are often used for large-scale matrix operations.

Karatsuba multiplication

Karatsuba multiplication is an efficient algorithm for multiplying two numbers. It employs a divide and conquer strategy to break down the multiplication problem into smaller subproblems and reduce the overall number of multiplications required. Karatsuba multiplication has a better asymptotic complexity compared to the standard multiplication algorithm, especially for large numbers.

Algorithm 5.6: Karatsuba multiplication

karatsuba(x, y)

if x or y is a single-digit number

return x * y

n = max(length(x), length(y))

m = ceil(n / 2)

// Divide x and y into two halves

a, b = split(x, m)

c, d = split(y, m)

// Recursively compute three products

ac = karatsuba(a, c)

bd = karatsuba(b, d)

adbc = karatsuba((a + b), (c + d)) - ac - bd

// Combine the results to obtain the final product

result = ac * 10^(2m) + adbc * 10^m + bd

return result

Karatsuba multiplication using divide and conquer

Karatsuba multiplication is an efficient algorithm for multiplying two numbers. It employs a divide and conquer strategy to break down the multiplication problem into smaller subproblems and reduce the overall number of multiplications required. Karatsuba multiplication has a better asymptotic complexity compared to the standard multiplication algorithm, especially for large numbers.

Consider the following pseudo-code:

karatsuba(x, y)

if x or y is a single-digit number

return x * y

n = max(length(x), length(y))

m = ceil(n / 2)

// Divide x and y into two halves

a, b = split(x, m)

c, d = split(y, m)

// Recursively compute three products

ac = karatsuba(a, c)

bd = karatsuba(b, d)

adbc = karatsuba((a + b), (c + d)) - ac - bd

// Combine the results to obtain the final product

result = ac * 10^(2m) + adbc * 10^m + bd

return result

Program 5.6: Implementation in C++

#include <iostream>

#include <cmath>

#include <string>

using namespace std;

// Function to split a number into two halves

pair<string, string> split(const string& num, int m) {

int len = num.length();

string left = num.substr(0, len - m);

string right = num.substr(len - m, m);

return {left, right};

}

// Function to pad a number with zeros

string padZeros(const string& num, int count) {

string result = num;

result.insert(result.begin(), count, '0');

return result;

}

// Function to perform Karatsuba multiplication

string karatsuba(const string& x, const string& y) {

if (x.length() == 1 || y.length() == 1) {

// Base case: Single-digit multiplication

int result = stoi(x) * stoi(y);

return to_string(result);

}

int n = max(x.length(), y.length());

int m = ceil(n / 2);

// Divide x and y into two halves

auto [a, b] = split(x, m);

auto [c, d] = split(y, m);

// Recursively compute three products

string ac = karatsuba(a, c);

string bd = karatsuba(b, d);

string adbc = karatsuba(to_string(stoi(a) + stoi(b)), to_string(stoi(c) + stoi(d))) - ac - bd;

// Combine the results to obtain the final product

string result = ac + padZeros(adbc, m) + bd;

return result;

}

int main() {

// Example usage

string x = "1234";

string y = "5678";

cout << "Number 1: " << x << endl;

cout << "Number 2: " << y << endl;

string result = karatsuba(x, y);

cout << "Product: " << result << endl;

return 0;

}

Example 5.6: Consider multiplying two numbers:

• x=1234

• y=5678

Solution: Complete the following steps:

1. Divide: The numbers are divided into two halves:

a=12, b=34

c=56, d=78

2. Recursive calls: Three recursive calls are made to compute three products:

ac=12×56

bd=34×78

adbc=(12+34)×(56+78)−ac−bd

3. Combine: The results are combined to obtain the final product:

ac×102m + adbc×10m + bd

The final product is:

1234×5678=7006652

Complexity analysis

The time complexity and space complexity are calculated as follows:

• Time complexity:

• Space complexity:

Karatsuba multiplication has a better time complexity compared to the standard multiplication algorithm, especially for large numbers. It reduces the number of multiplications required, leading to improved performance.

Applications

Karatsuba multiplication is used in scenarios where efficient multiplication of large numbers is required, such as cryptography, signal processing, and polynomial multiplication in algebraic algorithms. It serves as a building block for more advanced algorithms and applications where large integer or polynomial multiplications are common.

Closest pair of points

The closest pair of points problem involves finding the pair of points with the smallest distance among a set of points in a 2D plane. The divide and conquer approach is commonly used to efficiently solve this problem. The idea is to recursively divide the set of points into smaller subsets, find the closest pairs in each subset, and then combine the results to determine the overall closest pair.

Algorithm 5.7: Closest pair of points

closestPair(points)

// Sort points by x-coordinate

sort(points)

// Call the recursive function

return closestPairRec(points)

closestPairRec(points)

n = number of points

// Base case: if there are only two or three points, brute force

if n <= 3

return closestPairBruteForce(points)

// Divide the points into two halves

mid = n / 2

leftPoints = points[0...mid-1]

rightPoints = points[mid...n-1]

// Recursively find the closest pairs in each half

leftClosest = closestPairRec(leftPoints)

rightClosest = closestPairRec(rightPoints)

// Find the minimum distance between the two halves

minDistance = min(leftClosest, rightClosest)

// Merge the results from the two halves

strip = pointsWithinStrip(points, mid, minDistance)

return min(minDistance, closestPairStrip(strip, minDistance))

pointsWithinStrip(points, mid, minDistance)

// Extract points within the strip of width 2*minDistance around the middle line

strip = empty list

for each point in points

if abs(point.x - points[mid].x) < minDistance

append point to strip

// Sort points in the strip by y-coordinate

sort(strip, compare by y-coordinate)

return strip

closestPairStrip(strip, minDistance)

// Find the closest pair of points within the strip

n = size of strip

// Initialize the minimum distance to the given minDistance

minDistance = minDistance

for i from 0 to n-1

for j from i+1 to min(i+7, n-1)

distance = distanceBetween(strip[i], strip[j])

minDistance = min(minDistance, distance)

return minDistance

Program 5.7: Implementation in C++

#include <iostream>

#include <vector>

#include <algorithm>

#include <cmath>

#include <limits>

using namespace std;

struct Point {

double x, y;

};

bool compareByX(const Point& p1, const Point& p2) {

return p1.x < p2.x;

}

bool compareByY(const Point& p1, const Point& p2) {

return p1.y < p2.y;

}

double distanceBetween(const Point& p1, const Point& p2) {

return sqrt(pow(p1.x - p2.x, 2) + pow(p1.y - p2.y, 2));

}

double closestPairBruteForce(const vector<Point>& points) {

int n = points.size();

double minDistance = numeric_limits<double>::max();

for (int i = 0; i < n - 1; i++) {

for (int j = i + 1; j < n; j++) {

double distance = distanceBetween(points[i], points[j]);

minDistance = min(minDistance, distance);

}

}

return minDistance;

}

double closestPairStrip(const vector<Point>& strip, double minDistance) {

int n = strip.size();

double minStripDistance = minDistance;

for (int i = 0; i < n; i++) {

for (int j = i + 1; j < min(i + 7, n); j++) {

double distance = distanceBetween(strip[i], strip[j]);

minStripDistance = min(minStripDistance, distance);

}

}

return minStripDistance;

}

double closestPairRec(vector<Point>& points) {

int n = points.size();

if (n <= 3) {

// Base case: Use brute force for small number of points

return closestPairBruteForce(points);

}

// Divide the points into two halves

int mid = n / 2;

vector<Point> leftPoints(points.begin(), points.begin() + mid);

vector<Point> rightPoints(points.begin() + mid, points.end());

// Recursively find the closest pairs in each half

double leftClosest = closestPairRec(leftPoints);

double rightClosest = closestPairRec(rightPoints);

// Find the minimum distance between the two halves

double minDistance = min(leftClosest, rightClosest);

// Merge the results from the two halves

vector<Point> strip = pointsWithinStrip(points, mid, minDistance);

return min(minDistance, closestPairStrip(strip, minDistance));

}

vector<Point> pointsWithinStrip(const vector<Point>& points, int mid, double minDistance) {

vector<Point> strip;

for (const Point& point : points) {

if (abs(point.x - points[mid].x) < minDistance) {

strip.push_back(point);

}

}

sort(strip.begin(), strip.end(), compareByY);

return strip;

}

double closestPair(vector<Point>& points) {

// Sort points by x-coordinate

sort(points.begin(), points.end(), compareByX);

// Call the recursive function

return closestPairRec(points);

}

int main() {

// Example usage

vector<Point> points = {{0, 0}, {1, 1}, {2, 2}, {3, 3}, {4, 4}, {5, 5}, {6, 6}, {7, 7}, {8, 8}, {9, 9}};

double minDistance = closestPair(points);

cout << "Closest Pair Distance: " << minDistance << endl;

return 0;

}

Example 5.7: Consider the set of points:

{(0,0),(1,1),(2,2),(3,3),(4,4),(5,5),(6,6),(7,7),(8,8),(9,9)}

Solution: To solve, follow these steps:

1. Divide: The points are sorted by x-coordinate, and the recursive function closestPairRec is called.

2. Base case: Since there are more than three points, the set is divided into two halves, shown as follows:

{(0,0),(1,1),(2,2),(3,3),(4,4)} and {(5,5),(6,6),(7,7),(8,8),(9,9)}

3. Recursive calls: The recursive calls are made on the left and right halves.

4. Merge: The minimum distance between the two halves is found to be 1.4142.

5. Strip: Points within the strip of width 2×1.4142 around the middle line are extracted:

{(4,4),(5,5),(6,6)}

6. Closest pair in strip: The closest pair within the strip is found to be as follows:

(4,4) and (5,5) with a distance of 1.4142

7. Final result: The overall closest pair is determined to be (5,5) with a distance of 1.4142.

Complexity analysis

The complexity analysis is as follows:

• Time complexity: The time complexity of the divide and conquer approach for the closest pair of points problem O(nlogn).

• Space complexity: The space required for storing the sorted points and intermediate results is O(n).

Applications

The closest pair of points problem has applications in computational geometry, computer graphics, pattern recognition, and robotics. The efficient divide and conquer algorithm is widely used in various applications where finding the closest pair of points is crucial, such as collision detection in computer graphics or robotics path planning.

Maximum subarray sum

The maximum subarray sum problem involves finding the contiguous subarray with the largest sum within a given array of numbers. The divide and conquer approach is commonly employed to efficiently solve this problem. The idea is to recursively divide the array into smaller subarrays, find the maximum subarray sum in each subarray, and then combine the results to determine the overall maximum subarray sum.

Algorithm 5.8: Maximum subarray sum

maxSubarraySum(arr, low, high)

if low equals high

return arr[low]

mid = (low + high) / 2

// Find the maximum subarray sum in the left subarray

leftMax = maxSubarraySum(arr, low, mid)

// Find the maximum subarray sum in the right subarray

rightMax = maxSubarraySum(arr, mid + 1, high)

// Find the maximum subarray sum crossing the midpoint

crossMax = maxCrossingSubarraySum(arr, low, mid, high)

// Return the maximum of the three sums

return max(leftMax, rightMax, crossMax)

maxCrossingSubarraySum(arr, low, mid, high)

// Find the maximum subarray sum in the left of the midpoint

leftSum = -infinity

sum = 0

for i from mid downto low

sum = sum + arr[i]

leftSum = max(leftSum, sum)

// Find the maximum subarray sum in the right of the midpoint

rightSum = -infinity

sum = 0

for i from mid+1 to high

sum = sum + arr[i]

rightSum = max(rightSum, sum)

// Return the sum of the maximum subarray sum in the left and right

return leftSum + rightSum

Program 5.8: Implementation in C++

#include <iostream>

#include <limits>

using namespace std;

int maxCrossingSubarraySum(int arr[], int low, int mid, int high) {

int leftSum = numeric_limits<int>::min();

int sum = 0;

for (int i = mid; i >= low; i--) {

sum += arr[i];

leftSum = max(leftSum, sum);

}

int rightSum = numeric_limits<int>::min();

sum = 0;

for (int i = mid + 1; i <= high; i++) {

sum += arr[i];

rightSum = max(rightSum, sum);

}

return leftSum + rightSum;

}

int maxSubarraySum(int arr[], int low, int high) {

if (low == high) {

return arr[low];

}

int mid = (low + high) / 2;

int leftMax = maxSubarraySum(arr, low, mid);

int rightMax = maxSubarraySum(arr, mid + 1, high);

int crossMax = maxCrossingSubarraySum(arr, low, mid, high);

return max({leftMax, rightMax, crossMax});

}

int main() {

// Example usage

int arr[] = {-2, -3, 4, -1, -2, 1, 5, -3};

int n = sizeof(arr) / sizeof(arr[0]);

int result = maxSubarraySum(arr, 0, n - 1);

cout << "Maximum Subarray Sum: " << result << endl;

return 0;

}

Example 5.8: Consider the array: [-2, -3, 4, -1, -2, 1, 5, -3]

Solution: The following steps are involved:

1. Divide: The array is divided into two halves: [-2, -3, 4, -1] and [-2, 1, 5, -3].

2. Recursive calls: The recursive calls are made to find the maximum subarray sums in the left and right halves.

Left half: [-2, -3, 4, -1] has a maximum subarray sum of 4.

Right half: [-2, 1, 5, -3] has a maximum subarray sum of 6.

3. Combine: The maximum subarray sum crossing the midpoint is found. The subarray [4, -1, -2, 1, 5] has a sum of 7.

4. Final result: The overall maximum subarray sum is the maximum of the left, right, and crossing sums, which is 7.

Complexity analysis

The complexity analysis is as follows:

• Time complexity: The time complexity of the divide and conquer approach for the maximum subarray sum problem is O(n logn).

• Space complexity: The space required for recursive function calls is O(log n).

Applications

The maximum subarray sum problem has applications in various fields, including finance, signal processing, and data analysis. It is commonly used in algorithms for financial portfolio optimization, audio signal processing, and in the analysis of time-series data. Efficient algorithms for finding the maximum subarray sum are essential in solving optimization problems related to resource allocation and signal processing.

Merge-based inversion count

Inversion count refers to the number of pairs of elements in an array that are out of order. The merge-based inversion count algorithm uses a divide and conquer approach to efficiently count inversions in an array. By recursively dividing the array into halves, counting inversions within each half, and then merging the results, this algorithm determines the total number of inversions.

Algorithm 5.9: Inversion count

inversionCount(arr, low, high)

count = 0

if low < high

mid = (low + high) / 2

// Recursively count inversions in the left half

count += inversionCount(arr, low, mid)

// Recursively count inversions in the right half

count += inversionCount(arr, mid + 1, high)

// Merge the two halves and count inversions

count += mergeAndCount(arr, low, mid, high)

return count

mergeAndCount(arr, low, mid, high)

leftSize = mid - low + 1

rightSize = high - mid

// Create temporary arrays for the left and right halves

leftArr[leftSize], rightArr[rightSize]

// Copy data to temporary arrays

for i from 0 to leftSize - 1

leftArr[i] = arr[low + i]

for j from 0 to rightSize - 1

rightArr[j] = arr[mid + 1 + j]

i = 0, j = 0, k = low

count = 0

// Merge the two halves while counting inversions

while i < leftSize and j < rightSize

if leftArr[i] <= rightArr[j]

arr[k] = leftArr[i]

i++

else

arr[k] = rightArr[j]

j++

count += leftSize - i

k++

// Copy the remaining elements of leftArr, if any

while i < leftSize

arr[k] = leftArr[i]

i++

k++

// Copy the remaining elements of rightArr, if any

while j < rightSize

arr[k] = rightArr[j]

j++

k++

return count

Program 5.9: Implementation in C++

#include <iostream>

#include <vector>

using namespace std;

long long mergeAndCount(vector<int>& arr, int low, int mid, int high) {

int leftSize = mid - low + 1;

int rightSize = high - mid;

// Create temporary arrays for the left and right halves

vector<int> leftArr(leftSize);

vector<int> rightArr(rightSize);

// Copy data to temporary arrays

for (int i = 0; i < leftSize; i++)

leftArr[i] = arr[low + i];

for (int j = 0; j < rightSize; j++)

rightArr[j] = arr[mid + 1 + j];

int i = 0, j = 0, k = low;

long long count = 0;

// Merge the two halves while counting inversions

while (i < leftSize && j < rightSize) {

if (leftArr[i] <= rightArr[j]) {

arr[k] = leftArr[i];

i++;

} else {

arr[k] = rightArr[j];

j++;

count += leftSize - i;

}

k++;

}

// Copy the remaining elements of leftArr, if any

while (i < leftSize) {

arr[k] = leftArr[i];

i++;

k++;

}

// Copy the remaining elements of rightArr, if any

while (j < rightSize) {

arr[k] = rightArr[j];

j++;

k++;

}

return count;

}

long long inversionCount(vector<int>& arr, int low, int high) {

long long count = 0;

if (low < high) {

int mid = (low + high) / 2;

// Recursively count inversions in the left half

count += inversionCount(arr, low, mid);

// Recursively count inversions in the right half

count += inversionCount(arr, mid + 1, high);

// Merge the two halves and count inversions

count += mergeAndCount(arr, low, mid, high);

}

return count;

}

int main() {

// Example usage

vector<int> arr = {1, 20, 6, 4, 5};

long long result = inversionCount(arr, 0, arr.size() - 1);

cout << "Inversion Count: " << result << endl;

return 0;

}

Example 5.9: Consider the array: [1, 20, 6, 4, 5]

Solution: Perform these steps:

1. Divide: The array is divided into two halves: [1, 20] and [6, 4, 5].

2. Recursive calls: The recursive calls are made to find the inversion count in the left and right halves.

Left half: [1, 20] has no inversions.

Right half: [6, 4, 5] has two inversions: (6, 4) and (6, 5).

3. Merge: The two halves are merged while counting inversions. The merging step identifies an inversion (20, 6).

4. Final result: The overall inversion count is the sum of inversions in the left, right, and merging steps, which is 0 + 2 + 1 = 3.

Complexity analysis

The complexity analysis is as follows:

• Time complexity: The time complexity of the merge-based inversion count algorithm is O(n logn).

• Space complexity: The space required for temporary arrays during merging is O(n).

Applications

Inversion count algorithms have applications in various fields, including data analysis, social network analysis, and optimization problems. They are used in scenarios where understanding the degree of disorder or the number of pairwise conflicts is important. For example, in sorting algorithms, inversion count can provide insights into the complexity of sorting an array, and it is also used in applications related to ranking and preference analysis.

Conclusion

The divide and conquer methodology serve as an efficient solution for complex problems because it transforms them into smaller workable components. Through this approach, the algorithm became more efficient and reusable because it maintained the system better while supporting parallel processing. Substructure optimization that uses recursion enables developers to build scalable products that function throughout different domains and application areas. This approach creates an organized framework, which enhances the computational execution as well as algorithm design effectiveness for problem-solving.

Exercise

1. Given an array of 16 elements, how many comparisons are needed to find both the maximum and minimum values using the divide and conquer approach?

2. For two 4×4 matrices, how many multiplications and additions are required using Strassen’s matrix multiplication algorithm?

3. Consider an array of size n sorted in descending order and sorted using quick sort with the last element as the pivot.

a) Derive the recurrence relation for the worst-case time complexity.

b) Solve for T(n) using the recurrence method.

4. We are going to play the game of smaller, bigger. The goal is to guess a number between 1 and n (integer). We consider that the algorithm takes as input the value of n and the number x to guess, and that it returns the number of moves to find this number.

a) State the problem more formally.

b) Write the iterative algorithm solving this problem.

c) Write the algorithm recursive solving this problem.

d) Compare complexities.

CHAPTER 6Greedy Algorithms

Introduction

In this chapter of the book, we will discuss greedy algorithms and their utility in the field of computer science engineering. The basic idea behind greedy algorithms is to make the best choice at each step without considering the overall consequences or future steps. This means that at each decision point, the algorithm selects the option that appears to be the best choice at that moment, without considering the potential impact on future decisions. Despite their limitations, greedy algorithms are widely used in various applications such as scheduling, minimum spanning trees, shortest path algorithms, and more, due to their simplicity and efficiency. Therefore, greedy algorithms make locally optimal choices at each step to construct a globally optimal solution. While they do not guarantee the absolute best solution, they are often used when finding an optimal solution is either computationally expensive or unnecessary.

Structure

The greedy algorithm chapter covers the following topics:

• Introduction to greedy algorithms

• General method

• Fractional knapsack problem

• Minimum cost spanning tree via Kruskal algorithm

• Minimum cost spanning tree via Prim’s algorithm

• Huffman coding

• Optimal merge pattern

• Interval scheduling

• Activity selection problem

• Job scheduling problem

Objectives

The primary objective of a greedy algorithm is to find an optimal solution to an optimization problem by making a series of locally optimal choices at each step, with the aim of eventually reaching a globally optimal solution. This entails selecting the best possible choice available at each decision point without considering the consequences of those choices on future steps. The goal is to iteratively build up a solution that incrementally improves upon the previous ones, ultimately leading to a solution that is optimal or close to optimal for the entire problem. Greedy algorithms prioritize immediate gains and rely on the hope that these locally optimal choices will culminate in the best possible outcome. However, it is important to note that while greedy algorithms are often efficient and simple to implement, they do not guarantee an optimal solution for all problems, and careful consideration is necessary to ensure their effectiveness in specific contexts.

Introduction to greedy algorithms

Greedy algorithms are a class of algorithms used for solving optimization problems. Unlike dynamic programming, which may solve the problem by considering all possible solutions and choosing the best one, greedy algorithms make a series of choices at each step with the hope that these locally optimal choices will lead to a globally optimal solution. One of the key characteristics of greedy algorithms is that they are generally efficient and simple to implement. However, they do not always guarantee an optimal solution for every problem. In some cases, a greedy algorithm may produce a suboptimal solution that is close to the optimal solution but not exactly the best one. Therefore, it is important to carefully analyze the problem and ensure that the greedy approach is appropriate and yields the desired results.

Thus, a choice made at each step in the greedy method should be:

• Feasible: Choice should satisfy problem constraints.

• Locally optimal: The best solution from all feasible solutions at the current stage should be selected.

• Irrevocable: Once the choice is made, it cannot be altered, i.e., if a feasible solution is selected (rejected) in step 1, it cannot be rejected (selected) in subsequent stages.

Characteristics of greedy algorithms

Problems that can be solved using the greedy method generally possess the following two properties:

• Greedy choice property: The global optimal solution is found by selecting locally optimal choices, or the ones that appear to be the best at the time. If the choice is feasible, include it in the solution set and reduce the problem by the same amount. The current decision may be influenced by prior decisions, but it is independent of future decisions.

• Optimal substructure: We say that the given problem exhibits optimal substructure if the optimal solution to the given problem contains the optimal solution to its sub-problems too. In a problem that possesses the optimal substructure, the next best choice always leads to an optimal solution.

General method

The general method of a greedy algorithm involves making the locally optimal choice at each step with the hope that it will lead to a globally optimal solution.

Algorithm 6.1: Algorithm of the general approach for the greedy method

1. Initialization: Begin with an empty solution set or an initial solution.

2. Selection: At each step, choose the best available option without considering the consequences of this choice on future steps. This choice is based on a specific criterion or heuristic.

3. Feasibility check: Ensure that the chosen option does not violate any constraints or rules.

4. Acceptance: If the chosen option meets the criteria or improves the solution, add it to the solution set.

5. Termination: Repeat steps 2 to 4 until a solution is found or the problem is solved satisfactorily.

6. Optimization (optional): Sometimes, additional steps are taken to optimize the solution further if needed.

The key characteristic of greedy algorithms is that they make decisions based solely on the current state of the problem without considering future consequences. This can lead to suboptimal solutions in some cases, but in many situations, greedy algorithms are efficient and provide acceptable solutions.

Fractional knapsack problem

In the context of a collection of items characterized by weight and profit values, the objective is to optimize the cumulative profit by strategically choosing items to fit into a knapsack, considering its constrained capacity. Unlike the conventional knapsack dilemma, where entire items must be selected, the fractional knapsack conundrum allows for breaking items into fractions to enhance the total value achieved.

Working of the fractional knapsack problem

The working of the fractional knapsack problem is as follows:

• Given: A list of items, each item represented as a pair of (profit, weight). Additionally, the capacity of the knapsack (W) is known.

• Goal: Optimize the selection of items to maximize the cumulative profit while ensuring that the total weight does not surpass W.

• Greedy strategy: Compute the profit-to-weight ratio for each item and arrange the items in descending order based on this ratio.

• Process:

1. Begin with an empty knapsack.

2. For every item in the sorted sequence:

a. If the item's weight fits within the remaining capacity, include it entirely.

b. Otherwise, add a fraction of the item to occupy the remaining capacity.

3. Continue until the knapsack reaches full capacity or all items are considered.

• Result: The total profit attained.

Arrange the packages in descending order based on their unit cost (vi/wi), shown as follows:

Sequentially examine the arranged packages; place the current package into the knapsack if the remaining capacity allows it (ensuring that the combined weight of the already placed packages and the weight of the current package does not surpass the knapsack's capacity).

Selection of the packages

To select the packages, review the array of unit costs and opt for packages based on decreasing unit costs, shown as follows:

Let us assume you found a temporary solution: (x1, …, xi).

The value of the knapsack is obtained as follows:

Corresponding to the weight of packages that have been put into the knapsack, the total weight is calculated as follows:

Therefore, the remaining weight limit of the knapsack is:

Procedure

The steps involved are as follows:

1. The root node denotes the knapsack's initial state, where no packages have been selected yet.

TotalValue = 0.

The upper bound of the root node, UpperBound, equals M multiplied by the maximum unit cost.

2. The root node will generate child nodes corresponding to the option of selecting the package with the highest unit cost. For each node, recalculations are made for the parameters as follows:

a. TotalValue = Previous TotalValue + number of selected packages * value of each package

b. M = Previous M - number of selected packages * weight of each package

c. UpperBound = TotalValue + New M * The unit cost of the next package to be considered

3. In the child nodes, priority is given to branching from the node with the higher upper bound. The child nodes of this branch correspond to selecting the next package with the highest unit cost. The parameters TotalValue, M, and UpperBound must be recalculated for each node using the formula mentioned in Step 2.

4. Step 3 is repeated with the caveat that nodes with upper bounds lower than or equal to the temporary maximum cost of an option found do not need further branching.

5. If all nodes are branched or cut off, the most valuable option is identified.

Algorithm 6.2: Algorithm of fractional knapsack using greedy method

Initialize an array called "cost" with the same size as V.

For i from 1 to the size of V, calculate cost[i] = V[i] / W[i].

Sort the array "cost" in descending order.

Initialize a variable i to 1.

While i is less than or equal to the size of V:

a. If W[i] is less than or equal to M:

- Decrease M by W[i].

- Add V[i] to the total.

b. If W[i] is greater than M, move to the next index (i = i + 1).

Example 6.1: Let us consider data items having the following parameter values:

I1: Profit is 60, and weight is 10

I2: Profit is 100, and weight is 20

I3: Profit is 120, and weight is 30

Additionally, the capacity of the knapsack is 50.

Solution: Perform these steps:

1. Sort the given data according to the profit-to-weight ratio.

2. Completely add I1, having a weight of 10, and I2, having a weight of 20.

3. Now, examine the remaining knapsack capacity. It is 20, so add 2/3 portion of I3, i.e., 20 units.

4. Final profit is = 60 + 100 + 120*(2/3) = 240

Program 6.1: Implementation of fractional knapsack

#include <bits/stdc++.h>

using namespace std;

struct Item {

int profit, weight;

Item(int p, int w) {

profit = p;

weight = w;

}

};

static bool cmp(struct Item a, struct Item b) {

double r1 = (double)a.profit / (double)a.weight;

double r2 = (double)b.profit / (double)b.weight;

return r1 > r2;

}

double fractionalKnapsack(int W, struct Item arr[], int N) {

sort(arr, arr + N, cmp);

// Rest of the implementation...

// Return the maximum profit achieved.

}

// Example usage:

int main() {

Item items[] = {{60, 10}, {100, 20}, {120, 30}};

int knapsackCapacity = 50;

double result = fractionalKnapsack(knapsackCapacity, items, 3);

cout << "Maximum profit: " << result << endl;

return 0;

}

Complexity of the fractional knapsack problem

The time complexity of the fractional knapsack problem hinges on the choice of sorting algorithm employed to arrange items by their value-to-weight ratios. Typically, common sorting algorithms like quick sort or heap sort are utilized, both boasting an average time complexity of O(n log n), where n represents the number of items.

Following the sorting process, the greedy approach is applied to pick items based on their value-to-weight ratios until the knapsack reaches its capacity. This greedy strategy entails iterating through the sorted list of items once, resulting in a time complexity of O(n).

Consequently, the overall time complexity of the fractional knapsack problem is chiefly determined by the sorting algorithm's time complexity, yielding an overall complexity of O(n log n).

Minimum cost spanning tree via Kruskal algorithm

This algorithm serves as a fundamental technique within graph theory aimed at discovering a minimum-weight spanning tree for a weighted, connected, undirected graph. A minimum spanning tree (MST) is a tree encompassing all vertices of the graph while minimizing the cumulative sum of edge weights. Kruskal’s algorithm adopts a greedy strategy to progressively construct the MST. The operational sequence is as follows:

1. Sorting edges: Commence by arranging all edges of the given graph in ascending order based on their weights. This step ensures the initiation with edges possessing the smallest weights.

2. Adding edges to MST: Initialize an empty MST. Continuously append edges and nodes to the MST if the newly included edge does not create a cycle. If the edge results in a cycle, discard it.

3. Iterate: Iterate through this procedure until the spanning tree comprises (V-1) edges, where V denotes the number of vertices in the graph.

4. Union-find algorithm: Kruskal’s algorithm employs the union-find data structure to identify cycles. The union-find algorithm aids in determining whether the addition of an edge would lead to a cycle in the MST.

5. Greedy choice: At each step, Kruskal’s algorithm makes a locally optimal selection to derive the overall optimal solution. It chooses the edge with the smallest weight that avoids creating a cycle in the constructed MST thus far.

Example 6.2: Let us assume the input graph having 9 V and 12 E as per the following table:

Weight value

Source node

Destination

1

7

6

2

8

2

2

6

5

4

0

1

4

2

5

6

8

6

7

7

8

8

0

7

9

3

4

10

5

4

11

1

7

14

3

5

Table 6.1: Table contains weights of edges from source to destination

Solution: The MST formed will comprise eight edges, as indicated by the equation (9 - 1 = 8). Let us proceed through the following steps:

1. Select edge 7-6: No cycle detected, so it is included.

2. Choose edge 8-2: No cycle detected, so it is included.

3. Opt for edge 6-5: No cycle detected, so it is included.

4. Select edge 0-1: No cycle detected, so it is included.

5. Opt for edge 2-5: No cycle detected, so it is included.

6. Choose edge 8-6: It forms a cycle, so it is discarded. Instead, pick edge 2-3: Include it.

7. Select edge 7-8: It forms a cycle, so it is discarded. Instead, pick edge 0-7: Include it.

8. Choose edge 1-2: It forms a cycle, so it is discarded. Instead, pick edge 3-4: Include it.

The resulting MST comprises the following edges: 7-6, 8-2, 6-5, 0-1, 2-5, 2-3, 0-7, and 3-4. It is important to note that the number of edges included in the MST equals (V-1), where V represents the number of vertices in the graph. Kruskal’s algorithm efficiently identifies the minimum cost spanning tree, rendering it a valuable tool in network design, transportation planning, and various other applications.

Algorithm 6.3: Kruskal’s algorithm for MST

Initialize with parameters (E, cost, n, t)

Let E be the set of edges in graph G, which consists of n vertices. The cost[u, v] represents the cost of edge (u, v), and 't' denotes the set of edges in the minimum-cost spanning tree. The function returns the final cost as follows:

{

Create a heap from the edge costs using heapify;

for i := 1 to n do parent[i] := -1; // Assign each vertex to a different set.

i := 0;

mincost := 0.0;

while ((i < n - 1) and (heap is not empty)) do

{

Remove a minimum-cost edge (u, v) from the heap and re-heapify using Adjust;

j := Find(u);

k := Find(v);

if (j ≠ k) then

{

i := i + 1;

t[i, 1] := u;

t[i, 2] := v;

mincost := mincost + cost[u, v];

Union(j, k);

}

}

if (i ≠ n-1) then

write("no spanning tree");

else

return mincost;

}

Program 6.2: Implementation of Kruskal’s algorithm for MST

#include <iostream>

#include <vector>

#include <algorithm>

using namespace std;

// Structure to represent an edge

struct Edge {

int src, dest, weight;

};

// Structure to represent a graph

class Graph {

public:

int V, E;

vector<Edge> edges;

Graph(int V, int E) {

this->V = V;

this->E = E;

}

// Add an edge to the graph

void addEdge(int src, int dest, int weight) {

Edge edge;

edge.src = src;

edge.dest = dest;

edge.weight = weight;

edges.push_back(edge);

}

// Find operation for disjoint set

int find(vector<int>& parent, int i) {

if (parent[i] == -1)

return i;

return find(parent, parent[i]);

}

// Union operation for disjoint set

void Union(vector<int>& parent, int x, int y) {

int xset = find(parent, x);

int yset = find(parent, y);

parent[xset] = yset;

}

// Kruskal's algorithm to find MST

void KruskalMST() {

vector<Edge> result; // This will store the MST

int i = 0; // Index used to pick next edge

// Step 1: Sort all the edges in non-decreasing order of their weight

sort(edges.begin(), edges.end(), [](const Edge& a, const Edge& b) {

return a.weight < b.weight;

});

// Allocate memory for creating V subsets

vector<int> parent(V, -1);

// Number of edges to be taken is equal to V-1

while (result.size() < V - 1 && i < E) {

Edge next_edge = edges[i++];

int x = find(parent, next_edge.src);

int y = find(parent, next_edge.dest);

// If including this edge does not cause cycle, include it

if (x != y) {

result.push_back(next_edge);

Union(parent, x, y);

}

}

// Print the contents of result[] to display the built MST

cout << "Edges in MST:\n";

for (i = 0; i < result.size(); ++i)

cout << result[i].src << " - " << result[i].dest << " Weight: " << result[i].weight << endl;

}

};

int main() {

// Example usage

int V = 4; // Number of vertices

int E = 5; // Number of edges

Graph graph(V, E);

// Adding edges to the graph

graph.addEdge(0, 1, 10);

graph.addEdge(0, 2, 6);

graph.addEdge(0, 3, 5);

graph.addEdge(1, 3, 15);

graph.addEdge(2, 3, 4);

// Finding minimum spanning tree

graph.KruskalMST();

return 0;

}

Complexity of Kruskal’s algorithm

The time complexity of Kruskal's algorithm is primarily influenced by the sorting of edges and the operations performed on the disjoint-set data structure. The breakdown is as follows:

• Sorting edges: Sorting all edges takes O(E log E) time, where E is the number of edges in the graph.

• Disjoint-set operations:

o Initializing the disjoint-set data structure takes O(V) time, where V is the number of vertices.

o Each find operation (to find the representative element of a set) and union operation (to merge two sets) takes nearly constant time. However, due to path compression and union by rank optimizations, the amortized time complexity for each of these operations is close to O(1).

• Iteration through edges: Iterating through sorted edges and performing disjoint-set operations takes O(E) time in the worst case.

Overall, the time complexity of Kruskal's algorithm is dominated by the sorting step, resulting in O(E log E) time complexity. Since E is typically upper-bounded by O(V2) in a dense graph and O(V) in a sparse graph, the time complexity can be expressed as O(E log V) or O(V2 log V) depending on the graph's characteristics.

The space complexity is dominated by the disjoint-set data structure, requiring O(V) space for storing parent and rank arrays. Additionally, the sorted list of edges may require O(E) space. Therefore, the total space complexity is O(V + E).

Minimum cost spanning tree via Prim’s algorithm

Prim’s algorithm, a greedy approach utilized for discovering the MST within a connected, weighted graph, operates as follows:

1. Initialization:

a. Commence with an empty spanning tree.

b. Select an arbitrary vertex as the starting point.

2. Core procedure:

a. While there remain vertices not yet encompassed in the MST (referred to as fringe vertices):

b. Identify edges linking any tree vertex with the fringe vertices.

c. Choose the edge with the minimum weight from these connections.

d. Incorporate the selected edge into the MST if it doesn't create a cycle.

Iterate through step 2 until the complete spanning tree is obtained.

The advantages of Prim’s algorithm are as follows:

• Ensures the discovery of the MST in a connected, weighted graph.

• Demonstrates a time complexity of O(E log V) when employing a binary heap or Fibonacci heap, where E represents the number of edges and V signifies the number of vertices.

Algorithm 6.4: Prim’s algorithm for MST

Initialize with parameters (E, cost, n, t)

// Let E denote the set of edges in graph G. The cost [1: n, 1: n] represents the cost adjacency matrix of an n-vertex graph, where cost [i, j] is either a positive real number or infinity if no edge (i, j) exists. A minimum spanning tree is calculated and stored as a set of edges in the array t [1: n-1, 1: 2]. Each entry (t [i, 1], t [i, 2]) represents an edge in the minimum-cost spanning tree. The final cost is then returned.

{

Let (k, l) be an edge of minimum cost in E;

mincost := cost [k, l];

t [1, 1] := k;

t [1, 2] := l;

for i := 1 to n do // Initialize near

{

if (cost [i, l] < cost [i, k]) then

near [i] := l;

else

near [i] := k;

near [k] := near [l] := 0;

}

for i := 2 to n - 1 do // Find n - 2 additional edges for t.

{

Let j be an index such that near [j] != 0 and cost [j, near [j]] is minimum;

t [i, 1] := j;

t [i, 2] := near [j];

mincost := mincost + cost [j, near [j]];

near [j] := 0;

}

for k := 1 to n do // Update near[].

{

if ((near [k] != 0) and (cost [k, near [k]] > cost [k, j])) then

near [k] := j;

}

return mincost;

}

Program 6.3: Implementation of Prim’s algorithm for MST

#include <cstring>

#include <iostream>

using namespace std;

#define INF 9999999

// number of vertices in graph

#define V 5

// create a 2d array of size 5x5

//for adjacency matrix to represent graph

int G[V][V] = {

{0, 9, 75, 0, 0},

{9, 0, 95, 19, 42},

{75, 95, 0, 51, 66},

{0, 19, 51, 0, 31},

{0, 42, 66, 31, 0}};

int main() {

int no_edge; // number of edge

// create a array to track selected vertex selected will become true otherwise false

int selected[V];

// set selected false initially

memset(selected, false, sizeof(selected));

// set number of edge to 0

no_edge = 0;

// the number of egde in minimum spanning tree will be always less than (V -1), where V is number of vertices in graph

// choose 0th vertex and make it true

selected[0] = true;

int x; // row number

int y; // col number

// print for edge and weight

cout << "Edge" << " : "<< "Weight";

cout << endl;

while (no_edge < V - 1) {

//For every vertex in the set S, find the all-adjacent vertices, calculate the distance from the vertex selected at step 1.

// if the vertex is already in the set S, discard it otherwise choose another vertex nearest to selected vertex at step 1.

int min = INF;

x = 0;

y = 0;

for (int i = 0; i < V; i++) {

if (selected[i]) {

for (int j = 0; j < V; j++) {

if (!selected[j] && G[i][j]) { // not in selected and there is an edge

if (min > G[i][j]) {

min = G[i][j];

x = i;

y = j;

}

}

}

}

}

cout << x << " - " << y << " : " << G[x][y];

cout << endl;

selected[y] = true;

no_edge++;

}

return 0;

}

Complexity of Prim’s algorithm

The time complexity of Prim's algorithm varies depending on the data structures utilized for implementation and their respective implementations. Primarily, it can be implemented using either a priority queue or an adjacency matrix/adjacency list combined with arrays or heaps. The analysis is as follows:

• Priority queue implementation: In this approach, each vertex is added to the priority queue at most once, and each edge is processed at most once. Inserting and updating the priority queue operations take O(log V) time, where V is the number of vertices. Thus, the time complexity using a priority queue is O((V + E) log V), where E is the number of edges.

• Arrays or heaps implementation: Using arrays or heaps for maintaining the priority queue, updating the priorities might take O(V) or O(log V) time, respectively. Consequently, the overall time complexity using this approach would be O(V2) or O(V log V), depending on the data structure chosen.

• Adjacency matrix representation: If the graph is represented using an adjacency matrix, finding the minimum weight edge adjacent to each vertex takes O(V2) time. Therefore, the total time complexity would be dominated by this step, resulting in O(V2).

• Adjacency list with heap implementation: When the graph is represented using an adjacency list and a heap (priority queue), finding the minimum weight edge adjacent to each vertex can be accomplished in O(log V) time (assuming a binary heap). Since each edge is processed exactly once, the overall time complexity would be O((V + E) log V).

In summary, the time complexity of Prim's algorithm is generally O((V + E) log V) if a priority queue is used and O(V^2) or O(V log V) if arrays or heaps are employed, respectively. The choice of data structures significantly influences the algorithm's performance.

Regarding space complexity, it typically ranges from O(V) to O(V^2), contingent on the selected data structures and graph representation.

Huffman coding

Another utilization of the greedy algorithm manifests in file compression.

Consider a scenario where we possess a file containing only the characters a, e, i, s, t, spaces, and newline characters. The frequencies of appearance for these characters are as follows: 10 occurrences of a, 15 occurrences of e, 12 occurrences of i, 3 occurrences of s, 4 occurrences of t, 13 spaces, and 1 newline.

Utilizing a standard coding scheme, wherein 3 bits are allocated for each character, the file containing 58 characters necessitates 174 bits for representation. This encoding scheme is delineated in the following table:

Character

Code

Frequency

Total bits

A

000

10

30

E

001

15

45

I

010

12

36

S

011

3

9

T

100

4

12

Space

101

13

39

Newline

110

1

3

Table 6.2: Encoding scheme of characters

Expressed through a binary tree structure, the binary code corresponding to each alphabet is outlined as follows:

To determine the representation of each character, commence at the root and record the pathway. Employ a 0 to denote the left branch and a 1 to denote the right branch. If a character, denoted as ci, exists at a depth di and appears fi times, the code's cost is equivalent to the summation of di multiplied by fi (∑di fi).

Consequently, with this representation, the total number of bits sums up to 3x10 + 3x15 + 3x12 + 3x3 + 3x4 + 3x13 + 3x1 = 174.

An improved code can be achieved with the subsequent representation:

Figure 6.1: Binary tree structure

Algorithm 6.5: Huffman’s algorithm using the greedy approach

Huffman's algorithm can be outlined as follows:

• A forest of trees is preserved, where each tree's weight equals the total frequency sum of its leaves.

• When the total number of characters is c, this process is iterated c-1 times.

• During each iteration, the two trees, T1 and T2, with the smallest weights are selected, and a new tree is constructed incorporating these two subtrees.

• By continuously repeating this process, we eventually attain an optimal Huffman coding tree.

Example 6.3: Let us assume the initial forest with the weight value of each tree is as follows:

Figure 6.2: Initial forest with weight value

Solution: After merging the two trees with the least weight, a forest is formed. Following the initial merge in the Huffman algorithm, where a new root, denoted as T1, is created, the process is as follows: The total weight of the resulting tree equals the combined weights of the original trees, shown as follows:

Figure 6.3: T1 tree structure

Once more, we pick the two trees with the lowest weights. In this case, these are T1 and t, which are then combined into a fresh tree having T2 as its root and a weight of 8, shown as follows:

Figure 6.4: T2 tree structure

In the subsequent step, we merge T2 and a to form T3, with a weight of 10+8=18. The outcome of this operation is shown in the following figure:

Figure 6.5: T3 tree structure

Following the third merge, the two trees with the lowest weights are the individual node trees representing i and the blank space, as shown in the following figure; these trees are combined to form a new tree with the root denoted as T4:

Figure 6.6: T4 tree structure

The fifth step involves merging the trees with roots e and T3. The outcome of this step is as follows:

Figure 6.7: T5 tree structure

Ultimately, the optimal tree is achieved by merging the two remaining trees. The optimal tree, with root denoted as T6, is depicted in the following figure:

Figure 6.8: Final Huffman tree structure

Table 6.3 shows the final encoding schemes for these characters. The complete binary tree with the lowest total cost, where all characters are found in the leaves, requires only 146 bits.

Character

Code

Frequency

Total bits

(Code bits X frequency)

A

001

10

30

E

01

15

30

I

10

12

24

S

00000

3

15

T

0001

4

16

Space

11

13

26

New line

00001

1

5

Total:

146

Table 6.3: Final encoding scheme of characters

Complexity of the Huffman coding algorithm

The time and space complexities of Huffman coding are influenced by various factors, such as the length of the input text, character distribution, and implementation specifics. A general insight into its complexity is as follows:

• Time complexity:

o Building the Huffman tree: This involves iterating through the frequency map of characters and constructing a priority queue, which usually takes O(n log n) time, where n represents the number of unique characters in the text.

o Generating Huffman codes: Traversing the Huffman tree to derive Huffman codes for each character requires O(n) time, where n is the count of unique characters.

o Encoding and decoding: Performing encoding and decoding for each character in the input text using Huffman codes consumes O(m) time, where m denotes the length of the input text.

In total, the time complexity of Huffman coding stands at O(n log n + m), where n denotes the count of unique characters and m signifies the length of the input text.

• Space complexity:

o Building the Huffman tree: The priority queue utilized for constructing the Huffman tree occupies O(n) space, where n denotes the number of unique characters.

o Huffman tree: The space requirement for storing the Huffman tree relies on its size, which is proportional to the number of characters in the input text.

o Huffman codes: The space needed to store Huffman codes for each character varies based on the code lengths, typically ranging from O(1) to O(log n), where n represents the count of unique characters.

Overall, the space complexity of Huffman coding amounts to O(n + m), where n represents the count of unique characters and m signifies the length of the input text.

Optimal merge pattern

When presented with n sorted files, numerous methods exist to merge them pairwise into a single sorted file. Since various pairings entail different computational time requirements, our aim is to identify an optimal approach, specifically, one necessitating the fewest comparisons, to pairwise merge n sorted files. This process is commonly referred to as a 2-way merge pattern. Combining an n-record file with an m-record file may involve up to n + m record movements. The straightforward strategy involves merging the two smallest files together at each step. These 2-way merge patterns can be depicted using binary merge trees.

Algorithm 6.6: Generation of 2-way merge tree

struct tree_node

{

tree_node * lchild; tree_node * rchild;

};

Algorithm TREE (n)

// list is a global of n single node binary trees

{

for i := 1 to n – 1 do

{

pt ß new tree_node

(pt -> lchild) ß least (list); // merge two trees with smallest lengths

(pt -> rchild) ß least (list);

(pt -> weight) ß ((pt -> lchild) -> weight) + ((pt -> rchild) -> weight);

insert (list, pt);

}

return least (list); // The tree left in list is the merge tree

}

Example 6.4: Let us consider three sorted files, denoted as X1, X2, and X3, containing 30, 20, and 10 records, respectively.

Solution: The merging of these files can proceed as outlined in the following table:

S. No.

First merging

Record moves in the first merging

Second merging

Record moves in the second merging

Total no. of records moves

1.

X1 and X2 = T1

50

T1 and X3

60

50 + 60 = 110

2.

X2 and X3 = T1

30

T1 and X1

60

30 + 60 = 90

Table 6.4: Merging of files using optimal merge pattern

The second case is optimal.

Example 6.5: Given five files (X1, X2, X3, X4, X5) with sizes (20, 30, 10, 5, 30), employ the greedy rule to determine the optimal method for pairwise merging, resulting in an optimal solution represented by a binary merge tree.

Solution: Figure 6.9 represents the initial case of file records with their size in the nodes.

Figure 6.9: Initial case of files

Merge X4 and X3 to get 15 record moves. Call this Z1, illustrated as follows:

Figure 6.10: Merging of files in iteration 1

Merge Z1 and X1 to get 35 record moves. Call this Z2, as shown in the following figure:

Figure 6.11: Merging of files in iteration 2

Merge X2 and X5 to get 60 record moves. Call this Z3, depicted in the following figure:

Figure 6.12: Merging of files in iteration 3

Merge Z2 and Z3 to get 95 record moves. This is the answer. Call this Z4, shown as follows:

Figure 6.13: Final optimal merge pattern

Therefore, the total number of record moves is 15 + 35 + 60 + 95 = 205. This is an optimal merge pattern for the given problem.

Program 6.4: Implementation of the optimal merge pattern

#include <iostream>

#include <queue>

#include <vector>

using namespace std;

// Function to find the minimum computations required to merge sorted arrays

int optimalMergePattern(vector<int>& arr) {

// Create a priority queue to store the sizes of the sorted arrays

priority_queue<int, vector<int>, greater<int>> pq(arr.begin(), arr.end());

int totalComputations = 0;

// Merge the sorted arrays until there's only one array left

while (pq.size() > 1) {

// Extract the two smallest arrays

int first = pq.top();

pq.pop();

int second = pq.top();

pq.pop();

// Merge the two arrays and update the total computations

int mergedSize = first + second;

totalComputations += mergedSize;

// Add the merged array back to the priority queue

pq.push(mergedSize);

}

return totalComputations;

}

int main() {

// Example usage

vector<int> arr = {4, 3, 2, 6};

int minComputations = optimalMergePattern(arr);

cout << "Minimum computations required: " << minComputations << endl;

return 0;

}

Complexity of the optimal merge pattern algorithm

The time and space complexities of the optimal merge pattern algorithm are contingent on the implementation specifics, particularly the method employed for the merging process. The following is a breakdown of the complexity:

• Time complexity:

o Building the priority queue: The initial construction of the priority queue from the input vector of array sizes demands O(n log n) time, where n represents the number of arrays.

o Merging arrays: Within each iteration of the while loop, two arrays of minimum sizes are extracted from the priority queue and merged. Given the use of a priority queue, extracting the two minimum elements and inserting the merged array back into the priority queue necessitates O(log n) time, where n denotes the count of arrays in the priority queue.

Overall, the time complexity of the algorithm is chiefly influenced by the sorting phase, resulting in O(n log n) time complexity, where n signifies the number of arrays.

• Space complexity:

o Priority queue: The space allocation required for storing the priority queue depends on the count of arrays, amounting to O(n).

Overall, the space complexity of the algorithm is O(n).

Interval scheduling

Provided a collection of N events, each characterized by its start and end times, our objective is to devise a schedule that accommodates the maximum number of events. However, the caveat lies in the fact that we cannot choose an event partially; it is an all-or-nothing proposition.

There exist several greedy algorithms capable of addressing this challenge. Let us explore a few of them.

Algorithm 6.7.1: Choosing the shortest events

The primary concept involves selecting the shortest events initially.

Nonetheless, this strategy does not consistently produce the optimal solution. For instance, contemplate the subsequent events:

• Event A: Commences at 1 and concludes at 5.

• Event B: Commences at 3 and concludes at 6.

• Event C: Commences at 4 and concludes at 7.

If we opt for the shortest events, we would only select one (either A or B), thereby overlooking the opportunity to select both B and C.

Algorithm 6.7.2: Selecting the earliest start time

Another method involves consistently choosing the subsequent event that begins at the earliest possible time.

Although effective in certain scenarios, this approach falters in others. Consider the following instance:

• Event D: Commences at 1 and concludes at 4.

• Event E: Commences at 3 and concludes at 6.

• Event F: Commences at 5 and concludes at 7.

Adhering to this strategy would lead to selecting only D, thereby overlooking both E and F.

Algorithm 6.7.3: Opt for the earliest end time (optimal)

The most efficient greedy algorithm entails selecting the subsequent event that concludes at the earliest possible time.

This strategy unfailingly yields an optimal solution.

Contemplate the outcome of selecting an event that concludes later than the earliest-ending event, in such a case:

• We would have an equal number of choices for the subsequent event.

• Consequently, opting for an event with a later conclusion time can never result in an improved solution.

Hence, this greedy algorithm guarantees an optimal schedule.

Note: Keep in mind that these algorithms serve as tools within our problem-solving repertoire. Depending on the particular context and constraints, we can select the most appropriate approach.

Program 6.5: Implementation of the interval scheduling algorithm

#include <iostream>

#include <vector>

#include <algorithm>

using namespace std;

// Structure to represent an interval

struct Interval {

int start;

int end;

};

// Comparison function to sort intervals by their end times

bool compareIntervals(const Interval& a, const Interval& b) {

return a.end < b.end;

}

// Function to find the maximum number of non-overlapping intervals

int intervalScheduling(vector<Interval>& intervals) {

// Sort intervals by their end times

sort(intervals.begin(), intervals.end(), compareIntervals);

int count = 1; // Count of non-overlapping intervals

int currentEnd = intervals[0].end;

// Iterate through the sorted intervals

for (int i = 1; i < intervals.size(); ++i) {

// If the start time of the current interval is after the end time of the previous interval,

// it can be included in the schedule

if (intervals[i].start >= currentEnd) {

++count;

currentEnd = intervals[i].end;

}

}

return count;

}

int main() {

// Example usage

vector<Interval> intervals = {{1, 3}, {2, 4}, {3, 6}, {5, 7}, {8, 10}, {9, 11}};

int maxNonOverlapping = intervalScheduling(intervals);

cout << "Maximum number of non-overlapping intervals: " << maxNonOverlapping << endl;

return 0;

}

Complexity of the interval scheduling algorithm

The interval scheduling problem revolves around selecting the maximum number of non-overlapping intervals from a given set. This problem can be effectively tackled using the greedy algorithm, characterized by the following time and space complexities:

• Time complexity:

o Sorting: If the intervals are not pre-sorted, the initial step typically involves sorting them based on their end times. This sorting operation generally consumes O(n log n) time, where n denotes the count of intervals.

o Greedy selection: Following sorting, the greedy algorithm traverses through the sorted intervals once. At each iteration, it chooses the interval that concludes earliest and avoids overlapping with previously selected intervals. This process demands O(n) time, as each interval is scrutinized once.

Overall, the time complexity of the greedy algorithm for interval scheduling is predominantly governed by the sorting phase, resulting in O(n log n) time complexity.

• Space complexity: The space complexity of the greedy algorithm for interval scheduling amounts to O(n) since it necessitates storing the sorted intervals in memory.

Activity selection problem

Overall, the time complexity of the greedy algorithm for interval scheduling is predominantly.

The activity selection problem represents a renowned optimization challenge that can be efficiently resolved employing a greedy algorithm. Let us delve deeper into its intricacies:

• Problem description: You are provided with 'n' activities, each characterized by its start time and end time. The objective is to choose the highest number of non-overlapping activities feasible for execution by either an individual or a machine within a stipulated timeframe. It is assumed that an individual or machine can solely engage in one activity concurrently.

• Greedy algorithm strategy: The key greedy selection in this scenario involves consistently selecting the subsequent activity with the earliest finish time among the remaining activities, provided its start time is later than or equal to the finish time of the previously chosen activity.

Adhering to this approach guarantees the maximization of non-conflicting activities.

Example 6.6: Let us illustrate the activity selection problem with the given parameters:

Activities: A, B, and C

Start time: [10, 12, 20]

Finish time: [20, 25, 30]

Solution: The greedy algorithm unfolds as follows:

Choose activity A (10-20).

Activity B overlaps with A; therefore, it is skipped.

Select activity C (20-30).

The maximum set of activities that can be executed is {A, C}.

Applications of greedy algorithms

Greedy algorithms find utility in optimization problems where solutions are gradually constructed, selecting each component to provide the most immediate benefit.

Some common algorithms employing the greedy approach include:

• Kruskal’s minimum spanning tree (MST): Opting for edges with the smallest weight that avoid forming cycles.

• Prim’s MST: Establishing connections between vertices via the smallest weight edge.

• Dijkstra’s shortest path: Constructing the shortest-path tree by incrementally adding edges.

• Huffman coding: Assigning variable-length bit codes to characters based on their frequencies.

• Approximation for challenging optimization problems: For instance, the traveling salesman problem can be tackled by greedily selecting the nearest unvisited city at each stage.

Program 6.6: Implementation of the activity selection problem

// C++ program for activity selection problem.

// The following implementation assumes that the activities

// are already sorted according to their finish time

#include <bits/stdc++.h>

using namespace std;

// Prints a maximum set of activities that can be done by a

// single person, one at a time.

void printMaxActivities(int s[], int f[], int n)

{

int i, j;

cout << "Following activities are selected" << endl;

// The first activity always gets selected

i = 0;

cout << i << " ";

// Consider rest of the activities

for (j = 1; j < n; j++) {

// If this activity has start time greater than or

// equal to the finish time of previously selected

// activity, then select it

if (s[j] >= f[i]) {

cout << j << " ";

i = j;

}

}

}

int main()

{

int s[] = { 1, 3, 0, 5, 8, 5 };

int f[] = { 2, 4, 6, 7, 9, 9 };

int n = sizeof(s) / sizeof(s[0]);

// Function call

printMaxActivities(s, f, n);

return 0;

}

Complexity of the activity selection problem

The time complexity of the greedy algorithm for the activity selection problem is generally O(n log n), where n represents the number of activities. This complexity arises primarily from the sorting step, which typically requires O(n log n) time if the activities are not already sorted. After sorting, the greedy selection of activities usually takes linear time, O(n), since each activity is examined once. Therefore, the overall time complexity is dominated by the sorting step.

Job scheduling problem

The goal of the job scheduling problem is to assign jobs to resources in such a way that optimizes an objective while satisfying the constraints. The procedure of job sequencing with deadlines is given as follows:

When presented with a set of n jobs, each job i is associated with a deadline di > 0 and a profit Pi > 0. The profit Pi is earned only if the job is completed by its respective deadline. With only one machine available for processing jobs, the objective is to determine an optimal solution, defined as the feasible solution that maximizes profit.

First, the jobs are sorted in non-decreasing order by their deadlines. This sorting process creates an array d of size n to store the deadlines in accordance with their associated profit values.

To verify the feasibility of adding a new job i to the set of jobs J, we simply insert i into J while preserving the deadline ordering. Subsequently, we ensure that the deadlines of the jobs in J satisfy the condition d[J[r]] ≤ r for 1 ≤ r ≤ k+1.

Example 6.7 Let us consider the example for job scheduling problem using greedy approach, n = 4, (P1, P2, P3, P4,) = (100, 10, 15, 27) and (d1 d2 d3 d4) = (2, 1, 2, 1).

Solution: The feasible solutions and their values are as shown in the following table:

Serial number

Feasible solution

Procuring sequence

Value

Remarks

1

1,2

2,1

110

2

1,3

1,3 or 3,1

115

3

1,4

4,1

127

Optimal

4

2,3

2,3

25

5

3,4

4,3

42

6

1

1

100

7

2

2

10

8

3

3

15

9

4

4

27

Table 6.5: Job scheduling solution

Algorithm 6.8: Job scheduling using the greedy approach

The algorithm builds an optimal set J comprising jobs that can be completed within their respective deadlines.

Algorithm GreedyJob (d, J, n)

// J represents a set of jobs that meet their deadlines.

{

J := {1};

for i := 2 to n do

{

if (all jobs in J union {i} can be completed by their deadlines) then J := J union {i};

}

}

Program 6.7: Implementation of the job scheduling problem

#include <iostream>

#include <algorithm>

#include <vector>

using namespace std;

// Structure to represent a job

struct Job {

char id; // Job ID

int deadline; // Deadline of job

int profit; // Profit associated with the job

};

// Function to compare jobs based on their profit

bool compare(Job a, Job b) {

return (a.profit > b.profit);

}

// Function to find the maximum profit by scheduling jobs

void findMaxProfit(Job arr[], int n) {

// Sort jobs based on profit in decreasing order

sort(arr, arr + n, compare);

vector<int> result(n, -1); // To store result (Sequence of jobs)

vector<bool> slot(n, false); // To keep track of free time slots

// Iterate through all given jobs

for (int i = 0; i < n; i++) {

// Find a free slot for this job (Note that we start from the last possible slot)

for (int j = min(n, arr[i].deadline) - 1; j >= 0; j--) {

// Free slot found

if (slot[j] == false) {

result[j] = i; // Add this job to result

slot[j] = true; // Make this slot occupied

break;

}

}

}

// Print the sequence of jobs

cout << "The sequence of jobs with maximum profit is: ";

for (int i = 0; i < n; i++) {

if (result[i] != -1)

cout << arr[result[i]].id << " ";

}

cout << endl;

}

int main() {

// Example jobs

Job arr[] = { {'a', 2, 100}, {'b', 1, 19}, {'c', 2, 27}, {'d', 1, 25}, {'e', 3, 15} };

int n = sizeof(arr) / sizeof(arr[0]);

// Find the maximum profit

findMaxProfit(arr, n);

return 0;

}

Complexity of the job scheduling problem

The time complexity of the job scheduling problem solved using the greedy approach is O(n log n), where n denotes the number of jobs. A breakdown of the complexity is as follows:

• Sorting the jobs by profit: Sorting n jobs by their profit typically requires O(n log n) time complexity, as efficient sorting algorithms like quicksort or merge sort are commonly employed.

• Iterating through the sorted jobs: After sorting, iterating through the sorted jobs takes linear time, O(n).

In summary, the sorting step dominates the time complexity, resulting in O(n log n) overall time complexity for solving the job scheduling problem using the greedy approach.

Conclusion

By maximizing the most advantageous decision at every stage, greedy algorithms provide a strong and natural way to solve optimization challenges. Their efficiency and simplicity make them perfect for a wide range of applications, particularly when the issue shows the greedy choice property and optimum substructure. Although they do not always provide the best solution, particularly in complicated situations like the traveling salesman problem, their performance in real-world uses like Dijkstra's algorithm, Prim's and Kruskal's MST algorithms, and Huffman coding highlights their value. Though they have shortcomings, greedy approaches are nonetheless crucial for algorithm designers. Using their strengths means knowing when and how to use them properly.

In the next chapter, the reader will examine a method called dynamic programming (DP), which resembles the divide and conquer strategy by decomposing the problem into smaller subproblems.

Exercise

1. Write the algorithm for the general greedy method. Compare how it is better than the traditional approach.

2. Explain the Huffman coding algorithm using the greedy method with a suitable example.

3. Explain how Kruskal's algorithm can be used to find the MST.

4. Explain how Prim's algorithm can be used to find the MST.

5. For the activity selection problem, consider the following set of activities with their start and end times:

Activity Start time End time

A 1 3

B 2 5

C 3 6

D 5 7

E 1 8

Using a greedy approach, determine the maximum number of activities that can be selected without overlapping. Additionally, provide the sequence of selected activities.

6. You are given a set of jobs, each with a unique ID, a deadline by which it must be completed, and the profit gained upon completion. The goal is to schedule these jobs in a way that maximizes the total profit. Using a greedy approach, how would you determine the sequence of jobs to schedule, and what would be the resulting maximum profit for the following set of jobs?

Job ID Deadline Profit

A 2 100

B 1 19

C 2 27

D 1 25

E 3 15

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 7Dynamic Programming

Introduction

In this chapter, we will explore a technique known as dynamic programming (DP), which shares similarities with the divide and conquer approach, as it breaks down the problem into smaller subproblems. However, unlike divide and conquer, these subproblems are not solved independently. Instead, the results of these smaller subproblems are stored and utilized for similar or overlapping subproblems. This chapter also introduces various algorithms, including the 0/1 knapsack problem, the subset sum problem, edit distance (Levenshtein distance), optimal binary search tree, matrix chain multiplication, longest common subsequence problem, and the traveling salesman problem.

Structure

The chapter covers the following topics:

• Dynamic programming

• 0/1 knapsack problem

• Subset sum problem

• Levenshtein distance or edit distance

• Optimal binary search tree

• Matrix chain multiplication

• Longest common subsequence problem

• Traveling salesperson problem

Objectives

The objective of DP is to optimize the use of computational resources by breaking down complex problems into smaller, overlapping subproblems and solving each subproblem only once. By storing the results of these subproblems in a memory structure such as an array or a table, DP eliminates redundant calculations, thereby improving efficiency. This approach significantly reduces the time complexity of algorithms compared to naive recursive methods. DP is particularly useful for solving optimization problems where the goal is to find the best possible solution while minimizing computational effort. It is widely applied in areas such as graph algorithms, sequence alignment, and resource allocation.

Dynamic programming

DP is a powerful algorithmic technique utilized in computer science and mathematics to address problems efficiently. It involves breaking down intricate problems into smaller, interconnected subproblems and storing their solutions to avoid redundant calculations. This method is particularly advantageous for optimization challenges, as it allows for the creation of optimal solutions based on the optimal outcomes of these smaller subproblems.

DP encompasses the following three primary stages:

1. Identification of overlapping subproblems: Divide the primary problem into smaller subproblems that exhibit overlapping or recurrence.

2. Establishment of a recurrence relation: Formulate a recursive formula expressing the solution to the primary problem in terms of solutions to its subproblems.

3. Memorization or tabulation: Store solutions to subproblems in a structured data format (either through memorization or tabulation) to avoid redundant calculations and facilitate efficient solution retrieval.

Algorithm 7.1: Dynamic programming

// Function for solving the problem utilizing dynamic programming

int dynamicProgramming(int problem) {

// Set up data structures for storing interim outcomes.

int dp[SIZE][SIZE];

// Initialization of base cases

initializeBaseCases(dp);

// Calculate and store the optimal solutions for subproblems.

for (int i = 0; i < SIZE; i++) {

for (int j = 0; j < SIZE; j++) {

dp[i][j] = computeOptimalSolution(i, j, dp);

}

}

// Retrieve and return the ultimate solution.

return extractFinalSolution(dp);

}

// Function for initializing base cases.

void initializeBaseCases(int dp[SIZE][SIZE]) {

// Establish the base cases in the dynamic programming table according to the constraints of the problem.

}

// Function to calculate the optimal solution for a specific subproblem.

int computeOptimalSolution(int i, int j, int dp[SIZE][SIZE]) {

// Compute the optimal solution for the given subproblem using previously computed solutions

return dp[i-1][j-1] + dp[i-1][j]; // Example of recurrence relation

}

// Function to retrieve the final solution.

int extractFinalSolution(int dp[SIZE][SIZE]) {

// Retrieve the ultimate solution from the DP table according to the specifications of the problem.

}

int main() {

int solution = dynamicProgramming(problem);

// Subsequent processing or output of the solution

return 0;

}

In this algorithm, the variable SIZE denotes the size of the problem or the dimensions of the DP table. The dynamicProgramming() function is responsible for initializing the DP table, computing optimal solutions for subproblems, and extracting the final solution. The initializeBaseCases() function establishes the base cases in the DP table, computeOptimalSolution() calculates the optimal solution for each subproblem, and extractFinalSolution() retrieves the final solution from the DP table. Ultimately, the main() function invokes the dynamicProgramming() function to solve the problem utilizing DP techniques.

The dynamicProgramming function serves as the main entry point for solving the problem using DP. The following list follows the process of dynamicProgramming():

1. It initializes a DP table (dp) to store intermediate results.

2. Base cases are initialized in the DP table to handle the simplest cases of the problem.

3. It iteratively computes optimal solutions for subproblems and stores them in the DP table.

4. Finally, it extracts the final solution from the DP table and returns it.

5. Each step (initialization, base case handling, solution computation, and final solution extraction) is represented by a separate function for clarity and modularity.

Detailed examples of DP span various domains and problem types. Some common examples include the following:

• Fibonacci sequence: Computing Fibonacci numbers using DP can greatly reduce the time complexity compared to a naive recursive approach.

• Shortest path problems: DP algorithms, such as Floyd-Warshall and Bellman-Ford, are used to find the shortest path between all pairs of vertices in a weighted graph.

• Longest common subsequence (LCS): Finding the longest subsequence that is common to two sequences is a classic example of a problem solved using DP.

• Knapsack problem: DP can be used to find the most valuable combination of items that can be included in a knapsack of limited capacity.

• Matrix chain multiplication: Optimally parenthesizing matrix multiplication to minimize the number of scalar multiplications can be achieved using DP.

These examples illustrate the versatility and effectiveness of DP in solving a wide range of problems efficiently.

0/1 knapsack problem

The 0/1 knapsack problem is a classic optimization problem in computer science and mathematics. In this problem, a set of items is given, each with a weight and a value, and one has to determine the number of each item to include in a knapsack so that the total weight is less than or equal to a given limit and the total value is maximized. The 0/1 in the problem name signifies that an item can only be selected either entirely or not at all.

Working of 0/1 knapsack

For each i ≤ n and each w ≤ W, solve the knapsack problem for the first i objects when the capacity is w. This will work because solutions to larger subproblems can be built up easily from solutions to smaller ones. We construct a matrix V[0 . . . n, 0 . . .W]. For 1 ≤ i ≤ n, and 0 ≤ j ≤ W, V[i, j] will store the maximum value of any set of objects {1, 2, . . . , i} that can fit into a knapsack of weight j. V[n,W] will contain the maximum value of all n objects that can fit into the entire knapsack of weight W.

To compute entries of V, we will use an inductive approach. As a basis, V[0, j] = 0 for 0 . j . W.

If we have no items, then we have no value. We consider the following two cases:

• Leave object i: If we choose not to take object i, then the optimal value will come about by considering how to fill a knapsack of size j with the remaining objects {1, 2, . . . , i − 1}. This is just V[i − 1, j].

• Take object i: If we take object i, then we gain a value of vi. But we use up wi of the capacity. With the remaining j − wi capacity in the knapsack, we can fill it in the best possible way with objects {1, 2, . . . , i − 1}. This is vi + V[i − 1, j − wi]. This is only possible if wi ≤ j.

This leads to the following recursive formulation:

A naive evaluation of this recursive definition is exponential. So, as usual, we avoid recomputation by making a table.

DP is a powerful technique for solving optimization problems like the knapsack problem. In the case of the 0/1 knapsack problem, DP works by building a table where each cell represents the maximum value that can be obtained using a certain subset of items and a certain weight limit. The table is filled iteratively, considering each item and each possible weight limit and determining whether it is beneficial to include the item or not based on its weight and value.

Algorithm 7.2: 0/1 knapsack

function knapsack(values[], weights[], capacity, n):

let dp be a 2D array of size (n+1) x (capacity+1)

for i from 0 to n:

for w from 0 to capacity:

if i = 0 or w = 0:

dp[i][w] = 0

else if weights[i-1] <= w:

dp[i][w] = max(values[i-1] + dp[i-1][w-weights[i-1]], dp[i-1][w])

else:

dp[i][w] = dp[i-1][w]

return dp[n][capacity]

Program 7.1: Implementation in C++

#include <iostream>

#include <vector>

using namespace std;

int knapsack(vector<int>& values, vector<int>& weights, int capacity) {

int n = values.size();

vector<vector<int>> dp(n + 1, vector<int>(capacity + 1, 0));

for (int i = 1; i <= n; ++i) {

for (int w = 1; w <= capacity; ++w) {

if (weights[i - 1] <= w) {

dp[i][w] = max(values[i - 1] + dp[i - 1][w - weights[i - 1]], dp[i - 1][w]);

} else {

dp[i][w] = dp[i - 1][w];

}

}

}

return dp[n][capacity];

}

int main() {

vector<int> values = {10, 15, 40};

vector<int> weights = {1, 2, 3};

int capacity = 6;

cout << "Maximum value: " << knapsack(values, weights, capacity) << endl;

return 0;

}

Example 7.1: Suppose we have the following items:

• Item 1: Weight = 2, Value = 6

• Item 2: Weight = 2, Value = 10

• Item 3: Weight = 3, Value = 12

The knapsack capacity is 5.

Solution: We will create a table to store the solutions to subproblems. Each cell DP[i][w] will represent the maximum value that can be obtained using the first I items and a knapsack with capacity w.

1. Initialization: Create a 2D array with dimensions (n+1) x (W+1), where n is the number of items and W is the knapsack capacity. Initialize all cells to 0. The following table shows the initialization of the table for the 0/1 knapsack problem:

0

1

2

3

4

5

0

0

0

0

0

0

0

1

0

2

0

3

0

Table 7.1: Table initialization

2. Fill the table: The table is filled for each item as follows:

Item 1 (Weight = 2, Value = 6):

• For each capacity from 0 to 5:

o When capacity is 0, we cannot take any item. So, the value remains 0.

o For capacities 1 and 2, we cannot take item 1, so the value remains 0.

o When capacity is 3, we can take item 1 (Weight = 2) as it fits, so the maximum value is 6.

o For capacities 4 and 5, we can take item 1 (Weight = 2) as it fits, so the maximum value is 6.

Table 7.2 shows the filling entries when item 1 is selected, having weight=2 and value=6.

0

1

2

3

4

5

0

0

0

0

0

0

0

1

0

0

0

6

6

6

2

0

3

0

Table 7.2: Filling table for item 1

The next phase fills the table with the parameters for item number 2.

Item 2 (Weight = 2, Value = 10):

• For each capacity from 0 to 5:

o When capacity is 0, we cannot take any item. So, the value remains 0.

o For capacities 1 and 2, we cannot take item 2, so the value remains 6 (the value from the previous row).

o When capacity is 3, we can either exclude item 2 or include it. Excluding it gives us the value of 6 (value from the previous row). Including it, we get a total value of 10 (value of item 2) + value from the remaining capacity (3 - 2 = 1), which is 0. So, the maximum value is 10.

o For capacities 4 and 5, we can take item 2 (Weight = 2) as it fits, so the maximum value is 10.

Table 7.3 shows the filling entries when item 1 is selected, having weight=2 and value=10.

0

1

2

3

4

5

0

0

0

0

0

0

0

1

0

0

0

6

6

6

2

0

0

10

10

16

16

3

0

Table 7.3: Filling table for item 2

The next phase fills the table with the parameters for item number 3.

Item 3 (Weight = 3, Value = 12):

• For each capacity from 0 to 5:

o When capacity is 0, we cannot take any item. So, the value remains 0.

o For capacities 1 and 2, we cannot take item 3, so the value remains 10 (the value from the previous row).

o When capacity is 3, we can either exclude item 3 or include it. Excluding it gives us the value of 10 (value from the previous row). Including it, we get a total value of 12 (value of item 3) + value from the remaining capacity (3 - 3 = 0), which is 0. So, the maximum value is 12.

o For capacities 4 and 5, we can either exclude item 3 or include it. Excluding it gives us the value of 16 (value from the previous row). Including it, we get a total value of 12 (value of item 3) + value from the remaining capacity (4 - 3 = 1), which is 6. So, the maximum value is 18.

Table 7.4 shows the filling entries when item 1 is selected, having weight=3 and value=12.

0

1

2

3

4

5

0

0

0

0

0

0

0

1

0

0

0

6

6

6

2

0

0

10

10

16

16

3

0

0

12

12

16

18

Table 7.4: Filling table for item 3

3. Retrieve optimal solution: The maximum value that can be obtained is 18, which is in cell dp[3][5]. To retrieve the selected items, we backtrack from this cell to the first row:

• Starting from cell dp[3][5], we see that the value is different from the value in the cell, dp[2][5], above it. This means that item 3 (Weight = 3, Value = 12) was selected.

• We move diagonally up and left to cell dp[2][2]. The value here is different from the value in the cell, dp[1][2] above it. This means that item 2 (Weight = 2, Value = 10) was selected.

• We move diagonally up and left to cell dp[1][0]. The value here is the same as the value in the cell above it dp[0][0]. This means that item 1 (Weight = 2, Value = 6) was not selected.

So, the selected items are items 2 and 3.

This completes the step-by-step solution of the 0/1 knapsack problem.

Complexity analysis

The time complexity of the DP solution for the 0/1 knapsack problem is O(n*capacity), where n is the number of items and capacity is the maximum capacity of the knapsack.

Let us consider a few applications. The 0/1 knapsack problem has various real-world applications, such as resource allocation, portfolio optimization, and project selection. It is commonly used in fields like operations research, finance, and computer science for solving optimization problems involving resource constraints.

Subset sum problem

The subset sum problem is a classic algorithmic problem in computer science. Given a set of positive integers and a target sum, the task is to determine whether there exists a subset of the given set whose elements sum up to the target sum.

Algorithm 7.3: Subset sum problem

Function subsetSum(set[], n, target_sum):

Create a 2D array dp[n+1][target_sum+1] and initialize it with false values.

Set dp[0][0] = true, as an empty subset can sum up to 0.

For i = 1 to n:

For j = 0 to target_sum:

If set[i-1] > j:

Set dp[i][j] = dp[i-1][j] (current element cannot be included)

Else:

Set dp[i][j] = dp[i-1][j] OR dp[i-1][j-set[i-1]] (include or exclude the current element)

Return dp[n][target_sum]

Program 7.2: Implementation in C++

#include <iostream>

#include <vector>

using namespace std;

bool subsetSum(vector<int>& set, int target_sum) {

int n = set.size();

vector<vector<bool>> dp(n + 1, vector<bool>(target_sum + 1, false));

// Base case: an empty subset can sum up to 0

dp[0][0] = true;

for (int i = 1; i <= n; ++i) {

for (int j = 0; j <= target_sum; ++j) {

if (set[i - 1] > j) {

dp[i][j] = dp[i - 1][j]; // Exclude current element

} else {

dp[i][j] = dp[i - 1][j] || dp[i - 1][j - set[i - 1]]; // Include or exclude current element

}

}

}

return dp[n][target_sum];

}

int main() {

vector<int> set = {3, 34, 4, 12, 5, 2};

int target_sum = 9;

if (subsetSum(set, target_sum)) {

cout << "Subset with the given sum exists." << endl;

} else {

cout << "Subset with the given sum does not exist." << endl;

}

return 0;

}

Example 7.2: Consider the set {3, 34, 4, 12, 5, 2} and the target sum 9. Check whether a subset with a given sum exists or not.

Solution: It involves the following steps:

1. Initialization: Create a 2D DP table with dimensions (n + 1) x (target_sum + 1), where n is the number of elements in the set and target_sum is the given target sum, shown as follows:

Subset sum

0

1

2

3

4

5

6

7

8

9

{} (0)

T

F

F

F

F

F

F

F

F

F

3

T

34

T

4

T

12

T

5

T

2

T

Table 7.5: Initialization of the subset sum problem

2. DP: For each element in the set, we iterate over the target sum and update the DP table according to the recurrence relation.

For element 3:

When considering 3, we update the DP table as follows:

Subset sum

0

1

2

3

4

5

6

7

8

9

{} (0)

T

F

F

F

F

F

F

F

F

F

3

T

F

F

T

F

F

F

F

F

F

34

T

4

T

12

T

5

T

2

T

Table 7.6: Subset sum for element 3

For element 34:

When considering 34, we update the DP table as follows:

Subset sum

0

1

2

3

4

5

6

7

8

9

{} (0)

T

F

F

F

F

F

F

F

F

F

3

T

F

F

T

F

F

F

F

F

F

34

T

F

F

T

F

F

F

F

F

F

4

T

12

T

5

T

2

T

Table 7.7: Subset sum for element 34

For element 4:

When considering 4, we update the DP table as follows:

Subset sum

0

1

2

3

4

5

6

7

8

9

{} (0)

T

F

F

T

F

F

F

F

F

F

3

T

F

F

T

F

F

F

F

F

F

34

T

F

F

T

F

F

F

F

F

F

4

T

F

F

T

T

F

F

F

F

F

12

T

5

T

2

T

Table 7.8: Subset sum for element 4

For element 12:

When considering 12, we update the DP table as follows:

Subset sum

0

1

2

3

4

5

6

7

8

9

{} (0)

T

F

F

T

F

F

F

F

F

F

3

T

F

F

T

F

F

F

F

F

F

34

T

F

F

T

F

F

F

F

F

F

4

T

F

F

T

T

F

F

F

F

F

12

T

F

F

T

T

F

F

F

F

F

5

T

2

T

Table 7.9: Subset sum for element 12

For element 5:

When considering 5, we update the DP table as follows:

Subset sum

0

1

2

3

4

5

6

7

8

9

{} (0)

T

F

F

F

F

F

F

F

F

F

3

T

F

F

T

F

F

F

F

F

F

34

T

F

F

T

F

F

F

F

F

F

4

T

F

F

T

T

F

F

F

F

F

12

T

F

F

T

T

F

F

F

F

F

5

T

F

F

T

T

T

F

F

F

T

2

T

Table 7.10: Subset sum for element 5

For the remaining elements in the set, repeat this process and update the table as follows:

Subset sum

0

1

2

3

4

5

6

7

8

9

{} (0)

T

F

F

F

F

F

F

F

F

F

3

T

F

F

T

F

F

F

F

F

F

34

T

F

F

T

F

F

F

F

F

F

4

T

F

F

T

T

F

F

F

F

F

12

T

F

F

T

T

F

F

F

F

F

5

T

F

F

T

T

T

F

F

F

T

2

T

F

T

T

T

T

T

F

F

T

Table 7.11: Subset sum for element 2

Traceback to find the subset:

• Start at dp[6][9] (which is True).

• Traceback through the table:

Check if the value came from dp[i-1][j] or dp[i-1][j-arr[i-1]].

From the table:

• dp[6][9] is True because of dp[5][7].

• dp[5][7] is True because of dp[4][4].

• dp[4][4] is True because of dp[3][4].

• dp[3][4] is True because of dp[2][0].

So, the subset that gives the sum 9 is {3, 4, 2}.

Levenshtein distance or edit distance

Levenshtein distance, or edit distance, quantifies the similarity between two strings by determining the minimum number of single-character edits needed to transform one string into another. These edits may include insertions, deletions, or substitutions of characters.

DP is a widely adopted method for computing the Levenshtein distance efficiently. The process involves creating a matrix where each cell represents the distance between the substrings of the two strings being compared. It functions according to the following algorithm.

Algorithm 7.4: Levenshtein distance

1. Initialize the matrix: Create a matrix DP with dimensions (m+1) x (n+1), where m is the length of the first string and n is the length of the second string.

2. Set base cases:

a. dp[i][0] = i for all i from 0 to m (distance from string of length i to an empty string).

b. dp[0][j] = j for all j from 0 to n (distance from an empty string to a string of length j).

c. Fill the matrix: For each cell dp[i][j], compute the value based on the following rules:

i. If the characters of the strings at positions i-1 and j-1 are the same, dp[i][j] = dp[i-1][j-1].

ii. If the characters are different, compute the minimum cost of the three possible edits:

• Insertion: dp[i][j-1] + 1

• Deletion: dp[i-1][j] + 1

• Substitution: dp[i-1][j-1] + 1

iii. Set dp[i][j] to the minimum of these values.

d. Retrieve the result: The value in the bottom-right cell of the matrix dp[m][n] is the Levenshtein distance between the two strings.

Program 7.3: Implementation in C++

#include <iostream>

#include <vector>

#include <algorithm>

using namespace std;

// Function to compute the Levenshtein Distance between two strings

int levenshteinDistance(const string& str1, const string& str2) {

int m = str1.size();

int n = str2.size();

// Create a 2D vector to store the distances

vector<vector<int>> dp(m + 1, vector<int>(n + 1));

// Initialize the base cases

for (int i = 0; i <= m; ++i) {

dp[i][0] = i; // Distance from str1[0..i-1] to an empty str2

}

for (int j = 0; j <= n; ++j) {

dp[0][j] = j; // Distance from an empty str1 to str2[0..j-1]

}

// Fill the rest of the dp matrix

for (int i = 1; i <= m; ++i) {

for (int j = 1; j <= n; ++j) {

if (str1[i - 1] == str2[j - 1]) {

dp[i][j] = dp[i - 1][j - 1]; // No change needed

} else {

dp[i][j] = min({

dp[i - 1][j] + 1, // Deletion

dp[i][j - 1] + 1, // Insertion

dp[i - 1][j - 1] + 1 // Substitution

});

}

}

}

// The bottom-right cell contains the Levenshtein Distance

return dp[m][n];

}

int main() {

string str1 = "kitten";

string str2 = "sitting";

int distance = levenshteinDistance(str1, str2);

cout << "The Levenshtein Distance between \"" << str1 << "\" and \"" << str2 << "\" is: " << distance << endl;

return 0;

}

Example 7.3: Consider the strings kitten and sitting.

Solution: Initialization:

s

i

t

t

i

n

g

0

1

2

3

4

5

6

7

k

1

i

2

t

3

t

4

e

5

n

6

Table 7.12: Kitten and sitting initialization

Base condition:

s

i

t

t

i

n

g

0

1

2

3

4

5

6

7

k

1

i

2

t

3

t

4

e

5

n

6

Table 7.13: Base condition filling

Filling the matrix:

s

i

t

t

i

n

g

0

1

2

3

4

5

6

7

k

1

1

2

3

4

5

6

7

i

2

2

1

2

3

4

5

6

t

3

3

2

1

2

3

4

5

t

4

4

3

2

1

2

3

4

e

5

5

4

3

2

2

3

4

n

6

6

5

4

3

3

2

3

Table 7.14: Matrix filling

Final matrix and result:

s

i

t

t

i

n

g

0

1

2

3

4

5

6

7

k

1

1

2

3

4

5

6

7

i

2

2

1

2

3

4

5

6

t

3

3

2

1

2

3

4

5

t

4

4

3

2

1

2

3

4

e

5

5

4

3

2

2

3

4

n

6

6

5

4

3

3

2

3

Table 7.15: Resultant matrix

The Levenshtein distance between kitten and sitting is 3, which is the value in the bottom-right cell of the matrix. This means three edits are required to transform kitten into sitting:

• Substitute k with s

• Substitute e with i

• Insert g at the end

Complexity analysis

The Levenshtein distance (edit distance) algorithm, implemented using DP, has specific computational complexities in terms of time and space. The following is a detailed analysis:

• Time complexity: The time complexity of the Levenshtein distance algorithm is determined by the number of operations performed to fill the DP table.

o Initialization: Initializing the DP table requires O(m + n) operations, where m is the length of the first string and n is the length of the second string.

o Filling the DP table: The main computation involves filling a table of size (m+1) x (n+1).

For each cell dp[i][j], a constant amount of work (comparing characters and taking the minimum of three values) is performed.

Therefore, the total work done to fill the table is O(m x n).

Combining these steps, the overall time complexity is O(m x n).

• Space complexity: The space complexity of the algorithm is determined by the amount of memory required to store the DP table.

o The DP table requires (m+1) x (n+1) cells.

o Each cell stores a single integer value, resulting in a space complexity of O(m x n).

Optimized space complexity: In some implementations, it is possible to optimize the space complexity by using only two rows (or columns) of the DP table at any time because the calculation of each cell, dp[i][j], only depends on the current and previous rows (or columns).

Two-row optimization: By maintaining only two rows (or two columns) of the DP table at any time, the space complexity can be reduced to O(min(m, n)).

This is particularly useful when one string is much shorter than the other.

Summary is as follows:

• Time complexity: O(m x n))

• Space complexity: O(m x n) (can be optimized to O(min(m, n))

Let us look at the detailed complexity analysis and break down the complexity further for clarity:

• Time complexity analysis

o Initialization of the DP table: O(m + n)

Initialize the first row: O(n)

Initialize the first column: O(m)

o Filling the DP table: O(m x n)

For each cell dp[i][j] (where 1≤ i ≤ m and 1 ≤ j ≤n):

 Compare characters str1[i-1] and str2[j-1].

 Compute the minimum of the three values (insertion, deletion, substitution).

• Space complexity analysis

o DP table storage: O(m x n)

The table size is (m+1) x (n+1).

o Optimized storage (using two rows or columns): O(min(m, n))

Only two rows (or columns) are maintained at any time.

Optimal binary search tree

Let us assume that the given set of identifiers is {a1, . . . , an} with a1 < a2 < …< an. Let p(i) be the probability with which we search for ai. Let q(i) be the probability that the identifier x being searched for is such that ai < x < ai+1, 0 < i < n (assume a0 = -∞ and an+1 = +∞). We have to arrange the identifiers in a binary search tree in a way that minimizes the expected total access time.

In a binary search tree, the number of comparisons needed to access an element at depth d is d + 1, so if ai is placed at depth di, then we want to minimize:

Let P(i) be the probability with which we shall be searching for ai. Let Q(i) be the probability of an unsuccessful search. Every internal node represents a point where a successful search may terminate. Every external node represents a point where an unsuccessful search may terminate.

The expected cost contribution for the internal node for ai is:

P(i)* level (ai)

An unsuccessful search terminates with i = 0 (i.e., at an external node). Hence, the cost contribution for this node is:

Q(i)* level ((Ei)- 1)

The expected cost of the binary search tree is:

The total time to evaluate all the c(i, j)’s and r(i, j)’s is therefore:

Example 7.4: Consider four elements a1, a2, a3 and a4 with Q0 = 1/8, Q1 = 3/16, Q2 = Q3 = Q4 = 1/16 and p1 = 1/4, p2 = 1/8, p3 = p4 =1/16. Construct an optimal binary search tree. Solving for C(0, n):

Solution: First, computing all C(i, j), such that j - i = 1; j = i + 1 and as 0 < i < 4; i = 0, 1, 2 and 3; i < k ≤ J. Start with i = 0; so j = 1; as i < k ≤ j, so the possible value for k = 1

W (0, 1) = P (1) + Q (1) + W (0, 0) = 4 + 3 + 2 = 9

C (0, 1) = W (0, 1) + min {C (0, 0) + C (1, 1)} = 9 + [(0 + 0)] = 9

R (0, 1) = 1 (value of K that is minimum in the above equation)

Next with i = 1; so, j = 2; as i < k ≤ j, so the possible value for k = 2 W (1, 2) = P (2) + Q (2) + W (1, 1) = 2 + 1 + 3 = 6

C (1, 2) = W (1, 2) + min {C (1, 1) + C (2, 2)} = 6 + [(0 + 0)] = 6

R (1, 2) = 2

Next with i = 2; so, j = 3; as i < k ≤ j, so the possible value for k = 3 W (2, 3) = P (3) + Q (3) + W (2, 2) = 1 + 1 + 1 = 3

C (2, 3) = W (2, 3) + min {C (2, 2) + C (3, 3)} = 3 + [(0 + 0)] = 3

R (2, 3) = 3

Next with i = 3; so, j = 4; as i < k ≤ j, so the possible value for k = 4 W (3, 4) = P (4) + Q (4) + W (3, 3) = 1 + 1 + 1 = 3

C (3, 4) = W (3, 4) + min {[C (3, 3) + C (4, 4)]} = 3 + [(0 + 0)] = 3

R (3, 4) = 4

Second, computing all C (i, j), such that j - i = 2; j = i + 2 and as 0 < i < 3; i = 0, 1, 2; i < k ≤ J

Start with i = 0; so, j = 2; as i < k ≤ j, so the possible values for k = 1 and 2. W (0, 2) = P (2) + Q (2) + W (0, 1) = 2 + 1 + 9 = 12

C (0, 2) = W (0, 2) + min {(C (0, 0) + C (1, 2)), (C (0, 1) + C (2, 2))}

= 12 + min {(0 + 6, 9 + 0)} = 12 + 6 = 18

R (0, 2) = 1

Next, with i = 1; so, j = 3; as i < k ≤ j, so the possible value for k = 2 and 3

W (1, 3) = P (3) + Q (3) + W (1, 2) = 1 + 1+ 6 = 8

C (1, 3) = W (1, 3) + min {[C (1, 1) + C (2, 3)], [C (1, 2) + C (3, 3)]}

= W (1, 3) + min {(0 + 3), (6 + 0)} = 8 + 3 = 11

R (1, 3) = 2

Next, with i = 2; so, j = 4; as i < k ≤ j, so the possible value for k = 3 and 4. W (2, 4) = P (4) + Q (4) + W (2, 3) = 1 + 1 + 3 = 5

C (2, 4) = W (2, 4) + min {[C (2, 2) + C (3, 4)], [C (2, 3) + C (4, 4)]}

= 5 + min {(0 + 3), (3 + 0)} = 5 + 3 = 8

R (2, 4) = 3

Third, computing all C(i, j), such that J - i = 3; j = i + 3 and as 0 < i < 2; i = 0, 1; i < k ≤ J. Start with i = 0; so, j = 3; as i < k ≤ j, so the possible values for k = 1, 2 and 3.

W (0, 3) = P (3) + Q (3) + W (0, 2) = 1 + 1 + 12 = 14

C (0, 3) = W (0, 3) + min {[C (0, 0) + C (1, 3)], [C (0, 1) + C (2, 3)], [C (0, 2) + C (3, 3)]}

= 14 + min {(0 + 11), (9 + 3), (18 + 0)} = 14 + 11 = 25

R (0, 3) = 1

Start with i = 1; so, j = 4; as i < k ≤ j, so the possible values for k = 2, 3 and 4. W (1, 4) = P (4) + Q (4) + W (1, 3) = 1 + 1 + 8 = 10

C (1, 4) = W (1, 4) + min {[C (1, 1) + C (2, 4)], [C (1, 2) + C (3, 4)], [C (1, 3) + C (4, 4)]}

= 10 + min {(0 + 8), (6 + 3), (11 + 0)} = 10 + 8 = 18

R (1, 4) = 2

Fourth, computing all C (i, j) such that J - i = 4; j = i + 4 and as 0 < i < 1; i = 0; i < k ≤ J. Start with i = 0; so, j = 4; as i < k ≤ j, so the possible values for k = 1, 2, 3 and 4.

W (0, 4) = P (4) + Q (4) + W (0, 3) = 1 + 1 + 14 = 16

C (0, 4) = W (0, 4) + min {[C (0, 0) + C (1, 4)], [C (0, 1) + C (2, 4)], [C (0, 2) + C (3, 4)], [C (0, 3) + C (4, 4)]}

= 16 + min [0 + 18, 9 + 8, 18 + 3, 25 + 0] = 16 + 17 = 33

R (0, 4) = 2

The table for recording W (i, j), C (i, j) and R (i, j) is as follows:

Column row

0

1

2

3

4

0

2,0,0

1,0,0

1,0,0

1,0,0

1,0,0

1

9,9,1

6,6,2

3,3,3

3,3,4

2

12,18,1

8,11,2

5,8,3

3

14,25,2

11,18,2

4

16,33,2

Table 7.16: Recording table for variable W, C, R

From the preceding table, we see that C (0, 4) = 33 is the minimum cost of a binary search tree for (a1, a2, a3, a4).

The root of the tree T04 is a2.

Hence, the left subtree is T01, and the right subtree is T24. The root of T01 is a1, and the root of T24 is a3.

The left and right subtrees for T01 are T00 and T11, respectively. The root of T01 is a1

The left and right subtrees for T24 are T22 and T34, respectively. The root of T24 is a3.

The root of T22 is null. The root of T34 is a4.

The following figure shows the final structure of the optimal binary search tree:

Figure 7.1: Final optimal binary search tree

Matrix chain multiplication

It is a method under DP in which the previous output is taken as input for the next. Here, chain means one matrix's column is equal to the second matrix's row (always).

In general:

If A = ⌊aij⌋ is a p x q matrix, B = ⌊bij⌋ is a q x r matrix, C = ⌊cij⌋ is a p x r matrix.

Then,

Given following matrices {A1,A2,A3,...An} and we have to perform the matrix multiplication, which can be accomplished by a series of matrix multiplications:

A1 x A2 x, A3 x.....x An

The matrix multiplication operation is associative rather than commutative in nature. By this, we mean that we have to follow the preceding matrix order for multiplication, but we are free to parenthesize the multiplication depending upon our needs.

In general, for 1≤ i ≤ p and 1 ≤ j ≤ r:

It can be observed that the total entries in matrix C is pr as the matrix is of dimension p x r. Also, each entry takes O(q) time to compute. Thus, the total time to compute all possible entries for the matrix C, which is a multiplication of A and B, is proportional to the product of the dimensions p, q, and r.

It is also noticed that we can save the number of operations by reordering the parentheses.

Example 7.5: Let us have three matrices, A1, A2, A3 of order (10 x 100), (100 x 5) and (5 x 50) respectively.

Solution: Three matrices can be multiplied in two ways:

1. A1,(A2,A3): First multiplying (A2 and A3) then multiplying and resultant with A1.

2. (A1,A2),A3: First multiplying (A1 and A2) then multiplying and resultant with A3.

The number of scalar multiplication in case 1 will be:

(100 x 5 x 50) + (10 x 100 x 50) = 25000 + 50000 = 75000

The number of scalar multiplication in case 2 will be:

(100 x 10 x 5) + (10 x 5 x 50) = 5000 + 2500 = 7500

To find the best possible way to calculate the product, we could simply parenthesize the expression in every possible fashion and count each time how many scalar multiplications are required.

The matrix chain multiplication problem can be stated as finding the optimal parenthesization of a chain of matrices to be multiplied such that the number of scalar multiplication is minimized.

Number of ways for parenthesizing the matrices

There are numerous ways of parenthesizing these matrices. If there are n items, there are (n-1) ways in which the outermost pair of parentheses can be placed:

(A1) (A2,A3,A4,................An)

Or (A1,A2) (A3,A4 .................An)

Or (A1,A2,A3) (A4 ...............An)

Or (A1,A2,A3.............An-1) (An)

It can be observed that after splitting the kth matrices, we are left with two parenthesized sequences of matrices: one consists of k matrices, and another consists of n-k matrices.

Now there are L ways of parenthesizing the left sub-list and R ways of parenthesizing the right sub-list, then the total will be L.R:

Also, p (n) = c (n-1), where c (n) is the nth Catalan number:

On applying Stirling’s formula, we have,

This shows that 4n grows faster, as it is an exponential function, than n1.5.

The development of dynamic programming consists of the following steps:

1. Characterize the structure of an optimal solution.

2. Define the value of an optimal solution recursively.

3. Compute the value of an optimal solution in a bottom-up fashion.

4. Construct the optimal solution from the computed information.

Dynamic programming approach

Let Ai,j be the result of multiplying matrices i through j. It can be seen that the dimension of Ai,j is pi-1 x pj matrix.

DP solution involves breaking up the problems into subproblems whose solutions can be combined to solve the global problem.

At the greatest level of parenthesization, we multiply two matrices, such that:

A1.....n=A1....k x Ak+1....n

Thus, we are left with the following two questions:

1. How to split the sequence of matrices?

2. How to parenthesize the subsequence A1.....k and Ak+1......n?

One possible answer to the first question for finding the best value of k is to check all possible choices of k and consider the best among them. It can be observed that checking all possibilities will lead to an exponential number of total possibilities. It can also be noticed that there exist only O(n2) different sequences of matrices. In this way, they do not reach exponential growth.

1. Structure of an optimal parenthesization: The first step in the dynamic paradigm is to find the optimal substructure and then use it to construct an optimal solution to the problem from an optimal solution to subproblems.

Let Ai....j where i≤ j denotes the matrix that results from evaluating the product.

Ai Ai+1....Aj

If i < j, then any parenthesization of the product Ai Ai+1 ......Aj must split so that the product between Ak and Ak+1 for some integer k is in the range i ≤ k ≤ j. That is for some value of k, we first compute the matrices Ai.....k and Ak+1....j and then multiply them together to produce the final product Ai....j. The cost of computing Ai....k plus the cost of computing Ak+1....j plus the cost of multiplying them together is the cost of parenthesization.

2. A recursive solution: Let m[i, j] be the minimum number of scalar multiplication needed to compute the matrix Ai....j.

If i=j, the chain consists of just one matrix Ai....i=Ai so no scalar multiplication is necessary to compute the product. Thus, m[i, j] = 0 for i= 1, 2, 3....n.

If i<j, we assume that to optimally parenthesize the product, we split it between Ak and Ak+1 where i≤ k ≤j. Then m[i,j] equals the minimum cost for computing the subproducts Ai....k and Ak+1....j+ cost of multiplying them together. We know Ai has dimension pi-1 x pi, so computing the product Ai....k and Ak+1....j takes pi-1 pk pj scalar multiplication we obtain the following:

m [i,j] = m [i, k] + m [k + 1, j] + pi-1 pk pj

There are only (j-1) possible values for k, namely k = i, i+1.....j-1. Since the optimal parenthesization must use one of these values for k, we only need to check them all to find the best.

So, the minimum cost of parenthesizing the product Ai Ai+1......Aj becomes:

To construct an optimal solution, let us define s[i,j] to be the value of k at which we can split the product Ai Ai+1 .....Aj To obtain an optimal parenthesization, i.e., s [i, j] = k, such that:

m [i,j] = m [i, k] + m [k + 1, j] + pi-1 pk pj

MATRIX-CHAIN-ORDER (P)

1 n = p.length -1

2 let m[1 . . . n , 1 . . . n] and s[1 . . . n - 1 , 2 . . . n] be new tables

3 for i = 1 to n

4 m[ i, i ] = 0

5 for l = 2 to n // l is the chain length

6 for i = 1 to n – l + 1

7 j = i + l – 1

8 m[ i , j ] = ….

9 for k = i to j – 1

10 q = m[ i, k ] + m[ k + 1, j ] + Pi-1 Pk Pj

11 if q < m[ i, j ]

12 m[ i, j ] = q

13 s[ i, j ] = k

14 return m and s

The following figure represents two different tables, where table m and table s are defined for the matrix chain having n=6:

Figure 7.2: Input matrix chain for n=6

The m and s tables are computed by MATRIX-CHAIN-ORDER for n = 6, and the following matrix dimensions are calculated:

Matrix

A1 A2 A3 A4 A5 A6

Dimension

30 X 35 35 X 15 15 X 5 5 X 10 10 X 20 20 X 25

Table 7.17: Matrix dimensions calculation

The tables are rotated so that the main diagonal runs horizontally. The m table uses only the main diagonal and upper triangle, and the s table uses only the upper triangle. The minimum number of scalar multiplications needed to multiply the six matrices is m[1, 6] = 15,125. On the darker entries, the pairs that have the same shading are taken together in line 10 when computing.

PRINT-OPTIMAL_PARENS (s, i, j)

In the example, the call PRINT-OPTIMAL_PARENS (s, 1, 6) prints the parenthesization ((A1(A2A3)) ((A4A5) A6)), calculated as follows:

1. if i == j

2. print “A” i

3. else print “(“

4. PRINT-OPTIMAL_PARENS (s, i, s[i, j])

5. PRINT-OPTIMAL_PARENS (s, s[i, j] + 1, j)

6. print “)”

Example 7.6: We are given the sequence {4, 10, 3, 12, 20, and 7}. The matrices have size 4 x 10, 10 x 3, 3 x 12, 12 x 20, 20 x 7. We need to compute M [i,j], 0 ≤ i, j≤ 5. We know M [i, i] = 0 for all i.

Solution: The initial case of matrix chain multiplication is mentioned in Figure 7.3:

Figure 7.3: Initial matrix with m[i,i]=0

Let us proceed with working away from the diagonal. We compute the optimal solution for the product of 2 matrices.

Figure 7.4 shows the sequence for the next step of multiplication:

Figure 7.4: Sequence for chaining multiplication

Here, P0 to P5 are the positions, and M1 to M5 are the matrix of size (pi to pi-1).

On the basis of the sequence, we make the following formula:

For Mi p [i] as column

p[i-1] as row

In DP, the initialization of every method is done by 0. So, we initialize it by 0. It will sort out diagonally.

We have to sort out all the combinations, but the minimum output combination is taken into consideration.

Calculation of the product of two matrices is as follows:

1. m (1,2) = m1 x m2

= 4 x 10 x 10 x 3

= 4 x 10 x 3 = 120

2. m (2, 3) = m2 x m3

= 10 x 3 x 3 x 12

= 10 x 3 x 12 = 360

3. m (3, 4) = m3 x m4

= 3 x 12 x 12 x 20

= 3 x 12 x 20 = 720

4. m (4,5) = m4 x m5

= 12 x 20 x 20 x 7

= 12 x 20 x 7 = 1680

Figure 7.5 shows the multiplication result of 2 matrices:

Figure 7.5: Product matrix of 2 matrices

In the matrix:

• We initialize the diagonal element with equal i, j value with 0.

• After that, the second diagonal is sorted out, and we get all the values corresponding to it.

Now, the third diagonal will be solved in the same way.

Product of three matrices is as follows:

M [1, 3] = M1 M2 M3

1. There are two cases by which we can solve this multiplication: (M1 x M2) + M3, M1+ (M2x M3)

2. After solving both cases, we choose the case with the minimum output.

Comparing the outputs, 264 is the minimum in both cases, so we insert 264 in the table and (M1 x M2) + M3. This combination is chosen for the output making.

M [2, 4] = M2 M3 M4

1. There are two cases by which we can solve this multiplication: (M2x M3)+M4, M2+(M3 x M4)

2. After solving both cases, we choose the case with the minimum output.

Comparing both the outputs, 1320 is the minimum in both cases, so we insert 1320 in the table and M2+(M3 x M4). This combination is chosen for the output making.

M [3, 5] = M3 M4 M5

1. There are two cases by which we can solve this multiplication: (M3 x M4) + M5, M3+ ( M4xM5)

2. After solving both cases, we choose the case with the minimum output.

Comparing both the outputs, 1140 is the minimum in both cases, so we insert 1140 in the table and (M3 x M4) + M5.This combination is chosen for the output making.

Figure 7.6 shows the multiplication result when we take three matrices as a chain:

Figure 7.6: Chain result up to 3 matrices

Product of four matrices is as follows:

M[1, 4] = M1 M2 M3 M4

There are three cases by which we can solve this multiplication:

1. (M1 x M2 x M3) M4

2. M1 x(M2 x M3 x M4)

3. (M1 xM2) x ( M3 x M4)

After solving these cases, we choose the case with the minimum output.

Comparing the output of different cases, 1080 is the minimum output, so we insert 1080 in the table, and (M1 xM2) x (M3 x M4) combination is taken out in output making.

M[2, 5] = M2 M3 M4 M5

There are three cases by which we can solve this multiplication:

1. (M2 x M3 x M4)x M5

2. M2 x( M3 x M4 x M5)

3. (M2 x M3)x ( M4 x M5)

After solving these cases, choose the case with the minimum output:

Comparing the output of different cases, 1350 is the minimum output, so we insert 1350 in the table, and M2 x( M3 x M4 xM5) combination is taken out in output making, shown as follows:

Figure 7.7: Chain result up to 4 matrices

Product of five matrices is as follows:

M [1, 5] = M1 M2 M3 M4 M5

There are five cases by which we can solve this multiplication:

1. (M1 x M2 xM3 x M4 )x M5

2. M1 x( M2 xM3 x M4 xM5)

3. (M1 x M2 xM3)x M4 xM5

4. M1 x M2x(M3 x M4 xM5)

After solving these cases, we choose the case with the minimum output, shown as follows:

Comparing the output of the different cases, 1344 is the minimum output, so we insert 1344 in the table, and M1 x M2 x(M3 x M4 x M5) combination is taken out in output making.

The final output is as follows:

Figure 7.8: Chain result up to 5 matrices

Computing optimal costs

Let us assume that matrix Ai has dimension pi-1xpi for i=1, 2, 3....n. The input is a sequence (p0,p1,......pn), where length [p] = n+1. The procedure uses an auxiliary table m[1....n, 1.....n] for storing m[i, j] costs and an auxiliary table s[1.....n, 1.....n] that records which index of k achieved the optimal costs in computing m[i, j].

The algorithm first computes m[i, j] ← 0 for i=1, 2, 3.....n, the minimum costs for the chain of length 1.

Longest common subsequence problem

The longest common subsequence problem is finding the longest sequence that exists in both the given strings.

Let us consider a sequence S = <s1, s2, s3, s4, …,sn>.

A sequence Z = <z1, z2, z3, z4, …,zm> over S is called a subsequence of S, if and only if it can be derived from S deletion of some elements.

For the common subsequence, suppose X and Y are two sequences over a finite set of elements. We can say that Z is a common subsequence of X and Y, if Z is a subsequence of both X and Y.

Let us look at the longest common subsequence. If a set of sequences is given, the longest common subsequence problem is to find a common subsequence that is of the maximal length of all the sequences.

The longest common subsequence problem is a classic computer science problem, the basis of data comparison programs such as the diff-utility, and has applications in bioinformatics. It is also widely used by revision control systems, such as SVN and Git, for reconciling multiple changes made to a revision-controlled collection of files.

For the naïve method, let X be a sequence of length m and Y a sequence of length n. Check for every subsequence of X, whether it is a subsequence of Y, and return the longest common subsequence found.

There are 2m subsequences of X. Testing if the sequences are a subsequence of Y takes O(n) time. Thus, the naive algorithm would take O(n2m) time.

Dynamic programming

Let X = < x1, x2, x3,…, xm > and Y = < y1, y2, y3,…, yn > be the sequences. To compute the length of an element, the following algorithm is used:

In this procedure, table C[m, n] is computed in row-major order, and another table B[m,n] is computed to construct an optimal solution.

Algorithm 7.5: Longest common sequence

LCS-LENGTH (X, Y)

1. m ← length [X]

2. n ← length [Y]

3. for i ← 1 to m

4. do c [i,0] ← 0

5. for j ← 0 to m

6. do c [0,j] ← 0

7. for i ← 1 to m

8. do for j ← 1 to n

9. do if xi= yj

10. then c [i,j] ← c [i-1,j-1] + 1

11. b [i,j] ← "↖"

12. else if c[i-1,j] ≥ c[i,j-1]

13. then c [i,j] ← c [i-1,j]

14. b [i,j] ← "↑"

15. else c [i,j] ← c [i,j-1]

16. b [i,j] ← "← "

17. return c and b

Example 7.7: Given two sequences X [1...m] and Y [1.....n]. Find the longest common sub-sequences to both.

x: A B C B D A B

y: B D C A B A

Solution: Here, X = (A,B,C,B,D,A,B) and Y = (B,D,C,A,B,A)

m = length [X] and n = length [Y]

m = 7 and n = 6

Here, x1= x [1] = A y1= y [1] = B

x2= B y2= D

x3= C y3= C

x4= B y4= A

x5= D y5= B

x6= A y6= A

x7= B

Now, fill the values of c [i, j] in m x n table.

Initially, for i=1 to 7 c [i, 0] = 0

For j = 0 to 6 c [0, j] = 0

It is shown as follows:

Figure 7.9: Initial case of longest common sequence

Now for i=1 and j = 1

x1 and y1 we get x1 ≠ y1 i.e. A ≠ B

And, c [i-1,j] = c [0, 1] = 0

c [i, j-1] = c [1,0 ] = 0

That is, c [i-1,j]= c [i, j-1] so c [1, 1] = 0 and b [1, 1] = ' ↑ '

Now for i=1 and j = 2

x1 and y2 we get x1 ≠ y2 i.e. A ≠ D

c [i-1,j] = c [0, 2] = 0

c [i, j-1] = c [1,1 ] = 0

That is, c [i-1,j]= c [i, j-1] and c [1, 2] = 0 b [1, 2] = ' ↑ '

Now for i=1 and j = 3

x1 and y3 we get x1 ≠ y3 i.e. A ≠ C

c [i-1,j] = c [0, 3] = 0

c [i, j-1] = c [1,2 ] = 0

So, c [1,3] = 0 b [1,3] = ' ↑ '

Now for i=1 and j = 4

x1 and y4 we get x1=y4 i.e A = A

c [1,4] = c [1-1,4-1] + 1

= c [0, 3] + 1

= 0 + 1 = 1

c [1,4] = 1

b [1,4] = ' ↖ '

Now for i=1 and j = 5

x1 and y5 we get x1 ≠ y5

c [i-1,j] = c [0, 5] = 0

c [i, j-1] = c [1,4 ] = 1

Thus, c [i, j-1] > c [i-1,j], i.e., c [1, 5] = c [i, j-1] = 1. So b [1, 5] = '←'

Now for i=1 and j = 6

x1 and y6 we get x1=y6

c [1, 6] = c [1-1,6-1] + 1

= c [0, 5] + 1 = 0 + 1 = 1

c [1,6] = 1

b [1,6] = ' ↖ '

Figure 7.10 represents the table for 1st pass i = 1:

Figure 7.10: Longest common sequence table for i=1

Now for i=2 and j = 1

We get x2 and y1 B = B i.e. x2= y1

c [2,1] = c [2-1,1-1] + 1

= c [1, 0] + 1

= 0 + 1 = 1

c [2, 1] = 1 and b [2, 1] = ' ↖ '

Similarly, we fill all the values of c [i, j], and we get the following matrix:

Figure 7.11: Construction of matrix for remaining values of i

Construct an LCS. The initial call is PRINT-LCS (b, X, X.length, Y.length)

PRINT-LCS (b, x, i, j)

1. if i=0 or j=0

2. then return

3. if b [i,j] = ' ↖ '

4. then PRINT-LCS (b,x,i-1,j-1)

5. print x_i

6. else if b [i,j] = ' ↑ '

7. then PRINT-LCS (b,X,i-1,j)

8. else PRINT-LCS (b,X,i,j-1)

Example 7.8: Determine the LCS of (1,0,0,1,0,1,0,1) and (0,1,0,1,1,0,1,1,0).

Solution: Let X = (1,0,0,1,0,1,0,1) and Y = (0,1,0,1,1,0,1,1,0).

We are looking for c [8, 9]. The following table is built:

Figure 7.12: Longest common sequence result

From the table, we can deduce that LCS = 6. There are several such sequences, for instance, (1,0,0,1,1,0) (0,1,0,1,0,1) and (0,0,1,1,0,1).

Example 7.9: In this example, we have two strings, X = BACDB and Y = BDCB, to find the longest common subsequence.

Solution: Following the algorithm LCS-Length-Table-Formulation (as stated previously), we have calculated Table C (shown on the left-hand side) and Table B (shown on the right-hand side) in Figure 7.13:

Figure 7.13: Length table formulation

In Table B, instead of D, L, and U, we are using the diagonal arrow, left arrow, and up arrow, respectively. After generating table B, the LCS is determined by function LCS-Print. The result is BCB.

Let us explore some applications. The longest common subsequence problem is a classic computer science problem, the basis of data comparison programs such as the diff-utility, and has applications in bioinformatics. It is also widely used by revision control systems, such as SVN and Git, for reconciling multiple changes made to a revision-controlled collection of files.

Traveling salesperson problem

The traveling salesperson problem (TSP) is a classic optimization problem in computer science and operations research. The objective is to find the shortest possible route such that the traveler visits each city exactly once and returns to the origin city. Using DP, we can solve TSP efficiently for a moderate number of cities.

Let us look at the problem definition. Given a set of n cities and a distance matrix dist where dist[i][j] represents the distance between city i and city j, the goal is to find the minimum distance tour that visits all cities exactly once and returns to the starting city.

Dynamic programming solution

To solve TSP using DP, we use a bitmask to represent subsets of cities and a DP table to store the minimum cost of visiting each subset of cities. The following steps are involved in the solution:

1. State representation: This step involves the following:

a. dp[mask][i] represents the minimum cost of visiting the set of cities represented by mask, ending at city i.

b. Mask is a bitmask of size n, where the j-th bit is 1 if city j is included in the subset.

2. Initialization: dp[1 << i][i] = dist[0][i] for all i (starting from the first city).

3. State transition:

For each subset of cities represented by mask and each city i in the subset, update dp[mask][i] using:

dp[mask][i]=min(dp[mask][i],dp[mask\(1≪i)][j]+dist[j][i])

Where j is any city in the subset mask except i.

4. Final state: The answer is the minimum cost of visiting all cities and returning to the starting city:

min(dp[(1≪n)-1][i]+dist[i][0])

For all i.

Algorithm 7.6: Traveling salesperson problem

function tsp(dist)

n = length(dist) # Number of cities

dp = array of size [2^n][n] initialized to infinity # DP table

# Initialize base cases

for i from 0 to n-1

dp[1 << i][i] = dist[0][i]

# Iterate over all subsets of cities

for mask from 0 to (1 << n) - 1

for i from 0 to n-1

if mask & (1 << i) != 0 # If city i is in the subset represented by mask

for j from 0 to n-1

if mask & (1 << j) != 0 and i != j # If city j is also in the subset and different from i

dp[mask][i] = min(dp[mask][i], dp[mask ^ (1 << i)][j] + dist[j][i])

# Calculate the final answer

answer = infinity

for i from 1 to n-1 # Iterate over all cities except the starting city

answer = min(answer, dp[(1 << n) - 1][i] + dist[i][0])

return answer

Program 7.4: Implementation in C++

#include <iostream>

#include <vector>

#include <algorithm>

#include <limits.h>

using namespace std;

// Define the number of cities

const int N = 4;

// Define a large number representing infinity

const int INF = INT_MAX;

// Function to implement the Travelling Salesman Problem using dynamic programming

int tsp(const vector<vector<int>>& dist) {

int n = dist.size();

int VISITED_ALL = (1 << n) - 1;

// dp[mask][i] will store the minimum cost to visit all cities in mask and end at city i

vector<vector<int>> dp(1 << n, vector<int>(n, INF));

// Initialize the base cases

for (int i = 0; i < n; ++i) {

dp[1 << i][i] = dist[0][i];

}

// Iterate over all subsets of cities

for (int mask = 0; mask < (1 << n); ++mask) {

for (int i = 0; i < n; ++i) {

if (mask & (1 << i)) { // If city i is in the subset represented by mask

for (int j = 0; j < n; ++j) {

if ((mask & (1 << j)) && i != j) { // If city j is also in the subset and different from i

dp[mask][i] = min(dp[mask][i], dp[mask ^ (1 << i)][j] + dist[j][i]);

}

}

}

}

}

// Calculate the final answer

int answer = INF;

for (int i = 1; i < n; ++i) { // Iterate over all cities except the starting city

answer = min(answer, dp[VISITED_ALL][i] + dist[i][0]);

}

return answer;

}

int main() {

// Define the distance matrix

vector<vector<int>> dist = {

{0, 10, 15, 20},

{10, 0, 35, 25},

{15, 35, 0, 30},

{20, 25, 30, 0}

};

// Call the tsp function and print the result

cout << "The minimum cost of visiting all cities and returning to the start is: " << tsp(dist) << endl;

return 0;

}

Example 7.10: Consider Figure 7.14 as a connected graph where nodes represent cities and edge value shows the distance between cities. Find the optimal tour using the traveling salesman problem with dynamic programming.

Figure 7.14: Connected graph between cities

Solution: Naive solution is as follows:

1. Consider city 1 as the starting and ending point.

2. Generate all (n-1)! permutations of cities.

3. Calculate the cost of every permutation and keep track of the minimum cost permutation.

4. Return the permutation with minimum cost.

n-1! = (4-1)! = 3 X 2 X 1 =6

1,2,3,4,1 = 10+35+30+20 = 95

1,2,4,3,1 = 10+25+30+15 = 80

1,3,2,4,1 = 15+35+25+20 = 95

1,3,4,2,1 = 15+30+25+10 = 80

1,4,2,3,1 = 20+25+35+15 = 95

1,4,3,2,1 = 20+30+35+10 = 95

The distance matrix with respect to the preceding graph is as follows:

1

2

3

4

1

0

10

15

20

2

10

0

35

25

3

15

35

0

30

4

20

25

30

0

Table 7.18: Distance matrix

Let us start from node 1:

C(4,∞) = 20 ; C(3,∞) =15 ; C(2,∞) =10

C(2,{3,4}) = min( d(2,3)+C(3,{4}), d(2,4)+C(4,{3} )

C(3,{2,4}) = min( d(3,2)+C(2,{4}), d(3,4)+C(4,{2} )

C(4,{2,3}) = min( d(4,2)+C(2,{3}), d(4,3)+C(3,{2} )

C(2,{3}) = d(2,3)+C(3,phi) = 35 + 15 = 50

C(2,{4}) = d(2,4)+C(4,phi) = 25 + 20 = 45

C(3,{2}) = d(3,2)+C(2,phi) = 35 + 10 = 45

C(3,{4}) = d(3,4)+C(4,phi) = 30 + 20 = 50

C(4,{2}) = d(4,2)+C(2,phi) = 25 + 10 = 35

C(4,{3}) = d(4,3)+C(3,phi) = 30 + 15 = 45

C(2,{3,4}) = min(35+50), 25+45 ) = min(85,70)=70

C(3,{2,4}) = min(35+45), 30+35 ) = min(80,65)=65

C(4,{2,3}) = min(25+50), 30+45 ) = min(75,75)=75

C(1,{2,3,4}) = min(d(1,2)+ C(2,{3,4)), d(1,3) + C(3,{2,4},

d(1,4) + C(4,{2,3})

= min(10+70, 15+65, 20+75)

= min(80,80,95) = 80

The optimal tour for the graph has length =80.

The optimal tour is 1, 3, 4, 2, 1 or 1, 2, 4, 3, 1.

Complexity analysis

The time and space complexity of the traveling salesman problem using dynamic programming is as follows:

• Time complexity: O(n2 * 2n) because we have 2n subsets, and for each subset, we perform O(n2) operations.

• Space complexity: O(n * 2n) for storing the DP table.

The application is as follows:

The TSP has numerous practical applications across various industries. Its ability to optimize routes and reduce costs makes it an invaluable tool for businesses and services that rely on efficient routing and scheduling. By leveraging DP and other advanced algorithms, solutions to TSP can significantly enhance operational efficiency and resource management.

A comparison of divide and conquer and DP techniques is shown in the following table:

Divide and conquer

Dynamic programming

Approach

It breaks down a problem into smaller independent subproblems.

It breaks down a problem into smaller subproblems, but subproblems may overlap.

Subproblems

Subproblems are solved independently.

The results of subproblems are stored and reused.

Memory

Does not store the results of subproblems.

Requires memory to store results of subproblems (usually in a table or array).

Usage

Typically used when subproblems do not overlap.

Used when subproblems overlap.

Examples

Merge sort, quick sort

Fibonacci series, knapsack problem, longest common subsequence

Table 7.19: Comparison of divide and conquer and dynamic programming

Comparison

Some major factors for the difference between the two methods mentioned in Table 7.18 are described here:

• Overlap of subproblems: Divide and conquer solves each subproblem independently, while DP stores the results of overlapping subproblems.

• Memory usage: DP requires additional memory to store results, whereas divide and conquer does not.

• Applicability: Divide and conquer is more applicable when subproblems do not overlap, whereas DP is suitable when subproblems overlap and results can be reused.

• Efficiency: DP can be more efficient in terms of time complexity when compared to divide and conquer for problems with overlapping subproblems due to the reuse of results.

In summary, both techniques aim to break down problems into smaller subproblems, but DP optimizes by storing and reusing results of overlapping subproblems, which can lead to more efficient solutions for certain types of problems. However, regarding memory concerns, divide and conquer requires less space as compared to DP.

Conclusion

Dynamic programming is a powerful technique that enhances computational efficiency by breaking problems into smaller subproblems and storing their solutions to prevent redundant calculations. This approach significantly reduces time complexity compared to naive recursive methods, making it ideal for solving complex optimization problems. DP is widely used in various fields, including graph theory, sequence alignment, and resource allocation. By efficiently utilizing memory and computational resources, it helps in solving large-scale problems that would otherwise be infeasible. Its ability to provide optimal solutions while improving performance makes it a fundamental concept in computer science and algorithm design.

In the next chapter, the user will learn about backtracking algorithms and their utility in the field of computer science engineering. Furthermore, get knowledge about the complexity analysis in both ways, i.e., time and space complexity.

Exercise

1. What do you mean by the longest common subsequence? Explain.

2. Explain matrix chain multiplication with a suitable example.

3. Explain the traveling salesman problem using dynamic programming.

4. Write a short note on optimal binary search trees.

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 8Backtracking

Introduction

In this chapter of the book, we will discuss the various kinds of backtracking algorithms and their utility in the field of computer science engineering. Backtracking is a methodical algorithmic strategy that is utilized to solve problems by investigating all of the potential solutions through the process of exploration and experimentation. When you are faced with an issue in which you need to find one or more answers out of a huge number of options, it is especially effective. Puzzles, combinatorial optimization challenges, and constraint satisfaction issues are all examples of situations that frequently call for the application of backtracking.

Structure

The chapter covers the following topics:

• Overview of backtracking

• N-Queens problem

• Sudoku solver

• Graph coloring problem

• Constraints satisfaction problems

• Sum of subsets

• Hamiltonian cycles

• Solving crossword puzzles

Objectives

The objective of the chapter on backtracking algorithms is to provide a thorough understanding of this problem-solving technique commonly used in computer science and mathematics. The chapter begins by introducing the concept of backtracking and differentiating it from other algorithmic approaches. It then explores the fundamental principles of backtracking, including recursive exploration and systematic search. The chapter covers various aspects of problem-solving with backtracking, illustrating how it can be applied to solve combinatorial optimization problems, constraint satisfaction problems, and graph-related challenges. Implementation techniques for backtracking algorithms are also discussed, including recursive and iterative approaches, as well as strategies for pruning and optimization. Real-world examples and case studies are included throughout the chapter to demonstrate the practical application of backtracking algorithms in solving complex problems. By the end of the chapter, readers will have a solid grasp of backtracking algorithms, enabling them to analyze problems methodically, design efficient algorithms, and apply backtracking techniques effectively in problem-solving scenarios.

Overview of backtracking

Backtracking is a powerful algorithmic technique used to systematically search for solutions to combinatorial problems, constraint satisfaction problems, and optimization challenges. It is particularly useful when the problem space is large and exhaustive search methods become impractical. At its core, backtracking involves exploring all possible candidate solutions incrementally and abandoning a search path as soon as it is determined that the path cannot lead to a valid solution. This process of systematic exploration and pruning helps in efficiently navigating through the problem space, ultimately leading to the discovery of optimal or satisfactory solutions.

One of the defining characteristics of backtracking algorithms is their recursive nature, where decisions are made at each step to choose a candidate solution and explore further or backtrack and try a different path. This recursive exploration mimics a tree-like structure known as a decision tree, where each node represents a decision point, and each branch represents a possible choice. Backtracking algorithms are widely used in various domains, including artificial intelligence, operations research, algorithm design, and software development. They are especially suited for problems that involve searching for a feasible solution among a large number of possibilities, such as the N-Queens problem, Sudoku puzzles, and graph coloring problems.

General process of backtracking

In the process of backtracking, we initially take a step and then assess its correctness, determining whether it aligns with the desired outcome or not. Should it fail to yield the desired result, we backtrack and modify our initial step. Typically, this is achieved through recursion. Essentially, in backtracking, we commence with a partial sub-solution of the problem, which may or may not guide us towards the final solution. Subsequently, we evaluate whether we can proceed further based on this partial solution. If progress is not feasible, we backtrack and adjust the sub-solution before resuming the process. Hence, the typical steps involved in backtracking are as follows:

1. Begin with a sub-solution.

2. Evaluate if this sub-solution leads to the desired solution.

3. If not, backtrack and revise the sub-solution, then resume the process.

N–Queens problem

A classic illustration of backtracking involves arranging N queens on an NxN chessboard without any queen being able to attack another queen. Queens have the ability to strike horizontally, vertically, or diagonally. Similarly, the solution to this challenge follows a comparable approach. Initially, we position the first queen anywhere on the board, then proceed to place subsequent queens in safe positions. This sequence repeats until either all queens are placed (resulting in a solution) or no safe positions remain. In the latter scenario, where no safe positions are available, we adjust the placement of the previously positioned queen.

Algorithm 8.1: Algorithm of N-Queen using backtracking

1. Commence from the leftmost column.

2. If all queens are successfully placed, return true. Go to step 4.

3. Iterate through all rows in the current column. Perform the following steps for each row:

a. Check if placing the queen in this row is safe.

b. If safe, mark the position [row, column] as part of the solution and recursively check if placing the queen here leads to a valid solution.

c. If placing the queen in [row, column] leads to a solution, return true.

d. If placing the queen in [row, column] does not lead to a solution, unmark the position [row, column], backtrack, and try other rows.

e. If no safe position is found in the current column, return false to initiate backtracking.

4. Exit.

The following figure shows the initial state of the 4-Queens problem:

Figure 8.1: Initial state of the 4-Queens problem

Initial state

The N-Queens problem is one that involves the placement of N number of chess queens on a chessboard that is N×N in size, assuring that no two queens will attack each other.

The expected output is in the form of a matrix, with Qs representing the blocks in which queens are placed and periods (.) representing the empty spaces in the matrix. For instance, the output matrix for the 4-Queens solution described previously is presented in the following figure:

Figure 8.2: Goal state of the 4-Queens problem

Final state

_ Q _ _

_ _ _ Q

Q _ _ _

_ _ Q _

Program 8.1: Implementation of N-Queens problem

#include <bits/stdc++.h>

#define N 4

using namespace std;

// A utility function to print solution

void printSolution(int board[N][N])

{

for (int i = 0; i < N; i++) {

for (int j = 0; j < N; j++)

if(board[i][j])

cout << "Q ";

else cout<<". ";

printf("\n");

}

}

// A utility function to check if a queen can be placed on board[row][col]. Note that this function is called when "col" queens are already placed in columns from 0 to col -1. So we need to check only left side for attacking queens

bool isSafe(int board[N][N], int row, int col)

{

int i, j;

// Check this row on left side

for (i = 0; i < col; i++)

if (board[row][i])

return false;

// Check upper diagonal on left side

for (i = row, j = col; i >= 0 && j >= 0; i--, j--)

if (board[i][j])

return false;

// Check lower diagonal on left side

for (i = row, j = col; j >= 0 && i < N; i++, j--)

if (board[i][j])

return false;

return true;

}

// A recursive utility function to solve N Queen problem

bool solveNQUtil(int board[N][N], int col)

{

// base case: If all queens are placed then return true

if (col >= N)

return true;

// Consider this column and try placing this queen in all rows one by one

for (int i = 0; i < N; i++) {

// Check if the queen can be placed on board[i][col]

if (isSafe(board, i, col)) {

// Place this queen in board[i][col]

board[i][col] = 1;

// recur to place rest of the queens

if (solveNQUtil(board, col + 1))

return true;

// If placing queen in board[i][col] doesn't lead to a solution, then remove queen from board[i][col]

board[i][col] = 0; // BACKTRACK

}

}

// If the queen cannot be placed in any row in this column col then return false

return false;

}

// This function solves the N Queen problem using Backtracking. It mainly uses solveNQUtil() to solve the problem. It returns false if queens cannot be placed, otherwise, return true and prints placement of queens in the form of 1s. Please note that there may be more than one solutions, this function prints one of the feasible solutions.

bool solveNQ()

{

int board[N][N] = { { 0, 0, 0, 0 },

{ 0, 0, 0, 0 },

{ 0, 0, 0, 0 },

{ 0, 0, 0, 0 } };

if (solveNQUtil(board, 0) == false) {

cout << "Solution does not exist";

return false;

}

printSolution(board);

return true;

}

// Driver program to test above function

int main()

{

solveNQ();

return 0;

}

Complexity of N-Queens problem

The time and space complexity of the N-Queens problem are as follows:

• Time complexity: The N-Queens problem is a combinatorial problem that involves finding a way to place N number of queens on an N×N chessboard such that no two queens threaten each other. The time complexity of the problem can be represented as O(N!), where N is the size of the chessboard and also the number of queens to be placed. The factorial complexity arises from the number of possible permutations and combinations that need to be checked to find a valid solution.

• Space complexity: The space complexity of the N-Queens problem depends on the algorithm used to solve it. In the backtracking algorithm, which is commonly used to solve the N-Queen problem, the space complexity is O(N2) to represent the chessboard. This is because the algorithm typically uses a 2D array to represent the chessboard and keeps track of the queen placements.

Overall, the N-Queens problem is considered to have a high computational complexity, especially for larger values of N, due to its exponential time complexity in terms of the number of queens to be placed. Various optimization techniques, such as pruning and constraint propagation, can be applied to improve the performance of algorithms used to solve this problem.

Sudoku solver

Sudoku is a puzzle that involves logic and combinatorial techniques to place numbers correctly on a grid. The objective is to complete a 9×9 grid, represented as a 2D array grid [9][9], by filling the empty cells with digits from 1 to 9. The goal is to ensure that each row, column, and 3×3 sub-grid contains all digits from 1 to 9 exactly once.

Initial state of the puzzle

The initial state of the puzzle is as follows:

Figure 8.3: Initial state of the puzzle

Let us look at the solution:

• Using the naïve approach: The basic method involves generating every possible arrangement of numbers from 1 to 9 to fill in the empty cells. Each arrangement is then checked one by one until the correct one is discovered. Specifically, for each empty position, a number from 1 to 9 is assigned. Once all empty positions are filled, the matrix's safety is verified. If it is safe, it is printed; otherwise, the process recurs for other possibilities.

• Using the backtracking approach: In backtracking algorithms, you construct a solution incrementally, step by step. When it becomes evident at a certain step that the current path cannot lead to a solution, you backtrack to the previous step and explore an alternative path. In essence, you backtrack when you have exhausted all possibilities at a particular step. Backtracking is also termed as depth-first search.

Algorithm 8.2: Algorithm of the Sudoku solver using backtracking

1. Similar to other backtracking problems, Sudoku can be solved by sequentially assigning numbers to empty cells.

2. Before assigning a number, it is crucial to ensure that the number is not already present in the current row, the current column, and the current 3x3 sub-grid.

3. If the number is absent from the corresponding row, column, and sub-grid, proceed to assign that number. Then, recursively verify if this assignment contributes to a solution. If the assignment does not result in a solution, move on to the next number for the current empty cell. If none of the numbers from 1 to 9 lead to a solution, return false.

Final state of the puzzle

Figure 8.4 shows the goal state of the puzzle:

Figure 8.4: Goal state of puzzle

Program 8.2: Implementation of the Sudoku solver using backtracking

#include <iostream>

using namespace std;

// N is the size of the 2D matrix N*N

#define N 9

/* A utility function to print grid */

void print(int arr[N][N])

{

for (int i = 0; i < N; i++)

{

for (int j = 0; j < N; j++)

cout << arr[i][j] << " ";

cout << endl;

}

}

// Checks whether it will be legal to assign num to the given row, col

bool isSafe(int grid[N][N], int row, int col, int num)

{

// Check if we find the same num in the similar row , we return false

for (int x = 0; x <= 8; x++)

if (grid[row][x] == num)

return false;

// Check if we find the same num in the similar column , we return false

for (int x = 0; x <= 8; x++)

if (grid[x][col] == num)

return false;

// Check if we find the same num in the particular 3*3 matrix, we return false

int startRow = row - row % 3,

startCol = col - col % 3;

for (int i = 0; i < 3; i++)

for (int j = 0; j < 3; j++)

if (grid[i + startRow][j + startCol] == num)

return false;

return true;

}

/* Takes a partially filled-in grid and attempts to assign values to all unassigned locations in such a way to meet the requirements for Sudoku solution (non-duplication across rows, columns, and boxes) */

bool solveSudoku(int grid[N][N], int row, int col)

{

// Check if we have reached the 8th row and 9th column (0 indexed matrix) , we are returning true to avoid further backtracking

if (row == N - 1 && col == N)

return true;

// Check if column value becomes 9, we move to next row and column start from 0

if (col == N) {

row++;

col = 0;

}

// Check if the current position of the grid already contains value >0, we iterate for next column

if (grid[row][col] > 0)

return solveSudoku(grid, row, col + 1);

for (int num = 1; num <= N; num++)

{

// Check if it is safe to place the num (1-9) in the given row ,col ->we move to next column

if (isSafe(grid, row, col, num))

{

/* Assigning the num in the current (row,col) position of the grid and assuming our assigned num in the position is correct */

grid[row][col] = num;

// Checking for next possibility with next column

if (solveSudoku(grid, row, col + 1))

return true;

}

// Removing the assigned num , since our assumption was wrong , and we go for next assumption with diff num value

grid[row][col] = 0;

}

return false;

}

int main()

{

// 0 means unassigned cells

int grid[N][N] = { { 5, 3, 0, 0, 7, 0, 0, 0, 0 },

{ 6, 0, 0, 1, 9, 5, 0, 0, 0 },

{ 0, 9, 8, 0, 0, 0, 0, 6, 0 },

{ 8, 0, 0, 0, 6, 0, 0, 0, 3 },

{ 4, 0, 0, 8, 0, 3, 0, 0, 1 },

{ 7, 0, 0, 0, 2, 0, 0, 0, 6 },

{ 0, 6, 0, 0, 0, 0, 2, 8, 0 },

{ 0, 0, 0, 4, 1, 9, 0, 0, 5 },

{ 0, 0, 0, 0, 8, 0, 0, 7, 9 } };

if (solveSudoku(grid, 0, 0))

print(grid);

else

cout << "no solution exists " << endl;

return 0;

}

Complexity of the Sudoku solver algorithm

In a backtracking Sudoku solver, the time and space complexity can be analyzed as follows:

• Time complexity: In the worst-case scenario, a backtracking solver exhaustively tests all feasible number combinations for empty cells until it reaches a solution. This gives rise to a time complexity of O(9(n^2)), where n represents the grid size (e.g., for a standard 9x9 Sudoku, n=9). Nevertheless, due to the backtracking method's ability to discard invalid options early in the process, the effective number of operations tends to be considerably lower than this worst-case estimate. Consequently, backtracking Sudoku solvers frequently exhibit fast solving times, particularly for puzzles possessing unique solutions and moderate difficulty levels.

• Space complexity: The space complexity of a backtracking Sudoku solver is contingent on its implementation methodology. The primary space utilization arises from the recursion stack, which retains the puzzle's state at each recursive invocation. In a conventional recursive backtracking setup lacking optimizations, the space complexity stands at O(n2), where n denotes the Sudoku grid's dimensions. This arises because the recursion depth correlates with the number of empty cells in the grid. However, certain optimizations can curtail space requirements. For instance, employing iterative backtracking or optimizing tail recursion can diminish the stack space demand. Moreover, if the solver adopts an explicit data structure to monitor empty cells and their feasible values, the space complexity might slightly escalate but typically remains manageable, often hovering around O(n2) or slightly higher.

Graph coloring problem

Consider a graph G and a positive integer m. We aim to determine if the nodes of G can be colored so that adjacent nodes have distinct colors, using at most m colors. This problem is known as the m-colorability decision problem. The m-colorability optimization problem seeks to find the smallest value of m that allows such coloring of graph G.

When coloring a map, only four colors are necessary to ensure that no adjacent regions share the same color. Historically, it was believed that five colors were always sufficient, but no map requiring more than four colors had been identified. This challenge persisted for centuries until a group of mathematicians, assisted by computer analysis, conclusively demonstrated that four colors are indeed adequate for planar graphs.

The m-coloring function initially converts the graph into its adjacency matrix and initializes an array x[] to zero. Colors are denoted by integers 1 through m, and a valid solution is represented by an n-tuple (x1, x2, ..., xn), where xi indicates the color of node i.

To implement a recursive backtracking algorithm for graph coloring, the process starts by calling the function mcoloring(1).

Algorithm 8.3: Algorithm of graph coloring using backtracking

mcoloring(k) //It is an algorithm designed using a recursive backtracking framework. It utilizes the Boolean adjacency matrix G[1:n, 1:n] to represent the graph. The algorithm prints all assignments of integers from 1 to m to the graph's vertices, ensuring adjacent vertices are assigned different integers. The parameter k signifies the index of the next vertex to be colored.

{

Iterate until (false) {

// Generate all valid assignments for x[k].

NextValue(k); // Assign a valid color to x[k].

if (x[k] = 0) then return; // No new color is possible.

if (k = n) then // n vertices have been colored using at most m colors.

write(x[1: n]);

else

mcoloring(k+1);

}

}

Algorithm NextValue (k)

// x[1], ..., x[k-1] have been assigned integers within the range [1, m], ensuring adjacent vertices have unique integers. A value for x[k] is selected from the range [0, m], where x[k] is given the next highest number that hasn't been used by adjacent vertices of vertex k. If such a color isn't available, x[k] is set to 0.

{

Repeat

{

x[k] := (x[k] + 1) mod (m + 1); // Move to the next available color.

if (x[k] = 0)

then return; // All colors have been used.

for j := 1 to n do {

// Check if the color is distinct from adjacent colors.

if ((G[k, j] ≠ 0) and (x[k] = x[j])) then

{

// If (k, j) is an edge and adjacent vertices have the same color.

break;

}

}

if (j = n + 1) then

return; // A new distinct color has been found.

} until (false); // Otherwise, try finding another color.

}

Example 8.1: Color the following graph with the minimum number of colors by backtracking using the state space tree.

Solution: The following state space tree has been designed for all the possible solutions for given graph nodes:

Figure 8.5: Graph with possible coloring

Program 8.3: Implementation of graph coloring problem using backtracking

#include <bits/stdc++.h>

using namespace std;

#define V 4

void printSolution(int color[]);

bool isSafe(int v, bool graph[V][V], int color[], int c)

{

for (int i = 0; i < V; i++)

if (graph[v][i] && c == color[i])

return false;

return true;

}

// A recursive utility function to solve m coloring problem

bool graphColoringUtil(bool graph[V][V], int m, int color[], int v)

{

// base case: If all vertices are assigned a color, then return true

if (v == V)

return true;

// Consider this vertex v and try different colors

for (int c = 1; c <= m; c++) {

if (isSafe(v, graph, color, c)) {

color[v] = c;

if (graphColoringUtil(graph, m, color, v + 1) == true)

return true;

// If assigning color c doesn't lead to a solution then remove it

color[v] = 0;

}

}

//If no color can be assigned to this vertex then return false

return false;

}

/* This function solves the m Coloring problem using Backtracking. It mainly uses graphColoringUtil() to solve the problem. It returns false if the m colors cannot be assigned, otherwise return true and prints assignments of colors to all vertices. Please note that there may be more than one solutions, this function prints one of the feasible solutions.*/

bool graphColoring(bool graph[V][V], int m)

{

// Initialize all color values as 0. This initialization is needed correct functioning of isSafe()

int color[V];

for (int i = 0; i < V; i++)

color[i] = 0;

// Call graphColoringUtil() for vertex 0

if (graphColoringUtil(graph, m, color, 0) == false) {

cout << "Solution does not exist";

return false;

}

printSolution(color);

return true;

}

// A utility function to print solution

void printSolution(int color[])

{

cout << "Solution Exists:"

<< " Following are the assigned colors"

<< "\n";

for (int i = 0; i < V; i++)

cout << " " << color[i] << " ";

cout << "\n";

}

int main()

{

// Create following graph and test whether it is 3 colorable

bool graph[V][V] = {

{ 0, 1, 1, 1 },

{ 1, 0, 1, 0 },

{ 1, 1, 0, 1 },

{ 1, 0, 1, 0 },

};

// Number of colors

int m = 3;

// Function call

graphColoring(graph, m);

return 0;

}

Complexity of graph coloring using backtracking

The complexity of graph coloring using backtracking is exponential due to the nondeterministic polynomial time (NP)-hard nature of the problem. The algorithm's time complexity increases with the number of vertices, edges, and available colors. It involves exploring a potentially large search space and checking constraints to ensure adjacent vertices have different colors. While heuristics can help, the fundamental challenge remains computationally intensive, especially for large or constrained graphs.

Constraints satisfaction problems

A constraint satisfaction problem, often known as a CSP, is a type of computational issue that requires the identification of a solution to a set of variables that are subject to conditions that have been established. To solve CSPs, one method that is frequently utilized is called backtracking. If you want to solve a CSP using backtracking, first determine the variables that are included in the complexity of the issue. It is possible for each variable to assign a value from a domain. Specify the potential values (domain) for each variable. This is the process of defining domains. In order to define constraints, you must first identify the constraints that restrict the potential combinations of variable values.

We can use the CSP strategy for solving the cryptarithmetic problem, the graph coloring problem, and the N-Queens problem. In this section, we will understand CSP for the cryptarithmetic problem.

Algorithm 8.4: CSP using backtracking search algorithm

Procedure Backtrack(assignment, csp):

// Returns a solution or failure

if assignment is complete:

return assignment

var ← Select an unassigned variable from the CSP

for each value in the domain of var, respecting any ordering preferences, based on assignment and CSP:

if value is consistent with assignment:

add { var = value } to assignment

inferences ← Perform Inference(csp, var, value)

if inferences are not a failure:

add inferences to assignment

result ← Backtrack(assignment, csp

if result is not a failure:

return result

remove { var = value } and inferences from assignment

return failure

Solution: If all variables are assigned values that satisfy the constraints, then a solution has been found.

Cryptarithmetic problem using CSP

Cryptarithmetic problems involve assigning digits to letters in an arithmetic expression such that the resulting equation holds true. For instance, in the problem SEND + MORE = MONEY, each letter represents a digit (from 0 to 9), and the goal is to find a valid assignment that satisfies the equation.

CSP contains the following components:

• Variables: These represent the unknowns in the problem. In cryptarithmetic, each letter corresponds to a variable.

• Domains: The domain of a variable consists of the possible values it can take. For cryptarithmetic, the domain is the set of digits (0-9).

• Constraints: Constraints define relationships between variables. In our case, constraints ensure that no two letters represent the same digit and that the equation remains valid.

Let us look at CSP representation. A CSP is defined by the following:

• A finite set of variables (V₁, V₂, …, Vₙ).

• Non-empty domains for each variable (D₁, D₂, …, Dₙ).

• A finite set of constraints (C₁, C₂, …, Cₘ), where each constraint restricts the possible values for variables (e.g., V₁ ≠ V₂).

Example 8.2: Solve the cryptarithmetic TWO + TWO = FOUR.

Solution:

T W O

+ T W O

F O U R

Variables: F, T, U, W, R, O, X1, X2, X3 (carry generated)

Domains: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} (same domain for all)

Sample constraints are given as follows:

All_different(F,T,U, W , R, O)

First, assign the value 5 to character O, so carry would be generated and forwarded to the next pair of characters. The value of carry is 1. R got the value 0. Next, we consider value 6 for W, we get carry, and the value for U is 3. Similarly, for character T, the value is 7. So, the resultant carry signifies 1 for character F.

T W O 7 6 5

+ T W O + 7 6 5

F O U R 1 5 3 0

Program 8.4: Implementation of constraint satisfaction problem using backtracking

#include <iostream>

#include <vector>

#include <algorithm>

using namespace std;

bool solveCryptarithmetic(vector<string>& words, string& resultWord) {

string uniqueChars = "";

for (string word : words) {

for (char ch : word) {

if (uniqueChars.find(ch) == string::npos)

uniqueChars += ch;

}

}

if (uniqueChars.size() > 10) return false; // Too many unique characters for 0-9 digits

sort(uniqueChars.begin(), uniqueChars.end());

vector<int> mapping;

do {

if (uniqueChars.size() > 1 && mapping[0] == 0) continue; // Leading zeros are not allowed

int carry = 0, sum = 0;

for (int i = resultWord.size() - 1; i >= 0; i--) {

int digitSum = carry;

for (string word : words) {

if (word.size() > i)

digitSum += mapping[uniqueChars.find(word[i])];

}

carry = digitSum / 10;

sum += (digitSum % 10) * pow(10, resultWord.size() - 1 - i);

}

if (carry == 0 && sum == 0) continue; // No leading zeros

if (sum == 0) continue; // No leading zeros in result word

int result = 0;

for (char ch : resultWord)

result = result * 10 + mapping[uniqueChars.find(ch)];

if (sum == result) {

for (char ch : uniqueChars)

cout << ch << " = " << mapping[uniqueChars.find(ch)] << ", ";

cout << endl;

return true;

}

} while (next_permutation(mapping.begin(), mapping.end()));

return false;

}

int main() {

vector<string> words = {"TWO", "TWO"};

string resultWord = "FOUR";

vector<int> mapping;

for (int i = 0; i < 10; i++)

mapping.push_back(i);

if (!solveCryptarithmetic(words, resultWord))

cout << "No solution found!" << endl;

return 0;

}

Complexity of CSP using backtracking

The time complexity of a CSP solved by backtracking is commonly exponential, denoted as O(bd), where:

• The branching factor, denoted as b, is the average number of possibilities available for each decision.

• The variable d represents the depth of the search tree, which is the greatest depth reached throughout the recursion process.

Sum of subsets

The subset sum problem involves identifying a subset of elements whose sum is equal to a specified number. In terms of the worst-case scenario, the backtracking strategy is capable of generating all permutations. However, it generally outperforms the recursive approach when it comes to solving the subset sum problem.

Let us consider one example to solve this problem.

Example 8.3: Given a set A consisting of n positive integers and a target value sum, determine whether there exists a subset of A whose elements add up to the target total. {9, 10, 1, 3, 2, 99}

Sum=4.

Solution: {1, 3} is the subset whose sum is 4.

For better understanding, we will use Algorithm 8.5.

In the backtracking process, as we traverse down the depth of the tree, we accumulate elements that we have encountered so far. If the sum of these elements satisfies the explicit criteria, we proceed to construct child nodes. Whenever the conditions are not satisfied, we cease generating more subtrees of that particular node and revert to the previous node in order to investigate the nodes that have not yet been examined. We must investigate the nodes throughout the width and depth of the tree. The generation of nodes along the width is regulated by a loop, whereas the generation of nodes along the depth is accomplished through recursion, specifically employing post-order traversal. An improved code can be achieved with the subsequent representation.

Algorithm 8.5: Sum of subset using backtracking

1. Begin with an empty subset.

2. Include the next element from the list in the subset.

3. If the subset achieves the sum M, terminate with that subset as the solution.

4. If the subset is infeasible or if the end of the list is reached, backtrack through the subset until you find the most appropriate value.

5. If the subset is feasible (sum of subset < M), proceed to step 2.

6. If all elements have been visited without discovering a suitable subset and no further backtracking is possible, halt without a solution.

Program 8.5: Implementation of the sum of subset problem using backtracking

#include<iostream>

using namespace std;

class Subset_Sum

{

public:

void subsetsum_Backtracking(int Set[] , int pos, int sum, int tmpsum, int size, bool & found)

{

if (sum == tmpsum)

found = true;

for (int i = pos; i < size; i++)

{

if (tmpsum + Set[i] <= sum)

{

tmpsum += Set[i];

subsetsum_Backtracking(Set, i + 1, sum, tmpsum, size, found);

tmpsum -= Set[i];

}

}

}

};

int main()

{

int i, n, sum;

Subset_Sum S;

cout << "Enter the number of elements in the set" << endl;

cin >> n;

int a[n];

cout << "Enter the values" << endl;

for(i=0;i<n;i++)

cin>>a[i];

cout << "Enter the value of sum" << endl;

cin >> sum;

bool f = false;

S.subsetsum_Backtracking(a, 0, sum, 0, n, f);

if (f)

cout << "subset with the given sum found" << endl;

else

cout << "no required subset found" << endl;

return 0;

}

Complexity of sum of subset problem using backtracking

The time complexity of the sum of subset problem solved using backtracking is exponential, typically O(2n), where n is the number of elements in the set. However, this complexity can be reduced through optimization techniques like pruning and memorization, which help narrow down the search space and avoid redundant computations.

Hamiltonian cycles

Consider a connected graph G with n vertices, denoted as G = (V, E). A Hamiltonian cycle, proposed by William Hamilton, is a closed path in graph G that traverses n edges, visits each vertex exactly once, and ends at the starting vertex. If the vertices of G are visited in the order v1, v2, ..., vn+1, then the edges (vi, vi+1) are present in E, where 1 < i < n. The vertices vi are distinct, except for v1 and vn+1, which are the same. In Figure 8.6, the graph G1 exhibits a Hamiltonian cycle consisting of the vertices 1, 2, 8, 7, 6, 5, 4, 3, and 1, while the graph G2 does not possess a Hamiltonian cycle:

Figure 8.6: Connected graphs G1 and G2

The backtracking solution vector (X1, . . . . . Xn) is defined such that each xi represents the ith vertex visited in the proposed cycle. Given that k = 1, X1 can assume any of the n vertices. In order to prevent the repetition of the same cycle n times, it is necessary for X1 to be equal to 1. If 1 < k < n, then Xk can be any vertex v that is different from X1, X2, . . . , Xk–1, and v is next to kx-1. The vertex Xn must be the last remaining vertex, and it must be connected to both Xn-1 and X1.

The NextValue method can be used to specialize the recursive backtracking schema for finding all Hamiltonian cycles. The process begins by initializing the adjacency matrix G[1: n, 1: n]. Next, X[2: n] is set to zero, and X[1] is set to 1. Finally, the Hamiltonian (2) function is executed.

Algorithm 8.6: Hamiltonian cycle using backtracking

NextValue(k):

repeat

x[k]: = (x[k] + 1) mod (n + 1) // Get the next vertex

if x[k] = 0 then

return // If x[k] becomes 0, return

if G[x[k-1], x[k]] ≠ 0 then // Check if there is an edge

j: = 1

while j <= k - 1 do

if x[j] = x[k] then

break // Check for distinctness

j := j + 1

if j = k then

if k < n or (k = n and G[x[n], x[1]] ≠ 0) then

return // If the vertex is distinct and meets connectivity condition, return

until false

Algorithm Hamiltonian(k): //employs a recursive backtracking approach to discover all Hamiltonian cycles within a graph. The graph, represented as an adjacency matrix G[1:n, 1:n], is explored with cycles originating from node 1.

{

repeat

{// Generate values for x[k].

NextValue(k); //Assign a valid next value to x[k]. If x[k] equals 0, return.

if (k = n) then write (x[1: n]); else Hamiltonian(k + 1)

} until (false);

}

Program 8.6: Implementation of the Hamiltonian cycle using backtracking

This program prompts the user to enter the number of vertices and the adjacency matrix of the graph. Then, it checks whether a Hamiltonian cycle exists in the graph and prints the cycle if found.

#include <iostream>

#include <vector>

using namespace std;

const int MAX = 10;

bool isSafe(int v, bool graph[MAX][MAX], vector<int>& path, int pos, int V) {

if (!graph[path[pos - 1]][v])

return false;

for (int i = 0; i < pos; i++)

if (path[i] == v)

return false;

return true;

}

bool hamCycleUtil(bool graph[MAX][MAX], vector<int>& path, int pos, int V) {

if (pos == V) {

if (graph[path[pos - 1]][path[0]])

return true;

else

return false;

}

for (int v = 1; v < V; v++) {

if (isSafe(v, graph, path, pos, V)) {

path[pos] = v;

if (hamCycleUtil(graph, path, pos + 1, V))

return true;

path[pos] = -1;

}

}

return false;

}

void hamiltonianCycle(bool graph[MAX][MAX], int V) {

vector<int> path(V, -1);

path[0] = 0;

if (hamCycleUtil(graph, path, 1, V)) {

cout << "Hamiltonian cycle exists: ";

for (int i = 0; i < V; i++)

cout << path[i] << " ";

cout << path[0] << endl;

} else {

cout << "No Hamiltonian cycle exists." << endl;

}

}

int main() {

int V;

cout << "Enter the number of vertices: ";

cin >> V;

bool graph[MAX][MAX];

cout << "Enter the adjacency matrix: " << endl;

for (int i = 0; i < V; i++) {

for (int j = 0; j < V; j++) {

cin >> graph[i][j];

}

}

hamiltonianCycle(graph, V);

return 0;

}

Complexity of Hamiltonian cycle algorithm

The time complexity of the Hamiltonian cycle algorithm using backtracking is exponential, specifically O(V!), where V is the number of vertices in the graph. This makes the algorithm impractical for large graphs due to its factorial growth in time complexity.

Solving crossword puzzles

A crossword puzzle comprises a grid, often square or rectangular, with white and black-shaded squares. The objective is to populate the white squares with letters, forming words or phrases according to the hints provided. Words are entered horizontally (across) and vertically (down) in languages that are read from left to right. Shaded squares separate the words or phrases.

The backtracking algorithm can be applied to solve a crossword puzzle in the following ways:

• Naive approach: The naive approach fills in the empty cells without any logic and then checks whether it was a valid placement. However, this can be inefficient and time-consuming.

• Backtracking algorithm: Backtracking is a form of a brute-force approach used when evaluating several options where we do not know which one is correct. We try to solve the problem using a trial-and-error method, making one decision at a time until we find the desired answer.

The backtracking algorithm for solving a crossword puzzle traverses all the vacant cells, incrementally filling them with possible words retrieved from a dictionary file. The algorithm backtracks if a word filled does not comply with the constraint (the intersection point between horizontal and vertical words).

The following figure shows the crossword puzzle:

Figure 8.7: Crossword puzzle

Algorithm 8.7: Crossword puzzle problem using backtracking:

1. Begin by looking for a continuous row or column of empty cells.

2. Try to fill the continuous row or column with words retrieved from the dictionary file.

3. Examine the intersection point between the horizontal and vertical words for constraints before placing a word.

4. If a valid word can be placed, continue with the next continuous row or column of empty cells.

5. If none of the words can be placed, backtrack and alter the values for the cells visited previously.

Program 8.7: Implementation of crossword puzzle using backtracking algorithm

#include <iostream>

#include <vector>

#include <string>

#include <algorithm>

using namespace std;

const int GRID_SIZE = 10; // Change the grid size as needed

// Structure to represent a word in the crossword

struct Word {

string text;

int row, col;

bool isHorizontal;

Word(string t, int r, int c, bool h) : text(t), row(r), col(c), isHorizontal(h) {}

};

// Function to check if a word can be placed horizontally at the given position

bool canPlaceHorizontal(const vector<string>& grid, const string& word, int row, int col) {

if (col + word.length() > GRID_SIZE) return false; // Check if word exceeds grid boundary

for (int i = 0; i < word.length(); ++i) {

if (grid[row][col + i] != '-' && grid[row][col + i] != word[i]) {

return false; // Check for conflicting letters

}

}

return true;

}

// Function to check if a word can be placed vertically at the given position

bool canPlaceVertical(const vector<string>& grid, const string& word, int row, int col) {

if (row + word.length() > GRID_SIZE) return false; // Check if word exceeds grid boundary

for (int i = 0; i < word.length(); ++i) {

if (grid[row + i][col] != '-' && grid[row + i][col] != word[i]) {

return false; // Check for conflicting letters

}

}

return true;

}

// Function to place a word horizontally at the given position

void placeHorizontal(vector<string>& grid, const string& word, int row, int col) {

for (int i = 0; i < word.length(); ++i) {

grid[row][col + i] = word[i];

}

}

// Function to place a word vertically at the given position

void placeVertical(vector<string>& grid, const string& word, int row, int col) {

for (int i = 0; i < word.length(); ++i) {

grid[row + i][col] = word[i];

}

}

// Function to solve the crossword puzzle using backtracking

bool solveCrossword(vector<string>& grid, const vector<string>& words, int index) {

if (index == words.size()) return true; // Base case: all words are placed

for (int i = 0; i < GRID_SIZE; ++i) {

for (int j = 0; j < GRID_SIZE; ++j) {

if (grid[i][j] == '-' || grid[i][j] == words[index][0]) {

if (canPlaceHorizontal(grid, words[index], i, j)) {

vector<string> tempGrid = grid;

placeHorizontal(tempGrid, words[index], i, j);

if (solveCrossword(tempGrid, words, index + 1)) {

grid = tempGrid;

return true;

}

}

if (canPlaceVertical(grid, words[index], i, j)) {

vector<string> tempGrid = grid;

placeVertical(tempGrid, words[index], i, j);

if (solveCrossword(tempGrid, words, index + 1)) {

grid = tempGrid;

return true;

}

}

}

}

}

return false; // No solution found

}

// Function to display the crossword grid

void displayCrosswordGrid(const vector<string>& grid) {

for (const string& row : grid) {

cout << row << endl;

}

}

int main() {

vector<string> words = {"HELLO", "WORLD", "CROSSWORD", "PUZZLE"}; // Words to place in the crossword

sort(words.begin(), words.end(), [](const string& a, const string& b) { return a.length() > b.length(); }); // Sort words by length

vector<string> grid(GRID_SIZE, string(GRID_SIZE, '-')); // Initialize the crossword grid with dashes

if (solveCrossword(grid, words, 0)) {

cout << "Crossword puzzle solved:" << endl;

displayCrosswordGrid(grid);

} else {

cout << "No solution found." << endl;

}

return 0;

}

Complexity of crossword puzzle algorithm

The time and space complexity of the crossword puzzle problem using backtracking is as follows:

• Time complexity: Time complexity of the backtracking approach is O((M * P)D) where:

N is the number of continuous rows or columns of empty cells (where a word is to be filled) in the grid.

P is the list of possible words to be tested for the crossword constraint.

For the backtracking function, the depth (D) of this recursive function will be equal to the crossword constraint.

D = intersection point(s) between the horizontal and vertical words.

M is the average length of a word.

• Space complexity: The space complexity is O(L), where L is the length of the given word. This space is used for the recursion stack.

Conclusion

In this chapter, we explored various backtracking algorithms and their significance in solving complex computational problems. Backtracking provides an efficient way to navigate large solution spaces by systematically exploring and pruning possibilities. From classic problems like the N-Queens and Sudoku solver to constraint satisfaction and combinatorial challenges, backtracking proves to be a powerful tool in computer science. Its applications extend to graph problems, optimization, and even puzzle-solving. Understanding and implementing backtracking enhances problem-solving skills, making it a valuable approach in algorithm design and artificial intelligence.

In the next chapter, the user will learn about various kinds of branch and bound algorithms and their utility in the field of computer science engineering. Branch and bound is an algorithmic technique used to solve optimization problems by systematically exploring the solution space and pruning branches that cannot lead to an optimal solution.

Exercise

1. What is the N-Queens problem? Write a procedure to solve 8-Queens problem.

2. Explain the graph coloring problem using the backtracking technique.

3. Solve the cryptarithmetic SEND + MORE = MONEY using constraint satisfaction problem.

4. Find the subset from the given set whose sum is equal to 14. Use the backtracking strategy to solve the subset sum problem A = [2, 3, 5, 7, 10].

5. How can you solve a crossword puzzle problem using the constraint satisfaction mechanism? Justify your answer.

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 9Branch and Bound

Introduction

In this chapter of the book, we will discuss the various kinds of branch and bound algorithms and their utility in the field of computer science engineering. Branch and bound is an algorithmic technique used to solve optimization problems by systematically exploring the solution space and pruning branches that cannot lead to an optimal solution. It divides the problem into smaller subproblems and maintains bounds on the possible solutions within each subproblem. This chapter consists of detailed descriptions of various branch and bound algorithms with solved examples.

Structure

This chapter covers the following topics:

• Branch and bound approach

• 0/1 knapsack problem

• Traveling salesperson problem

• Resource allocation problem

• Quadratic assignment problem

• Job sequencing problem

Objectives

The objective of the chapter is to provide a systematic solution space search method. Like backtracking, boundary functions will prevent subtrees from being generated without a solution node. The branch and bound approach has two major differences from backtracking. First, the branching function of this system can use depth-first, breadth-first, or bounding functions. Secondly, the boundary function efficiently trims the search tree. The approach's primary goal is the optimal solution, the best optimization result. It assures the solution will be the highest or lowest value, depending on the problem. The branch and bound algorithm reduces the number of alternative solutions by methodically examining the solution space. By pruning fewer promising branches and favoring more promising ones, computational efficiency is improved. Bounding requires setting objective function upper and lower limits in subproblems. This technique eliminates branches that cannot produce better solutions than the best-known answer, narrowing the search. Since it can solve integer programming, the traveling salesman problem, the knapsack problem, and other combinatorial optimization problems, the branch and bound method is versatile.

Branch and bound approach

In computer science, there exist numerous optimization issues, some of which include finding the finite number of viable shortest paths in a graph or the least spanning trees. These problems can be addressed efficiently within polynomial time. Usually, these issues necessitate considering the most unfavorable outcome among all potential arrangements. The branch and bound algorithm generates alternative paths and constraints to get the optimal solution.

Using the branch and bound algorithm

Branch and bound is a highly efficient approach for solving some issues, as we have already mentioned. In this part, we will examine all instances in which branching and binding are suitable.

If the problem at hand involves discrete optimization, it is suitable to employ a branch and bound methodology. Discrete optimization pertains to issues when the variables are part of a discrete set. Some examples of these problems include 0-1 integer programming and network flow difficulties.

Branch and bound is an effective method for solving combinatory optimization problems. Combinatorial optimization is used to optimize an optimization problem by determining its maximum or lowest value, depending on its objective function. Combinatorial optimization challenges encompass Boolean satisfiability and integer linear programming.

The classification of branch and bound problems

It is based on the sequential order in which the state space tree is explored. There are three kinds of branch and bound methods, which are as follows:

• FIFO branch and bound is a search algorithm that explores the search space in a FIFO manner while using a bounding function to determine which branches to explore further.

• LIFO branch and bound is a method used in optimization problems to find the most efficient solution. It involves exploring the search space by prioritizing the nodes with the most potential for improvement.

• Least cost-branch and bound, which aims to minimize the cost of the solution by iteratively evaluating and refining the search space.

0/1 knapsack problem

Provided are two arrays, V[] and W[], which represent the values and weights associated with n elements, respectively. Determine the subset with the highest value (maximum profit) from V[] such that the total weight of this subset is less than or equal to the knapsack capacity W.

Note: The limitation in this scenario is that we can only place an item in the bag either in its entirety or not at all. It is not possible to partially place an item in the bag.

Algorithm 9.1: Algorithm of 0/1 knapsack using branch and bound

1. Arrange all items in descending order based on their value-to-weight ratio to facilitate the computation of an upper limit using the greedy approach.

2. Begin by setting the maximum profit, maxProfit, to 0. Next, construct an empty queue called Q. Then, create a fake node for the decision tree and add it to the queue, Q. The dummy node has a profit and weight of 0.

3. Continue executing the following steps as long as Q is not empty:

a. Retrieve an element from Q. Designate the extracted item as u.

b. Calculate the profitability of the next level node. Update maxProfit if the profit exceeds the current maxProfit.

c. Calculate the upper limit of the node in the following level. If the bound exceeds the maximum profit, then append the next level node to the Q.

d. In this scenario, the next-level node is excluded from the solution. A node is added to the queue with the level set as the next level, but the weight and profit are calculated without taking into account the nodes at the next level.

Example 9.1: Let us take the following example: M is equal to 15, n is equal to 4, and we get the values (P1, P2, P3, P4) = (10, 10, 12, 18) and (w1, w2, w3, w4) = (2, 4, 6, 9).

Solution: You can use the branch and bound strategy to solve the 0/1 knapsack problem. This problem entails the calculation of both the lower limit and upper bound for each node.

Insert the initial item into the knapsack. The remaining weight of the knapsack is calculated by subtracting 2 from 15, resulting in a value of 13. Put the next item, W2, inside the backpack. The remaining weight of the knapsack is calculated as 13 - 4 = 9. Put the next item, W3, into the knapsack. As a result, the remaining weight of the knapsack will be 3, calculated by subtracting 6 from 9. Fractions are prohibited in the calculation of the upper bound; hence, W4 cannot be included in the knapsack.

The total profit, denoted as P, is equal to the sum of three components: P1, P2, and P3. Specifically, P1 is 10, P2 is 10, and P3 is 12. Therefore, the total profit is 32.

The upper bound is equal to 32.

To determine the lower bound, we can include item W4 in the knapsack, as the calculation of the lower bound allows for the consideration of fractional values.

The lower bound is calculated by adding 10, 10, 12, and the product of 3 multiplied by 18, which equals 32 plus 6, resulting in a value of 38.

Lower bound = 10 + 10 + 12 + ((3/9) X 18) = 32 + 6 = 38

The knapsack problem is a maximization problem; however, the branch and bound technique can only be applied to minimization problems. To change a maximization problem into a minimization problem, we need to assign a negative sign to both the upper bound and lower bound.

Therefore, the upper bound (U) = -32

Lower bound (L) = -38

We select the route that has the smallest disparity between the upper limit and lower limit. If the difference is equal, we determine the path by comparing the upper bounds and eliminate the node with the highest upper bound, according to the following figure:

Figure 9.1: Branch and bound up to first level

Now, we will determine the upper bound and lower bound for nodes 2 and 3. For node 2, x1 = 1, which means we should place the first item in the knapsack.

U = 10 + 10 + 12 = 32, write it as -32.

L = 10 + 10 + 12 + ((3/9) X 18 = 32 + 6 = 38, write it as -38.

For node 3, x1 = 0, which means we should not place the first item in the knapsack.

U = 10 + 12 = 22, write it as -22.

L = 10 + 12 + ((5/9) X 18 = 22 + 10 = 32, write it as -32.

Next, we determine UB and LB for nodes 2 and 3.

Node2 UB – LB = -32 + 38 = 6

Node3 UB – LB = -22 + 32 = 10

Figure 9.2 represents a tree up to two levels:

Figure 9.2: Branch and bound up to the second level

Select node 2 due to its lowest difference value, i.e., 6.

Now, calculate LB and UB for node 4 and node 5, as follows:

For node 4, UB – LB = -32 + 38 = 6.

For node 5, UB – LB = -22 + 36 = 14.

Figure 9.3 represents a tree with up to three levels:

Figure 9.3: Branch and bound up to the third level

Select node 4, having the lowest difference value, i.e., 6.

Now, calculate LB and UB for node 6 and node 7.

For node 6, UB – LB = -32 + 38 = 6.

For node 7, UB – LB = -38 + 38 = 0.

Select node 7, having the lowest difference value, i.e., 0.

Figure 9.4 represents a tree up to four levels:

Figure 9.4: Branch and bound up to the fourth level

Now, we will calculate LB and UB of nodes 8 and 9. Calculate the difference of lower and upper bound of nodes 8 and 9.

For node 8, UB – LB = -38 + 38 = 0.

For node 9, UB – LB = -20 + 20 = 0.

For this particular instance, the difference is the same; hence, compare the upper bounds of nodes 8 and 9. The node that has the highest possible upper bound should be discarded. Choose node 8, and leave node 9 out of consideration because it has the highest possible upper bound.

Consider the path from 1 -> 2 -> 4 -> 7 -> 8.

X1 = 1

X2 = 1

X3 = 0

X4 = 1

The final solution for 0/1 knapsack problem is (x1, x2, x3, x4) = (1, 1, 0, 1).

Maximum profit is as follows:

∑Pi xi = 10 x 1 + 10 x 1 + 12 x 0 + 18 x 1

= 10 + 10 + 18 = 38

Program 9.1: Implementation of 0/1 knapsack problem using branch and bound

// C++ program to solve knapsack problem using branch and bound

#include <bits/stdc++.h>

using namespace std;

// Structure for Item which store weight and corresponding value of Item

struct Item

{

float weight;

int value;

};

// Node structure to store information of decision tree

struct Node

{

// level --> Level of node in decision tree (or index in arr[]

// profit --> Profit of nodes on path from root to this node (including this node)

// bound ---> Upper bound of maximum profit in subtree of this node/

int level, profit, bound;

float weight;

};

// Comparison function to sort Item according val/weight ratio

bool cmp(Item a, Item b)

{

double r1 = (double)a.value / a.weight;

double r2 = (double)b.value / b.weight;

return r1 > r2;

}

// Returns bound of profit in subtree rooted with u. This function mainly uses Greedy solution to find

// an upper bound on maximum profit.

int bound(Node u, int n, int W, Item arr[])

{

// if weight overcomes the knapsack capacity, return 0 as expected bound

if (u.weight >= W)

return 0;

// initialize bound on profit by current profit

int profit_bound = u.profit;

// start including items from index 1 more to current item index

int j = u.level + 1;

int totweight = u.weight;

// checking index condition and knapsack capacity condition

while ((j < n) && (totweight + arr[j].weight <= W))

{

totweight += arr[j].weight;

profit_bound += arr[j].value;

j++;

}

// If k is not n, include last item partially for upper bound on profit

if (j < n)

profit_bound += (W - totweight) * arr[j].value / arr[j].weight;

return profit_bound;

}

// Returns maximum profit we can get with capacity W

int knapsack(int W, Item arr[], int n)

{

// sorting Item on basis of value per unit weight.

sort(arr, arr + n, cmp);

// make a queue for traversing the node

queue<Node> Q;

Node u, v;

// dummy node at starting

u.level = -1;

u.profit = u.weight = 0;

Q.push(u);

// One by one extract an item from decision tree compute profit of all children of extracted item and keep saving maxProfit

int maxProfit = 0;

while (!Q.empty())

{

// Dequeue a node

u = Q.front();

Q.pop();

// If it is starting node, assign level 0

if (u.level == -1)

v.level = 0;

// If there is nothing on next level

if (u.level == n-1)

continue;

// Else if not last node, then increment level, and compute profit of children nodes.

v.level = u.level + 1;

// Taking current level's item add current level's weight and value to node u's weight and value

v.weight = u.weight + arr[v.level].weight;

v.profit = u.profit + arr[v.level].value;

// If cumulated weight is less than W and profit is greater than previous profit, update maxprofit

if (v.weight <= W && v.profit > maxProfit)

maxProfit = v.profit;

// Get the upper bound on profit to decide whether to add v to Q or not.

v.bound = bound(v, n, W, arr);

// If bound value is greater than profit, then only push into queue for further consideration

if (v.bound > maxProfit)

Q.push(v);

// Do the same thing, but Without taking the item in knapsack

v.weight = u.weight;

v.profit = u.profit;

v.bound = bound(v, n, W, arr);

if (v.bound > maxProfit)

Q.push(v);

}

return maxProfit;

}

int main()

{

int W = 10; // Weight of knapsack

Item arr[] = {{2, 40}, {3.14, 50}, {1.98, 100}, {5, 95}, {3, 30}};

int n = sizeof(arr) / sizeof(arr[0]);

cout << "Maximum possible profit = " << knapsack(W, arr, n);

return 0;

}

Complexity of 0/1 knapsack problem

The following explanations show how we determined the time and space complexity of the 0/1 knapsack problem using branch and bound.

• Time complexity: The time complexity of the 0/1 knapsack problem using the branch and bound technique is 𝑂(2𝑛 ⋅ 𝑛 ⋅ log𝑛), where 𝑛 is the number of items. This complexity arises due to the exponential nature of the branch and bound algorithm, which explores all possible subsets of items to find the optimal solution.

• Space complexity: Regarding the space complexity, the branch and bound approach for the 0/1 knapsack problem typically requires 𝑂(𝑛) space to store the current state of the problem during the search process. This space complexity is relatively lower compared to the time complexity, as it mainly involves storing information about the current state of the search rather than maintaining a large data structure.

Traveling salesperson problem

We can solve the problem by utilizing the dynamic programming approach, which has a time complexity of O(n2.2n) for the worst-case scenario. The branch and bound strategy, which makes use of an efficient bounding function, can be used to tackle this problem. The temporal complexity of the traveling salesperson issue has been determined to be O(n2.2n) when utilizing the least cost (LC) branch and bound algorithm. This indicates that there has been no change or reduction in complexity compared to the original method.

We begin at a specific node, travel to each node exactly once, and then return to the initial node with the least amount of expense.

Let G = (V, E) be a connected graph. Let C(i, J) be the cost of edge <i, j>. cij = ∞ if <i, j> ₡ E, and let |V| = n, the number of vertices. Every tour starts at vertex 1 and ends at the same vertex. So, the solution space is given by S = {1, π, 1 | π is a permutation of (2, 3, . . . , n)} and |S| = (n – 1)!. The size of S can be reduced by restricting S so that (1, i1, i2, . . . . in-1, 1) € S if <ij, ij+1> € E, 0 <= j <= n - 1 and i0 = in = 1.

Algorithm 9.2: TSP algorithm

Refer to the following steps for the algorithm of the traveling salesperson problem using the branch and bound approach:

1. Reduce the cost matrix that has been provided. By reducing each row and column of a matrix, the matrix is said to be reduced. When a row or column contains at least one zero and all of the other entries are non-negative, we say that the row or column has already been reduced. The following is how this can be accomplished:

a. First, take the minimum element from the first row and subtract it from all of the items in the first row. Next, take the minimum element from the second row and subtract it from the second row. In the same manner, apply the same process to each and every row.

b. Determine the total number of components that were removed without being added to rows.

c. The matrix that was obtained following the row reduction should then have column reductions applied to it.

Taking the minimum element from the first column and subtracting it from all of the items in the first column is the first step in the column reduction process. Next, pick the minimum element from the second column and remove it from the second column. In the same manner, apply the same approach to each and every column.

d. Determine the total number of components that were removed from the sections of the table.

e. Determine the total amount in which row-wise reduction and column-wise reduction have been added together.

Cumulative reduced sum = row-wise reduction sum + column-wise reduction sum.

Associate the cumulative reduced sum to the starting state as lower bound and ∞ as upper bound.

2. Determine the reduced cost matrix for each node R. Let A represent the cost matrix that has been lowered for node R. Consider a child node S of the node R, where the tree edge (R, S) represents the inclusion of the edge <i, j> in the tour. To acquire the reduced cost matrix for S, it is necessary to check if S is a leaf node. If S is not a leaf node, the reduced cost matrix can be produced using the following method:

a. Replace all elements in row i and column j of matrix A with the symbol ∞.

b. Set the element A(j, 1) to infinity.

c. Eliminate all rows and columns in the resultant matrix except for the rows and columns that contain only infinity (∞). Let r represent the total amount deducted to reduce the matrix.

d. The equation c(S) = c(R) + A(i, j) + r calculates the cost function c(S), where r represents the total amount deducted to reduce the matrix. c(R) represents the lower bound of the ith node in the (i, j) path.

3. Repeat step number 2 until all nodes are traversed.

Example 9.2: Let us take the following example: Calculate the LC branch and bound solution for the traveling salesperson problem whose cost matrix is as follows:

Solution: The solution is as follows:

1. Calculate the reduced cost matrix as follows:

Subtract 10 (which is the minimum) from all values in the 1st row.

Subtract 2 (which is the minimum) from all values in the 2nd row.

Subtract 2 (which is the minimum) from all values in the 3rd row.

Subtract 3 (which is the minimum) from all values in the 4th row.

Subtract 4 (which is the minimum) from all values in the 5th row.

Now, row-wise reduction sum = 10 + 2 + 2 + 3 + 4 = 21.

Apply column reduction for resultant matrix.

Subtract 1 (which is the minimum) from all values in the 1st column.

Subtract 3 (which is the minimum) from all values in the 3rd column.

Now, column wise reduction sum = 1 + 0 + 3 + 0 + 0 = 4.

Cumulative reduced sum = row reduction sum + column reduction sum

= 21 + 4 = 25

This is the cost of a root, i.e., node 1, because this is the initially reduced cost matrix. The lower bound for the node is 25, and the upper bound is ∞. Starting from node 1, we can next visit the vertices 2, 3, 4, and 5. So, consider exploring paths (1, 2), (1, 3), (1, 4), and (1, 5).

After step 1, the tree structure would be as follows, where variable i denotes the next node to visit:

Figure 9.5: Branch and bound tree after phase 1

2. Let us pick up path (1, 2):

Change all entries of row 1 and column 2 of the matrix to ∞ and also set A(2, 1) to ∞.

Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 0 + 0 + 0 + 0 = 0

Column reduction sum = 0 + 0 + 0 + 0 = 0

Cumulative reduction (r) = 0 + 0 = 0

That is why, c(S) = c(R) + A(1, 2) + r

C(S) = 25 + 10 + 0 = 35

Now, pick up path (1, 3):

Change all entries of row 1 and column 3 of the matrix to ∞ and also set A(3, 1) to ∞.

Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 0 + 0 + 0 + 0 = 0

Column reduction sum = 11 + 0 + 0 + 0 = 11

Cumulative reduction (r) = 0 + 11 = 11

That is why, c(S) = c(R) + A(1, 3) + r

C(S) = 25 + 17 + 11 = 53

Now, pick up path (1, 4):

Change all entries of row 1 and column 4 of the matrix to ∞ and also set A(4, 1) to ∞.

Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 0 + 0 + 0 + 0 = 0

Column reduction sum = 0 + 0 + 0 + 0 = 0

Cumulative reduction (r) = 0 + 0 = 0

That is why, c(S) = c(R) + A(1, 4) + r

C(S) = 25 + 0 + 0 = 25

Now, pick up path (1, 5):

Change all entries of row 1 and column 5 of the matrix to ∞ and also set A(5, 1) to ∞.

Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 2 + 0 + 3 + 0 = 5

Column reduction sum = 0 + 0 + 0 + 0 = 0

Cumulative reduction (r) = 5 + 0 = 5

That is why, c(S) = c(R) + A(1, 5) + r

C(S) = 25 + 1 + 5 = 31

The tree structure after step 2 is as follows:

Figure 9.6: Branch and bound tree after phase 2

The cost of the paths between (1, 2) = 35, (1, 3) = 53, (1, 4) = 25, and (1, 5) = 31. The cost of the path between (1, 4) is minimum. Hence, the matrix obtained for path (1, 4) is considered as the reduced cost matrix.

The new available paths are (4, 2), (4, 3), and (4, 5).

3. Consider the path (4, 2):

Change all entries of row 4 and column 2 of A to ∞ and also set A(2, 1) to ∞.

Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 0 + 0 + 0 + 0 = 0

Column reduction sum = 0 + 0 + 0 + 0 = 0

Cumulative reduction (r) = 0 + 0 = 0

That is why, c(S) = c(R) + A(4, 2) + r

C(S) = 25 + 3 + 0 = 28

Consider the path (4, 3):

Change all entries of row 4 and column 3 of A to ∞ and also set A(3, 1) to ∞.

Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 2

Column reduction sum = 11

Cumulative reduction (r) = 2 + 11 = 13

That is why, c(S) = c(R) + A(4, 3) + r

C(S) = 25 + 12 + 13 = 50

Consider the path (4, 5):

Change all entries of row 4 and column 5 of A to ∞ and also set A(5, 1) to ∞.

Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 11

Column reduction sum = 0

Cumulative reduction (r) = 11 + 0 = 11

That is why, c(S) = c(R) + A(4, 5) + r

C(S) = 25 + 0 + 11 = 36

After step 3, the tree structure is as follows:

Figure 9.7: Branch and bound tree after phase 3

The cost of the paths between (4, 2) = 28, (4, 3) = 50, and (4, 5) = 36. The cost of the path between (4, 2) is minimum. Hence, the matrix obtained for path (4, 2) is considered as the reduced cost matrix.

The new possible paths are (2, 3) and (2, 5).

4. Consider the path (2, 3).

Change all entries of row 2 and column 3 of A to ∞ and also set A(3, 1) to ∞.

Apply row and column reduction for the rows and columns whose rows and columns are not completely ∞.

Row reduction sum = 2

Column reduction sum = 11

Cumulative reduction (r) = 2 + 11 = 13

That is why, c(S) = c(R) + A(4, 5) + r

C(S) = 28 + 11 + 13 = 52

Take the next path (2, 5).

Change all entries of row 2 and column 5 of A to ∞ and also set A(5, 1) to ∞.

Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 0

Column reduction sum = 0

Cumulative reduction (r) = 0 + 0 = 0

That is why, c(S) = c(R) + A(2, 5) + r

C(S) = 28 + 0 + 0 = 28

After step 4, the tree structure is as follows:

Figure 9.8: Branch and bound tree after phase 4

The cost of the paths between (2, 3) = 52 and (2, 5) = 28. The cost of the path between (2, 5) is minimum. Hence, the matrix obtained for path (2, 5) is considered as the reduced cost matrix.

The new possible path is (5, 3).

Let us consider this new path,

Change all entries of row 5 and column 3 of A to ∞ and also set A(3, 1) to ∞. Apply row and column reduction for the rows and columns that are not completely ∞.

Row reduction sum = 0

Column reduction sum = 0

Cumulative reduction (r) = 0 + 0 = 0

That is why, c(S) = c(R) + A(5, 3) + r

C(S) = 28 + 0 + 0 = 28

The final tree structure will be as follows:

Figure 9.9: Branch and bound tree after phase 5

The path of the traveling salesperson problem is:

1 — 4 — 2 — 5 — 3 — 1

The minimum cost of the path is 10 + 6 +2+ 7 + 3 = 28.

Program 9.2: Implementation of TSP using branch and bound

#include <iostream>

#include <vector>

#include <queue>

#include <utility>

#include <limits>

using namespace std;

const int INF = numeric_limits<int>::max();

int n;

vector<vector<int>> cost;

vector<bool> visited;

vector<int> path;

int minCost = INF;

struct Node {

vector<int> path;

int cost;

int level;

Node(vector<int> p, int c, int l) : path(p), cost(c), level(l) {}

};

int reducedMatrix(vector<vector<int>>& matrix) {

int n = matrix.size(); int cost = 0;

for (int i = 0; i < n; i++) {

int minVal = *min_element(matrix[i].begin(), matrix[i].end());

if (minVal != 0) {

cost += minVal;

for (int j = 0; j < n; j++) {

if (matrix[i][j] != INF) {

matrix[i][j] -= minVal;

}

}

}

}

vector<int> minVals(n, INF);

for (int i = 0; i < n; i++) {

for (int j = 0; j < n; j++) {

if (matrix[j][i] != INF && matrix[j][i] < minVals[i]) {

minVals[i] = matrix[j][i];

}

}

}

for (int i = 0; i < n; i++) {

if (minVals[i] != INF) {

cost += minVals[i];

for (int j = 0; j < n; j++) {

if (matrix[j][i] != INF) {

matrix[j][i] -= minVals[i];

}

}

}

}

return cost;

}

int bound(Node& node) {

int n = node.path.size();

vector<vector<int>> matrix(n, vector<int>(n, INF));

for (int i = 0; i < n; i++) {

for (int j = 0; j < n; j++) {

matrix[i][j] = cost[node.path[i]][node.path[j]];

}

}

int bound = node.cost + reducedMatrix(matrix);

for (int i = 0; i < n; i++) {

matrix[i][node.path.back()] = INF;

}

int temp = reducedMatrix(matrix);

bound += temp;

return bound;

}

void solve() {

queue<Node> q;

vector<int> path = {0};

q.push(Node(path, 0, 0));

while (!q.empty()) {

Node node = q.front();

q.pop();

if (node.level == n) {

if (cost[node.path.back()][0] != INF) {

int totalCost = node.cost + cost[node.path.back()][0];

if (totalCost < minCost) {

minCost = totalCost;

path = node.path;

}

}

} else {

for (int i = 1; i < n; i++) {

if (find(node.path.begin(), node.path.end(), i) == node.path.end() && cost[node.path.back()][i] != INF) {

vector<int> newPath = node.path;

newPath.push_back(i);

int newCost = node.cost + cost[node.path.back()][i];

Node newNode(newPath, newCost, node.level + 1);

int boundVal = bound(newNode);

if (boundVal < minCost) {

q.push(newNode);

}

}

}

}

}

}

int main() {

cout << "Enter the number of cities: ";

cin >> n;

cost = vector<vector<int>>(n, vector<int>(n, INF));

visited = vector<bool>(n, false);

cout << "Enter the cost matrix:\n";

for (int i = 0; i < n; i++) {

for (int j = 0; j < n; j++) {

cin >> cost[i][j];

} }

solve();

cout << "Minimum cost: " << minCost << endl;

cout << "Optimal path: ";

for (int city : path)

cout << city + 1 << " ";

cout << endl; return 0; }

Complexity of TSP using branch and bound

The following explanations outline the calculation of the time and space complexity of the traveling salesman problem using branch and bound:

• Time complexity: The time complexity of solving the traveling salesman problem with branch and bound is O(n!) in the worst case, akin to brute force. Despite this, branch and bound can excel by efficiently pruning the search space using lower bounds, leading to improved practical performance, especially for smaller instances. The choice of bounding function is critical for effectiveness. In practice, branch and bound can outperform brute force due to its ability to discard unpromising branches, making it a valuable approach for optimizing TSP solutions.

• Space complexity: The space complexity of the branch and bound algorithm for TSP is O(n), as it needs to store the current partial solution and the set of unvisited cities.

Resource allocation problem

The resource allocation problem is a classic example of an optimization problem. The objective of this problem is to determine the most effective way to distribute limited resources (such as personnel, machinery, or finances) among a number of different projects or activities. The purpose is often to achieve some other target while satisfying constraints, such as minimizing costs, maximizing earnings, or achieving some other objective.

There is a specific variation of this problem known as the job assignment problem, which can be handled by employing the branch and bound method.

Let us begin by looking at the problem, which is as follows:

Given N workers and N jobs, we want to assign each job to exactly one worker and each worker to exactly one job. Each assignment has an associated cost (or profit), and we want to find an assignment that minimizes the total cost (or maximizes the total profit).

In branch and bound, we intelligently explore the state space tree by considering promising nodes first. The key idea is to use an approximate cost function to guide the search. We choose a live node with the least cost (promising cost) to explore the next node in the list.

Two common approaches for the cost function are as follows:

1. For each worker, choose the job with the minimum cost from the list of unassigned jobs (minimum entry from each row).

2. For each job, choose the worker with the lowest cost for that job from the list of unassigned workers (minimum entry from each column).

We can significantly reduce the search space and improve efficiency by following this approach.

Algorithm 9.3: Algorithm of job assignment using branch and bound

/* The findMinCost function uses the Least() and Add() functions to manage the list of live nodes. Least() retrieves and removes the live node with the least cost from the list, returning it. Add(x) computes the cost of x and adds it to the list of live nodes. The list of live nodes is implemented as a min-heap. */

// Node structure for the Search Space Tree

struct Node {

int job_number;

int worker_number;

Node* parent;

int cost;

};

// Input: Cost matrix for the Job Assignment problem, Output: Optimal cost and job assignment

void findMinCost(int costMatrix[][]) {

// Initialize the list of live nodes (min-Heap) with the root of the search tree, i.e., a dummy node

while (true) {

// Retrieve and remove the live node with the least estimated cost

Node* E = Least();

// Check if the retrieved node is a leaf node

if (isLeafNode(E)) {

printSolution();

return;

}

// For each child node x of E

for (each child x of E) {

Add(x); // Add x to the list of live nodes

x->parent = E; // Set the parent pointer to maintain the path to the root

}

}

}

Example 9.3: Given the matrix for workers and jobs, we have to find the minimum cost to assign the job to the workers.

9

2

7

8

6

4

3

7

5

8

1

8

7

6

9

4

Table 9.1: Input matrix

Solution: Here, rows are mapped by A, B, C, and D, while columns denote jobs like J1, J2, J3, and J4. Value 8 means worker A will take eight units of time to complete job J4.

Let us consider job J2 is assigned to worker A, mark it as green. cost becomes 2, and job J2 and worker A become unavailable (marked in red) as follows:

9

2

7

8

6

4

3

7

5

8

1

8

7

6

9

4

Table 9.2: Matrix after pass 1

Now, we give job J3 to worker B because it has the lowest cost of the jobs that have not been given yet. The price goes up to 2 + 3 = 5, and job J3 and worker B are no longer available, which is shown as follows:

9

2

7

8

6

4

3

7

5

8

1

8

7

6

9

4

Table 9.3: Matrix after pass 2

The last job, job J4, is given to worker D because it is the only one left. Job J1 was given to worker C because it had the lowest cost of all the jobs that had not been assigned. Final cost = 2 + 3 + 5 + 4 = 14. The matrix is as follows:

9

2

7

8

6

4

3

7

5

8

1

8

7

6

9

4

Table 9.4: Matrix after pass 3

Program 9.3: Implementation of resource allocation using branch and bound

#include <bits/stdc++.h>

using namespace std;

#define N 4

struct Node

{

Node* parent;

int pathCost;

int cost;

int workerID;

int jobID;

bool assigned[N];

};

// Function to allocate a new search tree node, here Person x is assigned to job y

Node* newNode(int x, int y, bool assigned[], Node* parent)

{

Node* node = new Node;

for (int j = 0; j < N; j++)

node->assigned[j] = assigned[j];

node->assigned[y] = true;

node->parent = parent;

node->workerID = x;

node->jobID = y;

return node;

}

// Function to calculate the least promising cost of node after worker x is assigned to job y.

int calculateCost(int costMatrix[N][N], int x, int y, bool assigned[])

{

int cost = 0;

bool available[N] = {true};

for (int i = x + 1; i < N; i++)

{

int min = INT_MAX, minIndex = -1;

for (int j = 0; j < N; j++)

{

if (!assigned[j] && available[j] && costMatrix[i][j] < min)

{

minIndex = j;

min = costMatrix[i][j];

} }

cost += min;

available[minIndex] = false;

}

return cost; }

struct comp

{

bool operator()(const Node* lhs, const Node* rhs) const

{

return lhs->cost > rhs->cost;

}

};

void printAssignments(Node *min)

{

if(min->parent==NULL)

return;

printAssignments(min->parent);

cout << "Assign Worker " << char(min->workerID + 'A') << " to Job " << min->jobID << endl;

}

int findMinCost(int costMatrix[N][N])

{

// Create a priority queue to store live nodes of search tree;

priority_queue<Node*, std::vector<Node*>, comp> pq;

bool assigned[N] = {false};

Node* root = newNode(-1, -1, assigned, NULL);

root->pathCost = root->cost = 0;

root->workerID = -1;

pq.push(root);

while (!pq.empty())

{

Node* min = pq.top();

pq.pop();

int i = min->workerID + 1;

if (i == N)

{

printAssignments(min);

return min->cost;

}

for (int j = 0; j < N; j++)

{

if (!min->assigned[j])

{

Node* child = newNode(i, j, min->assigned, min);

child->pathCost = min->pathCost + costMatrix[i][j];

child->cost = child->pathCost + calculateCost(costMatrix, i, j, child->assigned);

pq.push(child);

}

}

}

}

int main()

{

int costMatrix[N][N] = { {9, 2, 7, 8},

{6, 4, 3, 7},

{5, 8, 1, 8},

{7, 6, 9, 4}};

cout << "\n Optimal Cost is " << findMinCost(costMatrix);

return 0;

}

Complexity of resource allocation

O(M*N) is the time complexity. This is because a double for loop is used to go through the M x N matrix over and over again.

We denote space for extras as O(M+N) because it tracks applicants and jobs with two groups of size M and N.

Quadratic assignment problem

The objective is to identify the allocation that minimizes the overall cost or distance, considering both the distances and the flows. The quadratic assignment problem can be formulated as a quadratic objective function by utilizing the distance matrix, flow matrix, and restrictions that guarantee each facility is assigned to only one location and each location is assigned to only one facility.

The quadratic assignment problem (QAP) is a widely recognized instance of an NP-hard problem, implying that finding the optimal solution might be challenging for bigger instances. Consequently, numerous methods and heuristics have been developed to locate close approximations of solutions rapidly.

Algorithm 9.4: QAP using branch and bound algorithm

The steps of the algorithm are as follows:

1. Define the distance matrix (D) and the flow matrix (F).

a. Set the initial bound (can be calculated using a heuristic or set to infinity).

b. Initialize an empty solution and the best solution found so far.

2. At each node of the search tree, choose a facility-location pair to assign.

a. Generate child nodes by assigning the facility to each unassigned location.

3. Calculate a lower bound for each node. This can be done using relaxation techniques or heuristics.

a. If the lower bound of a node is greater than or equal to the current best-known solution, prune that branch.

4. Use a priority queue (or stack for depth-first search) to explore nodes based on their bounds.

a. Update the best solution when a complete assignment with a lower cost is found.

5. The algorithm terminates when all nodes have been explored or pruned. The best solution at the end is the optimal solution.

Example 9.4: Considering four facilities (F1, F2, F3, F4) and four sites (L1, L2, L3, L4). Our cost matrix shows the pairwise distances, that is, costs, between facilities. We also have a flow matrix to show the interactions, that is, the flow between sites. Based on interconnections between facilities and sites, identify the project that reduces the overall cost. Every facility has to be allocated to exactly one location, and that location can only house one other facility.

The cost matrix is as follows:

L1

L2

L3

L4

F1

0

2

3

1

F2

2

0

1

4

F3

3

1

0

2

F4

1

4

2

0

Table 9.5: Cost matrix

The flow matrix is as follows:

L1

L2

L3

L4

F1

0

1

2

3

F2

1

0

4

2

F3

2

4

0

1

F4

3

2

1

0

Table 9.6: Flow matrix

Solution: The final solution of the preceding problem is as follows:

In order to determine the overall expense, we examine every combination of facilities denoted by (i, j) and their corresponding positions (location1, location2). The product of the cost of allocating facility1 to facility2 (facilities[facility1][facility2]) and the flow from location1 to location2 (locations[location1][location2]) is calculated. This procedure is carried out for every combination of facilities in the assignment, and the costs are aggregated.

The optimal assignment is F1:L1, F3:L2, F2:L3, F4:L4

The total cost is 44.

Program 9.4: Implementation of QAP using branch and bound

#include <iostream>

#include <vector>

#include <algorithm>

using namespace std;

int calculateTotalCost(const vector<vector<int>>& facilities, const vector<vector<int>>& locations, const vector<int>& assignment) {

int totalCost = 0;

int n = facilities.size();

for (int i = 0; i < n; i++) {

for (int j = 0; j < n; j++) {

int facility1 = assignment[i];

int facility2 = assignment[j];

int location1 = i;

int location2 = j;

totalCost += facilities[facility1][facility2] * locations[location1][location2];

}

}

return totalCost;

}

int main() {

// Facilities cost matrix

vector<vector<int>> facilities = {

{0, 2, 3, 1},

{2, 0, 1, 4},

{3, 1, 0, 2},

{1, 4, 2, 0}

};

// Flow matrix

vector<vector<int>> locations = {

{0, 1, 2, 3},

{1, 0, 4, 2},

{2, 4, 0, 1},

{3, 2, 1, 0}

};

int n = facilities.size();

// Generate initial assignment (0, 1, 2, 3)

vector<int> assignment(n);

for (int i = 0; i < n; i++) {

assignment[i] = i;

}

// Calculate the initial total cost

int minCost = calculateTotalCost(facilities, locations, assignment);

vector<int> minAssignment = assignment;

// Generate all permutations of the assignment

while (next_permutation(assignment.begin(), assignment.end())) {

int cost = calculateTotalCost(facilities, locations, assignment);

if (cost < minCost) {

minCost = cost;

minAssignment = assignment;

}

}

// Print the optimal assignment and total cost

cout << "Optimal Assignment: ";

for (int i = 0; i < n; i++) {

cout << "F" << (minAssignment[i] + 1) << "->L" << (i + 1) << " ";

}

cout << endl;

cout << "Total Cost: " << minCost << endl;

return 0;

}

Complexity of QAP using branch and bound

The worst-case time complexity of QAP is (O(n!)) due to factorial growth in the number of possible assignments (permutations).

Job sequencing problem

One processor and n jobs are assigned to us. Every job i does corresponds with a three-tuples pi, di, ti. Job i need ti units of processing time. Should its processing not meet the deadline, di, a penalty, pi, results. The aim is to choose a subset J of the n jobs such that their deadlines allow every job in J to be finished. Consequently, a penalty might be paid just on those positions not under J. The subset J should be such that among all conceivable subsets J, the penalty obtained is minimum. Such J is optimal.

Example 9.5: Let us assume the following instances having n=4:

(p1, d1, t1) = (5, 1, 1),

(p2, d2, t2) = (10, 3, 2),

(p3, d3, t3) = (6, 2, 1),

(p4, d4, t4) = (3, 1, 1)

Solution: We will follow the FIFO approach to solve the problem.

The cost function c(x) for any circular node x is the minimum penalty corresponding to any node in the subtree with root x.

The value of c(x) = ∞ for a square node.

Let Sx be the subset of jobs selected for J at node x.

If m = max {i | i є Sx }, then Cx = ∑Pi.

S(2) = {1}

m=1

S(3) = {2}

m=2

S(4) = {3}

m=3

S(5) = {4}

m=4

S(6) = {1, 2}

m=2

S(7) = {1, 3}

m=3

S(8) = {1, 4}

m=4

S(9) = {2,3}

m=3

S(10) = {2, 4}

m=3

S(11) = {3, 4}

m=4

Table 9.7: Job sequencing problem solution

The calculation of the upper bound, U(x) = ∑ pi, where i does not belong to Sx.

U(1) = 0

U(2) = P2 + P3 + P4 = 10 + 6 + 3 = 19 job 1 eliminated

U(3) = P1 + P3 + P4 = 5 + 6 + 3 = 14 job 2 eliminated

U(4) = P1 + P2 + P4 = 5 + 10 + 3 = 18 job 3 eliminated

U(5) = P1 + P2 + P3 = 5 + 10 + 6 = 21 job 4 eliminated

U(6) = P3 + P4 = 6 + 3 = 9 jobs 1, 2 eliminated

U(7) = P2 + P4 = 10 + 3 = 13 jobs 1, 3 eliminated

U(8) = P2 + P3 = 10 + 6 = 16 jobs 1, 4 eliminated

U(9) = P1 + P4 = 5 + 3 = 8 jobs 2, 3 eliminated

U(10) = P1 + P3 = 5 + 6 = 11 jobs 2, 4 eliminated

U(11) = P1 + P2 = 5 + 10 = 15 jobs 3, 4 eliminated

Algorithm 9.5: Job sequencing using branch and bound

The algorithm steps for job sequencing using branch and bound are as follows:

1. Sort jobs by profit: Sort the jobs in decreasing order of profit because we want to maximize profit.

2. State space tree: The solution space is represented as a tree. Each node in the tree represents a partial solution (a subset of jobs scheduled up to that point).

3. Bounding function: For a node, compute an upper bound on the maximum profit that can be obtained from that node. If the upper bound is less than the best-known solution, prune that node.

4. Branching: For each node, generate child nodes by including or excluding the next job in the list.

5. Pruning: Use the bounding function to eliminate nodes that cannot yield a better solution than the current best.

Program 9.5: Implementation of job sequencing using branch and bound

#include <iostream>

#include <vector>

#include <algorithm>

#include <queue>

using namespace std;

struct Job {

int id;

int deadline;

int profit;

};

struct Node {

vector<bool> job_schedule;

int level;

int profit;

int bound;

};

// Comparison function to sort jobs according to their profits

bool jobComparator(Job a, Job b) {

return (a.profit > b.profit); }

// Function to calculate upper bound of profit in subtree rooted with 'u'

int calculateBound(Node u, int n, vector<Job>& jobs) {

if (u.level >= n) return 0;

int profit_bound = u.profit;

int total_time = 0;

for (int i = u.level; i < n; i++) {

total_time += jobs[i].deadline;

if (total_time <= jobs[i].deadline) {

profit_bound += jobs[i].profit;

}

} return profit_bound;

}

// Branch and Bound algorithm for Job Sequencing

void jobSequencing(vector<Job>& jobs, int n) {

sort(jobs.begin(), jobs.end(), jobComparator);

queue<Node> Q;

Node u, v;

u.level = -1;

u.profit = 0;

u.job_schedule.resize(n, false);

u.bound = calculateBound(u, n, jobs);

Q.push(u);

int max_profit = 0;

vector<bool> best_schedule;

while (!Q.empty()) {

u = Q.front();

Q.pop();

if (u.level == -1) v.level = 0;

if (u.level == n - 1) continue;

v.level = u.level + 1;

v.job_schedule = u.job_schedule;

v.job_schedule[v.level] = true;

v.profit = u.profit + jobs[v.level].profit;

if (v.profit > max_profit) {

max_profit = v.profit;

best_schedule = v.job_schedule;

}

v.bound = calculateBound(v, n, jobs);

if (v.bound > max_profit) Q.push(v);

v.job_schedule[v.level] = false;

v.profit = u.profit;

v.bound = calculateBound(v, n, jobs);

if (v.bound > max_profit) Q.push(v);

}

cout << "Scheduled Jobs: ";

for (int i = 0; i < n; i++) {

if (best_schedule[i]) {

cout << jobs[i].id << " ";

}

}

cout << "\n Max Profit: " << max_profit << endl;

}

int main() {

vector<Job> jobs = {

{1, 2, 100},

{2, 1, 19},

{3, 2, 27},

{4, 1, 25},

{5, 3, 15}

};

int n = jobs.size();

jobSequencing(jobs, n);

return 0;

}

Complexity of job sequencing using branch and bound

The time complexity and space complexity of the job sequencing problem using branch and bound approach is calculated as follows:

• Time complexity: The time complexity of the branch and bound algorithm for job sequencing depends on several factors, including the number of jobs 𝑛, the structure of the state space tree, and the efficiency of the bounding function. The following is a breakdown of the factors affecting the time complexity:

o State space tree size: The state space tree can have up to 2n nodes in the worst case, since each job can either be included or excluded from the schedule. Therefore, the size of the state space tree is 𝑂(2n).

o Bounding function: For each node, we compute the bound, which in the worst case might require 𝑂(𝑛) operations (to calculate the remaining profit from unscheduled jobs). However, this bounding function can often be computed more efficiently with optimizations.

o Queue operations: In the worst case, every node might be added to the queue, and each node might be processed once. Thus, the operations on the queue, such as insertion and extraction, can be 𝑂(2n).

Considering these factors, the worst-case time complexity of the Branch and bound algorithm for job sequencing is 𝑂(2n. n). However, the actual complexity can be much lower in practice due to effective pruning by the bounding function, which reduces the number of nodes that need to be explored.

• Space complexity: The space complexity of the branch and bound algorithm is determined by the storage required for the queue and the state space tree nodes, as follows:

o Queue size: In the worst case, the queue can hold up to 𝑂(2n) nodes. Each node contains information about the current state, including the job schedule (a Boolean vector of size n), profit, bound, and level.

o Auxiliary space: Additional space is required for storing job information and intermediate calculations. Thus, the space complexity of the branch and bound algorithm for job sequencing is 𝑂(2n.𝑛) in the worst case.

Conclusion

Branch and bound is an optimization method that breaks problems down into smaller problems and gets rid of non-optimal branches, which speeds up the search for solutions. This chapter discusses the applications of branch and bound in work scheduling, resource allocation, quadratic assignment, 0/1 knapsack, and traveling salesperson problems. Each of these problems shows how branch and bound can be used to solve difficult computer problems by keeping upper and lower solution bounds. Using heuristics and boundary functions is useful in computer science and engineering because it greatly reduces the amount of work that needs to be done on processing. Understanding these algorithms enables the development of superior solutions for various issues, including scheduling and logistics.

In the next chapter, the readers will learn about graph theory in computer science. Graph theory offers a versatile framework for representing relationships and connections between entities, making it essential for various applications across numerous fields.

Exercise

1. What do you mean by the branch and bound technique? Why is it better?

2. Find the solution using 0/1 knapsack problem, M is equal to 18, n is equal to 4, and we get the values (P1, P2, P3, P4) = (12, 10, 11, 16) and (w1, w2, w3, w4) = (3, 4, 7, 8).

3. What is the significance of branch and bound to solve the traveling salesman problem?

4. How can job sequencing be implemented using the branch and bound mechanism? Explain with a suitable example.

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 10Graph Algorithms

Introduction

This chapter will provide a concise overview of graph theory, a key area within computer science dedicated to solving problems related to graphs. Graph theory offers a versatile framework for representing relationships and connections between entities, making it essential for various applications across numerous fields. Graphs consist of nodes (or vertices) connected by edges (or links), and they can model many real-world systems. For example, in social networks, nodes represent individuals and edges represent friendships; in transportation networks, nodes represent cities and edges represent routes, and in biological networks, nodes and edges depict interactions between genes or proteins.

Structure

The chapter covers the following topics:

• Basic terminology and representation of graph

• Topological sort

• Strongly connected components

• Eulerian path and circuit

• Single-source shortest path algorithm

• All-pair shortest paths

• Max-flow min-cut

Objectives

This chapter aims to offer a thorough grasp of fundamental graph theory concepts and algorithms. Graph theory is a crucial discipline in computer science, providing effective tools for addressing intricate problems concerning networks, pathways, and connections. By engaging with this chapter, you will acquire the knowledge and skills necessary to model, analyze, and solve problems using graph-based approaches.

Basic terminology and representation of graph

A graph can be described as a collection of vertices and edges that connect these vertices. Unlike a tree structure, a graph allows for complex relationships between its vertices (nodes), rather than a strict parent-child hierarchy.

A graph 𝐺 can be formally defined as an ordered pair 𝐺(𝑉,𝐸s) where:

• V(G) represents the set of vertices.

• E(G) represents the set of edges that connect these vertices.

Consider a graph G(V, E) with five vertices (A, B, C, D, E) and six edges: (A, B), (B, C), (C, E), (E, D), (D, B), (D, A). This graph is illustrated in the following figure:

Figure 10.1: Graph

Directed and undirected graph

A graph can be either directed or undirected. In an undirected graph, edges do not have associated directions. Figure 10.1 illustrates an undirected graph, where edges are not marked with any specific direction. For example, if there is an edge between vertex A and B, you can traverse from B to A as well as from A to B.

In a directed graph, edges are represented as ordered pairs. Each edge signifies a specific directed path from a vertex A to another vertex B. Node A is referred to as the initial node, while node B is the terminal node. Figure 10.2 shows an example of a directed graph:

Figure 10.2: Directed graph

The following figure illustrates an undirected graph:

Figure 10.3: Undirected graph

Graph terminology

The following are important terms related to graphs:

• Path: A path is a sequence of nodes that are traversed in order to reach a terminal node V from an initial node U.

• Closed path: A path is called a closed path if the initial node is the same as the terminal node. A closed path is denoted as V0=VN.

• Simple path: A simple path P is a path where all nodes are distinct, except possibly the initial and terminal nodes V0 and VN.

• Cycle: A cycle is a path that has no repeated edges or vertices, except that the first and last vertices are the same.

• Connected graph: A connected graph is one in which there is a path between every pair of vertices (𝑢, 𝑣) in 𝑉. There are no isolated nodes in a connected graph.

• Complete graph: A complete graph is one in which every node is connected to all other nodes. A complete graph on n nodes contains n(n-1)/2 edges.

• Weighted graph: In a weighted graph, each edge is assigned some data, such as length or weight. The weight w(e) of an edge e is a positive value indicating the cost of traversing the edge.

• Digraph: A digraph (directed graph) is a graph in which each edge is associated with a direction, and traversal is allowed only in the specified direction.

• Loop: A loop is an edge that connects a vertex to itself.

• Adjacent nodes: Two nodes 𝑢 and 𝑣 are adjacent if they are connected by an edge 𝑒. They are also referred to as neighbors.

• Degree of the node: The degree of a node is the number of edges connected to that node. A node with degree 0 is called an isolated node.

Representation of graph

Graph representation refers to the method used to store a graph in a computer's memory. A graph is a data structure consisting of a set of vertices (also known as nodes) and edges. There are two primary methods to store graphs in a computer's memory:

• Sequential representation (or, adjacency matrix representation).

• Linked list representation (or, adjacency list representation).

In sequential representation, the graph is stored using an adjacency matrix. In a linked list representation, an adjacency list is used to store the graph.

Sequential representation

In sequential representation, an adjacency matrix is employed to depict the connections between vertices and edges within the graph. This matrix can be utilized to represent undirected graphs, directed graphs, weighted directed graphs, and weighted undirected graphs.

If adj[i][j] = w, it indicates that there exists an edge from vertex i to vertex j with weight w.

In the adjacency matrix representation of an undirected graph G, an entry Aij will be 1, if an edge exists between vertices Vi and Vj. If an undirected graph G consists of n vertices, then its adjacency matrix A, denoted as aij, is an n x n matrix defined as:

aij =1 {if there is a path that exists from Vi to Vj}

aij =0 {otherwise}

In an adjacency matrix, 0 indicates no connection between nodes, while 1 indicates the presence of an edge between two nodes.

If the graph does not contain self-loops, the diagonal entries of the adjacency matrix will be 0.

Now, let us examine the adjacency matrix representation of an undirected graph:

Figure 10.4: Adjacency matrix of an undirected graph

In the aforementioned figure, an image illustrates the mapping among the vertices (A, B, C, D, E) using the adjacency matrix.

There are different adjacency matrices for directed and undirected graphs. In a directed graph, an entry Aij will be 1 only when there is an edge directed from Vi to Vj.

Adjacency matrix for a directed graph

In a directed graph, edges represent a specific path from one vertex to another. If there is a path from vertex A to vertex B, it means that vertex A is the initial node and vertex B is the terminal node.

Consider the following directed graph and attempt to construct its adjacency matrix:

Figure 10.5: Adjacency Matrix of a directed graph

Adjacency matrix for a weighted directed graph

The adjacency matrix representation of a weighted directed graph is similar to that of a directed graph, with the difference that instead of using 1 to indicate the existence of a path, we use the weight associated with the edge. The weights of the graph edges are represented as the entries of the adjacency matrix. This can be illustrated with an example. Consider the graph below and its adjacency matrix representation, where the weights associated with the edges are shown as the entries in the adjacency matrix:

Figure 10.6: Adjacency matrix of a weighted directed graph

In Figure 10.6, observe that the adjacency matrix representation of the weighted directed graph differs from other representations. This variation arises from replacing non-zero values with the actual weights assigned to the edges. While an adjacency matrix is straightforward to implement and comprehend, it is more space-intensive. This matrix is beneficial for dense graphs with numerous edges. However, even for sparse graphs, the matrix occupies the same amount of space.

Linked list representation

In the linked list representation, an adjacency list is used to store the graph in the computer's memory. This method is efficient in terms of storage because we only need to store the values for edges. Let us understand the adjacency list representation of an undirected graph:

Figure 10.7: Adjacency list of an undirected graph

In Figure 10.7, we can observe that each node of the graph has a linked list or adjacency list associated with it. From vertex A, there are paths to vertices B and D. These nodes are linked to node A in the provided adjacency list. An adjacency list is maintained for each node in the graph, storing the node's value and a pointer to the next adjacent node. Once all adjacent nodes have been traversed, the pointer field of the last node in the list is set to NULL. The sum of the lengths of adjacency lists equals twice the number of edges present in an undirected graph.

Now, consider the directed graph, and let us examine its adjacency list representation:

Figure 10.8: Adjacency list of a directed graph

In a directed graph, the sum of the lengths of adjacency lists equals the number of edges present in the graph. Now, let us consider a weighted directed graph and examine its adjacency list representation as shown in the following figure:

Figure 10.9: Adjacency list of a directed graph

In a weighted directed graph, each node includes an additional field called the weight of the node. Using an adjacency list makes it easy to add a vertex, and the use of linked lists also helps save space.

Topological sort

Topological sort is a graph theory technique used to order the vertices of a directed acyclic graph (DAG). It ensures that for every directed edge from vertex A to vertex B, vertex A precedes vertex B in the ordering. This technique is particularly useful in scheduling problems where tasks must be performed in a specific sequence. The algorithm starts by selecting a vertex with no incoming edges, adding it to the ordering, and then removing all its outgoing edges. This process is repeated until all vertices are visited, resulting in a topological sort of the DAG.

Algorithm 10.1: Topological sort

function Topsort(G)

T ← empty list .

Z ← empty stack

in ← dictionary mapping all vertices to 0 .

for each v ∈ V do

for each u adjacent to v do

increment in[v]

for each v ∈ V do

if in[v] = 0 then

add v to Z

while S is not empty do

v ← Z.remove

append v to T

for each u adjacent to v do

decrement in[u]

if in[u] = 0 then

add u to Z

return T

Program 10.1: Implementation in C++

#include <iostream>

#include <vector>

#include <queue>

using namespace std:

void topologicalSort(vector<vector<int>>& graph, int numVertices) {

vector<int> inDegree(numVertices, 0):

queue<int> zeroInDegreeQueue:

vector<int> topoOrder:

// Calculate in-degrees of all vertices

for (int i = 0: i < numVertices: i++) {

for (int j : graph[i]) {

inDegree[j]++:

}

}

// Enqueue vertices with zero in-degree

for (int i = 0: i < numVertices: i++) {

if (inDegree[i] == 0) {

zeroInDegreeQueue.push(i):

}

}

while (!zeroInDegreeQueue.empty()) {

int vertex = zeroInDegreeQueue.front():

zeroInDegreeQueue.pop():

topoOrder.push_back(vertex):

// Decrease in-degree for all adjacent vertices

for (int adjVertex : graph[vertex]) {

inDegree[adjVertex]--:

if (inDegree[adjVertex] == 0) {

zeroInDegreeQueue.push(adjVertex):

}

}

}

// Check for a cycle

if (topoOrder.size() != numVertices) {

cout << "Graph has a cycle, topological sort not possible." << endl:

return:

}

// Print topological order

for (int vertex : topoOrder) {

cout << vertex << " ":

}

cout << endl:

}

int main() {

int numVertices = 6:

vector<vector<int>> graph(numVertices):

// Example graph (DAG)

graph[5] = { 2, 0 }:

graph[4] = { 0, 1 }:

graph[2] = { 3 }:

graph[3] = { 1 }:

topologicalSort(graph, numVertices):

return 0:

}

Example 10.1: Generate a topological ordering for the following directed graph:

Figure 10.10: Directed graph for topological ordering

Solution: Identify the first node in the topological ordering. This node must have no incoming directed edges, meaning its in-degree is zero. Find a node with an in-degree of zero and add it to the topological ordering.

Figure 10.11: Topological ordering first step

Once a node is added to the topological ordering, remove the node and its outgoing edges from the graph:

Figure 10.12: Topological ordering second step

Now, repeat:

Figure 10.13: Topological ordering third step

Then, repeat:

Figure 10.14: Topological ordering fourth step

Until we reach the last node:

Figure 10.15: Topological ordering final step

Complexity analysis

The complexity analysis is as follows:

• Time complexity: The DFS-based method employs a depth-first search and utilizes a stack to maintain vertices in the order they complete processing. The complexity of both entries is mentioned below.

o DFS traversal: Each vertex is visited once, and all its adjacent vertices are explored, resulting in O(V+E) time complexity, akin to standard graph traversal.

o Stack operations: Pushing and popping vertices from the stack totals O(V) time.

Consequently, the overall time complexity of the DFS-based method remains O(V+E).

• Space complexity: Both methods necessitate additional space for data structures employed during the process.

o Adjacency list: Storing the graph using an adjacency list requires O(V + E) space.

o DFS-based method: It uses a stack to store the topological order, requiring O(V) space, and a visited array also taking O(V) space.

Therefore, the space complexity for both methods is O(V + E).

Applications

Some significant applications of topological sort include:

• Scheduling jobs based on dependencies among jobs

• Instruction scheduling

• Determining the order of compilation tasks in make files

• Data serialization

Strongly connected components

Strongly connected components (SCCs) in a directed graph are subsets of vertices where every vertex is reachable from every other vertex in the same subset. In simpler terms, within an SCC, there is a path from any vertex to every other vertex.

An SCC in a directed graph 𝐺 is a maximal subgraph 𝐶 such that for every pair of vertices 𝑢, 𝑣∈𝐶, there exists both a path from 𝑢 to 𝑣 and a path from 𝑣 to 𝑢 in 𝐺.

An algorithm known as Kosaraju's Algorithm is used to identify strongly connected components.

Algorithm 10.2: Strongly connected component

procedure Kosaraju(G):

Input: Directed graph G

Output: List of strongly connected components (SCCs)

n = number of vertices in G

visited[n] = false

stack = empty stack

finishOrder = empty list

// Step 1: Perform a DFS and record the finishing order

procedure DFS1(v):

visited[v] = true

for each (v, u) in G.adjacentEdges(v):

if not visited[u]:

DFS1(u)

stack.push(v)

finishOrder.push(v)

for each v in G.vertices():

if not visited[v]:

DFS1(v)

// Step 2: Transpose the graph G

G_transposed = G.transpose()

// Step 3: Perform DFS on the transposed graph in the order of finishOrder

procedure DFS2(v):

visited[v] = true

add v to current SCC

for each (v, u) in G_transposed.adjacentEdges(v):

if not visited[u]:

DFS2(u)

visited[n] = false

SCCs = empty list

while stack is not empty:

v = stack.pop()

if not visited[v]:

create a new empty set for current SCC

DFS2(v)

add current SCC to SCCs

return SCCs

Program 10.2: Implementation in C++

#include <iostream>

#include <vector>

#include <stack>

#include <algorithm>

using namespace std:

class Graph {

int V: // Number of vertices

vector<vector<int>> adj: // Adjacency list

public:

Graph(int V) : V(V) {

adj.resize(V):

}

void addEdge(int u, int v) {

adj[u].push_back(v):

}

void DFS1(int v, vector<bool>& visited, stack<int>& finishStack) {

visited[v] = true:

for (int u : adj[v]) {

if (!visited[u]) {

DFS1(u, visited, finishStack):

}

}

finishStack.push(v):

}

void DFS2(int v, vector<bool>& visited, vector<int>& SCC) {

visited[v] = true:

SCC.push_back(v):

for (int u : adj[v]) {

if (!visited[u]) {

DFS2(u, visited, SCC):

}

}

}

Graph transpose() {

Graph transposed(V):

for (int v = 0: v < V: ++v) {

for (int u : adj[v]) {

transposed.addEdge(u, v):

}

}

return transposed:

}

vector<vector<int>> findSCCs() {

vector<bool> visited(V, false):

stack<int> finishStack:

// Step 1: Fill finishStack in order of finish times

for (int v = 0: v < V: ++v) {

if (!visited[v]) {

DFS1(v, visited, finishStack):

}

}

// Step 2: Get the transposed graph

Graph transposed = transpose():

// Reset visited array

fill(visited.begin(), visited.end(), false):

// Step 3: Process vertices in order defined by finishStack

vector<vector<int>> SCCs:

while (!finishStack.empty()) {

int v = finishStack.top():

finishStack.pop():

if (!visited[v]) {

vector<int> SCC:

transposed.DFS2(v, visited, SCC):

SCCs.push_back(SCC):

}

}

return SCCs:

}

}:

int main() {

Graph g(5): // Example graph with 5 vertices

// Adding edges

g.addEdge(0, 1):

g.addEdge(1, 2):

g.addEdge(2, 0):

g.addEdge(1, 3):

g.addEdge(3, 4):

// Finding Strongly Connected Components

vector<vector<int>> SCCs = g.findSCCs():

// Output SCCs

cout << "Strongly Connected Components:\n":

for (const auto& SCC : SCCs) {

for (int v : SCC) {

cout << v << " ":

}

cout << "\n":

}

return 0:

}

Example 10.2: Find SCC of the following graph:

Figure 10.16: Strongly connected graph

Solution: To find SCC, follow these steps:

1. In the initial step, conduct a DFS traversal to establish the priorities of the vertices. Specifically, for each vertex, record when the DFS completes its processing. The vertices that are processed later have higher priorities. This step resembles topological sorting but is distinct because it is applicable to any directed graph, not just DAGs.

The preceding section illustrates the outcome of executing this step for the graph. The process began with vertex 1 (though it could have started from any node). The priorities are indicated by the numbers outside the circles:

Figure 10.17: SCC step 1

During the DFS traversal, push the vertices onto a stack based on their priorities. At the end of the process, the stack will contain: [3, 4, 7, 8, 6, 5, 2, 1].

2. In the second step, we construct the transpose graph of G. The transpose graph TG has the same vertices as the original graph but with reversed edge directions:

Figure 10.18: SCC step 2

Construct TG as a separate graph where the edge directions are inverted compared to G.

3. After completing step 2, vertex 1 has the highest priority. Therefore, we initiate the first DFS from vertex 1. During this DFS traversal, vertices 1, 3, and 2 are visited in that order, identifying one SCC:

Figure 10.19: SCC step 3

The next vertex with the highest priority is 5, given that vertex 2 has already been visited. We execute DFS starting from node 5, identifying the second SCC, which contains only that node:

Figure 10.20: SCC step 4

Vertex 6 is the next vertex to be visited. During DFS, we traverse vertices 6, 7, and 8 in this order, resulting in:

Figure 10.21: SCC step 5

Lastly, invoke DFS for vertex 4 and find the final SCC containing only that vertex:

Figure 10.22: SCC step 6

At this point, all the SCCs are obtained, and the algorithm terminates.

Complexity analysis

In Kosaraju’s algorithm, the adjacency list representation for G ensures that DFS runs in O(N+M) time. The algorithm consists of three steps, each depending on the variables N (number of vertices) and M (number of edges). Therefore, the algorithm’s complexity is determined by the maximum complexity among these steps:

• Step 1 involves a DFS traversal of G, which runs in O(N+M).

• Transposing G requires iterating over all vertices and edges, also operating in O(N+M) time.

• Step 3 is a DFS traversal of TG (the transposed graph), which similarly runs in O(N+M).

Thus, the overall complexity of Kosaraju’s algorithm is O(N+M).

Applications

This algorithm is used for vehicle routing applications, maps, and model-checking in formal verification.

Eulerian path and circuit

The Euler path is a route that allows us to traverse every edge exactly once, and vertices can be revisited multiple times. A Euler circuit is a specific type of Euler path where the starting vertex of the path is also connected to its ending vertex:

Figure 10.23: Connected graph for Eulerian path and circuit

The graph contains Eulerian paths, such as 4 3 0 1 2 0, but lacks an Eulerian cycle. It is worth noting that two vertices, 4 and 0, have odd degrees:

Figure 10.24: Eulerian path

The graph contains an Eulerian cycle, such as 2 1 0 3 4 0 2, and all vertices have even degrees:

Figure 10.25: Eulerian cycle

The graph is not Eulerian because all vertices have odd degrees.

In an Eulerian path, each time we visit a vertex v, we traverse exactly two unvisited edges with one endpoint at v. Hence, all intermediate vertices in a Eulerian path must have an even degree. For an Eulerian cycle, any vertex can act as an intermediate vertex, so all vertices must have an even degree.

Single-source shortest path algorithm

For a weighted directed graph, the shortest path problem identifies the path with the lowest total weight between two vertices. The single-source shortest path problem (SSSP) seeks the shortest path from a single vertex (the source) to every other vertex in the graph.

Relaxation property

The single-source shortest paths rely on a technique called relaxation, which iteratively reduces an upper bound on the actual shortest path weight of each vertex until the upper bound equals the shortest path weight. For each vertex v in V, we maintain an attribute d[v], which serves as an upper bound on the weight of the shortest path from source s to v. This attribute d[v] is referred to as the shortest path estimate.

The following algorithm shows the SSSP process:

INITIALIZE - SINGLE - SOURCE (G, s)

1. for each vertex v ∈ V [G]

2. do d [v] ← ∞

3. π [v] ← NIL

4. d [s] ← 0

After initialization, π [v] = NIL for all v ∈ V, d [v] = 0 for v = s, and d [v] = ∞ for v ∈ V - {s}.

The process of relaxing an edge (u, v) involves checking if we can enhance the shortest path to vertex v by passing through vertex u. If this is possible, we update d[v], which represents the shortest path estimate to v, and π[v], the predecessor field of v. This step may decrease the shortest path estimate d[v] and update π[v]. In Figure 10.26, after the relaxation step, the value d[v] decreases because v.d > u.d + w(u, v) prior to the relaxation step:

Figure 10.26: Relaxation for SSSP case 1

Figure 10.27 represents v.d < u.d + w(u, v) before relaxing the edge, so the relaxation step leaves v.d unchanged:

Figure 10.27: Relaxation for SSSP case 2

The following pseudo-code performs a relaxation step on edge (u, v):

RELAX (u, v, w)

1. If d [v] > d [u] + w (u, v)

2. then d [v] ← d [u] + w (u, v)

3. π [v] ← u

Dijkstra’s algorithm

Dijkstra’s algorithm is utilized to determine the shortest path between any two vertices within a graph. These two vertices may either be adjacent or at the graph's most distant points. The algorithm commences from the source vertex. Its inputs consist of the graph G{V, E}, where V represents the set of vertices and E the set of edges, along with the source vertex S. The output is the shortest path spanning tree.

Algorithm 10.3: Dijkstra’s algorithm

The algorithm is as follows:

1. Initialize two arrays: distance[] to store distances from the source vertex to all other vertices in the graph, and visited[] to keep track of visited vertices.

2. Set distance[S] = 0, where S is the source vertex, and distance[v] = ∞ for all other vertices v in the graph.

3. Add the source vertex S to the visited[] array.

4. Find the adjacent vertices of S that have the minimum distance.

5. Choose the adjacent vertex A with the minimum distance that is not yet visited. Add A to the visited[] array and update its distance from ∞ to its assigned distance d1, where d1 < ∞.

6. Repeat the process for the adjacent vertices of all visited vertices until the shortest path spanning tree is constructed.

Program 10.3: Implementation in C++

#include<stdio.h>

#include<limits.h>

#include<stdbool.h>

int min_dist(int[], bool[]):

void dijsktra(int[][6],int):

int min_dist(int dist[], bool visited[]){ // finding minimum dist

int minimum=INT_MAX,ind:

for(int k=0: k<6: k++) {

if(visited[k]==false && dist[k]<=minimum) {

minimum=dist[k]:

ind=k:

}

}

return ind:

}

void dijsktra(int graph[6][6],int src){

int dist[6]:

bool visited[6]:

for(int k = 0: k<6: k++) {

dist[k] = INT_MAX:

visited[k] = false:

}

dist[src] = 0: // Source vertex dist is set 0

for(int k = 0: k<6: k++) {

int m=min_dist(dist,visited):

visited[m]=true:

for(int k = 0: k<6: k++) {

// updating the dist of neighbouring vertex

if(!visited[k] && graph[m][k] && dist[m]!=INT_MAX && dist[m]+graph[m][k]<dist[k])

dist[k]=dist[m]+graph[m][k]:

}

}

printf("Vertex\t\tdist from source vertex\n"):

for(int k = 0: k<6: k++) {

char str=65+k:

printf(„%c\t\t\t%d\n“, str, dist[k]):

}

}

int main(){

int graph[6][6]= {

{0, 1, 2, 0, 0, 0},

{1, 0, 0, 5, 1, 0},

{2, 0, 0, 2, 3, 0},

{0, 5, 2, 0, 2, 2},

{0, 1, 3, 2, 0, 1},

{0, 0, 0, 2, 1, 0}

}:

dijsktra(graph,0):

return 0:

}

Example 10.3: Find the shortest path from the source node S in the following graph:

Figure 10.28: Connected graph for shortest path

Solution: To find the shortest path:

1. Initialize the distances of all vertices as infinity, except for the source node S:

Vertex

S

A

B

C

D

E

Distance

0

∞

∞

∞

∞

∞

Table 10.1: Distance table initialization

Now, source node S is a visited node, so add it in the visited array:

visited={S}

2. The vertex S has three adjacent vertices with different distances, and the vertex with the minimum distance among them all is A. Therefore, A is visited, and dist[A] is updated from ∞ to 6.

S → A = 6

S → D = 8

S → E = 7

The values are updated in the table:

Vertex

S

A

B

C

D

E

Distance

0

6

∞

∞

8

7

Table 10.2: Updated distance table

visited = {S, A}, shown as follows:

Figure 10.29: First edge visited {S, A}

3. Since there are two vertices already visited in the visited array, we need to check the adjacent vertices for both of them.

Vertex S still has two adjacent vertices to be visited: D and E. Vertex A has one adjacent vertex, B.

Calculate the distances from S to D, E, and B, and select the minimum distance:

S → D = 8 and S → E = 7.

S → B = S → A + A → B = 6 + 9 = 15

The values are updated in the table:

Vertex

S

A

B

C

D

E

Distance

0

6

15

∞

8

7

Table 10.3: Distance table from S to D, E, B

visited = {S, A, E}, shown as follows:

Figure 10.30: Visited nodes from S

4. Calculate the distances of the adjacent vertices, S, A, and E, from all the vertices in the visited array and select the vertex with the minimum distance:

S → D = 8

S → B = 15

S → C = S → E + E → C = 7 + 5 = 12

The values are updated in the table:

Vertex

S

A

B

C

D

E

Distance

0

6

15

12

8

7

Table 10.4: Updated distance table from S to C

visited = {S, A, E, D}, illustrated as follows:

Figure 10.31: Visited nodes A, D, E from S

5. Recalculate the distances of unvisited vertices. If a new minimum distance is found, update the distance array accordingly:

S → C = S → E + E → C = 7 + 5 = 12

S → C = S → D + D → C = 8 + 3 = 11

dist[C] = minimum (12, 11) = 11

S → B = S → A + A → B = 6 + 9 = 15

S → B = S → D + D → C + C → B = 8 + 3 + 12 = 23

dist[B] = minimum (15,23) = 15

The new distance array:

Vertex

S

A

B

C

D

E

Distance

0

6

15

11

8

7

Table 10.5: Final distance table

visited = { S, A, E, D, C}, as depicted in the following figure:

Figure 10.32: Visited nodes A, D, E, C from S

6. The remaining unvisited vertex in the graph is B, which has a minimum distance of 15, and it is added to the output spanning tree as shown in Figure 10.33:

visited = {S, A, E, D, C, B}

Figure 10.33: Final shortest path spanning tree

The shortest path spanning tree is generated as the output using Dijkstra's algorithm.

Complexity analysis

The complexity is calculated as:

• Time complexity: O(E Log V), where E is the number of edges and V is the number of vertices.

• Space complexity: O(V)

Applications

The applications of this algorithm are:

• To determine the quickest route

• In applications for social networking

• Within a phone network

• To locate the places on the map

Bellman-Ford algorithm

The Bellman-Ford algorithm is a dynamic programming-based single-source shortest path algorithm. It starts from a vertex and calculates the shortest distance to other vertices in the graph.

The Bellman-Ford algorithm is a single-source shortest path algorithm based on the bottom-up approach of dynamic programming. It starts from a single vertex and calculates the shortest path from that vertex to all other nodes in a weighted graph. It is based on the principle of relaxation. Negative edge weights in the graph can create a negative cycle, which makes it challenging to find the shortest path using Dijkstra's algorithm and other methods. The Bellman-Ford algorithm resolves this issue by effectively handling graphs with negative edges.

Role of negative weight

The presence of negative edges in a weighted graph can create a negative cycle, which leads to incorrect results in shortest path algorithms like Dijkstra's. Dijkstra's algorithm may incorrectly find a shorter path due to the negative values in each cycle.

Negative weights in the graph must be handled carefully to avoid incorrect results.

The core concept behind the Bellman-Ford algorithm is to start from a single source and initially estimate the distance to each node as infinity. The algorithm then iteratively relaxes these paths by finding shorter paths, which is known as the principle of relaxation. To ensure the optimized result, the algorithm repeats these steps for all vertices. If a new shorter path is found for any vertex during this process, it means the previous result was not optimized.

Algorithm 10.4: Bellman-Ford algorithm

Bellmanford(G, S)

for each vertex V in G

dist[V] ←infinite

prev[V] ← NULL

dist[S] ← 0

for each vertex V in G

for each edge (U, V) in G

tempDist ← dist[U] + edge_weight(U, V)

if tempDist < dist[V]

dist[V] ← tempDist

prev[V] ← U

for each edge (U,V) in G

if dist[U] + edge_weight(U, V) < dist[V}

Error: Negative Cycle Exists

return dist[], prev[]

Program 10.4: Implementation in C++

#include <iostream>

#include <vector>

#include <limits>

using namespace std:

struct Edge {

int source, destination, weight:

}:

void BellmanFord(vector<Edge>& edges, int V, int E, int source) {

vector<int> dist(V, numeric_limits<int>::max()):

vector<int> prev(V, -1):

dist[source] = 0:

// Relax all edges |V| - 1 times

for (int i = 1: i <= V - 1: ++i) {

for (int j = 0: j < E: ++j) {

int u = edges[j].source:

int v = edges[j].destination:

int weight = edges[j].weight:

if (dist[u] != numeric_limits<int>::max() && dist[u] + weight < dist[v]) {

dist[v] = dist[u] + weight:

prev[v] = u:

}

}

}

// Check for negative-weight cycles

for (int i = 0: i < E: ++i) {

int u = edges[i].source:

int v = edges[i].destination:

int weight = edges[i].weight:

if (dist[u] != numeric_limits<int>::max() && dist[u] + weight < dist[v]) {

cout << "Error: Negative Cycle Exists" << endl:

return:

}

}

// Print the shortest paths

cout << "Vertex Distance from Source:" << endl:

for (int i = 0: i < V: ++i)

cout << i << "\t\t" << dist[i] << endl:

}

int main() {

int V = 5: // Number of vertices in graph

int E = 8: // Number of edges in graph

vector<Edge> edges = {

{0, 1, -1}, {0, 2, 4}, {1, 2, 3}, {1, 3, 2},

{1, 4, 2}, {3, 2, 5}, {3, 1, 1}, {4, 3, -3}

}:

int source = 0: // Source vertex

BellmanFord(edges, V, E, source):

return 0:

}

Example 10.4: Find the shortest path from the source node 'S' in the graph provided:

Figure 10.34: Directed graph for shortest path

Solution: Initially, the distance of each vertex from the source vertex is set to infinity, and the distance from the source to itself is zero. Let us consider vertex A as the source vertex:

Figure 10.35: Graph initialization with infinity distance from S

Relax all edges for n-1 iterations, where n is the number of vertices.

If, [(d(u) + c(u , v) < d(v)) ]

d(v) = d(u) + c(u , v)

So, for this problem, there will be a total of 5 iterations (6-1).

Enumerate all the edges in the graph.

(A, B), (A, C), (A, D), (B, E), (D, C), (D, F) (C, E), (C, B), (E, F)

First iteration:

Consider the edge (A, B), where d(u) is 0, d(v) is ∞, and c(u, v) is 6.

Applying the principle of relaxation:

If 0 + 6 < ∞… True

So, d(B) = 6

Now, consider the edge (A, C):

Applying the principle of relaxation

d(C) = 4

Now, consider the edge (A, D):

Applying the principle of relaxation:

d(D) = 5

Now, consider the edge (B, E):

Applying the principle of relaxation:

If 6 + (-1) < ∞… True

So, d(E) = 5

Now, consider the edge (C, E):

Applying the principle of relaxation:

If 4 + 3 < 5 … False

So, it is not updated.

Now, consider the edge (D, C):

Applying the principle of relaxation:

If 5 + (-2) < 4 … True

So, d(C) = 3

Now, consider the edge (D, F):

Applying the principle of relaxation:

If 5 + (-1) < ∞… True

So, d(F) = 4

Now, consider the edge (E, F):

Applying the principle of relaxation:

If 5 + 3 < 4 … False

So, no update is made.

Now, consider the edge (C, B):

Applying the principle of relaxation:

If 3 + (-2) < 6 … True

So, d(B) = 1

The graph after the first iteration looks as follows:

Figure 10.36: Directed graph after the first iteration

Second iteration:

Let us take (A, B):

Applying the principle of relaxation:

If 0 + 6 < 1 … False

So, do not update.

Now, let us take (A, C):

Applying the principle of relaxation:

If 0 + 4 < 3 … False

So, do not update.

Now, let us take (A, D):

Applying the principle of relaxation:

If 0 + 5 < 5… False

So, do not update.

Now, let us take (B, E):

Applying the principle of relaxation:

If 1 + (-1) < 5 … True

So, d(E) = 0

Now, let us take (C, E):

Applying the principle of relaxation:

If 3 + 3 < 0… False

So, do not update.

Now, let us take (D, C):

Applying the principle of relaxation:

If 5 + (-2) < 3 … False

So, do not update.

Now, let us take (D, F):

Applying the principle of relaxation:

If 5 + (-1) < 4 … False

So, do not update.

Now, let us take (E, F):

Applying the principle of relaxation:

If 0 + 3 < 4 … True

So, d(F) = 3

Now, let us take (C, B):

Applying the principle of relaxation:

If 3 + (-2) < 6 … False

So, do not update.

The graph after the second iteration:

Figure 10.37: Directed graph after the second iteration

Third iteration:

Let us take (A, B):

Applying the principle of relaxation:

If 0 + 6 < 1… False

So, do not update.

Now, let us take (A, C):

Applying the principle of relaxation:

If 0 + 4 < 3 … False

So, do not update.

Now, let us take (A, D):

Applying the principle of relaxation:

If 0 + 5 < 5 … False

So, do not update.

Now, let us take (B, E):

Applying the principle of relaxation:

If 1 + (-1) < 0… False

So, do not update.

Now, let us take (C, E):

Applying the principle of relaxation:

If 3 + 3 < 0 … False

So, do not update.

Now, let us take (D, C):

Applying the principle of relaxation:

If 5 + (-2) < 3… False

So, do not update.

Now, let us take (D, F):

Applying the principle of relaxation:

If 5 + (-1) < 3 … False

So, do not update.

Now, let us take (E, F):

Applying the principle of relaxation:

If 0 + 3 < 3… False

So, do not update.

Now, let us take (C, B):

Applying the principle of relaxation:

If 3 + (-2) < 6… False

So, do not update.

The graph after the third iteration is:

Figure 10.38: Directed graph after the third iteration

According to the relaxation condition, we need at most five iterations. As far as we can tell, the third iteration itself contains no updates or modifications. Therefore, we will stop here and document the shortest path for each vertex in A.

So, the cost for each vertex from A is:

A- A: 0

A- B: 1

A- C: 3

A- D: 5

A- E: 0

A- F: 3

Complexity analysis

The complexity analysis is as follows:

• Time complexity:

o Best case: O(E)

o Average case: O(VE)

o Worst case: O(VE)

• Space complexity: O(V)

Here, V is the number of vertices and E is the number of edges.

Applications

The applications of Bellman-Ford algorithm are:

• Network routing: This algorithm is used to determine the shortest path in a network with negative weights, such as the internet.

• Robot navigation: It finds the shortest path for a robot to navigate in an environment with obstacles.

• Resource allocation: The algorithm optimally allocates resources, such as assigning tasks to workers in a factory.

• Image processing: This algorithm finds the shortest path between two points in an image, aiding in image segmentation and object recognition.

• Game theory: It determines the optimal strategy for players in a game, like chess.

• Genetics: The algorithm identifies the shortest path in a genetic network, useful for analyzing genetic interactions and identifying potential drug targets.

All-pair shortest paths

The all-pairs shortest path algorithm, also known as the Floyd-Warshall algorithm, is used to solve the problem of finding the shortest paths between all pairs of nodes in a given weighted graph. This algorithm produces a matrix representing the minimum distances from each node to every other node in the graph. It is a dynamic programming technique used to find the shortest paths in a weighted graph, even when the graph contains negative weight cycles. The algorithm determines the shortest route between every pair of vertices by utilizing a matrix of intermediate vertices to keep track of the best-known path at each step.

It is similar to Dijkstra’s and Bellman-Ford’s algorithms. However, while Dijkstra’s and Bellman-Ford’s algorithms are single-source shortest path algorithms, Floyd-Warshall is an all-pairs shortest path algorithm. This means it can compute the shortest path between every pair of vertices in the graph.

Algorithm 10.5: Floyd-Warshall algorithm

n = number of vertices

D = matrix of dimension n*n

for k = 1 to n

for i = 1 to n

for j = 1 to n

Dk[i, j] = min (Dk-1[i, j], Dk-1[i, k] + Dk-1[k, j])

return D

Program 10.5: Implementation in C++

#include <iostream>

#include <vector>

#include <climits>

using namespace std:

// Number of vertices in the graph

#define V 4

// A function to print the solution matrix

void printSolution(const vector<vector<int>>& dist) {

cout << "The following matrix shows the shortest distances between every pair of vertices:\n":

for (int i = 0: i < V: i++) {

for (int j = 0: j < V: j++) {

if (dist[i][j] == INT_MAX)

cout << "INF" << " ":

else

cout << dist[i][j] << " ":

}

cout << endl:

}

}

// Function to implement Floyd-Warshall algorithm

void floydWarshall(vector<vector<int>>& graph) {

// Initialize the solution matrix same as input graph matrix

vector<vector<int>> dist = graph:

// Add all vertices one by one to the set of intermediate vertices

for (int k = 0: k < V: k++) {

// Pick all vertices as source one by one

for (int i = 0: i < V: i++) {

// Pick all vertices as destination for the above picked source

for (int j = 0: j < V: j++) {

// If vertex k is on the shortest path from i to j, then update the value of dist[i][j]

if (dist[i][k] != INT_MAX && dist[k][j] != INT_MAX && dist[i][k] + dist[k][j] < dist[i][j])

dist[i][j] = dist[i][k] + dist[k][j]:

}

}

}

// Print the shortest distance matrix

printSolution(dist):

}

int main() {

vector<vector<int>> graph = { {0, 5, INT_MAX, 10},

{INT_MAX, 0, 3, INT_MAX},

{INT_MAX, INT_MAX, 0, 1},

{INT_MAX, INT_MAX, INT_MAX, 0} }:

// Function call

floydWarshall(graph):

return 0:

}

Example 10.5: The Floyd-Warshall algorithm is utilized to determine the shortest paths between all pairs of vertices in a weighted graph. The following is a step-by-step explanation of its functioning, accompanied by an example:

Figure 10.39: Graph for Floyd-Warshall algorithm

Solution: Make the distance matrix or graph matrix. Insert the distance value, and if the distance is unknown, fill that with ∞, shown as follows:

D0= A square matrix with labels Aâ€“D on columns and rows. The cells contain numbers: diagonal is 0; other values include 6, 5, âˆž, 3, 2, 4, 7, with âˆž for missing connections between nodes.

Figure 10.40: Distance matrix

Now, calculating the distance of every pair via A:

DA = A table with rows and columns labeled A to D. Diagonal values are 0, and example values include: A-B is 6, A-C is 5, A-D is infinity, B-A is 3, C-A and D-A are infinity. Some cells are empty.

Figure 10.41: Distance matrix for node A

Applying Floyd-Warshall's algorithm:

D0[B, C] = D0[B, A] + D0[A, C]

∞ > 3 + 5

∞ > 8 \, (updating the distance)

D0[B, C] = 8

D0[B, D] = D0[B, A] + D0[A, D]

2 > 3 + ∞

2 > ∞, (no changes)

D0[B, D] = 2

D0[C, B] = 8

D0[C, D] = D0[C, A] + D0[A, D]

∞ > ∞ + ∞

∞ > ∞, (no changes)

D0[C, D] = ∞

D0[D, B] = D0[D, A] + D0[A, B]

∞ > ∞ + ∞

∞ > ∞, (no changes)

D0[D, B] = ∞

D0[D, C] = 7

Similarly, for all other vertices, the resulting matrix will look like:

DA = A table with rows and columns labeled A to D shows distances: Aâ†’B:6, Aâ†’C:5, Aâ†’D:âˆž; Bâ†’A:3, Bâ†’C:8, Bâ†’D:2; Câ†’A:âˆž, Câ†’B:4, Câ†’D:0; Dâ†’A:âˆž, Dâ†’B:âˆž, Dâ†’C:7, Dâ†’D:0.

Figure 10.42: Distance matrix for all vertices from A

Now, calculating the distance of every pair via B:

DB =

Figure 10.43: Distance matrix for vertex B

After applying Floyd-Warshall formula, we get:

D0 [A, C] = D0 [A, B] + D0 [B, C]

5 > 6 + ∞…… False, No changes required.

D0 [A, C] = 5

D0 [A, D] = D0 [A, B] + D0 [B, D]

∞ > 6 + 2…… Update the value.

D0 [A, D] = 8

D0 [C, A] = D0 [C, B] + D0 [B, C]

∞ > 4 + 3…… Update the value.

D0 [C, A] = 7

D0 [C, D] = D0 [C, B] + D0 [B, D]

∞ > 4 + 2…… Update the value.

D0 [C, D] = 6

D0 [D, A] = D0 [D, B] + D0 [B, A]

∞ > ∞ + 3…… No changes are required.

D0 [C, A] = ∞

D0 [D, C] = D0 [D, B] + D0 [B, C]

∞ > ∞ + 2…… No changes required.

D0 [C, D] = ∞

DB = A 4x4 table with rows and columns labeled A to D. Cell values: row A: 0, 6, 5, 8; row B: 3, 0, 8, 2; row C: 7, 4, 0, 6; row D: âˆž, âˆž, 7, 0.

Figure 10.44: Distance matrix for all vertices from B

Now, calculating the distance of each path via C:

DC = A 4x4 grid with row and column headers labeled Aâ€“D. The grid contains these numbers: A,B: 5; B,B: 8; C,A: 7; C,B: 4; C,C: 0; C,D: 6; D,B: 7. All other cells are blank.

Figure 10.45: Distance matrix for vertex C

After applying Floyd-Warshall formula, we get:

D0 [A, B] = D0 [A, C] + D0 [C, B]

6 > 5 + 4…… False, No changes required.

D0 [A, B] = 6

D0 [A, D] = D0 [A, C] + D0 [C, D]

8 > 5 + 6…… False, No changes required.

D0 [A, D] = 8

D0 [B, A] = D0 [B, C] + D0 [C, A]

3 > 0 + 7…… False, No changes required.

D0 [B, A] = 3

D0 [B, D] = D0 [B, C] + D0 [C, D]

2 > ∞ + 6…… False, No changes required.

D0 [B, D] = 2

D0 [D, A] = D0 [D, C] + D0 [C, A]

∞ > 7 + 7…… Update the value.

D0 [D, A] = 14

D0 [D, B] = D0 [D, C] + D0 [C, B]

∞ > 7 + 4…… Update the value.

D0 [D, B] = 11

Figure 10.46 shows the distance matrix from vertex C:

DC = A 4x4 table with rows and columns labeled A to D, showing numeric values at intersections. Notable numbers include 14 in row D, column A; 11 in row D, column B; and 8 in row A, column D.

Figure 10.46: Distance matrix for all vertices from C

Similarly, if we calculate the shortest distance via D, no changes will be required.

Hence, the shortest path matrix using Floyd-Warshall algorithm is as follows:

Figure 10.47: Distance matrix for all vertices from D

Complexity analysis

Time and space complexity for the shortest path using Floyd-Warshall algorithm are:

• Time complexity: There are three for loops in the pseudo-code of the algorithm, so the time complexity will be O (n3).

• Space complexity: The space complexity of Floyd-Warshall algorithm is O (n2).

Applications

The applications of Floyd-Warshall algorithm are:

• In routing data packets

• Calculating the inversion of the real matrix

• Calculating the transitive closure of directed graphs

• To check whether an undirected graph is bipartite

• To find the shortest path in a directed graph

Max-flow min-cut

In graph theory, a flow network is defined as a directed graph involving a source (S) and a sink (T) and several other nodes connected with edges. Each edge has an individual capacity, which is the maximum limit of flow that the edge could allow.

Flow in the network should follow these conditions:

• For any non-source and non-sink node, the input flow is equal to the output flow.

• For any edge (Ei) in the network, 0≤flow(Ei)≤Capacity(Ei).

• Total flow out of the source node is equal to total flow into the sink node.

• Net flow in the edges follows skew symmetry, i.e., F(u,v)=-F(v,u) where F(v,u) is flow from node u to node v. This leads to a conclusion where you have to sum up all the flows between two nodes (either direction) to find the net flow between the nodes initially.

Ford-Fulkerson algorithm

It was developed by L. R. Ford Jr. and D. R. Fulkerson in 1956. Here is a pseudocode for this algorithm. The required inputs are the network graph G, source node S, and sink node T.

An augmenting path is a simple path from the source to the sink that does not include any cycles and passes only through edges with positive weights. A residual network graph indicates how much more flow is allowed in each edge in the network graph. If no augmenting paths are possible from S to T, then the flow is maximum. The maximum flow result will be the total flow out of the source node, which is also equal to the total flow into the sink node.

Max-flow min-cut theorem

A s -t -cut is a partition of the vertices of a flow network into two sets, such that a set includes the source s and the other one includes the sink t. The capacity of a s -t -cut is defined as the sum of the capacities of the edges from the source side to the sink side. According to Max-flow min-cut theorem, the capacity of the maximum flow has to be equal to the capacity of the minimum cut.

A minimum cut can be found after performing a maximum flow computation using the Ford-Fulkerson method. One possible minimum cut is the following: the set of all the vertices that can be reached from s in the residual graph (using edges with positive residual capacity), and the set of all the other vertices.

Figure 10.48 shows the sample residual graph, which is used for max-cut and min-cut theorem:

Figure 10.48: Sample residual graph

The capacity of the cut {s, A, D} and {B, C, t} is 5 + 3 + 2 = 10, which matches the maximum flow we determined. Other cuts will have a higher capacity: for instance, the capacity between {s, A} and {B, C, D, t} is 4 + 3 + 5 = 12.

Algorithm 10.6: Ford-Fulkerson algorithm

FordFulkerson(Graph G,Node S,Node T):

Initialise flow in all edges to 0

while (there exists an augmenting path(P) between S and T in residual network graph):

Augment flow between S to T along the path P

Update residual network graph

Return

Program 10.6: Implementation in C++

#include <iostream>

#include <limits.h>

#include <queue>

#include <vector>

#include <cstring>

using namespace std:

#define V 6 // Number of vertices in the graph

// A utility function to perform BFS and find if there is a path from source to sink

bool bfs(int rGraph[V][V], int s, int t, int parent[]) {

// Create a visited array and mark all vertices as not visited

bool visited[V]:

memset(visited, 0, sizeof(visited)):

// Create a queue for BFS

queue<int> q:

q.push(s):

visited[s] = true:

parent[s] = -1:

// Standard BFS loop

while (!q.empty()) {

int u = q.front():

q.pop():

for (int v = 0: v < V: v++) {

if (visited[v] == false && rGraph[u][v] > 0) {

// If we find a connection to the sink, we return true

if (v == t) {

parent[v] = u:

return true:

}

q.push(v):

parent[v] = u:

visited[v] = true:

}

}

}

// We didn't reach sink in BFS starting from source, so return false

return false:

}

// Returns the maximum flow from s to t in the given graph

int fordFulkerson(int graph[V][V], int s, int t) {

int u, v:

// Create a residual graph and fill the residual graph with given capacities in the original graph as residual capacities in residual graph

int rGraph[V][V]:

for (u = 0: u < V: u++)

for (v = 0: v < V: v++)

rGraph[u][v] = graph[u][v]:

int parent[V]: // This array is filled by BFS and to store path

int max_flow = 0: // There is no flow initially

// Augment the flow while there is a path from source to sink

while (bfs(rGraph, s, t, parent)) {

// Find the maximum flow through the path found.

int path_flow = INT_MAX:

for (v = t: v != s: v = parent[v]) {

u = parent[v]:

path_flow = min(path_flow, rGraph[u][v]):

}

// update residual capacities of the edges and reverse edges along the path

for (v = t: v != s: v = parent[v]) {

u = parent[v]:

rGraph[u][v] -= path_flow:

rGraph[v][u] += path_flow:

}

// Add path flow to overall flow

max_flow += path_flow:

}

// Return the overall flow

return max_flow:

}

int main() {

// Create a graph given in the above diagram

int graph[V][V] = { {0, 16, 13, 0, 0, 0},

{0, 0, 10, 12, 0, 0},

{0, 4, 0, 0, 14, 0},

{0, 0, 9, 0, 0, 20},

{0, 0, 0, 7, 0, 4},

{0, 0, 0, 0, 0, 0}

}:

cout << "The maximum possible flow is " << fordFulkerson(graph, 0, 5):

return 0:

}

Example 10.6: A demonstration of the working of the Ford-Fulkerson algorithm is shown with the help of figures.

The following figures show the input network graph G and the residual graph GR.:

Figure 10.49: Graph G and residual graph GR

Solution: Using these two graphs, we calculate path 1 and also find the flow of the network from source S:

Figure 10.50: Path 1 with flow

Similarly, we find the path 2 with the updated flow:

Figure 10.51: Path 2 with flow

In the next step, we find the path 3 and also update the flow value:

Figure 10.52: Path 3 with flow

As per the previous process, we find the next path 4 and update the value of flow:

Figure 10.53: Final path 4 with max flow

Complexity analysis

The time and space complexity of Ford Fulkerson’s algorithm are as follows:

• Time complexity: O(|V| * E2), where E is the number of edges and V is the number of vertices.

• Space complexity: O(V)

Applications

The applications are:

• Water distribution pipeline.

• Bipartite matching problem.

• Circulation with demands.

Conclusion

This chapter provided an essential introduction to graph theory, highlighting its role in representing and analyzing relationships within various systems. Topics such as graph terminology, topological sorting, strongly connected components, shortest path algorithms, and flow networks were discussed to illustrate key principles and techniques. These graph-based methods are widely used in fields like communication, transportation, biology, and computing. Understanding these concepts allows learners to model and solve complex problems involving interconnected data. A solid grasp of graph theory also serves as a foundation for learning more advanced algorithms and systems.

The upcoming chapter will explore evaluating algorithm performance and understanding the theoretical boundaries of what can be computed efficiently.

Exercise

1. Write a function to compute the lengths of the shortest paths between all pairs of nodes for the given adjacency matrix:

Figure 10.54: Matrix for calculating shortest path

2. Perform a depth first search and breadth first search on the following graph starting at A.

Figure 10.55: Calculating depth first and breadth first search

3. Illustrate the execution of the Ford Fulkerson algorithm in the flow network:

Figure 10.56: Flow network for Ford Fulkerson algorithm

4. Apply Floyd-Warshall algorithm for constructing the shortest path. Show the matrix D(k) that results each iteration.

Figure 10.57: Applying Floyd-Warshall algorithm to construct the shortest path

5. For the graph shown as follows, obtain the following:

a. Kruskal’s method

b. Prim’s method

Figure 10.58: Applying Kruskal’s method and Prim’s method

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

CHAPTER 11Computational Complexity

Introduction

This chapter offers a comprehensive overview of computational complexity, highlighting its most essential aspects. Each section can be elaborated with in-depth discussions, examples, and proofs to thoroughly illustrate the field's depth and scope. Computational complexity studies the resources, such as time and space, needed to solve computational problems. It assesses how efficiently algorithms use these resources as input size increases. This field aims to understand the inherent difficulty of various computational problems and classify them based on the resources required to solve them. By doing so, computational complexity provides insights into which problems can be solved efficiently and which are computationally hard or even infeasible within practical limits.

Structure

This chapter will cover the following topics:

• Key aspects

• P and NP-classes

• Proof of NP-hardness and NP-completeness

• Polynomial space

• Exponential time

• Reductions and completeness

• Approximation algorithms

• Probabilistic algorithms

• Bounded-error probabilistic polynomial time

• Randomized polynomial time

• Complement of randomized polynomial time

Objectives

The objective of this chapter is to learn computational complexity, understand the inherent difficulty of computational problems, and categorize them based on the resources required to solve them. Specifically, it aims to measure and analyze the time, space, and other resources (like parallelism and randomness) that algorithms need. Classify problems into complexity classes based on their resource requirements, such as P, NP, and PSPACE. Readers will be able to use reductions and completeness concepts to determine how the complexity of one problem relates to another, develop efficient algorithms, and understand the limits of algorithmic efficiency, and explore the theoretical limits of what can be computed efficiently, including proving lower bounds on resource requirements. Finally, readers will also be able to apply insights from complexity theory to practical computing tasks, cryptography, optimization, and other areas, and address significant unresolved questions in the field, such as the P vs. NP problem, to advance computational theory.

Key aspects

The analysis in computational complexity involves several key aspects:

• Resource measurement: This process involves quantifying the time (number of computational steps) and space (amount of memory) an algorithm needs to solve a problem. Other resources like parallelism (use of multiple processors) and randomness (use of random numbers) are also considered.

• Complexity classes: Problems are grouped into classes based on their resource requirements. For example, problems in class P can be solved in polynomial time by a deterministic Turing machine, while problems in NP can be verified in polynomial time by a deterministic Turing machine. Other classes include PSPACE, which involves problems solvable in polynomial space, and EXP, which involves problems solvable in exponential time.

• Reductions and completeness: To understand the relative difficulty of problems, computational complexity uses reductions, which transform one problem into another. If a problem can be reduced to another problem known to be hard, it indicates that the original problem is at least as hard. Completeness concepts, like NP-completeness, identify the most challenging problems within a class.

• Algorithmic efficiency: The field seeks to develop the most efficient algorithms possible and to understand the limits of what can be achieved. This involves proving lower bounds on the resources needed for specific problems, demonstrating that certain problems cannot be solved more efficiently than a given threshold.

• Practical applications: Insights from computational complexity are applied to real-world problems in areas such as cryptography, where the hardness of certain problems ensures security, and optimization, where efficient algorithms are crucial for practical applications.

• Theoretical boundaries: Computational complexity explores the boundaries of what can be computed efficiently. This includes investigating theoretical limits and understanding why some problems are inherently difficult to solve.

• Open problems and research: The field addresses significant unresolved questions, such as the famous P vs. NP problem, which asks whether every problem whose solution can be quickly verified can also be quickly solved. Solving these open problems would have profound implications for both theory and practice.

By categorizing problems and analyzing their resource requirements, computational complexity helps in understanding the feasibility of finding efficient solutions and guides the development of algorithms that can effectively tackle practical computational challenges.

Basic concepts of computational complexity

The basic concepts of computational complexity focus on understanding the resources required to solve computational problems. The key concepts are as follows:

• Problem instances and input size: Every computational problem has different instances, and the size of an instance is defined by the length of its input. For example, in sorting a list, the input size is the number of elements in it.

• Algorithm efficiency: Algorithms are evaluated based on how efficiently they use computational resources (time and space) to solve problem instances of varying sizes. Efficient algorithms typically have polynomial time complexity, meaning their running time is bounded by a polynomial function of the input size.

• Complexity classes: Problems are categorized into complexity classes based on the amount of resources (time or space) required to solve them. Common complexity classes include P (problems solvable in polynomial time), NP (problems for which a solution can be verified in polynomial time), and NP-hard or NP-complete (problems at least as hard as the hardest problems in NP).

• Worst-case vs. average-case complexity: Algorithms may perform differently depending on the input data. Worst-case complexity measures the maximum resources an algorithm requires for any input of a given size, while average-case complexity considers the average resource usage over all possible inputs.

• Reduction: Reductions are transformations that map one problem (source problem) into another (target problem) such that a solution to the target problem can be used to solve the source problem. Reductions are fundamental for proving the complexity of problems and classifying them into complexity classes like NP-complete.

• Feasibility and intractability: Some problems are inherently difficult to solve efficiently. NP-complete problems, for example, are problems in NP for which no polynomial-time algorithm is known, and if one exists for any of them, it would solve all NP problems in polynomial time.

P and NP-classes

Let us explore the concept of P and NP-classes:

• Polynomial time (P): The class P includes all decision problems (problems with a yes/no answer) that can be solved by a deterministic Turing machine in polynomial time. Simply put, a problem belongs to P if an algorithm can solve it in polynomial time based on the size of its input.

The characteristics are as follows:

o Efficiently solvable: Problems in P can be solved within a reasonable timeframe for practical input sizes.

o Deterministic algorithms: The solutions are computed using deterministic algorithms, where each step follows a specific, predictable sequence.

Sorting algorithms are like merge sort, searching algorithms like binary search, and many basic arithmetic operations are in P because algorithms exist that solve these problems in polynomial time.

For example:

o Decision problem: Given a list of numbers, is there a number in the list greater than 100?

o Algorithm: Traverse the list once, checking each number against 100. If such a number is found, return yes; otherwise, return no. This algorithm runs in linear time, O(n), which is polynomial time for input size n.

• Nondeterministic polynomial time (NP): The class NP consists of all decision problems for which a solution can be verified in polynomial time by a deterministic Turing machine. In other words, if someone provides a solution to an NP problem, we can check its correctness in polynomial time.

The characteristics are as follows:

o Verification: While finding a solution might be challenging, verifying a proposed solution is straightforward and efficient.

o Nondeterministic algorithms: Conceptually, the verification process is performed by a nondeterministic algorithm that can guess the solution and verify it in polynomial time.

The traveling salesperson problem (TSP), where the goal is to find the shortest route visiting all cities, is in NP because verifying that a given route is indeed the shortest can be done in polynomial time, though finding the shortest route is not known to be achievable in polynomial time.

For example:

o Decision problem: Is there a route for the TSP that visits all cities and has a total length less than or equal to a given threshold?

o Algorithm: A nondeterministic algorithm might guess a route and then verify in polynomial time that it visits all cities and has a total length within the threshold.

o P vs. NP Problem: The central question in theoretical computer science is whether P equals NP. This unsolved millennium prize problem asks if every problem for which a solution can be verified in polynomial time can also be solved in polynomial time.

o Implications:

 If P = NP, it would mean that problems traditionally considered difficult, such as many combinatorial optimization problems, can be solved efficiently.

 If P ≠ NP, then there exist problems in NP that are inherently more difficult to solve than those in P.

• NP-complete and NP-hard

o NP-complete: These are problems in NP that are as hard as any problem in NP. If you can solve any NP-complete problem in polynomial time, you can solve all NP problems in polynomial time. NP-complete problems are both in NP and NP-hard.

o NP-hard: These are problems that are at least as difficult as the hardest problems in NP, but they may not be in NP themselves. This means they do not necessarily have polynomial-time verification.

P represents problems that are efficiently solvable, while NP includes problems where solutions can be verified efficiently. The relationship between P and NP is a fundamental open question in computer science with significant implications for cryptography, optimization, and artificial intelligence.

Proof of NP-hardness and NP-completeness

To understand NP-hardness and NP-completeness, we need to explore the concepts of reductions and how they are used in computational complexity theory to prove these properties for problems. Let us break down each concept and how proofs typically work.

NP-hardness

A problem is NP-hard if every problem in NP can be reduced to it in polynomial time. This means that if you can solve an NP-hard problem in polynomial time, you can solve all NP problems in polynomial time, making NP-hard problems among the most difficult in NP.

The proof technique is as follows:

• Reduction: To prove that a problem X is NP-hard, we reduce a known NP-hard problem Y to X. This reduction demonstrates that any algorithm that can solve X in polynomial time can also solve Y in polynomial time.

Reduction steps:

1. Assume we have a problem Y known to be NP-hard.

2. Construct a polynomial-time reduction from Y to X, which transforms instances of Y into instances of X.

3. Show that the reduction preserves the answer: if there is a solution to the transformed instance of X, there must also be a solution to the original instance of Y, and vice versa.

The conclusion is that by showing the reduction from a known NP-hard problem to X, we establish that X is NP-hard.

Consider the following example of NP-hardness proof:

• Problem Y: Vertex cover (Given a graph G and a number k, is there a set of k vertices that touches every edge in G?)

• Problem X: Dominating set (Given a graph G and a number k, is there a set of k vertices such that every vertex in G is either in the set or adjacent to a vertex in the set?)

The proof sketch is as follows:

• Reduction: To prove that dominating set (X) is NP-hard, reduce vertex cover (Y) to dominating set (X).

• Transformation: Given an instance of vertex cover (G, k), construct an instance of dominating set (G, k) where the transformation is straightforward (details depend on the exact reduction technique used).

• Preservation: Show that a solution to the transformed instance of dominating set corresponds to a solution of vertex cover, and vice versa.

• Conclusion: Since vertex cover is NP-hard and we have shown a polynomial-time reduction from vertex cover to dominating set, dominating set is also NP-hard.

NP-completeness

A problem is NP-complete if it is both NP-hard and in NP. NP-complete problems represent the hardest problems in NP for which a solution, if found, can be verified in polynomial time. Solving any NP-complete problem in polynomial time would imply P = NP.

To prove that a problem X is NP-complete, we follow these steps:

1. NP-hardness: First, prove that X is NP-hard using the reduction technique described earlier.

2. Membership in NP: Show that X belongs to NP by demonstrating that a proposed solution can be verified in polynomial time.

3. Conclusion: By proving both NP-hardness and membership in NP, we establish that X is NP-complete.

Consider the following example of NP-completeness proof:

• Problem X: Subset sum (Given a set of integers and a target sum, is there a subset of integers that sums to the target?)

The proof sketch is as follows:

• NP-hardness: Reduce a known NP-hard problem, such as 3SAT (Boolean satisfiability), to subset sum. This reduction shows that if we can solve subset sum in polynomial time, we c also solve 3SAT in polynomial time.

• Membership in NP: Show that a proposed subset that sums to the target can be verified in polynomial time (just add up the subset elements and check if they sum to the target).

• Conclusion: Since subset sum is both NP-hard and in NP, it is NP-complete.

Thus, it can be concluded that:

• NP-hardness: Proven by reducing a known NP-hard problem to the problem in question, demonstrating that a polynomial-time solution to the latter would imply a polynomial-time solution to all NP problems.

• NP-completeness: Proven by demonstrating both NP-hardness (via reduction) and membership in NP (efficient verification of solutions).

These proofs are foundational in computational complexity theory, helping to classify problems based on their inherent difficulty and the computational resources required for their solutions.

Polynomial space

Polynomial space (PSPACE) is a complexity class that focuses on the amount of memory (space) required by a Turing machine to solve problems. PSPACE consists of all decision problems that can be solved by either a deterministic or nondeterministic Turing machine using a polynomial amount of memory space relative to the size of the input.

The characteristics are as follows:

• Memory usage: Unlike the P and NP classes, which focus on time complexity, PSPACE is concerned with space complexity. A problem is in PSPACE if there exists an algorithm that solves it using a polynomial amount of space.

• Deterministic vs. nondeterministic: PSPACE includes problems solvable by both deterministic and nondeterministic Turing machines within polynomial space bounds.

Some examples of problems in PSPACE include:

• Solving games such as chess or Go on a game board of fixed size.

• Determining whether a given mathematical statement (e.g., a logical formula) is true or false within a specified range of integers.

Relationship to other complexity classes:

• PSPACE vs. P and NP: PSPACE includes P and NP as special cases:

• PSPACE ⊇ P: Problems that can be solved in polynomial time also require polynomial space, as time complexity inherently includes space complexity.

• PSPACE ⊇ NP: Problems with solutions that can be verified in polynomial time also have algorithms that are bounded by polynomial space.

• PSPACE vs. NP-hard and NP-complete: While PSPACE includes both P and NP, PSPACE-hard problems are not necessarily NP-hard, and vice versa. PSPACE-hardness pertains to the space complexity of problems rather than their time complexity.

Complexity theories and problems

PSPACE-complete problems are the most challenging problems within PSPACE; solving any PSPACE-complete problem in polynomial space would enable the solution of all problems in PSPACE. They are analogous to NP-complete problems in NP.

Similar to NP-complete problems, PSPACE-complete problems are the most difficult problems in the PSPACE complexity class. They are both in PSPACE and PSPACE-hard.

For examples, a well-known PSPACE-complete problem is the quantified Boolean formula (QBF), which involves determining whether a given formula with quantifiers (∀ and ∃) is true for all possible assignments.

SPACE is an essential complexity class that evaluates problems based on the memory resources needed to solve them. It broadens the focus from time complexity (as seen in the P and NP classes) to include problems that might require substantial memory but can still be solved within polynomial space limits. Understanding PSPACE aids in analyzing the memory requirements of algorithms and determining the theoretical limits of computation in terms of space usage.

Exponential time

Exponential time (EXP) is a complexity class that includes decision problems solvable by deterministic Turing machines within exponential time bounds. Here’s a detailed exploration of EXP:

EXP encompasses all decision problems that a deterministic Turing machine can solve in 2poly(n) time, where poly(n) denotes a polynomial function of the input size n.

In contrast to polynomial time complexity classes such as P, NP, and PSPACE, which limit computation time to a polynomial function of the input size, EXP permits computations that increase exponentially with the input size.

The relationship to other complexity classes are as follows:

• P vs. EXP: P is contained within EXP because any problem solvable in polynomial time is also solvable in exponential time.

• NP vs. EXP: EXP is more powerful than NP because there exist problems that can be solved in exponential time but not in polynomial time. However, the relationship between NP and EXP is not fully understood; whether NP problems can be solved in exponential time is a major open question (related to the P vs. NP problem).

Complexity theories and problems

EXP-complete problems are the hardest problems in EXP; solving any EXP-complete problem in exponential time would solve all problems in EXP. Examples include certain problems in formal language theory and specific combinatorial optimization problems.

EXP is an important complexity class that captures problems solvable in exponential time. It represents a significant increase in computational difficulty compared to polynomial time classes (P, NP, PSPACE) and is used to classify problems based on their resource requirements. Understanding EXP helps in analyzing the inherent complexity of computational problems and exploring the theoretical limits of what can be feasibly computed within exponential time constraints.

Reductions and completeness

Reductions and completeness are foundational concepts in theoretical computer science, particularly in the study of computational complexity. Let us explore both concepts in detail.

Reductions

Reduction is a technique used to transform one problem into another. If problem A can be reduced to problem B, it means that any instance of problem A can be transformed into an instance of problem B in polynomial time. This transformation allows us to solve problem A using a solution to problem B. There are different types of reductions, such as:

• Many-one reduction (Karp reduction): A problem A is many-one reducible to problem B (denoted as A ≤m B) if there is a polynomial-time computable function, f, such that for every instance x of A, x is a yes instance of A if and only if f(x) is a yes instance of B.

• Turing reduction: A problem A is Turing reducible to problem B if a solution to B can be used as an oracle to decide A in polynomial time. This means that we can solve A using an algorithm that is allowed to call a subroutine for solving B.

Completeness

Completeness is a concept that identifies the hardest problems within a complexity class. A problem is complete for a complexity class if it is among the most difficult problems in that class, and every problem in the class can be reduced to it. Here are the main types:

• NP-completeness: A problem is NP-complete if it is in NP and every problem in NP can be reduced to it in polynomial time. These problems are significant because if any NP-complete problem can be solved in polynomial time, then every problem in NP can also be solved in polynomial time (i.e., P = NP). Famous NP-complete problems include the TSP, the knapsack problem, and SAT (Boolean satisfiability problem).

• P-completeness: A problem is P-complete if it is in P and every problem in P can be reduced to it using a reduction that can be computed in logarithmic space. P-complete problems are the hardest problems in the class P under logarithmic space reductions.

• PSPACE-completeness: A problem is PSPACE-complete if it is in PSPACE (the class of problems solvable in polynomial space) and every problem in PSPACE can be reduced to it in polynomial time. An example of a PSPACE-complete problem is the QBF problem.

Consider the following examples of reductions and completeness:

• SAT and 3-SAT: The Boolean satisfiability problem (SAT) was the first problem proven to be NP-complete by Stephen Cook. The reduction from SAT to 3-SAT (a version of SAT where each clause has exactly three literals) is a classic example. SAT can be reduced to 3-SAT in polynomial time, showing that 3-SAT is also NP-complete.

• Vertex cover and independent set: The vertex cover problem can be reduced to the independent set problem. Given a graph G and an integer k, finding a vertex cover of size k can be transformed into finding an independent set of size |V| - k in the same graph.

Understanding reductions and completeness helps us classify problems based on their computational difficulty and relationships, guiding the development of efficient algorithms and the understanding of computational intractability.

Approximation algorithms

Approximation algorithms are algorithms designed to find near-optimal solutions to optimization problems where finding the exact optimal solution is computationally infeasible (often due to the problem being NP-hard). These algorithms are particularly valuable when exact solutions are too costly in terms of time or resources. Here is a detailed look into approximation algorithms:

The key concepts are as follows:

• Optimization problems: Problems where the goal is to find the best solution according to some criteria. Examples include the TSP (minimize the travel cost) and the knapsack problem (maximize the total value of items without exceeding capacity).

• Approximation ratio: The performance of an approximation algorithm is measured by its approximation ratio. For a minimization problem, if is the cost of the solution produced by the algorithm and C* is the cost of the optimal solution, the approximation ratio α is defined as:

For a maximization problem, it is:

A lower ratio (closer to 1) indicates a better approximation for a minimization problem, and a higher ratio (closer to 1) indicates a better approximation for a maximization problem.

• Polynomial-time approximation scheme (PTAS): A family of algorithms that can find a solution within (1+ϵ) times the optimal solution for any given ϵ>0. The running time of a PTAS is polynomial in the size of the input for fixed ϵ, but can become impractical as ϵ decreases.

• Fully polynomial-time approximation scheme (FPTAS): A PTAS where the running time is polynomial in both the size of the input and 1/ϵ. FPTAS is more efficient than PTAS in terms of how running time depends on ϵ.

Consider the following examples of approximation algorithms:

• Vertex cover: For the vertex cover problem, a simple 2-approximation algorithm works as follows:

o Initialize an empty set C for the vertex cover.

o While there are edges left in the graph, pick any edge (u, v).

o Add both endpoints u and v to C, and remove all edges incident to u or v from the graph.

o Return C.

This algorithm ensures that C is at most twice the size of the minimum vertex cover.

• TSP: For the metric TSP (where the distance between any two points satisfies the triangle inequality), a 2-approximation algorithm can be implemented using a minimum spanning tree (MST):

o Compute an MST of the graph.

o Perform a preorder traversal of the MST to get a Hamiltonian cycle.

o Shortcut the Hamiltonian cycle to remove any repeated vertices (thanks to the triangle inequality, this doesn't increase the tour length).

This algorithm produces a tour whose length is at most twice the optimal length.

• Knapsack problem: For the 0/1 knapsack problem, an FPTAS is available:

o Sort items by their value-to-weight ratio.

o Use dynamic programming to find the best combination of items up to a scaled and rounded version of their values.

o Adjust the dynamic programming approach to ensure the solution is within (1+ϵ) of the optimal value.

o The running time of this FPTAS is polynomial in both the number of items and 1/ϵ.

Importance and use cases

Approximation algorithms are critical in various fields such as:

• Operations research: Where finding exact solutions to large-scale optimization problems can be computationally infeasible.

• Network design: Where problems like network flow and connectivity must be optimized.

• Bioinformatics: For problems like sequence alignment and phylogenetic tree reconstruction.

• Machine learning: Where clustering and classification problems often rely on approximate solutions.

Approximation algorithms provide practical solutions for NP-hard optimization problems, balancing between solution quality and computational efficiency. They are essential tools in scenarios where exact solutions are impractical due to time or resource constraints.

Probabilistic algorithms

Probabilistic algorithms, also known as randomized algorithms, use random numbers to influence the steps or decisions made during computation. These algorithms can offer significant advantages in terms of simplicity, speed, and efficiency, especially for problems where deterministic algorithms might be too slow or complex. Here’s an overview of probabilistic algorithms:

The following are the types of probabilistic algorithms:

• Las Vegas algorithms: These algorithms always produce a correct result, but their running time is a random variable.

For example, randomized quicksort is a Las Vegas algorithm where the pivot is chosen randomly. This ensures that the algorithm avoids worst-case scenarios on average, leading to an expected O(nlogn) running time.

• Monte Carlo algorithms: These algorithms have a fixed running time, but they may produce incorrect results with a certain probability.

For example, the Monte Carlo algorithm for primality testing, such as the Miller-Rabin primality test, runs in polynomial time but has a small probability of incorrectly identifying a composite number as prime. The error probability can be reduced by increasing the number of iterations.

The examples of probabilistic algorithms are as follows:

• Randomized quick sort algorithm: In quick sort, the pivot element is chosen randomly. This randomization ensures that the expected running time remains O(nlogn), making it less likely to encounter the worst-case scenario O(n2).

The advantage of randomized quick sort is the simplicity and good average-case performance.

• Miller-Rabin primality test: This algorithm tests whether a number is prime by using random bases and checking certain mathematical conditions. Each test reduces the probability of error.

Its advantage is that it is fast and can handle very large numbers efficiently. The probability of error can be made arbitrarily small by running more iterations.

• Randomized algorithms for graph problems: For example, for the min-cut problem (finding the minimum cut in a graph), the Karger’s algorithm repeatedly contracts random edges until only two vertices remain. This process is repeated multiple times to increase the probability of finding the minimum cut.

It is simpler than deterministic counterparts and often faster.

• Monte Carlo simulation: It is used for numerical integration and optimization problems. Random samples are taken, and the results are averaged to approximate the solution.

It is applicable to a wide range of problems where deterministic methods are infeasible.

The following are the benefits of probabilistic algorithms:

• Simplicity: Randomization can simplify the design and implementation of algorithms.

• Speed: Randomized algorithms can often be faster than their deterministic counterparts, especially for large input sizes.

• Performance guarantees: They often have good average-case performance and can avoid worst-case scenarios.

• Versatility: Applicable to a wide range of problems, including those in cryptography, numerical analysis, and optimization.

The limitations of probabilistic algorithms are as follows:

• Uncertainty: Monte Carlo algorithms can produce incorrect results, which might not be acceptable in all applications.

• Dependence on randomness: The performance of these algorithms can depend heavily on the quality of the random number generator used.

• Complexity analysis: Analyzing the performance and correctness of probabilistic algorithms can be more challenging than for deterministic ones.

Probabilistic algorithms leverage randomness to provide efficient and often simpler solutions to complex problems. While they introduce some uncertainty, their advantages in terms of speed and average-case performance make them valuable tools in many areas of computer science and beyond.

Bounded-error probabilistic polynomial time

Bounded-error probabilistic polynomial time (BPP) is a class of decision problems that can be solved by probabilistic algorithms in polynomial time with a bounded probability of error. It is a key concept in computational complexity theory, highlighting the power and limitations of probabilistic computation.

A language L belongs to the complexity class BPP if there exists a probabilistic Turing machine M such that:

• Polynomial time: M runs in polynomial time for all inputs.

• Bounded error: For any input x:

o If x ∈ L, M accepts x with a probability of at least 2/3.

o If 𝑥 ∉ 𝐿, M rejects x with a probability of at least 2/3.

The constants 2/3 and 1/3 are arbitrary and can be replaced by any values between 1/2 and 1, as long as the difference is a constant. This error probability can be reduced further using techniques like majority voting or running the algorithm multiple times and taking the majority result.

The key characteristics of BPP are as follows:

• Randomness: BPP algorithms use randomness to make decisions during computation. The randomness allows the algorithm to potentially achieve better average-case performance or avoid worst-case scenarios.

• Efficiency: BPP algorithms run in polynomial time, making them efficient for practical use on large inputs.

• Bounded error: The probability of error is bounded away from 1/2 by a constant. This ensures that the algorithm is highly likely to produce the correct result.

The examples of BPP algorithms are as follows:

• Primality testing: The Miller-Rabin primality test is a classic example of a BPP algorithm. It can determine whether a number is prime with high confidence, and the probability of error can be made arbitrarily small by running the test multiple times.

• Graph properties: Testing certain properties of graphs, such as connectivity or bipartiteness, can often be done using probabilistic algorithms that fall within BPP.

• Approximate counting: Estimating the number of solutions to certain combinatorial problems can be efficiently done using randomized algorithms that belong to BPP.

The following is BPP’s relationship with other complexity classes:

• P: P is the class of problems solvable by deterministic algorithms in polynomial time. Clearly, P⊆BPP, as deterministic algorithms can be viewed as probabilistic algorithms with no randomness.

• RP: RP is a subclass of BPP where the algorithm has no false positives:

o If x ∈ L, the algorithm accepts with a probability of at least ½ .

o If 𝑥 ∉ 𝐿, the algorithm always rejects.

• Co-RP: The complement of RP, where the algorithm has no false negatives:

o If 𝑥 ∈ 𝐿, the algorithm always accepts.

o If 𝑥 ∉ 𝐿, the algorithm rejects with a probability of at least ½.

• Nondeterministic polynomial time (NP): The relationship between BPP and NP is not fully understood. It is known that P ⊆ BPP ⊆ PNP (P with an NP oracle).

The following is the significance of BPP:

• Practical applications: BPP algorithms are used in practical applications such as cryptography, randomized algorithms for network design, and machine learning, where deterministic algorithms might be too slow or complex.

• Theoretical insights: Studying BPP helps in understanding the power of randomness in computation and provides insights into the boundaries of efficient computation.

Reducing error probability

The error probability in BPP algorithms can be reduced by amplification techniques:

• Repeated runs: Run the algorithm multiple times and take the majority vote. For example, running a BPP algorithm k times and taking the majority result can reduce the error probability to 2−𝑘.

• Chernoff bound: This probabilistic bound can be used to show that the error probability decreases exponentially with the number of repetitions.

BPP represents problems that can be efficiently solved by probabilistic algorithms with a small, bounded error probability. It highlights the role of randomness in computation, providing powerful tools for solving complex problems and contributing to both practical applications and theoretical computer science.

Randomized polynomial time

Randomized polynomial time (RP) is a complexity class that characterizes decision problems that can be solved by probabilistic algorithms with one-sided error in polynomial time. This means that the algorithm may err on one side (either false positives or false negatives) but not both. Specifically, RP allows false negatives but not false positives.

A language L is in the complexity class RP if there exists a probabilistic Turing machine M such that:

• Polynomial time: M runs in polynomial time for all inputs.

• One-sided error:

o If x ∈ L, the probability that M accepts x is at least ½.

o If 𝑥 ∉ 𝐿, the probability that M accepts x is 0 (i.e., M always rejects x).

The following are the key characteristics of RP are as follows:

• Randomness: RP algorithms use randomness to make decisions during the computation.

• Efficiency: RP algorithms run in polynomial time, making them efficient.

• One-sided error: RP algorithms have no false positives. They only have false negatives, meaning they may fail to recognize a member of the language, but they will never incorrectly identify a non-member as a member.

Consider the following examples of RP algorithms:

• Primality testing: Before the discovery of deterministic polynomial-time algorithms for primality testing (such as the AKS primality test), the Miller-Rabin primality test was a common example of an RP algorithm. It can determine whether a number is composite with high confidence and no false positives (i.e., it never incorrectly identifies a composite number as prime).

• Polynomial identity testing: Given two polynomials, determine whether they are identical. The RP algorithm evaluates the polynomials at a randomly chosen point. If they differ at this point, they are not identical. If they are the same, the algorithm repeats the process to increase confidence.

The following is RP algorithm’s relationship with other complexity classes:

• P: P is the class of problems solvable by deterministic algorithms in polynomial time. Clearly, P⊆RP, as deterministic algorithms can be viewed as probabilistic algorithms with no randomness and no error.

• BPP: BPP algorithms allow two-sided error, meaning they can have both false positives and false negatives but with bounded probability. It is known that RP ⊆ BPP.

• Co-RP: Co-RP is the class of problems where the complements are in RP. In Co-RP:

o If x ∈ L, the algorithm always accepts.

o If 𝑥 ∉ 𝐿, the algorithm rejects with a probability of at least ½.

• Zero-error probabilistic polynomial time (ZPP): ZPP is the class of problems solvable by a randomized algorithm that always returns the correct answer or reports I don't know (in polynomial expected time). It is the intersection of RP and Co-RP, i.e., ZPP=RP ∩ Co-RP.

Reducing error probability

Amplification techniques can reduce the error probability in RP algorithms:

• Repeated runs: Run the algorithm multiple times. If the input x is in L, the probability that all runs fail to recognize x as a member decreases exponentially. For example, running an RP algorithm k times reduces the error probability to (1/2)k.

The significance of RP is as follows:

• Practical applications: RP algorithms are used in various fields where it is crucial to avoid false positives, such as cryptographic protocols and probabilistic verification.

• Theoretical insights: Studying RP helps in understanding the boundaries of efficient computation with randomness and the impact of one-sided error on algorithm design.

RP represents problems that can be efficiently solved by probabilistic algorithms with one-sided error. These algorithms use randomness and guarantee no false positives, making them valuable for applications requiring high confidence in the correctness of positive instances. Understanding RP and its relationships with other complexity classes provides insights into the power and limitations of randomness in computation.

Complement of randomized polynomial time

Complement of randomized polynomial time (Co-RP) is a complexity class that characterizes decision problems that can be solved by probabilistic algorithms with one-sided error in polynomial time, where the error is on the opposite side compared to RP. Specifically, Co-RP algorithms may have false positives but not false negatives.

Let us look at the definition of Co-RP. A language L is in the complexity class Co-RP if there exists a probabilistic Turing machine M such that:

• Polynomial time: M runs in polynomial time for all inputs.

• One-sided error:

o If 𝑥 ∈ 𝐿, the probability that M accepts 𝑥 is 1 (i.e., 𝑀 always accepts x).

o If 𝑥 ∉ 𝐿, the probability that M accepts 𝑥 is at most ½.

Key characteristics of Co-RP are as follows:

• Randomness: Co-RP algorithms use randomness to make decisions during the computation.

• Efficiency: Co-RP algorithms run in polynomial time, making them efficient.

• One-sided error: Co-RP algorithms have no false negatives. They may incorrectly identify a non-member as a member of the language (false positives), but they will never fail to recognize a member of the language.

Let us look at some examples of Co-RP algorithms:

• Polynomial identity testing: Given two polynomials, determine whether they are identical. The Co-RP algorithm evaluates the polynomials at a randomly chosen point. If they differ at this point, they are not identical, and the algorithm rejects. If they are the same at this point, the algorithm accepts, potentially with some probability of error if they are not truly identical.

• Minimum spanning tree verification: Given a graph and a spanning tree, verify if the spanning tree is a minimum spanning tree. A Co-RP algorithm can probabilistically verify this by checking certain properties or performing random checks.

Relationship with other complexity classes is given in the following list:

• P: P is the class of problems solvable by deterministic algorithms in polynomial time. Clearly, P ⊆ Co-RP, as deterministic algorithms can be viewed as probabilistic algorithms with no randomness and no error.

• RP: RP is the class of problems where the complements are in Co-RP. In RP:

o If 𝑥 ∈ 𝐿, the algorithm accepts with a probability of at least ½.

o If 𝑥 ∉ 𝐿, the algorithm always rejects.

• BPP: BPP algorithms allow two-sided error, meaning they can have both false positives and false negatives but with bounded probability. It is known that Co-RP ⊆ BPP.

• ZPP: ZPP is the class of problems solvable by a randomized algorithm that always returns the correct answer or reports I don't know (in polynomial expected time). It is the intersection of RP and Co-RP, i.e., ZPP = RP ∩ Co-RP.

Reducing error probability

The error probability in Co-RP algorithms can be reduced by amplification techniques:

• Repeated runs: Run the algorithm multiple times. If the input 𝑥 ∉ 𝐿, the probability that all runs incorrectly accept 𝑥 as a member decreases exponentially. For example, running a Co-RP algorithm 𝑘 times reduces the error probability to (1/2)k.

Significance of Co-RP is as follows:

• Practical applications: Co-RP algorithms are used in various fields where it is crucial to avoid false negatives, such as verification of mathematical proofs and combinatorial property testing.

• Theoretical insights: Studying Co-RP helps in understanding the boundaries of efficient computation with randomness and the impact of one-sided error on algorithm design.

Co-RP represents problems that can be efficiently solved by probabilistic algorithms with one-sided error, where the error is only on the positive side (false positives). These algorithms use randomness and guarantee no false negatives, making them valuable for applications, requiring high confidence in the correctness of negative instances. Understanding Co-RP and its relationships with other complexity classes provides insights into the power and limitations of randomness in computation.

Conclusion

This chapter examined the fundamental ideas of computational complexity, focusing on how different problems are grouped according to the time and space required to solve them. It covered a range of topics from the distinction between P and NP problems to more complex subjects like NP-hardness, polynomial space, and probabilistic approaches. Through the study of reductions, approximations, and randomized techniques, the chapter demonstrated various strategies for managing problems that are difficult or impossible to solve exactly. Understanding these classifications enables learners to recognize which problems are computationally feasible and which present significant challenges.

In the next chapter, we will explore a variety of sophisticated algorithmic techniques designed to tackle demanding and large-scale computational tasks encountered in real-world applications.

Exercise

1. Define and differentiate between the complexity classes P and NP.

2. Explain the concept of polynomial-time reductions.

3. What does it mean for a problem to be NP-hard but not NP-complete?

4. Compare and contrast deterministic and probabilistic algorithms in the context of complexity classes.

5. Discuss the relevance and limitations of approximation algorithms for NP-hard problems.

CHAPTER 12Other Advanced Algorithms

Introduction

This chapter explores advanced algorithms, organized into categories like cryptosystems, computational geometry, and complexity classes. It covers randomized algorithms and NP-completeness problems, providing a comprehensive look at each area. It also includes a variety of advanced algorithms, complete with detailed explanations and complexity analyses in both space and time formats.

Structure

The chapter covers the following topics:

• Randomized quick sort

• Graham’s scan algorithm

• Jarvis’s March algorithm

• Karger’s minimum cut

• Fisher-Yates shuffles

• Travelling salesman approximation algorithm

• Euclid's algorithm for GCD

• Modulo arithmetic

• Chinese remainder theorem

• RSA public-key cryptosystem

• 3-CNF satisfiability

• The clique problem

• The vertex problem

Objectives

The goal of this chapter is to explore advanced algorithms, grouping them into areas such as cryptosystems, computational geometry, and complexity classes. It seeks to provide a clear and detailed understanding of topics like randomized algorithms and NP-completeness problems within these fields.

Randomized quick sort

Randomized quick sort is a variation of the quick sort algorithm that improves performance on average by randomly selecting the pivot element. It works as follows:

1. Pick a random pivot: Rather than always choosing the first or last element as the pivot, randomly select an element from the array.

2. Partition the array: Arrange the elements so that those less than the pivot is on the left and those greater than the pivot are on the right.

3. Sort the subarrays: Apply these steps recursively to the left and right subarrays.

This approach helps ensure that the algorithm performs well across different scenarios.

Algorithm 12.1: Randomized quick sort

RandomizedQuickSort(arr, low, high):

IF low < high:

pivot_index = RandomizedPartition(arr, low, high)

RandomizedQuickSort(arr, low, pivot_index - 1)

RandomizedQuickSort(arr, pivot_index + 1, high)

RandomizedPartition(arr, low, high):

pivot_index = Random(low, high)

Swap(arr[pivot_index], arr[high])

RETURN Partition(arr, low, high)

Partition(arr, low, high):

pivot = arr[high]

i = low - 1

FOR j = low TO high - 1:

IF arr[j] <= pivot:

i = i + 1

Swap(arr[i], arr[j])

Swap(arr[i + 1], arr[high])

RETURN i + 1

Swap(a, b):

temp = a

a = b

b = temp

Program 12.1: Implementation in C

#include <stdio.h>

#include <stdlib.h>

#include <time.h>

// Function to swap two elements

void swap(int *a, int *b) {

int temp = *a;

*a = *b;

*b = temp;

}

// Partition function

int partition(int arr[], int low, int high) {

int pivot = arr[high];

int i = (low - 1);

for (int j = low; j <= high - 1; j++) {

if (arr[j] <= pivot) {

i++;

swap(&arr[i], &arr[j]);

}

}

swap(&arr[i + 1], &arr[high]);

return (i + 1);

}

// Randomized Partition function

int randomizedPartition(int arr[], int low, int high) {

int pivot_index = low + rand() % (high - low + 1);

swap(&arr[pivot_index], &arr[high]);

return partition(arr, low, high);

}

// Randomized QuickSort function

void randomizedQuickSort(int arr[], int low, int high) {

if (low < high) {

int pivot_index = randomizedPartition(arr, low, high);

randomizedQuickSort(arr, low, pivot_index - 1);

randomizedQuickSort(arr, pivot_index + 1, high);

}

}

// Utility function to print an array

void printArray(int arr[], int size) {

for (int i = 0; i < size; i++)

printf(“%d “, arr[i]);

printf(“\n”);

}

// Main function to test the Randomized QuickSort

int main() {

srand(time(0)); // Seed for random number generation

int arr[] = {10, 7, 8, 9, 1, 5};

int n = sizeof(arr) / sizeof(arr[0]);

printf("Original array: \n");

printArray(arr, n);

randomizedQuickSort(arr, 0, n - 1);

printf("Sorted array: \n");

printArray(arr, n);

return 0;

}

The following is the explanation of the preceding code:

• swap: This function swaps two elements in the array.

• partition: It rearranges the array based on the pivot, so elements less than the pivot end up on one side, and the elements greater than the pivot end up on the other.

• randomizedPartition: This function picks a random pivot, moves it to the end of the array, and then calls the partition function.

• randomizedQuicksort: The core recursive function that sorts the array using the randomized partitioning method.

• printArray: A helper function that prints out the array.

• main: Sets up the array, prints the original version, sorts it using randomizedQuicksort, and then prints the sorted result.

Complexity analysis

The time and space complexity of randomized quicksort are as follows:

• Time complexity: On average, randomized quick sort runs in O(n log n) time. Although it is quite rare, in the worst-case scenario, it can degrade to O(n²) if the randomization does not work out well.

• Space complexity: The space complexity is O(log n), which comes from the space needed for the recursive call stack.

The randomization in this algorithm helps ensure that the pivot element is more likely to split the array into two roughly equal parts, reducing the chances of encountering the worst-case performance.

Graham's scan algorithm

Graham's scan is an algorithm for finding the convex hull of a set of points on a plane. The convex hull is the smallest convex shape that can enclose all the given points.

The steps of Graham's scan algorithm are as follows:

1. Identify the starting point: Find the point with the lowest y-coordinate. If there are multiple such points, choose the one with the lowest x-coordinate. This point will be the pivot and is guaranteed to be on the convex hull.

2. Sort the points: The remaining points should be arranged in accordance with the angle that they create with the pivot. Whenever two points form the same angle, the one that is closer to the pivot should be selected.

3. Build the convex hull: The next step is to build the convex hull, which is done as follows:

a. Start with the pivot and the point with the smallest angle.

b. Go through the sorted points and check whether each point makes a left or right turn relative to the last two points in the hull.

c. If a point makes a right turn, it is removed from the hull (it is not part of the convex hull).

d. If it makes a left turn or is collinear, add it to the hull.

This approach ensures that you construct the convex hull efficiently and correctly.

Algorithm 12.2: Graham's scan algorithm

FUNCTION GrahamScan(points):

pivot = FindLowestPoint(points)

sorted_points = SortByPolarAngle(points, pivot)

hull = Stack()

PUSH(pivot, hull)

PUSH(sorted_points[0], hull)

FOR i = 1 TO LENGTH(sorted_points) - 1:

WHILE SIZE(hull) > 1 AND ccw(NextToTop(hull), Top(hull), sorted_points[i]) <= 0:

POP(hull)

PUSH(sorted_points[i], hull)

RETURN hull

FUNCTION FindLowestPoint(points):

lowest = points[0]

FOR point IN points:

IF point.y < lowest.y OR (point.y == lowest.y AND point.x < lowest.x):

lowest = point

RETURN lowest

FUNCTION SortByPolarAngle(points, pivot):

SORT points BY (angle_with_pivot, distance_from_pivot)

RETURN points

FUNCTION ccw(a, b, c):

RETURN (b.x - a.x) * (c.y - a.y) - (b.y - a.y) * (c.x - a.x)

The following is the explanation of the algorithm:

• FindLowestPoint: This function finds the point with the lowest y-coordinate, which becomes the pivot for the convex hull.

• SortByPolarAngle: This function sorts the remaining points by the angle they make with the pivot.

• ccw(counter-clockwise): This function checks whether three points make a left turn, a right turn, or are collinear.

• GrahamScan: This function uses a stack to build the convex hull by adding points that make a left turn and removing those that make a right turn.

Program 12.2: Implementation in C

#include <stdio.h>

#include <stdlib.h>

// Point structure

typedef struct Point {

int x, y;

} Point;

// Function to swap two points

void swap(Point *a, Point *b) {

Point temp = *a;

*a = *b;

*b = temp;

}

// Function to find the orientation of the triplet (p, q, r).

// The function returns the following values:

// 0 -> p, q and r are collinear

// 1 -> Clockwise

// 2 -> Counterclockwise

int orientation(Point p, Point q, Point r) {

int val = (q.y - p.y) * (r.x - q.x) - (q.x - p.x) * (r.y - q.y);

if (val == 0) return 0; // collinear

return (val > 0) ? 1 : 2; // clock or counterclock wise

}

// Function to sort points by polar angle with the pivot

int compare(const void *vp1, const void *vp2) {

Point *p1 = (Point *)vp1;

Point *p2 = (Point *)vp2;

int o = orientation(pivot, *p1, *p2);

if (o == 0)

return (p1->x - pivot.x) * (p1->x - pivot.x) + (p1->y - pivot.y) * (p1->y - pivot.y) <

(p2->x - pivot.x) * (p2->x - pivot.x) + (p2->y - pivot.y) * (p2->y - pivot.y) ? -1 : 1;

return (o == 2) ? -1 : 1;

}

// Function to return the next to top element of a stack

Point nextToTop(Point stack[], int *top) {

return stack[*top - 1];

}

// Function to print the convex hull

void printConvexHull(Point points[], int n) {

// Find the bottom-most point

int ymin = points[0].y, min = 0;

for (int i = 1; i < n; i++) {

int y = points[i].y;

if ((y < ymin) || (ymin == y && points[i].x < points[min].x))

ymin = points[i].y, min = i;

}

// Place the bottom-most point at first position

swap(&points[0], &points[min]);

// Sort n-1 points with respect to the first point. A point p1 comes before p2 in sorted output if p2

// has a larger polar angle (in counterclockwise direction) than p1

pivot = points[0];

qsort(&points[1], n - 1, sizeof(Point), compare);

// Create an empty stack and push first three points to it

Point stack[n];

int top = -1;

stack[++top] = points[0];

stack[++top] = points[1];

stack[++top] = points[2];

// Process remaining n-3 points

for (int i = 3; i < n; i++) {

// Keep removing top while the angle formed by points next-to-top, top, and points[i] makes a non-left turn

while (top > 0 && orientation(nextToTop(stack, &top), stack[top], points[i]) != 2)

top--;

stack[++top] = points[i];

}

// Now stack has the output points, print contents of stack

printf("The points in the convex hull are:\n");

for (int i = 0; i <= top; i++)

printf("(%d, %d)\n", stack[i].x, stack[i].y);

}

// Main function

int main() {

Point points[] = {{0, 3}, {1, 1}, {2, 2}, {4, 4}, {0, 0}, {1, 2}, {3, 1}, {3, 3}};

int n = sizeof(points) / sizeof(points[0]);

printConvexHull(points, n);

return 0;

}

The following is the explanation:

• Point structure: Defines a point with its x and y coordinates.

• swap: Switches the positions of two points.

• orientation: Determines how three points (p, q, r) are oriented relative to each other, whether they form a left turn, a right turn, or are collinear.

• compare: Sorts the points based on the angle they make with the pivot.

• nextToTop: Retrieves the second-to-last element from the stack.

• printConvexHull: Computes the convex hull of a set of points and displays it.

Complexity analysis

The time and space complexity of Graham’s scan algorithm are as follows:

• Time complexity: The time complexity is calculated as follows:

1. Finding the lowest point: This step involves going through all the points to find the one with the lowest y-coordinate, which takes O(n) time.

2. Sorting the points: Once the pivot is identified, the remaining points are sorted by their polar angle relative to the pivot. Sorting is the most time-consuming part and usually takes O(n log n) time using efficient algorithms like merge sort or quick sort.

3. Constructing the convex hull: After sorting, the convex hull is built by going through the sorted points and using a stack. This step takes O(n) time, as each point is added to and removed from the stack at most once.

Overall, the time complexity of Graham’s scan is dominated by the sorting step, making it O(n log n).

• Space complexity: The space complexity is as follows:

o Storage for points: The algorithm needs O(n) space to store the points.

o Stack space: It also requires O(n) space for the stack used to build the convex hull in the worst-case scenario where all points are part of the hull.

In conclusion, Graham’s scan is efficient with a time complexity of O(n log n) and a space complexity of O(n), making it a solid choice for finding the convex hull of a set of points.

Jarvis’s March algorithm

Jarvis's March, also known as the gift wrapping algorithm, is used to find the convex hull of a set of points on a plane. The convex hull is the smallest convex shape that can enclose all the given points.

The steps of Jarvis's March algorithm are as follows:

1. Find the leftmost point: Start by finding the point with the smallest x-coordinate. If there are ties, choose the one with the smallest y-coordinate. This point will definitely be part of the convex hull.

2. Select the next point: From your current point, find the next point such that all other points are to the right of the line drawn between the current point and this new point. Use the orientation function to check the direction of the turn.

3. Repeat: Continue this process, selecting the next point and checking the direction of the turn, until you loop back to the starting point.

This method effectively wraps around the set of points to form the convex hull.

Algorithm 12.3: Jarvis's March algorithm

FUNCTION JarvisMarch(points):

hull = []

# Find the leftmost point

leftmost = points[0]

FOR point IN points:

IF point.x < leftmost.x OR (point.x == leftmost.x AND point.y < leftmost.y):

leftmost = point

current = leftmost

REPEAT:

ADD current TO hull

next_point = points[0]

FOR point IN points:

IF next_point == current OR orientation(current, next_point, point) == -1:

next_point = point

current = next_point

UNTIL current == leftmost

RETURN hull

FUNCTION orientation(p, q, r):

# Returns -1 if r is to the left of line pq

# Returns 0 if p, q, r are collinear

# Returns 1 if r is to the right of line pq

val = (q.y - p.y) * (r.x - q.x) - (q.x - p.x) * (r.y - q.y)

IF val == 0: RETURN 0

IF val > 0: RETURN 1

ELSE: RETURN -1

The explanation is as follows:

• Orientation: Using the three locations (p, q, and r), this function determines the direction of the turn that has been made. If r is to the left of the line produced by p and q, it returns -1; if all three points are on a straight line, it returns 0; and if r is to the right of that line, it returns to the right of that line.

• JarvisMarch: This function carries out Jarvis's March algorithm to find the convex hull. It starts at the leftmost point and repeatedly selects the next point such that all other points are on the right side of the line connecting the current point and the new point.

Program 12.3: Implementation in C

#include <stdio.h>

typedef struct Point {

int x, y;

} Point;

// Function to swap two points

void swap(Point *a, Point *b) {

Point temp = *a;

*a = *b;

*b = temp;

}

// Function to find the orientation of the triplet (p, q, r).

// The function returns the following values:

// 0 -> p, q and r are collinear

// 1 -> Clockwise

// 2 -> Counterclockwise

int orientation(Point p, Point q, Point r) {

int val = (q.y - p.y) * (r.x - q.x) - (q.x - p.x) * (r.y - q.y);

if (val == 0) return 0; // collinear

return (val > 0) ? 1 : 2; // clock or counterclock wise

}

// Function to print the convex hull

void printConvexHull(Point points[], int n) {

// There must be at least 3 points

if (n < 3) return;

// Initialize Result

Point hull[n];

int hullSize = 0;

// Find the leftmost point

int leftmost = 0;

for (int i = 1; i < n; i++) {

if (points[i].x < points[leftmost].x)

leftmost = i;

}

// Start from leftmost point, keep moving counterclockwise

// until reach the start point again. This loop runs O(h)

// times where h is number of points in result or output.

int p = leftmost, q;

do {

// Add current point to result

hull[hullSize++] = points[p];

// Search for a point 'q' such that orientation(p, x,

// q) is counterclockwise for all points 'x'. The idea

// is to keep track of last visited most counterclock-

// wise point in q. If any point 'i' is more counterclock-

// wise than q, then update q.

q = (p + 1) % n;

for (int i = 0; i < n; i++) {

// If i is more counterclockwise than current q, then

// update q

if (orientation(points[p], points[i], points[q]) == 2)

q = i;

}

// Now q is the most counterclockwise with respect to p

// Set p as q for next iteration, so that q is added to

// result 'hull'

p = q;

} while (p != leftmost); // While we don't come to first point

// Print Result

printf("The points in the convex hull are:\n");

for (int i = 0; i < hullSize; i++)

printf("(%d, %d)\n", hull[i].x, hull[i].y);

}

// Main function

int main() {

Point points[] = {{0, 3}, {1, 1}, {2, 2}, {4, 4}, {0, 0}, {1, 2}, {3, 1}, {3, 3}};

int n = sizeof(points) / sizeof(points[0]);

printConvexHull(points, n);

return 0;

}

Let us look at the explanation. The preceding program consists of the following modules:

• Point structure: Defines a point with x and y coordinates.

• swap: Swaps the positions of two points.

• orientation: Checks the orientation of three points (p, q, r). It returns 0 if they are collinear, 1 if they form a clockwise turn, and 2 if they form a counterclockwise turn.

• printConvexHull: Uses the Jarvis's March algorithm to find and display the convex hull of a set of points.

• main: Sets up the points and calls printConvexHull to compute and show the convex hull.

Complexity analysis

The complexity analysis of Jarvis’s March algorithm is as follows:

• Time complexity: The time complexity is calculated as follows:

1. Finding the leftmost point: This step involves going through all the points to identify the one with the smallest x-coordinate. It takes O(n) time to do this.

2. Constructing the convex hull: The core of Jarvis’s March involves finding the next point for the convex hull by checking all points relative to the current one. In the worst-case scenario, this can mean checking every point each time, leading to a time complexity of O(n²). This inefficiency arises because each point may require a full scan to ensure it is correctly positioned relative to the convex hull.

So, Jarvis’s March has an overall time complexity of O(n²), which can be slower compared to other algorithms like Graham’s scan or QuickHull, especially with larger datasets.

• Space complexity: The space complexity is as follows:

o Storage for points: The algorithm needs O(n) space to store the points.

o Additional space: It also requires a small amount of extra space for variables and the final result, but this is minimal compared to the space used for storing the points.

Overall, Jarvis’s March has a space complexity of O(n), making it suitable for smaller datasets despite its higher time complexity.

In summary, Jarvis' March is a straightforward and simple algorithm to build; yet, due to the fact that it has an O(n2) time complexity, it may not be as effective for large datasets. It has a space complexity of O(n), which is well within the realm of possibility for more compact point sets.

Karger’s minimum cut

Karger's algorithm is a randomized method for finding the minimum cut of a connected graph. A cut splits the graph’s vertex into two non-empty groups, and the cut size is the number of edges connecting these groups. The minimum cut is the one with the fewest edges. Karger's algorithm works by repeatedly contracting random edges until only two nodes are left, with the remaining edges between these two nodes representing the cut.

The following are the steps of Karger's minimum cut algorithm:

1. Choose a random edge: Pick a random edge (u, v) from the graph.

2. Contract the edge: Merge the two vertices, u and v, into a single vertex. Make sure to remove any self-loops created by this merge.

3. Repeat: Continue contracting edges until only two vertices are left. The edges between these two vertices then represent the minimum cut.

4. Repeat multiple times: To improve the chances of finding the true minimum cut, run the entire algorithm multiple times and keep track of the smallest cut found.

This approach leverages randomness to approximate the minimum cut, and repeating the process multiple times helps in finding a more accurate result.

Algorithm 12.4: Karger's minimum cut algorithm

FUNCTION KargerMinCut(graph):

minCut = INFINITY

REPEAT log(n) * n TIMES:

copiedGraph = copy(graph)

WHILE copiedGraph has more than 2 vertices:

edge = randomEdge(copiedGraph)

contract(copiedGraph, edge)

currentCut = number of edges in copiedGraph

IF currentCut < minCut:

minCut = currentCut

RETURN minCut

FUNCTION contract(graph, edge):

u, v = edge

merge vertices u and v into a single vertex

remove self-loops

The explanation is as follows:

• KargerMinCut: This function runs the algorithm several times (usually log(n) * n times) to boost the chances of finding the true minimum cut. It tracks the smallest cut discovered during all these runs.

• contract: This function merges the two vertices connected by the chosen edge into a single vertex and removes any self-loops that result from this merge.

Program 12.4: Implementation in C

#include <stdio.h>

#include <stdlib.h>

#include <limits.h>

#include <time.h>

// A structure to represent a graph

struct Graph {

int V, E;

int** edges;

};

// A utility function to create a graph with V vertices and E edges

struct Graph* createGraph(int V, int E) {

struct Graph* graph = (struct Graph*)malloc(sizeof(struct Graph));

graph->V = V;

graph->E = E;

graph->edges = (int**)malloc(E * sizeof(int*));

for (int i = 0; i < E; i++) {

graph->edges[i] = (int*)malloc(2 * sizeof(int));

}

return graph;

}

// A utility function to find the subset of an element i

int find(int parent[], int i) {

if (parent[i] == i)

return i;

return find(parent, parent[i]);

}

// A function that does union of two sets of x and y (uses union by rank)

void Union(int parent[], int rank[], int x, int y) {

int xroot = find(parent, x);

int yroot = find(parent, y);

if (rank[xroot] < rank[yroot])

parent[xroot] = yroot;

else if (rank[xroot] > rank[yroot])

parent[yroot] = xroot;

else {

parent[yroot] = xroot;

rank[xroot]++;

}

}

// A function to contract an edge (u, v) in the given graph

void contractEdge(struct Graph* graph, int u, int v) {

for (int i = 0; i < graph->E; i++) {

if (graph->edges[i][0] == v)

graph->edges[i][0] = u;

if (graph->edges[i][1] == v)

graph->edges[i][1] = u;

}

}

// A function to copy a graph

struct Graph* copyGraph(struct Graph* graph) {

struct Graph* newGraph = createGraph(graph->V, graph->E);

for (int i = 0; i < graph->E; i++) {

newGraph->edges[i][0] = graph->edges[i][0];

newGraph->edges[i][1] = graph->edges[i][1];

}

return newGraph;

}

// A function to count the number of edges in the graph

int countEdges(struct Graph* graph) {

int count = 0;

for (int i = 0; i < graph->E; i++) {

if (graph->edges[i][0] != graph->edges[i][1])

count++;

}

return count;

}

// Karger's minimum cut algorithm

int KargerMinCut(struct Graph* graph) {

int minCut = INT_MAX;

for (int i = 0; i < graph->V * graph->V; i++) {

struct Graph* copiedGraph = copyGraph(graph);

int vertices = graph->V;

while (vertices > 2) {

int randomEdge = rand() % copiedGraph->E;

int u = copiedGraph->edges[randomEdge][0];

int v = copiedGraph->edges[randomEdge][1];

contractEdge(copiedGraph, u, v);

vertices--;

}

int cutEdges = countEdges(copiedGraph);

if (cutEdges < minCut)

minCut = cutEdges;

free(copiedGraph);

}

return minCut;

}

// Driver program to test above functions

int main() {

srand(time(NULL));

int V = 4; // Number of vertices in graph

int E = 5; // Number of edges in graph

struct Graph* graph = createGraph(V, E);

// add edge 0-1

graph->edges[0][0] = 0;

graph->edges[0][1] = 1;

// add edge 0-2

graph->edges[1][0] = 0;

graph->edges[1][1] = 2;

// add edge 0-3

graph->edges[2][0] = 0;

graph->edges[2][1] = 3;

// add edge 1-3

graph->edges[3][0] = 1;

graph->edges[3][1] = 3;

// add edge 2-3

graph->edges[4][0] = 2;

graph->edges[4][1] = 3;

printf("Minimum cut is %d\n", KargerMinCut(graph));

return 0;

}

The explanation is as follows:

• Graph structure: Defines a graph with a set number of vertices (V) and edges (E).

• createGraph: Sets up the graph by allocating memory and initializing its components.

• find: Identifies the subset to which a particular element belongs.

• Union: Merges two sets into one.

• contractEdge: Merges the two vertices connected by a chosen edge into a single vertex and removes any self-loops that are created.

• copyGraph: Creates a duplicate of the given graph.

• countEdges: Counts how many edges are present in the graph.

• KargerMinCut: Runs Karger’s minimum cut algorithm, repeating the edge contraction process multiple times to improve the chances of finding the minimum cut.

• main: Sets up the graph with a specified set of edges and calls KargerMinCut to find and display the minimum cut.

Complexity analysis

The time and space complexity of Karger’s algorithm are as follows:

• Time complexity: The time complexity is calculated in the following manner:

o Single run of Karger’s algorithm: In each run, the algorithm contracts edges until only two vertices are left. This requires O(E) operations, where E is the number of edges in the graph. Each contraction involves finding and merging sets, which can be efficiently handled with Union-Find data structures, taking nearly constant time for each operation.

The contraction process involves iterating through and merging edges, and since each edge can be contracted once, the overall time complexity per run is O(E).

o Number of runs: To increase the probability of finding the true minimum cut, Karger’s algorithm is typically run multiple times. The number of runs required to achieve a high probability of success is approximately O(n² log n), where n is the number of vertices. This is because the probability of finding the minimum cut in a single run is 1/n(n-1), and running the algorithm O(n² log n) times boosts the probability significantly.

Therefore, the overall time complexity of running Karger’s algorithm multiple times is O(n² log n * E), assuming each run takes O(E) time.

• Space complexity: The space complexity is as follows:

o Graph storage: The space needed to store the graph, including vertices and edges, is O(V + E), where V is the number of vertices and E is the number of edges.

o Additional space: Additional space is required for maintaining data structures like Union-Find to manage sets and handle edge contractions, which also requires O(V) space in the worst case.

Therefore, the overall space complexity is O(V + E).

In summary, Karger’s minimum cut algorithm has a time complexity of O(n² log n * E) when run multiple times to ensure a high probability of finding the minimum cut. Its space complexity is O(V + E), making it suitable for moderate-sized graphs.

Fisher-Yates shuffles

The Fisher-Yates shuffle, also known as the Knuth shuffle, is a method for shuffling a list of items so that each possible arrangement of the items is equally likely. In other words, it is a fair way to shuffle.

Imagine you have a playlist of songs or a deck of cards that you want to mix up. The Fisher-Yates shuffle ensures that every possible order of the songs or cards could happen, making the shuffle truly random.

By following these steps, every card has an equal chance of ending up in any position, ensuring a fair shuffle:

1. Start at the end: Begin with the last card in the deck.

2. Pick a random card: Choose a random card from the whole deck.

3. Swap the cards: Swap the randomly chosen card with the last card in the deck.

4. Move back: Move to the second-last card in the deck, and repeat the process, pick a random card from the remaining unshuffled cards, and swap.

5. Keep going: Continue this process, working backward through the deck, until you have reached the first card. By the time you get there, the entire deck will be shuffled.

Algorithm 12.5: Fisher-Yates shuffle

FUNCTION FisherYatesShuffle(arr, n):

FOR i FROM n - 1 DOWNTO 1:

j = randomInt(0, i)

SWAP arr[i] WITH arr[j]

The explanation is as follows:

• randomInt(0, i): This generates a random number between 0 and i, including both 0 and i. It is like rolling a die with (i + 1) sides and getting a random result.

• SWAP arr[i] WITH arr[j]: This means taking the element at position i in the list and swapping it with the element at position j. It is like trading places between two items in the list.

Program 12.5: Implementation in C

#include <stdio.h>

#include <stdlib.h>

#include <time.h>

// Function to swap two elements

void swap(int *a, int *b) {

int temp = *a;

*a = *b;

*b = temp;

}

// Function to perform the Fisher-Yates shuffle

void fisherYatesShuffle(int arr[], int n) {

// Seed the random number generator

srand(time(NULL));

for (int i = n - 1; i > 0; i--) {

// Generate a random index j such that 0 <= j <= i

int j = rand() % (i + 1);

// Swap arr[i] with the element at random index

swap(&arr[i], &arr[j]);

}

}

// Function to print an array

void printArray(int arr[], int n) {

for (int i = 0; i < n; i++) {

printf("%d ", arr[i]);

}

printf("\n");

}

// Main function to test the Fisher-Yates shuffle

int main() {

int arr[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

int n = sizeof(arr) / sizeof(arr[0]);

printf("Original array:\n");

printArray(arr, n);

fisherYatesShuffle(arr, n);

printf("Shuffled array:\n");

printArray(arr, n);

return 0;

}

The explanation is as follows:

• swap: This is a little helper function that switches the positions of two items in the list.

• fisherYatesShuffle: This is the main function that mixes up the list. It goes through the list from the end to the beginning, and for each item, it swaps it with a randomly chosen item that comes before it (or itself).

• printArray: This function just displays the items in the list.

• main: This function sets up the list, shows you what it looks like before shuffling, uses the fisherYatesShuffle to mix it up, and then shows you the shuffled list.

Complexity analysis

The complexity analysis of the Fisher-Yates shuffle is as follows:

• Time complexity: The Fisher-Yates shuffle algorithm is very efficient with time. It only needs to go through the list of items once, no matter how many items there are. This means if you have ten items, it takes roughly ten steps; if you have 100 items, it takes about 100 steps, and so on. We describe this efficiency as O(n), where n is the number of items in the list.

• Space complexity: Moreover, the Fisher-Yates shuffle is particularly effective in terms of space utilization. With the exception of a few variables of a predetermined size, it does not require any additional space to do its function. In other words, regardless of the size of your list, it will only require a minimal amount of additional space that is consistent throughout. Therefore, this efficiency is described as O(1).

Travelling salesman approximation algorithm

The TSP is a famous puzzle in computer science and operations research. Imagine a salesperson who needs to visit a list of cities and then return to the starting point. The challenge is to figure out the shortest route that visits each city exactly once and gets you back home.

Solving this problem perfectly is very hard and takes a lot of computing power (it is what we call NP-hard). However, there are smart shortcuts, known as approximation algorithms, that can find pretty good solutions much faster.

One of the most popular shortcuts is the Christofides algorithm. This algorithm can give you a route that is at most 1.5 times longer than the best possible route, but only if the distances between cities meet a specific condition (the triangle inequality, which means going from city A to city C is never shorter than going from A to B and then B to C).

So, while the exact solution to TSP might be tough to get, the Christofides algorithm helps you get a good-enough route in a reasonable amount of time.

The steps of the Christofides algorithm are as follows:

1. Build a simple backbone (minimum spanning tree): Start by creating a simple network (tree) that connects all cities with the shortest possible total distance without any loops.

2. Find lonely connections (odd degree vertices): In this tree, look for cities that have an odd number of connections.

3. Pair up the odd ones (minimum weight perfect matching): Match up these lonely cities in pairs, using the shortest possible paths.

4. Combine paths (combine MST and matching): Add these new shortest paths to your original tree. Now, you have a network where each city has an even number of connections.

5. Create a loop (Eulerian circuit): Use this network to create a path that visits each connection exactly once and returns to the starting city.

6. Simplify the path (Hamiltonian circuit): Convert this path into a final route by skipping any cities you have already visited, ensuring you visit each city exactly once.

Algorithm 12.6: Christofides algorithm

FUNCTION ChristofidesAlgorithm(graph):

# Step 1: Find MST

MST = findMST(graph)

# Step 2: Find vertices with odd degree

oddVertices = findOddDegreeVertices(MST)

# Step 3: Find minimum weight perfect matching for odd degree vertices

matching = findMinimumWeightPerfectMatching(graph, oddVertices)

# Step 4: Combine MST and matching to form Eulerian multigraph

multigraph = combineMSTAndMatching(MST, matching)

# Step 5: Find Eulerian circuit in multigraph

eulerianCircuit = findEulerianCircuit(multigraph)

# Step 6: Convert Eulerian circuit to Hamiltonian circuit

hamiltonianCircuit = convertToHamiltonianCircuit(eulerianCircuit)

RETURN hamiltonianCircuit

The explanation is as follows:

• findMST(graph): This step uses a method like Prim's or Kruskal's algorithm to create a basic network (tree) that connects all the cities with the shortest total distance, without any loops.

• findOddDegreeVertices(MST): Here, we look at the tree and find the cities that have an odd number of connections. These cities need special attention because they do not fit perfectly into our current network.

• findMinimumWeightPerfectMatching(graph, oddVertices): Next, we find the shortest paths to pair up these odd cities. This way, we make sure that every city has an even number of connections.

• combineMSTAndMatching(MST, matching): We then merge the original tree with these new shortest paths. This creates a network where every city now has an even number of connections, which is what we need for the next step.

• findEulerianCircuit(multigraph): In this network, we create a route that visits each connection exactly once and returns to the starting city. This type of route is called an Eulerian circuit.

• convertToHamiltonianCircuit(eulerianCircuit): Finally, we take this route and adjust it so that we visit each city only once. We do this by skipping over any cities we have already visited in the route.

Program 12.6: Implementation in C

#include <stdio.h>

#include <stdlib.h>

#include <limits.h>

// Define structure for a graph

struct Graph {

int V;

int** adjMatrix;

};

// Create a graph with V vertices

struct Graph* createGraph(int V) {

struct Graph* graph = (struct Graph*) malloc(sizeof(struct Graph));

graph->V = V;

graph->adjMatrix = (int**) malloc(V * sizeof(int*));

for (int i = 0; i < V; i++) {

graph->adjMatrix[i] = (int*) malloc(V * sizeof(int));

}

return graph;

}

// Utility function to print the adjacency matrix of the graph

void printGraph(struct Graph* graph) {

for (int i = 0; i < graph->V; i++) {

for (int j = 0; j < graph->V; j++) {

printf("%d ", graph->adjMatrix[i][j]);

}

printf("\n");

}

}

// A utility function to find the vertex with minimum key value, from

// the set of vertices not yet included in MST

int minKey(int key[], int mstSet[], int V) {

int min = INT_MAX, minIndex;

for (int v = 0; v < V; v++)

if (mstSet[v] == 0 && key[v] < min)

min = key[v], minIndex = v;

return minIndex;

}

// Function to construct and print MST for a graph represented using adjacency matrix

void primMST(struct Graph* graph) {

int V = graph->V;

int parent[V]; // Array to store constructed MST

int key[V]; // Key values used to pick minimum weight edge in cut

int mstSet[V]; // To represent set of vertices included in MST

// Initialize all keys as INFINITE

for (int i = 0; i < V; i++)

key[i] = INT_MAX, mstSet[i] = 0;

// Always include first 1st vertex in MST.

key[0] = 0; // Make key 0 so that this vertex is picked as first vertex

parent[0] = -1; // First node is always root of MST

// The MST will have V vertices

for (int count = 0; count < V - 1; count++) {

// Pick the minimum key vertex from the set of vertices not yet included in MST

int u = minKey(key, mstSet, V);

// Add the picked vertex to the MST Set

mstSet[u] = 1;

// Update key value and parent index of the adjacent vertices of the picked vertex.

for (int v = 0; v < V; v++)

if (graph->adjMatrix[u][v] && mstSet[v] == 0 && graph->adjMatrix[u][v] < key[v])

parent[v] = u, key[v] = graph->adjMatrix[u][v];

}

// Print the constructed MST

printf("Edge \tWeight\n");

for (int i = 1; i < V; i++)

printf("%d - %d \t%d \n", parent[i], i, graph->adjMatrix[i][parent[i]]);

}

int main() {

int V = 5;

struct Graph* graph = createGraph(V);

graph->adjMatrix[0][1] = 2;

graph->adjMatrix[0][2] = 3;

graph->adjMatrix[0][3] = 1;

graph->adjMatrix[0][4] = 4;

graph->adjMatrix[1][2] = 1;

graph->adjMatrix[1][3] = 5;

graph->adjMatrix[1][4] = 3;

graph->adjMatrix[2][3] = 2;

graph->adjMatrix[2][4] = 1;

graph->adjMatrix[3][4] = 6;

// Since the graph is undirected, fill the other half of the matrix

for (int i = 0; i < V; i++) {

for (int j = i; j < V; j++) {

graph->adjMatrix[j][i] = graph->adjMatrix[i][j];

}

}

printf("Adjacency Matrix of the graph:\n");

printGraph(graph);

printf("Minimum Spanning Tree (MST) using Prim's Algorithm:\n");

primMST(graph);

return 0;

}

The explanation is as follows:

● Graph structure: Think of this as a way to represent a network of cities (or nodes) and the connections between them using a table (adjacency matrix).

● createGraph: This step sets up the network, creating space in memory to store information about the cities and their connections, and then fills in the initial details.

● printGraph: This function simply displays the table that shows how all the cities are connected to each other.

● minKey: This function finds the city with the smallest connection cost that has not been added to the network yet. It is like picking the cheapest city to add to your growing network.

● primMST: Here is where we use Prim's algorithm to build a minimum spanning tree, a way to connect all the cities with the shortest total connection cost. This function also shows the connections (edges) and their costs.

● main: This is the main setup where we create our network of cities, show the connection details, and then use Prim's algorithm to find and display the minimum spanning tree.

Complexity analysis

The complexity analysis is calculated as follows:

● Time complexity: O(n3)

This algorithm is much faster than the brute-force method. It gives you a solution that is close to the best possible route, within 1.5 times the optimal length. The reason it works relatively quickly is that it handles the problem in a few key steps, finding a minimum spanning tree, matching up some connections, and creating a path that covers all the cities. Each of these steps involves calculations that are manageable within O (n3) time, meaning it is feasible for larger numbers of cities compared to the exhaustive approach.

● Space complexity: O(n2)

The amount of extra space the Christofides algorithm needs mainly comes from storing the graph's connections. Specifically, it uses an adjacency matrix, which is a big table showing how each city is connected to every other city. This table needs space proportional to n2, where n is the number of cities.

Euclid’s algorithm for GCD

Euclid's algorithm is a clever way to find the greatest common divisor (GCD) of two numbers. The GCD is the largest number that can evenly divide both the original numbers without leaving any remainder.

For example, if you want to find the GCD of 12 and 15, Euclid’s algorithm helps you figure out the biggest number that divides both 12 and 15 evenly, which in this case is 3. This method is efficient and quick, making it easy to handle even for larger numbers.

Euclid’s algorithm is a simple and efficient way to find the GCD of two numbers. It works as follows:

1. Start with two numbers: Begin with two numbers, a and b, where a is the larger number or at least equal to b.

2. Repeat the process: Repeat the following steps:

a. Check if b is 0: If b is 0, then the GCD is a because any number divided by 0 is itself.

b. Update the numbers: If 𝑏 is not 0, replace a with b, and replace b with the remainder when a is divided by b (this is written as a mod b).

c. Repeat: Keep doing this until b becomes 0.

3. Get the result: When b is 0, the remaining value of a is the GCD of the original two numbers.

In simple terms, keep swapping and reducing the numbers until one of them becomes 0, and the non-zero number is the greatest number that can evenly divide both of the original numbers.

Algorithm 12.7: Euclid's algorithm

FUNCTION EuclidGCD(a, b):

WHILE b != 0:

temp = b

b = a % b

a = temp

RETURN a

The explanation is as follows:

• temp = b: First, save the value of b in a temporary spot so you do not lose it.

• b = a % b: Next, update b to be the remainder when a is divided by b. This gives you the part of a that does not fit evenly into b.

• a = temp: Then, set a to be the old value of b (which you saved in step 1).

• Repeat: Keep doing these steps and updating a and b as described until b becomes 0.

• Return: When b is 0, the value of a is the GCD of the original numbers.

Program 12.7: Implementation in C

#include <stdio.h>

// Function to calculate GCD using Euclid's algorithm

int euclidGCD(int a, int b) {

while (b != 0) {

int temp = b;

b = a % b;

a = temp;

}

return a;

}

// Main function to test the GCD function

int main() {

int a = 56;

int b = 98;

printf("The GCD of %d and %d is %d\n", a, b, euclidGCD(a, b));

return 0;

}

The explanation is as follows:

• euclidGCD: This function uses Euclid’s algorithm to find the GCD of two numbers, a and b. It works by repeatedly swapping a and b with b and the remainder of a divided by b, until b becomes 0. At that point, a holds the GCD.

• main: This part of the code tests the euclidGCD function with some example numbers and prints out the result, showing you the GCD of those numbers.

Complexity analysis

The following is the time and space complexity of Euclid's algorithm:

• Time complexity: The time complexity is O(logmin(a,b)). It is very efficient due to the rapid reduction in problem size.

• Space complexity: There is a space complexity of O(1). Only a modest and consistent amount of additional space is required for it.

Modulo arithmetic

Modulo arithmetic is like working with a clock where you only care about the time on the face, not the actual number of hours that have passed. For example, on a standard clock, if you add three hours to 10 o'clock, you end up at 1 o'clock, not 13 o'clock.

In the same way, modulo arithmetic takes large numbers and wraps them around a fixed value, called the modulus. So, if you are working with modulo 12, adding 7 to 8 would bring you back to 3 (because 15 hours is the same as three hours on a 12-hour clock).

This kind of math is useful for simplifying problems in computer science, cryptography, and other fields, where dealing with huge numbers or repeating patterns is common.

Basic concepts

Imagine you are dividing a number by another number and only caring about the remainder. For example, if you divide 17 by 5, you get a remainder of 2 (because 17=3×5+2). In modular arithmetic, we say that 17 is congruent to 2 modulo 5, which we write as follows:

17 ≡ 2 (mod 5)

This means both 17 and 2 leave the same remainder when divided by 5.

Properties

The following list of operations is used with modulo operations:

• Addition: Adding numbers and then taking the remainder is the same as taking the remainder of each number first, then adding them, and taking the remainder again. For example, (17+7)%5=24%5=4, which is the same as [(17%5)+(7%5)]%5=(2+2)%5=4.

• Subtraction: Subtracting numbers and then taking the remainder works similarly. For instance, (17-7)%5=10%5=0, and [(17%5)-(7%5)]%5=(2-2)%5=0.

• Multiplication: Multiplying numbers and then taking the remainder is the same as taking the remainders first, multiplying them, and then taking the remainder. For example, (17×7)%5=119%5=4, which is the same as [(17%5)×(7%5)]%5=(2×2)%5=4.

• Division: This is a more difficult one. The process of dividing in modular arithmetic frequently includes locating a unique integer, which is referred to as the modular inverse, that enables the division to be performed.

Applications

Modulo arithmetic has applications in the following fields:

• Cryptography: In secure communication, like encrypting messages, modular arithmetic helps keep data safe. It is used in algorithms like Rivest–Shamir–Adleman (RSA) to ensure that only the right people can read the message.

• Computer science: It helps in various tasks like creating unique identifiers (hashes), generating random numbers, and more.

• Mathematics: It is useful for solving certain types of mathematical problems and understanding number properties.

Program 12.8: Implementation in C

#include <stdio.h>

// Function to perform addition modulo

int modAdd(int a, int b, int m) {

return (a % m + b % m) % m;

}

// Function to perform subtraction modulo

int modSub(int a, int b, int m) {

return (a % m - b % m + m) % m; // Add m to ensure non-negative result

}

// Function to perform multiplication modulo

int modMul(int a, int b, int m) {

return (a % m * b % m) % m;

}

// Function to perform division modulo using modular inverse (only valid for prime m)

int modDiv(int a, int b, int m) {

// Compute modular inverse of b modulo m using Fermat's Little Theorem

int inverse = 1;

int exponent = m - 2; // For prime m, a^(m-1) ≡ 1 (mod m) => a^(m-2) ≡ a^(-1) (mod m)

int base = b % m;

while (exponent > 0) {

if (exponent % 2 == 1) {

inverse = (inverse * base) % m;

}

base = (base * base) % m;

exponent /= 2;

}

return (a % m * inverse) % m;

}

int main() {

int a = 17, b = 5, m = 7;

printf("Addition: (%d + %d) %% %d = %d\n", a, b, m, modAdd(a, b, m));

printf("Subtraction: (%d - %d) %% %d = %d\n", a, b, m, modSub(a, b, m));

printf("Multiplication: (%d * %d) %% %d = %d\n", a, b, m, modMul(a, b, m));

printf("Division: (%d / %d) %% %d = %d\n", a, b, m, modDiv(a, b, m));

return 0;

}

The explanation is as follows:

• modAdd: This is like adding two numbers as usual, but then taking the result and seeing what is left if you only care about a fixed range. For example, if you add 8 and 9 in a system that wraps around after 10, you get 17. Since we are only interested in numbers from 0 to 9, you would end up with 7 (because 17 minus 10 leaves a remainder of 7).

• modSub: This works similarly to addition, but for subtraction. If you subtract one number from another and get a negative result, this function adjusts it so you still end up with a positive number within the desired range. For instance, subtracting 5 from 3 in a system where you wrap around after 10 would give you -2, but adjusting it to be within 0 to 9 would give you 8 (because -2 plus 10 equals 8).

• modMul: This involves multiplying two numbers and then finding out what is left if you divide by the modulus. For example, multiplying 4 by 6 and then taking the result modulo 7 gives you 24. Since 24 divided by 7 leaves a remainder of 3, you get 3.

• modDiv: Dividing in modular arithmetic is a bit trickier. It involves finding a special number (called the modular inverse) that helps you perform the division. This method usually works only when the modulus is a prime number. Essentially, it is like finding a number that, when multiplied by your original number, gives you the result you are looking for under the modulus.

Complexity analysis

The complexity analysis is as follows:

• Addition, subtraction, multiplication modulo: The complexity is O(1) as these operations are direct and efficient.

• Division modulo: The complexity is O(log m) as computing modular inverses introduces logarithmic complexity due to the algorithms involved.

• Exponentiation modulo: Due to the fact that efficient methods for exponentiation ensure that this operation is still manageable even with large exponents, it is calculated as O(log e).

Chinese remainder theorem

The chinese remainder theorem (CRT) is a mathematical tool that helps solve problems where you have several equations with remainders. If you have different clocks (or moduli) that do not share common factors (they are pairwise coprime), CRT lets you figure out a single number that fits all these clocks simultaneously. For example, imagine you have two clocks: one that ticks every three hours and another that ticks every five hours. If you know a number has specific remainders when divided by 3 and 5, CRT can help you find out exactly what that number is.

This theorem is handy in various fields, such as the following:

• Cryptography: It helps in making encryption and decryption faster.

• Computer science: It is used in algorithms and computations involving large numbers.

• Algebra: It aids in solving equations and understanding number patterns.

In short, CRT simplifies complex problems involving multiple modular conditions, making it easier to work with in different applications.

Example 12.1: You have a set of equations like this:

Where each mi is a different modulus, and they are all pairwise coprime (meaning no two of them share any common factors except 1).

Solution: The steps to find the solution are as follows:

1. Calculate the product of moduli: Multiply all the moduli together to get MMM.

For example, if your moduli are 3, 4, and 5, then M=3×4×5=60.

2. Compute partial products: For each modulus mi, calculate Mi by dividing M by mi.

So, for m1=3, M1=60/3=20; for m2=4, M2=60/4=15; for m3=5, m3 =60/5=12.

3. Find modular inverses: For each Mi, find a number Ni such that when Mi is multiplied by Ni, the result is 1 modulo mi. This number Ni is the modular inverse.

For and M1=20 and m1=3, you find N1 such that 20×N1≡1 (mod 3). The inverse N1 is 2 because 20×2 ≡1 (mod 3) or  20×2 (mod 3) = 1.

4. Calculate the solution: Combine everything using the following formula:

For each congruence, multiply the remainder ai by its partial product Mi and its modular inverse Ni, then sum these values and take the result modulo M.

For our example, the calculation would be as follows:

x=(2×20×2)+(3×15×3)+(2×12×3) (mod 60)

x=(80)+(135)+(72) (mod 60)

x=287 (mod 60)=47

So, the unique solution that fits all the given congruences is as follows:

x=47 modulo 60

Program 12.9: Implementation in C

#include <stdio.h>

// Function to compute the modular inverse of a modulo m

int modInverse(int a, int m) {

int m0 = m, t, q;

int x0 = 0, x1 = 1;

if (m == 1) return 0;

while (a > 1) {

q = a / m;

t = m;

// m is remainder now, process same as Euclid's algo

m = a % m, a = t;

t = x0;

// Update x0 and x1

x0 = x1 - q * x0;

x1 = t;

}

// Make x1 positive

if (x1 < 0) x1 += m0;

return x1;

}

// Function to solve the system of congruences using CRT

int chineseRemainderTheorem(int num[], int rem[], int k) {

int prod = 1;

for (int i = 0; i < k; i++)

prod *= num[i];

int result = 0;

for (int i = 0; i < k; i++) {

int pp = prod / num[i];

result += rem[i] * modInverse(pp, num[i]) * pp;

}

return result % prod;

}

// Main function to test the CRT function

int main() {

int num[] = {3, 4, 5}; // Moduli

int rem[] = {2, 3, 2}; // Remainders

int k = sizeof(num) / sizeof(num[0]);

int result = chineseRemainderTheorem(num, rem, k);

printf("x = %d\n", result);

return 0;

}

The explanation is as follows:

• modInverse: This function finds a special number called the modular inverse. Imagine you need to find a number that, when multiplied by another specific number, gives you a remainder of 1 when divided by a given modulus. The extended Euclidean algorithm is a method used to find this modular inverse efficiently.

• chineseRemainderTheorem: This function is where the magic happens. It takes a set of equations with remainders and moduli and finds a single number that satisfies all those equations at once. It uses the CRT algorithm, which combines the remainders and moduli in a clever way to find the unique solution.

• main: This part of the code is like a test run. It uses the CRT function with a specific set of moduli and remainders to demonstrate how the algorithm works. It is like checking if everything is functioning correctly by giving it some example problems to solve.

RSA public-key cryptosystem

RSA is a method of encrypting and decrypting messages so that only the intended recipient can read them. It was invented by three mathematicians, Ron Rivest, Adi Shamir, and Leonard Adleman, in 1977. RSA relies on the mathematical properties of large prime numbers to create secure keys.

The working of RSA involves the following steps:

1. Key generation: For a key to be generated, the following steps must be completed:

a. Choose two large prime numbers: The security of RSA comes from the difficulty of factoring large numbers, so two large prime numbers (let us call them p and q) are chosen.

b. Calculate their product: Multiply these primes to get n=p×q. This number n is part of both the public and private keys.

c. Compute Euler’s Totient function: Calculate ϕ(n)=(p-1)×(q-1). This value is used to create the keys.

d. Generate public key: Choose a number e (the public exponent) that is coprime with ϕ(n) (typically, e=65537 is used).

e. Generate private key: Calculate d (the private exponent) such that e×d≡1 (mod ϕ(n)). The number d is kept secret.

2. Encryption: The sender uses the recipient’s public key (n and e) to encrypt the message. If m is the message, the encrypted message c is calculated as c=me (mod n).

3. Decryption: The recipient uses their private key (n and d) to decrypt the message. The original message m is recovered by calculating m=cd (mod n).

The applications of RSA are as follows:

• Secure data transmission: Ensures that only the intended recipient can read the message.

• Digital signatures: Verifies the authenticity and integrity of a message or document.

The security of RSA is based on the fact that while it is easy to multiply two large primes, it is extremely difficult to reverse the process (i.e., factor the product back into the original primes). This difficulty ensures that even if someone has the public key, they cannot easily determine the private key. Thus, RSA is a cornerstone of modern encryption, allowing secure data exchange and authentication across the internet.

Consider the following example:

• Choose p = 3 and q = 11

• Compute n = p * q = 3 * 11 = 33

• Compute φ(n) = (p - 1) * (q - 1) = 2 * 10 = 20

• Choose e such that 1 < e < φ(n) and e and φ (n) are coprime. Let e = 7

• Compute a value for d such that (d * e) % φ(n) = 1. One solution is d = 3 [(3 * 7) % 20 = 1]

• Public key is (e, n) => (7, 33)

• Private key is (d, n) => (3, 33)

• The encryption of m = 2 is c = 27 % 33 = 29

• The decryption of c = 29 is m = 293 % 33 = 2

Program 12.10: Implementation in C

#include <stdio.h>

#include <math.h>

// Function to compute (base^exp) % mod using modular exponentiation

long long modExp(long long base, long long exp, long long mod) {

long long result = 1;

base = base % mod;

while (exp > 0) {

if (exp % 2 == 1) // If exp is odd

result = (result * base) % mod;

exp = exp >> 1; // Divide exp by 2

base = (base * base) % mod;

}

return result;

}

// Function to compute gcd of a and b

long long gcd(long long a, long long b) {

if (b == 0)

return a;

return gcd(b, a % b);

}

// Function to find modular inverse of a with respect to mod

long long modInverse(long long a, long long mod) {

long long m0 = mod, t, q;

long long x0 = 0, x1 = 1;

if (mod == 1)

return 0;

while (a > 1) {

q = a / mod;

t = mod;

mod = a % mod;

a = t;

t = x0;

x0 = x1 - q * x0;

x1 = t;

}

if (x1 < 0)

x1 += m0;

return x1;

}

int main() {

long long p = 61, q = 53;

long long n = p * q;

long long phi = (p - 1) * (q - 1);

long long e = 17;

long long d = modInverse(e, phi);

printf("Public Key: (%lld, %lld)\n", e, n);

printf("Private Key: (%lld, %lld)\n", d, n);

long long message = 65;

long long encrypted = modExp(message, e, n);

long long decrypted = modExp(encrypted, d, n);

printf("Original Message: %lld\n", message);

printf("Encrypted Message: %lld\n", encrypted);

printf("Decrypted Message: %lld\n", decrypted);

return 0;

}

Complexity analysis

The complexity analysis is as follows:

• Key generation: O(k4) for finding primes and O(k3) for computing d.

• Encryption: O(k3)

• Decryption: O(k3)

3-CNF satisfiability

Imagine you have a bunch of statements, and you want to figure out if there is a way to make all of them true at the same time by deciding whether certain things are true or false. In this specific problem, each statement is a combination of three smaller parts. These smaller parts are either a variable (which can be true or false) or the opposite of a variable (if the variable is true, this part is false, and vice versa).

Structure of 3-CNF

The statements are connected in a way that is called 3-Conjunctive Normal Form (3-CNF). We can define the statement in two different forms, which are as follows:

• Conjunctive: All the statements are combined using AND, meaning they all need to be true for the whole thing to be true.

• Normal form: Each statement is a combination of three parts using OR, meaning if at least one part of a statement is true, the whole statement is true.

Let us look at an example formula, shown as follows:

(x1∨¬x2∨x3)∧(¬x1∨x2∨x4)∧(x2∨¬x3∨¬x4)

This means the following:

• Either x1 is true, or x2 is false, or x3 is true (or any combination of these).

• Either x1 is false, or x2 is true, or x4 is true.

• Either x2 is true, or x3 is false, or x4 is false.

Your goal is to find a way to assign true or false values to x1, x2, x3, x4, so that all these statements are true simultaneously.

The challenge is to see if there is any way to make all the statements true by carefully picking true or false for each variable. If you can do that, the formula is satisfiable. If not, it is not satisfiable.

Algorithm 12.8: 3-CNF Satisfiability

The 3-CNF Satisfiability problem is what is known as NP-complete. This term means that we do not have a quick and easy way to solve every possible version of this problem. There is no known method that works efficiently for all cases. However, there are several ways we can try to tackle it as follows:

• Brute force: This method involves trying every possible way to assign true or false to the variables and seeing if the formula ends up being true. It is like checking every possible combination until you find one that works. But it takes a lot of time because the number of combinations grows very fast as the number of variables increases.

• Backtracking: This is a smarter version of brute force. You start assigning truth values, and if you hit a dead end where the formula cannot be true, you go back and try a different path. It is more efficient than pure brute force because it avoids many unnecessary checks.

• Heuristic methods: These are clever shortcuts. Instead of trying all combinations, these methods make educated guesses to find a solution. Techniques like local search, simulated annealing, or genetic algorithms help in finding a satisfying assignment more quickly, although they might not always find the best solution.

• Satisfiability (SAT) solvers: These are advanced tools specifically designed to solve SAT problems like 3-CNF. They use complex methods like the Davis–Putnam–Logemann–Loveland (DPLL) algorithm (which systematically tries and backtracks on assignments) and Conflict-Driven Clause Learning (CDCL), which learns from conflicts to avoid similar mistakes. These solvers are highly optimized and can handle large and complex problems efficiently.

Complexity analysis

The complexity analysis is calculated based on the approaches, shown as follows:

• Brute force approach: The most straightforward way to solve the 3-SAT problem is by trying every possible combination of true or false assignments for the variables as follows:

o Number of combinations: If you have n variables, there are 2n possible ways to assign true or false values to these variables.

o Checking each combination: For each combination, you need to check if the whole formula becomes true, which can be done relatively quickly (in polynomial time) for each combination.

o Time complexity: Since you have to check 2n combinations, this method has exponential time complexity, O(2n). This means the time it takes to solve the problem grows very quickly as the number of variables increases.

• Backtracking: Backtracking is a smarter version of brute force. Instead of trying all combinations blindly, you start assigning values and backtrack (go back and try different values) when you find a conflict. The time complexity is as follows:

o Efficiency: While backtracking can still take an exponential amount of time in the worst case, it often works much faster in practice because it avoids many unnecessary checks.

• Heuristic methods and SAT solvers: These are advanced techniques designed to solve specific instances more efficiently, and work as follows:

o Heuristic methods: These methods make educated guesses to find a solution faster. They do not guarantee a quick solution for every problem, but often work well in practice.

o SAT solvers: These tools use sophisticated techniques like the DPLL algorithm and CDCL to solve SAT problems much faster than brute force in many cases. However, they can still take exponential time in the worst case.

The clique problem

A clique is a group of people (vertices) where every pair of people is friends (there is an edge between every pair). Imagine you have a network of friends (an undirected graph G) and you want to find out if there is a group of k friends (a clique) where everyone in the group knows each other.

The working of the clique problem is as follows:

1. Input: The input consists of the following values:

• An undirected graph G = (V, E), where V represents the people and E represents the friendships.

• An integer k, the size of the group you are looking for.

2. Output: You need to determine if there is a group of k friends (a clique) in the graph.

For example, consider a network with five people and the following friendships:

{(1,2),(1,3),(1,4),(1,5),(2,3),(2,4),(3,4),(3,5),(4,5)}

This network forms a complete subgraph with all five people, which means there is a clique of size 5. If you want to find a group of four friends, there are several options, such as the group consisting of people {1, 2, 3, 4}.

The clique problem is NP-complete, meaning we do not have a quick method to solve every instance of this problem efficiently. It gets harder to solve as the network grows and as the value of k increases.

Exact algorithms

The following algorithms can be used to solve the clique problem:

• Brute force approach: Check all possible groups of k people to see if they form a clique. This method takes a lot of time and is not practical for large networks.

• Backtracking: This is a smarter brute-force approach that avoids unnecessary checks by pruning the search space based on certain criteria.

• Exact algorithms: Algorithms like Bron-Kerbosch can find cliques more efficiently than brute force.

Approximation algorithms

For large networks, finding an exact solution might be too time-consuming, so we use approximation methods, such as the following:

• Greedy algorithms: Build a clique by adding one person at a time based on some criteria.

• Local search: Use techniques like simulated annealing to find large cliques in practice, even if they might not be the largest possible.

Complexity analysis

The complexity analysis is as follows:

• Brute force approach: The efficiency of the brute force approach is as follows:

o Time complexity: The brute force method involves checking every possible group of k vertices to see if they form a clique. The number of possible groups (combinations) is given by , and for each group, we need to check if every pair of vertices is connected, which takes k2 time.

o Exponential growth: This means the total time complexity is . Since , grows exponentially with n (the number of vertices), this approach becomes impractical for large graphs.

• Backtracking and exact algorithms: The complexity analysis of backtracking and exact algorithms reveals the following:

o Efficiency in practice: These methods are designed to be more efficient than brute force by reducing the number of checks needed. They do this by pruning the search space, meaning they skip over parts of the graph that cannot possibly contain a clique of the desired size.

o Still exponential: Despite these improvements, backtracking and exact algorithms still face exponential growth in the worst case as the size of the graph and the desired clique size k increase.

The vertex problem

These problems focus on the vertices (nodes) of a graph and can vary widely.

Vertex cover problem

Imagine you have a network (graph) of cities (vertices) connected by roads (edges). You want to place police stations in a few cities so that every road has at least one police station at either end. The following is a brief explanation of the vertex cover problem:

• Goal: Find the smallest number of cities (vertices) where placing police stations ensures all roads are covered.

• Example: For a network with cities {1, 2, 3, 4} and roads {(1, 2), (1, 3), (2, 4), (3, 4)}, placing police stations in cities {1, 4} would cover all roads.

• Complexity: It is a tough problem (NP-complete), but there are ways to find good enough solutions quickly, like the 2-approximation algorithm, which determines a cover that is at most twice the size of the smallest possible.

Vertex coloring problem

Think of a network where each city needs a unique color, but neighboring cities (connected by a road) cannot share the same color. Refer to the following list for a brief overview of the vertex coloring problem:

• Goal: Use the fewest number of colors to color the cities.

• Example: For a triangle (three cities each connected to the other two), you will need three different colors.

• Complexity: This problem is generally hard (NP-hard), but for certain networks like bipartite graphs, it is easier and can be done with just two colors.

Dominating set problem

Picture a network where you want to place a few central hubs (vertices), such that every city either has a hub or is directly connected to a city with a hub. A summarized overview of the Dominating Set problem is given as follows:

• Goal: Find the smallest number of hubs needed.

• Example: In a network where each city is connected to its neighbors, placing a hub in a central city that connects to all others might work.

• Complexity: This problem is also very challenging (NP-complete), but there are practical methods to find good solutions.

Maximum independent set problem

Imagine you want to form the largest group of cities (vertices) where no two cities in the group are directly connected by a road. A brief description of the maximum independent set problem is as follows:

• Goal: Find the largest such group.

• Example: In a network where a group of cities all connect to each other (a clique), the largest group where no two cities are connected is just one city.

• Complexity: This is another hard problem (NP-complete). There are strategies for finding approximate solutions or solving specific types of networks efficiently.

Vertex connectivity problem

This problem looks at how connected a network is. Specifically, it asks how many cities (vertices) need to be removed to make the network fall apart (disconnect). A brief overview of the vertex connectivity problem is as follows:

• Goal: Determine the minimum number of cities that, when removed, break the network into separate parts.

• Example: In a fully connected network (complete graph), you would have to remove almost all cities to disconnect it.

• Complexity: Unlike the others, this problem can be solved more easily (in polynomial time) with algorithms like the max-flow min-cut theorem.

Conclusion

This chapter examined fundamental algorithms in sorting, computational geometry, graph theory, number theory, cryptography, and complexity theory. Randomized quicksort demonstrated efficient sorting, while Graham's scan and Jarvis's March tackled convex hull issues. The Fisher-Yates algorithm ensured unbiased permutations, whereas Karger's minimum cut technique elucidated graph partitioning. Euclid's algorithm and the Chinese remainder theorem were essential in number theory, while the traveling salesman approximation method addressed NP-hard optimization challenges. The RSA cryptosystem emphasized computational complexity, 3-CNF satisfiability, the clique problem, and secure encryption; RSA also highlighted security. Collectively, RSA, 3-CNF, and the clique problem provide a robust foundation for understanding algorithmic problem-solving across several domains.

Numerical questions

Let us look at some numerical questions based upon the concepts discussed in this chapter.

Key generation

1. Given two prime numbers p=61 and q=53, generate the RSA modulus n and the Euler’s totient function ϕ(n).

Answer:

• n=p×q

• ϕ(n)=(p−1)×(q−1)

So,

• n=61×53=3233

• ϕ(n)=(61−1)×(53−1)=60×52=3120

Public and private keys

2. For the given n=3233 and ϕ(n)=3120, choose a public key e such that 1<e<ϕ(n) and e is coprime with ϕ(n). Then, find the corresponding private key d such that d×e≡1 (mod ϕ(n)).

Answer:

• Let us choose e=17 (a common choice for simplicity).

• To find d, we need to solve 17d≡1 (mod 3120).

Using the extended Euclidean algorithm:

d=2753

Encryption

3. Given the public key (e, n) = (17, 3233), encrypt the plaintext message m=65.

Answer:

• Ciphertext c is calculated as c≡ me (mod n).

• So, c≡6517 (mod 3233).

Using modular exponentiation:

• c=2790

Decryption

4. Using the private key d=2753 and modulus n=3233, decrypt the ciphertext c=2790.

Answer:

• Plaintext mmm is calculated as m ≡ cd (mod n) m.

• So, m≡27902753 (mod 3233).

Using modular exponentiation:

• m=65

Verification

5. Verify that the decryption process works correctly by checking that the original plaintext m is recovered.

Answer:

• We previously encrypted m=65 to get c=2790.

• Then we decrypted c=2790 to get m=65.

• Since the original plaintext m=65 matches the decrypted plaintext, the encryption and decryption processes are verified to work correctly.

Finding e

6. For a given ϕ(n)=3120, find a suitable value for e.

Answer:

• e must be coprime with ϕ(n) and 1<e<ϕ(n).

• Common choices are small prime numbers like 3, 5, 17, or 65537.

• Let us verify e=65537.

Using the Euclidean algorithm to check gcd:

• gcd (65537,3120)=1, so e=65537 is suitable.

Exercise

1. Can randomized quick sort be implemented in a way that it is always in-place?

2. How does Graham’s scan algorithm find the convex hull of a set of points?

3. How can the probability of success in Karger’s algorithm be increased?

4. Can Euclid’s algorithm be extended to find GCD of more than two numbers?

5. Explain the process of reconstructing a number using Chinese Remainder Theorem given its remainders.

6. Why is the clique problem significant in graph theory?

APPENDIXMost Frequently Asked Questions

Introduction

Understanding the analysis of algorithms is fundamental to computer science and critical for developing efficient, high-performing software. As you delve into this intricate field, you will likely encounter numerous questions that may seem daunting. This chapter aims to address some of the most frequently asked questions to help clarify and reinforce key concepts. Whether you are a student beginning your journey in algorithms, a professional seeking to refresh your knowledge, or someone with a keen interest in computational theory, this chapter provides concise, clear answers to common queries. We will explore topics such as time complexity, space complexity, big O notation, algorithm optimization, and more. By the end of this chapter, you should have a better understanding of the principles that underpin algorithm analysis, enabling you to apply this knowledge to your coding practices and problem-solving techniques. This chapter provides a compilation of the most frequently asked questions related to various algorithms. It aims to reinforce key concepts, clarify doubts, and enhance understanding of algorithms in solving optimization problems. The questions cover fundamental principles, problem-solving approaches, and real-world use cases to help readers grasp the practical significance of branch and bound techniques.

Questions

Question 1: Let 𝑇(𝑛) be the recurrence relation defined as follows:

𝑇(0)=1,

𝑇(1)=2, and

𝑇(𝑛)=5𝑇(𝑛−1)−6𝑇(𝑛−2) for 𝑛≥2

Which one of the following statements is TRUE?

a. 𝑇(𝑛)=ϴ (2𝑛)

b. 𝑇(𝑛)= ϴ (𝑛2𝑛)

c. 𝑇(𝑛)= ϴ (3𝑛)

d. 𝑇(𝑛)= ϴ (𝑛3𝑛) [Gate 2024]

Question 2: Let f and g be functions of natural numbers given by f(n) = n and g(n) = n2. Which of the following statements is or are TRUE?

a. f ∈ O(g)

b. f ∈ Ω(g)

c. f ∈ o(g)

d. f ∈ Θ(g) [Gate 2023]

Question 3: Let G be a simple, finite, undirected graph with vertex set {v1, . . . , vn}. Let Δ(G) denote the maximum degree of G and let N = {1, 2, . . .} denote the set of all possible colors. Color the vertices of G using the following greedy strategy:

for i = 1, . . . , n

color(vi) ← min{j ∈ N : no neighbor of vi is colored j}

Which of the following statements is or are TRUE?

a. This procedure results in a proper vertex coloring of G.

b. The number of colors used is at most Δ(G) + 1.

c. The number of colors used is at most Δ(G).

d. The number of colors used is equal to the chromatic number of G. [Gate 2023]

Question 4: Which statement is not false about comparison-based sorting algorithms?

a. Comparison-based sorting algorithms take O(n(log(n)) time for random input arrays.

b. A comparison-based sorting algorithm can be stable by comparing two elements by location.

c. The counting sort method does not belong to comparison-based sorting.

d. The heap sort method does not belong to comparison-based sorting.

Question 5: What will be the recurrence relation of the code of recursive selection sort?

a. t(n) = 2t(n/2) + n

b. t(n) = 2t(n/2) + c

c. t(n) = t(n-1) + n

d. t(n) = t(n-1) + c

Question 6: The recursive program to raise an integer x to the power y uses which of the following algorithms?

a. Dynamic programming

b. Backtracking

c. Divide and conquer

d. Greedy algorithm

Question 7: Which of the following problems should be solved using dynamic programming?

a. Merge sort

b. Binary search

c. Longest common subsequence

d. Quick sort

Question 8: Suppose you have coins of denominations 1, 3, and 4. You use a greedy algorithm, in which you choose the largest denomination coin that is not greater than the remaining sum. For which of the following sums will the algorithm produce an optimal answer?

a. 10

b. 14

c. 100

d. 6

Question 9: Which of the following problems is equivalent to the 0/1 knapsack problem?

a. You are given a bag that can carry a maximum weight of w. You are given n items which have a weight of {w1, w2, w3,…., wn} and a value of {v1, v2, v3,…., vn}. You can break the items into smaller pieces. Choose the items in such a way that you get the maximum value.

b. You are studying for an exam and you have to study n questions. The questions take {t1, t2, t3,…., tn} time (in hours) and carry {m1, m2, m3,…., mn} marks. You can study for a maximum of t hours. You can either study a question or leave it. Choose the questions in such a way that your score is maximized.

c. You are given infinite coins of denominations {v1, v2, v3,….., vn} and a sum s. You have to find the minimum number of coins required to get the sum s.

d. You are given a suitcase that can carry a maximum weight of 15 kg. You are given four items which have a weight of {10, 20, 15,40} and a value of {1, 2, 3,4}. You can break the items into smaller pieces. Choose the items in such a way that you get the maximum value.

Question 10: What is the objective of the knapsack problem?

a. To get the maximum total value in the knapsack.

b. To get the minimum total value in the knapsack.

c. To get the maximum weight in the knapsack.

d. To get the minimum weight in the knapsack.

Question 11: Which of the following statements about 0/1 knapsack and fractional knapsack problem is correct?

a. In the 0/1 knapsack problem, items are divisible, and in the fractional knapsack, items are indivisible.

b. Both are the same.

c. 0/1 knapsack is solved using a greedy algorithm, and fractional knapsack is solved using dynamic programming.

d. In the 0/1 knapsack problem, items are indivisible, and in the fractional knapsack, items are divisible.

Question 12: Given G is a bipartite graph, and the bipartitions of this graph are U and V, respectively. What is the relation between them?

a. Number of vertices in u = number of vertices in v

b. Sum of degrees of vertices in u = sum of degrees of vertices in v

c. Number of vertices in u > number of vertices in v

d. Nothing can be said.

Question 13: Which algorithm is used to solve a maximum flow problem?

a. Prim’s algorithm

b. Kruskal’s algorithm

c. Dijkstra’s algorithm

d. Ford-Fulkerson algorithm

Question 14: Which of the following problems is not NP-complete?

a. Hamiltonian circuit

b. Bin packing

c. Partition problem

d. Halting problem

Question 15: Which of the following problems is similar to that of a Hamiltonian path problem?

a. Knapsack problem

b. Closest pair problem

c. Traveling salesman problem

d. Assignment problem

Question 16: What is the worst-case time complexity of the dynamic programming solution of the set partition problem (sum=sum of set elements)?

a. o(n)

b. o(sum)

c. o(n2)

d. o(sum*n)

Question 17: A networking company uses a compression technique to encode the message before transmitting it over the network. Suppose the message contains the following characters with their frequency:

Characters

Frequency

A

5

B

9

C

12

D

13

E

16

F

45

Each character in the input message takes 1 byte. If the compression technique used is Huffman coding, how many bits will be saved in the message?

a. 224

b. 576

c. 324

d. 328

Question 18: Which of the following is true about Huffman coding?

a. Huffman coding may become lossy in some cases.

b. Huffman codes may not be optimal lossless codes in some cases.

c. In Huffman coding, no code is a prefix of any other code.

d. All of the above

Question 19: A text is made up of the characters a, b, c, d, and e, each occurring with the probability 0.11, 0.40, 0.16, 0.09, and 0.24, respectively. The optimal Huffman coding technique will have an average length of:

a. 2.24

b. 2.16

c. 2.26

d. 2.15

Question 20: F1, F2, F3, F4, F5, and F6 are six files. They have 100, 200, 50, 80, 120, and 150 items, respectively. How should they be stored so that the act works best? Let us say that each file is read the same number of times.

a. F3, F4, F1, F5, F6, F2

b. F3, F4, F1, F5, F6, F2

c. F3, F4, F1, F5, F6, F2

d. All files can be accessed with the same frequencies.

Question 21: Which of the following is true about Kruskal and Prim minimum spanning tree (MST) algorithms?

Assume that Prim is implemented for adjacency list representation using binary heap and Kruskal is implemented using union by rank.

a. Worst-case time complexity of both algorithms is the same.

b. Worst-case time complexity of Kruskal is better than Prim.

c. Worst-case time complexity of Prim is better than Kruskal.

d. None of these

Question 22: Consider the string abbccddeee. Each letter in the string must be assigned a binary code satisfying the following properties:

• For any two letters, the code assigned to one letter must not be a prefix of the code assigned to the other letter.

• For any two letters of the same frequency, the letter that occurs earlier in the dictionary order is assigned a code whose length is at most the length of the code assigned to the other letter.

Among the set of all binary code assignments that satisfy the preceding two properties, what is the minimum length of the encoded string (refer to the following figure for the Huffman tree structure)?

Figure 13.1: Huffman tree structure

a. 29

b. 25

c. 23

d. 21

Question 23: Consider a job scheduling problem with four jobs, J1, J2, J3, and J4, and with corresponding deadlines, (d1, d2, d3, d4) = (4, 2, 4, 2). Which of the following is not a feasible schedule without violating any job schedule?

a. J2, J4, J1, J3

b. J4, J1, J2, J3

c. J4, J2, J1, J3

d. J24 J2, J3, J1

Question 24: Suppose the letters a, b, c, d, e, and f have probabilities 1/2, 1/4, 1/8, 1/16, 1/32, and 1/32, respectively. Which of the following is the Huffman code for the letter a, b, c, d, e, and f?

a. 0, 10, 110, 1110, 11110, 11111

b. 11, 10, 011, 010, 001, 000

c. 11, 10, 01, 001, 0001, 0000

d. 110, 100, 010, 000, 001, 111

Question 25: Which of the following standard algorithms is not a greedy algorithm?

a. Dijkstra's shortest path algorithm

b. Prim's algorithm

c. Kruskal algorithm

d. Bellman-Ford shortest path algorithm

Question 26: Consider two strings, A = "qpqrr" and B = "pqprqrp". Let x be the length of the longest common subsequence (not necessarily contiguous) between A and B, and let y be the number of such longest common sub-sequences between A and B. Then, x + 10y = ___.

a. 23

b. 33

c. 34

d. 43

Question 27: In the 0/1 knapsack problem, if an item's weight is greater than the remaining capacity of the knapsack, what action is typically taken?

a. Add the fractional part of the item that can fit into the knapsack.

b. Remove the least valuable item from the knapsack to make space.

c. Remove the most valuable item from the knapsack to make space.

d. Ignore the item and move to the next one.

Question 28: The worst-case efficiency of solving a problem in polynomial time is?

a. O(p(n))

b. O(p( n log n))

c. O(p(n2))

d. O(p(m log n))

Question 29: Problems that cannot be solved by any algorithm are called?

a. Tractable problems

b. Intractable problems

c. Undecidable problems

d. Decidable problems

Question 30: Which of the following problems is not NP-complete?

a. Hamiltonian circuit

b. Bin packing

c. Partition problem

d. Halting problem

Question 31: Consider a simple graph G with 18 vertices. What will be the size of the maximum independent set of G if the size of the minimum vertex cover of G is 10?

a. 18

b. 08

c. 10

d. 28

Question 32: The recursive solution of the Tower of Hanoi problem is an example of which of the following algorithms?

a. Dynamic programming

b. Backtracking

c. Greedy algorithm

d. Divide and conquer

Question 33: The fractional knapsack problem is solved most efficiently by which of the following algorithms?

a. Divide and conquer

b. Dynamic programming

c. Greedy algorithm

d. Backtracking

Answers

1. C

2. A, C

3. A, B

4. D

5. C

6. C

7. C

8. C

Explanation: Using the greedy algorithm, three coins {4,1,1} will be selected to make a sum of 6. However, the optimal answer is two coins {3,3}. Similarly, four coins {4,4,1,1} will be selected to make a sum of 10, but the optimal answer is three coins {4,3,3}. Also, five coins {4,4,4,1,1} will be selected to make a sum of 14, but the optimal answer is four coins {4,4,3,3}. For a sum of 100, 25 coins {all 4’s} will be selected, and the optimal answer is also 25 coins {all 4’s}.

9. B

10. A

11. D

Explanation: In the fractional knapsack problem, we can partially include an item into the knapsack, whereas in 0/1 knapsack, we have to either include or exclude the item wholly.

12. B

Explanation: We can prove this by induction. By adding one edge, the degree of vertices in u is equal to 1, as well as in the v vertex. Let us assume that this is true for n-1 edges and add one more edge. Since the given edge adds exactly once to both u and v, we can tell that this statement is true for all n vertices.

13. D

14. D

15. C

16. D

Explanation: The set partition problem has both recursive as well as dynamic programming solutions. The dynamic programming solution has a time complexity of o(n*sum) as it has a nested loop with limits from 1 to n and 1 to sum, respectively.

17. B

Explanation: Total number of characters in the message = 100.

Each character takes 1 byte. So, total number of bits needed = 800.

After Huffman coding, the characters can be represented as follows:

F: 0

C: 100

D: 101

A: 1100

B: 1101

E: 111

Total number of bits needed = 224

Hence, number of bits saved = 800 - 224 = 576

18. C

19. B

Explanation: a = 0.11 b = 0.40 c = 0.16 d = 0.09 e = 0.24

Huffman coding for character:

a = 1111

b = 0

c = 110

d = 1111

e = 10

Length for each character = number of bits * frequency of occurrence:

a = 4 * 0.11 = 0.44

b = 1 * 0.4 = 0.4

c = 3 * 0.16 = 0.48

d = 4 * 0.09 = 0.36

e = 2 * 0.24 = 0.48

Now, add these lengths for average length:

0.44 + 0.4 + 0.48 + 0.36 + 0.48 = 2.16

20. A

Explanation: This question is based on the optimal storage on tape problem, which uses the greedy approach to find the optimal time to retrieve programs from tape. There are n programs of length L that are to be stored on a computer tape. Associated with each program i is a length Li. So, in order to retrieve these programs most optimally, we need to store them in the non-decreasing order of length Li. So, the correct order is F3, F4, F1, F5, F6, F2.

21. A

22. C

Explanation:

Alphabets

Frequency

a

1

b

2

c

2

d

2

e

3

1×3 + 2×3 + 3×2 + 2×2 + 2×2 = 23

23. B

Explanation:

Option A: 2,2,4,4. No problem.

Option B: 2,4,2,4. The third job in this sequence had to be completed under 2-time units. But it took 3-time units. This is not an optimal solution, hence the answer. 

Option C: 2,2,4,4. No problem. 

Option D: 2,2,4,4. No problem.

24. A

Explanation: The idea is to keep the least probable characters as low as possible by picking them first.

The letters a, b, c, d, e, and f have the probabilities 1/2, 1/4, 1/8, 1/16, 1/32, and 1/32, respectively, as shown in the following Huffman tree after applying the Huffman coding algorithm:

Figure 13.2: Huffman tree structure after applying Huffman coding

25. D

26. C

Explanation: The LCS is of length 4. There are three LCS of length 4: qprr, pqrr, and qpqr. A subsequence is a sequence that can be derived from another sequence by selecting zero or more elements from it without changing the order of the remaining elements. The subsequence need not be contiguous. Since the lengths of the given strings, A = “qpqrr” and B = “pqprqrp”, are very small, we do not need to build a 5x7 matrix and solve it using dynamic programming. Rather, we can solve it manually just by brute force. We will first check whether there exists a subsequence of length 5, since min_length (A, B) = 5. Since there is no subsequence, we will now check for length 4. “qprr”, “pqrr”, and “qpqr” are common in both strings. X = 4 and Y = 3.

X + 10Y = 34

27. D

28. A

29. C

30. D

31. B

Explanation: Removing the vertex cover vertices from the graph results in an isolated graph, and so the remaining vertices would be the independent set in the original graph. Hence, the size of the maximum independent set = 18 – 10 = 8.

32. D

33. C

Conclusion

Reiterating the ideas of branch and bound algorithms is made possible in great part by the section on most often asked questions. It improves the reader's knowledge of optimization methods and their pragmatic uses by addressing typical questions and offering concise answers. This part guarantees a better understanding of problem-solving techniques, helps to strengthen important concepts, and gets readers ready to use branch and bound approaches in practical situations.

Join our Discord space

Join our Discord workspace for latest updates, offers, tech happenings around the world, new releases, and sessions with the authors:

https://discord.bpbonline.com

Index

Symbols

0/1 knapsack problem 210, 283-286

complexity 215, 290

implementation in C++ 212-215

implementation, using branch and bound 287

working 210, 211

3-CNF satisfiability 424

complexity analysis 426

structure 425, 426

A

activity selection problem 199, 200

applications of greedy algorithms 200

complexity 202

implementation 201

adjacency matrix representation 322

examining 322, 323

for weighted directed graph 323

for directed graph 323

algorithm 2

application 3

characteristics 4, 5

designing 5

examples 5, 6

for cooking food 3, 4

need for 4

algorithm complexity 6, 7

analysis of algorithms 8

space complexity 7, 8

time complexity 7

all-pairs shortest path algorithm 352

implementation in C++ 353

approximation algorithms 377, 378

use cases 379

array 23

2D array declaration 25, 26

address calculation, of array elements 24

applications 24

declaring 24

operations 27-34

storing 24

two sorted arrays, merging 35, 36

asymptotic notations 8

Big–OH (O) notation 9

Big–OMEGA (Ω) notation 9

THETA (θ) notation 10-13

auxiliary space 7

AVL tree 69, 74

insertion operation 74, 75

operations 75

rotations 75

B

backtracking 251, 252, 425

general process 253

Basic Linear Algebra Subprograms (BLAS) 156

Bellman-Ford algorithm 343

applications 352

complexity analysis 351

role of negative weight 344-351

Big–OH (O) notation 9

Big–OMEGA (Ω) notation 9

binary search algorithm 113-115, 140

applications 142, 143

complexity 117, 142

implementation 115

implementation in C++ 141

binary search tree (BST) 69-71

binomial heap 78, 79

bounded-error probabilistic polynomial time (BPP) 381

characteristics 381

error probability, reducing 382

examples 381

relationship 382

significance 382

branch and bound algorithm 282

classification of problems 283

using 282

brute force 425

B-tree 71

features 71-73

bubble sort algorithm 86, 87

complexity analysis 88, 89

implementation 87, 88

bucket sort algorithm 103, 104

complexity analysis 106, 107

implementing 105, 106

C

chinese remainder theorem (CRT) 419-421

implementation in C 420, 421

Christofides algorithm 408, 409

complexity analysis 413

implementation in C 410-413

circular doubly linked list 62, 63

deletion 64, 65

insertion 63, 64

clique problem 427

approximation algorithms 427, 428

complexity analysis 428

exact algorithms 427

working 427

closest pair of points 160

applications 165

complexity analysis 165

implementation in C++ 162-164

Complement of randomized polynomial time (Co-RP) 384

characteristics 385

error probability, reducing 385, 386

examples 385

relationship, with other complexity classes 385

completeness 376

examples 377

NP-completeness 376

P-completeness 376

PSPACE-completeness 376

computational complexity 367

basic concepts 369, 370

class NP 370

class P 370

key aspects 368, 369

Conflict-Driven Clause Learning (CDCL) 426

constraint satisfaction problem 266

complexity 269

implementation 267

using with cryptarithmetic problem 266, 267

counting sort algorithm 97

complexity analysis 99, 100

implementing 98, 99

crossword puzzle 274

complexity 278

implementation 275

solving 274, 275

D

data structures 21, 22

purpose 22, 23

Davis–Putnam–Logemann–Loveland (DPLL) algorithm 426

deque 48

Dijkstra’s algorithm 338

implementation in C++ 338-343

directed acyclic graph (DAG) 325

directed graph 320

divide and conquer method 137

general method 138-140

division method 66

doubly linked list (DLL) 58, 59

deletion 61, 62

insertion 59-61

dynamic programming (DP) 207-210

E

Euclid's algorithm for GCD 413, 414

complexity analysis 415

implementation in C 415

Euler circuit 335

Eulerian path 336

Euler path 335

exponential search algorithm 125

complexity 127, 128

implementation 126

phases 125, 126

Exponential time (EXP) 375

complexity theories and problems 375

F

Fibonacci heap 79

properties 79, 80

Fibonacci search 128-130

complexity 132

implementation 130

find maximum and minimum in number sequence 143

applications 145

complexity analysis 144

implementation in C++ 143, 144

findMST(graph) 409

findOddDegreeVertices(MST) 409

first-in, first-out (FIFO) queue 43

Fisher-Yates shuffle 405, 406

complexity analysis 408

implementation in C 406, 407

flow network 359

Floyd-Warshall algorithm 352-354

applications 359

applying 355-358

complexity analysis 358

folding technique 67, 68

Ford-Fulkerson algorithm 359

applications 364

complexity 364

fractional knapsack problem 176

complexity 179

package selection 176, 177

procedure 177, 178

working 176

G

geometric progression (GP) 18

gift wrapping algorithm 396

Graham's scan algorithm 391, 392

complexity analysis 395, 396

implementation in C 393-395

graph 320

directed graph 320

representation 320

undirected graph 320

graph coloring problem 261-263

complexity 265

implementation 263

graph representation 322

linked list representation 324

sequential representation 322

graph terminology

adjacent nodes 322

closed path 321

connected graph 321

cycle 321

degree of node 322

digraph 321

loop 321

path 321

simple path 321

weighted graph 321

greedy algorithms 174

characteristics 175

general method 175

H

Hamiltonian cycle 271, 272

complexity 274

implementation 273

hash function 65

hashing 65

hash table 66

heap 69, 77

max heap 77

min heap 78

operations 78

heuristic methods 425

Huffman coding algorithm 188-192

complexity 192, 193

I

insertion sort algorithm 91-93

complexity analysis 93, 94

interpolation search algorithm 118, 119

complexity 120, 121

implementation 119

interval scheduling 197, 198

complexity 199

implementation 198

inversion count 168

iterative method 16

J

Jarvis’s March algorithm 396, 397

complexity analysis 399, 400

implementation in C 397-399

job assignment problem 305

job scheduling problem 202

complexity 204

implementation 203

job sequencing 313, 314

complexity 317

implementation 315

jump search algorithm 121, 122

advantages, over binary search 123

advantages, over linear search 122

complexity 124, 125

implementation 123

optimization 123

step length, selecting 123

K

Karatsuba multiplication 157

applications 160

complexity analysis 160

divide and conquer, using 157

implementation in C++ 158, 159

Karger's minimum cut algorithm 400, 401

complexity analysis 405

implementing in C 401-404

Knuth shuffle 405

Kruskal's algorithm 179, 180

complexity 184

for MST 181

implementation for MST 182

minimum cost spanning with 180, 181

L

last-in-first-out (LIFO) data structure 38

Las Vegas algorithms 379

Levenshtein distance 219, 220

complexity analysis 223, 224

implementation in C++ 220-222

Linear Algebra Package (LAPACK ) 156

linear search algorithm 111, 112

implementation 112

linked list 53

circular doubly linked list 62, 63

doubly linked list (DLL) 58, 59

singly linked list (SLL) 53

linked list representation 324

longest common subsequence problem 237

dynamic programming 238-243

M

matrix chain multiplication 227, 228

computing optimal costs 237

dynamic programming approach 229-236

matrices, parenthesizing 228, 229

max-flow min-cut theorem 359-364

max heap 77

maximum subarray sum 165

applications 168

complexity analysis 167

implementation in C++ 166, 167

merge-based inversion count 168

applications 172

complexity analysis 171

implementation in C++ 169, 171

merge sort algorithm 145

applications 148

complexity analysis 148

implementation in C++ 146-148

mid-square method 67

min heap 78

minimum spanning tree (MST) 179

modAdd 418

modDiv 418

modInverse 421

modMul 418

modSub 418

Modulo arithmetic 415, 416

applications 416

basic concepts 416

complexity analysis 418

implementation in C 417, 418

properties 416

Monte Carlo algorithm 379

multiplication method 66

N

Nondeterministic polynomial time (NP) 370

characteristics 370, 371

NP-complete 371

proof technique 373

NP-hard

proof technique 372

N–Queens problem 253

complexity 257

final state 254

implementation 255

initial state 254

O

optimal binary search tree 224-227

optimal merge pattern algorithm 193-195

complexity 196, 197

implementation 195

P

P-hard 371

pointer 49

key concepts 50

using 50-52

Polynomial space (PSPACE) 374

complexity theories and problems 374, 375

Polynomial time (P) 370

characteristics 370

Prim’s algorithm 185

advantages 185

complexity 188

for MST 185

implementation for MST 186

principle of relaxation 344

probabilistic algorithms 379

benefits 380

examples 379, 380

Las Vegas algorithms 379

limitations 380

Monte Carlo algorithms 379

program analysis

general rules 13

Q

quadratic assignment problem (QAP) 309-311

complexity 313

implementation 311

queue 43, 44

circular queue 47, 48

deque 48

multiple queue 49

operations 44-47

priority queue 48, 49

types 47

quick sort algorithm 149

applications 151

complexity analysis 151

implementation in C++ 149, 150

R

radix sort algorithm 100, 101

complexity analysis 103

implementing 101-103

randomized algorithms 379

randomized polynomial time (RP) 382

characteristics 383

error probability, reducing 384

examples 383

relationship, with classes 383

randomized quick sort 388

complexity analysis 391

implementation in C 389, 390

recurrence relation 13

iterative method 16, 17

master's method 18, 19

recurrence 14

recursion tree method 17, 18

substitution method 14-16

red-black (RB) tree 76, 77

reductions 376

examples 377

many-one reduction (Karp reduction) 376

Turing reduction 376

resource allocation problem 304-307

complexity 309

implementation 307

RSA public-key cryptosystem 421

applications 422

complexity analysis 424

implementation in C 423

working 421, 422

S

Satisfiability (SAT) solvers 426

searching 110

purposes 110, 111

selection sort algorithm 89

complexity analysis 91

implementation 89-91

sequential representation 322

sequential search 111

shell sort algorithm 94

complexity analysis 96

gap sequence 94

implementation 95

single-source shortest path problem (SSSP) 336

applications 343

complexity 343

Dijkstra’s algorithm, using 338

relaxation property 336, 337

singly linked list (SLL) 53

deletion 57, 58

element, searching 54

insertion 54-56

traversing 54

sorting 83, 84

adaptive sorting 85

comparison-based sorting 84

in-place sorting 85

non-comparison-based sorting 84

scenarios 84

stable sorting 85

time and space complexity 85, 86

unstable sorting 85

stack 38

applications 38, 39

stack operations 39

linked list representation 43

peek operation 41

pop operation 40

push operation 39

Strassen’s matrix multiplication 151, 152

applications 156

complexity analysis 156

implementation 152-156

strongly connected components (SCCs) 329, 330

applications 335

complexity analysis 335

implementation in C++ 331-335

sublist search algorithm 132, 133

complexity 134

implementation 133

subset sum problem 215

implementation in C++ 216-219

substitution method 14

Sudoku solver 257

complexity 261

final state 258

initial state 257, 258

sum of subset problem 269, 270

complexity 271

implementation 270

T

THETA (θ) notation 10-12

topological sort 325

applications 329

complexity analysis 329

implementation in C++ 325-328

traveling salesperson problem (TSP) 243, 290-301

comparison 248, 249

complexity 248, 304

dynamic programming solution 243-248

implementation 301

travelling salesman approximation algorithm 408

trees 68

application 69

AVL tree 69

binary search tree (BST) 69

binary tree 68

heap 69

U

undirected graph 320, 321

V

vertex problem 428

dominating set problem 429

maximum independent set problem 429

vertex coloring problem 429

vertex connectivity problem 430

vertex cover problem 428