Methods and models in mathematical programming 9783030270445, 9783030270452

Citation preview

S. A. MirHassani · F. Hooshmand

Methods and Models in Mathematical Programming

Methods and Models in Mathematical Programming

S. A. MirHassani • F. Hooshmand

Methods and Models in Mathematical Programming

S. A. MirHassani Department of Mathematics and Computer Science Amirkabir University of Technology (Tehran Polytechnic) Tehran, Iran

F. Hooshmand Department of Mathematics and Computer Science Amirkabir University of Technology (Tehran Polytechnic) Tehran, Iran

ISBN 978-3-030-27044-5 ISBN 978-3-030-27045-2 https://doi.org/10.1007/978-3-030-27045-2

(eBook)

© Springer Nature Switzerland AG 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG. The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Abbreviations and Acronyms

The following table summarizes all abbreviations and acronyms, used throughout this book, and the reader can turn to this list at any time. Abbreviation B&B B&C BIP CLP CSP CTP DCP FSSP IFP IIS IP IPM KKT LHS LP LPR (mu) MCER MILP MINLP MIP MIS MPDP NDP NFP NLP

Definition Branch-and-bound Branch-and-cut Binary integer programming Constraint logic programming Cutting stock problem Course timetabling problem Disease control problem Flow shop scheduling problem Inner fit polygon Irreducible infeasible subsystem Integer programming Interior point method Karush Kuhn Tucker Left hand side Linear programming Linear programming relaxation Monetary unit McCormick envelope relaxation Mixed integer linear programming Mixed integer nonlinear programming Mixed integer programming Minimal infeasible subsystem Multi-parametric disaggregation technique Network design problem No fit polygon Nonlinear programming (continued) v

vi Abbreviation OR PDIP PIP PLF PPSP PSP QP RHS RLT SHP SNP SOS1 SOS2 SSP TSP VRP VRPTW WRMP

Abbreviations and Acronyms Definition Operations research Pre-disaster investment problem Pure integer programming Piecewise linear function Petroleum pipeline scheduling problem Portfolio selection problem Quadratic programming Right-hand side Reformulation and linearization technique Sell or hold problem Social network problem Special ordered set of type 1 Special ordered set of type 2 Sports scheduling problem Traveling salesman problem Vehicle routing problem Vehicle routing problem with time window Water resource management problem

Preface

Operations research (OR) is a branch of mathematical sciences that aims to utilize mathematical modeling, statistical analysis, and mathematical optimization systematically to identify the best choice among available options. The source of optimization problems is the need for industrial, economic, and social decision-makers, who are looking for the best possible way to solve the problems in their area of expertise. In most cases, the number of options discussed is exceptionally high, so that it is not possible to express all of them explicitly. Therefore, these problems are initially described by managers (problem description). Then, in order to identify acceptable alternatives, possible choices are implicitly introduced in the form of mathematical relations, so that any solution satisfying these relations is a valid choice. In this way, mathematical models are formulated as a combination of variables and parameters in the form of mathematical equalities and inequalities (model construction). Finally, methods of solving mathematical models are utilized to determine the optimal choice from a range of possible options (model solving). Tracking and replicating this process in an interactive and dynamic environment can lead to the desired outcome, i.e., the selection of the best option. Today, with the efforts of the scientific community, efficient algorithms have been developed and embedded in mathematical software to satisfy the need of OR practitioners. These software can receive the mathematical models, compile the collected information in a specific form, and call an appropriate solver. Then, the model is solved via mathematical algorithms, and the result is returned to the user. In this way, the user will be able to design, formulate, and then solve the problem by himself or herself without dealing with the mathematical complexity of algorithms. The success in this area is reliant on the construction of an accurate and appropriate model that not only observes the problem conditions but also considers the limitations of algorithms as an effective factor. The initial description of a practical problem is often based on the expression of a collection of logical statements that should be used in the form of mathematical equations and inequalities. Although describing a problem in the form of a mathematical model is an essential step in the real-world application of optimization, little attention has been paid in the literature to the challenges and delicacies of the vii

viii

Preface

modeling process. This textbook focuses on mathematical modeling and introduces its principles and inherent weaknesses. It attempts to describe the process of constructing and evaluating models and explicitly outlines the required rules and regulations so that the reader is able to generalize and reuse concepts in other problems by relying on mathematical logic. This textbook is organized into six chapters: General principles of modeling, different types of models and their importance, the main steps of modeling, and the general structure of an optimization model are described in Chap. 1. Chapter 2 introduces the main components of an optimization model, provides an accurate classification of models’ types, and presents a short review of solution methods. Chapter 3 expresses how to incorporate logical propositions as linear constraints into optimization models. Chapter 4 refers to some nonlinear functions frequently appearing in optimization problems (e.g., multiplication of two or more variables, piecewise linear and separable nonlinear functions, minimax and maximin functions, absolute value and fractional functions) and explains some techniques for their linearization. In the cases in which linearization is not possible, some approximation techniques are introduced. Chapter 5 addresses the qualitative comparison of models, discusses the impact of the number of constraints and variables on the quality of models, and introduces the ideal formulation. Furthermore, some techniques of improving formulations are presented, and at the end of this chapter, constraint logic programming is introduced. Chapter 6 develops a set of real-world applications of mathematical programming, which are closely related to the methodology presented in the previous chapters. In each case, the problem is defined, the model’s components are introduced, and then, the model is presented in the general form. Afterward, the model is solved for a given set of data with the aid of a computer, and the results are analyzed. Each chapter contains different exercises; some are relatively simple, and others are more challenging. Undergraduate and postgraduate students of different academic disciplines, including applied mathematics, computer science, operations research, industrial engineering, and management science would find this textbook a suitable option preparing them for jobs and researches requiring modeling techniques. Furthermore, this textbook can be used as a reference for experts requiring advanced skills of model building in their jobs. The first version of this book was published in Persian by Amirkabir University of Technology Press. Here, it is the second version written in English. This book is the result of several years of teaching, and ideas brought forward by readers will be appreciated for improving possible shortcomings in the future. Tehran, Iran

S. A. MirHassani F. Hooshmand

Contents

1

Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Model and Its Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Value of Mathematical Models . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Steps of Model Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4.1 Step 1: Identify and Define the Problem . . . . . . . . . . . 1.4.2 Step 2: Collect Data and Present the Model . . . . . . . . . 1.4.3 Step 3: Solve the Model . . . . . . . . . . . . . . . . . . . . . . . 1.4.4 Step 4: Validate the Model . . . . . . . . . . . . . . . . . . . . . 1.4.5 Step 5: Provide Results to the Organization . . . . . . . . . 1.5 Structure of a Mathematical Model . . . . . . . . . . . . . . . . . . . . . 1.5.1 Decision Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.2 Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.3 Objective Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.4 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.5 Solution Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.6 Model Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Notes and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

1 1 2 3 4 4 4 4 5 5 5 6 7 7 8 8 9 10 10

2

Main Components of Mathematical Models . . . . . . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Free Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Finite Domain Variables . . . . . . . . . . . . . . . . . . . . . . 2.2.3 Semi-continuous Variables . . . . . . . . . . . . . . . . . . . . . 2.2.4 Partial Integer Variable . . . . . . . . . . . . . . . . . . . . . . . 2.3 Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Simple Bound and General Bound . . . . . . . . . . . . . . . 2.3.2 Soft and Hard Constraints . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

13 13 13 13 14 15 15 16 16 17 ix

x

Contents

2.3.3 Redundant and Binding Constraints . . . . . . . . . . . . . . 2.3.4 Cuts and Lazy Constraints . . . . . . . . . . . . . . . . . . . . . 2.3.5 Constraints and Special Ordered Sets . . . . . . . . . . . . . 2.3.6 Chance Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.7 Converting Constraints . . . . . . . . . . . . . . . . . . . . . . . Objective Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mathematical Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Continuous and Discrete Models . . . . . . . . . . . . . . . . 2.5.2 Linear and Nonlinear Models . . . . . . . . . . . . . . . . . . . 2.5.3 Single and Multi-Objective Models . . . . . . . . . . . . . . . 2.5.4 Constrained and Unconstrained Models . . . . . . . . . . . 2.5.5 Static and Dynamic Models . . . . . . . . . . . . . . . . . . . . 2.5.6 Deterministic and Non-deterministic Models . . . . . . . . 2.5.7 LP Models with Interval Coefficients . . . . . . . . . . . . . 2.5.8 Single and Multi-stage Models . . . . . . . . . . . . . . . . . . 2.5.9 Single and Multi-level Models . . . . . . . . . . . . . . . . . . Solution Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.1 Solving LP Problems . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.2 Solving MIP Problems . . . . . . . . . . . . . . . . . . . . . . . . 2.6.3 Solving NLP Problems . . . . . . . . . . . . . . . . . . . . . . . 2.6.4 Solving Multi-objective Problems . . . . . . . . . . . . . . . . 2.6.5 Dynamic Programming to Solve Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.6 Solving Bi-level Problems . . . . . . . . . . . . . . . . . . . . . 2.6.7 Solving Problems with Interval Coefficients . . . . . . . . Notes and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

18 19 21 23 24 25 26 26 26 30 31 31 32 32 34 36 41 41 43 44 50

. . . . .

52 52 55 57 57

Models and Mathematical Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Atomic Propositions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Composite Propositions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Propositional Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Propositions and Binary Variables . . . . . . . . . . . . . . . . . . . . . . 3.6 Constraints and Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Conditional Indicators . . . . . . . . . . . . . . . . . . . . . . . . 3.6.2 Complementarity Rule . . . . . . . . . . . . . . . . . . . . . . . . 3.6.3 Bi-Conditional Indicators . . . . . . . . . . . . . . . . . . . . . . 3.6.4 Either-or Constraints . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.5 Constraint Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.6 Not-Equality Constraints . . . . . . . . . . . . . . . . . . . . . . 3.6.7 If-Then Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.8 If-Then-Else Constraints . . . . . . . . . . . . . . . . . . . . . . 3.7 Miscellaneous Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Notes and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

67 67 68 68 70 70 74 75 79 80 88 93 94 96 100 101 105 105

2.4 2.5

2.6

2.7 2.8 3

Contents

xi

4

Linearization of Nonlinear Functions . . . . . . . . . . . . . . . . . . . . . . . 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Variables Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Multiplication of Binary Variables . . . . . . . . . . . . . . . 4.2.2 Multiplying Binary and Continuous Variables . . . . . . . 4.2.3 Multiplying Binary and Integer Variables . . . . . . . . . . 4.2.4 Converting Integer Variable to Binary . . . . . . . . . . . . . 4.2.5 Multiplying Two Integer Variables . . . . . . . . . . . . . . . 4.2.6 Multiplying Integer and Continuous Variables . . . . . . . 4.2.7 Multiplying Two Continuous Variables . . . . . . . . . . . . 4.2.8 Approximation by McCormick Relaxation . . . . . . . . . 4.2.9 The Multi-parametric Disaggregation Technique . . . . . 4.2.10 Approximation by PLFs . . . . . . . . . . . . . . . . . . . . . . . 4.3 Piecewise Linear Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 The First Convex Combination Method . . . . . . . . . . . 4.3.2 The Second Convex Combination Method . . . . . . . . . 4.3.3 The Incremental Method . . . . . . . . . . . . . . . . . . . . . . 4.3.4 Multiple Choice Method . . . . . . . . . . . . . . . . . . . . . . 4.3.5 Discontinuous PLF . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Separable Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Separable Models Transformations . . . . . . . . . . . . . . . 4.5 Maximum and Minimum Functions . . . . . . . . . . . . . . . . . . . . . 4.6 Maximin and Minimax Functions . . . . . . . . . . . . . . . . . . . . . . 4.7 Absolute Value Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7.1 Absolute Value Function in Constraints . . . . . . . . . . . 4.7.2 Absolute Value Function in Objective . . . . . . . . . . . . . 4.7.3 Vectors Comparison . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Fractional Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8.1 Fractional Functions in Constraints . . . . . . . . . . . . . . . 4.8.2 Fractional Objective Function . . . . . . . . . . . . . . . . . . . 4.8.3 Minimax Fractional Objective Function . . . . . . . . . . . 4.8.4 Problems Involving Mean Value Function . . . . . . . . . . 4.9 Knapsack Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Notes and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

115 115 116 116 118 119 119 120 120 121 121 128 135 138 140 145 146 149 150 153 159 162 164 167 167 172 176 177 178 178 183 184 185 192 192

5

Qualitative Comparison of Models . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Model’s Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Comparison of Models Based on LPR . . . . . . . . . . . . . . . . . . . 5.4 Ideal Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Totally Unimodular Models . . . . . . . . . . . . . . . . . . . . 5.4.2 Reformulation and Linearization Technique . . . . . . . . 5.5 Improving Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 Improvement by Cutting Planes . . . . . . . . . . . . . . . . . 5.5.2 Improve by Symmetry Breaking . . . . . . . . . . . . . . . . .

. . . . . . . . . .

205 205 205 207 217 218 222 239 239 242

xii

Contents

5.6 5.7

Irreducible Infeasible Subsystems . . . . . . . . . . . . . . . . . . . . . . Constraint Logic Programming . . . . . . . . . . . . . . . . . . . . . . . . 5.7.1 A Comparison Between CLP and IP . . . . . . . . . . . . . . 5.7.2 Expressing the Domain of Variables . . . . . . . . . . . . . . 5.7.3 Not-Equal Constraint (6¼) . . . . . . . . . . . . . . . . . . . . . . 5.7.4 The At-Least Constraint . . . . . . . . . . . . . . . . . . . . . . . 5.7.5 The Cardinality Rule Constraint . . . . . . . . . . . . . . . . . 5.7.6 All-Different Constraint . . . . . . . . . . . . . . . . . . . . . . . 5.7.7 Element Constraint . . . . . . . . . . . . . . . . . . . . . . . . . . Notes and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . .

243 248 248 249 250 250 250 255 263 264 264

Applications of Mathematical Modeling . . . . . . . . . . . . . . . . . . . . . 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Course Timetabling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Description of the CTP . . . . . . . . . . . . . . . . . . . . . . . 6.2.2 Model of the CTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.3 Solving the CTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Portfolio Selection Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Description of the PSP . . . . . . . . . . . . . . . . . . . . . . . . 6.3.2 Model of the PSP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.3 Solving the PSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Traveling Salesman Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Description of the TSP . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 Model of the TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.3 Solving the TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.4 Solution Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Disease Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.1 Description of the DCP . . . . . . . . . . . . . . . . . . . . . . . 6.5.2 Model of the DCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.3 Solving the DCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Flow Shop Scheduling Problem . . . . . . . . . . . . . . . . . . . . . . . 6.6.1 Description of the FSSP . . . . . . . . . . . . . . . . . . . . . . . 6.6.2 Model of the FSSP . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6.3 Solving the FSSP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Pre-disaster Investment Problem . . . . . . . . . . . . . . . . . . . . . . . 6.7.1 Description of the PDIP . . . . . . . . . . . . . . . . . . . . . . . 6.7.2 Model of the PDIP . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7.3 Solving the PDIP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Vehicle Routing Problem with Time Window . . . . . . . . . . . . . 6.8.1 Description of the VRPTW . . . . . . . . . . . . . . . . . . . . 6.8.2 Model of the VRPTW . . . . . . . . . . . . . . . . . . . . . . . . 6.8.3 Solving the VRPTW . . . . . . . . . . . . . . . . . . . . . . . . . 6.9 Petroleum Pipeline Scheduling Problem . . . . . . . . . . . . . . . . . . 6.9.1 Description of the PPSP . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

269 269 270 270 272 275 281 282 282 283 288 289 289 292 293 294 294 294 296 298 298 298 302 303 304 304 307 308 308 309 311 316 317

5.8 5.9 6

Contents

6.10

6.11

6.12

6.13

6.14

6.15

6.16 6.17

xiii

6.9.2 Model of the PPSP . . . . . . . . . . . . . . . . . . . . . . . . . . 6.9.3 Solving the PPSP . . . . . . . . . . . . . . . . . . . . . . . . . . . Cutting Stock Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.10.1 Description of the CSP . . . . . . . . . . . . . . . . . . . . . . . 6.10.2 Model of the CSP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.10.3 Solving the CSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sports Scheduling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11.1 Description of the SSP . . . . . . . . . . . . . . . . . . . . . . . . 6.11.2 Model of the SSP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11.3 Solving the SSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sell or Hold Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.12.1 Description of the SHP . . . . . . . . . . . . . . . . . . . . . . . 6.12.2 Model of the SHP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.12.3 Solving the SHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . Network Design Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.13.1 Description of the NDP . . . . . . . . . . . . . . . . . . . . . . . 6.13.2 Model of the NDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.13.3 Solving the NDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . Social Network Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.14.1 Description of the SNP . . . . . . . . . . . . . . . . . . . . . . . 6.14.2 Model of the SNP . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.14.3 Solving the SNP . . . . . . . . . . . . . . . . . . . . . . . . . . . . Water Resource Management Problem . . . . . . . . . . . . . . . . . . 6.15.1 Description of the WRMP . . . . . . . . . . . . . . . . . . . . . 6.15.2 Model of the WRMP . . . . . . . . . . . . . . . . . . . . . . . . . 6.15.3 Solving the WRMP . . . . . . . . . . . . . . . . . . . . . . . . . . Notes and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

318 323 324 325 325 329 330 331 332 338 339 340 340 342 343 344 344 346 350 350 351 356 360 360 361 364 368 369

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

383

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

387

Chapter 1

Preliminaries

1.1

Introduction

The issue of modeling, which often includes linear and nonlinear programming, mixed integer programming, and so forth, has been received considerable attention from researchers. In mathematical literature, different illustrative examples and realworld cases are discussed to demonstrate the applications and the importance of using the mathematical methods, while hoping that the reader would be able to generalize and reuse the ideas in other related cases. There are a few books specifically focusing on model building and dedicating to its methods and delicacies. Most of the integer programming books contain many problems, formulations, generalizations, and explanations on solution methods without discussing other issues such as proper formulation, or providing strategies to evaluate the quality of models. The fact that formulating a problem is also a process that needs to be improved is a topic that remains neglected and requires further attention. Modeling is more an art than a science; however, the meaning of this statement is not that modeling does not follow any specific rule and is a taste matter. Modeling and solving are not two independent issues and both have their own rules and features. Modeling often requires exact consideration of the problem’s conditions and its expression via mathematical symbols. This book is devoted to mathematical modeling and intends to answer questions such as; what is a mathematical model? Is there a systematic approach for model building? How is the quality of a model assessed? Therefore, the purpose of this book is to provide a systematic and methodical approach in the form of rules and regulations that familiarize the reader with the process of extracting a model from a problem. Our goal is to present the art of model building in a way that enables the reader to build and improve his model independently and evaluate its quality.

© Springer Nature Switzerland AG 2019 S. A. MirHassani, F. Hooshmand, Methods and Models in Mathematical Programming, https://doi.org/10.1007/978-3-030-27045-2_1

1

2

1.2

1 Preliminaries

Model and Its Types

A model is a description of a system that addresses some aspects of reality and leads to a better understanding of concepts and facilitates the decision-making process. In terms of the way the models are used to describe reality, they can be categorized into four groups containing iconic, analogical, verbal, and mathematical models (Fig. 1.1). An Iconic model represents the physical characteristics of a system in smaller or larger sizes. For example, the model of the solar system represents the sun and its planets on a smaller scale, while the atom-model describes the atom structure on a larger scale. In the same way, to investigate the resistance and efficiency of an airplane or a car, their models are constructed in smaller size and tested in the wind tunnel. An analogous model is a physical model describing some features of a given system with indicators. For example, a city map is an analogous model in which colors represent certain concepts. An organizational chart that shows the positions in an organization and their relationship, as well as the car speedometer, are other examples of analogous models. A verbal model describes reality using phrases and sentences. For example, we use a verbal model when we describe to a person who is looking for an address to travel along which streets and paths. A mathematical model uses letters, numbers, symbols and mathematical relations to represent reality, and reflects the internal relations between the system components. Mathematical models constitute a vast area. Some of them are merely descriptive and express the connection between various factors of a system via a mathematical formula. For example, the following formula is a mathematical model that shows the method of calculating the break-even point for a company producing a product. The break-even point indicates that for which level of production the total income is equal to the total expense incurred, and accordingly if the company produces less than this level, it will suffer losses while producing more than this level will result in profitability. Break‐even point ¼

Fig. 1.1 Classification of models

fixed costs income  variable costs

1.3 Value of Mathematical Models

3

The simulation model is another kind of mathematical models that predicts and analyzes the behavior of a system by the utilization of statistical methods, and constant observation and examination of the system in different situations. Another kind of mathematical models that have been received considerable attention is the mathematical programming model. The characteristic of most of these models is that they deal with optimization and correspond a set of mathematical relationships (such as equations, inequalities, logical statements, and so forth) to practical situations in the real world (such as technical communication, physical rules, market constraints, and so forth). Indeed, such models usually involve four main components: parameters, decision variables, constraints, and objective function. We are often interested in maximizing or minimizing the objective function, subject to meet all constraints. Mathematical models have been widely used, and their main advantages include: 1. They are precise and can be used to explain the ideas and phenomena in a mathematical form. Also, there is a wide range of valuable mathematical theorems and algorithms that are computationally helpful. 2. They are useful tools for analyzing systems, reveal relationships that are not known to individuals, and also lead to a better understanding of systems’ components. 3. Often, it is costly and dangerous to execute and evaluate a decision at once in the real world. Generally, compensation for wrong decisions, not only imposes enormous costs but also wastes time. With a mathematical model, we can predict and evaluate the outcome of the decisions without realizing the decision-making risk in the real world, and thus, make a significant saving in cost and time. 4. Due to the advancement of technology and computer science, computers can be efficiently used to perform calculations and solve mathematical models.

1.3

Value of Mathematical Models

Decision-making and looking for the best choice are managerial challenges which may have a great impact on the success or failure of an organization. Therefore, a manager who is familiar with decision-making tools such as mathematical programming can make the right decisions at the right time. Nowadays, with the growth of computational ability, mathematical programming models are increasingly effective in different fields such as transportation, investment, facility location, network design, production planning, human resources management, train scheduling, course timetabling and so on. The most crucial goal of this book is to teach the techniques and methods of formulating real-world problems in the form of mathematical programming models. Of course, note that modeling has both scientific and artistic dimensions. Its scientific dimension is learnable, but the artistic dimension needs taste, creativity, practice, and practice. To gain the modeling skills the user is required to think about various examples and problems, addressed in the book,

4

1 Preliminaries

and provide a model for them; then, compare the results with the current solution and do not worry about making mistakes. Henceforth, wherever we refer to the “model”, “mathematical model” or “optimization model”, we mean the mathematical programming model.

1.4

Steps of Model Building

The process of solving a real-life problem with a mathematical programming model involves several steps as described below. It is worth mentioning that depending on the requirements and needs of an institution or organization; the modeler may change the sequence of these steps or even follow some of them simultaneously.

1.4.1

Step 1: Identify and Define the Problem

The first step is to understand the problem properly and get a deep insight into the issues that the organization is dealing with, the decisions to be made, and the goals, limitations, and assumptions of the problem. This time-consuming step requires careful consideration because it has a direct effect on all subsequent steps. If the modeler has not a complete insight into the problem, a lot of time and money may be spent on the model development and problem resolution phases without any success in the achievement of organization’s goals. It is necessary to point out that sometimes the real-life problem is so comprehensive and incorporating all of its details may lead to a complex and intractable model. In this situation, the modelers usually consider simplifying assumptions, so that the presented model shows the main features and their relationship and at the same time, is simple and tractable.

1.4.2

Step 2: Collect Data and Present the Model

At this step, the modeler collects the necessary information to estimate the problem’s effective parameters, and then, formulates the problem as a mathematical programming model. Full familiarity with techniques and methods of modeling is the main requirement of this step, which is addressed in this book.

1.4.3

Step 3: Solve the Model

Now, it is necessary to use an appropriate algorithm to solve the model. The methods of solving mathematical programming models, both exact and heuristic, have been

1.5 Structure of a Mathematical Model

5

widely discussed in the operations research related books and papers. Also, there are a variety of solvers which are capable to efficiently solve the models.

1.4.4

Step 4: Validate the Model

The validity of the model should be carefully checked in terms of different aspects including adaptation to reality, compatibility with assumptions, possible errors in the data and formulation, the complexity and finally the solution time. One criterion that can be taken for model evaluation is to examine its validity for the current situation. Another method is to compare the level of performance that the model predicts, with the previous performance level of the system. If the studies carried out in this step do not confirm the validity of the model, we need to go back to the previous steps, revise the main simplified assumptions, if necessary, and modify the possible errors, or even construct a new model.

1.4.5

Step 5: Provide Results to the Organization

The modeler presents the results of the model resolution to the organization’s managers. Of course, it is possible that managers do not approve the proposed solution. In this case, the modeler should return to the previous steps and make the necessary adjustments. In order to avoid the managers’ dissatisfaction, not only the modeler should be careful in doing the previous steps, but also it is better to involve and justify the managers from the beginning of the process. After the managers have accepted the results of the study, the modeler should help them in using the results and give them the necessary training. Also, when performing the results, the system should be fully monitored, and its changes are checked regularly to ensure that the results of the model are effective in achieving the organization’s goals.

1.5

Structure of a Mathematical Model

Model 1.1 represents a general form of a mathematical model, consisting of four main components: decision variables, constraints, objective function, and parameters; each one is discussed in details below.

6

1 Preliminaries

Model 1.1 A general mathematical program
 min or max z ¼ f x1 , x2 , . . . , xn1 , y1 , y2 , . . . , yn2 , δ1 , δ2 , . . . , δn3 s.
t.  gi x1 , x2 , . . . , xn1 , y1 , y2 , . . . , yn2 , δ1 , δ2 , . . . , δn3 or  or ¼ bi 8i ¼ 1, . . ., m xj  0 8j 2 1 ¼ f1, 2, . . . , n1 g yj  0, Int 8j 2 2 ¼ f1, 2, . . . , n2 g 8j 2 3 ¼ f1, 2, . . . , n3 g δj 2 f0, 1g

1.5.1

(1.1) (1.2)

Decision Variables

Variables represent the decisions to be made, and their values are determined after the model is solved. For example, decisions like how much raw material to buy, how much to produce, how to invest in a project, and so on can be expressed in terms of decision variables, which are generally divided into three groups of continuous, integer, and binary variables. Continuous variables are the most common types of variables where their bounds are restricted to a connected subset of real numbers and represent uncountable quantities such as the amount of water stored behind a dam. Regarding the sign restriction, these variables may be either nonnegative or unrestricted in sign (i.e., free). A nonnegative continuous variable can only take nonnegative real numbers, while the domain of a free continuous variable includes zero, positive and negative real numbers. In this book, we display the continuous variable with the symbol x . To express the sign restriction on x, we use the terms (x  0) and (x free) indicating nonnegative and free variables, respectively. The free variable will be explained in Sect. 2.2.1. Integer variables that their domain is limited to sequential integer values are used to represent countable quantities such as the number of cars produced, the number of dams constructed, and the number of workforces required for a project. Regarding the sign restriction, these variables may be either nonnegative or free. In this book, we show the integer variable with the symbol y. To express the sign restriction on y, we use the terms (y  0, Int.) and (y Int. free) indicating nonnegative and free integer variables, respectively. A binary variable also referred to as zero-one variable, is a particular case of an integer variable where its domain is limited to the set {0, 1}. Binary variables support two-state decisions, such as yes or no, open or close, agree or disagree, true or false. Also, they treat as indicator variables to incorporate logical propositions, which will be described in the next chapters. In this book, we show the binary variable with the symbol δ, γ and η and use the term δ 2 {0, 1} to specify the domain of a binary variable.

1.5 Structure of a Mathematical Model

1.5.2

7

Constraints

Constraints and restrictions indicate the conditions governing the problem. Each constraint can be stated as (1.1), which consists of two parts, namely the left-handside (LHS) and the right-hand-side (RHS) components. The LHS component is a function of decision variables, and the RHS component is a constant number. These two components are linked together with the symbols , , or ¼. Remark 1.1 When formulating a constraint, the units of measurement of the factors aggregated with each other should be the same. For example, the summation of two amounts, one expressed in the dollar and the other expressed in Euro, is not correct. Also, both sides of equations or inequalities should have the same unit of measurement. For example, assume that the production time of each unit of products 1 and 2 are 10 and 15 min, respectively, and the production line is active for 8 h. We want to present a constraint ensuring that the total time spent on production is at most 8 h. The following constraint is not correct, because the units of measurement on different sides are not the same: 10x1 þ 15x2  8 The correct form is as follows where both sides are expressed in minutes. 10x1 þ 15x2  8  60 In Model 1.1, the type and the sign of variables are described via the restriction set (1.2). Remark 1.2 In a mathematical programming model, only the constraints with symbols , ¼ or  are allowed; and constraints with symbols>, 6¼, and < are not permitted to be directly involved in the model. Such constraints should, indirectly and in a proper way, be included in the model, which will be described in the next chapters.

1.5.3

Objective Functions

The objective function is a function of decision variables that evaluates the quality of solutions concerning a specific criterion, and generally, the decision-maker seeks the best solution by minimizing or maximizing this function. Usually, the value of the objective function is represented by the free variable z. Depending on the organization’s preferences, different objectives may be considered, such as maximizing the profit and customer satisfaction, or minimizing the cost, waste, and so forth.

8

1 Preliminaries

Always it is possible to transform any maximization problem to an equivalent minimization problem and vice versa. For example, the objective max f(x) can be written in the form of min (f(x)).

1.5.4

Parameters

Parameters are the input data of the problem and appear in the objective function and constraints. For example, in a production planning problem, the data ai, j, bi, and cj, can be considered as parameters which respectively indicate the amount of consumption of the raw material i per unit of product j, the inventory level of the raw material i and the sale profit of each unit of product j. In this case, if the decision variable xj denotes the production level of product j, then the aforementioned P ai, j xj  bi and the objective parameters will appear in the constraint j P function max z ¼ cj xj . j

1.5.5

Solution Types

Concerning the above discussion, it can be concluded that Model 1.1 seeks to find the value of the variables xj, yj and δj such that the constraint sets (1.1) and (1.2) are satisfied, and the objective function Any combination of the values
is optimized.  assigned to vectors ðx1 , . . . , xn1 Þ, y1 , . . . , yn2 , ðδ1 , . . . , δn3 Þ, subject to satisfying the constraint sets (1.1) and (1.2), is called a feasible solution. The set of all feasible solutions is called the feasible region that may be empty, countable or uncountable. A member of the feasible region with the best objective function value is called the optimal solution. We usually use asterisk () to refer to the optimal solution. For the solution of Model 1.1, one of the following four cases may occur: 1. The model has a unique optimal solution. In this case, there exists a unique member in the feasible region that its objective function value is finite and better than that of other feasible solutions. 2. The model has alternative (multiple) optimal solutions. In this case, the problem has an optimal solution, but it is not unique. Indeed, there is more than one optimal solution all of which take the same objective function value. 3. The model is infeasible. In this case, the feasible region is empty, and the problem has no feasible solution. 4. The model is unbounded. In this case, the feasible region involves solutions with arbitrarily large objective function values (in maximization problems) or solutions with arbitrarily small objective function values (in minimization problems), and therefore, the problem has no optimal solution.

1.5 Structure of a Mathematical Model

1.5.6

9

Model Assumptions

Mathematical models are influenced by a set of assumptions, which will be described in more details below. Divisibility Assumption This assumption indicates the continuity of variables xj and according to which, these variables are allowed to take any real value. Integrality Assumption This assumption implies the integrality of variables yj as well as the binary restriction of variables δj, and according to which, these variables are not allowed to take fractional values. Certainty Assumption This assumption implies that all input parameters of the problem are known with certainty and have deterministic values. For more clarity, consider an example that the profit of one unit of a particular product is dependent on the economic situation and may take three values of 0.3, 0.9 and 1.5 with probabilities of 0.3, 0.3 and 0.4, respectively. Such a problem violates the certainty assumption since the amount of the profit is not known with certainty and depends on the economic situation. Models violating the certainty assumption are called non-deterministic models and will be further described in the next chapters. Proportionality Assumption The objective function is satisfying the proportionality assumption if its contribution from each decision variable is proportional to the value of that variable. For example, the contribution to the objective function z ¼ 2x1 + 3x2, from the variable x1 equals 2x1, which is proportional to the value of x1. Thus, the contribution to the objective function from x1 ¼ k, equals k times the contribution to the objective function from x1 ¼ 1. Similarly, the constraint (1.1) is satisfying the proportionality assumption, if the contribution to the LHS of this constraint from each variable is proportional to the value of that variable. For example, the constraint 3x1 + 5x2  10 satisfies this assumption. However, if the objective function or the LHS of constraint (1.1), contains expressions such as x21 , logx1, |x1|, or ex1 , the proportionality assumption is violated. Additivity Assumption The objective function is said to satisfy the additivity assumption if it can be written as a summation of some terms, each containing only one variable. This assumption implies that the contribution to the objective function from each variable is independent of the value of other variables. For example, the objective function z ¼ 2x1  3x22 satisfies the additivity assumption, since it can be considered as a sum of two terms 2x1 and3x22 , each containing only one variable. Similarly, the constraint (1.1) is said to satisfy the additivity assumption, if its LHS can be shown as a group of summands, each containing only one variable. For example, this assumption holds for the constraint 3x1 þ 4x22  5, but it is violated by constraint x1x2 + 2x3  5.

10

1 Preliminaries

The objective function and constraints are said to be linear if they satisfy both the proportionality and additivity assumptions; otherwise, they are called nonlinear. Simultaneousness Assumption Simultaneousness assumption indicates that every feasible solution must simultaneously satisfy all constraints, and any violation is not permitted. Some problems may violate this assumption; for example, if a problem requires at least k constraints to be satisfied, or if the satisfaction of one of the constraints is conditional on establishing another constraint, then the simultaneous assumption is violated. In this situation, it is necessary to properly transform the model to an equivalent model satisfying simultaneous assumption. This topic will be further explained in the next chapters. In this chapter, the main components, different solution’s types, and model assumptions were described in general forms. In the next chapter, we introduce different types of variables, the constraints, and the objective functions, and categorize mathematical models from different aspects.

1.6

Notes and References

Some exercises of Sect. 1.7 have been derived from Winston (2003).

1.7

Exercises

Exercise 1.1 A company produces two products. The profit of each unit of the first product equals 2 monetary units (mu) (for the first 10 units of sale), and 1.5 (mu) (for the 11–30th units of sale). The profit of each unit of the second product equals 3 (mu) (for the first 20 units of sale) and 1 (mu) (for the 20–30th units of sale). Does this problem meet the additivity and proportionally assumptions? Exercise 1.2 Do the following problems meet the additivity and proportionally assumptions? Why? (a) 1 3

1 3

max z ¼ x1 þ x2 s. t. x1 + x2 ¼ 1 x 1, x 2  0

(b) max z ¼ x1x2 s. t. 4x1 + x2  1 x 1, x 2  0

1.7 Exercises

11

Exercise 1.3 A company is going to advertise its products during the broadcasting TV series and pffiffiffiffiffi football games. If the ad is displayed for x1 seconds during TV series, 5 x1 million pffiffiffiffiffi men and 20 x1 million women will see it. If the ad is displayed for x2 seconds pffiffiffiffiffi pffiffiffiffiffi during the football games, 17 x2 million men and 7 x2 million women will see it. The company wants the ads are seen by at least 40 million men and 60 million women. The cost of an advertisement during the time of broadcasting TV series and football game equals 50,000 and 10,000 (mu), respectively, and the goal is to minimize the total cost of advertising. (a) Provide an optimization model for this problem. (b) Does the model violate the proportionality and additivity assumptions? Exercise 1.4 Write an optimization model to find the circle with the smallest radius containing three points (0, 0), (1, 1) and (2, 3). Does this problem violate the additivity assumption? Exercise 1.5 Consider a company producing three types of products, and let the continuous variables, x1, x2 and x3 represent the production level of the first, second, and third products, respectively. Answer the following questions: (a) The total time required to produce one unit of the first product is three times and twice that of the second and the third product, respectively. If the company devotes the whole day to produce the first product, it can produce 270 units of that product. Provide a constraint to limit the daily production of the company, based on these assumptions. (b) The ratio between the sales of the first product and the total sales of the second and third products must be at least equal to 0.5. To comply with this assumption, provide a constraint. Does this constraint meet the proportionality and additivity assumptions? Exercise 1.6 Show that if xis the optimal solution to the model (a), then it is the optimal solution to the model (b), as well, assuming that the coefficients are known and d > 0. (a) min z1 ¼

n P

(b) cj xj

min z2 ¼ d

j¼1

s. t. n P ai , j x j  bi

cj xj þ d 0

j¼1

8i ¼ 1, . . . , m

j¼1

xj  0

n P

s. t. n P ai, j xj  bi 8i ¼ 1, . . . , m j¼1

8 j ¼ 1, . . ., n

xj  0

8 j ¼ 1, . . ., n

Chapter 2

Main Components of Mathematical Models

2.1

Introduction

In the previous chapter, the main components of a mathematical model (decision variables, constraints, objective function, and parameters) were described. This chapter provides more details about the types of variables, constraints, and objective functions and categorizes mathematical models. Also, a brief reference to the solution methods of mathematical models is provided.

2.2

Variables

Variables constitute the main part of mathematical models and are divided into different types according to their domains. In practice, the nature of decisions determines the type of required variables, and the choice of the solution method also mainly depends on the type of model’s variables. In the previous chapter, the most common types of variables, namely, continuous, integer, and binary variables, were introduced. Except for these variables, there are other types of variables, such as the finite domain, semi-continuous, and partial integer variables, which will be discussed later. It is worth mentioning that directly incorporating such variables into the models is not allowed and they should be rewritten appropriately in terms of other variables.

2.2.1

Free Variables

The domain of a free continuous variable consists of zero, positive, and negative numbers. For example, the monthly changes of the production level for a given © Springer Nature Switzerland AG 2019 S. A. MirHassani, F. Hooshmand, Methods and Models in Mathematical Programming, https://doi.org/10.1007/978-3-030-27045-2_2

13

14

2

Main Components of Mathematical Models

product can be defined as a free variable, because the production level may increase, decrease, or remain unchanged from one month to the next. To utilize an algorithm that is designed to deal with only nonnegative variables, it would be necessary to write free variables in terms of nonnegative ones. Suppose that {x1, x2, . . ., xk} is a set of free variables. There are two ways to rewrite these variables in terms of nonnegative continuous ones. The first way is to introduce two nonnegative continuous variables xj0 and xj00 , for j ¼ 1, . . ., k, and replace the variable xj with xj0  xj00 . This method doubles the number of variables. The second way is to introduce k + 1 nonnegative continuous variables 0 0 x10 , x20 , . . . , xk0 , xkþ1 and replace the variable xj with the term xj0  xkþ1 , for j ¼ 1, . . ., k. This method increases the number of variables by only one unit. Obviously, each feasible solution ðx1 , . . . , xk Þ for the initial model is equivalent to the feasible solution ðx01 , . . . , x0k , x0kþ1 Þ for the new model and we have: x0kþ1 ¼ max ð0,  min ðx1 , . . . , xk ÞÞ x0j ¼ xj þ x0kþ1 8j ¼ 1, . . . , k To put it more clearly, assume that x1, x2 and x3 are three free variables taking the values 1, 0 and  2, respectively, in a given solution. If the new variables x01 , x02 , x03 , x04 , are introduced and the variable xj is replaced by the term x'j-x'4, for j¼1,2,3, then, regarding the aforementioned solution, the value of new nonnegative continuous variables would be as follows: x40 ¼ max ð0,  min ð1, 0, 2ÞÞ ¼ 2, x10 ¼ 3,

2.2.2

x20 ¼ 2,

x30 ¼ 0

Finite Domain Variables

A variable y which takes its value among the members of a finite set, say {a1, . . ., an}, is called a finite domain variable. This variable can be rewritten in terms of binary variables. For this purpose, it is necessary to define the binary variables δ1, . . ., δn and add the following constraints: y ¼ a1 δ 1 þ . . . þ an δ n n X δj ¼ 1 j¼1

δj 2 f0, 1g

8j ¼ 1, 2, . . . , n

2.2 Variables

2.2.3

15

Semi-continuous Variables

If the domain of a variable is limited to the set {0} [ [a, b], where 0 < a < b, it is called a semi-continuous variable. If a semi-continuous variable indicates the level of an activity, it implies that the activity is either not performed at all or, performed at a level more than or equal to a minimum pre-specified value. For example, in a company, it may be costly to produce a specific product below a certain amount of a. On the other hand, the company may have no extra capacity to produce more than a specified amount of b. In this case, if the variable w represents the amount of production, its domain would be {0} [ [a, b]. Such Semi-continuous variables can be expressed with the help of other variables. To this end, two ways are available. The first method is to introduce the binary variable δ and add the following constraints: w  bδ w  aδ δ 2 f0, 1g The second method is to introduce a binary variable δ and two nonnegative continuous variables x1 and x2 and add the following constraints: w ¼ ax1 þ bx2 δ þ x1 þ x 2 ¼ 1 δ 2 f0, 1g x1 , x2  0 In a more general case, the domain of a semi-continuous variable w may be {c} [ [a, b], which c 2 = [a, b]. To express this variable in terms of other variables, it is sufficient to define a binary variable δ, two nonnegative continuous variables x1, x2 and add the following constraints: w ¼ cδ þ ax1 þ bx2 δ þ x1 þ x2 ¼ 1 δ 2 f0, 1g x 1 , x2  0 If w is restricted to integer values, it is called a semi-continuous integer variable.

2.2.4

Partial Integer Variable

If the domain of the variable w is limited to the set {0, 1, . . ., n  1} [ [n, b], where 0 < n < b, it is called a partial integer variable. This variable behaves, to

16

2

Main Components of Mathematical Models

some extent, as an integer variable, and then, acts as a continuous one. The constant value n can be interpreted as a level after which one unit of change in the value of w does not have a significant effect on the problem. This variable can also be expressed in terms of other variables. For this purpose, we introduce δ, x, and y as binary, continuous, and integer variables, respectively, and add the following constraints: w¼yþx x  bδ x  nδ 0  y  ð n  1Þ  ð 1  δ Þ δ 2 f0, 1g x0 y  0, Int:

2.3

Constraints

We now focus on some of the more general considerations of the constraints that a modeler has to be aware of them. In mathematical models, constraints can be categorized and studied in various aspects, which will be discussed in this section.

2.3.1

Simple Bound and General Bound

The simplest type of constraints in mathematical models is those limiting the domain of variables. Such constraints are usually written as follows: xj  u xj  l These are called simple upper and lower bounds, respectively. Also, a variable may have both lower and upper bounds at the same time. In this case, we have: l  xj  u By a little change, above constraint can be converted into a simple upper bound constraint as follows: 0  xj  l  u  l

2.3 Constraints

17

In practice, such constraints are not required to include in the set of constraints explicitly, and most algorithms can satisfy them implicitly. Another set of constraints denotes bound on a set of variables that are known as a generalized upper bound (GUB). The constraint below represents a generalized upper bound: X xj  U j2

If the number of such constraints is high and the set of variables involved in each one is independent of the other one, it would be computationally valuable to use specific algorithms designed for this structure. Some constraints, such as the one below, include both upper and lower bounds: L

X a j xj  U j2

However, by utilizing a simple upper bound, it is possible to display it as below: X aj x j þ s ¼ U j2

0sUL In this way, we have an equality constraint with a simple upper bound on the slack variable s.

2.3.2

Soft and Hard Constraints

Consider the following linear constraint of type : X

aj x j  b

ð2:1Þ

j2

This constraint denies any solution for which the LHS exceeds the value of b. In some cases, there is a constraint that is never permitted to be violated. For example, assume that constraint (2.1) represents a technical limit imposed by the capacity of a pipe. In this case, since the capacity of the pipe cannot be expanded, it is never possible to violate this requirement. Constraints that cannot be violated under any circumstances are known as hard constraints. In some situations, it is unrealistic to consider a constraint as a hard one. For example, if (2.1) states the production’s capacity or the raw material’s level, there may exist situations in which (2.1) can be relaxed. For example, it may be possible to

18

2

Main Components of Mathematical Models

expand the capacity or purchase some additional raw materials at a higher price. In such situations, we deal with a soft constraint, which can be violated, at a specified cost. To formulate this, we need to introduce the nonnegative continuous variable e and rewrite the constraint (2.1) as follows. X a j xj  b þ e j2

Then, the variable e, called excess or surplus variable, should be penalized in the objective function with the positive cost coefficient (in the minimization problem) and the negative cost coefficient (in the maximization problem). For example, assuming that b represents the current level of the raw material, the above method implies that through the optimization process, if the capacity expansion is recognized to be beneficial, it can be done with the expense of c  e. Of course, the level of excess variable e can be controlled with a simple upper bound when needed. If constraint (2.1) is of type , a similar result will be obtained by using the slack variable s as: X aj x j  b  s j2

Finally, if constraint (2.1) is of type “¼”, model it as: X

aj x j ¼ b þ e  s

j2

Moreover, by giving proper weights to the variables e and s in the objective function, it is possible to increase or decrease the RHS properly. Obviously, in the optimal solution, at least one of the variables e or s equals zero, because if both e and s are positive in a feasible solution, a better solution can be obtained by subtracting the smaller value from the other one.

2.3.3

Redundant and Binding Constraints

The redundant constraint, as its name implies, is a constraint whose elimination has no impact on the feasible region. Indeed, the feasible region created by other constraints is a subset of the points satisfying the redundant constraint. Although it is not straightforward to detect redundancy, as soon as the redundancy of a constraint is identified, it can be eliminated from the model. However, since the model’s feasible region depends on the value of its parameters, one constraint may be redundant for some values of the parameters and is non-redundant for other settings

2.3 Constraints

19

of parameter values. Hence, it is better to avoid the general elimination of such constraints and leave it to the optimizer preprocessing tools. Sometimes, it is necessary to deactivate the effect of a constraint on the feasible region. For example, consider a situation in which the only one of the two given constraints has to be satisfied. In this case, it is sufficient to increase the RHS of one of the constraints to an appropriate value (in the constraint ) or decrease it (in the constraint ). In these cases, the decision about which constraint should be removed is determined by the model. For example, the following constraints P guarantee that a P solution is feasible if one of the two constraints j2 a1, j xj  b1 or j2 a2, j xj  b2 is satisfied. X a1, j xj  b1 þ Mδ1 j2

X a2, j xj  b2 þ Mδ2 j2

δ1 þ δ2 ¼ 1 Where δ1 and δ2 are binary variables and M > 0 is a sufficiently large number. Obviously, if δ1 ¼ 1, then the first constraint is redundant, otherwise, the second one is redundant. Another category of constraints is known as binding constraints. An inequality constraint is called binding if it holds with equality at the optimal point. Therefore, a binding constraint can be identified after the problem is solved to optimality.

2.3.4

Cuts and Lazy Constraints

In the case of pure or mixed integer programming problems (which will be described in Sect. 2.5), the optimization process often begins by solving the LP relaxation (LPR) problem (LPR problem is referred to a problem in which the integrality restriction is temporarily relaxed, and all variables are considered continuous). It is well known that whatever the bound generated by the LPR problem is closer to the optimal value, the solution process typically converges faster to the optimal solution (if any). A cut is a type of constraint that is added to the model before or during the execution of the optimization process to decrease the gap between the LPR bound and the optimal objective value and accelerate the solution process. Therefore, the cuts are not a part of the model and do not affect the quality of the solution, but they may reduce the problem solution time. In general, a cut has two features: 1. It reduces the feasible region of the LPR problem. 2. None of the solutions available to the IP or MIP problem is eliminated. Cuts are usually produced and added to the model by the modeler or the optimization solvers. In this regard, there are precise algorithms, but their consideration is beyond the scope of this book.

20

2

Main Components of Mathematical Models

Example 2.1 Valid cuts for a facility location problem Suppose that we want to decide on the location of several service centers in order to serve customers. The location of the centers will be selected from a set of potential sites so that the ability to serve all customers are provided. Let dj be the demand of customer & P ’ j, and consider U as the capacity of each center. Then clearly, at least dj centers should be opened. Suppose that δi is a binary variable that is 1 if the j U

center i is opened; otherwise 0. Then, the following constraint can be used as an effective cut (valid inequality), which can accelerate the process of problem solving. X

δi 

&P ’ dj

i

j

U

On the other hand, lazy constraints are another type of constraints that appear in the main structure of the model, but make difficulties to its solution process and increase the solution time. Lazy constraints are a part of the model, i.e., any solution violating them is an infeasible solution. However, since the number of such constraints is significantly high, in order to speed up the solution process, they are first removed from the model. Then, the model with the remaining constraints is solved to generate a solution. Next, it is examined whether or not the generated solution satisfies the lazy constraints? If so, the current solution is optimal, and the process terminates. Otherwise, the current solution is deleted (by a cut) from the feasible region and this process is repeated. Example 2.2 Subtour elimination in a traveling salesman problem A Traveling salesman intends to start his travel from city 1, visit the cities 2, 3, . . ., n, and return to city 1 so that each city is visited exactly once. How can the salesman make his travel so that the total traveled distance is minimized? This example has a great reputation as the traveling salesman problem (TSP). Suppose that ci, j represents the distance between two cities i and j. To formulate the problem, a binary variable δi, j is introduced that is 1 if the seller immediately travels from city i to city j; otherwise 0. Thus, TSP is formulated as follows: min z ¼

n P n P

ci, j δi, j

i¼1 j¼1 n P j¼1 n P

δi , j ¼ 1

8i ¼ 1, . . . , n

δi, j ¼ 1 8j ¼ 1, . . . , n i¼1 P δi, j  jj  1,   f2, . . . , ng, 2  jj  n  1 i, j2 δi, j 2 f0, 1g 8i, j ¼ 1, . . . , n : i 6¼ j

ð2:2Þ

2.3 Constraints

21

Constraints (2.2) prevent the creation of any subtour (a trip plan that does not include all cities is called subtour) and can be considered as lazy constraints. By removing these constraints, the remaining model has the structure of an assignment problem (see Exercise 2.5) and can easily be solved. Therefore, a primary approach to tackle this problem may be as follows. First, the model without any lazy constraint, which is referred to as a relaxed model, is solved. If the obtained solution contains any subtour, a feasibility cut is added to the relaxed model to remove the current infeasible solution from its feasible region. Then, the relaxed model is solved again, and the process is repeated. The first complete tour produced by this method shows the optimal solution to the problem. Another application of this type of constraints is the removal of an undesirable solution. Most of the integer programming problems cannot be solved quickly. Therefore, users try to find a solution by using heuristic and explorative methods. Such a solution is then eliminated from the feasible region if it has no required quality. This task is accomplished by generating a cut that eliminates the current solution. Consider the following example: Example 2.3 Elimination of a feasible suboptimal solution Suppose that a solution to a binary integer programming problem is given as follows: δ^j ¼



1 0

j 2 1 j 2 0

, 0

[

1 ¼ 

If this solution has not the required quality, it should be removed from the feasible region. For this purpose, we add the following cut ensuring that at least one of the variables δj takes a value different from δ^j : X j20

δj þ

X

 1  δj  1

j21

The cut indicates that at least one of the terms ð1  δj Þ ðfor j 2 1 Þ and ðδj Þ ðfor j 2 0 Þ must be changed to one. Equivalently, the above constraint can be stated as follows: X j2

2.3.5

X    1  δ^j δj þ δ^j 1  δj  1 j2

Constraints and Special Ordered Sets

Two most commonly used types of constraints in mathematical programming problems are associated with the special ordered sets of type 1 (SOS1) and the special ordered sets of type 2 (SOS2).

22

2

Main Components of Mathematical Models

The set SOS1 is a set of variables (continuous P or integer), precisely one of which should take a nonzero value. The constraint j2 ai, j xj  bi has the characteristic of SOS1, if exactly one of its variables allowed being nonzero. To indicate that a constraint has the SOS1 attribute, it is stated as follows: X

ai, j xj  bi

ðSOS1Þ

j2

For example, let δj be a binary variable and assume that the following constraint should be observed. X δj ¼ 1 j2

In this case, instead of defining δj as a binary variable, we can treat it as a continuous one and rewrite the above constraint as follows: X δj ¼ 1

ðSOS1Þ

j2

The SOS1 attribute of the above constraint causes the variables δj to take only zero or one values. AnPimportant property of this set is that for any 0  , if we define x ¼ δj then x is equal to zero or one. Therefore, x is j20

called an implicit binary variable, which is a continuous variable but acts as a discrete variable. The set SOS2 is an ordered set of variables, at most two of which can take nonzero values and if exactly two variables are nonzero, they must necessarily be adjacent in theirP ordering. A constraint j2 ai, j xj  bi has the characteristic of SOS2, if at most two of its variables allowed being nonzero and these two variables should be consecutive. To indicate that a constraint has the SOS2 attribute, it is stated as follows: X

ai, j xj  bi

ðSOS2Þ

j2

In practice, it is possible to define models that have a set of integer or continuous variables belonging to the sets SOS1 or SOS2. The use of these variables has computational advantages in solution algorithms. Many optimization solvers support the methods in which the modified branch-and-bound (B&B) algorithm works with the sets SOS1 and SOS2. The description of these methods is beyond the scope of this book, and the interested reader is referred to the relevant literature (Beale and Tomlin 1969).

2.3 Constraints

23

Here are some examples of these sets in optimization problems and the way they are created. Example 2.4 Warehouse location problem Suppose that precisely one of the five positions 1, . . ., 5 should be selected for the construction of a new warehouse. Let δi be a binary variable that is 1 if the warehouse is located at position i; otherwise 0. The set {δ1, .., δ5} can be considered as a SOS1 set. The SOS1 property of the following constraint ensures that only one variable takes 1 and the others take 0. δ1 þ δ2 þ δ3 þ δ4 þ δ5 ¼ 1,

ðSOS1Þ

Therefore, it is not necessary to explicitly specify the binary restriction on variables. Example 2.5 Capacity expansion problem A company wants to increase its production capacity. In this regard, it deals with five options, one of which should be selected. Choosing the jth option ( j ¼ 1, . . ., 5) increases the capacity to rj, and requires fj units of budget. Let δj be a binary variable that is 1 if the jth option is selected; otherwise 0, and consider x, and v as variables displaying the increase in the capacity and the amount of investment, respectively. In this case, if the set of variables {δ1, .., δ5} is considered to be of type SOS1, then, the capacity expansion modeling can be stated as follows: x ¼ r 1 δ1 þ r 2 δ2 þ r 3 δ3 þ r 4 δ4 þ r 5 δ5 v ¼ f 1 δ1 þ f 2 δ2 þ f 3 δ3 þ f 4 δ4 þ f 5 δ5 δ1 þ δ2 þ δ3 þ δ4 þ δ5 ¼ 1 ðSOS1Þ The domain of variables x, and v is finite. The SOS1 property of the last constraint ensures that the variable δj treats as binary. There are other applications for the sets SOS1 and SOS2 as well. The most common use of the set SOS2 is in the modeling of piecewise linear functions (PLF), which will be described in Sect. 4.3.

2.3.6

Chance Constraints

In some problems that are affected by uncertainty, the consequences due to the violation of a constraint may be irreparable. For example, the water level behind a dam is limited for safety reasons, and its violation can lead to severe damage. Such constraints, except in the event of an emergency and unforeseeable circumstance, shall not be violated. In such cases, by using the chance constraints, it is possible to keep hold such limitations for a high percentage of observations of uncertain parameters.

24

2

Main Components of Mathematical Models

For example, assuming that a~ is an uncertain parameter, Pr ðgðx, a~Þ  0Þ  1  α shows a chance constraint that gðx, a~Þ  0 is a regular constraint and α 2 ½0, 1 is a parameter determined by the decision-maker to meet the safety and security needs. This chance constraint ensures that the regular constraint gðx, a~Þ  0 is met with a probability of at least 1  α. Under certain conditions, chance constraints can be rewritten as regular constraints. For more details, see Birge and Louveaux (2015).

2.3.7

Converting Constraints

Sometimes we need to replace the inequalities in the model with equations. Each constraint of type  (or ) can be rewritten in equality format by adding (or subtracting) a continuous and nonnegative called the slack (excess) variable. In n P ai, j xj  bi is equivalent to the following constraint: other words, the constraint j¼1 n X

ai, j xj þ s i ¼ bi

j¼1

s i  0 Also, the constraint

n P

ai, j xj  bi is equivalent to the following constraint:

j¼1 n X

ai, j xj  sþ i ¼ bi

j¼1 sþ i 

0

Similarly, it is sometimes necessary to replace equations in a model with inequalities. Suppose that the following model includes m equality constraints: n X

ai, j xj ¼ bi

8i ¼ 1, . . . , m

ð2:3Þ

j¼1

There are two ways to rewrite this constraint as inequality. The first method n n P P is to replace each constraint ai, j xj ¼ bi with two constraints ai, j xj  bi , and

n P

j¼1

j¼1

ai, j xj  bi . The second method is to replace the constraint set (2.3) with

j¼1

the following constraints:

2.4 Objective Functions

25 n X

ai, j xj  bi

8i ¼ 1, . . . , m

ð2:4Þ

j¼1 m X n X

ai , j x j 

i¼1 j¼1

m X

ð2:5Þ

bi

i¼1

In the second method, a smaller number of constraints are added to the problem. The constraints (2.3) result in the constraints (2.4) and (2.5). Therefore, to prove the validity of the above relations, it is enough to show that the reverse is also valid. Given the relations (2.4) and (2.5), for i0 ¼ 1, . . ., m, we have: n X

m X n X

ai 0 , j x j ¼

j¼1

! ai, j xj



i¼1

bi 

@

m n X X i¼1ji6¼i

i¼1 j¼1 m X

0

m X

0

1 ai, j xj A

j¼1

bi ¼ bi 0

i¼1ji6¼i 0

The above relation with inequality (2.4) results in Eq. (2.3).

2.4

Objective Functions

In each mathematical model, the function that evaluates the quality of any feasible solution is called the objective function. Different goals can be put forward for optimization problems, such as finding the most profit, cost saving, maximum service, maximum capital turnover, maximum return on capital, maximum net present value, minimum number of employees, minimum wastage, the risk of loss, maximum customer satisfaction, maximum likelihood of stability, maximum stability of operational plan, minimizing maximum error, maximizing minimum customer satisfaction, etc. Also, it is possible to have no specific goal in the optimization problem at all. Moreover, we may deal with several conflicting goals at the same time. For a given set of constraints, different goals usually lead to different optimal solutions. This is not the case only in rare situations: for example, when the goals are dependent, and there is no confliction between them the optimal solutions corresponding to all objectives are the same. Additionally, when the problem’s feasible region is a singleton set, regardless of the type of objective functions, the same optimal solution is always obtained. Such a discovery suggests that there is no need for optimization. Of course, if the model is very complex, the recognition that whether or not the feasible region is singleton is not an easy task, but the existence of such a property would be more likely when different objective functions of a given

26

2

Main Components of Mathematical Models

model produce the same optimal solutions. Also, more understanding of the problem that is modeled may provide more complete result. It is worth mentioning that the objective function of every optimization problem can be included in the constraints. To put it more clearly, consider the following problem: min z ¼ f ðx1 , x2 , . . . , xn Þ s:t: gi ðx1 , x2 , . . . , xn Þ  bi xj 2 j

8i ¼ 1, . . . , m

8j ¼ 1, . . . , n

The above model can be rewritten as follows: min z ¼ w s:t: gi ðx1 , x2 , . . . , xn Þ  bi

8i ¼ 1, . . . , m

w  f ð x1 , x2 , . . . , xn Þ xj 2 j 8j ¼ 1, . . . , n w free This issue can be raised for maximization problems, as well.

2.5

Mathematical Models

In this section, we will categorize and study mathematical models from different points of view. A better understanding of the model’s structure may aid to select proper algorithms to solve the model effectively and create a significant saving in the solution time.

2.5.1

Continuous and Discrete Models

This classification is based on the type of decision variables. If all of the variables of the model are continuous, the model is called continuous, but if at least one of the variables is restricted to a countable set of values (e.g., a binary or integer values), then the model is called discrete.

2.5.2

Linear and Nonlinear Models

This categorization is based on the linearity or nonlinearity of constraints and objective function of the model. As previously stated, the constraints and the

2.5 Mathematical Models

27

objective function of the model are linear if and only if the problem proportionality and additivity assumptions are valid. In a mathematical programming model, if the objective function and constraints are linear, the model is called linear, and if at least one nonlinear term, such as x1 x2 , xx12 , ex1 , and jx1 j appears in the objective function or model constraints, it is called nonlinear. Depending on whether the objective function and constraints are linear or nonlinear, and whether the variables are continuous or discrete, there are different types of models, which will be further introduced here. Consider the general structure of a mathematical model as follows (this structure was previously described in Sect. 1.5): Model 2.1 A general mathematical model   min or max z ¼ f x1 , x2 , . . . , xn1 , y1 , y2 , . . . , yn2 , δ1 , δ2 , . . . , δn3 s. t.  gi x1 , x2 , . . . , xn1 , y1 , y2 , . . . , yn2 , δ1 , δ2 , . . . , δn3  or  or ¼ bi 8i ¼ 1, . . ., m xj  0 8j 2 1 ¼ f1, 2, , . . . , n1 g yj  0, Int 8j 2 2 ¼ f1, 2, , . . . , n2 g δj 2 f0, 1g 8j 2 3 ¼ f1, 2, , . . . , n3 g Linear Programming Model If in Model 2.1 only continuous variables are present (i.e., 1 6¼ ∅, 2 , 3 ¼ ∅) and its constraints and objective function are linear, it is considered as a linear programming (LP) model with the following general structure: Model 2.2 Linear programming model n P min or max z ¼ cj xj j¼1

s. t. n P ai, j xj  or  or ¼ bi

8i ¼ 1, . . . , m

j¼1

xj  0 or free

8 j ¼ 1, . . ., n

In this model, there are n continuous variables that may be nonnegative or free in sign; cj, bi and ai, j are three types of parameters that are called objective function coefficients, RHS values, and technological coefficients, respectively, and their values are known with certainty. In an LP model, a linear function of decision variables is maximized or minimized, and the constraints are expressed as linear equalities or inequalities.

28

2

Main Components of Mathematical Models

In general, an LP model can be represented in a compact symbolic matrix form, as follows: Model 2.3 An LP in matrix form min or max z ¼ cTx s. t. Ax  or  or ¼ b x0 Where cn1, bm1 and Amn represent the model parameters, xn1 is the vector of variables, and the problem constraints can be of any type , ¼ or . Nonlinear Programming Model If all variables of Model 2.1 are continuous (i.e., 1 6¼ ∅, 2 , 3 ¼ ∅) and there is at least one nonlinear term in the constraints or objective function, we get a nonlinear programming (NLP) model. Integer Programming Model In many real-life problems, divisibility assumption is not valid, and it is necessary to use integer and binary variables. If in Model 2.1, we have 2 [ 3 6¼ ∅, we get an integer programming model. In a general classification, integer programming models are divided into several categories, which we refer to some of them here. Pure Integer Programming Model If Model 2.1 has no continuous variables (i.e., 1 ¼ ∅, 2 [ 3 6¼ ∅) and its constraints and objective function are linear, we get a linear model with only integer variables which is called pure integer programming (PIP) model. Binary Model If Model 2.1 has only binary variables (i.e., 1 , 2 ¼ ∅, 3 6¼ ∅) and its constraints and objective function are linear, we get a linear model with only binary variables which is called binary integer programming (BIP) model. Mixed Integer Programming Model If Model 2.1 simultaneously contains both continuous and discrete variables (i.e., 1 6¼ ∅, 2 [ 3 6¼ ∅) and its constraints and objective function are linear, we get a mixed integer linear programming (MILP) model which belongs to the category of mixed integer programming (MIP) problems. Nonlinear MIP If Model 2.1 has both continuous and discrete variables, (i.e., 1 6¼ ∅, 2 [ 3 6¼ ∅) and there is at least one nonlinear term in the constraints or objective function, we get a mixed integer nonlinear programming (MINLP) problem. The models mentioned above are summarized in Table 2.1 and Fig. 2.1. Therefore, LP and NLP are in the category of continuous models, and PIP, BIP, MIP, and MINLP belong to the category of discrete models. LPs are typically easier to solve while solving NLP models, due to their nonlinear structure, and discrete models, due to the presence of discrete variables, are more complicated.

2.5 Mathematical Models

29

Table 2.1 Model’s types Model

Abbreviation

Type of variables

Linear programming Nonlinear programming

LP NLP

1 ¼ 6 ∅, 2 , 3 ¼ ∅ 1 ¼ 6 ∅, 2 , 3 ¼ ∅

Pure integer programming Binary integer programming Mixed integer linear programming Mixed integer nonlinear programming

PIP BIP MILP

1 ¼ ∅, 2 [ 3 6¼ ∅ 1 , 2 ¼ ∅, 3 6¼ ∅ 1 6¼ ∅, 2 [ 3 6¼ ∅

MINLP

1 6¼ ∅, 2 [ 3 6¼ ∅

Objective and constraints Linear At least one nonlinear term Linear Linear Linear At least one nonlinear term

Fig. 2.1 Classification of models

Whenever we relax the integrality assumption from PIP, BIP, and MILP models and assume that all variables are continuous, the resulting problem is called the linear programming relaxation model which is referred to as LPR, for short, and the bound obtained by the resolution of LPR is called the LP-relaxation bound. In most cases, the process of determining the optimal solution begins by solving the LPR problem, and whatever the LPR bound is closer to the optimal objective value of the main problem, the resolution process may converge faster to the optimal solution (if any). Remark 2.1 For the sake of brevity, throughout the text, in addition to the abbreviation PIP, we may use the term IP to refer to a pure integer programming problem. Additionally, to refer to a mixed integer linear programming problem, in addition to MILP, we may use MIP, as well.

30

2

Main Components of Mathematical Models

Remark 2.2 Throughout the text, we refer to the following mathematical model as a standard model: Model 2.4 Standard model n3 n1 n2 P P P min or max z ¼ cj xj þ c0j yj þ c00j δj j¼1

j¼1

j¼1

s. t. n3 n1 n2 P P P ai, j xj þ a0i, j yj þ a00i, j δj  or  or ¼ bi j¼1

j¼1

xj  0 yj  0, Int: δj 2 f0, 1g

8i ¼ 1, . . ., m

j¼1

8j 2 1 ¼ f1, 2, , . . . , n1 g 8j 2 2 ¼ f1, 2, , . . . , n2 g 8j 2 3 ¼ f1, 2, , . . . , n3 g

In some situations, it may be possible to convert a non-standard model to a standard one or provide an appropriate approximation. This topic is described in Chap. 4.

2.5.3

Single and Multi-Objective Models

As mentioned earlier, an optimization problem may involve one or several objective functions, or even, sometimes there is no specific objective function. Accordingly, single-objective, multi-objective and non-objective models are possible. In a non-objective case, only finding a feasible solution is enough. For example, in a course timetabling problem, we may look for assigning a collection of courses to timeslots of the week and college classes, so that the training standards and related operational requirements are satisfied. In such cases, it is only necessary to find a solution observing all the problem constraints. In a multi-objective model, simultaneous optimization of several goals (which are generally conflicting) is considered. The following model is a multi-objective model in which the simultaneous maximization of the objectives f 1 ðxÞ, f 2 ðxÞ, . . . , f k ðxÞ is required, and  represents the problem’s feasible region. Model 2.5 Multi-objective model max f(x) ¼ ( f1(x), f2(x), . . ., fk(x)) s:t:x 2  Remark 2.3 Note that in the above model, if for some objective functions such as f i0 ðxÞ, the minimization direction is desired, then f i0 ðxÞ can be replaced by  f i0 ðxÞ so that all objective functions be expressed in maximization direction. In a multi-objective problem, in general, no single solution simultaneously optimizes all objective functions since the objectives are usually conflicting. Instead,

2.5 Mathematical Models

31

Pareto or efficient solutions are sought to such problems. Before describing these solutions, it is required to explain the concept of domination. Let x and x^ be two feasible solutions to Model 2:5: We say that x dominates x^ if for every objective, x is not worse than x^ ði:e:, f i ðxÞ  f i ð^ xÞ for all i ¼ 1, . . . , kÞ and in at least one objective, x is strictly better than x^ ði:e:, there exists i 2 f1, . . . , kg for xÞ). which f i ðxÞ > f i ð^ Pareto or Efficient Solution We say that a feasible solution x^ is an efficient or Pareto solution to the multiobjective problem (Model 2.5), if it is not dominated by any other feasible solution. In other words, if there is no other feasible solution improving at least one objective without degrading other objectives. There are several methods to identify the Pareto solutions that some of the most important ones are listed in Sect. 2.6.4.

2.5.4

Constrained and Unconstrained Models

If the optimization problem has no constraint and merely minimization or maximization of the function f(x1, . . ., xn) on the real space, ℝn, is desired, it is called unconstrained optimization problem. In contrast, the constrained optimization problem refers to a problem having at least one constraint. For some problems, such as NLP problems, the study of the unconstrained state is of great importance. Because most of the algorithms proposed to solve NLP problems, either solve a sequence of unconstrained problems or, in a proper way, turn the constrained problem into an unconstrained one. Also, sometimes it is possible to extend the unconstrained solution methods to solve the constrained problems.

2.5.5

Static and Dynamic Models

How the parameters of the problem change over time has a significant effect on its model. In a static model, the problem parameters are assumed to be independent of time. In this type of problems, it is assumed that the parameters of the problem do not change over time, and the results obtained for a given time-period can be utilized for other time-periods, as well. For example, the weekly course timetable that is available for a week is reused for other weeks. In practice, there are models in which parameters and decisions vary during different time-periods. For example, the price of a product in different months is a time-dependent parameter. The timedependency of the problem’s parameters leads to a dynamic model.

32

2.5.6

2

Main Components of Mathematical Models

Deterministic and Non-deterministic Models

If the model’s parameters are known with certainty, then, we have a deterministic model. Uncertainty in parameters is a phenomenon we encounter with in practice. In this regard, statistical information (e.g., the probability distribution function of the parameters or their possible outcomes) is available that may lead to a non-deterministic model. Stochastic programming is a branch of decision-making under uncertainty. The main difference with the deterministic case is the presence of uncertainty in some of the input parameters, such as the RHS values, the coefficients of the objective function and the technological coefficients that may not be known at the time of decision-making. In a stochastic model, solutions having the minimum average cost are sought, while taking into account the events that may occur in the future. In other words, the goal is to determine the optimal solution to a problem that its parameters are not precisely known at the time of decision-making, but the probability distribution of uncertain parameters or the historical data are available. The scenario-based stochastic model is one of the methods allows to incorporate uncertainty in optimization models and has successfully been used in various fields. In this method, random parameters are considered as random variables, and a scenario is a hypothesis about the future that expresses the interaction between different factors under certain conditions. Scenarios are a combination of random parameters and summarize different data states in a few simple ones. The managers who can better predict the possible values for random parameters are more prepared and have a chance to take advantage of opportunities. This method tries to avoid two common errors caused by extremes. Naturally, scenarios are made based on the possible states of the parameters. Of course, it is not necessary to consider all possible states, and using several simple modes is sufficient. For example, three low, medium, and high-interest rates among hundreds of possible outcomes may be sufficient. In general, the goal is to obtain a value for strategic decisions that are valid for almost all future situations and, to some extent, close to the optimal solution. Scenario-based stochastic programming models are divided into two groups of two-stage and multi-stage problems that will be discussed in Sect. 2.5.8. Also, chance constraint programming and robust optimization models are other approaches to confronting uncertainty in optimization problems.

2.5.7

LP Models with Interval Coefficients

In most of the LP problems, it is assumed that the value of the parameters is known with a sufficient precision, while in the real world, this assumption is not necessarily correct and, generally, the exact values of parameters are not known, and merely, some information on the range of their variations is available. One of the exciting applications in linear programming is the formulation and the resolution of problems in which the coefficients are not precisely known, but defined over intervals.

2.5 Mathematical Models

33

Decisions made by solving these models may be more realistic. When the constraint coefficients are defined over a range, there is no fixed and definite feasible region, but each selection of the coefficients may lead to a specific region, and hence, there are many feasible regions. Various feasible regions can lead to different optimal solutions. Also, if the objective function coefficients are defined over a range, there exist many different objective functions. In such cases, we need to identify the coefficients for which the best and the worst optimal solutions are obtained. To clarify the discussion, consider the following example: Example 2.6 Model with interval RHS coefficients Consider the following problem in which the RHS of the second constraint can take any number within the interval [3, 5]. We deal with different feasible regions each one is associated with a value for b2 2 [3, 5], and accordingly, corresponding to each feasible region, the optimal objective function value is computable. We want to know that for which value of b2 the worst optimal solution and for which one the best optimal solution is obtained. min z ¼ x2 s:t: 2x1 þ x2  4 3x1  2x2  ½3, 5 x1  0, x2  0 Consider the two constraints 3x1  2x2  3 and 3x1  2x2  5. As shown in Fig. 2.2, these two constraints are parallel and, with gradually changing the RHS, we can get from one constraint to the other. In this example, it can easily be seen that the worst case occurs when we consider the second constraint as 3x1  2x2  3, which leads to the following solution: x1 ¼ 1:571, x2 ¼ 0:857, z ¼ 0:857,

Fig. 2.2 Optimal solutions for the best and the worst cases

34

2

Main Components of Mathematical Models

Moreover, the best case happens when the second constraint is taken as 3x1  2x2  5 and the following solution is reached: x1 ¼ 1:857, x2 ¼ 0:286, z ¼ 0:286

2.5.8

Single and Multi-stage Models

In a model, decision-making may take place in a single or multiple stages. Two-stage stochastic programming models are among the most common and well-known problems addressing multiple stages. The most important feature of a two-stage stochastic programming model is that the decisions are divided into two stages where the decision-maker makes a decision in the first stage (i.e., the first-stage decisions). Then, a random event occurs (i.e., the uncertainty resolves) which affects the decisions of the first stage. Then, a recursive decision (i.e., the second-stage decision) is taken to offset the likely adverse effects of the first-stage decisions. The reason this problem is called two-stage stochastic program is that it is not necessary to make decisions of the first and second stages at the same time, but we can postpone the second-stage decisions until an event occurs and the uncertainty resolves. The generalization of this process to the several stages results in the multi-stage stochastic programming models. The general presentation of a two-stage stochastic LP model is as follows: Model 2.6 Two-stage stochastic programming model P min z ¼ cT x þ ps bsT ws s2

s. t. Ax ¼ d Bs x þ Ds ws ¼ hs 8s 2  x  0, ws  0 8s 2  Where,  is a finite set of possible scenarios, each scenario s 2  represents a state of random parameters, and ps is the occurrence probability of the scenario s. Each parameter having the superscript s is a random parameter that takes a specific value under scenario s. In addition, x is the vector of first-stage decisions and ws is the vector of second-stage decisions under scenario s. The parameters d and A are the RHS values and the coefficients of the first-stage constraints, respectively. The parameters hs, Bs and Ds, respectively, show the RHS values and the coefficients of the first- and second-stage decision variables in the second-stage constraints under scenario s. In addition, c and bs, represent the coefficients of the first- and second-stage decisions in the objective function, respectively. The sequence of decisions and events is as follows: first, the first-stage decision, x, is determined. Then, a scenario s occurs, and accordingly, the second-stage decision ws is made. The objective function minimizes the cost of the first-stage decisions and the average cost of the second-stage decisions.

2.5 Mathematical Models Table 2.2 Demand and fuel price under each scenario

35

Scenario 1 2 3

Weather Normal Cold Very cold

Fuel price 5 6 7.5

Demand 100 150 180

Probability 0.33 0.33 0.34

In the following, the two-stage stochastic programming is briefly described on a simple example. For a complete overview of the two-stage and multi-stage stochastic programming problems, see Birge and Louveaux (2015). Example 2.7 Fuel planning under uncertainty A company has a 2 years plan to supply the requested fuel. In the first year, the company has to buy fuel and use as much as it needs and store the remaining for the next year’s consumption. In the second year, it will be able to meet its demand from the previous year’s stock and by purchasing from the market. Fuel price and demand in the first year are known with certainty and are equal to 5 (mu) and 100 units, respectively. However, for the second year, three scenarios normal, cold and very cold are possible for weather condition, and hence, fuel price and demand level are uncertain. The actual value of non-deterministic parameters is realized at the beginning of the second year and the corresponding information is given in Table 2.2. The cost of storing each unit of fuel is equal to 1 (mu), and the storage capacity is 90 units. Therefore, the company’s manager faces with a two-stage stochastic programming problem. At the beginning of the first year, he must decide on the amount of fuel purchased and stored in the first year, so that the demand for that year is met. In the second year, once the uncertainty is realized, he must decide on the amount of fuel purchased in the second year, so that its needs are met via the first year’s reserve and the second year’s purchase. The objective is to minimize the first-stage cost and the average of the second-stage cost. To this end, we present a stochastic programming model. The first-stage decisions are as follows: The amount of purchased fuel in the first year The amount of stored fuel in the first year

x 1: x 2:

These decisions should be made now and before the weather condition is realized for the coming year. We show the scenario set with  ¼ f1, 2, 3g, indexed by s, and define the second-stage decision as follows which is taken after the weather condition is realized and the uncertainty is resolved. ws:

The amount of fuel purchased in the second year under scenario s

Thus, the problem is formulated as follows:

36

2

Main Components of Mathematical Models

Model 2.7 SP model for the fuel supply problem min z ¼ 5x1 + x2 + (0.33  5)w1 + (0.33  6)w2 + (0.34  7.5)w3 s. t. x1  x2 ¼ 100 x2  90 x2 + w1 ¼ 100 x2 + w2 ¼ 150 x2 + w3 ¼ 180 x1, x2, ws  0 8 s ¼ 1, 2, 3 The model’s optimal solution is as follows: 





x1 ¼ 190, x2 ¼ 90, w1 ¼ 10, w2 ¼ 60, w3 ¼ 90, z ¼ 1404:8 If the company proceeds according to the above solution, then, if the normal scenario happens in the second year, the cost would be 1090. Similarly, for the cold and very cold weather, the cost would be 1400 and 1715, respectively. The value of z ¼ 1404.8 represents the expected cost in the long run. That is, if the company’s manager uses the same solution during n successive years, under the same conditions, on average, his yearly cost would be 1404.8, which is actually equal to the average cost over three scenarios (normal, cold, and very cold). In other words, we have: ð0:33  1090Þ þ ð0:33  1400Þ þ ð0:34  1715Þ ¼ 1404:8

2.5.9

Single and Multi-level Models

Consider a company having several distinct factories. The company’s board of directors is at the first level of decision-making, and the manager of associated factories are required to execute its decisions. Therefore, the board of directors, regarding its responsibilities and information, makes central decisions for the development of the company to achieve its goals and benefits. However, each factory manager can also make decisions to improve its factory performance, according to his authority. On the other hand, decisions taken at the factory level can influence the decision-making space of the board of directors and force the company to change its decision. In systems with the above structure, optimal decision-making is not easy, because the decisions are made decentralized; meaning that there are two decisionmakers at two different levels of authority and each one manages only one part of decisions. In other words, the first-level decision-maker alone cannot decide on all variables or make decisions without regarding the other parts of the system. Bi-level programming and, more generally, multi-level programming are techniques proposed to model and solve decentralized decision-making problems. Almost all of the decision-making issues that include the government, the private

2.5 Mathematical Models

37

sector, and the public can be described as a bi-level or multi-level programming problem. For example, in the transportation system, the government, as the first-level decision-maker, decides on the level of duties, taxes, and subsidy allocation and aims at increasing the general welfare in the transportation sector. Transportation companies in the second-level of decision-making, decide on the types of vehicles, the transportation routes and costs, and so forth. Finally, people, as the third level decision-maker, for the sake of greater prosperity and lower cost, choose the type of travel device, travel time, and so on. In general, in a single-level or centralized programming problem, there is only one decision-maker. However, the multi-level programming is a tool developed for decentralized decision-making. Multi-level programming with a hierarchical structure has the following characteristics: 1. Decision-making levels affect each other based on a hierarchical structure. 2. Decisions are taken from the top level to the bottom level. In other words, the lower level decision-makers are required to implement the decisions taken by the higher levels. 3. Decisions of each level can influence the goals and decision space of other levels. 4. Each level tries to optimize its goals while is influenced by decisions and activities of other levels. Bi-level and multi-level programming is an extension of the popular Stackelberg game addressed in the game theory. Game theory, with different applications in economic, business, political science and biology, is the study of strategic interaction between rational decision-makers (called players). The game environment is defined based on different features including the set of players, the order of play, the strategies a player can take during the game, the information each player has at each decision point, possible outcomes and the payoff a player receives from a given outcome. Although each player is interested in gaining more payoff, the outcome of the game is not merely under his/her control and is dependent on the decisions taken by other players, as well. In this regard, the issue of conflict and cooperation is raised. A two-person zero-sum game is the most straightforward game in which, players make decisions simultaneously, and one player wins what the other one loses. All possible outcomes of play and the corresponding gains of players are known, and there is no room for cooperation. Stackelberg game is another game in which the winning of one player is not equal to the loss of the other player and decisions are not taken simultaneously. Rather, the first player, as the leader, starts the game; and the second player, as the follower, with full knowledge of the decisions taken by the leader, exhibits rational reaction. Analyzing the small instances of these games is easy; however, it becomes more complicated when each player has many strategies. The Stackelberg game can be stated as a bi-level programming problem and allows incorporating some constraints in the strategies taken by players. For more details on game theory, see Straffin (1993). In the following, we briefly describe the formulation and features of the bi-level linear programming problems.

38

2

Main Components of Mathematical Models

Bi-level Models Bi-level problem is an optimization problem in which two decision-makers, called the leader and follower, hierarchically adopt their decisions. The leader controls a set of decision variables through an optimization problem called the upper-level problem, and the follower controls other variables through another optimization problem, called the lower-level problem. The main feature of the bi-level programming problems is that the lower-level problem appears in the constraints set of the upper-level one. The decision-making process is that first, the leader decides in his area of authority. Then, the follower with complete knowledge of the decisions taken by the leader tries to optimize his own objective function. Thus, the leader must make decisions in order to optimize his own objective, while taking into account the response of the follower. Assume that the vectors x and y represent the decisions that are under the control of the leader and the follower, respectively. Some of the components of these vectors may be continuous or discrete. Model 2.8 describes a general bi-level problem with linear constraints, where F 1 ðx, yÞ and F 2 ðx, yÞ, respectively, represent the leader and follower objective functions, and the sets  and  contain constraints defining the type of variables. Model 2.8 A linear bi-level programming problem max F 1 ðx, yÞ ¼ cT x þ dT y x

s. t. Ax + By  b x2 max F 2 ðx, yÞ ¼ c0T x þ d 0T y y

s. t.

A0x + B0y  b0 y2

In general, some of the components of vectors x and y are continuous and others are discrete. Thus, a bi-level programming problem belongs to one of the following four categories: 1. Continuous-continuous: In this case, all components of x and y are continuous. 2. Continuous-discrete: In this case, all components of x are continuous and at least one component of y is discrete. 3. Discrete-continuous: In this case, at least one component of x is discrete and all components of y are continuous. 4. Discrete-discrete: In this case, at least one component of x and one component of y are discrete. Occasionally, it is assumed that for a decision x taken by the leader, the follower’s problem has a unique optimal solution. However, this assumption is not necessarily established in practice, and for a given decision taken by the leader, the problem of follower may have alternative optimal solutions. Each of these solutions has

2.5 Mathematical Models

39

different effects on the objective function value of the leader’s problem. In such cases, the bi-level problem is usually considered under one of the following extra assumptions: Optimistic Bi-level Programming In this case, it is assumed that if for a leader’s decision, the follower’s problem has alternative optimal solutions, the follower shows a friendly and cooperative attitude towards the leader, and in response to the leader’s decision, among all optimal solutions, he chooses the one that is in favor of the leader. Pessimistic Bi-level Programming In this case, it is assumed that if for a leader’s decision, the follower’s problem has alternative optimal solutions, the follower shows an aggressive behavior toward the leader, and in response to the leader’s decision, among all optimal solutions, he chooses the one that is in detriment of the leader. For the bi-level problem presented in Model 2.8, the following sets are defined: The constraint region:  ¼ fðx, yÞ : x 2 , y 2 , Ax þ By  b, A 0 x þ B 0 y  b 0 g The projection of  on to the leader’s decision space: P ðÞ ¼ fx 2  : ∃y : ðx, yÞ 2 g The follower’s feasible region for a fixed x 2 : ðxÞ ¼ fy : y 2 , A 0 x þ B 0 y  b 0 g The Follower’s rational reaction set for a fixed x 2 P ðÞ: ðxÞ ¼ fy^ : y^ 2 ðxÞ, y^ optimizes the objective function F 2 ðx, yÞ g The feasible region of the bi-level problem that is called the inducible region:  ¼ fðx, yÞ : ðx, yÞ 2 , y 2 ðxÞ g For a given value of x, the follower solves the lower  level optimization problem to optimize the objective function F 2 ðx, yÞ: The set ðxÞ defines the optimal solutions that the follower can show in response to the decision x: Suppose that, for every decision x, the set ðxÞ is a singleton set: The leader examines the responses to each solution x 2 P ðÞ: The inducible region , which is a nonconvex set, contains all feasible solutions to the bi-level problem, and the leader must choose the right solution from it. Therefore, the optimal solution of the bi-level problem belongs to  and maximizes the upper  level objective function F 1 ðx, yÞ: Thus, the optimal solution, denoted by ðx , y Þ, satisfies the two following properties:

40

2

Main Components of Mathematical Models

Property 1: ðx , y Þ 2 . Property 2: If ðx^, y^Þ 2 , then F 1 ðx , y Þ  F 1 ðx^, y^Þ. Therefore, Model 2.8 can be rewritten as follows: Model 2.9 An equivalent restatement of Model 2.8 max F1(x, y) s:t: ðx, yÞ 2  Finally, we note that any MIP model with at least one binary variable can be equivalently reformulated in the form of a bi-level model. To clarify the discussion, consider the following problem: max z ¼ cT x þ dT δ s:t: Ax þ Bδ  b x2 δ 2 f0, 1gn2 Where, c, d and b are vectors with n1 , n2 and m components, and A, and B are matrices with dimensions of m  n1 and m  n2 , respectively:The vectors x and δ, respectively, consist of continuous and binary decision variables, and the set  expresses the boundaries and the sign restrictions on the x components. Note that the binary restriction on variable δj (for j ¼ 1, . . ., n2) can be expressed as:   min δj , 1  δj ¼ 0 In this case, the model is equivalent to a bi-level model in which wj (for j ¼ 1, . . ., n2) is a nonnegative continuous variable. Model 2.10 Bi-level model corresponding to an MIP problem max F 1 ðx, δ, w, w0 Þ ¼ cT x þ dT δ x , δ , w , w0 s. t. Ax + Bδ  b x2 w0 ¼ 0 0  δj  1 8 j ¼ 1, . . ., n2 n2 P w0 ¼ max F 2 ðδ, wÞ ¼ wj w

j¼1

s. t. wj  δj 8 j ¼ 1, . . ., n2 wj  (1  δj) 8 j ¼ 1, . . ., n2 wj  0 8 j ¼ 1, . . ., n2

2.6 Solution Methods

41

In this way, in Model 2.10, it is not necessary to explicitly express the binary restriction on variables δj. Because on the one hand, the optimality of the follower’s problem ensures that wj ¼ min (δj, 1  δj) and, on the other hand, the restriction n2 P w0 ¼ 0 in the leader’s problem and the equation w 0 ¼ max wj ensures j¼1

min (δj, 1  δj) ¼ 0, which is equivalent to δj 2 {0, 1}. This reformulation may be useful in developing efficient algorithms for some problems. For example, see Marcotte et al. (2004).

2.6

Solution Methods

Knowledge of mathematical solution methods, algorithms, and their practical limitations play an essential role in creating a suitable model and rapid achievement to the optimal solution. In reality, making a good model, without considering the solution process, is not possible. In this section, we attempt to introduce the popular methods for solving LP, NLP and MILP problems and their field of application. Details can be found in the relevant sources and books.

2.6.1

Solving LP Problems

Simplex Method The George Dantzig’s algorithm (1947), known as the Simplex algorithm, is one of the best methods to solve LP problems. This algorithm solves an LP in the following standard form. Other settings such as the minimization direction, or the constraints of type  or  can easily be converted to the standard form. max z ¼ cT x s:t: Ax ¼ b x0 In this problem,  ¼ fx 2 ℝn jAx ¼ b, x  0g is called the feasible region that is a convex set of feasible solutions. If  is empty, the problem is infeasible. Moreover, if we can increase the value of the objective function over  to any arbitrary value (in the maximization problem) or decrease it to any arbitrary value (in the minimization problem), we face with an unbounded problem. The Simplex algorithm starts with a vertex or an extreme point of the feasible region, and at each iteration, improves the objective function value by moving toward the adjacent vertex, and generating a new basic feasible solution (pivoting).

42

2

Main Components of Mathematical Models

It is proved that if the problem has an optimal solution, there exists at least one optimal extreme point. Moreover, the algorithm converges to the optimal solution after a finite number of iterations. This algorithm always moves along the boundary of the feasible region and produces a basic feasible solution at each iteration. The basic feasible solutions that provide the possibility of post-optimality analysis are very valuable. For more details, see Winston (2003). Although the Simplex algorithm, theoretically, has an exponential time complexity, it shows high performance in most cases of small and medium-sized problems. Because of its capabilities and speed, it is the main part of almost all existing optimization software. Interior Point Method The Karmarkar’s optimization algorithm (1984), known as the interior point method (IPM), is another technique to solve LP problems. This method starts with a point inside the feasible region, and in each iteration, identifies a suitable direction and moves along it in order to improve the solution. The algorithm stops when it sufficiently approaches the optimal solution. Since the solution generated in each iteration is always inside the feasible region, the method is known as the interior point method. For more details, see Winston (2003). Although this method can approach the optimal solution well, it cannot produce a basic feasible solution. Therefore, it is necessary to combine it with another algorithm so that the final solution has the characteristics of a basic feasible solution. Figure 2.3 compares the convergence paths to the optimal solution in the Simplex and IPM methods. Theoretically, interior point method has a polynomial time complexity, and the most critical point of its strength is the number of steps required to reach the final solution, which is almost independent of the size of the problem. In contrast, the computational time spent in each step is high. Therefore, in solving large-scale LPs that the Simplex algorithm is hardly able to solve, the interior point method is very efficient, while in solving small or medium-sized problems, the efficiency of the Simplex algorithm is better. So far, many studies have been devoted to improve the IPM, and provide some versions for NLPs.

Fig. 2.3 Comparing the simplex and interior point methods

2.6 Solution Methods

2.6.2

43

Solving MIP Problems

Solving optimization problems with discrete variables is more complicated and requires special algorithms. In general, the solution methods of these problems are based on three processes of search, inference, and relaxation. The search process examines the feasible region, while inference and relaxation processes play a role in intelligent search guidance. By repeatedly solving the LPR problem (with the Simplex algorithm or the IPM), the method searches in the feasible region and gradually narrows it down until the optimal solution is found or the problem infeasibility is detected. Branch-and-Bound Algorithm The Land and Doig optimization algorithm (1960), known as B&B algorithm, has been presented for solving MIP problems. It can efficiently solve the MIP, PIP, and BIP problems by a systematic and implicit enumeration of all feasible solutions. This method is based on the frequent division of the feasible region and solving the LPR problem. First, the LPR problem is solved. If in the current solution, there is an integer variable holding a fraction of a decimal, two sub-problems are constructed with simple bound constraints and added to the search tree. In this way, sets of sub-problems are gradually identified (search tree), and then, with a direct or indirect consideration (taking into account the best available bound), the sub-problems are removed from the tree. The algorithm terminates when all sub-problems are handled, and then, the best solution is returned (if any). Although the B&B technique is hard to terminate in large scale problems, its capability is suitable for small and mediumsized problems. For details of this method, see Taha (1975). Given the steps of the B&B algorithm, it is evident that in the IP and MIP problems, continuous and discrete variables have different effects on the resolution process. Indeed, the computational efforts often increase exponentially with the number of discrete variables and hence, the number of discrete variables has a great effect on the running time of the solution process. Cutting Plane Algorithm The Gomory optimization algorithm (1950), known as the cutting plane algorithm, is another powerful method for solving MIP, PIP, and BIP problems. In this method, first, the LPR problem is solved, and if there exists a discrete variable taking an inappropriate value (i.e., a fraction of a decimal), a new constraint (cut) is generated and added to the problem, and the problem with the new constraint is resolved. The cut removes the current infeasible LP solution without removing any IP feasible solution. This process continues until the optimal solution is found or it is detected that the problem is infeasible. So far, different methods have been developed for generating strong cuts. Alternatively, the algorithm combining B&B and cutting plane method is called the branch-and-cut (B&C) algorithm. For more details, see Taha (1975). Additive Algorithm Although the B&B algorithm is capable of solving MIP, PIP, and BIP problems, utilization of the model structure plays a significant role in the solution process. The

44

2

Main Components of Mathematical Models

Balas algorithm (1982), known as the additive algorithm or the implicit enumeration algorithm, is a version of the B&B algorithm that is devised for solving zero-one (BIP) problems. In this method, the fact that the value of each variable must be 0 or 1 is used for branching and calculating the proper bound. In this algorithm, an effective method is also provided to recognize the infeasibility of the nodes of the search tree. In this case, every node or branch of the search tree indicates the state (i.e., taking zero or one values) of a variable. The values of variables are gradually determined and fixed. If the fixed variables indicate that a node is infeasible or there is no suitable bound, that node is truncated. Otherwise, two new sub-problems are generated and added to the search tree. The algorithm terminates and returns the best solution as soon as all of the sub-problems are evaluated. For details of this method, refer to Taha (1975).

2.6.3

Solving NLP Problems

Consider the following optimization problem: min z ¼ f ðxÞ

ð2:6Þ

s:t: x 2 

In which, f : n !  is a real  valued function, x ¼ ðx1 , . . . , xn Þ is the vector of decision variables, and  represents the feasible region of the problem and is defined as follows:  ¼ fðx1 , . . . , xn Þ : gi ðx1 , . . . , xn Þ  or  or ¼ 0

8i ¼ 1, . . . , mg

where, gi : ℝn ! ℝ is a real-valued function representing the LHS of the problem constraints or the sign restrictions. Definition 2.1 Global minimum Suppose that x belongs to the set : We say that x is a global minimum point for the objective function f ðxÞ over the set , whenever 8x 2 

f ðx Þ  f ðxÞ

A global minimum solution is a point for which the objective function value is at least as good as the objective function value of any other solution. Definition 2.2 Local minimum Suppose that x belongs to the set :We say that x is a local minimum for function f ðxÞ over the set , if there exists a neighborhood  x around the point x such that:

2.6 Solution Methods

45

  f x  f ð xÞ

  8x 2  \  x

The global and local maximum is defined similarly. Solving a nonlinear problem means to find a global optimal point. However, in practice, the computational complexity is very high so that we confine ourselves to find just a locally optimal solution. Of course, the convex programming problems, which will be described later, are a special category of nonlinear problems in which every locally optimal solution is a global one, as well. Because of this interesting feature of convex programming problems, efficient solution methods have been developed for them. Also, there are efficient algorithms for quadratic programming problems. See Bazaraa et al. (2006) for more details. Nevertheless, in general, the resolution of NLP problems is difficult, and most of the methods are not able to find a global optimal solution and only provide a local optimal solution. As mentioned earlier, in LP problems, at least one of the extreme points of the feasible region is always associated with the optimal solution; however, this property is not valid in NLP problems, and the optimal solution may be an internal or boundary point. Therefore, NLP problems are greatly more challenging to solve, and the methods such as the Simplex algorithm, which only search over the extreme points are not applicable. Definition 2.3 Convex set A set of points   ℝn is convex if for every pair of points, x^, x 2 , the line between x^ and x is entirely inside the . In other words 8 x^, x 2 ,

8 0  α  1,

α^ x þ ð1  αÞx 2 

Figure 2.4 shows a convex set and a nonconvex set in a two-dimensional space. Definition 2.4 Convex function Let   ℝn be a convex set. The real-valued function f : ℝn ! ℝ on convex set  is called convex if for each x^ and x in  we have:

Fig. 2.4 Convex and nonconvex sets

46

2

Main Components of Mathematical Models

f ðα^ x þ ð1  αÞxÞ  αf ð^ xÞ þ ð1  αÞf ðxÞ

80α1

For example, the single-variable functions x21 , x41 , ex1 , ex1 and log x1 are convex on their domain. Definition 2.5 Concave function Let   ℝn be a convex set. The real-valued function f : ℝn ! ℝ on convex set  is called concave if for each x^ and x in  we have: xÞ þ ð1  αÞf ðxÞ f ðα^ x þ ð1  αÞxÞ  αf ð^

80α1

pffiffiffiffiffi For example, the single-variable functions x21 , log ðx1 Þ and x1 are concave on their domain. If f ðxÞ is a convex function,  f ðxÞ is concave and vice versa: In addition, the n P affine function f ðxÞ ¼ aj xj þ b is both convex and concave: Additionally, it can j¼1

be easily verified that if f 1 ðxÞ and f 2 ðxÞ are convex ðconcaveÞ functions and c > 0 is a constant number, then cf 1 ðxÞ þ f 2 ðxÞ is also a convex (concave) function. In Fig. 2.5, four functions are depicted. The first one is convex, the second is concave, the third is neither convex nor concave, and the fourth function is both convex and concave.

Fig. 2.5 Convex and concave functions

2.6 Solution Methods

47

It is easy to see that if f ðxÞ is a convex function over the set   ℝn , and a is a constant number, then the following set is convex:  ¼ fx 2  : f ð x Þ  ag It is clear that the intersection of two convex sets is also a convex set. Definition 2.6 Convex programming problem Consider the following minimization problem: Model 2.11 A convex programming problem min z ¼ f(x) s. t. gi(x)  0 8 i ¼ 1, . . ., m Where the main constraints and the sign restrictions (if any) are considered in the form of gi ðxÞ  0: Define the sets 1 , 2 , . . . , m , as follows: 1 ¼ fx 2 n : g1 ðxÞ  0 g, . . . , m ¼ fx 2 n : gm ðxÞ  0g The problem feasible region is:  ¼ fx 2 ℝn : gi ðxÞ  0 ð8i ¼ 1, . . . , mÞg ¼ 1 \ 2 \ . . . \ m Model 2.11 is a convex programming problem, if the feasible region, , is a convex set, and the objective function f (x) is a convex function on . In other words, we call the Model 2.11 a convex programming problem if gi(x) is a convex function for i ¼ 1, . . . , m, and f (x) is a convex function on the feasible region . Note that the above definition is valid only for an optimization problem that is written in accordance with the standard of Model 2.11 (i.e., minimization direction, and constraints of type ) and for other states (such as maximization direction and constraints of type ¼ or ), the problem must be rewritten in accordance with Model 2.11. For example, suppose that we want to find the conditions under which, the problem below is a convex programming problem. Model 2.12 A programming problem max z ¼ f(x) s. t. gi(x)  bi 8 i ¼ 1, . . ., m hi(x) ¼ 0 8 i ¼ 1, . . ., m0

48

2

Main Components of Mathematical Models

By rewriting Model 2.12 in accordance with Model 2.11, we get: Model 2.13 Restatement of Model 2.12 min z ¼  f(x) s. t. gi(x) + bi  0 8 i ¼ 1, . . ., m hi(x)  0 8 i ¼ 1, . . ., m0 hi(x)  0 8 i ¼ 1, . . ., m0 Model 2.13 is a convex programming problem if the functions gi(x), hi(x) and  hi(x) are convex functions, and on the problem’s feasible region, the function  f(x) is convex. In other words, Models 2.13 and 2.12 are convex programming problems, if the function gi(x) is concave, the function hi(x) is affine and the function f(x) on the problem’s feasible region is concave. Theorem 2.1 Global Optimal Solution of a Convex Problem In a convex programming problem with the aim of minimization, every local minimum is also a global minimum. Proof Refer to Bazaraa et al. (2006). □ Because of this feature, convex programming problems are of great importance, and efficient solution algorithms have been proposed for them that guarantee the achievement of a globally optimal solution. Nevertheless, for many nonlinear problems, it is still not possible to determine the globally optimal solution, and merely, the achievement of global solutions is appreciated. For a complete overview of NLP problems and their solution methods, refer to Bazaraa et al. (2006), Beck (2014) and Boyd and Vandenberghe (2004). In most of the methods developed to solve these problems, the solution process is based on the resolution of several linear problems. In this regard, the use of the optimality conditions proposed by Karush, Kuhn, and Tucker (KKT) is of particular importance. KKT Conditions for NLP Problems Consider an NLP  problem  as Model 2.11. The necessary conditions for a feasible solution x ¼ x1 , . . . , xn 2  to be optimal were first introduced by Karush, Kuhn and Tucker (1951), and are known as KKT or KT conditions. Suppose that the “constraint qualification” property is established for Model 2.11 ðe:g:, the condition that the gradients of the binding constraints are linearly independent at x is one of the many types of assumptions, which are referred to as constraint qualificationÞ: The necessary conditions for x 2  to be an optimal solution to Model 2:11 is that there exists the vector λ ¼ ðλ1 , . . . , λm Þ so that x and λ satisfy the following conditions which are referred to as KKT conditions.

2.6 Solution Methods

49

∂f ðx Þ X  ∂gi ðx Þ þ λi ¼ 0, ∂xj ∂xj i¼1 m

gi ð x  Þ  0 λi

8j ¼ 1, . . . , n

8i ¼ 1, . . . , m

 0, 8i ¼ 1, . . . , m

λi ðgi ðx ÞÞ ¼ 0, 8i ¼ 1, . . . , m

ð2:7Þ ð2:8Þ ð2:9Þ ð2:10Þ

  The vector λ ¼ λ1 , . . . , λm , the ith component of which corresponds to the ith constraint of Model 2.11, is called the vector of dual variables. Note that the above conditions are valid only for an optimization problem that is written in accordance with Model 2.11. Remark 2.4 Constraint qualification is always satisfied if all of the problem constraints are linear. Also, if Model 2.11 is a convex programming problem, the KKT conditions will be sufficient for optimality. Of course, the KKT condition is rarely used directly to determine the optimal solution, and it is often applied in an iterative process to approximate the optimal solution. We conclude this discussion by mentioning some points about the KKT conditions for LP problems. Consider an LP problem as follows: Model 2.14 A linear programming problem n P min z ¼ ci, j xj j¼1

s. t. n P ai , j x j  bi

8i ¼ 1, . . . , m

j¼1

xj  0 8 j ¼ 1, . . ., n The KKT conditions for the above problem can be rewritten in the form of the following relations in which λi is the dual variable corresponding to the ith constraint (i ¼ 1, . . ., m). The relations (2.11), (2.12) and (2.13), respectively, correspond to the constraints of the primal problem, the constraints of the dual problem and the complementary slackness conditions. n P

ai, j xj  bi

8i ¼ 1, . . . , m

j¼1

xj  0 m P

ai, j λi  cj

8j ¼ 1, . . . , n

i¼1

λi  0

Primal feasibility conditions

ð2:11Þ

Dual feasibility conditions

ð2:12Þ

8j ¼ 1, . . . , n

8i ¼ 1, . . . , m

50

λi

2 n P

!

xj

¼0

ai , j x j  bi

j¼1 m P

Main Components of Mathematical Models

8i ¼ 1, . . . , m

! ai, j λi  cj

Complementary slackness conditions ¼ 0 for all j ¼ 1, . . . , n

i¼1

ð2:13Þ

2.6.4

Solving Multi-objective Problems

As mentioned in Sect. 2.5.3, the general form of a multi-objective problem is as follows: Model 2.15 A multi-objective problem max f(x) ¼ ( f1(x), f2(x), . . ., fk(x)) s:t: x 2  In this section, some solution methods for this problem are introduced. First Method: Prioritization of Objectives In this method, the decision-maker needs to rank his goals from the most important to the least one. Suppose that in Model 2.15, the objectives are arranged as f1(x), f2(x), . . ., fk(x). Then, first, by solving the following problem, the most important goal is optimized: max f 1 ðxÞ s:t: x 2  Subsequently, assume that the optimal objective value of the above problem is equal to f 1 , among all of the feasible solutions optimizing the first goal, the solution that is the best for the second objective is searched. To this end, the following singleobjective model is solved: max f 2 ðxÞ s:t: x2 f 1 ðxÞ ¼ f 1 Now, assuming that the optimal objective value of the above problem is equal to f 2 , among all of the feasible solutions x, satisfying f 1 ðxÞ ¼ f 1 and f 2 ðxÞ ¼ f 2 , the one that is the best for the third objective is searched. This process continues similarly, and at the end, the solution obtained by the last model is introduced as a

2.6 Solution Methods

51

Pareto solution to the multi-objective problem. Note that this method is useful when the single-objective model that is solved in each step has alternative optimal solutions. Second Method: The Weighted Sum Method In this method, with each objective function fi(x), a positive weight ρi is associated. Then, a Pareto solution can be achieved by solving a single-objective model in which the new objective function is a weighted summation of all original objectives: max

k X

ρi f i ð x Þ

i¼1

s:t: x 2  Of course, the unit of measurement of the functions fi(x) are not necessarily the same for i ¼ 1, . . ., k. For example, one goal may represent profit, while the other may represent the degree of customer satisfaction. In this case, their summation is not correct, and the quantities must be normalized. For this purpose, we can define f i 6¼ 0 as the maximum value of f i ðxÞ ðif for every feasible solution x we have k P f i ðxÞ  0Þ, and rewrite the model objective function as max ρi f ifðxÞ. i¼1

i

Third Method: Goal Programming In this method, with each objective function f i ðxÞ a target level denoted by t i is associated to represent the value of f i ðxÞ that the decision-maker tends to achieve (e.g., the target of earning a profit of 100 (mu)). Since in practice, the decision-maker may not achieve the desired level, nonnegative variables sþ i and th are introduced to measure the deviation from the target value of the i goal. In s i this case, a solution can be obtained by solving a single-objective model, in which the weight ρi is a positive value indicating the importance of the target i. The objective function minimizes the weighted amount of deviations from the target values. min

Xk

ρ i¼1 i



 sþ i þ si



s:t: x 2    f i ð xÞ  s þ i þ si ¼ t i 8i ¼ 1, 2, :::, k  sþ i , si  0 8i ¼ 1, 2, :::, k

In the above model, attention is required for some points. First, the variables representing the deviation from targets have not the same unit of measurement for all þ   goals. For example, sþ 1 and s1 may denote profit, while s2 and s2 may represent the degree of customer satisfaction. In this case, their summation þ is meaningless and,  it is necessary to first normalize the terms si þ si : For this purpose,

52

2

Main Components of Mathematical Models

assuming that t i > 0, the objective function can be restated as min

k P ðsþi þsi Þ i¼1

t i

.

The second point is that deviations can be considered on one-side, as well. For example, a decision-maker might tend to reach a profit of 100 (mu), but if his profit exceeds 100 units, he does not hurt. Therefore, it is sufficient to only penalize the profit reduction (i.e., the variable s i ) in the objective function. The third point is about the method of determining the target levels. These levels can be either determined by decision-makers according to their knowledge of the problem, or obtained by solving the following single-objective model for i ¼ 1, . . ., k: max f i ðxÞ s:t: x 2 

2.6.5

Dynamic Programming to Solve Optimization Problems

Bellman first introduced dynamic programming in 1954 as a method applicable to solve a variety of optimization problems. This method breaks the problem down into simpler sub-problems in a recursive manner, and by consecutive resolution of sub-problems, the optimal solution to the original problem is found. There is no standard framework regarding the implementation of this method; instead, the creativity of persons plays an important role. The detailed description of the dynamic programming method is out of the scope of this book, and the interested reader is referred to Lew and Mauch (2006).

2.6.6

Solving Bi-level Problems

As described in Sect. 2.5.9, bi-level problems are referred to as a category of problems that two decision-makers (leader and follower) decide on their own interests. Mathematically, a bi-level programming problem is a nonconvex NPhard problem (Jeroslow 1985). So far, various methods have been proposed to solve bi-level problems that are not explicitly discussed in this book, and the interested reader is referred to Colson et al. (2007), Dempe (2002) and Lu et al. (2016). Here, we describe only a KKT-based method for a particular type of problems, in which the bi-level problem can be rewritten as a single-level programming problem. The general form of a bi-level programming problem with linear constraints and objective function in the continuous-continuous or discretecontinuous modes is as follows:

2.6 Solution Methods

53

Model 2.16 A bi-level model n n0 P P ck xk þ d j yj max F 1 ðx, yÞ ¼ x

j¼1

k¼1

s. t. n P

ai, k xk þ

n0 P

bi, j yj  bi

8i ¼ 1, . . . , m

(2.14)

j¼1

k¼1

x2

(2.15)

max F 2 ðx, yÞ ¼ y

n P k¼1

s. t. n P k¼1

ai0, k xk þ

yj  0

n0 P j¼1

c0k xk þ

n0 P j¼1

bi0, j yj  bi0

d 0j yj

8i ¼ 1, . . . , m 0

8 j ¼ 1, . . ., n0

For the follower’s problem, the vector x is assumed to be fixed and hence, only the components of y are considered as decision variables. Therefore, by considering λ as the vector of dual variables of the follower’s problem, with respect to the KKT conditions for LP problems, the follower’s problem can be substituted by the following constraints: 8 n0 n P 0 P > > bi, j yj  b0i  a0i, k xk > < j¼1

> > > :

k¼1

8i ¼ 1, . . . , m0 yj  0 8j ¼ 1, . . . , n0

Primal feasibility conditions

ð2:16Þ

8 m0 > < P bi0, j λi  d j0 8j ¼ 1, . . . , n 0 Dual feasibility conditions ð2:17Þ i¼1 > : λi  0 8i ¼ 1, . . . , m 0 8 ! n0 n > P P > 0 0 > bi , j y j  bi þ ai0, k xk ¼ 0 > λi > > j¼1 k¼1 < Complementary slackness conditions 8i ¼ 1, . . . , m 0 > m  > > > P > > bi0, j λi  dj0 ¼ 0 8j ¼ 1, . . . , n 0 : yj i¼1

ð2:18Þ In this way, we get a single-level problem equivalent to Model 2.16. Model 2.17 A single-level model equivalent to Model 2.16 n n0 P P max z ¼ ck xk þ d j yj k¼1

s. t. (2.14)–(2.18)

j¼1

54

2

Main Components of Mathematical Models

In this problem, the constraints corresponding to complementary slackness conditions are nonlinear. Due to the difficulty of solving nonlinear problems, it is desirable to replace nonlinear constraints (2.18) with appropriate linear constraints. In Sect. 3.6.4, the method of linearization of such constraints is described. Example 2.8 Resolution of a continuous-continuous bi-level problem with KKT conditions In the following bi-level programming problem, x is under the control of the leader and y is controlled by the follower. Using the KKT conditions, formulate and solve the problem as a single-level problem. Model 2.18 A continuous-continuous bi-level problem max F 1 ðx, yÞ ¼ x þ 4y x

s. t. x0 max F 2 ðyÞ ¼ y y

s. t. xy 3  2x + y  0 2x + y  12  3x + 2y   4 y0 Answer: Let λ1, . . ., λ4 represent the dual variables associated with the first to the fourth constraints of the follower’s problem, respectively. The following single-level model is equivalent to Model 2.18: Model 2.19 A single-level model equivalent to Model 2.18 max z ¼  x + 4y s. t. x0 x  y  3 Primal feasibility conditions 2x þ y  0 2x þ y  12 3x þ 2y  4 y0 λ1 þ λ2 þ λ3 þ 2λ4  1 λ1 , λ2 , λ3 , λ4  0 (x  y + 3)λ1 ¼ 0 (2x + y)λ2 ¼ 0 (2x + y  12)λ3 ¼ 0 (3x + 2y + 4)λ4 ¼ 0 (λ1 + λ2 + λ3 + 2λ4 + 1)y ¼ 0

Dual feasibility conditions Complementary slackness conditions

(2.19) (2.20) (2.21) (2.22) (2.23)

2.6 Solution Methods

55

The above model is nonlinear due to the complementary slackness conditions. In Sect. 3.6.4, the method of linearization of these constraints is described (see Example 3.10). By solving the linearized model, the optimal solution to the bi-level problem is obtained as (x, y) ¼ (4, 4), for which the objective function value associated with the first and the second levels are equal to 12 and 4, respectively.

2.6.7

Solving Problems with Interval Coefficients

Consider an LP problem with interval coefficients as follows in which each interval is a subset of real numbers. Note that the problem contains no equality constraints and there are not any free variables. min z ¼

X h

i cj , cj xj

j21 [2

s:t: h X j21 [2

xj  0 xj  0

i h i ai, j , ai, j xj  bj , bj

8i 2 

8j 2 1 8j 2 2

In such problems, the decision-maker is interested in to identify the best and the worst optimal solutions of the model and the settings of the interval coefficients that yields these two solutions. In the following, we introduce two LP models able to identify the best and the worst performances. Since the variable xj is either nonnegative or nonpositive, we define the following parameters: ( a0i, j ¼ ( c0j

¼

ai , j

j 2 2

ai , j

j 2 1

cj

j 2 1

cj

j 2 2

8i 2 , 8j 2 1 [ 2 8j 2 1 [ 2

The best optimal solution is obtained by solving the following model: min z ¼

X j21 [2

s:t: X j21 [2

c0j xj

a0i, j xj  bj

xj  0 xj  0

8j 2 1 8j 2 2

8i 2 

56

2

Main Components of Mathematical Models

Similarly, consider the following parameters: ( a00i, j

¼ (

c00j ¼

ai, j ;

j 2 1

ai, j ;

j 2 2

cj ;

j 2 2

cj ;

j 2 1

8i 2 , 8j 2 1 [ 2 8j 2 1 [ 2

The worst optimal solution is obtained by solving the following model: min z ¼

X j21 [2

s:t: X j21 [2

c00j xj

a00i, j xj  bi

xj  0

8j 2 1

xj  0

8j 2 2

8i 2 

It is easy to show that the optimal objective values of the models created by different choices for specific values for the interval coefficients are in the range ½ z, z : In other words, there is no optimal solution with an objective value less than z or greater than z. In addition, if the model associated with the best optimal solution is infeasible, then the model associated with the worst optimal solution is also infeasible. Furthermore, if the model associated with the worst optimal solution is unbounded, then the model associated with the best optimal solution is unbounded, as well. For more details see Chinneck and Ramadan (2000) and Ramadan (1996). Example 2.9 A problem with interval coefficients Consider the following problem with interval coefficients. Determine the best and the worst optimal solutions. min z ¼ ½0:5, 2x1 þ ½1, 3x2 s:t: ½3, 4x1 þ ½2, 1:5x2  ½4, 3:5 ½4, 5x1 þ ½16, 17x2  16 6x1 þ 3x2  ½8, 12 x1 , x2  0 Answer: By solving the following model, the best optimal solution is obtained as x1 ¼ 1.0115, x2 ¼ 0.6437 and z ¼ 1:149425.

2.8 Exercises

57

min z ¼ 0:5x1 þ x2 s:t: 4x1  1:5x2  4 5x1 þ 17x2  16 6x1 þ 3x2  8 x1 , x2  0 By solving the following model, the worst optimal solution is obtained as x1 ¼ 1.7143, x2 ¼ 0.5714, and z ¼ 5:142857. min z ¼ 2x1 þ 3x2 s:t: 3x1  2x2  3:5 4x1 þ 16x2  16 6x1 þ 3x2  12 x1 , x2  0

2.7

Notes and References

In this chapter, the main components of an optimization model and a precise description of solution methods were described. For more details see Appa et al. (2006), Bertsimas and Weismantel (2005), Bradley et al. (1977), Castillo et al. (2002), Chen et al. (2010), Conforti et al. (2014), Rao (1978), Sarker and Newton (2007), Schrijver (1998) and Wolsey and Nemhauser (1999). Most of the exercises addressed in Sect. 2.8 have been derived from Boyd and Vandenberghe (2004), Ramadan (1996) and Winston (2003).

2.8

Exercises

Exercise 2.1 Provide appropriate linear constraints for each of the following cases: (a) Assume that nonnegative continuous variables x1 and x2 represent the production level of the products 1 and 2, respectively. Write a constraint to guarantee that for each unit of product 1 produced, at least 5 units of product 2 should be produced.

58

2

Main Components of Mathematical Models

(b) A device is capable of producing three types of products. Suppose that every day this device is launched, it can produce only one type of product. If on a given day, the machine is launched for the production of type 1, it produces 11 units during the day. However, if it is requested for the production of types 2 and 3, respectively, 45 and 30 units are produced during the day. Assume that x1, x2 and x3 are nonnegative continuous variables representing the amount of productions of types 1, 2, and 3 over a period of 10 days. What is the relation between x1, x2 and x3? (c) Assume that the nonnegative continuous variables x1 and x2 represent the production levels of the products 1 and 2, respectively. Write a constraint to ensure that for 9 units of product 2 produced, exactly 7 units of product 1 should be produced. (d) Assume that the nonnegative continuous variables x1, x2 and x3 represent the production level of products 1, 2, and 3, respectively. Provide a constraint ensuring that the production level of product 1 must be at least 33% of the total production of products 1, 2 and 3. (e) Whenever a machine is utilized for 1 h, 24.5 units of product 1 and 73.1 units of product 2 are produced. Assume that the nonnegative continuous variable x denotes the machine working hours, and the nonnegative continuous variables x1 and x2 represent the production levels of the products 1 and 2, respectively. Provide constraints to show the relation between these variables. Now, suppose that at each hour, the machine could be used to produce either 24.5 units of product 1 or 73.1 units of product 2. Let the nonnegative continuous variable x and x0 denote the machine working hours spent to produce the products 1 and 2, respectively. Moreover, assume that the nonnegative continuous variables x1 and x2 represent the production levels of the products 1 and 2, respectively. Provide constraints to show the relation between these variables. (f) When a warehouse is built, at least 56 tons of goods must be stored, and the volume of goods in stock cannot exceed 141 tons. Suppose that δ is a binary variable that is 1 if the warehouse is constructed; otherwise 0. Moreover, let x be a nonnegative continuous variable representing the amount of goods stored in the warehouse. Provide a constraint to describe the relation between these variables. Exercise 2.2 A company produces only one type of product. At the end of each month, after the sale is made, its leftover is stored in the warehouse. Define the following continuous variables: xt : yt, k: wt, k:

The number of units produced during month t The number of units that have produced k months before the current month, t, and are left the warehouse during the month t (for sale). The number of units that have produced k months before the current month, t, and are kept in the warehouse during the month t.

Provide constraints to calculate inventory level at the end of each month.

2.8 Exercises

59

Exercise 2.3 Does a minimization LP problem with all constraints of type  show an unbounded problem? Exercise 2.4 The feasible region of a binary programming problem is as follows: n o ðδ1 , δ2 , . . . , δ5 Þ 2 f0, 1g5: 5δ1 þ 2δ2  3δ3  δ4 þ 4δ5  6 Which of the following cuts are valid for this problem? ðaÞ δ1 ¼ 1

ðbÞ δ3 ¼ 0

ðcÞ δ1 þ δ2 þ δ5  2

ðdÞ δ3 þ δ4  1

Exercise 2.5 Let  and  be the sets of persons and jobs, respectively, and suppose that cj, i represents the cost of assigning job j to person i. To each job, exactly one person, and to each person, exactly one job should be assigned so that the total cost is minimized. Provide a mathematical model for this problem, which is known as an assignment problem, and compare its feasible region with that of TSP. Exercise 2.6 A distribution center utilizes a fleet of K homogenous vehicles with capacity Q to deliver goods from depot to customers: Consider the set  ¼ f0g [  ðindexed by i, jÞ, where i ¼ 0 denotes the depot and  is the set of customers: The demand of customer i 2 , denoted by di , should be observed by exactly one vehicle and in one visit: The travel time between every pair of customers as well as between the depotto each customer is known and denoted by t i, j : Each vehicle should start the travel from the depot, visit some customer and, return to the depot and the total travel time cannot exceed the limit T max , which is interpreted as the driver is working hour: The fixed cost of utilizing each vehicle ðinterpreted as driver wageÞ and the travel cost ðinterpreted as the cost of consumed fuel the cost of consumed fuelÞ are known and denoted by c and c0i, j , respectively. The aim is to decide on the number of required vehicles and determine the sequence of customers visited by each vehicle. This problem in known as vehicle routing problem (VRP). Provide a model and compare it with the model of TSP. Exercise 2.7 Answer the following questions: (a) On what interval, the function f(x) ¼ 12x + x3 is convex and on what interval it is concave ? Draw the function for  3  x  3 and determine its local and global optimal solutions on this interval. (b) Which of the following functions is, convex, concave, or neither convex nor concave?

60

2

Main Components of Mathematical Models

1 x1

f ðx1 Þ ¼ log ðx1 Þ x1 > 0 ( 0 x1  0 f ðx1 Þ ¼ x21 x1 > 0

f ð x1 Þ ¼

x1 > 0

f ðx1 , x2 Þ ¼ x21 þ x22 f ðx1 , x2 Þ ¼ x21  x22

f ð x1 , x2 Þ ¼ x1 x2 f ðx1 , x2 Þ ¼ x2  jx1 j

f ðx1 Þ ¼ ex1 2

f ðx1 , x2 , x3 Þ ¼ 4x21 þ 3x22 þ 5x23 þ 6x1 x2 þ x1 x3  3x1  2x2 þ 15 (c) Which of the following sets is convex?   

ðx1 , x2 Þ 2 ℝ2 : x21 þ x22  4 ðx1 , x2 Þ 2 ℝ2 : x21 þ x22 ¼ 4 ðx1 , x2 Þ 2 ℝ2 : x21 þ x22  4



fx 1 2 ℝ : x 1  j x 1 j  0g



fx 1 2 ℝ : x 1  j x 1 j  0g  ðx1 , x2 Þ 2 ℝ2 : x1 x2  4, x1  4, x2  4 ðx1 , x2 Þ 2 ℝ2 : x1 x2  25, x1  x2  10, x2  7, x1 , x2  0  ðx1 , x2 Þ 2 ℝ2 : jx1  x2 j  4, x1 þ x2  10, x1  2



(d) Show that the feasible region of an LP problem is a convex set. Exercise 2.8 By using the KKT conditions, find the optimal solution to the following NLP problems. ð aÞ min z ¼ ðx  1Þ2

ðbÞ min z ¼ x1  x2

s:t: x2

s:t: x21 þ x22  1

ðcÞ max z ¼ s:t:

ðdÞ x21

x1 þ x2  6 x1  3 x2  4 x1 , x 1  0



x22

þ 4x1 þ 6x2

min z ¼ z ¼ ex1 þ ex2 s:t: x1 þ x2  1 x1 , x1  0

2.8 Exercises

61

Exercise 2.9 Answer the following questions: (a) Let f ðx1 , . . . , xn Þ be a convex function on the set   ℝn :Show that if λ is a positive scalar, then λf ðx1 , . . . , xn Þ is a convex function over , and if it is a negative scalar, λf ðx1 , . . . , xn Þ is a concave function. (b) Assume that λ is a positive scalar and f ðx1 , . . . , xn Þ and gðx1 , . . . , xn Þ are convex functions over   ℝn : Show that f ðx1 , . . . , xn Þ þ λgðx1 , . . . , xn Þ is a convex function on . Exercise 2.10 Answer the following questions: (a) Show that for every natural number n 2 ℕ, f(x) ¼ x2n is a convex function on ℝ. However, f(x) ¼ x2n+1 is a convex function over the set of nonnegative real numbers and it is concave over the set of nonpositive real numbers. (b) Show that for every scalar a 2 ℝ, the function f(x) ¼ eax is a convex function over ℝ. (c) Show that f(x) ¼ xa is convex over the set of positive real numbers when a  1 or a  0, and concave for 0  a  1. (d) Show that f(x) ¼ |x|p, for p  1, is a convex function on ℝ. (e) Show that f(x) ¼ log (x) is a concave function over the set of positive real numbers. (f) Show that every norm function over ℝn is convex. Exercise 2.11 Answer the following questions: (a) Suppose that the functions hðxÞ and gðxÞ are convex on   ℝ: Show that f ðxÞ ¼ max fgðxÞ, hðxÞg is also convex on . (b) Assume that h(x) is convex and nondecreasing and g(x) is convex. Show that f(x) ¼ hog(x) is convex. (c) Assume that h(x) is convex and nonincreasing and g(x) is concave. Show that f(x) ¼ hog(x) is convex. (d) Assume that h(x) is concave and nondecreasing and g(x) is concave. Show that f(x) ¼ hog(x) is concave. (e) Assume that h(x) is concave and nonincreasing and g(x) is convex. Show that f(x) ¼ hog(x) is concave. Exercise 2.12 A company produces two products. The relevant information for each product is provided in Table 2.3. The first goal is to achieve at least 48 (mu) of profit, and a penalty of 1 (mu) is incurred for each unit the profit falls short of this goal. The second goal is to utilize a total of 32 hours of available labor, and a penalty of 2 (mu) is incurred for each hour of overtime (labor over 32 hours); furthermore, a penalty of 1 (mu) is imposed for each hour of available labor that is unused. The third goal is to produce at least 10 units of each product, and a penalty of 5 (mu) is

62

2

Table 2.3 Data for Exercise 2.12

Main Components of Mathematical Models

Product Labor required (hour) Profit (mu)

1 4 4

2 2 2

incurred for each unsatisfied demand. What approach can be used to solve this multiobjective problem? Exercise 2.13 Consider the following two-objective problem: max f ðx1 , x2 Þ ¼ ðf 1 ðx1 , x2 Þ, f 2 ðx1 , x2 ÞÞ s:t: x1  1:5 x2  1:5 x1 þ x2  2 x1 , x2  0 Where f1(x1, x2) ¼ x1 and f2(x1, x2) ¼ x2. Draw the problem feasible region and find all Pareto solutions. Exercise 2.14 Consider the multi-objective programming problem as expressed in Model 2.15. (a) Show that the solution obtained by the prioritization of goals is a Pareto solution. (b) Show that if each goal is associated with a positive weight, the solution obtained by the weighted sum method is a Pareto solution. If some of the weights are zero, is the solution necessarily Pareto? (c) Is it possible to identify all of the Pareto solutions by the weighted sum method? Exercise 2.15 Consider the following two-objective problem: max f ðx1 , x2 Þ ¼ ðf 1 ðx1 , x2 Þ, f 2 ðx1 , x2 ÞÞ 8i ¼ 1, . . . , m s:t: gi ðx1 , x2 Þ  0 Assume that the optimal solution to the single-objective problem associated with the objective function f 1 ðx1 , x2 Þ is not unique and let ðx1 , x2 Þ be an alternative optimal solution: How can we determine whether or not ðx1 , x2 Þ is a Pareto solution? Exercise 2.16 Consider the following problems with interval coefficients. Show that for any choice of coefficients, the optimal objective value for (a) and (b) lies in the intervals [35,60.3333], and (1, 4], respectively.

2.8 Exercises

63

ðaÞ max z ¼ 4x1 þ ½8, 12x2

ð bÞ min z ¼ x1 þ x2

s:t:

s:t:

6x1 þ ½4:25, 5:75x2  30 ½1, 4x1  2x2  2 x1 þ ½1, 2x2  3 ½0:95, 1:05x1  3 x2  ½3:6, 4:4 x1 , x2  0

x1 , x2  0

Exercise 2.17 Consider the following problem with interval coefficients. Show that the problem corresponding to the best optimal solution is feasible while there is no solution for the worth case problem. min z ¼ x1  x2 s:t: ½3, 1:5x1 þ x2  3 x1 þ ½3, 1:5x2  3 x1  4 x2  4 x1 , x2  0 Exercise 2.18 Show that the following problem is infeasible. min z ¼ 2x1 þ 3x2 s:t: ½2, 4x1 þ x2  4 3x1 þ 2x2  ½7, 8 x2  2 x1 , x2  0 Exercise 2.19 Consider the following problem. The best and the worth optimal solutions are (0.75, 2.5) and (1.25,1.5), respectively (see Fig. 2.6).

64

2

Main Components of Mathematical Models

Fig. 2.6 The best and the worst optimal solutions

min z ¼ x2 s:t: ½4, 2x1 þ 2x2  2 2x1 þ x2  ½4, 4 x1 , x2

free

Let x1 ¼ x10  x100 and x2 ¼ x20  x200 where x10 , x10 0 , x20 , x20 0 are nonnegative continuous variables. Rewrite the problem as: min z ¼ x02  x002 s:t: ½4, 2x01 þ ½2, 4x001 þ 2x02  2x002  2 2x01  2x001 þ x02  x002  ½4, 4 x01 , x02 , x001 , x002  0

Why are not the best and the worth optimal solutions of these two problems the same? Modify the method for problems having free variables. For more details see Chinneck and Ramadan (2000) and Ramadan (1996).

2.8 Exercises

65

Exercise 2.20 Consider the following problem: min z ¼

i Xh cj , cj xj j2

s:t: h i h i X ai, j , ai, j xj  bj , bj 8i 2  j2

xj 2 f0, 1g

8j 2 

Each number in the interval [α, β] can be represented as λα + (1  λ) β, where 0  λ  1. Is it possible to obtain the best optimal solution by solving the following model? min z ¼

X

   λ j cj þ 1  λ j cj xj j2

s:t:

 



  X λ0i, j ai, j þ 1  λ0i, j ai, j xj  λ00j bj þ 1  λ00j bj 8i 2  j2

xj 2 f0, 1g, 0  λj  1 0 0

λ0i, j  1 λ00i  1

8j 2 

8i 2 , 8j 2  8i 2 

Exercise 2.21 Compare the IPM and the Simplex algorithm in terms of the complexity. Explain which one is more appropriate for LP problems. Why does the Simplex algorithm, despite having exponential complexity, not lose its desirability and is used in many optimization tools? See Bertsimas and Tsitsiklis (1997) for more details. Exercise 2.22 Answer the following questions: (a) Compare the B&B, the B&C, and the branch and price algorithms with each other. For more details on these methods see Chen et al. (2010) and Taha (1975). (b) What is the difference between the Gomory cut and the Chvatal-Gomory cut in the cutting plane algorithm? For more details on these methods see Chen et al. (2010) and Taha (1975). Exercise 2.23 One way to solve the following unconstrained optimization problem is the steepest descent method. Investigate the method and its convergence conditions. See Bazaraa et al. (2006) and Winston (2003) for more details.

66

2

Main Components of Mathematical Models

min z ¼ f ðx1 , . . . , xn Þ ð x1 , . . . , xn Þ 2 ℝ n Exercise 2.24 Answer the following questions: (a) Quadratic Programming (QP) problem is a class of NLP problems in which the objective function is a polynomial function in one or more variables in which the terms are at most of degree 2, and all constraints are linear. Wolfe’s algorithm is one of the well-known methods to solve QP problems based on the KKT conditions (Bazaraa et al. 2006; Winston 2003). Explore the details of this algorithm. (b) A company uses raw material to produce two types of products. The cost of purchase and process of each unit of raw material is 5 (mu) When processed, each unit of raw material yields two units of product 1 and one unit of product 2. The sale price of each product depends on its production level. Each unit of product 1 and product 2 can be sold at prices of 49  x1 (mu) and 30  x2 (mu), respectively, where x1 and x2 represent the production level of the first and the second products. Formulate this problem as a QP. Exercise 2.25 Another method for solving a multi-objective programming problem is the ε-constraint method (Ehrgott 2005). Explore this method.

Chapter 3

Models and Mathematical Logic

3.1

Introduction

Mathematics like any other branches of science has its rules, and the mathematical logic is committed to figure out the instructions and study the logical structure. In this regard, the mathematical logic or the symbolic logic employs a set of rules, principles, and directions that allows us to decide what to do or not. Roughly speaking, the mathematical method is a set of symbols that are used to refer to mathematical objects and acts. Obviously, the higher the level of utilizing mathematical methods, the better the results and understanding we get. Regardless of the logic definition, the development of a proper language is certainly the point of progress of mathematical activities. It is well known that discipline is embodied in thinking in the form of language. On the other hand, everything that does not follow in the form of language is not within the realm of mathematics. The symbols of such language are virtually invented and make it different from the natural language, but at the same time, the choice and invention of these symbols are not baseless and are made by analyzing the natural language and its components. Therefore, by analyzing and using mathematical language, we try to construct a mathematical model that is supported by math’s language and is free of contradictions. These are concepts that are more often used in mathematical models. Model building is one of the creative aspects of operations research. In complex and real-world problems without using a systematic and methodological process, it is not possible to construct a precise and acceptable model. On the one hand, we deal with concepts that must be defined accurately; on the other hand; we should arrange the decision-making process simple and straightforward. In this regard, mathematical logic plays a fundamental role as a powerful tool and provides scientific rules and methods to achieve this goal. One of the problems of beginners in this field is how to understand and define the scientific concepts, especially theoretical notions. Because they often have to be imagined and the mind is the main factor in © Springer Nature Switzerland AG 2019 S. A. MirHassani, F. Hooshmand, Methods and Models in Mathematical Programming, https://doi.org/10.1007/978-3-030-27045-2_3

67

68

3

Models and Mathematical Logic

understanding their characteristics. Fortunately, by practicing and thinking, the mind is also able to understand theoretical concepts. Modeling as an abstraction of a real phenomenon requires rational reasoning and inference. To this end, we introduce and define some of the commonly used concepts and terminologies in this field.

3.2

Atomic Propositions

Mathematicians intend to prove the truth or falsity of sentences with arguments. Thus, these sentences should be able to accept false or true attributes. In this regard, we introduce one of the most fundamental concepts of mathematical logic, that is, atomic or simple proposition. An atomic proposition is a simple statement that accepts one and only one of the two attributes “True” and “False”. Each of the following statements is a proposition: • Purchase shares have a budget limitation. • If the plant is to be built, production will be possible. • This product will be produced at a maximum of 200 units if at least 50 units are requested.

3.3

Composite Propositions

A proposition consisting of a combination of atomic propositions is called a composite proposition. For example, the statement “π is larger than 3 and smaller than 4” is a composite proposition which is made up of two simple propositions, p: “π is smaller than 4” and, q: “π is larger than 3”. In general, propositions can be combined in different ways. The negation of a proposition is also a proposition itself. Moreover, simple propositions can be put together and produce composite propositions in the form of “conjunction”, “disjunction”, “if-then”, and “if and only if” combinations. These compounds can also be used again to create more complex propositions. Let p and q are two propositions, some of the main combinations are provided in Table 3.1. Many of the other propositions are made by repeating and combining these methods. It is clear that the truth or falsity of any composite proposition depends Table 3.1 Composition of simple propositions

# 1 2 3 4 5 6

Name Negation Conjunction Disjunction Implication Equivalence Exclusive disjunction

Description Not p p and q p or q If p then q p iff q p or q not both

Math’s symbol ~p p^q p_q p)q p,q L p q

3.3 Composite Propositions Table 3.2 The value of composite propositions

69 p T T F F

q T F T F

~p F F T T

p^q T F F F

p_q T T T F

p)q T F T T

p,q T F F T

p F T T F

L

q

Table 3.3 Some theorems # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Description Idempotence Property of F Property of T Absorption Absorption Equivalency Equivalency Equivalency Commutativity Associativity Associativity Distributivity Distributivity Contrapositive DeMorgan’s Laws DeMorgan’s Laws Implication Implication Implication Implication Implication Bi-implication

Proposition p_pp p_Fp p_TT p _ (p ^ q)  p p ^ (p _ q)  p (p _ q  q) , (p ) q) (p ^ q  p) , (p _ q  q) p ⨁ ~q  ~(p ⨁ q)  ~p ⨁ q p_qq_p p _ (q _ r)  (p _ q) _ r L (p q) ⨁ r  p ⨁ (q ⨁ r) p _ (q ^ r)  (p _ q) ^ (p _ r) p ^ (q _ r)  (p ^ q) _ (p ^ r) (p ) q)  (~q ) ~p) ~(p _ q)  ~p ^ ~q ~(p ^ q)  ~p _ ~q (p ) q)  ~p _ q (p ) (q ^ r))  (p ) q) ^ (p ) r) (p ) (q _ r))  (p ) q) _ (p ) r) ((p ^ q) ) r)  (p ) r) _ (q ) r) ((p _ q) ) r)  (p ) r) ^ (q ) r) (p , q)  (p ) q) ^ (~p ) ~q)

on the correctness or incorrectness of its constructive propositions. Table 3.2 expresses the value of the above statements in terms of the propositions that make up the statement. Usually, we show a proposition that is always True with ‘T’ or 1, and a proposition that is always False with ‘F’ or 0. We also consider the two propositions equivalent (or “”) if their values are the same. Any proposition that has been proved to be true is a theorem. In other words, the proof of a proposition is a convincing argument of its correctness in an acceptable way. By assuming that p, q and r are simple statements, one can easily deduce the theorems provided in Table 3.3.

70

3.4

3

Models and Mathematical Logic

Propositional Functions

A propositional function is a sentence in which there is a variable x with unspecified value and by placing a value instead of x, it is converted into a proposition with the value of True or False. For example, “x belongs to the set of nonnegative real numbers”, is a propositional function that by placing a value instead of x converts into a proposition, True or False. In most mathematical models, we deal with propositional functions that we need to make sure that they are True or False with respect to the value attributed to the variables, and consequently make a decision with respect to the results obtained.

3.5

Propositions and Binary Variables

We often find equalities or inequalities in the mathematical models that we cannot properly understand their meaning. However, we want to know what role they play in the model and, more importantly, how these relations are achieved. For example, consider the following equation that links the binary variables δ1, δ2, δ3, δ4. δ1  δ2 þ δ3 þ δ4  2 What is the meaning of this constraint? How is it obtained? It is easy to see that when δ1 ¼ 1, δ2 ¼ 0, δ3 ¼ 1, then, the above inequality is converted to δ4  0, which is equivalent to δ4 ¼ 0. Therefore, it can be concluded that this is an if-then logical proposition that can be expressed as follows. ðδ1 ¼ 1 ^ δ2 ¼ 0 ^ δ3 ¼ 1Þ ) δ4 ¼ 0 or ð1  δ1 ¼ 0 ^ δ2 ¼ 0 ^ 1  δ3 ¼ 0Þ ) δ4 ¼ 0 or δ4  ð1  δ1 Þ þ δ2 þ ð1  δ3 Þ Which is more meaningful than the original inequality, and can easily be interpreted and analyzed. Various examples of combinatorial problems can be found that utilize similar relations and express the requirements of the problem in similar formats. Often, the problem conditions are expressed in the form of logical propositions. Some decisions are made provided that some other decisions have already been taken. For example, establishing any of the following conditions may be somehow required by a mathematical model.

3.5 Propositions and Binary Variables

71

• If the warehouse is built, then it can be used. • If product 1 is produced, then the minimum amount of product 2 should be produced. • Job 1 must be completed before job 2 is started. Concerning the close relationship between logical propositions and the way of expressing the problem’s conditions and constraints, it is necessary to define a precise framework ensuring that a logical proposition is exactly equivalent to its corresponding constraints. Suppose that pk represents a simple logical proposition taking the values True (T ) or False (F) to represent an action or a choice or a decision. We consider a binary variable δk taking the values 1 or 0. The relation of the proposition pk with the binary variable δk is defined as follows: δk ¼ 1 if and only if pk is a true statement. δk ¼ 0 if and only if pk is a false statement. The logical conditions required for each problem are introduced to the mathematical model by means of the binary variables in the form of (non) linear constraints. Temporarily, ignore the need for the linearity of the constraints. In this case, the statement p1 ^ p2 is true if δ1δ2 ¼ 1. In general, if we show the value of the proposition p1 ^ p2 with the binary variable δ, we have: δ ¼ δ1 δ2 which means:  δ¼

1

If p1 and p2 are both true:

0

Otherwise

Of course, there is a linear and straightforward form for expressing this proposition, which will be described later. Similarly, if we show the value of the proposition p1 _ p2 with the binary variable δ, we have: δ ¼ δ1 þ δ2  δ1 δ2 which means:  δ¼

1

If p1 or p2 is true:

0

Otherwise

In the same way, if we represent the value of the proposition ~p1, with the binary variable δ, we have: δ ¼ 1  δ1

72

3

Models and Mathematical Logic

In any case, with the aid of the three operators “and”, “or”, “not” all the required composite propositions can be expressed. For example, the proposition pk _ ~pk+1 is expressed as the linear inequality δk + (1  δk+1)  1. As another example, consider the following three simple propositions: p1: p2: p3:

I am happy. My exam is good. Weather is sunny.

If we show the value of these propositions with variables δ1, δ2, and δ3, then the value of the composite proposition “I am happy and my test is good and the weather is sunny” is simply obtained from the product of these variables. That is: δ ¼ δ1 δ2 δ3 Also, the expression (1  δ1) + δ3  (1  δ1)δ1δ3  1 indicates that “If the weather is sunny, then I will be happy”. This nonlinear inequality can also be expressed linearly. Note that the expression (1  δ1)δ1δ3 is always zero, so the inequality δ1  δ3 is enough to express this proposition. Table 3.4 provides a list of these conversions. In this table, the logical propositions and their counterpart constraints are equivalent. Therefore, in a mathematical model, any logical statement can be replaced by its corresponding constraints. The following examples illustrate the relation between propositions and constraints. Example 3.1 Converting logical propositions into linear constraints. Binary variables δ1, δ2 and δ3 are corresponding to three propositions p1, p2 and p3, respectively. In each case, rewrite the logical proposition as a linear constraint. (a) p3 ¼ p1 ^ p2 Answer: The proposition is equivalent to the following constraints: 2δ3  δ1 þ δ2 δ1 þ δ2  1 þ δ3 (b) p3 ¼ p1 _ p2 Answer: The proposition is equivalent to the following constraints: δ3  δ1 δ3  δ2 δ3  δ1 þ δ2

3.5 Propositions and Binary Variables

73

Table 3.4 Logical propositions vs linear constraints Logical propositions ~p1 p1 _ p2 p1 ⨁ p2 p1 ^ p2 ~( p1 _ p2) ~( p1 ^ p2) p1 ) ~p2 p1 ) p2 p1 , p2 p1 ) ( p2 ^ p3) p1 ) ( p2 _ p3) ( p1 ^ p2) ) p3 ( p1 _ p2) ) p3 p1 ^ ( p2 _ p3) p1 _ ( p2 ^ p3) p1 _ p2 _ . . . _ pn L . . . ⨁ pn p1 ⨁ p2 ( p1 ^ . . . ^ pk) ) ( pk+1 _ . . . _ pn) The statement is true if at least k out of n propositions are true. The statement is true if exactly k out of n propositions are true. The statement is true if at most k out of n propositions are true. ( p1 _ p2 _ . . . _ pk) , pn ( p1 ^ p2 ^ . . . ^ pk) , pn

Linear constraints δ1 ¼ 0 δ1 + δ2  1 δ1 + δ2 ¼ 1 δ1 ¼ 1, δ2 ¼ 1 δ1 ¼ 0, δ2 ¼ 0 δ1 + δ2  1 δ1 + δ2  1 δ1  δ2  0 δ1  δ2 ¼ 0 δ1  δ2, δ1  δ3 δ1  δ2 + δ3 δ1 + δ2  δ3  1 δ1  δ3, δ2  δ3 δ1 ¼ 1, δ2 + δ3  1 δ1 + δ2  1, δ1 + δ3  1 δ1 + δ2 + . . . + δn  1 δ1 + δ2 + . . . + δn ¼ 1 (1  δ1) + . . . + (1  δk) +δk+1 + . . . + δn  1 δ1 + δ2 + . . . δn  k δ1 + δ2 + . . . + δn ¼ k δ1 + δ2 + . . . + δn  k δ1 + δ2 + . . . + δk  δn δn  δj, 8 j ¼ 1, . . ., k δ1  δ2  . . .  δk + δn  1  k δn  δj, 8 j ¼ 1, . . ., k

(c) p3 ¼ p1 ⨁ p2 Answer: The proposition is equivalent to the following constraints: δ3  δ1 þ δ2 δ3  2  δ1  δ2 δ3  δ1  δ2 δ3  δ2  δ1

74

3

Models and Mathematical Logic

Example 3.2 Linear constraint equivalent to a composite proposition. Convert the following composite proposition to linear constraints. ðp1 _ p2 _ p3 Þ ^ ð ðp1 ^  p3 ÞÞ Answer: We know that: ð p1 _ p2 _ p3 Þ  δ1 þ δ2 þ δ3  1 moreover,  ð p1 ^  p3 Þ  p1 _  p3  p1 _ p3  ð1  δ 1 Þ þ δ 3  1  δ1  δ3  0 Therefore, ( ðp1 _ p2 _ p3 Þ ^ ð ðp1 ^  p3 ÞÞ 

δ1 þ δ2 þ δ3  1 δ1  δ3  0

In general, the following steps are followed to determine constraints corresponding to composite logical propositions. 1. Use the DeMorgan’s law and other equivalency relations to move the negation operator “~” from the composite propositions to simple propositions. 2. Write the result as groups of disjunctive propositions “_” in which any two groups are linked together with the operator “^”. 3. Convert any group of disjunctive propositions to a set of constraints. 4. Collection of all constraints is equivalent to the original composite proposition. Most of the materials of Sects. 3.2–3.5 have been derived from Mitra et al. (1994) and Williams (2009).

3.6

Constraints and Indicators

In most of the mathematical models, we need to know whether the current solution satisfies a given constraint, and make sure that the problem conditions can be observed. Indicators are mathematical tools for achieving this requirement, properly. In this section, we introduce indicators and outline how to use them.

3.6 Constraints and Indicators

3.6.1

75

Conditional Indicators

A conditional indicator is a binary variable that is associated with the status of the satisfaction or non-satisfaction of a constraint. In other words, if the indicator takes the value of 1, it can be concluded that the current solution is satisfying the constraint, and if it takes 0, no implication is obtained. In this way, we encounter with an “if-then” conditional statement that the first part is a simple proposition and the second part is a constraint. In this section, we show how such a statement can be introduced into an MIP model. Assume that lj and uj, are, respectively, the lower and upper bounds of the and Li and Ui are respectively the lower and upper bounds of the variable xj;P expression j2 ai, j xj  bi . In other words, suppose that: lj  xj  uj , 8j 2  X ai, j xj  bi  U i , Li 

8i 2 

j2  Let þ i ¼ fj 2  : ai, j > 0g, i ¼ fj 2  : ai, j < 0g. Then, we have:

Li ¼

X

ai, j lj þ

j2þ i

Ui ¼

X

X ai, j uj  bi , 8i 2  j2 i

ai, j uj þ

j2þ i

X j2 i

ai, j lj  bi , 8i 2 

Remark 3.1 Sometime, we may not explicitly define the parameters Li and Ui. Instead, we may simply use M and M as upper and lower bounds, respectively, where M is a sufficiently large positive integer number. In the following, we examine different states that are possible for a constraint and an indicator. Conditional Indicator for Constraint 0  0 δi ¼ 1 )

X

ai, j xj  bi

j2



X ai, j xj  bi  U i ð1  δi Þ j2

In the case of δi ¼ 0, the constraint becomes redundant. Note that these relations are valid for Li  0.

76

3

Models and Mathematical Logic

Conditional Indicator for Constraint 0  0 δi ¼ 1 )

X

ai, j xj  bi

j2



X ai, j xj  bi  Li ð1  δi Þ j2

In the case of δi ¼ 0, the constraint becomes redundant. Note that these relations are valid for Ui  0. Conditional Indicator for Constraint 0 < 0 δi ¼ 1 )

X

ai, j xj < bi

j2

Because we are not allowed to have strict inequalities (i.e., < or >) in the mathematical model, the above constraint is approximated as follows, where ε > 0 is a sufficiently small positive number. X ai , j x j  bi  ε

δi ¼ 1 ) 

X

j2

ai, j xj  ðbi  εÞ  ðU i þ εÞ ð1  δi Þ

j2

For δi ¼ 0, the constraint becomes redundant. However, if δi ¼ 1, we get. X

ai, j xj  bi  ε:

j2

Since ε > 0 is sufficiently small, it is equivalent to the initial constraint. Obviously, these relations are valid only for Li   ε, and they are not applicable in the case of Li > 0. Conditional Indicator for Constraint 0 > 0 δi ¼ 1 ) 

X

X ai , j x j > bi j2

ai, j xj  ðbi þ εÞ  ðLi  εÞð1  δi Þ

j2

For δi ¼ 0, the constraint becomes redundant, and if δi ¼ 1, we get: X ai , j x j  bi þ ε j2

3.6 Constraints and Indicators

77

Where, since ε > 0 is sufficiently small, it is equivalent to the initial constraint. Clearly, these relations are valid only for Ui  ε, and in the case of Ui < 0, they are not applicable. Conditional Indicator for Constraint 0 ¼ 0 δi ¼ 1 )

X

ai, j xj ¼ bi

j2

 δi ¼ 1 )

X

ai, j xj  bi ^

j2

X

! ai, j xj  bi

j2

8P a x  bi  U i ð 1  δ i Þ > < j2 i, j j  P > : ai, j xj  bi  Li ð1  δi Þ j2

In the case of δi ¼ 0, the constraint becomes redundant. Clearly, these relations are valid for Ui  0 and Li  0, and in the case of Ui < 0 or Li > 0, they are not applicable. Conditional Indicator for Constraint 0 6¼ 0 δi ¼ 1 )

X

ai, j xj 6¼ bi

j2

 δi ¼ 1 )

X j2

ai, j xj < bi _

X

! ai, j xj > bi

j2

8P   ai, j xj  ðbi  εÞ  ðU i þ εÞ 1  δ0i > > > j2 > >

> j2 > > > : δi ¼ δ0i þ δ00i Since ε > 0 is sufficiently small, these are equivalent to the initial constraint. If δi ¼ 0, then we get δ0i þ δ00i ¼ 0, and hence, both constraints are redundant. If δi ¼ 1, then 

   δ0i ¼ 1 ^ δ00i ¼ 0 _ δ0i ¼ 0 ^ δ00i ¼ 1

These relations are valid for Ui  ε and Li  ε. Example 3.3 Launching a production line Consider a company producing different products. Suppose that δj is a binary variable that is 1 if the product line of the product j is lunched; otherwise 0. Present a

78

3

Models and Mathematical Logic

constraint ensuring that the production line of product 4 can be lunched provided that the production lines of at least two of products 1, 2 and 3 are lunched. Answer: The following statement should be observed: δ4 ¼ 1 ) δ1 þ δ2 þ δ3  2 That is equivalent to the following constraint: δ1 þ δ2 þ δ3  2δ4 Note that this constraint does not reject the solution in which the products 1, 2 and 3 are produced but not the product 4. Example 3.4 Production planning in a refinery Suppose that a refinery produces two types of products 1 and 2. In order to prevent product mixture, the products are stored in two separate tanks with capacities a and b. If the refinery wishes to produce both products, at most a units of product 1 and b units of product 2 can be produced. However, if it wants to produce only one product, both tanks can be allocated to that product and, in this case, the production capacity is equal to a + b units. Describe these conditions in the form of linear constraints. Answer: We define the propositions p1, p2 and p3 as follows: Proposition p1: At most, a units of product 1 and b units of product 2 are produced; Proposition p2: Only product 1 is produced and its production level is at most a + b units; Proposition p3: Only product 2 is produced and its production level is at most a + b units; Assume that x1 and x2 are the production levels of products 1 and 2, respectively. The statements p1, p2 and p3 can be rewritten as follows: Proposition p1: (x1  a ^ x2  b) Proposition p2: (x1  a + b ^ x2  0) Proposition p3: (x1  0 ^ x2  a + b) Now, it is necessary that the statement p1 ⨁ p2 ⨁ p3 be true. We associate the binary variables δ1, δ2 and δ3 with the propositions p1, p2 and p3, respectively. Then, the following statements should be satisfied simultaneously: ðδ1 ¼ 1 ) ðx1  a ^ x2  bÞÞ, ðδ2 ¼ 1 ) ðx1  a þ b ^ x2  0ÞÞ, ðδ3 ¼ 1 ) ðx1  0 ^ x2  a þ bÞÞ, δ1  δ2  δ3

3.6 Constraints and Indicators

79

These statements are equivalent to the following linear constraints: x1  a  ða þ bÞð1  δ1 Þ x2  b  ða þ bÞð1  δ1 Þ x1  ða þ bÞ  ða þ bÞð1  δ2 Þ x2  ða þ bÞð1  δ2 Þ x1  ða þ bÞð1  δ3 Þ x2  ða þ bÞ  ða þ bÞð1  δ3 Þ δ1 þ δ2 þ δ3 ¼ 1

3.6.2

Complementarity Rule

Consider the binary variable δi for i 2 . Suppose that 0 and 1 are two disjoint subsets of . We intend to rewrite the following statement linearly: ðδi ¼ 0 ð8i 2 0 Þ ^ δi ¼ 1 ð8i 2 1 ÞÞ )

X

aj x j  b

j2

The main difference between this situation and the previous state is that there is more than one indicator variable in the RHS of the statement. In this case, as mentioned in Plastria (2002), we can use the complementarity rule, indicating that for i 2 1 , the term δi ¼ 1 is replaced by the term 1  δi ¼ 0. Regarding this rule, the above statement is equivalent to: ðδi ¼ 0 ð8i 2 0 Þ ^ 1  δi ¼ 0 ð8i 2 1 ÞÞ )

X aj x j  b j2

moreover, its linear form is as follows: X

aj x j  b þ U

j2

where U is an upper bound on

X

δi þ

i20

P

j2 aj xj

X

! ð1  δi Þ

i21

 b.

Example 3.5 Complementarity rule in a location problem Consider a location problem in which the set  (with index i) and the set  (with index j) represent the potential centers and customers, respectively. Moreover, let di, j be the distance between the center i and the customer j. The aim is to decide on opening centers so that each customer is allocated to exactly one open center.

80

3

Models and Mathematical Logic

Suppose that δi is a binary variable that is 1 if the center i is opened; otherwise 0. Also, let γ i, j be a binary variable that is 1 if the customer j is assigned to center i; otherwise 0. Without using any objective function, provide a constraint to ensure that each customer is assigned to the nearest open center. Answer: Suppose that aðjÞ 2  is the closest center to the customer j. In other words, aðjÞ ¼ argminðdi, j Þ i2

If a( j) is open, the customer j has to be assigned to it. Thus, we have: γ aðjÞ, j  δaðjÞ

8j 2 

Also, suppose that i, j is the set of all centers that are closer to customer j than center i. In other words, i, j ¼ fi0 2  : di0 , j  di, j g

8i 2 , j 2  : i 6¼ aðjÞ

In this case, it is enough to satisfy the following statement: ðδi ¼ 1 ^ δi0 ¼ 0 ð8i0 2 i, j ÞÞ ) γ i, j ¼ 1 8i 2 , j 2  : i 6¼ aðjÞ The statement means that if the center i is open and all the centers that are closer to customer j than i are closed, then the customer j must be served by center i. Using the complementarity rule, the above statement is equivalent to the following linear constraint: 0 γ i, j  1  @1  δi þ

3.6.3

X

1 δi0 A

8i 2 , j 2  : i 6¼ aðjÞ

i0 2i, j

Bi-Conditional Indicators

A bi-conditional indicator is a binary variable that is associated with satisfaction and non-satisfaction statuses of a constraint concerning the current solution. In other words, if the indicator takes the value of 1, it can be concluded that the current solution is satisfying the constraint; and if it takes 0, it means that the current solution

3.6 Constraints and Indicators

81

is not meeting the constraint. Thus, we face with a bi-conditional statement, namely “if and only if”, where its first part is a simple proposition and its second part is a linear constraint. In this section, we illustrate how bi-conditional statements can be introduced into an MIP model. Bi-Conditional Indicator for Constraint 0  0 P Suppose that we want to determine the status of the constraint j2 ai, j xj  bi with respect to the current solution. Therefore, we should incorporate the following statement into the model: δi ¼ 1 ⟺

X

ai, j xj  bi

ð3:1Þ

j2

We know that the following statement is established: p ) q  q ⟹  p Therefore, we have: p ⟺ q  ð p ) qÞ ^ ð p ) qÞ Thus, the proposition (3.1) is equivalent to: X δi ¼ 1 ) ai , j x j  bi j2



X ai , j x j  bi δi ¼ 1 ) j2

! ^

X ai, j xj Ý bi δi ¼ 0 )

!

j2

! ^

1  δi ¼ 1 )

8P a x  bi  U i ð 1  δ i Þ > < j2 i, j j  P > : a i , j x j  ð b i þ ε Þ  ð Li  ε Þ δ i

X

! ai, j xj > bi

j2

j2

Obviously, in the case δi ¼ 0, the first constraint and for δi ¼ 1, the second constraint are redundant. Also, δi ¼ 0 indicates that the current solution does not satisfy the constraint that appeared in the RHS of proposition (3.1). These relations are valid for Li  0 and Ui  ε. Now, we can achieve similar results for other types of constraints.

82

3

Models and Mathematical Logic

Bi-Conditional Indicator for Constraint 0  0 Suppose that we want to incorporate the following statement into the model: δi ¼ 1 ⟺

X

ai, j xj  bi

j2

This proposition is equivalent to: X ai , j x j  bi δi ¼ 1 )

! ^

X δi ¼ 0 ) ai, j xj ß bi

j2

8P a x  ðbi  εÞ  ðU i þ εÞðδi Þ > < j2 i, j j  P > : ai, j xj  bi  Li ð1  δi Þ

!

j2

j2

These relations are valid for Li  ε and Ui  0. Bi-Conditional Indicator for Constraint 0 < 0 Suppose that we want to incorporate the following statement into the model: δi ¼ 1 ⟺

X j2

ai, j xj < bi

This proposition is equivalent to: X ai, j xj < bi δi ¼ 1 )

! ^

X δi ¼ 0 ) ai , j x j  bi

j2

8 P a x  ðbi  εÞ  ðU i þ εÞð1  δi Þ > < j2 i, j j  P > : ai, j xj  bi  Li δi

!

j2

j2

These relations are valid for Li  ε and Ui  0. Bi-Conditional Indicator for Constraint 0 > 0 Suppose that we want to incorporate the following statement into the model: δi ¼ 1 ⟺

X j2

ai, j xj > bi

3.6 Constraints and Indicators

83

This proposition is equivalent to: X ai, j xj > bi δi ¼ 1 )

! ^

X δi ¼ 0 ) ai , j x j  bi

j2

8 P a x  ðbi þ εÞ  ðLi  εÞð1  δi Þ > < j2 i, j j  P > : ai, j xj  bi  U i δi

!

j2

j2

These relations are valid for Li  0 and Ui  ε. Bi-Conditional Indicator for Constraint 0 ¼ 0 Suppose that we want to consider the following statement in the model: δi ¼ 1 ⟺

X

ai, j xj ¼ bi

j2

This proposition is equivalent to: δi ¼ 1 )

X

!



δi ¼ 1 )

X

^

ai, j xj ¼ bi

j2

X δi ¼ 0 ) ai, j xj ¼ 6 bi j2

! ai, j xj ¼ bi

!

^

δi ¼ 0 )

j2

X

ai, j xj < bi _

j2

8P   > ai, j xj  bi  U i 1  δ0i > > j2 > > > >   P > > > ai, j xj  bi  Li 1  δ00i > > > j2 >

> j2 > > > P > > ai, j xj  ðbi  εÞ  ðU i þ εÞδ00i > > > > j2 > > > : 0 δi þ δ00i ¼ 1 þ δi

X

!! ai, j xj > bi

j2

These relations are valid for Li   ε and Ui  ε . Bi-Conditional Indicator for Constraint 0 6¼ 0 Suppose that we want to consider the following statement in the model: δi ¼ 1 ⟺

X j2

ai, j xj 6¼ bi

84

3

Models and Mathematical Logic

This proposition is equivalent to: δi ¼ 1 )

X

! ai, j xj 6¼ bi

^

δi ¼ 0 )

j2



δi ¼ 1 )

X

ai, j xj ¼ bi

j2

X X ai , j x j < bi _ ai , j x j > bi j2

!

!! ^

j2

  8P ai, j xj  ðbi  εÞ  ðU i þ εÞ 1  δ0i > > > j2 > > >   P > > > ai, j xj  ðbi þ εÞ  ðLi  εÞ 1  δ00i > > > > j2 < P  ai, j xj  bi  U i δi > > j2 > > >P > > > > ai, j xj  bi  Li δi > j2 > > > : δi ¼ δ0i þ δ00i

δi ¼ 0 )

X

! ai, j xj ¼ bi

j2

These relations are valid for Li   ε and Ui  ε. Example 3.6 Suppose that y is a nonnegative integer variable and δ is a binary variable. Provide a constraint to guarantee the following statements: y>0 ) δ¼1

ð3:2Þ

y¼0 )δ¼0

ð3:3Þ

Answer: To answer this question, we present two methods where, the first one requires a predefined upper bound on variable y; however, the second one is also applicable when no upper bound is identified on y. Method 1: Let u be a known upper bound on y. Proposition (3.2) is equivalent to: δ¼0

) y0

Which is corresponding to: y  uδ Also, the integrality of y implies that the proposition (3.3) is equivalent to: δ¼1) y1 Which is corresponding to: δy

3.6 Constraints and Indicators

85

Method 2: We define a new integer variable y0 and use the following constraint: y0 ¼ y  δ Furthermore, y0 must be appropriately penalized in the objective function. Obviously, by this method, proposition (3.2) is met. Because, if y has a value greater than zero (i.e., 1, 2, . . .), then, penalizing y0 in the objective function ensures that δ holds the value of 1. Example 3.7 Suppose that x is a free continuous variable and δ is a binary variable. Provide a set of constraints to guarantee the following proposition: δ ¼ 1 , x 6¼ 0 Answer: The proposition above is equivalent to the following propositions: x¼0 )δ¼0

ð3:4Þ

x 6¼ 0 ) δ ¼ 1

ð3:5Þ

Suppose that u and l are respectively the upper and lower bounds on the free variable x, where l < 0 < u. The statement (3.4) is equivalent to: δ ¼ 1 ) x 6¼ 0 Considering ε > 0 as a sufficiently small number, the above proposition can be approximated as the following one: δ ¼ 1 ) ðx  ε _ x  εÞ The statement is equivalent to: 8 x  ε þ ðl  εÞð1  δ0 Þ > > > < x  ε þ ðu þ εÞð1  δ00 Þ > δ0 þ δ00 ¼ δ > > : 0 00 δ , δ 2 f0, 1g Similarly, proposition (3.5) is equivalent to: δ ¼ 0 ) ð x  0 ^ x  0Þ It can be stated as the following constraints: 

x  uδ x  lδ

86

3

Models and Mathematical Logic

Fixed Cost In optimization problems, the cost associated with doing or not doing an activity is considered as the fixed cost or the setup cost. This cost does not depend on the level of activity and is solely due to the decision of carrying out that activity. For example, suppose that a car company that had stopped the production of a specific type of car would decide to reproduce this type of car for the coming season. Before its production, the company must pay for the setup of the production line. The cost of doing this is constant and does not depend on the production level. Assume that the variable x represents the level of activity we want to do and u is an upper bound on it. Also, let c1 be the fixed cost associated with doing or not doing the activity and c2 be the cost to be paid for each unit of activity. Thus, the total cost is determined as follows:  C ð xÞ ¼

0

x¼0

c1 þ c2 x

x>0

Figure 3.1 shows the graph of the function C(x). As can be seen, the function of the total cost is not a linear function of x and is discontinuous at the origin. Suppose that C(x) is a part of the objective function of a minimization model as follows: min z ¼ CðxÞ To linearize the objective function, we use the indicator δ which is equal to 1 if x > 0; and 0 otherwise. Thus, the function C(x) is substituted by the linear function c1δ + c2x, and the objective function is rewritten as follows: min z ¼ c1 δ þ c2 x

ð3:6Þ

Now, it is only required to add a constraint ensuring the following statement: x>0,δ¼1

Fig. 3.1 Discontinuous cost function

3.6 Constraints and Indicators

87

Above statement is an example of a bi-conditional indicator for a ‘ > ’ constraint and is equivalent to: ð x > 0 ) δ ¼ 1Þ ^ ð δ ¼ 1 ) x > 0Þ  ðx > 0 ) δ ¼ 1Þ ^ ðx ¼ 0 ) δ ¼ 0Þ Therefore, we must add some constraints to guarantee the following propositions: x>0)δ¼1

ð3:7Þ

x¼0)δ¼0

ð3:8Þ

The proposition (3.7) can be guaranteed by the linear constraint x  uδ. However, the objective function (3.6) is in minimization direction and implies that between two solutions (x, δ) ¼ (0, 0) and (x, δ) ¼ (0, 1), the first one is preferred. Therefore, the proposition (3.8) is automatically observed and there is no need to add any constraint for its satisfaction. Example 3.8 Production plan for a car manufacturing company A car manufacturing company examines the production of three types of vehicles. The fixed cost of launching and preparing the equipment of the production line of vehicle types 1, 2 and 3 is 500, 400 and 350, respectively. The variable cost and sale price of a vehicle of any type, as well as the amount of steel and workforce required to produce, are listed in Table 3.5. Assuming that there are 5000 tons of steel and 40,000 h of human labor, provide an optimization model to make a decision on the number of production so that the company’s profit is maximized. Answer: For each j ¼ 1, 2, 3, we define the integer variable yj as the production level of the vehicle of type j. Also, let δj be a binary variable that is 1 if yj > 0; otherwise 0. Thus, the problem is formulated as follows: max z ¼ 10y1 þ 6y2 þ 7y3  500δ1  400δ2  350δ3 s:t: 5y1 þ 3y2 þ 1:5y3  5000

ð3:9Þ

40y1 þ 25y2 þ 30y3  40000 y 1  u1 δ 1 ,

ð3:10Þ

y2  u2 δ2 , y3  u3 δ3

y1 , y2 , y3  0, Int: δ1 , δ2 , δ3 2 f0, 1g Table 3.5 Characteristics of each vehicle type Vehicle type 1 2 3

Fixed cost 500 400 350

Variable cost 10 8 14

Sale price 20 14 21

Steel (ton) 5 3 1.5

Workforce (persons/h) 40 25 30

88

3

Models and Mathematical Logic

Given the constraints (3.9) and (3.10), the upper bound on the variables y1, y2, and y3 can be defined as u1 ¼ 1000, u2 ¼ 1600, and u3 ¼ 1333, respectively.

3.6.4

Either-or Constraints

Suppose that two constraints R1() and R2() are as follows: R1 ðÞ :

X a1 , j x j  b1  0 j2

X a2 , j x j  b2  0 R2 ðÞ : j2

where, U1, UP denote the upper and lower bounds on the 2, L1, and L2, respectively, P expressions j2 a1, j xj  b1 and j2 a2, j xj  b2 . We want to ensure that at least one of these two constraints is met. In other words, we deal with the following proposition: X X a1 , j x j  b1  0 _ a2 , j x j  b2  0 j2

j2

This type of constraint is often called the “Either-Or constraint”; and appears in mathematical programming problems. Now, we explain how to replace it by linear constraints. Assuming that δ is a binary variable, the above proposition is equivalent to the following ones: δ¼1)

X

! a1, j xj  b1  0

j2



δ¼1)

X j2

^

8P a x  b1  U 1 ð 1  δ Þ > < j2 1, j j  P > : a2 , j x j  b2  U 2 δ

! a2 , j x j  b2  0

j2

! a1, j xj  b1  0

δ¼0)

X

^

1δ¼1)

X

! a2 , j x j  b2  0

j2

j2

Similar results for different types of constraints can be obtained, as summarized in Table 3.6.

3.6 Constraints and Indicators

89

Table 3.6 Different cases for either-or constraints Type of constrains R1(), R2() R1(), R2() R1(), R2(>) R1(), R2(>)

R1(), R2(6¼)

R1( > < R2 þ ε  ðU 2 þ εÞð1  δ0 Þ > R2  ε  ðL2  εÞð1  δ00 Þ > > > > : δ ¼ δ0 þ δ00 ( R1 þ ε  ðU 1 þ εÞδ R2 þ ε  ðU 2 þ εÞð1  δÞ

(

R1  ε  ðL1  εÞδ

R2 þ ε  ðU 2 þ εÞð1  δÞ 8 R1 þ ε  ðU 1 þ εÞδ > > < R2  U 2 ð1  δÞ > > : R2  L2 ð1  δÞ 8 R > 1 þ ε  ðU 1 þ εÞδ > > > > < R2 þ ε  ðU 2 þ εÞð1  δ0 Þ > R2  ε  ðL2  εÞð1  δ00 Þ > > > > : δ ¼ δ0 þ δ00 8 R1 þ ε  ðU 1 þ εÞð1  δ0 Þ > > > > > 00 > > > R1  ε  ðL1  εÞð1  δ Þ < R2  U 2 ð1  δÞ > > > > R2  L2 ð1  δÞ > > > > : δ ¼ 1  δ0  δ00

Type of constrains R1(), R2() R1(), R2() R1(>), R2(¼)

R1(>), R2(6¼)

R1(¼), R2(¼)

Equivalent constraints ( R1  L1 δ ( (

R2  U 2 ð1  δÞ R1  U 1 δ R2 þ ε  ðU 2 þ εÞð1  δÞ R1  L1 δ

R2 þ ε  ðU 2 þ εÞð1  δÞ 8 R1  U 1 δ > > < R2  U 2 ð1  δÞ > > : R2  L2 ð1  δÞ 8 R1  L1 δ > > < R2  U 2 ð1  δÞ > > : R2  L2 ð1  δÞ 8 R1  U 1 δ > > > > > < R2 þ ε  ðU 2 þ εÞð1  δ0 Þ > R2  ε  ðL2  εÞð1  δ00 Þ > > > > : δ ¼ δ0 þ δ00 ( R1  ε  ðL1  εÞδ R2  ε  ðL2  εÞð1  δÞ 8 R1  ε  ðL1  εÞδ > > < R2  U 2 ð1  δÞ > > : R2  L2 ð1  δÞ 8 R > 1  ε  ðL1  εÞδ > > > > < R2 þ ε  ðU 2 þ εÞð1  δ0 Þ > R2  ε  ðL2  εÞð1  δ00 Þ > > > > : δ ¼ δ0 þ δ00 8 R1  U 1 δ > > > > > < R1  L1 δ > R2  U 2 ð1  δÞ > > > > : R2  L2 ð1  δÞ (continued)

90

3

Models and Mathematical Logic

Table 3.6 (continued) Type of constrains R1(6¼), R2(6¼)

Equivalent constraints 8 R1 þ ε  ðU 1 þ εÞð1  δ0 Þ > > > > > 00 > > > R1  ε  ðL1  εÞð1  δ Þ > > > < R2 þ ε  ðU 2 þ εÞð1  γ 0 Þ

Type of constrains

Equivalent constraints

> R2  ε  ðL2  εÞð1  γ 00 Þ > > > > > > > 1  δ ¼ δ0 þ δ00 > > > : δ ¼ γ 0 þ γ 00

Fig. 3.2 Nonconvex feasible region

Fig. 3.3 The region 1

Example 3.9 Optimization over nonconvex regions The feasible region of the following problem is depicted in Fig. 3.2. Formulate it in the form of linear constraints. max z ¼ 2x1 þ x2 s:t: x ¼ ðx1 , x2 Þ 2  Answer: The nonconvex region  can be considered as a union of two convex regions 1 and 2 (i.e.,  ¼ 1 [ 2 ) which are shown in Figs. 3.3 and 3.4, respectively (note that the representation of  as a union of convex sets is not necessarily unique; in this regard, see Exercise 3.11).

3.6 Constraints and Indicators

91

Fig. 3.4 The region 2

The regions 1 and 2 are described in the form of the following constraints: 1 ¼ fðx1 , x2 Þ : x1 þ x2  1, x1  0, x2  0g 2 ¼ fðx1 , x2 Þ : x1  x2  0, x2  1, x1  0, x2  0g Therefore, ðx1 , x2 Þ 2 1 [ 2 is equivalent to: 8 > < x1 þ x2  1 x1  0 > : x2  0

_

8 x1  x2  0 > > >

x1  0 > > : x2  0

ð3:11Þ

The constraints x1  0, x2  0 and x2  1 are common in both sides of disjunction (3.11). Therefore, (3.11) is equivalent to: ð x1 þ x2  1

_

x1  x2  0Þ,

x2  1, x1  0, x2  0 Thus, with the help of the binary variable δ, we can declare the nonconvex region  in the form of the following linear constraints, in which M is a sufficiently large positive number. x1 þ x2  1 þ Mδ, x1  x2  M ð1  δÞ, x2  1, x1  0, x2  0, Example 3.10 Linearization of complementary slackness conditions In Model 2.19, linearize the complementary slackness conditions and solve the resulting linear model.

92

3

Models and Mathematical Logic

Answer: First, consider (x  y + 3)λ1 ¼ 0. This condition is equivalent to the following proposition: ðx  y þ 3 ¼ 0Þ _ ðλ1 ¼ 0Þ Due to the initial condition x  y + 3  0 and the nonnegativity restriction on variable λ1, the above proposition is equivalent to the following statement: ðx  y þ 3  0Þ _ ðλ1  0Þ Now, considering δ1 as a binary variable, the above statement is equivalent to the following constraints, in which M is a sufficiently large positive number. λ1  M ð1  δ1 Þ x  y þ 3  Mδ1 Similarly, the complementary slackness conditions (2.20)–(2.23) can be linearized by introducing the binary variables δ2, . . ., δ5, respectively. Thus, Model 2.19 is rewritten as follows: Model 3.1 Liner restatement of Model 2.19 max z ¼  x + 4y s. t. x0 x  y  3 2x þ y  0 2x þ y  12  3x þ 2y  4 y0 λ1 þ λ2 þ λ3 þ 2λ4  1 λ1 , λ2 , λ3 , λ4  0 λ1  M(1  δ1) x  y + 3   Mδ1 λ2  M(1  δ2) 2x + y   Mδ2 λ3  M(1  δ3) 2x + y  12   Mδ3 λ4  M(1  δ4) 3x + 2y + 4   Mδ4 y  M(1  δ5) λ1 + λ2 + λ3 + 2λ4 + 1  Mδ5 δ1, δ2, . . ., δ5 2 {0, 1}

3.6 Constraints and Indicators

93

If M is substituted by 20 and Model 3.1 is solved, the optimal solution is obtained as follows:

3.6.5

z ¼ 12

x ¼ 4,

y ¼ 4,

λ1 δ1

λ2 δ2

λ3 ¼ 0,

¼ 0, ¼ 1,

¼ 0, ¼ 1,

δ3 ¼ 0,

λ4 ¼ 0,

δ4 ¼ 1,

δ5 ¼ 0

Constraint Selection

The either-or constraint, described in the previous section, addresses the choice of one constraint between the two constraints. In general, it may be required to choose k constraints among m constraints (in other words, a solution is feasible if it satisfies at least k constraints, where k  m). For example, assume that we need to select a single constraint (k ¼ 1) among the following m constraints: n X

ai, j xj  bi

8i ¼ 1, . . . , m

j¼1

To implement this situation, consider the binary variable δi that is 1 if the ith constraint is chosen; otherwise 0. Thus, the following constraints should be included in the model: n X j¼1 m X

ai, j xj  bi þ M ð1  δi Þ

8i ¼ 1, . . . , m

δi ¼ k

i¼1

δi 2 f0, 1g

8i ¼ 1, . . . , m

Example 3.11 Selecting two constraints Consider the following constraints: x1 þ x2 þ x3 þ x4  5 x1 þ 2x2 þ x3  x4  4 2x1 þ x2 þ 3x3  4 x1  2x3  3 x1 þ x2  3x4  2 x1 þ 2x2 þ 4x4  8

94

3

Models and Mathematical Logic

(a) How can we ensure that at least two constraints of these constraints are selected? Answer: The following constraints guarantee this: x1 þ x2 þ x3 þ x4  5 þ M ð 1  δ 1 Þ

ð3:12Þ

x1 þ 2x2 þ x3  x4  4  M ð1  δ2 Þ

ð3:13Þ

2x1 þ x2 þ 3x3  4  M ð1  δ3 Þ

ð3:14Þ

x1  2x3  3 þ M ð1  δ4 Þ

ð3:15Þ

x1 þ x2  3x4  2  M ð1  δ5 Þ

ð3:16Þ

x1 þ 2x2 þ 4x4  8 þ M ð1  δ6 Þ

ð3:17Þ

δ1 þ δ2 þ . . . þ δ6 ¼ 2

ð3:18Þ

δ1 , δ2 , . . . , δ6 2 f0, 1g

ð3:19Þ

(b) What difference does it make if in constraint (3.18), the sign “¼” is substituted by “”? Answer: There is no difference (why?).

3.6.6

Not-Equality Constraints

In linear programming, the type of constraints can only be one of the three states ¼, , or . However, in optimization problems, sometimes, we need to use the not-equal constraint (6¼). In this case, it is necessary to rewrite the constraint appropriately in the form of ¼, , or . For example, consider the following constraint: X

aj xj 6¼ b

j2

To rewrite this constraint in an appropriate form, we first write it in the form of either-or constraint: X

aj x j < b _

j2

X aj x j > b j2

Then, since we cannot have strict inequalities in the mathematical programming problem, we rewrite it as follows: X j2

aj x j  b  ε _

X aj x j  b þ ε j2

3.6 Constraints and Indicators

95

Where ε > 0 is a sufficiently small number. Then, we introduce the binary variable δ and replace the above statement with the following inequalities: X

aj x j  b  ε þ M ð 1  δ Þ

j2

X

aj xj  b þ ε  Mδ

j2

Example 3.12 Linearization of a composite proposition Suppose that y1 and y2 are two integer variables and a1 and a2 are two integer parameters. Provide a set of constraints to ensure that the constraints y1 ¼ a1 and y2 ¼ a2 are not satisfied simultaneously. In other words, a situation in which both of them are met is not acceptable. Answer: The following statement should be held: ðy1 6¼ a1 Þ _ ðy2 6¼ a2 Þ Because of the integrality of variables and parameters, the above proposition is equivalent to: ð y 1  a1  1Þ _ ð y 1  a1 þ 1Þ _ ð y 2  a2  1Þ _ ð y 2  a2 þ 1Þ The statement is equivalent to: y1  a1  1 þ M ð1  δ1 Þ y1  a1 þ 1  M ð1  δ2 Þ y2  a2  1 þ M ð1  δ3 Þ y2  a2 þ 1  M ð1  δ4 Þ δ1 þ δ2 þ δ3 þ δ4 ¼ 1 δ1 , δ2 , δ3 , δ4 2 f0, 1g Example 3.13 Variables with not-equal values In some optimization problems, it is necessary to impose conditions on two specific variables ensuring that they cannot take the same values. Provide appropriate constraints for each of the following cases. (a) Case 1: δ1 6¼ δ2 assuming that δ1 and δ2 are binary variables. Answer: δ1 + δ2 ¼ 1 (b) Case 2: y1 6¼ y2 assuming that y1 and y2 are integer variables. Answer: Constraint y1 6¼ y2 is equivalent to:

96

3

Models and Mathematical Logic

y1  y2 < 0 _ y1  y2 > 0 Because of the integrality condition on variables, we get: y1  y2  1 _ y1  y2  1 Assuming that δ is a binary variable, the above propositions can be replaced by the following constraints: y1  y2  1 þ M ð1  δÞ y1  y2  1  Mδ Case 3: x1 6¼ x2 assuming that x1 and x2 are continuous variables. Answer: constraint x1 6¼ x2 is equivalent to: x1 < x2 _ x1 > x2 By considering ε > 0 as a sufficiently small number, the above proposition is substituted by the following statement: x1  x2  ε _ x1  x2 þ ε Now, similar to the previous case, it can be rewritten as two constraints.

3.6.7

If-Then Constraints

The logical proposition “if-then”, known as a conditional proposition, is a basis for mathematical models and is frequently used in expressing the problem requirements. Therefore, it is entirely natural to apply it as an efficient tool for making models. The restatement of this proposition in the form of mathematical inequality constraints can be a basis to express different types of a model’s requirements. For example, in many cases, we want to ensure that if the constraint R1 holds, then constraint R2 must be held. However, if R1 does not hold, then R2 may be established or not. Thus, we deal with a proposition in the following form: X a1 , j x j  b1  0 j2

! )

X

! a2 , j x j  b2  0

j2

Now, by using the indicator δ, we can split the above statement into two independent parts:

3.6 Constraints and Indicators

X

97

! a1 , j x j  b1  0 ) δ ¼ 1

j2



δ¼0)

X

^ !

a1, j xj  b1 > 0

1δ¼1)

X

! a2 , j x j  b2  0

j2

^

j2



δ¼1)

X

δ¼1)

X

a2 , j x j  b2  0

j2

! a1, j xj  b1 > 0

!

^

δ¼1)

X

j2

! a2 , j x j  b2  0

j2

Concerning the relations discussed in Sect. 3.6.1 for conditional indicators and the constraint of type “”, we have: X

a1, j xj  b1  ðL1  εÞδ þ ε

j2

X

a2, j xj  b2  U 2 ð1  δÞ

j2

These relations are valid for U1  ε and L2  0. For other situations, similar constraints can be obtained. Example 3.14 An application of If-Then statement Suppose that x1 and x2 are two continuous variables such that 0  x1  10 and 0  x2  10. Provide linear inequality constraints ensuring that these two variables satisfy the following statement: x 1 > 4 ) x2  6 Answer: With the help of the binary variable δ, it can be expressed as follows: x1  4 þ 6δ x2  10  4δ If x1 > 4 then δ ¼ 1 that implies x2  6. Example 3.15 Express the following statement in the form of linear constraints. δ1 þ δ2  1 ) δ3 þ δ4 þ δ5  1 Answer: Using the indicator δ, we can divide this statement into two independent parts. ðδ1 þ δ2  1 ) δ ¼ 1Þ ^ ðδ ¼ 1 ) δ3 þ δ4 þ δ5  1Þ Proposition δ1 + δ2  1 ) δ ¼ 1 is equivalent to:

ð3:20Þ

98

3

Models and Mathematical Logic

δ ¼ 0 ) δ1 þ δ2 < 1 By considering the upper bound U ¼ 2 on the expression δ1 + δ2, the above if-then proposition can be stated as the following linear constraint: δ1 þ δ2  2δ Similarly, by considering the lower bound L ¼  1 on the expression δ3 + δ4 + δ5  1, the proposition δ ¼ 1 ) δ3 + δ4 + δ5  1 of (3.20), is restated as the following linear constraint: δ3 þ δ4 þ δ5  δ Example 3.16 Expressing in linear form Suppose that y1 and y2 are integer variables, and k1 and k2 are integer parameters. Add constraints to guarantee the following statement: y1  k 1 ) y2  k 2 Answer: Using the indicator δ, the above statement is rewritten as follows: ð y 1  k 1 ) δ ¼ 1Þ ^ ð δ ¼ 1 ) y 2  k 2 Þ The conditional proposition y1  k1 ⟹ δ ¼ 1 is equivalent to: δ ¼ 0 ) y1 > k 1 Since y1 and k1 both are an integer the above statement is equivalent to: δ ¼ 0 ) y1  k 1 þ 1 Therefore, the statement is guaranteed by the following constraints: y1  k1 þ 1  Mδ y2  k 2 þ M ð 1  δ Þ Example 3.17 Cover determination problem Let  (indexed by i) be a set with the cardinality n and denote the subsets of  with k , where k 2  ¼ f1, 2, . . . , 2n g. We say that the set 0  covers the member i 2 , if there exists at least one k 2 0 such that i 2 k . Moreover, we define the set i  as follows: i ¼ fk 2 ji 2 k g

8i 2 

3.6 Constraints and Indicators

99

Therefore, we can say that 0  covers the member i 2 , if the intersection of  and i is not empty. 0

(a) Find the set 0 so that it covers all members of the set . Answer: Let δk be a binary variable that is 1 if k 2 0 ; otherwise 0. Therefore, we have 0 ¼ fk 2 jδk ¼ 1g, and the following constraint ensures that the set 0 covers all members of . X δk  1

8i 2 

k2i

(b) Suppose that γ i is a binary variable that is 1 if the member i 2  is covered; otherwise 0. Also, define the variable δk as before. Provide constraints to ensure that the following statement is satisfied: γ i ¼ 1 ⟺ ∃k 2 i : δk ¼ 1

ð3:21Þ

Answer: This is a bi-conditional proposition. The first part is: γ i ¼ 1 ) ð∃k 2 i : δk ¼ 1Þ Which is equivalent to: ð8k 2 i : δk ¼ 0Þ ) γ i ¼ 0 The above statement is equivalent to the following constraint: γi 

X

δk

k2i

The second part is: ð∃k 2 i : δk ¼ 1Þ ) γ i ¼ 1 Which is equivalent to: δk  γ i Remark 3.2 If the problem’s objective function is to maximize the number of covered members P (i.e., max i2 γ i ), then the second part of the proposition (3.21) is always established, and we do not need to add any constraint.

100

3.6.8

3

Models and Mathematical Logic

If-Then-Else Constraints

Sometimes, there is a situation where we want to ensure that if the constraint R1 holds, then the constraint R2 must be held; otherwise the constraint R3 must be held. In other words, we deal with the following statement: "

X

! a1, j xj  b1  0

X

)

j2

" 

!# a2 , j x j  b2  0

^

j2

X

! a1, j xj  b1  0

X a3 , j x j  b3  0

)

j2

!#

j2

Which is equivalent to: "

X

! a1, j xj  b1  0

)

j2

"

X a2 , j x j  b2  0

!# ^

j2

X a1 , j x j  b1 > 0

! )

j2

X

!# a3 , j x j  b3  0

j2

Now, by considering the indicator δ, we have: " δ¼1,

X

!# a1, j xj  b1  0

^

j2

" δ¼1)

X

!# a2, j xj  b2  0

^

j2

" δ¼0)

X a3 , j x j  b3  0

!#

j2

Concerning the conditional and bi-conditional indicators, explained earlier, the above statement is equivalent to the following constraints: X

a1, j xj  b1  U 1 ð1  δÞ

j2

X

a1, j xj  ðb1 þ εÞ  ðL1  εÞδ

j2

X

a2, j xj  b2  U 2 ð1  δÞ

j2

X j2

a3 , j x j  b3  U 3 δ

3.7 Miscellaneous Examples

101

Table 3.7 Logical propositions for project selection Logical proposition At most k projects are selected.

Mathematical constraint n P δj  k

At least k projects are selected.

j¼1 n P

Exactly k projects are selected.

j¼1 n P

δj  k δj ¼ k

j¼1

If project 1 is selected then project 2 must be selected. Project 2 is not selected. If project 1 is selected then project 2 must not be selected. If project 1 is not selected then project 2 must be selected. Project 1 is selected if and only if project 2 is selected. If project 1 is selected then projects 2 and 3 must be selected. If project 1 is selected then projects 2 or 3 must be selected. If projects 2 or 3 are selected then project 1 must be selected. If projects 2 and 3 are selected then project 1 must be selected If at least two of the projects {2, 3, 4, 5} are selected then project 1 must be selected. If at least k of n projects are selected then project 1 must be selected.

δ1  δ2 δ2 ¼ 0 δ1 + δ2  1 δ1 + δ2  1 δ1 ¼ δ2 δ1  12 ðδ2 þ δ3 Þ δ1  δ2 + δ3 δ1  12 ðδ2 þ δ3 Þ δ1  δ2 + δ3  1 δ1  13 ðδ2 þ δ3 þ δ4 þ δ5  1Þ 1 δ1  nkþ1

n P

! δj

! kþ1

j¼1

These relations are valid for L1  0, U1  ε, L2  0 and L3  0. Similarly, for other states, appropriate constraints can be presented. Example 3.18 Project selection Suppose that  ¼ f1, 2, . . . , ng, indexed by j, is the set of projects and δj is a binary variable that is 1 if project j 2  is selected; otherwise 0. Express the logical propositions provided in Table 3.7 as mathematical equations or inequalities. Answer: The inequalities and equations associated with each proposition are provided in the second column of Table 3.7.

3.7

Miscellaneous Examples

In this section, different examples are considered. Example 3.19 Select k nonzero successive components Suppose that δ ¼ (δ1, . . ., δn) is a vector of n binary variables assuming that exactly k of its components should be equal to 1 and others should be 0. Furthermore, the nonzero components should be successive. Provide appropriate constraints to impose this condition. Answer: In this regard, we present three methods.

102

3

Models and Mathematical Logic

Method 1: We define the binary variable ηj for j ¼ 1, . . ., n  k + 1 as follows:  ηj ¼

if j is the first nonzero component of binary vector δ Otherwise

1 0

Then, we consider the following constraints: nkþ1 X j¼1 n X

ηj ¼ 1 ð3:22Þ

δj ¼ k

j¼1

η1 ¼ δ1   δj1 ¼ 0 ^ δj ¼ 1 ) ηj ¼ 1

8j ¼ 2, . . . , n  k þ 1

ð3:23Þ

Proposition (3.23) guarantees the successiveness of nonzero components and is equivalent to:    δj1 ¼ 0 ^ δj ¼ 1 _ ηj ¼ 1  δj1 ¼ 1 _ δj ¼ 0 _ ηj ¼ 1    δj1 þ 1  δj þ ηj  1  δj  δj1  ηj Method 2: This method is similar to the previous one with the difference that instead of constraints (3.22) and (3.23), the successiveness of nonzero components is guaranteed by the following statement: ηj ¼ 1 )

jþk1 X

δj0  k

j0 ¼j

Which is equivalent to: jþk1 X

δj0  kηj

j0 ¼j

Method 3: In this method, there is no need to define any new variable. Instead, we only need to consider the following constraints: n X j¼1

δj ¼ k

3.7 Miscellaneous Examples

103 k X

δ1 ¼ 1 )

δj  k  1

ð3:24Þ

j¼2



jþk1 X  δj1 ¼ 0 ^ δj ¼ 1 ) δj0  k  1 j0 ¼jþ1

ð3:25Þ

8j ¼ 2, . . . , n  k þ 1 Proposition (3.24) is linearized as can be replaced by: jþk1 X

Pk

j¼2 δj

 ðk  1Þδ1 , and the proposition (3.25)

    δj0  k  1  M 2  1  δj1  δj 8j ¼ 2, . . . , n  k þ 1

j0 ¼jþ1

where M can be set equal to k  1. Example 3.20 Lower bound on the number of nonzero successive components Suppose that δ ¼ (δ1, . . ., δn) is a vector of n binary variables. In this vector, we refer to the successive components δj, δj + 1, . . ., δj + t as a block provided that they have the same values and their values are different from that of δj  1 and δj + t + 1. Provide appropriate linear constraints ensuring that in vector δ, every block with nonzero components has the length of at least 3. Answer: The following propositions should be followed: 

δj1

δ1 ¼ 1 ) ðδ2 ¼ 1 ^ δ3 ¼ 1Þ    ¼ 0 ^ δj ¼ 1 ) δjþ1 ¼ 1 ^ δjþ2 ¼ 1 8j ¼ 2, . . . , n  2

Proposition (3.26) may be replaced by: δ2  δ1 δ3  δ1 Proposition (3.27) is equivalent to:  

δj1 ¼ 0 ^ δj ¼ 1 δj1 ¼ 0 ^ δj ¼ 1

 

) δjþ1 ¼ 1

8j ¼ 2, . . . , n  2

) δjþ2 ¼ 1

8j ¼ 2, . . . , n  2

ð3:26Þ

ð3:27Þ

104

3

Models and Mathematical Logic

Which can be stated as follows: δj1  δj þ δjþ1  0

8j ¼ 2, . . . , n  2

δj1  δj þ δjþ2  0

8j ¼ 2, . . . , n  2

Example 3.21 Blocks with the exact length of k Suppose that δ ¼ (δ1, . . ., δn) is a vector of n binary variables. Provide appropriate linear constraints ensuring that in vector δ, every block with nonzero components has an exact length of k. In other words, for k ¼ 3, the vector (0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1) is feasible; while the vectors (0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1) and (1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0) are infeasible. Answer: It is only required to add constraints satisfying the following propositions: δ1 ¼ 1 )

k X

δj  k  1

ð3:28Þ

j¼2



jþk1 X  δj1 ¼ 0 ^ δj ¼ 1 ) δj0  k  1 j0 ¼jþ1

ð3:29Þ

8j ¼ 2, . . . , n  k þ 1 jþk X

δj0  k

8j ¼ 1, . . . , n  k

j0 ¼j

Restatement of propositions (3.28) and (3.29) as linear constraints are simple and straightforward and are given to the reader. Example 3.22 Blocks with the length of integer multiplier of k In Example 3.21, assume that every block with nonzero components should have the length of exactly equal to an integer multiplication of k. In other words, for k ¼ 3, the vectors (0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1) and (1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1) are feasible, while the vectors (0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1) and (1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1) are infeasible. To satisfy this condition, provide appropriate linear constraints. Answer: If δ satisfies the aforementioned condition, its nonzero components can be partitioned so that every k successive nonzero components are constituted a specific partition. For example, for k ¼ 3, the components with nonzero values of the vector (1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0) are partitioned into three groups, containing the components 1–3, 4–6, and 9–11, respectively. Concerning the above explanation, for j ¼ 1, 2, . . ., n  k + 1, we define the binary variable ηj as follows:

3.9 Exercises

105

 ηj ¼

1 If there exists a partition starting with jth component 0 Otherwise j ¼ 1, 2, . . . , n  k þ 1

The following proposition guarantees that each partition contains at least k components: ηj ¼ 1 )

jþk1 X

δj0  k

8j ¼ 1, . . . , n  k þ 1

ð3:30Þ

0

j ¼j

The next proposition ensures that there is no overlap between partitions. 

ηj ¼ 1 ^ ηj0 ¼ 1



) j ηj  j0 ηj0  k

8j, j0 : j > j0

ð3:31Þ

The following constraint together with the proposition (3.30) ensure that each block contains exactly k successive nonzero components. nkþ1 X j¼1

Pn ηj ¼

j¼1 δj

k

The linearization of proposition (3.30) is simple, and the linear counterpart of proposition (3.31) is as follows:   j ηj  j0 ηj0  k  M 2  ηj  ηj0 where M can be set equal to k + n.

3.8

Notes and References

Some examples of this chapter and several exercises provided in Sect. 3.9 have been borrowed from Castillo et al. (2002) and Plastria (2002).

3.9

Exercises

Exercise 3.1 Provide appropriate linear constraints for each of the following cases.

106

3

Models and Mathematical Logic

(a) Suppose that δ1 is a binary variable that is 1 if the device is purchased; otherwise 0. Also, assume that δ2 is another binary variable that is 1 if the worker A works on the device; otherwise 0. Write a constraint to indicate that worker A can work on the device if the device is purchased. (b) Suppose that δ1 is a binary variable that is 1 if product A is produced; otherwise 0. Also, assume that δ2 is a binary variable that is 1 if the machine is available for production; otherwise 0. Similarly, define the binary variable δ3 indicting the availability of the worker for production. Provide constraints to ensure that the product A is produced if and only if the machine and the relevant worker are available. (c) Suppose that δi is a binary variable that is 1 if the project i is done; otherwise 0. Write constraints to ensure that project 3 is carried out if and only if project 1 or project 2 or both are done. (d) Suppose that δi is a binary variable that is 1 if the project i is done; otherwise 0. Write constraints to ensure that project 3 is carried out if and only if project 1 or project 2 but not both are carried out. (e) By using the binary variables δ1, δ2, δ3, and δ4, provide constraints limiting the value of the variable x to one of the members  of the set {5, 6, . . ., 15}. (f) Consider the binary vector δ1 , δ2 , . . . , δn as a solution vector. Provide a   constraint ensuring that (δ1, δ2, . . ., δn) is different from δ1 , δ2 , . . . , δn in at least one component. Exercise 3.2 Suppose that δ1, δ2, . . ., δn, and γ 1 and γ 2 are binary variables. Provide linear constraints to comply with the following conditional proposition: n X

δi ¼ 0 ) γ 1 ¼ γ 2

i¼1

Exercise 3.3 Suppose that for j 2 , pj is a proposition and δj is the corresponding binary variable that is 1 if the proposition pj is true; otherwise 0. Also, assume that xi and wi (for i 2 ) are nonnegative continuous variables. Provide appropriate linear constraints to comply with the following statement: ^ ð pj Þ

)

ð8i 2  xi ¼ wi Þ

j2

This statement is a simplified propositional state appearing in stochastic programming problems under endogenous uncertainty. For more details, see Goel and Grossmann (2006) and Hooshmand and MirHassani (2016a, b).

3.9 Exercises

107

Exercise 3.4 Answer the following questions concerning the section of conditional indicators (i.e., Sect. 3.6.1). (a) (b) (c) (d) (e) (f)

For constraint of type , why the relation is valid only for Li  0? For constraint of type , why the relation is valid only for Ui  0? For constraint of type , why the relation is valid only for Ui  ε? For constraint of type ¼, why the relation is valid only for Li  0 and Ui  0? For constraint of type 6¼, why the relation is valid only for Li  ε and Ui  ε?

Exercise 3.5 Answer the following questions concerning the section of bi-conditional indicators (i.e., Sect. 3.6.3). (a) (b) (c) (d) (e) (f)

For constraint of type , why the relation is valid only for Li  0 and Ui  ε? For constraint of type , why the relation is valid only for Li  ε and Ui  0? For constraint of type , why the relation is valid only for Li  0 and Ui  ε? For constraint of type ¼, why the relation is valid only for Li  ε and Ui  ε? For constraint of type 6¼, why the relation is valid only for Li  ε and Ui  ε?

Exercise 3.6 Suppose that for j 2 , pj is a simple proposition, and δj is the corresponding binary variable that is 1 if the proposition pj is true; otherwise 0. Also, assume that 1 and 2 are two nonempty subsets of  such that 1 \ 2 ¼ ∅. Explore the validity of linear constraints provided for each composite proposition of Table 3.8.

Table 3.8 Linear constraints associated with some composite propositions

Composite proposition W W pj ) pj j21

V

j22

pj )

j21

V

pj

j22

pj )

j21

W

V W

pj

j22

pj 

W

j21

j22

V

V

pj

Linear constraints P δj  δj0 8j 2 1 j0 22

P j0 21

P

δj0  δj þ j1 j  1

δj 

j21

δj  δj 

pj 

j21

V j21

pj

j22

pj 

W j22

P j0 21

P

j0 22

pj

P

P

P

8j 2 2

δj þ j1 j  1

j22

j0 22

P

j0 21

δj0

8j 2 1

δj0

8j 2 2

δj0  δj þ j1 j  1

8j 2 2

δj0  δj þ j2 j  1

8j 2 1

P

δj  δj þ j1 j  1 Pj22 δj  δj0 8j 2 1

j21

j0 22

108

3

Models and Mathematical Logic

Exercise 3.7 Rewrite the statement (80  x1  100) _ (5  x1  10) as appropriate linear constraints. Exercise 3.8 Suppose that x 2 [0, 1] is a continuous variable, and δj, for each j 2 , is a binary variable. (a) Rewrite the following logical statement as a linear constraint: ð8j 2  δj ¼ 0Þ ) x ¼ 0 (b) Now, suppose that x 2 [0, a] is a continuous variable. Rewrite the logical statement of part (a) as a linear constraint. Exercise 3.9 (a) Suppose that x1 and x2 are nonnegative continuous variables. Rewrite the constraint x1x2 ¼ 0 linearly. (b) Suppose that x1 and x2 are free continuous variables. Rewrite the constraint x1x2 ¼ 0 linearly. Exercise 3.10 In Example 3.5, prove that if each center has an unlimited capacity and the goal is to minimize the total distance between customers and their service provider, then in the optimal solution each customer will be served by the nearest open center and, no constraints are needed to grantee this requirement. Exercise 3.11 Describe the feasible region of the following problem by linear constraints. max z ¼ 2x1 þ x2 s:t: x ¼ ðx1 , x2 Þ 2  Where Fig. 3.2 demonstrates the feasible region . This problem has already been examined in Example 3.9. Now, consider the nonconvex region  as the union of two convex regions 1 and 2 , as shown in Fig. 3.5, and solve the problem again. Exercise 3.12 Describe the following nonconvex regions in the form of linear constraints (Fig. 3.6).

3.9 Exercises

109

Fig. 3.5  as the union of two convex regions

Fig. 3.6 Information of Exercise 3.12

Exercise 3.13 Rewrite the following constraints in terms of linear constraints: x1 þ x2  4 x1  1  x2  1 x1 , x2  0 Exercise 3.14 An indirect graph G ¼ ð, Þ is given. The graph G0 ¼ ð0 , 0 Þ is a subgraph of G if 0  and 0 . Provide a model to find a perfect subgraph of G (a subgraph is perfect if there exist an arc between each pair of its vertices). Hint: For each i 2 , define the binary variable δi that is 1 if the vertex i is in the subgraph; otherwise 0. Moreover, for each arc ði, jÞ 2 , consider the binary variable γ i, j, that is 1 if the arc (i, j) is in the subgraph; otherwise 0. For every pair of vertices i, j 2 , one of the two following cases may occur: • ði, jÞ 2 = : In this case, a constraint should be added to ensure that if δi ¼ 1, then δj ¼ 0. • ði, jÞ 2 : In this case, a constraint should be added to ensure that γ i, j ¼ 1 if and only if δi ¼ 1 and δj ¼ 1.

110

3

Models and Mathematical Logic

Exercise 3.15 Suppose that y1, y2 and y3 are integer variables, and k1, k2, and k3, and k4 are integer constants. Add appropriate constraints to ensure the following propositions: (a) y1 ¼ k1 ) y2 ¼ k2 (b) [( y1  k1) ^ (k2  y2  k3)] ) y3  k4 Exercise 3.16 In the following, another approach is presented to observe the propositions provided in Example 3.7. Check the correctness of this method. Consider the constant ε > 0, and assume that u and l are, respectively, the upper and lower bounds on the free variable x with l < 0 < u. Define the binary variable δ as follows:  δ¼

0

x¼0

1 Otherwise

Approximate the above definition as follows:  δ¼

0

ε  x  ε

1

Otherwise

Now, define binary variables δ1, δ2 and δ3, and add the following constraints: εδ1 þ lδ2 þ ðεÞδ3  x  uδ1 þ ðεÞδ2 þ εδ3 δ1 þ δ2 þ δ3 ¼ 1 δ ¼ 1  δ3 δ1 , δ2 , δ3 2 f0, 1g Hint: Three situations may occur for x. Check the following statements: • If ε  x  u then δ ¼ 1. • If l  x  ε then δ ¼ 1. • If ε  x  ε then δ ¼ 0. Exercise 3.17 In the following model,rewrite the conditional constraints as a set of inequalities. The optimal solution is x1 , x2 , y1 , y2 ¼ ð7, 7, 1, 1Þ, z ¼ 18.

3.9 Exercises

111

min z ¼ ðx1  10Þ2 þ ðx2  10Þ2 s:t: x1 x2  1 ) x21 þ x22  98 y1 6¼ y2 ) x1 þ x2  10 0  x1 , x2  10 y1 , y2 2 f1, 2g Exercise 3.18 Consider the following proposition: ð1  x  4 ^ y 2 f1, 3, 6gÞ ) ðw ¼ 2 _ 5  w  8Þ To rewrite this proposition in the form of linear constraints, one has defined binary variables δ1, . . ., δ4 and δ01 , δ02 , δ03 and δ001 , . . . , δ005 and has presented the following constraints: x ¼ 1δ1 þ 2δ2 þ 3δ3 þ 4δ4 y ¼ 1δ01 þ 3δ02 þ 6δ03 w ¼ 2δ001 þ 5δ002 þ 6δ003 þ 7δ004 þ 8δ005    ðδ1 þ . . . þ δ4 ¼ 1Þ ^ δ01 þ δ02 þ δ03 ¼ 1 ) δ001 þ . . . þ δ005 ¼ 1 δ1 , . . . , δ4  0, δ01 , δ02 , δ03 2 f0, 1g, δ001 2 f0, 1g, δ002 , . . . , δ005  0

First, explain why the above formulation is incorrect. Then, provide appropriate linear constraints to satisfy the given proposition. Exercise 3.19 Consider a port in which several ships travel (the set of ships is represented by  with the index i). In this port, each ship may be kept for repairs for two consecutive months. On the other hand, during each month, the capacity of the port is sufficient to hold a maximum of four ships. Suppose that the ship’s scheduling is carried out for T months and at the beginning of the planning horizon, no ship is stopped at the port. (a) Let δi, t be a binary variable that is 1 if the repair of the ship i begins at the beginning of the month t and terminates at the end of month t + 1; otherwise 0. Provide appropriate constraints to ensure that the capacity of the port is observed in each month. (b) Now, change the definition for the binary variable δi, t, and assume that it is 1 if the ship i is being repaired during the month t; otherwise 0. Again provide appropriate constraints to ensure that the capacity of the port is observed in each month.

112

3

Models and Mathematical Logic

Exercise 3.20 Consider a production line for which three states “active”, “repairing”, and “inactive” are possible during each period, t, and denote the states by values 1, 2 and 3, respectively. At the beginning of the planning horizon, the production line is active. Assume that the binary variable δk, t is 1 if the production line is in state k in period t; otherwise 0. Provide constraints ensuring that in each period, the production line should be in only one status, and if it becomes inactive during a period, it must remain inactive until the end of the planning horizon (i.e., period T ). Exercise 3.21 Let  (with index i) include a variety of diseases and  (with index j) be a set of symptoms according to which physicians make decisions when identifying and diagnosing the diseases. Assume that ci, j represents the level and intensity of the jth symptom in disease i, and let di, i0 , j be a binary parameter that is 1 if i and i0 are different in terms of the intensity of the symptom j; otherwise 0. In order to expedite the identification of diseases, physicians intend to merely decide based on a small subset of symptoms, which we refer to as 0 . Nevertheless, in order to avoid any error in the diagnostic process, the set 0 must be chosen so that each pair P of diseases i and 0 0 i are different in at least k symptoms of the set  , in other words, j20 d i, i0 , j  k. (a) Provide an optimization model to minimize the number of selected symptoms. (b) Suppose that ¼{1, . . ., 5},  ¼ f1, . . . , 8g, and let the intensity of the symptoms in each disease is following Table 3.9. According to the given information, solve the model of the part (a) for k ¼ 1 and k ¼ 3. Exercise 3.22 A company produces a specific commodity and should supply the customers’ demand. The company’s director plans to decide on the location of the selling centers. Suppose that  (with index i) is a set of potential sites where the selling centers can be located, and  (with index j) is the set of customers. The demand of customer j is equal to dj, the fixed cost of opening the center placed at i is equal to fi, the profit per unit of sale of commodities transferred from location i to customer j is equal to pi, j, and the capacity of the selling center placed at i, is equal to ai. The company must decide where to locate the selling centers and determine the amount of commodity sent from center i to customer j so that the total profit of the company is maximized.

Table 3.9 The intensity of the symptoms in each disease

Disease 1 2 3 4 5

Symptoms 1 2 2 3 1 1 3 4 2 2 1 1

3 1 1 2 2 1

4 1 1 3 2 2

5 1 3 2 2 1

6 2 1 2 1 1

7 1 2 3 2 1

8 2 1 2 3 2

3.9 Exercises Table 3.10 Information of Exercise 3.22

113

Site 1 2 3 4 5 6 Demand

Customer 1 2 4 4.5 4 4.5 3.5 5 1.3 3 0.5 1 1 0 1.5 2

3 2.5 2.5 4 5 1.5 1.5 3

4 0.5 4.2 3.5 3.3 5 3.3 4

5 1 3.5 4.5 5.5 4 4 2.5

6 0.5 1.5 1.5 1.8 5.5 4.5 1

7 3.5 0.5 0 1.3 3 2 2

(a) Provide an optimization model for this problem. (b) Suppose that ¼{1, . . ., 6},  ¼ f1, . . . , 7g, f i ¼ 10, ai ¼ 6. The transfer profit and the customer demand are in accordance with Table 3.10. According to this information, solve the model of the part (a). Exercise 3.23 The network G ¼ ð, Þ is given and the aim is to find the maximum number of paths in G from the origin O to the destination D such that they are arc-disjoint (two different paths are called arc-disjoint if they do not have any common arc). Provide an IP model for this problem. Exercise 3.24 Suppose that  ¼ f1, . . . , ng, with index j, and  ¼ f1, . . . , mg, with index i, are the sets of jobs and persons, respectively where the number of jobs is greater than the number of persons (i.e., n > m). Each job should be allocated to at most one person, and each person can do more than one job. The profit of assigning the person i to the job j is known and denoted by pi, j. The jobs should be allocated to persons with the aim of maximizing the total profit while keeping a balance between the individual’s profits. In other words, the goal is to establish a balance between the company’s total profit and individual’s profits.

Chapter 4

Linearization of Nonlinear Functions

4.1

Introduction

Many optimization models describing real-life problems may include nonlinear terms in their objective function or constraints. For example, in the natural gas pipeline scheduling problem, the relation between pressure drop and flow-rate are described by nonlinear equations. In a portfolio optimization problem, the problem requirements may create nonlinear terms in the constraints or the objective function. The range of NLP applications is extensive. For a recent comprehensive review, see Boukouvala et al. (2016). In particular, any BIP and MIP problem can be rewritten as an NLP problem. To this end, it is enough to replace the restriction δ 2 {0, 1} by δ 2 [0, 1] to establish the continuity assumption for δ and add the following nonlinear constraints to the model: δð1  δÞ ¼ 0 Also, to express the integrality condition on the integer variable y, instead of using the restriction y Int., we can use the following nonlinear constraint: sinðπyÞ ¼ 0 Therefore, all problems formulated in the form of IP, MIP, and BIP can also be expressed as nonlinear models. However, because of the computational complexity of solving nonlinear models, it is often not computationally feasible to rewrite and solve a linear integer programming model as a nonlinear one. Practically, it is almost preferred to rewrite nonlinear models in the form of an equivalent linear formulation or to obtain an appropriate linear approximation. If one can turn a nonlinear problem into an equivalent linear problem and solve it by appropriate LP-based methods, it is expected that a precise solution is obtained at a more reasonable time. Thus, in this chapter, we introduce some nonlinear functions that frequently appear in © Springer Nature Switzerland AG 2019 S. A. MirHassani, F. Hooshmand, Methods and Models in Mathematical Programming, https://doi.org/10.1007/978-3-030-27045-2_4

115

116

4 Linearization of Nonlinear Functions

optimization problems and discuss how they can be represented in the form of linear functions. In this regard, we may also need to use integer and binary variables. Throughout the text, anywhere we talk about “linearization of a model”, we mean that the model to be restated as an equivalent standard one (see Remark 2.2). First, we examine the terms often seen in nonlinear functions.

4.2

Variables Multiplication

The multiplication of two or more variables is a phenomenon that is often seen in mathematical models. Since such expressions destroy the model linearity and bring complexities to the solution process, the modelers always try to avoid the multiplication of variables as much as possible, or replace them with appropriate linear terms, if possible. In this section, all situations for the variables multiplication are examined, and in each case, a suitable linearization strategy is presented.

4.2.1

Multiplication of Binary Variables

Suppose that δ1, δ2, . . ., δn are
binary variables. If the objective function or conn straints contain the expression δj j (nj is a nonnegative integer), it is obvious that this nonlinear expression Q can be replaced by δj. However, if the model contains the nonlinear expression nj¼1 δj , we replace it by binary variable γ and add some constraints to the model to guarantee the following propositions: ðδ1 ¼ 0 _ δ2 ¼ 0 _ . . . _ δn ¼ 0Þ , γ ¼ 0

ð4:1Þ

ðδ1 ¼ 1 ^ δ2 ¼ 1 ^ . . . ^ δn ¼ 1Þ , γ ¼ 1

ð4:2Þ

Since the statement p ) q is equivalent to ~q ) ~p, the backward implication of proposition (4.1) is equivalent to ðδ1 ¼ 1 ^ δ2 ¼ 1 ^ . . . ^ δn ¼ 1Þ ) γ ¼ 1 By a similar argument, it can be concluded that the backward implication of proposition (4.2) is equivalent to ð δ 1 ¼ 0 _ δ 2 ¼ 0 _ . . . _ δ n ¼ 0Þ ) γ ¼ 0 Therefore, it is only required to satisfy the forward implication of propositions (4.1) and (4.2) as follow:

4.2 Variables Multiplication

117

ðδ1 ¼ 0 _ δ2 ¼ 0 _ . . . _ δn ¼ 0Þ ) γ ¼ 0

ð4:3Þ

ðδ1 ¼ 1 ^ δ2 ¼ 1 ^ . . . ^ δn ¼ 1Þ ) γ ¼ 1

ð4:4Þ

Proposition (4.3) is equivalent to the following constraint set: γ  δj

8j ¼ 1, . . . , n

ð4:5Þ

Proposition (4.2) is equivalent to the following constraint: n X

δj  γ þ ðn  1Þ

j¼1

Remark 4.1 Proposition (4.3) is also equivalent to the following linear constraint, as well. nγ 

n X

δj

ð4:6Þ

j¼1

However, in Chap. 5 we will explain that the constraint set (4.5) is computationally more efficient than (4.6). Note that if the proposition (4.3) is linearized by constraint set (4.5), then γ can be considered as a continuous variable in the interval [0, 1] [see Glover and Woolsey (1974) for more details]. For some applications of this nonlinear expression, refer to Plastria (2002). According to the above explanations, any polynomial programming problem with binary variables can be rewritten as a BIP problem. To clarify the hdiscussion, assume Q
n i that f(δ1, . . ., δn) is a polynomial function of binary variables. Let ai kj¼1 δj j , as the product of k binary variables, be a general representation of the ith term of function f. Then, we can get the equivalent linear model by implementing the following steps: n

1. Replace δj j with δj, Q 2. Replace kj¼1 δj with binary variable γ, 3. Add the following constraints to the model, γ  δj k X j¼1

8j ¼ 1, . . . , k:

δ j  γ þ ð k  1Þ

118

4.2.2

4 Linearization of Nonlinear Functions

Multiplying Binary and Continuous Variables

Assume that x is a nonnegative continuous variable, δ is a binary variable, and the parameter u is a known upper bound for x. If the objective function or constraints of the model contains the term xδ, to linearize the model, we define the nonnegative continuous variable w and replace the expression xδ with w. Also, we add some constraints to the model to guarantee the following propositions: δ¼0)w¼0

ð4:7Þ

δ¼1)w¼x

ð4:8Þ

These statements are equivalent to the following linear inequalities: w  uδ

ð4:9Þ

w0

ð4:10Þ

wx

ð4:11Þ

w  x  uð 1  δ Þ

ð4:12Þ

Constraints (4.9) and (4.10) guarantee the statement (4.7), while constraints (4.11) and (4.12) ensure the validity of statement (4.8). For some applications of this nonlinear expression, see Wu (1997). Example 4.1 Suppose that x is a nonnegative continuous variable and δ1, . . ., δn are binary variables. Also, let u be a known upper bound on x. If the model consists of a multiplication term δ1. . .δnx, provide a suitable replacement for linearization. Answer: The term δ1. . .δnx is replaced by nonnegative continuous variable w and following constraints are added to the model. w  uδj

j ¼ 1, . . . n

wx w  x  uð n  ð δ 1 þ . . . þ δ n Þ Þ w0 Above constraints guarantee the following statements: ð δ 1 ¼ 0 _ δ 2 ¼ 0 _ . . . _ δ n ¼ 0Þ ) w ¼ 0 ð δ 1 ¼ 1 ^ δ 2 ¼ 1 ^ . . . ^ δ n ¼ 1Þ ) w ¼ x

4.2 Variables Multiplication

4.2.3

119

Multiplying Binary and Integer Variables

Suppose that y is a nonnegative integer variable, δ is a binary variable and the integer constant n is a known upper bound on y. If the objective function or constraints of the model include the term yδ, to linearize the model, we define the nonnegative continuous variable w and replace the term yδ with w. Then, similar to Sect. 4.2.2, the following constraints are added to the model: w  nδ wy w  y  nð 1  δ Þ w0

4.2.4

Converting Integer Variable to Binary

Among the methods proposed to solve MIP problems, there are techniques specially designed to solve BIP problems. If we want to apply such methods to solve problems that also include integer variables, we need to rewrite the integer variables in terms of binary variables. Suppose that y is a nonnegative integer variable, and n represents a known integer upper bound on y. The following three methods can be used to restate y in terms of binary variables: Method 1: P Introduce n binary variables δ1, . . ., δn, and replace the variable y with nj¼1 δj . Method 2: δ1, . . ., δn, and replace the variable P Introduce n binary variables P y with nj¼1 jδj . Also, add the constraint nj¼1 δj  1 to the model. The above two methods are not desirable because they require a large number of binary variables. Method 3: Since any integer number can be represented in the base-2 numeral P system, then y can be written as y ¼ kj¼0 2j δj where δj 2 {0, 1}. But the question arisen here, is that what is the value of k? P The maximum value of kj¼0 2j δj is obtained when every δj takes the value of 1. Thus, we have: k X j¼0

2j δj 

k X j¼0

2j ¼ 2kþ1  1

120

4 Linearization of Nonlinear Functions

Since n is an upper bound on y, it is enough to consider k as the smallest integer number such that 2k  n < 2k+1, in other words, k ¼ blog2(n)c. Example 4.2 Replacement of an integer variable by binaries Consider the integer variable y and assume that 0  y  10. Rewrite y in terms of binary variables. Answer: We use the third method. For this purpose, we set k ¼ blog210c ¼ 3 and P define four binary variables δ0, . . ., δ3 and replace y with the expression 3j¼0 2j δj . Therefore, we have: 3 X

2j δj  10

j¼0

δj 2 f0, 1g

4.2.5

8j ¼ 0, 1, 2, 3

Multiplying Two Integer Variables

Suppose that y1 and y2 are nonnegative integer variables, and let n1 and n2 be known integers upper bounds on y1 and y2, respectively. If the model objective function or constraints contain the multiplicative expression y1y2, to linearize the model, first, P we express y1 in terms of binary variables (i.e., y1 ¼ kj¼0 2j δj , where k ¼ blog2(n1)c). Then, we replace y1y2 by the following term: k X

! 2 δ j y2 j

j¼0

Now, each multiplicative term δjy2 can also be replaced by a linear expression in the way described in Sect. 4.2.3.

4.2.6

Multiplying Integer and Continuous Variables

Assume that x is a nonnegative continuous variable, and y is a nonnegative integer variable, and suppose that u and n are the known upper bounds of the variables x and y, respectively. If the objective function or the model constraints contain the multiplicative expression xy, to linearize the model, first, we rewrite the variable y in P terms of binary variables (i.e., y ¼ kj¼0 2j δj , where k ¼ blog2(n)c). In this way, the expression xy is replaced by the following term:

4.2 Variables Multiplication

121

x

k X

! 2 δj j

j¼0

Now, each multiplicative expression xδj can also be replaced by appropriate linear expressions as described in Sect. 4.2.2.

4.2.7

Multiplying Two Continuous Variables

Let x1 and x2 be nonnegative continuous variables. If the objective function or the model constraints contain the multiplicative expression x1x2, we encounter with a nonconvex problem that is complex and difficult to solve. Of course, sometimes, it is possible to apply approximation techniques to the problem and replace it with a simpler problem. Here, three approximation methods are presented. The first method is based on the McCormick envelop relaxation (MCER), the second one utilizes the multi-parametric disaggregation technique (MPDT) and, the third method is based on PLF.

4.2.8

Approximation by McCormick Relaxation

Nonconvex NLP problems are complex and difficult to solve, but sometimes they can be relaxed and approximated by convex problems. In a convex problem, every locally optimal solution is also global, and hence, it has less complexity than the original problem. Of course, the optimal solution to the relaxed problem is not necessarily in agreement with the solution of the original problem, but it generally provides lower (upper) bound for the minimization (maximization) problems. Therefore, the relaxations approximating the problem with a convex one, while producing stronger bounds are more desirable. The McCormick envelop relaxation (MCER) is one of the relaxation approaches that are suitable for a group of nonlinear problems including multiplicative terms as xixj. Suppose that x1 and x2 are continuous variables with upper bounds u1 and u2 and lower bounds l1 and l2, respectively. l 1  x 1  u1 ,

l 2  x 2  u2

Moreover, assume that the constraints or the objective function of the model contain the multiplicative term x1x2. It is clear that the following inequalities are valid.

122

4 Linearization of Nonlinear Functions

ð u1  x 1 Þ ð u2  x 2 Þ  0 ð x1  l 1 Þ ð x2  l 2 Þ  0 ð u1  x 1 Þ ð x 2  l 2 Þ  0 ð x 1  l 1 Þ ð u2  x 2 Þ  0 The above inequalities can be rewritten as follows: x1 x2  u1 u2 þ u2 x1 þ u1 x2

ð4:13Þ

x1 x2  l1 l2 þ l2 x1 þ l1 x2

ð4:14Þ

x1 x2  u1 l2 þ l2 x1 þ u1 x2

ð4:15Þ

x1 x2  l1 u2 þ u2 x1 þ l1 x2

ð4:16Þ

Given the inequalities mentioned above, if the problem involves the nonlinear term x1x2, it can be approximated by an LP (and also convex) problem. Therefore, it is enough to introduce a new variable w, replace the term x1x2 with w, and finally, add the following constraints to the model. w  u1 u2 þ u2 x1 þ u1 x2

ð4:17Þ

w  l1 l2 þ l2 x1 þ l1 x2

ð4:18Þ

w  u1 l2 þ l2 x1 þ u1 x2

ð4:19Þ

w  l1 u2 þ u2 x1 þ l1 x2

ð4:20Þ

The inequalities (4.17) and (4.18) provide under-estimators for w, while the inequalities (4.19) and (4.20) introduce over-estimators. For more illustration, see Fig. 4.1. Figure 4.2 demonstrates the cross-section view of surfaces of Fig. 4.1. Remark 4.2 If the variables x1 and x2 are binary, the formulation that is obtained from the MCER is precise and it is equivalent to the model obtained by the linearization process described in Sect. 4.2.1. Also, if one of the variables x1 or x2 is continuous and the other is binary, the formulation resulting from the MCER is exact and equivalent to the process described in Sect. 4.2.2 for the multiplication of a binary variable and a continuous one. Example 4.3 Approximation via MCER Approximate the following problem by MCER.

4.2 Variables Multiplication

123

Fig. 4.1 MCER of the term x1x2. (a) Multiplicative term w ¼ x1x2. (b) Under-estimator for w. (c) Over-estimator for w

124

4 Linearization of Nonlinear Functions

Fig. 4.2 Cross-section view of surfaces of Fig. 4.1

min z ¼ x1 x2  2x1 s:t: x1 x2  12 1  x1  6 1  x2  3 Answer: Using the MCER approach, the problem is relaxed as follows: min z ¼ w  2x1 s:t: w  12 1  x1  6 1  x2  3 w  18 þ 3x1 þ 6x2 w  1 þ x1 þ x2 w  6 þ x1 þ 6x2 w  3 þ 3x1 þ x2 w0 The optimal solution to the original problem is as follows:

4.2 Variables Multiplication

125

x1 ¼ 6, x2 ¼ 2, z ¼ 24 However, the optimal solution of the relaxed problem is: x1 ¼ 6, x2 ¼ 2, w ¼ 12, z ¼ 24 In this example, the solution obtained by MCER is equal to the optimal solution to the original problem. Example 4.4 Approximation via MCER Approximate the following problem by MCER. min z ¼ 1000  x21  2x22  x23  x1 x2  x1 x3 s:t: 8x1 þ 14x2 þ 7x3 ¼ 56 x21 þ x22 þ x23 ¼ 25 xi  0

8i ¼ 1, 2, 3

Answer: Given the problem constraints, the upper bounds of variables x1, x2 and x3 are 5, 4, and 5, respectively. Now, we replace the multiplicative terms as follows: w1,1 ¼ x1x1, w2,2 ¼ x2x2, w3,3 ¼ x3x3, w1, 2 ¼ x1x2, w1, 3 ¼ x1x3 The relaxed problem obtained by MCER approach is as follows: min z ¼ 1000  w1, 1  2w2, 2  w3, 3  w1, 2  w1, 3 s:t: 8x1 þ 14x2 þ 7x3 ¼ 56 w1, 1 þ w2, 2 þ w3, 3 ¼ 25 xi  0

8i ¼ 1, 2, 3

w1, 1  25 þ 10x1 w1, 1  0 w1, 1  5x1 w2, 2  16 þ 8x2 w2, 2  0 w2, 2  4x2 w3, 3  25 þ 10x3 w3, 3  0 w3, 3  5x3

126

4 Linearization of Nonlinear Functions

w1, 2  20 þ 4x1 þ 5x2 w1, 2  0 w1, 2  4x1 w1, 2  5x2 w1, 3  25 þ 5x1 þ 5x3 w1, 3  0 w1, 3  5x1 w1, 3  5x3 The optimal solution to the original problem is as follows: x1 ¼ 3:512118414, x2 ¼ 0:2169881741, x3 ¼ 3:552174034, z ¼ 961:7151721 The optimal solution to the relaxed problem is as follows: x1 ¼ 2:137405, x2 ¼ 1:709924, x3 ¼ 2:137405, w1, 1 ¼ 7:473282, w2, 2 ¼ 6:839694, w3, 3 ¼ 10:687023, w1, 2 ¼ 8:549619, w1, 3 ¼ 10:687023, z ¼ 948.92366 It should be noted that this method requires to identify lower and upper bounds on each variable which can be calculated according to the problem structure and concerning other constraints. However, it is clear that the more precise the bounds generated, the better the relaxation achieved. In Nagarajan et al. (2016), an iterative method is proposed to improve the lower and upper bounds. To illustrate the method, ^Þ is a feasible solution. we consider the following problem and suppose that ðx^, w min z ¼ f ðx, wÞ s:t: gðx, wÞ  0 wi, j ¼ xi xj 8i, j ¼ 1, . . . , n : i  j ^lj  xj  u^j 8j ¼ 1, . . . , n 8i, j ¼ 1, . . . , n : i  j wi, j free Initially, the primary boundaries ^lj and u^j are identified with respect to the problem structure. Then, the following steps are followed: Step 1. Set lk ≔^lk and uk ≔^ uk for k ¼ 1, . . ., n. Step 2. Solve the following problems for k ¼ 1, . . ., n, where the term MC(xixj) means that the relaxation is enforced on the term xixj with respect to the bounds lj and uj.

4.2 Variables Multiplication

127

~lk ¼ min xk

u~k ¼ max xk

s:t:

s:t:

^Þ f ðx, wÞ  f ðx^, w gðx, wÞ  0
 wi, j ¼ MC xi xj 8i, j ¼ 1, . . . , n : i  j

f ðx, wÞ  f ðx^, w^Þ gðx, wÞ  0
 wi, j ¼ MC xi xj 8i, j ¼ 1, . . . , n : i  j

l j  x j  uj

l j  x j  uj

wi, j

free

8j ¼ 1, . . . , n 8i, j ¼ 1, . . . , n : i  j

wi, j

free

8j ¼ 1, . . . , n 8i, j ¼ 1, . . . , n : i  j

 P  Step 3. Assuming that ε > 0 is a certain threshold, if both criteria nj¼1 ~lj  lj  < ε  P  and nj¼1 u~j  uj  < ε are satisfied, stop; the bounds ~lj and u~j are suitable for uj , and go to step 2. j ¼ 1, . . ., n. Otherwise, set lj ≔~lj and uj ≔~ Example 4.5 Bounds improvement In Example 4.4, apply the bound improvement method and implement the MCER concerning the new bounds. Then, compare the new results with that of Example 4.4. Answer: Initially, the upper bounds of variables x1, x2 and x3 are set at 5, 4, and 5, respectively, and the lower bounds of all variables are set at 0. Additionally, ε ¼ 0.0001 is considered as desired precision and the following feasible solution to the original problem is generated: x^1 ¼ 4:846153846,

x^2 ¼ 1:230769231,

x^3 ¼ 0,

z^ ¼ 967:5207101 The bounds obtained in different iterations of the bound improvement method are reported in Table 4.1. If the MCER is implemented based on the improved bounds obtained in iteration 10, the optimal solution to the relaxed problem is as follows: x1 ¼ 3:310812742, x2 ¼ 0:7432315675, x3 ¼ 2:729750874, w1, 1 ¼ 10:97085588, w2, 2 ¼ 0:9682244087, w3, 3 ¼ 13:06091971, w1, 2 ¼ 3:716157837, w1, 3 ¼ 13:64875437, z ¼ 956:6668634 The results indicate that by improving the bounds, the optimal solution to the relaxed problem becomes closer to that of the original problem. Remark 4.3 Note that if the optimization model contains multi-linear terms in the form of x1x2 . . . xk with k  3, the MCER can be used repeatedly during k  1 steps. For example, for the nonlinear term x1x2x3, we have:

128

4 Linearization of Nonlinear Functions

Table 4.1 Improved bounds Iteration 1 2 3 4 5 6 7 8 9 10

l1 0.0 0.0 0.3414104125 0.6822400516 0.9048415285 0.027809401 0.059372379 0.065453609 0.066644597 1.066884462

l2 0 0 0 0 0 0 0 0 0 0

l3 0 0 0 0 0 0 0 0 0 0

u1 5 5 5 5 5 5 5 5 5 5

u2 4.0 2.5 1.967484723 1.623222244 1.365808531 1.316918527 1.305886913 1.303369061 1.302833159 1.302722396

u3 5.0 5.0 5.0 4.934524911 4.864667828 4.798690351 4.787172422 4.785124837 4.784734567 4.784656299

x1 x2 x3 ¼ ðx1 x2 Þx3 ¼ wx3 ¼ w0 The following substitutions are used step-by-step, and accordingly, the constraints associated with the MCER are added: w ¼ x1 x2 w0 ¼ wx3 The grouping is not unique. For example, for the nonlinear term x1x2x3, three choices (x1x2)x3, (x2x3)x1 and (x1x3)x2 are possible. The way of grouping variables can affect the quality of the relaxation. For more details, see Speakman and Lee (2017).

4.2.9

The Multi-parametric Disaggregation Technique

Each real number x can be represented as follows: x¼

X


ak  10k



k2ℤ

For each k 2 ℤ, the value of ak belongs to the set {0, 1, . . ., 9} and represents a digit of x in the decimal representation, where the ak with k  0 refers to a digit placed before the decimal point and the ak with k < 0 represents a digit placed after the decimal point. If x is bounded above, there exists an integer number K > 0 where x < 10Kþ1 . Also, by taking into account a desired accuracy, we can choose an integer number K < 0 and ignore all digits of x with the place value less than 10K . Given the above assumptions, x can be approximated by x~ as follows:

4.2 Variables Multiplication

129

x ffi x~ ¼ 

X ðak  10k Þ k2

 where  ¼

 K, K þ 1, . . . , 0, 1, . . . , K  1, K .

Now, according to the above explanation, we can describe the method of MPDT. Assume that x1 and x2 are nonnegative continuous variables with upper bounds u1 and u2, respectively, such that 0  x1  u1, 0  x2  u2. Also, suppose that the model contains the term x1x2, and the aim is to approximate it as an MILP. Concerning the upper bound u1, we select the smallest integer number K > 0 so that u1 < 10Kþ1 and determine the integer number K < 0 indicating that in the approximation of x1, up to jKj decimal places are considered and hence, the inaccuracy will be less than 10K. Accordingly, we form the set  ¼

K, K þ 1, . . . , 0, 1, . . . , K  1, K , and define the variable δj,k as follows:

δj,k :

A binary variable that is 1 if in the decimal representation of x1, the digit with the place value of 10k equals j; otherwise 0 ðj 2 f0, 1, . . . , 9g, k 2 Þ.

Therefore, x1 is approximated as follows: x1 ffi

9 XX

ðj  10k  δj, k Þ

ð4:21Þ

k2 j¼0

where, 9 X

δj, k ¼ 1

8k 2 

ð4:22Þ

j¼0

Therefore, we have: x1 x2 ffi

9 XX

ðj  10k  δj, k  x2 Þ

k2 j¼0

If the multiplicative term δj, kx2 is replaced by a nonnegative continuous variable vj,k, then we get: x1 x2 ffi

9 XX

ðj  10k  vj, k Þ

k2 j¼0

The following propositions describe the relationship between the variables δj,k, vj,k and x2.

130

4 Linearization of Nonlinear Functions

δj, k ¼ 0 ) vj, k ¼ 0

8j 2 f0, 1, . . . , 9g, k 2 

ð4:23Þ

δj, k ¼ 1 ) vj, k ¼ x2

8j 2 f0, 1, . . . , 9g, k 2 

ð4:24Þ

(4.23) is equivalent to v j , k  u2 δ j , k

8j 2 f0, . . . , 9g, k 2 

ð4:25Þ

According to (4.22), it can be concluded that:
 δj, k ¼ 1 ) δj0 , k ¼ 0 8j0 2 f0, 1, . . . , 9g : j0 6¼ j The above proposition together with (4.25) ensures that:
 δj, k ¼ 1 ) vj0 , k ¼ 0 8j0 2 f0, 1, . . . , 9g : j0 6¼ j Therefore, we have: 0 δj, k ¼ 1 ) @

9 X

1 vj0 , k ¼ vj, k A

0

j ¼0

Concerning the above proposition, the following constraint ensures (4.24). x2 ¼

9 X

vj 0 , k

8k 2 

ð4:26Þ

j0 ¼0

According to the above explanations, if the model contains the term x1x2, we can replace the multiplicative expression with nonnegative continuous variables w and add the following constraints to the model. In this way, an MILP model is obtained, which is approximately the same as the original model. w¼

9
XX

j  10k  vj, k



k2 j¼0

ð4:21Þ, ð4:22Þ, ð4:25Þ, ð4:26Þ w0 δj, k 2 f0, 1g 8j 2 f0, 1, . . . , 9g, k 2  vj, k  0 8j 2 f0, 1, . . . , 9g, k 2  This method can also be extended to approximate the nonlinear problems containing the multiplicative terms in the form of xn11 xn22 . . . xnmm . For more details, see Teles et al. (2013).

4.2 Variables Multiplication

131

Remark 4.4 As mentioned earlier, proposition (4.24) is ensured by constraint set (4.26). As another approach, note that the statement (4.24) can be guaranteed through the following constraints, as well. vj, k  x2

8j 2 f0, 1, . . . , 9g, k 2 

vj, k  x2  u2 ð1  δj, k Þ

8j 2 f0, 1, . . . , 9g, k 2 

ð4:27Þ ð4:28Þ

Example 4.6 Use MPDT to approximate the following model: min z ¼ x1 x2  2x1 s:t: x1 x2  12 1  x1  6 1  x2  3 Answer: Let K ¼ 4, K ¼ 0, and  ¼ f4,  3,  2,  1, 0g. We substitute the term x1x2 by the nonnegative continuous variables w. Then, the model is approximated by an MILP model as follows: min z ¼ w  2x1 s:t: w  12 1  x1  6 1  x2  3 x1 ¼

0 X 9
X  j  10k  δj, k k¼4 j¼0

9 X

δj, k ¼ 1

8k 2 f4, 3, 2, 1, 0g

j¼0

w¼

0 X 9
X  j  10k  vj, k k¼4 j¼0

vj, k  3δj, k

8j 2 f0, . . . , 9g, k 2 f4, 3, 2, 1, 0g

132

4 Linearization of Nonlinear Functions

x2 ¼

9 X

vj , k

8k 2 f4, 3, 2, 1, 0g

j¼0

w0 δj, k 2 f0, 1g vj, k  0

8j 2 f0, 1, . . . , 9g, k 2 f4, 3, 2, 1, 0g

8j 2 f0, 1, . . . , 9g, k 2 f4, 3, 2, 1, 0g

The optimal solution to the above model is as follows: δ0, 4 ¼ δ0, 3 ¼ δ0, 2 ¼ δ0, 1 ¼ δ6, 0 ¼ 1 v0, 4 ¼ v0, 3 ¼ v0, 2 ¼ v0, 1 ¼ v6, 0 ¼ 2 The above solution indicates that




 x1 ¼ 0  104 þ 0  103 þ 0  102 þ 0  101 þ 6  100 ¼ 6,

x2 ¼ 2:

Also, the optimal solution to the original model is as follows: x1 ¼ 6, x2 ¼ 2 and z ¼ 24 In this example, the solution obtained by the approximation method is equal to the solution to the original problem. Example 4.7 Use MPDT to approximate the following model: min z ¼ 9x21 þ x22 s:t: x1 x2  1 0  x1  2 1  x2  2 Answer: Let K ¼ 4, K ¼ 0, and  ¼ f4,  3,  2,  1, 0g. We replace the nonlinear terms x21 , x22 and x1x2 by the nonnegative continuous variables w1,1, w2,2 and w1,2 respectively, and define the following variables: δj,k : δ0j, k :

A binary variable that is 1 if in the decimal representation of x1, the digit with the place value of 10k equals j; otherwise 0. ðj 2 f0, 1, . . . , 9g, k 2 Þ A binary variable that is 1 if in the decimal representation of x2, the digit with the place value of 10k equals j; otherwise 0. ðj 2 f0, 1, . . . , 9g, k 2 Þ

4.2 Variables Multiplication

133

Therefore, we have: x21 ffi x1

0 X 9
X

j  10k  δj, k



k¼4 j¼0

x22 ffi x2

0 X 9   X j  10k  δ0j, k k¼4 j¼0 0 X 9
X

x1 x2 ffi x2

j  10k  δj, k



k¼4 j¼0

Note that in this example, variables x1 and x2 vary in the same domain, and hence, the same set  is considered for both variables. This is not the case in practice and it may be required to define the set  for each variable, independently. The multiplicative terms x1δj,k, x2 δ0j, k and x2δj,k are replaced by nonnegative continuous variables v1,1,j,k, v2,2,j,k and v1,2,j,k, respectively. Thus, the model is approximated by the following MILP model: min z ¼ 9w1, 1 þ w2, 2 s:t: w1, 2  1 0  x1  2 1  x2  2 x1 ¼

0 X 9
X

j  10k  δj, k



k¼4 j¼0

x2 ¼

0 X 9   X j  10k  δ0j, k k¼4 j¼0

9 X

δj, k ¼ 1

8k 2 f4, 3, 2, 1, 0g

j¼0 9 X

δ0j, k ¼ 1

8k 2 f4, 3, 2, 1, 0g

j¼0

w1, 1 ¼

0 X 9
X

j  10k  v1, 1, j, k



k¼4 j¼0

w2, 2 ¼

0 X 9
X k¼4 j¼0

j  10k  v2, 2, j, k



134

4 Linearization of Nonlinear Functions

w1, 2 ¼

0 X 9
X

j  10k  v1, 2, j, k



k¼4 j¼0

v1, 1, j, k  2δj, k

8j 2 f0, . . . , 9g, k 2 f4, 3, 2, 1, 0g

2δ0j, k

8j 2 f0, . . . , 9g, k 2 f4, 3, 2, 1, 0g

v1, 2, j, k  2δj, k

8j 2 f0, . . . , 9g, k 2 f4, 3, 2, 1, 0g

v2, 2, j, k 

x1 ¼

9 X

v1, 1, j, k

8k 2 f4, 3, 2, 1, 0g

v2, 2, j, k

8k 2 f4, 3, 2, 1, 0g

v1, 2, j, k

8k 2 f4, 3, 2, 1, 0g

j¼0

x2 ¼

9 X j¼0

x2 ¼

9 X j¼0

w1, 1 , w2, 2 , w1, 2  0 δj, k , δ0j, k 2 f0, 1g

8j 2 f0, 1, . . . , 9g, k 2 f4, 3, 2, 1, 0g

v1, 1, j, k , v2, 2, j, k , v1, 2, j, k  0 8j 2 f0, 1, . . . , 9g, k 2 f4, 3, 2, 1, 0g The optimal solution to the above model is as follows: δ4, 4 ¼ δ7, 3 ¼ δ7, 2 ¼ δ5, 1 ¼ δ0, 0 ¼ 1




 x1 ¼ 4  104 þ 7  103 þ 7  102 þ 5  101 þ 0  100 ¼ 0:5774 v1, 1, 4, 4 ¼ v1, 1, 7, 3 ¼ v1, 1, 7, 2 ¼ v1, 1, 5, 1 ¼ v1, 1, 0, 0 ¼ 0:5774 0 0 0 0 δ0 0, 4 ¼ δ2, 3 ¼ δ3, 2 ¼ δ7, 1 ¼ δ1, 0 ¼ 1




 x2 ¼ 0  104 þ 2  103 þ 3  102 þ 7  101 þ 1  100 ¼ 1:7320

v1, 2, 4, 4 ¼ v1, 2, 7, 3 ¼ v1, 2, 7, 2 ¼ v1, 2, 5, 1 ¼ v1, 2, 0, 0 ¼ 1:7320 v2, 2, 0, 4 ¼ v2, 2, 2, 3 ¼ v2, 2, 3, 2 ¼ v2, 2, 7, 1 ¼ v2, 2, 1, 0 ¼ 1:7320 However, the optimal solution to the original model is: pffiffiffi 1 x1 ¼ pffiffiffi , x2 ¼ 3 and z ¼ 6: 3 The results indicate that the solution obtained by the approximation method is in good agreement with the optimal solution to the original model.

4.2 Variables Multiplication

135

4.2.10 Approximation by PLFs In this method, by using the conversion that will be described in Sect. 4.4.1, the term x1x2 is separated and replaced by the summation of appropriate single-variable functions. Then, the second-order expressions are approximated by PLFs to obtain a relatively good solution. Example 4.8 Multiplication of integer variables to continuous variables Assume that x1, x2 and y, are, respectively, free continuous, nonnegative continuous, and nonnegative integer variables with appropriate bounds as follows: l1  x1  u1 , 0  x2  u2 , y 2 f0, 1, 2, . . . , ng Provide linear constraints to ensure that the following propositions are guaranteed. x1  0 ) y ¼ 0

ð4:29Þ

x 1 < 0 ) ð x 1 þ x 2 y > 0 ^ x 1 þ x 2 ð y  1Þ < 0Þ

ð4:30Þ

Answer: We begin with the linearization of the multiplication term x2y. Therefore, we set k ¼ blog2(n)c, introduce binary variables δ0, δ1, . . ., δk, and rewrite the P variable y in terms of binary variables as y ¼ kj¼0 2j δj . In this way, the term x2y is P  k j replaced by the expression x2 j¼0 2 δj . Now, we replace the term x2δj by x3,j and add the following constraints: x3, j  x2 x3, j  u2 δj


 x3, j  x2  u2 1  δj

Let x4 be a nonnegative continuous variable such that: x4 ¼

k X

2j x3, j

j¼0

It is clear that the term x2y can be replaced by x4, and hence, the propositions (4.29) and (4.30) are equivalent to the following statements: x1  0 ) y ¼ 0 x 1 < 0 ) ð x 1 þ x 4 > 0 ^ x 1 þ x 4  x 2 < 0Þ Using the binary variable η, we rewrite the above statements as follows:

136

4 Linearization of Nonlinear Functions

x1  0 ) η ¼ 1

ð4:31Þ

η¼1)y¼0

ð4:32Þ

x1 < 0 ) η ¼ 0

ð4:33Þ

η ¼ 0 ) x1 þ x4 > 0 ^ x1 þ x4  x 2 < 0

ð4:34Þ

Proposition (4.31) is equivalent to: η ¼ 0 ) x1 < 0 Assuming that ε1 > 0 is a sufficiently small number, we have: η ¼ 0 ) x1  ε1 Which is equivalent to: x1  ε1 þ ðu1 þ ε1 Þη In the same way, the proposition (4.32) is equivalent to: n X

δj  nð1  ηÞ

j¼1

Proposition (4.33) can be stated as follows: η ¼ 1 ) x1  0 Which is equivalent to: x1  l 1 ð 1  η Þ Assuming that ε2 > 0 is a sufficiently small number, proposition (4.34) is equivalent to: η ¼ 0 ) x1 þ x4  ε2 ^ x1 þ x4  x2  ε2 Which can be guaranteed by the following constraints: x1 þ x4  ε2 þ ðl1  ε2 Þη x1 þ x4  x2  ε2 þ ðu1 þ u2 n þ ε2 Þη

4.2 Variables Multiplication

137

Example 4.9 Linearize the following constraint Let x1, x2 and δ, be nonnegative continuous, free continuous, and binary variables, respectively, such that lδ  x2  uδ. Linearize the constraint x1(aδ  x2) ¼ 0. Answer: Since lδ  x2  uδ, it is clear that if δ ¼ 0, then x2 ¼ 0. Therefore, if δ ¼ 0, the given constraint is always established. Thus, it is only sufficient to provide a constraint to ensure the following statement: δ ¼ 1 ) ð x1 ¼ 0 _ x 2 ¼ a Þ Which is equivalent to: δ1 þ δ2 ¼ δ x1  M ð1  δ1 Þ M ð1  δ2 Þ  x2  a  M ð1  δ2 Þ where, δ1, and δ2 are binary variables. Example 4.10 Assume that the coefficient cj, j0 is a constant and belongs to the set {1,1}. Rewrite the following problem as a BIP. max z ¼

n X n X

cj, j0 ηj ηj0

j¼1 j0 ¼1

s:t: ηj 2 f1, 1g

8j ¼ 1, . . . , n

Answer: Two methods can be proposed. Method 1: Associated with each variable ηj, we define two binary variables δj,1 and δj,2, and replace ηj by δj,1  δj,2. Thus, the problem is rewritten as follows: max z ¼

n X n X

cj, j0 ðδj, 1  δj, 2 Þðδj0 , 1  δj0 , 2 Þ

j¼1 j0 ¼1

s:t: δj, 1 þ δj, 2 ¼ 1 δj, 1 , δj, 2 2 f0, 1g

8j ¼ 1, . . . , n 8j ¼ 1, . . . , n

The objective function of the above model will be simplified as: max z ¼

n X n X

cj, j0 ðδj, 1 δj0 , 1  δj, 1 δj0 , 2  δj, 2 δj0 , 1 þ δj, 2 δj0 , 2 Þ

j¼1 j0 ¼1

It is necessary to linearize the multiplicative terms appropriately.

138

4 Linearization of Nonlinear Functions

Method 2: The variable ηj is replaced with 1  2δj where δj 2 {0,1}. Then, the terms of multiplication of two binary variables are appeared which can be linearized as described in Sect. 4.2.1.

4.3

Piecewise Linear Functions

One of the most important issues in the optimization field is the modeling of piecewise linear functions (PLF) that may appear in various applications. For example, in cases where the production cost per unit of a product increases or decreases by increasing the amount of production, we face with PLFs. Another application of these functions is the approximation of a nonlinear function with a PLF. The general formulation of a PLF is as follows: 8 m1 x þ d 1 > > > < m2 x þ d 2 f ð xÞ ¼ > ⋮ > > : mn x þ d n

a0  x < a1 a1  x < a2

ð4:35Þ

an1  x  an

where the domain of x is limited to the interval [a0, an], and this interval is broken down into n subintervals by the set of breakpoints {a0, a1, . . ., an}. For each i 2 {1, . . ., n } and x 2 [ai1, ai), we have f(x) ¼ mix + di, where mi is the slope of the line and di is its intercept (i.e., the point where the line crosses the vertical axis). Fig. 4.3 illustrates the function f(x) with five breakpoints. In this figure, the function f(x) is assumed to be continuous. Of course, in general, a PLF is not necessarily continuous. For example, Fig. 4.4 represents the diagram of the following PLF. Fig. 4.3 PLF with five breakpoints

4.3 Piecewise Linear Functions

139

Fig. 4.4 A convex piecewise linear function

8 > < 0:5x f ðxÞ ¼ 1:5x  1 > : 3x  4

0x1 1x2 2x4

A PLF may appear in the objective function or model constraints. The following example illustrates how a PLF may act in applications. Example 4.11 Discount on purchase Assume that f(x) is the amount that a company has to pay for x gallons of crude oil where x is a nonnegative continuous variable representing the amount of crude oil purchased (expressed in gallons). A maximum of 1500 gallons of crude oil can be purchased, in the following fashion. The first 500 gallons of purchased oil cost 25 (mu) per gallon, the next 500 gallons cost 20 (mu) per gallon; and the next 500 gallons cost 15 (mu) per gallon. Provide a formulation for f(x). Answer: It is obvious that if 0  x  500, then f(x) ¼ 25x. If 500  x  1000, the amount paid for the first 500 gallons equals 25  500 (mu), and the amount paid for the next (x  500) gallons equals 20  (x  500) (mu) Thus, we have: f ðxÞ ¼ 12500 þ 20 ðx  500Þ ¼ 20x þ 2500 Finally, if 1000  x  1500, then, the amount paid for the first 1000 gallons equals (25  500) + (20  500) ¼ 22500, and the amount of 15  (x  1000) (mu) is paid for the next (x  1000) gallons. Thus, in this case, we have: f ðxÞ ¼ 22500 þ 15 ðx  1000Þ ¼ 15x þ 7500 Therefore, f(x) is stated as follows: 8 > < 25x f ðxÞ ¼ 20x þ 2500 > : 15x þ 7500

0  x  500 500  x  1000 1000  x  1500

140

4 Linearization of Nonlinear Functions

The slope of the lines from one interval to the next is decreasing that make the function concave. A PLF is not a linear function. However, by using binary variables, it can be represented in linear form. In this regard, we consider the PLF, f(x), introduced in (4.35), as a base, and propose four methods for its linearization. In these methods, which are derived from Croxton et al. (2003), it is assumed that f(x) is continuous.

4.3.1

The First Convex Combination Method

Consider the interval defined by two successive breakpoints, such as [ai1, ai]. The line between two points (ai1, f(ai1)) and (ai, f(ai)) is as follows: f ð x Þ  f ð ai Þ ¼

f ðai Þ  f ðai1 Þ ð x  ai Þ ai  ai1

ð4:36Þ

For each x 2 [ai1, ai] we have: x  ai ¼ ai1 λi1 þ ai λi  ai ¼ ai1 λi1 þ ai ðλi  1Þ ¼ ðai  ai1 Þλi1 λi1 þ λi ¼ 1 λi1 , λi  0 By substitution in (4.36), we get: f ð x Þ  f ð ai Þ ¼

f ðai Þ  f ðai1 Þ ððai  ai1 Þλi1 Þ ai  ai1

The above equation is simplified as follows: f ðxÞ ¼ f ðai1 Þλi1 þ f ðai Þλi By now, we show that for x 2 [ai1, ai], there exist two coefficients λi1 and λi such that: x ¼ ai1 λi1 þ ai λi f ðxÞ ¼ f ðai1 Þλi1 þ f ðai Þλi λi1 þ λi ¼ 1 λi1 , λi  0 According to the above explanations, we describe the convex combination method for representing PLFs in a linear form. This method implies that if the PLF f(x) appears in constraints or the objective function of an optimization model,

4.3 Piecewise Linear Functions

141

we can associated the nonnegative continuous variable λi to each breakpoint ai, and add the following constraints to the model. x¼

n X

ai λ i

ð4:37Þ

i¼0 n X

λi ¼ 1

ð4:38Þ

i¼0

λi  0

8i ¼ 0, 1, . . . , n

ð4:39Þ

Also, it is necessary to replace f(x) by the following summation: f ðxÞ ¼

n X

f ðai Þλi

ð4:40Þ

i¼0

In this way, we obtain a linear model. Obviously, in the above relations, each of the points ðai0 , f ðai0 ÞÞ is obtained by setting λi0 ¼ 1 and λi ¼ 0 (for each i 6¼ i0). The other points on the graph of f(x) are obtained when at most two of the coefficients λi take nonzero values and the nonzero coefficients are adjacent. Therefore, in addition to constraints (4.37)–(4.39), it is necessary to add other constraints to ensure the adjacency condition. The adjacency condition implies that at most two variables of the set {λ0, λ1, . . ., λn} can take nonzero values, and these two variables must be adjacent. Two variables are said to be adjacent if they have consecutive indices such as λi and λi+1. To observe this condition, we can consider the variables λ0, λ1, . . ., λn as a special ordered set of type 2 (SOS2) and rewrite constraint (4.38) as follows: n X

λi ¼ 1

ðSOS2Þ

ð4:41Þ

i¼0

Most optimization software can implicitly observe the condition of SOS2. However, if we want to express this condition directly as a set of linear constraints, we have to define, for every i ¼ 1, . . ., n, the binary variable δi that is 1 if x 2 [ai1, ai]; otherwise 0. Then, by adding the following constraints, the adjacency condition is established. λ0  δ1 λ1  δ1 þ δ2 λ2  δ2 þ δ3 ⋮ λn1  δn1 þ δn λn  δn n X δi ¼ 1 i¼1

142

4 Linearization of Nonlinear Functions

Fig. 4.5 The need for adjacency for a concave function (min)

To illustrate the necessity of the adjacency condition, consider Fig. 4.5 representing the PLF f(x) with four breakpoints (n ¼ 3). The point ðx^, f ðx^ÞÞ ¼ ð2, 30Þ is on the graph. P Since x^ ¼ 2 isPbetween the breakpoints a1 ¼ 1 and a2 ¼ 3, in order to set the terms ni¼0 ai λi and ni¼0 f ðai Þλi , respectively, equal to x^ and f ðx^Þ, it is necessary that λ1 and λ2 take nonzero values and others become zero. To, this end, it is sufficient to set λ1 ¼ λ2 ¼ 1/2. Then, we have: 1 1 1 1 x^ ¼ a1 þ a2 ¼  1 þ  3 ¼ 2 2 2 2 2 1 1 1 1 f ðx^Þ ¼ f ða1 Þ þ f ða2 Þ ¼  20 þ  40 ¼ 30 2 2 2 2 Now suppose that in representing x^ ¼ 2, the coefficients λ0 and λ2 which are not adjacent, take nonzero values and the others become zero. In this case, λ0 ¼ 13 , λ2 ¼ 2 3 and we have: 1 2 1 2 x^ ¼ a0 þ a2 ¼  0 þ  3 ¼ 2 3 3 3 3 1 2 1 2 f ðx^Þ 6¼ f ða0 Þ þ f ða2 Þ ¼  0 þ  40 ¼ 26:67 3 3 3 3 P In this case, it can be concluded that ni¼0 f ðai Þλi will have an error relative to f ðx^Þ. In fact, if the adjacency condition is not observed, e.g. λ0 and λ2 (which are not adjacent) take nonzero values, then we have x^ ¼ a0 λ0 þ a2 λ2 and the point ðx^, f ða0 Þλ0P þ f ða2 Þλ2 Þ is placed on the dashed line specified in Fig. 4.5. Clearly, in this case, ni¼0 f ðai Þλi is not equal to f ðx^Þ. Therefore, the satisfaction of the adjacency condition is essential. This situation occurs when the minimum of a nonconvex function f(x) or the maximum of a nonconcave function f(x) is required.

4.3 Piecewise Linear Functions

143

In some cases, there is no need to write an adjacency condition explicitly, and the condition is self-contained. For example, if f(x) is a convex PLF with a positive coefficient in the objective function of a minimization problem, the adjacency condition is self-observing. To prove this issue, assume that in the optimal solution to such problem, the variable x takes the value x^ where x^ 2 ðai1 , ai Þ. We show that in this solution, only the coefficients λi  1 and λi take nonzero values. Suppose on the contrary that in the optimal solution of the problem, two non-adjacent coefficients λi1 and λi2 take nonzero values where i1 < i2 and i2 6¼ i1 + 1. Therefore x^ ¼ ai1 λi1 þ ai2 λi2 and λi1 þ λi2 ¼ 1; and with respect to the convexity of the function f(x), we have: f ðx^Þ ¼ f ðai1 λi1 þ ai2 λi2 Þ  f ðai1 Þλi1 þ f ðai2 Þλi2 ¼

n X

f ðai Þλi

i¼0

However, if the coefficients λi  1 and λi take nonzero values, we have: f ðx^Þ ¼ f ðai1 λi1 þ ai λi Þ ¼ f ðai1 Þλi1 þ f ðai Þλi ¼

n X

f ðai Þλi

i¼0

Pn PnTherefore, in the first case, f ðx^Þ  i¼0 Pfnðai Þλi and in the second case f ðx^Þ ¼ f ð a Þλ , and since the expression i i i¼0 i¼0 f ðai Þλi appears with the positive coefficient in the objective function of a minimization problem, its lower value is more desirable. Hence, in the optimal solution, the first state does not occur, and only the coefficients λi1 and λi are allowed to be nonzero. Figure 4.6 indicates this fact. Similarly, if f(x) is a concave PLF with a positive coefficient in the objective function of a maximization problem, no constraint is required to guarantee the adjacency condition. Example 4.12 Linearization with the first convex combination method A petrol producer company produces two types of gasoline (gas 1 and gas 2) by combining two types of oil (oil 1 and oil 2). Each gallon of gas 1 and gas 2 must contain at least 50 percent and 60 percent oil 1, respectively. Currently, 500 gallons Fig. 4.6 No need for adjacency for a convex function (min)

144

4 Linearization of Nonlinear Functions

of oil 1 and 1000 gallons of oil 2 are available. However, if needed, at most 1500 gallons of oil 1 can be purchased. The company can receive a discount on the purchase of oil 1 according to the style described in Example 4.11. Each gallon of gas 1 and gas 2 can be sold for 12 and 14 (mu), respectively. Provide a linear model to maximize the company’s profit. Answer: Let x be the amount of oil 1 purchased and wi, j denote a continuous variable indicating the amount of oil of type i used to produce gasoline of type j. Similar to Example 4.11, assume that f(x) represents the cost of purchasing x gallons of oil 1: 8 > < 25x f ðxÞ ¼ 20x þ 2500 > : 15x þ 7500

0  x  500 500  x  1000 1000  x  1500

Then, the problem is formulated as follows:

 max z ¼ 12 w1, 1 þ w2, 1 þ 14 w1, 2 þ w2, 2  f ðxÞ

ð4:42Þ

s:t: w1, 1 þ w1, 2  500 þ x

ð4:43Þ

w2, 1 þ w2, 2  1000
 w11  0:5 w1, 1 þ w2, 1
 w12  0:6 w1, 2 þ w2, 2

ð4:44Þ ð4:45Þ

x, w1, 1 , w2, 1 , w1, 2 , w2, 2  0

ð4:47Þ

ð4:46Þ

In the objective function, f(x) is a PLF with breakpoints a0 ¼ 0, a1 ¼ 500, a2 ¼ 1000, a3 ¼ 1500 which must be replaced by P3 i¼0 f ðai Þλi ¼ 0λ0 þ 12, 500λ1 þ 22, 500λ2 þ 30, 000λ3 . Also, the following constraints are added to the model: x ¼ 0λ0 þ 500λ1 þ 1000λ2 þ 1500λ3 λ0 þ λ1 þ λ2 þ λ3 ¼ 1 λ0  δ1 λ1  δ1 þ δ2 λ2  δ2 þ δ3 λ3  δ3 δ1 þ δ2 þ δ3 ¼ 1 λ0 , λ1 , λ2 , λ3  0 δ1 , δ2 , δ3 2 f0, 1g

4.3 Piecewise Linear Functions

145

Note that in this maximization problem, the function f(x) is concave and hence, f(x) which is appeared in the objective function of a maximization problem, is convex; therefore, we cannot omit the adjacency constraints.

4.3.2

The Second Convex Combination Method

This method is based on the fact that in the continuous PLF f(x), expressed in (4.35), if x 2 [ai1, ai], then x can be stated as a convex combination of ai1 and ai, and f(x) can be stated as a convex combination of f(ai1) and f(ai). The nonnegative continuous variables λi and λ0i are defined as the weights of the breakpoints ai1 and ai in the linear combination, respectively. We also define the binary variable δi that is 1 if x takes a value inside the ith interval; otherwise 0. Therefore, P
for linearization, it is enough to replace the function f(x) with the summation ni¼1 f ðai1 Þλi þ f ðai Þλ0i , and add the following constraints to the model: x¼

n
X

ai1 λi þ ai λ0i



i¼1

λi þ λ0i ¼ δi n X δi ¼ 1

8i ¼ 1, . . . , n

i¼1

λi , λ0i  0, δi 2 f0, 1g

8i ¼ 1, . . . , n

In this model, we can define the property SOS1 on the set {δi : i ¼ 1, . . ., n}, and relax the binary Prestriction of δi. In this case, the SOS1 property should be stated on the constraint ni¼1 δi ¼ 1 as follows: n X

δi ¼ 1

ðSOS1Þ

i¼1

Example 4.13 Linearization with the second convex combination method Consider the gasoline production problem addressed in Example 4.12. Apply the second convex combination method to linearize the PLF f(x). Answer: The function f(x) is defined on three intervals [0,500], [500, 1000], and [1000, 1500]. We denote the ith interval by [ai1, ai], where i ¼ 1,2,3. For linearization, the nonnegative continuous variables λi , λ0i and the binary variable δi are defined for i ¼ 1,2,3, and the problem is formulated as follows:

146

4 Linearization of Nonlinear Functions



 max z ¼ 12 w1, 1 þ w2, 1 þ 14 w1, 2 þ w2, 2 
 0λ1 þ 12500λ01 þ 12500λ2 þ 22500λ02 þ 22500λ3 þ 30000λ03 s:t:ð4:43Þ  ð4:47Þ x ¼ 0λ1 þ 500λ01 þ 500λ2 þ 1000λ02 þ 1000λ3 þ 1500λ03 λi þ λ0i ¼ δi 8i ¼ 1, 2, 3 δ1 þ δ2 þ δ3 ¼ 1 δi 2 f0, 1g, λi , λ0i  0

8i ¼ 1, 2, 3

As mentioned earlier, we can remove the zero-one restriction from the variable δi, impose the SOS1 characteristic on the constraint δ1 + δ2 + δ3 ¼ 1, and rewrite it as follows: δ1 þ δ2 þ δ3 ¼ 1

4.3.3

ðSOS1Þ

The Incremental Method

The incremental method is another approach for linearizing the continuous PLF f(x), introduced in (4.35). In this method the nonnegative continuous variable vi 2 [0, ai  ai1] as the load of the ith interval (i.e., [ai, ai1]) is defined and the following constraint is added to the model: x ¼ a0 þ

n X

vi

i¼1

The point that should be noted here is that, as long as the load of the ith interval is not reached to its highest value, the load of the (i + 1)th interval cannot take a positive value. In other words, vi+1 > 0 only if vi ¼ ai  ai1. To impose this condition, we define the binary variable δi that is 1 if vi >P0; otherwise 0. Thus, it is enough to replace the function f(x) with the expression ni¼1 mi vi þ f ða0 Þ, and add the following constraints to the model: x ¼ a0 þ

n X

vi

i¼1

ðai  ai1 Þδiþ1  vi  ðai  ai1 Þδi vn  ðan  an1 Þδn δi 2 f0, 1g, vi  0

8i ¼ 1, . . . , n  1

ð4:48Þ ð4:49Þ

8i ¼ 1, . . . , n

4.3 Piecewise Linear Functions

147

Remark 4.5 Consider a minimization problem with the objective function min f(x), and assume that f(x) is a convex PLF. In this case, the convexity of f(x) implies that the slope of the line-segments in consecutive intervals are ascending, i.e., mi < mi+1. Therefore, it is not necessary to introduce Pn binary variables and it is enough to rewrite the objective function as min z ¼ i¼1 mi vi þ f ða0 Þ and use the following constraints instead of the constraints (4.48) and (4.49): 0  vi  ðai  ai1 Þ

8i ¼ 1, . . . , n

Indeed, since the objective function is in minimization direction and the coefficient vi is smaller than that of vi+1, as long as vi does not reach its maximum value, vi+1 does not take a positive value. This also holds true for any maximization problem with the objective max f(x) assuming that f(x) is a concave PLF. Note that the above conclusions are valid due to the assumption that f(x) appears in the objective function and it is not necessarily correct for constraints. See Exercise 4.11. Example 4.14 Linearization with incremental method Consider the gasoline production problem addressed in Example 4.12 and use the incremental method to linearize the PLF f(x). Answer: We consider the interval [ai1, ai] for i ¼ 1, 2, 3 as described in Example 4.13. Additionally, for i ¼ 1,2,3, we define the continuous variable vi 2 [0, ai  ai1] and the binary variables δi as explained in the description of the incremental method. Then, the problem is formulated as follows: max z ¼ 12ðw11 þ w21 Þ þ 14ðw12 þ w22 Þ  ð25v1 þ 20v2 þ 15v3 Þ s:t:ð4:43Þ  ð4:47Þ x ¼ v1 þ v2 þ v3 500δ2  v1  500δ1 500δ3  v2  500δ2 v3  500δ3 δi 2 f0, 1g, vi  0

8i ¼ 1, 2, 3

Example 4.15 Advertisement problem A company plans to advertise its products in two magazines 1 and 2. In each magazine, the maximum number of times that an ad can be published is 15, and the number of people who see the ad for the first time is different depending on the magazine and the advert’s turn. The estimation of the number of people who see an ad for the first time is provided in Table 4.2. For example, if 13 ads are published in magazine 2, the total number of people who see these ads for the first time is calculated as follows:

148

4 Linearization of Nonlinear Functions

Table 4.2 Number of people see the ad for the first time Magazine 1 Ad’s turn 1–6 7–10 11–15

Number of people 2500 3000 10,000

Magazine 2 Ad’s turn 1–4 5–12 13–15

Number of people 8000 6000 2000

ð4  8000Þ þ ð8  6000Þ þ 2000 ¼ 82000 Assuming that the cost of each ad in each magazine is one (mu) and the company’s advertising budget is 20 (mu), provide a model to determine the number of ads to be published in each magazine, so that the number of people who see these ads for the first time is maximized. Answer: Suppose that y1 and y2 are the number of times the ad is published in magazines 1 and 2, respectively, and assume that f(y1) and g( y2) are the number of people who see these ads for the first time, accordingly. We have: 8 0  y1  6 > < 2500y1 f ð y1 Þ ¼ 3000y1  3000 6  y1  10 > : 10000y1  73000 10  y1  15 8 0  y2  4 > < 8000y2 gð y 2 Þ ¼ 4  y2  12 6000y2 þ 8000 > : 2000y2 þ 56000 12  y2  15 Therefore, the problem is formulated as follows: max f ðy1 Þ þ gðy2 Þ s:t: y1 þ y2  20 y1 , y2  0, Int: Function f(y1) is defined on three intervals. To linearize this function, we introduce the variable vi (as the load of the ith interval) and the binary variable δi, where i ¼ 1,2,3. Similarly, the function g(y2) is defined on three intervals and to linearize it, we introduce the variable wi (as the load of the ith interval); however, since this function is concave and has appeared with a positive coefficient in the objective function, it is not required to associate any binary variable with wi. Thus, the model above is linearized as follows:

4.3 Piecewise Linear Functions

149

max z ¼ 100  ð25v1 þ 30v2 þ 100v3 þ 80w1 þ 60w2 þ 20w3 Þ s:t: y1 ¼ v1 þ v2 þ v3 y2 ¼ w1 þ w2 þ w3 6δ2  v1  6δ1 ,

4δ3  v2  4δ2 ,

v3  5δ3

w1  4, w2  8, w3  3 y1 þ y2  20 y1 , y2 , v1 , v2 , v3 , w1 , w2 , w3  0, Int: δ1 , δ2 , δ3 2 f0, 1g

4.3.4

Multiple Choice Method

Multiple choice method is another approach for linearizing the continuous PLF f(x) introduced in (4.35). In this method, we define the nonnegative continuous variable vi which is equal to x, if x takes a value inside the ith interval; otherwise 0. Also, we define the binary variable δi that P is 1 if vi 6¼ 0; otherwise 0. Now, it is enough to replace f(x) with the expression ni¼1 ðmi vi þ di δi Þ and add the following constraints to the model (note that according to the relation (4.35), di represents the intercept of the ith line-segment): x¼

n X

vi

i¼1

ai1 δi  vi  ai δi n X δi ¼ 1

8i ¼ 1, . . . , n

i¼1

δi 2 f0, 1g

8i ¼ 1, . . . , n

Note that in this model, we must define the sign restriction of variable vi the same as variable x. It is worth mentioning Pnthat we can relax the binary restriction of variable δi, and replace the constraint i¼1 δi ¼ 1 by the following constraint: n X i¼1

δi ¼ 1

ðSOS1Þ

150

4 Linearization of Nonlinear Functions

Example 4.16 Linearization with multiple choice method Consider the gasoline production problem addressed in Example 4.12 and use the multiple choice method to linearize the PLF f(x). Answer: We consider the interval [ai1, ai] for i ¼ 1,2,3 as described in Example 4.13. Additionally, for i ¼ 1,2,3, we define the continuous variable vi and the binary variables δi as explained in the description of the multiple choice method. Then, the problem is formulated as follows:

 max z ¼ 12 w1, 1 þ w2, 1 þ 14 w1, 2 þ w2, 2 ð25v1 Þ  ð20v2 þ 2500δ2 Þ  ð15v3 þ 7500δ3 Þ s:t: ð4:43Þ  ð4:47Þ x ¼ v1 þ v2 þ v3 0  v1  500δ1 500δ2  v2  1000δ2 1000δ3  v3  1500δ3 δ1 þ δ2 þ δ3 ¼ 1 δi 2 f0, 1g, vi  0 8i ¼ 1, 2, 3 Also, we can relax the binary restriction of variable δi and replace the constraint P3 i¼1 δi ¼ 1 by the following constraint: 3 X

δi ¼ 1

ðSOS1Þ

i¼1

4.3.5

Discontinuous PLF

The topics discussed in the previous sections can also be generalized for cases where the function f(x) is not continuous at the breakpoints. In this regard, consider the following examples. Example 4.17 Step function Consider the following step (or staircase) function: 8 b1 > > > > > > < b2 f ð x Þ ¼ b3 > > >⋮ > > > : bn

a0  x < a1 a1  x < a2 a2  x < a3 an1  x < an

4.3 Piecewise Linear Functions

151

Fig. 4.7 Step function

The general diagram of such a function for n ¼ 3 is shown in Fig. 4.7. If f(x) appears in the objective function or constraints of a model, how can the model be rewritten linearly? Answer: For i ¼ 0, 1, . . ., n, we define the nonnegative continuous variable λi. Additionally, we define the binary variable δi that is 1 if x takes a P value in the interval [ai1, ai); otherwise 0. Then, we replace f(x) by the summation ni¼1 bi δi and add the following constraints to the model: x¼

n X

ai λ i

i¼0 n X i¼0 n X

λi ¼ 1 δi ¼ 1

i¼1

λ0  δ1 λi1  δi1 þ δi 8i ¼ 2, . . . , n λn  δn λi  0

8i ¼ 0, 1, . . . , n

δi 2 f0, 1g

8i ¼ 1, 2, . . . , n

Note that the above constraints do not necessarily work well in discontinuous points. For example, if the above constraints are applied to Fig. 4.7, then for x ¼ a3, the function f(x) can take both values b3 and b4, while according to its graph f(a3) 6¼ b3. Example 4.18 Discontinuity in a PLF Consider the PLF f(x) depicted in Fig. 4.8, and assume that 0 < a1 < a2 < a3. If f(x) appears in the objective function or constraints of a model, how can the model be rewritten linearly? Answer: For i ¼ 1,2,3, the nonnegative continuous variable vi 2 [0, ai  ai1] is defined as the load of the ith interval [ai1, ai] . Also, the binary variable δi is introduced that is 1 if vi > 0; otherwise 0. Thus, it is enough to replace the function f(x) with the following expression:

152

4 Linearization of Nonlinear Functions

Fig. 4.8 Discontinuous PLF

b1 þ

2 X

ðmi vi þ biþ1 δiþ1 Þ þ m3 v3

i¼1

moreover, the following constraints should be added to the model. x ¼ a0 þ

3 X

vi

i¼1

ðai  ai1 Þδiþ1  vi  ðai  ai1 Þδi v3  ða3  a2 Þδ3 δi 2 f0, 1g, vi  0

8i ¼ 1, 2

8i ¼ 1, 2, 3

Note that the above constraints do not necessarily work well in discontinuous points. For example for x ¼ a1, the function f(x) can take both values b1 + m1(a1  a0) and b1 + m1(a1  a0) + b2, while based on Fig. 4.8, f(x) 6¼ b1 + m1(a1  a0). Of course, this difficulty is resolved if f(x) appears with a positive coefficient in the objective function of a maximization problem. Example 4.19 Discontinuous nonconvex PLF The following PLF f(x) has appeared in an optimization model. Write the model linearly. 8 0 > > > < b þ m ðx  a Þ 0 1 1 f ðxÞ ¼ > b0 þ m1 ða2  a1 Þ þ m2 ðx  a2 Þ > > : b0 þ m1 ða2  a1 Þ þ m2 ða3  a2 Þ þ m3 ðx  a3 Þ

0  x < a1 a 1 < x  a2 a2 < x  a3 a3 < x  a4

where, 0 < a1 < a2 < a3 < a4, b0 > 0 and 0 < m2 < m1 < m3 (see Fig. 4.9).

4.4 Separable Functions

153

Fig. 4.9 PLF with one discontinuity point

Answer: We introduce the binary variables δ0,δ1,δ2,δ3 and the nonnegative continuous variables w,v1,v2,v3, and replace f(x) with the expression b0δ1 + m1v1 + m2v2 + m3v3. Finally, we add the following constraints to the model: x ¼ a1 w þ v 1 þ v 2 þ v 3 δ1  w  δ0 ða2  a1 Þδ2  v1  ða2  a1 Þδ1 ða3  a2 Þδ3  v2  ða3  a2 Þδ2 v3  ða4  a3 Þδ3 δ0 , δ1 , δ2 , δ3 2 f0, 1g w, v1 , v2 , v3  0 Note that the above constraints do not work at the discontinuous point x ¼ a1. Of course, no matter if f(x) appears with a positive coefficient in the objective function of the maximization problem.

4.4

Separable Functions

A function is called separable if it can be represented as a summation of some singlevariable functions, and the separable programming problem is referred to a model in which all constraints as well as the objective function are separable. The overall structure of these problems is as follows: max z ¼ f 1 ðx1 Þ þ f 2 ðx2 Þ þ . . . þ f n ðxn Þ s:t: gi, 1 ðx1 Þ þ gi, 2 ðx2 Þ þ . . . þ gi, n ðxn Þ  bi xj 2 j 8j ¼ 1, . . . , n

8i ¼ 1, . . . , m

154

4 Linearization of Nonlinear Functions

Fig. 4.10 Approximation of an NLP function by a PLF

Where j denotes the sign and bound restrictions on the continuous variable xj. For example, Model 4.1 represents a separable programming problem in which the objective function is the summation of two single-variable functions f 1 ðx1 Þ ¼ 30x1  2x21 and f 2 ðx2 Þ ¼ 35x2  3x22 . Also, the first constraint consists of two single-variable functions g1, 1 ðx1 Þ ¼ x21 and g1, 2 ðx2 Þ ¼ 2x22 . Model 4.1 A separable programming problem max z ¼ 30x1  2x21 þ 35x2  3x22 s. t. x21 þ 2x22  250 x1 + x2  20 x 1, x 2  0 It is easier and more cost-effective to solve these nonlinear problems in the way of LP or MIP problems. To this end, we
can each of the nonlinear  approximate
 functions fj(xj) and gi, j(xj) with PLFs f^j xj and g^i, j xj , respectively. In this way, we get the following model which is an approximation of the original one: max z ¼ f^1 ðx1 Þ þ f^2 ðx2 Þ þ . . . f^n ðxn Þ s:t: g^i, 1 ðx1 Þ þ g^i, 2 ðx2 Þ þ . . . þ g^i, n ðxn Þ  bi xj 2  j

8i ¼ 1, . . . , m

8j ¼ 1, . . . , n

By solving the above model, an approximate solution to the original problem is obtained. To clarify the discussion, consider the single-variable function f(x) as shown in Fig. 4.10. To approximate f(x) with a PLF, first, the breakpoints {a0,a1, . . ., an}, which are not necessarily equidistant, are selected from the domain of f(x). Then, by connecting the points (ai, f(ai)) and (ai+1, f(ai+1)) by line-segments, we get a PLF approximating f(x) within the desired range. Then, by using the approaches described in Sect. 4.3, a linear restatement of the approximated model can be formulated and solved. Obviously, a better approximation is achievable if the

4.4 Separable Functions Table 4.3 Breakpoints associated with variable x1

155 i 0 1 2 3 4

ai 0 5 10 15 20

f1(ai) 0 100 100 0 200

g1,1(ai) 0 25 100 225 400

number of breakpoints increases (i.e., smaller intervals are utilized). In Fig. 4.10, six breakpoints are considered. It is worth mentioning that in the approximation of a nonlinear function with a PLF, the domain of function has to be bounded. For example, in Fig. 4.10, the variable x is restricted to the interval [a0, a5]. However, sometimes it is not easy to determine proper upper and lower bounds for variables. Example 4.20 Approximation of a separable programming problem Approximate the Model 4.1 as a linear model. Answer: Based on the model constraints, the domain [0, 20] is considered for both variables. For x1, we consider five breakpoints as {0, 5, 10,15,20}, where the related information is provided in Table 4.3. Therefore, the functions f1(x1) and g1,1(x1) are replaced by the PLFs f^1 ðx1 Þ and g^1, 1 ðx1 Þ, respectively, as follows: 8 20x1 > > > < 100 f^1 ðx1 Þ ¼ > > 20x1 þ 300 > : 40x1 þ 600 8 5x1 > > > < 15x  50 1 g^1,1 ðx1 Þ ¼ > 25x 1  150 > > : 35x1  300

0  x1  5 5  x1  10 10  x1  15 15  x1  20 0  x1  5 5  x1  10 10  x1  15 15  x1  20

Similarly, for x2, we consider five breakpoints {0, 5, 10,15,20}. The related information is provided in Table 4.4. Therefore, the functions f2(x2) and g1,2(x2) are replaced by the PLFs f^2 ðx2 Þ and g^1,2 ðx2 Þ, respectively, as follows:

156

4 Linearization of Nonlinear Functions

Table 4.4 Breakpoints associated with variable x2

i 0 1 2 3 4

a0i 0 5 10 15 20


 f 2 a0i


 g1, 2 a0i

0 100 50 150 500

0 50 200 450 800

8 20x2 0  x2  5 > > > < 10x2 þ 150 5  x2  10 f^2 ðx2 Þ ¼ > 40x2 þ 450 10  x2  15 > > : 70x2 þ 900 15  x2  20 8 10x2 0  x2  5 > > > < 30x  100 5  x2  10 2 g^1, 2 ðx2 Þ ¼ > 50x2  300 10  x2  15 > > : 70x2  600 15  x2  20 Now, to linearize the above PLFs, we define the nonnegative continuous variables λi and λ0i associated with the breakpoints ai and a0i , respectively, for i ¼ 0, 1, . . ., 4. Then, we consider the following constraints: x1 ¼ 0λ0 þ 5λ1 þ 10λ2 þ 15λ3 þ 20λ4 x2 ¼ 0λ00 þ 5λ01 þ 10λ02 þ 15λ03 þ 20λ04 moreover, the PLFs are replaced by the following expressions: f^1 ðx1 Þ ¼ 0λ0 þ 100λ1 þ 100λ2 þ 0λ3  200λ4 f^2 ðx2 Þ ¼ 0λ00 þ 100λ01 þ 50λ02  150λ03  500λ04 g^1,1 ðx1 Þ ¼ 0λ0 þ 25λ1 þ 100λ2 þ 225λ3 þ 400λ4 g^1,2 ðx2 Þ ¼ 0λ00 þ 50λ01 þ 200λ02 þ 450λ03 þ 800λ04 Thus, we get the following linear approximation in which the SOS2 condition can be stated as a suitable set of linear constraints: max z ¼ 100λ1 þ 100λ2 þ 0λ3  200λ4 þ 100λ01 þ 50λ02  150λ03  500λ04 s:t: 25λ1 þ 100λ2 þ 225λ3 þ 400λ4 þ 50λ01 þ 200λ02 þ 450λ03 þ 800λ04  250

4.4 Separable Functions

157

5λ1 þ 10λ2 þ 15λ3 þ 20λ4 þ 5λ01 þ 10λ02 þ 15λ03 þ 20λ04  20 λ0 þ λ1 þ λ2 þ λ3 þ λ4 ¼ 1 ðSOS2Þ λ00 þ λ01 þ λ02 þ λ03 þ λ04 ¼ 1 ðSOS2Þ λi , λ0i  0

8i ¼ 0, 1, . . . , 4

Example 4.21 MCER and PLFs Consider the following problem and its optimal solution. Compare the outcome of approximate solutions obtained by two methods “MCER” and “PLF”. min z ¼ ð1  x1 Þ2 s:t:
 10 x2  x21 ¼ 0 3 5  xi  8i ¼ 1, 2 4 4

Optimal solution :
  x1 , x2 ¼ ð1, 1Þ, z ¼ 0

Answer: In the MCER, the multiplicative term x21 is replaced by the variable w, and the following model is solved: min z ¼ 1  2x1 þ w s:t: 10ðx2  wÞ ¼ 0 3 5  xi  8i ¼ 1, 2 4 4 25 5 w þ x1 16 2 9 3 þ x w 16 2 1 w

15 þ 2x1 16

w0

The optimal solution to the above model is as follows:


 x1 , x2 , w ¼ ð1, 0:9375, 0:9375Þ, z ¼ 0:0625

For PLFs, we consider three breakpoints related information is provided in Table 4.5.

3

5 4 , 1, 4



for the variable x1, where the

158

4 Linearization of Nonlinear Functions

Table 4.5 Breakpoints associated with variable x1

i 0

ai

f ðx1 Þ ¼ x21

3 4

9 16

1 2

1

1

5 4

25 16

Thus, f(x1) is approximated by the PLF f^ðx1 Þ: 8 7 3 > < x1  4 4 ^ f 1 ð x1 Þ ¼ > : 9 x1  5 4 4

3  x1  1 4 5 1  x1  4

Now, to linearize the PLF f^1 ðx1 Þ, we define the nonnegative continuous variables λi corresponding to the breakpoint ai, for each i ¼ 0,1,2, and consider the following constraint: 3 5 x1 ¼ λ 0 þ λ 1 þ λ 2 4 4 Thus, the function f^ðx1 Þ can be replaced by the following equation: 9 25 f^1 ðx1 Þ ¼ λ0 þ λ1 þ λ2 16 16 In this way, we get the following linear approximation of the original model:

3 5 9 25 min z ¼ 1  2 λ0 þ λ1 þ λ2 þ λ0 þ λ1 þ λ2 4 4 16 16 s:t:

9 25 10 x2  λ0  λ1  λ2 ¼ 0 16 16 3 5  x2  4 4 λ0 þ λ1 þ λ2 ¼ 1 ðSOS2Þ λi  0

8i ¼ 0, 1, 2

The optimal solution of this model is as follows: x2 ¼ 1, λ0 ¼ 0, λ1 ¼ 1, λ2 ¼ 0, z ¼ 0

4.4 Separable Functions

159

In this example, the approximation with PLFs, in contrast to the MCER, is more accurate and provides the optimal solution to the original problem.

4.4.1

Separable Models Transformations

A model containing the terms x1x2, x1 x2 and 2ðx1 þx2 Þ , is not separable. However, by using the transformations described below, it can be rewritten as a separable programming model. These transformation are derived from Bradley et al. (1977) and Rao (1978). 2

T1: Transformation for x1x2 Suppose that x1 and x2 are nonnegative continuous variables and the parameters u1 and u2 are their upper bounds, respectively. Also, suppose that the model contains a multiplicative expression x1x2. For example, if the sale price of each unit of a commodity and its level of production are both decision variables, such a multiplication phrase is created in the calculation of total outcome. We know that x1 x2 ¼



x1 þ x2 2

2





x1  x2 2

2

We introduce the continuous variables w1 and w2, and add the following constraints to the model: 1 w1 ¼ ðx1 þ x2 Þ 2 1 w2 ¼ ðx1  x2 Þ 2 where, 0  w1  12 ðu1 þ u2 Þ and  12 u2  w2  12 u1 . Then, we replace the term x1x2 with the term w21  w22 . The advantage of this replacement is that it establishes the separability. Afterwards, we can apply the method of separable programming to get a linear model as an approximation of the original one. If the continuous variables x1 and x2 are strictly positive, then there is another transformation for the expression x1x2. In this method, x1x2 is replaced by nonnegative continuous variable v. Then, by taking logarithm on both sides of the nonlinear expression v ¼ x1x2, we get the following constraint which is added to the model: log v ¼ log x1 þ log x2 Thus, separation is established. Now, we can approximate each of the terms log v, log x1 and log x2, separately with the PLFs and find an approximate solution to the original problem. This method cannot be used if x1 and x2 are allowed to take zero values (i.e., x1x2  0). In this situation, we can define two new variables w1 and w2 as follows:

160

4 Linearization of Nonlinear Functions

w1 ¼ x1 þ Δ1 w2 ¼ x2 þ Δ2 where Δ1 and Δ2 are arbitrary positive numbers and therefore, w1 and w2 are strictly positive. In this way, we have: x1 x2 ¼ ðw1  Δ1 Þðw2  Δ2 Þ ¼ w1 w2  Δ2 w1  Δ1 w2 þ Δ1 Δ2 Therefore, we replace x1x2 by w1w2  Δ2w1  Δ1w2 + Δ1Δ2 and substitute w1w2 with the new variable v. Also, we take logarithm on both sides of the nonlinear expression v ¼ w1w2, and add the following constraint to the model: log v ¼ log w1 þ log w2 Now, we have a separable problem. T2: Transformation for x1 x2 Assume that x1 > 0. We replace x1 x2 with the nonnegative continuous variable v. v ¼ x1 x2

ð4:50Þ

By taking logarithm on both sides of the above equation, we get: log v ¼ x2 log x1

ð4:51Þ

We introduce the free continuous variable w and let w ¼ log x1

ð4:52Þ

Concerning (4.51) and (4.52), we have: log v ¼ x2 w

ð4:53Þ

In summary, to establish separability, it is enough to replace the expression x1 x2 with nonnegative continuous variables v, add constraints (4.52) and (4.53) to the model, and then, apply T1 on the term x2w. T3: Transformation for 2ðx1 þx2 Þ 2 We replace 2ðx1 þx2 Þ with nonnegative continuous variable v. 2

v ¼ 2ðx1 þx2 Þ 2

ð4:54Þ

Then, we rewrite the above nonlinear equation as a separable constraint by taking logarithm, and add the following constraint to the model:

4.4 Separable Functions

161

Table 4.6 Separability transformation Term x1x2

Replacement x1 x2 ¼

x1x2

x1x2 ¼ v

x1 x2

x1 x2 ¼ v

2ðx1 þx2 Þ 2

w21



New constraints w22

2ðx1 þx2 Þ ¼ v 2

1 2 ðx1 1 2 ðx1

w1 ¼ þ x2 Þ w2 ¼  x2 Þ w1  0 w2 free log v ¼ log x1 + log x2 v0 log v ¼ x2w w ¼ log x1 v0 w free
 log v ¼ ð log 2Þ x1 þ x22


 log v ¼ ð log 2Þ x1 þ x22

Conditions x1  0 x2  0

x1 > 0 x2 > 0 x1 > 0



ð4:55Þ

The above transformations are summarized in Table 4.6. Example 4.22 Conversion to a separable function x x2

1 2 in separable form using appropriate Put the function f ðx1 , x2 , x3 Þ ¼ ð1þx 3Þ transformations. 1 , and add the following separable Answer: We use the substitution v ¼ 1þx 3 constraint to the model:

v¼ Therefore, the term

x1 x22 ð1þx3 Þ

1 1 þ x3

ð4:56Þ

is equivalent to x1 x22 v. Now, two cases may occur:

Case 1: x1 > 0, x22 > 0 and v > 0 In this case, we set w ¼ x1 x22 v where, w is a nonnegative continuous variable. Then, by taking logarithm on both sides of this equation, we get the following constraint which is added to the model. log w ¼ logx1 þ 2 log x2 þ log v

ð4:57Þ

Case 2: At least one of the conditions x1 > 0, x22 > 0 and v > 0 is not established In this case, we set w ¼ x1 x22 where, w is a nonnegative continuous variable. w ¼ x1 x22 Then, we apply T1 on multiplicative terms wv and x1 x22 .

ð4:58Þ

162

4 Linearization of Nonlinear Functions

Remark 4.6 Concerning the above transformations, although it might be theoretically claimed that almost every optimization problem of these types can be rewritten as a separable model, this is not necessarily an appropriate choice from a computational point of view. Because sometimes the number of variables and constraints added is so high that the resulting separable model is large and inapplicable.

4.5

Maximum and Minimum Functions

In optimization problems, the term maximum or minimum appears in the objective function to determine the optimal value of the objective function as a linear or nonlinear function of decision variables. Sometimes, these functions may appear in the model constraints as well. For example, to determine the value of a given variable, it may be necessary to know the value of other variables and the interaction between them. For example, assume that the variable xi represents the production level of the product i, the variable w is the maximum production level, and the variable y takes the index value of the product whose production level equals w. The relation between these variables can be expressed in the form of nonlinear constraints, which can be written as linear constraints by an appropriate reformulation. Assume that xj  0 ( j ¼ 1, . . ., n) and w  0 are nonnegative continuous variables whose relation is established through the following constraint: w ¼ max ðx1 , x2 , . . . , xn Þ

ð4:59Þ

The above nonlinear constraint is equivalent to the following proposition: ð w  x1 ^ . . . ^ w  xn Þ ^ ð w  x1 _ . . . _ w  xn Þ

ð4:60Þ

Now, we define indicator variable δj as follows: δj ¼ 1 ) xj  w

8j ¼ 1, . . . , n

In this way, the proposition (4.60) will be equivalent to the following linear constraints: w  xj

8j ¼ 1, . . . , n
 w  xj þ M 1  δj 8j ¼ 1, . . . , n n X δj ¼ 1 j¼1

4.5 Maximum and Minimum Functions

163

Now suppose that in addition to determining the value of w, we want to know that for which index j 2 {1, . . ., n} the maximum value is obtained. To this end, we introduce the integer variable y taking its value from the set {1, . . ., n}. Clearly, we have:

 y ¼ argmax xj j¼1, ..., n The following linear constraint can restate the above relation: y¼

n X

jδj

j¼1

Example 4.23 Maximum function in constraints Suppose that x1, x2 and x3 are continuous variables, and x1 and x2 are bounded. Rewrite the following constraint linearly. x3  max ðx1 , x2 Þ Answer: For x1 and x2, two cases may occur: x1  x2 or x2  x1. We define the binary variable δ as follows:  δ¼

1 0

x2 < x1 x1 < x2

Note that for x1 ¼ x2, there is no condition on δ and it can choose zero or one. Thus, we have the following statements: x1 < x2 ) δ ¼ 0 x2 < x1 ) δ ¼ 1 Therefore, we have: δ ¼ 0 ) x3  x2 δ ¼ 1 ) x3  x1 The aforementioned propositions can be stated as the following linear constraints: x2  x1 þ M ð 1  δ Þ

ð4:61Þ

x1  x2 þ Mδ

ð4:62Þ

164

4.6

4 Linearization of Nonlinear Functions

x3  x2 þ Mδ

ð4:63Þ

x3  x1 þ M ð 1  δ Þ

ð4:64Þ

Maximin and Minimax Functions

The maximin problem seeks to maximize the minimum value of a given set of expressions. A similar definition can be expressed for the minimax problem. For example, in a jobs scheduling problem, the aim is to minimize the time required to finish all jobs and hence, we deal with a minimax problem. Suppose that the problem’s objective function is as follows: min max ff i ðxÞg x i¼1, ..., m It is well known that if fi(x) is a convex function for i ¼ 1, . . ., m, then f ðxÞ ¼ max ff i ðxÞg is also a convex function. Figure 4.11 illustrates this issue, in which

i¼1, ..., m

f1(x) and f2(x) are convex functions. If we replace the inner maximization term with a new variable w, we have:

Fig. 4.11 Maximization of two convex functions

4.6 Maximin and Minimax Functions

165

min z ¼ w s:t: w ¼ max ff i ðxÞg i¼1, ..., m

ð4:65Þ

Now, similar to the previous section, (4.65) can be replaced by the following linear constraints: w  f i ð xÞ

8i ¼ 1, . . . , m

ð4:66Þ

w  f i ðxÞ þ M ð1  δi Þ 8i ¼ 1, . . . , m

ð4:67Þ

m X

δi ¼ 1

ð4:68Þ

i¼1

δi 2 f0, 1g 8i ¼ 1, . . . , m

ð4:69Þ

Note that since w is minimized in the objective function, it is practically not required to consider the constraints (4.67)–(4.69). Indeed, even without these constraints, at least one of the constraints w  fi(x) (i ¼ 1, . . ., m) will be binding. Similarly, the above method can be used for the following objective function, as well. max min ff i ðxÞg x i¼1, ..., m Which is equivalent to: max z ¼ w s:t:w  f i ðxÞ

8j ¼ 1, . . . , m

One of the applications of the minimax objective function is the curve fitting problem, which we will be discussed in the following example. Example 4.24 Curve fitting problem Curve fitting is the process of constructing a curve or mathematical function that has the best fit to a series of points. Suppose that n points with coordinates ðx^i , y^i Þ are given on the plane. The aim is to find a linear function y ¼ ax + b, which best fits to the points. For each point ðx^i , y^i Þ, the deviation between the approximated value and the actual one is equal to jy^i  ða^ xi þ bÞj. The goal is to find the coefficients a and b so that the maximum value of deviations is minimized. Figure 4.12 describes the problem graphically. Answer: The problem is formulated as follows:  min

 x i þ bÞ j g max fjy^i  ða^

i¼1, ..., n

s:t: a, b

free

166

4 Linearization of Nonlinear Functions

Fig. 4.12 Curve fitting

For linearization, we denote the inner maximization term with the nonnegative  continuous variables w, and define the nonnegative continuous variables vþ i and vi as the amount of deviation of approximated value from y^i ; then, the problem is rewritten as follows: min z ¼ w þ ^i a^ xi þ b þ v  i  vi ¼ y

w  v i

8i ¼ 1, . . . , n

8i ¼ 1, . . . , n

w  vþ 8i ¼ 1, . . . , n i  þ w, vi , vi  0 8i ¼ 1, . . . , n a, b

free

Example 4.25 Job scheduling problem This example examines a generalized version of the assignment problem in which,  ¼ f1, 2, . . . , Ig (with the index i) is the set of machines and  ¼ f1, 2, . . . , Jg (with the index j) is the set of jobs. Also, for each job j, j is a set of machines capable to perform job j, where j 6¼ ∅. In addition, ti, j is the processing time of job j on machine i, and b shows the total time that each machine is available. Two jobs are called incompatible, if they should be performed on distinct machines (i.e., their allocation to the same machine is not allowed). The binary parameter aj, j0 is equal to 1 if two jobs j and j0 are incompatible; otherwise 0. Finally, the set  ¼ fðj, j0 Þ : j < j0 , aj, j0 ¼ 1 g contain all pairs of incompatible jobs. The goal is to assign jobs to machines so that problem requirements are satisfied and the jobs are completed as soon as possible. Answer: For every j 2  and i 2 j , we define the binary variable δi,j that is 1 if job j is assigned to machine i; otherwise 0. Thus, the problem is formulated as follows:

4.7 Absolute Value Functions

167

X

min max i2

s:t: X

t i, j δi, j

j2ji2j

δi, j ¼ 1 8j 2 

i2j

X

t i, j δi, j  b

8i 2 

j2ji2j

8i 2 , 8ðj, j0 Þ 2 

δi, j þ δi, j0  1 δi, j 2 f0, 1g

8j 2 , 8i 2 j

The model objective is a minimax function minimizing the completion time of the last job. To linearize it, we define the nonnegative continuous variable x, replace the objective function with min x and add the following constraints to the model: X

x

t i, j δi, j

8i 2 

j2ji2j

x0

4.7

Absolute Value Functions

The absolute value function appears in optimization problems in different forms. It may appear in the objective function, constraints, or simultaneously in both. This function can be expressed in LP or MIP frameworks, depending on the problem’s conditions.

4.7.1

Absolute Value Function in Constraints

In this section, we examine different cases that the absolute value function appears in the problem’s constraints. P P Case 1: Constraints j j2 aj xj j¼ 0 or j j2 aj xj j 0. It is clear that each of these constraints is equivalent to the following constraint: X

aj x j ¼ 0

j2

P Case 2: Constraint j j2 aj xj j b in which b > 0. We know that jXj  b if and only if b  X  b. Therefore, to linearize the above constraint, it is enough to replace it with the following constraints:

168

4 Linearization of Nonlinear Functions

X

aj x j  b

j2

X

aj xj  b

j2

By using the same argument constraint j with the following constraints:

P

j2 aj xj j

þ

P

0 0 j20 aj xj

 b can be replaced

X X aj x j  b  a0j x0j j20

j2

X X aj xj  b þ a0j x0j j20

j2

P Case 3: Constraint j j2 aj xj j b We know that |X|  b if and only if X  b or X  b. Therefore, the above constraint is equivalent to the following proposition: X

aj x j  b _

X aj xj  b

j2

j2

Therefore, we can replace it with the following constraints: X

aj x j  b  M ð 1  δ Þ

j2

X

aj xj  b þ Mδ

j2

where δ is a binary variable. P P By using the same argument, constraint j j2 aj xj j þ j20 a0j x0j  b can be replaced by the following constraints: X X aj x j  b  a0j x0j  M ð1  δÞ j20

j2

X X aj xj  b þ a0j x0j þ Mδ j20

j2

P Case 4: Constraint j j2 aj xj j ¼ b where b > 0 This constraint is equivalent to the following proposition: X j2

aj xj ¼ b or  b

4.7 Absolute Value Functions

169

Therefore, we can replace it with the following constraint: X aj xj ¼ bδ  bð1  δÞ j2

where δ is a binary variable. Case 5: Model includes the following constraint: X  X    aj x j  þ a0j x0j ¼ b  

ð4:70Þ

j20

j2

One way to linearize the above equality constraint is to replace it with the following two inequality constraints, and then use the methods described in the previous cases.   X  X 0 0  aj x j  þ aj x j  b   j20

j2

  X  X 0 0  aj x j  þ aj x j  b   j20

j2

There are also two other methods for the linearization of constraint (4.70) which are more general than the previous one and can be used to linearize any expression containing absolute value function that appears in the constraints. One way is to introduce the nonnegative continuous variable w and replace the constraint (4.70) with the following constraints:   X    w ¼  aj x j   j2  X wþ a0j x0j ¼ b

ð4:71Þ

j20

Now, it is necessary to linearize the constraint (4.71). Given the definition of the absolute value function, we have jXj ¼ max (X, X). Therefore, the constraint (4.71) can be rewritten as follows: X X w ¼ max aj x j ,  aj x j j2

That is equivalent to:

j2

170

"

4 Linearization of Nonlinear Functions

X w aj x j j2

! ^

X w   aj x j

!#

" ^

w

j2

X j2

! aj x j

_

w

X

!# aj x j

j2

By using the indicator variable δ, the above proposition is stated as follows: w

X

aj x j

j2

w

X

aj x j

j2

w

X

aj xj þ M ð1  δÞ

j2

w

X

aj xj þ Mδ

j2

δ 2 f0, 1g Another approach for the linearization of constraint (4.70) is to define nonnegative continuous variables x0, x00, and add the following constraints to the model. X aj xj ¼ x0  x00 j2

x0 , x00  0 P Also, the expression j j2 aj xj j in the constraint (4.70) should be replaced by the sum of these variables, i.e., x0 + x00. In addition, it is necessary to add the to ensure that one of the variables x0 or x00 is zero constraint x0x00 ¼ 0 to the model P and the other is equal to j j2 aj xj j. The linearization of the nonlinear constraint x0x00 ¼ 0 is addressed inPExercise 3.8. Case 6: Constraint j2 jxj j  b In this case, one approach is to apply the method described in Case 5 to the expression |xj| for j 2 . However, there is also a simpler approach in which it is enough to introduce the nonnegative continuous variables vj and add the following constraints to the model: vj  xj  vj X vj  b j2

8j 2 

ð4:72Þ ð4:73Þ

4.7 Absolute Value Functions

171

P P Constraint (4.72) ensures that |xj|  vj and hence, we have j2 jxj j  j2 vj . The last inequality together with constraint (4.73), ensures the establishment of the P constraint j2 jxj j  b. Example 4.26 Linearize the following constraints. (a) j3x1  2x2 + x3 j  10 Answer: 3x1  2x2 þ x3  10 3x1  2x2 þ x3  10 (b) |x1  x2| ¼ 5 or 10 or 20 Answer: Considering δi as a binary variable for i ¼ 1, . . .6, the above constraint is equivalent to the following constraints: x1  x2 ¼ 5δ1  5δ2 þ 10δ3  10δ4 þ 20δ5  20δ6 6 X δi ¼ 1 i¼1

δi 2 f0, 1g

8i ¼ 1, . . . , 6

(c) |3x1  2x2| + |x3 + 4x4|  10 Answer: Considering w1 and w2 as nonnegative continuous variables, the above constraint is equivalent to: w1 ¼ j3x1  2x2 j

ð4:74Þ

w2 ¼ jx3 þ 4x4 j

ð4:75Þ

w1 þ w2  10 Therefore, we have w1 ¼ maxð3x1  2x2 , ð3x1  2x2 Þ Þ w2 ¼ maxðx3 þ 4x4 , ðx3 þ 4x4 Þ Þ Using binary variables δ1, δ2, above constraints, and equivalently, the constraints (4.74) and (4.75) can be replaced by:

172

4 Linearization of Nonlinear Functions

w1  3x1  2x2 w1  ð3x1  2x2 Þ w1  3x1  2x2 þ M ð1  δ1 Þ w1  ð3x1  2x2 Þ þ Mδ1 w2  x3 þ 4x4 w2  ðx3 þ 4x4 Þ w2  x3 þ 4x4 þ M ð1  δ2 Þ w2  ðx3 þ 4x4 Þ þ Mδ2 δ1 , δ2 2 f0, 1g

4.7.2

Absolute Value Function in Objective

The absolute value function is a convex function, and the summation of some absolute value functions is also aPconvex function. Figure 4.13 illustrates this fact. Suppose that the expression j j2 cj xj j has appeared in the objective function of an optimization problem and the sign of its coefficient is positive. In the linearization of such problem, there is a main difference between the two cases maximization and minimization, which will be discussed below.P The first way is to replace the expression j j2 cj xj j in the objective function by the nonnegative continuous variable w, and add the following constraint to the problem:   X  w ¼  cj xj  j2

The above constraint is equivalent to: X X w ¼ max cj xj ,  cj xj j2

j2

moreover, it can be substituted by the following constraints: w

X cj xj

ð4:76Þ

j2

w

X cj x j j2

ð4:77Þ

4.7 Absolute Value Functions

173

Fig. 4.13 Convexity of the absolute value function

w

X

cj xj þ Mð1  δÞ

ð4:78Þ

X cj xj þ Mδ

ð4:79Þ

j2

w

j2

δ 2 f0, 1g

ð4:80Þ

ConstraintsP (4.76) and (4.77) P guarantee that w is greater than or equal to the maximum of j2 cj xj and  j2 cj xj . While the constraints (4.78)–(4.80) ensure that strict case does not occur and w exactly takes the maximum value. In the minimization problems, since w is minimized in the objective function, the constraints P (4.76) and (4.77) P solely enforce w to be exactly the maximum of the two values j2 cj xj and  j2 cj xj , and it is not necessary to add constraints (4.78)– (4.80). However, in the maximization problems, it is necessary to write the constraints (4.78)–(4.80) to ensure P Pthat w does not take a value strictly greater than the maximum of j2 cj xj and  j2 cj xj . Remark 4.7 P The same approach is applicable when the expression j j2 cj xj j appears with a negative coefficient in the objective function. In this case, we have to keep constraints (4.78)–(4.80) when the problem is in minimization direction, while, we can remove them in the maximization P case. The second way to linearize j j2 cj xj j is to introduce the nonnegative continuous variables x0 and x00 and then add the following constraints to the model. X

cj xj ¼ x0  x00

j2

x0 , x00  0 P Also, the expression j j2 cj xj j in the objective function should be replaced by the sum of these variables (i.e., x0 + x00). In this case, if the problem is in minimization

174

4 Linearization of Nonlinear Functions

direction, then always one of the variables x0 and x00 takesP zero value in the optimal solution and consequently, the other one is equal to j j2 cj xj j. However, in a maximization problem, it is necessary to add the constraint x0x00 ¼ 0 to the model to ensure that one of the variables x0 or x00 will be zero. Remark 4.8 P The same approach is applicable when the expression j j2 cj xj j appears with a negative coefficient in the objective function. In this case, we have to keep the constraint x0x00 ¼ 0 in minimization problems. However, we can remove it in maximization case. Example 4.27 Absolute value function in the objective of a maximization problem Rewrite the following optimization problem as a linear model. max z ¼ j2x1  3x2 j 4x1 þ x2  4 2x1  x2  0:5 x1 , x2  0 Answer: max z ¼ w 4x1 þ x2  4 2x1  x2  0:5 w  2x1  3x2 w  ð2x1  3x2 Þ w  2x1  3x2 þ M ð1  δÞ w  ð2x1  3x2 Þ þ Mδ x1 , x2  0, δ 2 f0, 1g Example 4.28 Absolute value function in the objective of a minimization problem Rewrite the following optimization problem as a linear model. min z ¼ x1 þ 3 j x2 j þ4x3 6x1 þ 2x2 þ 5x3 ¼ 5 7x1 þ 3x2 þ 4x3  4 x1 , x3  0

4.7 Absolute Value Functions

175

Answer:
 min z ¼ x1 þ 3 x02 þ x002 þ 4x3
 6x1 þ 2 x02  x002 þ 5x3 ¼ 5
 7x1 þ 3 x02  x002 þ 4x3  4 x1 , x02 , x002 , x3  0 Note that in this case, there is no need to add the constraint x02 x002 ¼ 0. Example 4.29 Absolute value function in the objective of a minimization problem Rewrite the following optimization problem as a linear model. min z ¼

K  X  ck, 1 x1 þ ck, 2 x2 þ . . . þ ck, n xn  k¼1

s:t:

n X

ai, j xj ¼ bi

8i ¼ 1, . . . , m

j¼1

Answer: We introduce the nonnegative continuous variable yk, replace the term |ck,1x1 + ck,2x2 + . . . + ck,nxn| in the objective function with yk, and add the following constraints to the problem: yk  ck, 1 x1 þ ck, 2 x2 þ . . . þ ck, n xn 8k ¼ 1, . . . , K
 8k ¼ 1, . . . , K yk   ck , 1 x1 þ ck , 2 x2 þ . . . þ ck , n xn Therefore, we get: min z ¼

K X

yk

k¼1

s:t:

n X

ai , j x j ¼ bi

8i ¼ 1, . . . , m

j¼1

yk  ck, 1 x1 þ ck, 2 x2 þ . . . þ ck, n xn 8k ¼ 1, . . . , K
 yk   ck , 1 x1 þ ck , 2 x2 þ . . . þ ck , n xn 8k ¼ 1, . . . , K yk  0 8k ¼ 1, . . . , K

176

4.7.3

4 Linearization of Nonlinear Functions

Vectors Comparison

In some cases, it is necessary to recognize that among two vectors of the same dimension which one dominates the other. For example, each component of the vectors can indicate the performance of an objective function in a multi-objective context. Definition 4.1 Domination Let x and y be two vectors with nonnegative components. We say that the vector x dominates the vector y (or the vector y is dominated by the vector x), if any component of the vector x is greater than or equal to its corresponding component in the vector y, and at least one of the x components has a strictly larger value. In other words, x dominates y if:




8j xj  yj ^

n X

xj >

n X

j¼1

! yj

j¼1

Suppose that we need to know whether or not the vector x dominates the vector y in the optimal solution. To this end, we define the binary variable δ that is 1 if x dominates y; otherwise 0. Thus, in the optimal solution, with respect to the value of δ, we can find out the status of the domination. Therefore, it is necessary to provide a constraint to observe the following proposition: δ¼1 ,


 8j xj  yj ^

n X

xj >

j¼1

n X

!! yj

j¼1

First, we restate the following proposition (implication) as linear constraints: δ¼1 )






8j xj  yj ^

n X j¼1

xj >

n X

!! yj

j¼1

That is equivalent to the following constraints where ε > 0 is a sufficiently small number, and M > 0 is a sufficiently large number. xj  yj  M ð 1  δ Þ n n X X xj  yj þ ε  M ð 1  δ Þ j¼1

j¼1

Now, we need to express the following proposition (reverse implication) as linear constraints:

4.8 Fractional Functions




177



8j xj  yj ^

n X j¼1

xj >

n X

!! yj

)δ¼1

j¼1

That is equivalent to: δ¼0)

ð x1 < y 1 Þ _ ð x 2 < y 2 Þ _ . . . _ ð x n < yn Þ _

n X j¼1

xj 

n X

!! yj

j¼1

Considering ηj and β as binary variables, the above proposition is stated as follows:
 8j ¼ 1, . . . , n xj  yj  ε þ M 1  η j n n X X xj  yj þ M ð 1  β Þ j¼1 n X

j¼1

ηj þ β  1  δ

j¼1

4.8

Fractional Functions

Fractional functions may appear in the constraints or the objective function of optimization problems. The problems containing fractional functions in their objective function or constraints are known as fractional programming problems and appear in a variety of applications. As an example, we can point to the problem of blending materials to produce new products, in which the quality of new products can be expressed in the form of fractional constraints. In producing petroleum products from the crude oil, the octane percentage in gasoline production is controlled through a fractional constraint. As another example, consider the data envelopment analysis in which the goal is to maximize the efficiency of the system which is defined as the ratio of the weighted summation of outputs of the system to the weighted summation of its inputs. Maximizing the ratio of the production level to the number of employees in a manufacturing company, maximizing the ratio of the number of nurses to the number of patients in a hospital, and minimizing the ratio of the payments to the received funds are other examples of fractional objective functions. In this section, some points on the linearization of the fractional functions are discussed.

178

4.8.1

4 Linearization of Nonlinear Functions

Fractional Functions in Constraints

Suppose that the following constraint has appeared in an optimization problem: Pn aj x j þ a0 Pnj¼1 r j¼1 d j xj þ d 0 Assume that the denominator has a constant sign (positive or negative) for all feasible solutions. Then, this constraint can be rewritten linearly. For example, if Pn d x j¼1 j j þ d 0 > 0 for any feasible solution, then the fractional constraint can be stated as follows: n X

aj x j þ a0  r

j¼1

n X

! d j xj þ d 0

j¼1

Example 4.30 Fractional constraint in the blending problem A company produces gasoline and heating oil by blending two types of crude oil, namely oil 1 and oil 2, and at least 100 barrels must be produced per product. The octane ratings of oil 1 and oil 2 are equal to 10 and 5, respectively. Let xi and wi be the amount of oil i used to produce gasoline and heating oil, respectively. Provide linear constraints ensuring that the produced gasoline and heating oil must have an average octane rating of at least 8 and 6, respectively. Answer: The following constraint can control the octane rating of gasoline: 10x1 þ 5x2 8 x1 þ x2 Since x1 + x2 > 0, it is equivalent to 2x1  3x2  0. Similarly, the octane rating of heating oil can be controlled by: 10w1 þ 5w2 6 w1 þ w2 which is equivalent to 4w1  w2  0.

4.8.2

Fractional Objective Function

Consider the following problem in which the objective function is fractional, and the constraints are linear.

4.8 Fractional Functions

179

Pn j¼1 cj xj þ c0 min z ¼ Pn j¼1 d j xj þ d 0 s:t: n X ai, j xj  bi 8i ¼ 1, . . . , m j¼1

xj  0

8j ¼ 1, . . . , n

If the denominator has a fixed sign (always positive or always negative) for all feasible solutions, then the model can be rewritten linearly. Suppose that the denominator is always positive for all feasible solutions. We do the following steps to linearize the model: Step 1: We apply the following substitution: 1 j¼1 d j xj þ d 0

w ¼ Pn

Since the denominator is always positive, we have w > 0. Step 2: We set vj ¼ wxj. Obviously, vj  0. Step 3: Considering the variables substitution introduced in Step 2, the objective function is rewritten as follows:

min z ¼

n X

cj vj þ c0 w

j¼1

Also, by multiplying w in

Pn

j¼1 ai, j xj n X

 bi , we get:

ai, j vj  bi w

j¼1

Moreover, concerning the substitution, introduced in Step 1, we have: n X

d j vj þ d 0 w ¼ 1

j¼1

Therefore, the following linear model is obtained:

180

4 Linearization of Nonlinear Functions

min z ¼

n X

cj vj þ c0 w

j¼1

s:t: n X

ai , j v j  bi w

8i ¼ 1, . . . , m

j¼1 n X

d j vj þ d 0 w ¼ 1

j¼1

w, vj  0

8j ¼ 1, . . . , n

In the case that the sign of the denominator is always negative for all feasible solutions, it is enough to multiply the denominator by (1) and then, change the objective function from minimization to maximization. Then, the linearization steps can be followed in the same way. Below, some examples with fractional objective functions are introduced. Example 4.31 Optimizing the productivity index A company is producing m different types of products (i ¼ 1, . . ., m) by utilizing n different machines ( j ¼ 1, . . ., n). Currently, existing machines are old and the company has to replace them by new ones. The company can purchase at most aj machines of type j that each one is able to work Tj hours per day. At least li units of product i should be produced per day. Each unit of product i produced should be processed for ti,j hours on one of the machines of type j. Note that the processing of product i on a given machine of type j may be interrupted due to the time capacity Tj, and it is resumed on the another machine of that type. The company’s utilization index is defined as the ratio of the effective time in which the machines are being actually used and the maximum time for which the machines could be used. Assume that the variable xi represents the production level of product i and the variable yj is the number of the purchased machine of type of j. Provide a model to maximize the company’s utilization index. Answer: The problem is formulated as the following model in which the objective function is fractional. Pn Pm t i, j xi j¼1 Pn i¼1 max z ¼ j¼1 T j yj s:t: xi  l i

" i ¼ 1, . . . , m

0  y j  aj m X

" j ¼ 1, . . . , n

t i , j xi  T j yj

" j ¼ 1, . . . , n

i¼1

xi , yj  0, Int:

8i ¼ 1, . . . , m, 8j ¼ 1, . . . , n

4.8 Fractional Functions

181

The model’s objective function can be rewritten linearly by the method described earlier. Example 4.32 Sensitivity analysis and fractional function Sensitivity analysis is a process that examines the effect of changing the problem’s parameters (i.e., the objective function’s coefficients, RHS values, constraints’ coefficients) on the optimal solution. In this example, we use a fractional objective function to perform a specific case of sensitivity analysis. To this end, consider a transportation problem in which i ¼ 1, . . ., m represents the depots and j ¼ 1, . . ., n denotes the customers, ai is the capacity of the depot i, bj is the demand of customer j, and ci,j shows the cost of transferring one unit of product from depot i to customer j. Assume that the decision variable xi, j is the amount of products sent from the depot i to the customer j. The following optimization model minimizes the transmission costs. min z ¼

m X n X

ci, j xi, j

i¼1 j¼1

s:t: n X

xi, j  ai

8i ¼ 1, . . . , m

ð4:81Þ

xi, j  bj

8j ¼ 1, . . . , n

ð4:82Þ

j¼1 m X i¼1

xi, j  0 8i ¼ 1, . . . , m,

8j ¼ 1, . . . , n

Suppose that xi, j and z are the optimal solution and the optimal objective value, respectivly, and let i0 and j0 be a given depot and a given customer, correspondingly. We want to determine how big can the coefficient ci0 , j0 be (the interval of its variation) so that z is reduced by at least αz, where α is a given number in the interval (0, 1). In this regard, it is enough to solve the following optimization model: max z ¼ ci0 , j0 s:t:ð4:81Þ, ð4:82Þ X
 ci, j xi, j þ ci0 , j0 xi0 , j0  ð1  αÞz ði, jÞ:ði, jÞ6¼ði0 , j0 Þ ci0 , j0 , xi, j  0

8i ¼ 1, . . . , m,

8j ¼ 1, . . . , n

182

4 Linearization of Nonlinear Functions

This is a model with a nonlinear term ci0 , j0 xi0 , j0 . If the term ci0 , j0 xi0 , j0 is replaced by the new variable wi0 , j0 (i.e., wi0 , j0 ¼ ci0 , j0 xi0 , j0 ), then the model changes to a fractional programming model as follows: max z ¼

wi0 , j0 xi0 , j0

s:t:ð4:81Þ, ð4:82Þ X
 ci, j xi, j þ wi0 , j0  ð1  αÞz ði, jÞ:ði, jÞ6¼ði0 , j0 Þ wi0 , j0 , xi, j  0 8i ¼ 1, . . . , m,

8j ¼ 1, . . . , n

Now, it is easy to follow the linearization. Example 4.33 Maximum function and fractional function In this problem, a combination of the fractional function and the maximum value function is used. Let aj be a positive constant, and lj and uj be nonnegative constants. Provide a linear restatement of the following model: w max z ¼ v w ¼ x1 þ x2 þ þ xn

x1 x2 xn v ¼ max , , , a1 a2 an lj  xj  uj 8j ¼ 1, . . . , n Answer: To linearize the model, we introduce the continuous variable x0j such that x ¼ ajj . Therefore, we have xj ¼ aj x0j , and hence, the model is rewritten as the following fractional optimization problem, and can be linearized in the way described earlier. x0j

Pn max z ¼ v  x0j

0 j¼1 aj xj

v 8j ¼ 1, . . . , n

lj  aj x0j  uj Note that in this problem, v ¼ max

8j ¼ 1, . . . , n   x0j , . . . , x0j has been replaced by the

constraint v  x0j . Check why this constraint alone is enough to ensure that v precisely represents the maximum value of x0j s?

4.8 Fractional Functions

4.8.3

183

Minimax Fractional Objective Function

Consider the following problem which is a generalization of the problem with fractional objective function: min

max

k¼1, ..., K

z¼

ck , 1 x1 þ ck , 2 x2 þ þ c k , n xn þ c0 d k , 1 x1 þ d k , 2 x2 þ þ d k , n xn þ d 0

s:t: n X

ai , j x j  bi

8i ¼ 1, . . . , m

j¼1

xj  0

8j ¼ 1, . . . , n

Suppose that the denominator of the fractional term is positive for all feasible c x þc x þ...þc x þc solutions. If we set w ¼ max dkk,,11x11þdkk,, 22 x22 þ...þdkk,,nn xnn þd00 , then the model is rewritten k¼1, ..., K as follows: min w s:t: n X

ai, j xj  bi

8i ¼ 1, . . . , m

j¼1

xj  0 8j ¼ 1, . . . , n ! n X ck , j xj þ c0  w j¼1

w

n X

! d k , j xj

! þ d0 8k ¼ 1, . . . , K

j¼1

free

In general, it is not possible to rewrite this formulation linearly, but it can be solved with appropriate accuracy by solving a sequence of linear problems. Step 1: Assume that ε  0 is the desirable accuracy, and l and u are upper and lower bounds on the optimal objective function value (i.e., l  w  u). Initialize l and u to appropriate values and go to the step 2. Þ Step 2: Put w^ ¼ ðlþu 2 . Step 3: Solve the following linear problem:

184

4 Linearization of Nonlinear Functions

min z ¼ s s:t: n X ai , j x j  bi

8i ¼ 1, . . . , m

j¼1 n X

^ ck , j xj þ c0  s  w

j¼1

w

n X

! dk, j xj þ d 0 8k ¼ 1, . . . , K

j¼1

free

xj  0

8j ¼ 1, . . . , n

Step 4: If s > 0, set l ¼ w, and if s ¼ 0, then set u ¼ w. Step 5: If u  l  ε stop; otherwise, go to step 2. Example 4.34 A simplified problem of economic growth A company uses n processes to produce m products. If the process j is used for a full day, it consumes ai,j  0 units of product i and produces bi,j  0 units of product i. Suppose that for each product i there is at least one process j, with ai,j > 0 and for each process j there is at least one product i with bi,j > 0. Define the nonnegative continuous vector X, where the component xj is a nonnegative continuous variable indicating the duration of the utilization of process j (expressed as a fraction of a full day). For each vector X, consider wX as follows: (Pn wX ¼ min i¼1, ..., m

bi , j x j Pj¼1 n j¼1 ai, j xj

)

The value of wX implies that if we run the process according to the vector X, the output of the ith product will be at least wX times its input. The goal is to set X so that the value of wX is maximized. This problem is formulated as follows: (P n max s:t:

min

i¼1, ..., m n X

bi, j xj Pj¼1 n j¼1 ai, j xj

)

xj ¼ 1

j¼1

xj  0

4.8.4

8j ¼ 1, . . . , n

Problems Involving Mean Value Function

Consider a model containing the mean value function in its constraints as follows:

4.9 Knapsack Problems

185

Pn j¼1 cj δj x ¼ Pn j¼1 δj

ð4:83Þ

Where,P δj is a binary variable and cj is a constant, and for each feasible solution, n we have j¼1 δj 6¼ 0. In this case, to linearize the problem, we introduce the nonnegative continuous variables w, and set w ¼ P1n . Additionally, we set vj ¼ wδj. δj P j¼1 Thus, the constraint (4.83) is rewritten as x ¼ nj¼1 cj vj and the relation w ¼ P1n is δ j¼1 j P rewritten as nj¼1 vj ¼ 1. But vj is the product of a continuous variable and a binary variable. Therefore, the following statement is needed:



 δ j ¼ 1 ) vj ¼ w ^ δ j ¼ 0 ) vj ¼ 0

8j ¼ 1, . . . , n

According to the above explanations, the nonlinear constraint (4.83) can be replaced by the following linear constraints in which l ¼ 1/n, u ¼ 1. x¼

n X

cj vj

j¼1 n X

vj ¼ 1

j¼1

vj  uδj

8j ¼ 1, . . . , n

vj  w 8j ¼ 1, . . . , n
 vj  w  u 1  δj 8j ¼ 1, . . . , n lwu vj  0 8j ¼ 1, . . . , n

4.9

Knapsack Problems

A knapsack problem is a kind of programming problems having only one constraint. These problems have various usages in theory as well as in practice. Theoretically, their simple structure provides the opportunity to develop effective solution methods. Also, they appear as a sub-problem in the solution methods of many combinational optimization problems. Also, this structure is seen in several practical problems, such as timber cutting, product packing, budget control, investment, and so on. In this family, the problems complexity varies from pseudo-polynomial to NP-hardness (Kellerer et al. 2004). Here, some variants of knapsack problems, derived from Pisinger (1995), are introduced.

186

4 Linearization of Nonlinear Functions

A set of n items, numbered from 1 to n, is given. With each item j ¼ 1, . . ., n, the weight aj > 0, the value cj > 0 and the inventory level uj > 0 are associated. Assume that b is the capacity of the knapsack. The goal is to determine the number of each item to be included in the knapsack so that the total weight of included items is less than or equal to the knapsack capacity and the total value is as large as possible. The simplest form of the problem is the case where the nonnegative continuous variable xj represents the share of jth item in the knapsack. These assumptions lead to the following mathematical model: max z ¼

n X

cj xj

j¼1

s:t: n X

aj x j  b

j¼1

0  xj  u j

8j ¼ 1, . . . , n c

The best items have the largest values of ajj and the worst items have the smallest c values of ajj . Thus, without loss of generality, we assume that the items are in c non-increasing order of the ratio ajj ; i.e., ac11  ac22  . . .  acnn (if this is not the case, we can rearrange the variables based on this ratio and re-index them, accordingly). In this case, to obtain the problem optimal solution, first, the best item is put in the knapsack as much as possible. Then, the second best item is put as much as possible. This fashion is continued until the knapsack is filled. In practice, there are problems in which divisibility assumption is not valid and, merely, the selection or rejection of the items is considered. Suppose that δj is a binary variable that is 1 if the item j is put in the knapsack; otherwise 0. Therefore, the zero-one knapsack problem addresses the selection of a subset of items so that the profitability is maximized while observing the knapsack capacity. This problem is formulated as follows: max z ¼

n X

cj δ j

j¼1

s:t: n X

aj δ j  b

j¼1

δj 2 f0, 1g

8j ¼ 1, . . . , n

In this model, assume that the items are arranged in increasing order of their weight (i.e., a1  a2  . . . an) and suppose that the coefficients aj are nonnegative. Define the parameter m as follows:

4.9 Knapsack Problems

187

 X  k kj aj > b

m ¼ min

j¼1

P If m is undefined, i.e., kj¼1 aj  b for k, it can be concluded that the knapsack capacity is sufficiently large to accommodate all items; and hence, in the optimal solution, we have δj ¼ 1 for every j . Otherwise, based on the value of m, we can provide a valid cut, indicating that the knapsack cannot contain more than m  1 items. This cut is useful in improving the performance of the B&B solution method. n X

δj  m  1

j¼1

If a finite number of each item is available, we get the bounded knapsack problem which is formulated as follows: max z ¼

n X

cj yj

j¼1

s:t: n X

aj y j  b

j¼1

 yj 2 0, 1, . . . , uj

8j ¼ 1, . . . , n

Where, uj represents the availability of item j, and yj is an integer variable indicating the number of items of type j which are put in the knapsack. The quadratic zero-one knapsack problem is another type of the knapsack problem in which the variables are binary, and the objective function is quadratic. In this problem ci,j represents the value of the simultaneous placement of items i and j in the knapsack. max z ¼

n X n X j¼1

s:t: n X

ci, j δi δj

i¼j

aj δ j  b

j¼1

δj 2 f0, 1g

8j ¼ 1, ::, n

The model objective function is nonlinear. The nonlinear term δ2j can be easily replaced by δj. There are two methods to linearize the expressions δiδj. The first method is based on the approach outlined in Sect. 4.2.1, which a new introduces

n binary variable for each multiplicative expression and leads to new variables 2

188

4 Linearization of Nonlinear Functions

(n is the number of items). But the second method, which only needs n new variables, is based on the fact that the objective function can be expressed P P P as nj¼1 cj, j δj þ nj¼1 δj ni¼jþ1 ci, j δi . This method uses the new variable xj as follows: xj ¼ δ j

!

n X

ci, j δi

ð4:84Þ

i¼jþ1

objective function of the problem is rewritten as PnTherefore Pthe n c δ þ x . j¼1 j, j j j¼1 j Now, it is necessary to express the relation (4.84) in the form of linear constraints. P To this end, assume that Lj and Uj are lower and upper bounds on the expression ni¼jþ1 ci, j δi , we need constraints guaranteeing the following proposition:
 δ j ¼ 0 ) xj ¼ 0 ^

δ j ¼ 1 ) xj ¼

n X

! ci, j δi

i¼jþ1

Finally, the problem is linearized as follows: max z ¼

n X

cj, j δj þ

j¼1

s:t: xj  Lj δj

n X

xj

j¼1

8j ¼ 1, . . . , n

xj  U j δj 8j ¼ 1, . . . , n n X
 ci, j δi  U j 1  δj xj  xj 

i¼jþ1 n X

8j ¼ 1, . . . , n


 8j ¼ 1, . . . , n ci, j δi  Lj 1  δj

i¼jþ1 n X

aj δj  b

j¼1

δj 2 f0, 1g, xj

free

8j ¼ 1, . . . , n

Multiple knapsack problem is a generalization of the knapsack problem from a single knapsack to m knapsacks with different capacities. In this problem, the number of items of type j put in knapsack i should be determined so that the total weight in each knapsack is less than its capacity, and the total value of knapsacks is maximized. This problem is formulated as follows:

4.9 Knapsack Problems

189

max z ¼

m X n X

cj yi , j

i¼1 j¼1

s:t:

n X

aj y i , j  bi

8i ¼ 1, . . . , m

j¼1 m X

y i , j  uj

8j ¼ 1, . . . , n

yi, j  0, Int:

8i ¼ 1, . . . , m,

i¼1

8j ¼ 1, . . . , n

where, bi is the capacity of knapsack i, and yi,j is a nonnegative integer variable indicating the number of the items of type j put in the knapsack i. Clearly, this problem belongs to the category of MIP problems. Another generalization of the knapsack problem is the multiple-choice knapsack problem where the set of items is partitioned into m mutually disjoint classes ℕ1, . . ., ℕm. Each item j 2 ℕi has a profit cj and a weight aj, and the aim is to choose exactly one item from each class so that the total profit is maximized and the knapsack capacity is observed. This problem is formulated as follows: max z ¼

m X X

cj δ j

i¼1 j2ℕi

s:t: m X X aj δ j  b i¼1 j2ℕi

X δj ¼ 1

8i ¼ 1, . . . , m

j2ℕi

δj 2 f0, 1g

8j ¼ 1, . . . , n

where, δj is a binary variable that is 1 if the item j is selected; otherwise 0. It is worth mentioning that any IP problem with integer coefficients can be transformed into a knapsack problem by using constraint aggregation method (Ram et al. 1988). To clarify the discussion, consider the following model with two equality constraints:

190

4 Linearization of Nonlinear Functions

max z ¼

n X

cj yj

j¼1

s:t: n X j¼1 n X

a1, j yj ¼ b1 a2, j yj ¼ b2

j¼1

yj  uj 8j ¼ 1, . . . , n yj  0, Int: 8j ¼ 1, . . . , n Suppose that the two functions below represent the difference between the RHS and LHS of the equality constraints. gð y Þ ¼ b1 

n X

a1 , j y j

j¼1

hð y Þ ¼ b2 

n X

a2 , j y j

j¼1

 þ

 We define a i, j ¼ min 0, ai, j , ai, j ¼ max 0, ai, j and with respect to the variables’ upper bounds, we have: b1 

n X

aþ 1, j yj  gðyÞ  b1 

j¼1

n X

a 1, j yj

j¼1

We select the positive integer number θ such that: ( θ > max

b1 

n X j¼1

a 1, j uj , b1

þ

n X

) aþ 1, j uj

j¼1

Thus, we have jg( y)j < θ. Now, by multiplying the second constraint by θ and adding it to the first constraint, we get:

4.9 Knapsack Problems

191

max z ¼

n X

cj yj

j¼1

s:t: n
X

 a1, j þ θa2, j yj ¼ b1 þ θb2

j¼1

yj  uj 8j ¼ 1, . . . , n yj  0, Int: 8j ¼ 1, . . . , n It is easy to show that the feasible region of the above model and that of the primary IP model are the same. Suppose that h( y) ¼ K; obviously, K is an integer number. In addition, the new constraint is g( y) + θh( y) ¼ 0, where the definition of θ implies that K ¼ 0, hence, h( y) ¼ 0, and consequently, g( y) ¼ 0. In this way, two constraints are merged. Similarly, in the presence of more constraints, this process can be repeated for other constraints as well; however, this may lead to coefficients enlargement. Therefore, this method is useful for a few constraints. In the following, we present two examples having the structure of the knapsack problem. Example 4.35 Paying back the customer change Suppose that a seller must pay back b (mu) in a change to a customer using the minimum number of coins. Assuming that the value of the jth coin is aj (mu), the number of coins required is minimized by solving the following model: min z ¼

n X

yj

j¼1

s:t: n X

aj y j ¼ b

j¼1

yj  0, Int: 8j ¼ 1, . . . , n Example 4.36 Packing problem Another case in the family of knapsack problems is the problem of minimizing the number of packages needed to pack all items. This problem is formulated as follows:

192

4 Linearization of Nonlinear Functions

min z ¼

m X

δi

i¼1

s:t: n X j¼1 m X i¼1 m X

aj γ i, j  bδi 8i ¼ 1, . . . , m γ i, j ¼ 1 8j ¼ 1, . . . , n γ i, j ¼ 1

i¼1

δi , γ i, j 2 f0, 1g

8i ¼ 1, . . . , m, 8j ¼ 1, . . . , n

Where, b represents the volume of each package, γ i,j is a binary variable that is 1 if the item j is put in the package i; otherwise 0; and δi is a binary variable that is 1 if the package i is used; otherwise 0.

4.10

Notes and References

Some examples of this chapter and a number of exercises of Sect. 4.11 have been borrowed from Bertsimas and Tsitsiklis (1997), Castillo et al. (2002), Hock and Schittkowski (2009), Jamil and Yang (2013), Martello and Toth (1990), StancuMinasian (1997), Taha (1975) and Winston (2003).

4.11

Exercises

Exercise 4.1 Assume that x is a continuous variable, δ1, δ2 and δ3 are binary variables, and a, b, c, and d are arbitrary constants. Linearize the following constraint. x ¼ að1  ðbδ1 Þðcδ2 Þðdδ3 ÞÞ Exercise 4.2 P An optimization problem with the objective function max i2 δi x1, i wi  x2, i is given, where x1,i and x2,i are nonnegative continuous variables, δi is a binary variable, and the domain of variable wi is {1.2, 1.4, 1.6, 2, 2.1}. How can this function be rewritten linearly? Exercise 4.3 Assume that x1, x2 and x3 are nonnegative continuous variables and the constraints or the objective function of the model includes the multiplicative expression x1x2x3.

4.11

(a) (b) (c) (d)

Exercises

193

How to apply the MCER method to find an appropriate solution? How to apply the MPDT method to find an appropriate solution? How to apply the PLFs to find an appropriate solution? Consider the following optimization problem, apply the above three methods, and compare the results. max z ¼ x1 x2 x3 Optimal solution:
   s:t: x1 , x2 , x3 ¼ ð20, 11, 15Þ, z ¼ 3300 x1 þ 2x2 þ 2x3  72 0  x1  20 0  x2  11 0  x3  36

Exercise 4.4 Return to Example 4.6. Use the constraints (4.27) and (4.28) to guarantee the statement (4.24). Exercise 4.5 Consider the following optimization model: min z ¼ 9x21 þ x22 þ 9x23 s:t:

Optimal solution


   1 pffiffiffi x1 , x2 , x3 ¼ pffiffiffi , 3, 0 , z ¼ 6 3

x1 x2  1 2  x1  2 1  x2  2 1  x3  1 Obtain an appropriate solution by three methods MCER, MPDT, and the PLF. Compare the result with the problem’s optimal solution. Exercise 4.6 Consider the following optimization model. min z ¼ x1 þ x2 þ x3 s:t: 0:0025ðx4 þ x6 Þ  1  0 0:0025ðx4 þ x5 þ x7 Þ  1  0 0:01ðx5 þ x8 Þ  1  0 100x1  x1 x6 þ 833:33252x4  83333:333  0

194

4 Linearization of Nonlinear Functions

Table 4.7 Data for Exercise 4.6 j lj uj

1 573.1 585.1

2 1351.2 1368.5

3 5102.1 5117.5

4 181.5 182.5

5 295.3 296.0

6 217.5 218.5

7 286.0 286.9

8 395.3 396.0

x2 x4  x2 x7  1250x4 þ 1250x5  0 x3 x5  x3 x8  2500x5 þ 1250000  0 100  x1  10000 1000  x2 , x3  10000 10  x4 , x5 , x6 , x7 , x8  1000 With the optimal solution: x1 ¼ 579:3167, x2 ¼ 1359:943, x3 ¼ 5110:071, x4 ¼ 182:0174, x5 ¼ 295:5985, x6 ¼ 217:9799, x7 ¼ 286:4162, x8 ¼ 395:5979, z ¼ 7049:330923

(a) Solve the model by MCER method and compare the result with the optimal solution. (b) Assume that more precise estimations of the variables’ boundaries are available as shown in the following table. Repeat the method and compare the result with part (a) (Table 4.7). Exercise 4.7 We know that each real number x can be represented in the base-2 numeral system as follows: x¼

X


ak  2k



k2ℤ

where, ak 2 {0, 1} for k 2 ℤ. Assume that x1 and x2 are nonnegative continuous variables with upper bounds u1 and u2, respectively. 0  x 1  u1 ,

0  x 2  u2

Assume that the model contains the term x1x2, and we want to approximate it by an MILP model. To this end, with respect to the upper bound u1, we choose the integer K > 0 such that u1 < 2Kþ1 . Additionally, by taking into account a desired accuracy, we can choose an integer number K < 0 and ignore all digits with the place value less

4.11

Exercises

195

 than 2K . Therefore, we define the set  ¼

 K, K þ 1, . . . , 0, 1, . . . , K  1, K , and

consider the following binary variable: δk:

A binary variable that is 1 if in the base-2 representation of x1, the digit with the place value of 2k is equal to 1; otherwise 0 ðj 2 f0, 1g, k 2 Þ. Therefore, the variable x1 can be displayed as follows: x1 ¼

X ð2k  δk Þ k2

(a) Consider the base-2 numeral representation of x1 and extend the MPDT method, appropriately. (b) Compare the MILP models obtained by the MPDT method in both base-10 and base-2 numeral systems in terms of the number of constraints and variables. Which one is more suitable? (c) In Examples 4.6 and 4.7, perform the MPDT method by considering the base-2 numeral system and compare the obtained MILP model with the model that we already presented in base-10 numeral representation in terms of the number of variables and constraints. (d) Suppose that b  2 is an arbitrary integer number. Generalize the MPDT method, assuming that the variable x1 is displayed in the base-b numeral system (see Teles et al. 2012). Exercise 4.8 In what follows, some optimization problems along with their optimal solutions are presented. In each case, check whether or not the use of the MCER results in an appropriate solution. Given the fact that we know the problems optimal solution, examine the impact of tightening the variables bounds on the solution quality obtained by MCER.

196

4 Linearization of Nonlinear Functions

a) min z ¼ x21  x1 x2 þ x22 s:t: 50  xi  50 8i ¼ 1, 2 x1 ¼ 0, x2 ¼ 0, z ¼ 0 b)


2 min z ¼ 100 x2  x21 þ ð1  x1 Þ2

s:t: 1:2  xi  1:2 8i ¼ 1, 2 x1 ¼ 1, x2 ¼ 1, z ¼ 0 c) min z ¼ ðx1 þ 2x2  7Þ2 þ ð2x1 þ x2  5Þ2 s:t: 10  xi  10 x1

¼

1, x2

8i ¼ 1, 2 

¼ 3, z ¼ 0

d) min z ¼ x1 x4 ðx1 þ x2 þ x3 Þ þ x3 s:t: x1 x2 x3 x4  25  0 x21 þ x22 þ x23 þ x24  40 ¼ 0 8i ¼ 1, . . . , 4 1  xi  5 x1 ¼ 1, x2 ¼ 4:7429994, x3 ¼ 3:8211503, x4 ¼ 1:3794082, z ¼ 17:0140173 Exercise 4.9 In the following, some optimization models along with their optimal solutions are presented. In each case, linearize the problem, if possible, or choose an appropriate approximation method and compare the result with the given solution.

4.11

Exercises

197

x1 ¼ 1 ðaÞ min z ¼ max ðjx1 þ 2x2  7j, j2x1 þ x2  5jÞ

z ¼ 0

s:t:  10  xi  10 8i ¼ 1, 2 ðbÞ min z ¼

n
P i¼1

x2i  i




2

s:t:  500  xi  500

8i ¼ 1, 2

ðcÞ min z ¼ ðjx1 j  5Þ2 þ ðjx2 j  5Þ2 s:t:  500  xi  500 8i ¼ 1, 2

x2 ¼ 3

pffiffiffi 
x1 , x2 ¼ 1,  2

z ¼ 0


 x1 , x2 ¼ ð5, 5Þ

z ¼ 0

min z ¼ 0:2x3  0:8x1 x1 ¼ 0:1841264879

s:t: ðdÞ x2  ex1  0

x2 ¼ 1:202167873

x3  ex2  0

x3 ¼ 3:327322322

0  x1 , x2  100

z ¼ 0:5181632741

0  x3  10 Exercise 4.10 Suppose that f(x) has been appeared in an optimization problem where a, b and c are parameters with known values and b > 0. How can this function be rewritten linearly?  f ð xÞ ¼

0

x¼0

b þ cx

0 < α1 x f ðxÞ ¼ α1 β1 þ α2 ðx  β1 Þ > : α1 β1 þ α2 ðβ2  β1 Þ þ α3 ðx  β2 Þ

0  x  β1 β 1  x  β2 β 2  x  β3

Exercise 4.13 Return to Example 4.20. To create an approximate model, only replace the nonlinear functions h1 ðxÞ ¼ x21 and h2 ðxÞ ¼ x22 with PLFs. Solve the subsequent approximate model, and compare the result with the previous one. Exercise 4.14 Consider the following model: max z ¼ 60x1  5x21 þ 80x2  4x22 s:t: 6x1 þ 5x2  60 10x1 þ 12x2  150 x1  8 x1 , x2  0 Let a0 ¼ 0, a1 ¼ 2, a2 ¼ 4, a3 ¼ 6, a4 ¼ 8 and a00 ¼ 0, a01 ¼ 3, a02 ¼ 6, a03 ¼ 10, a04 ¼ 12 be the breakpoints in the domain of variables x1 and x2, respectively. Use PLFs to find an approximate solution to the model. Exercise 4.15 In Fig. 4.14, 10 units of goods should be sent from node 1 to 4 at the lowest cost assuming that the capacities of the arcs are unlimited. The decision variable xi,j denotes the amount of goods shipped over the arc (i, j). The network cost function is as follows: 8x1, 2 þ 2x2, 3 þ 4x3, 4 þ x21, 3 þ x22, 4 (a) Given the above assumptions, provide an NLP model. (b) Consider the breakpoints {0, 2, 6, 10} for both variables x1,3 and x2,4. Use appropriate PLFs, and approximate the nonlinear terms.

Fig. 4.14 Information of Exercise 4.15

4.11

Exercises

199

(c) Compare the result of part (b) with the problem’s optimal solution x1, 2 ¼ 5, x1, 3 ¼ 5, x2, 3 ¼ 3:5, x2, 4 ¼ 1:5, x3, 4 ¼ 8:5 Exercise 4.16 Suppose that G ¼ ð, Þ is a directed network in which each arc ði, jÞ 2  represents a communication link between two nodes i and j and is associated with a failure probability ai,j. Therefore, under the assumption of independence, for every path P
through the network, the probability of the path being survival is Q equal to ði, jÞ2P 1  ai, j . Provide an optimization model to find a path from the node s to t, having the highest probability of survival. Which of the linearization techniques, addressed in this chapter, are suitable for this model? Exercise 4.17 Rewrite any of the following models as a separable programming problem. ð aÞ

ðbÞ

max z ¼ x21 þ 3x1 x2  x22

max z ¼ x1 x2 x3

s:t:

s:t:

x1 x2  4

2x1 þ 3x2 þ 4x3 ¼ 36

x21

þ x2  6

x1 , x2  0 ð cÞ

ðdÞ

max z ¼ 10x21 þ 8x1 x2  5x1 ex1 þ 2x2 cos x2

max z ¼ 20e3x1 þx2 þ 5x21

s:t:

s:t:

x1 , x2  0

4x1 x2 þ 5x21 ¼ 60 x1 , x 2  0

Exercise 4.18 First, rewrite the following model as a separable programming problem. Then, by using PLFs, approximate it with a linear model. max z ¼ x21 þ 2x2 þ 3x3 s:t: ln x1 þ x2  2 x2 x3  20 2  x1  3 x2  5 x3  20 x1 , x2 , x3  0

200

4 Linearization of Nonlinear Functions

Exercise 4.19 Rewrite the following constraint linearly. x3  min ðx1 , x2 Þ Exercise 4.20 Answer the following questions. (a) Check why the maximum value function of Example 4.23 is equivalent to the linear constraints (4.61)–(4.64)? (b) Is it enough to only use the following two constraints to guarantee the proposition presented in Example 4.23? x3  x2 x3  x1 Exercise 4.21 Consider the following model in which, c, x, and aTk are n-component vectors, b is an m-component vector, A is an m  n, matrix and h and, dk are scalars. min z ¼ cT x s:t: Ax  b max aTk x þ dk  h

k¼1, ..., K

ð4:85Þ

x0 Can the above model be rewritten linearly without using binary variables? What if the constraint (4.85) is max aTk x þ dk  h? k¼1, ..., K Exercise 4.22 Linearize the model presented in Example 4.29, assuming that the objective function should be maximized. Exercise 4.23 (a) Rewrite the following model as an LP. min z ¼ 2x1 þ 3jx2  10j s:t:

jx1 þ 2j þ jx2 j  5 x1 , x2 free

(b) Answer to part (a) assuming that the objective function should be maximized.

4.11

Exercises

201

Exercise 4.24 (a) Assuming that cj is a nonnegative parameter for j ¼ 1, . . ., n, present two methods to linearize the following model. min z ¼

n X   cj  xj  j¼1

Ax  b

s:t:

free 8j ¼ 1, . . . , n

xj

(b) Assuming that cj is a real-valued parameter for j ¼ 1, . . ., n, rewrite the above model linearly. Exercise 4.25 Consider the following model assuming that the coefficients bi,j, di, and c0j are nonnegative. min z ¼

n   X cj xj þ c0j wj j¼1

s:t:

n
X

 ai, j xj þ bi, j wj  di

8i ¼ 1, . . . , m

j¼1

  wj ¼ xj  xj

free,

8j ¼ 1, . . . , n wj  0

8j ¼ 1, . . . , n

(a) Rewrite the model linearly in two different methods. (b) Show that if some of the coefficients bi,j are negative, then rewriting the model as LP is impossible. Exercise 4.26 Present a linear reformulation for the following model. max z ¼

1 þ 3x1 þ 3x2 þ 2x3 1 þ 2x1 þ x2 þ x3

s:t: 2x1 þ 5x2 þ x3  2 x1 þ 2x2 þ 3x3  3 x1 , x2 , x3  0 Exercise 4.27 Assuming that aj  1 for j ¼ 1, . . ., n, present a linear reformulation for the following model.

202

4 Linearization of Nonlinear Functions

min z ¼

n X

cj xj

j¼1

s:t: n X

xj ¼ 1

j¼1

lδj  xj  uδj 0 1

8j ¼ 1, . . . , n

B xj C C min B @1, Pn0 xj0 A  aj j ¼1 Pn

8j ¼ 1, . . . , n

ð4:86Þ

δ0 j0 ¼1 j

xj  0, δj 2 f0, 1g

8j ¼ 1, . . . , n

Hint: Constraint (4.86) implies that: aj  1

ð4:87Þ

xj aj  Pn x0 Pnj0 ¼1 j

ð4:88Þ

δ0 j0 ¼1 j

(4.87) is always established and concerning (4.88) and concluded that (4.86) is equivalent to: xj Pn1 j0 ¼1

Pn

¼ 1, it can be

j¼1 xj

 aj δ j0

Therefore, for j ¼ 1, 2, . . ., n, (4.86) can be replaced by only the term xj δj0 should be linearized.

Pn

j0 ¼1 xj δj

0

 aj and then,

Exercise 4.28 In this exercise, fractional programming is employed to solve a multi-objective problem. Consider the transportation problem described in Example 4.32 and suppose that ui, j is an upper bound on the amount of goods sent from the depot i to the customer j, and ti, j is the travel time from i to j. Consider three objectives: minimizing the transportation cost, minimizing the travel time and maximizing the amount of goods delivered. According to Stancu-Minasian (1978), an answer to this multiobjective problem can be obtained by solving the following model:

4.11

Exercises

203

P



Pm Pn m Pn i¼1 j¼1 ci, j xi, j i¼1 j¼1 t i, j xi, j Pm Pn i¼1 j¼1 xi, j

min z ¼



s:t: ð4:81Þ, ð4:82Þ n X xi, j  ai 8i ¼ 1, . . . , m j¼1 m X

xi, j  bj

8j ¼ 1, . . . , n

i¼1

0  xi, j  ui, j

8i ¼ 1, . . . , m,

8j ¼ 1, . . . , n

Provide an appropriate method to linearize the model. Exercise 4.29 In Sect. 4.9, we described a method for constraint aggregation according to which, any IP problem with integer coefficients can be transformed into a knapsack problem. However, the drawback of this method is that by increasing the number of constraints, the coefficients of the resulting constraint quickly become very large. Log prime method (Ram et al. 1988) is another approach overcoming this drawback. Explore this method and explain its advantages and disadvantages compared with the first method. Exercise 4.30 Consider the following zero-one knapsack problem: max z ¼

n X

cj δ j

j¼1

s:t: n X

aj δ j  b

j¼1

δj 2 f0, 1g

8j ¼ 1, . . . , n

Without loss of generality, we assume that:

P 1. Parameters cj, aj and b are positive integer numbers and nj¼1 aj > b. 2. aj  b, 8j ¼ 1, . . ., n. 3. The variables are arranged such that the following non-increasing order is satisfied: c1 c2 c   ...  n a1 a2 an Let m ¼ min

n P o P k : kj¼1 aj > b , and b ¼ b  m1 j¼1 aj .

204

4 Linearization of Nonlinear Functions

(a) Show that the solution to the LPR problem is as follows: δj ¼ 1

8j ¼ 1, . . . , m  1

δj ¼ 0

8j ¼ m þ 1, . . . , n

δm ¼

b am

and accordingly, show that U1 is an upper bound for the optimal objective value of the zero-one knapsack problem, where U1 ¼

m1 X j¼1

  b cj þ cm am

(b) Let V1 ¼

m1 X j¼1

V2 ¼

m1 X j¼1



c cj þ b mþ1 amþ1



 
c cj þ cm  am  b m1 am1

Show that U2 is an upper bound for the optimal objective value of the zeroone knapsack problem and U2  U1, where U 2 ¼ max ðV 1 , V 2 Þ (c) Consider the following model: max z ¼ 15δ1 þ 100δ2 þ 90δ3 þ 60δ4 þ 40δ5 þ 15δ6 þ 10δ7 þ δ8 s:t: 2δ1 þ 20δ2 þ 20δ3 þ 30δ4 þ 40δ5 þ 30δ6 þ 60δ7 þ 10δ8  102 δj 2 f0, 1g 8j ¼ 1, . . . , 8 The optimal solution to the above model is: δ ¼ ð1, 1, 1, 1, 0, 1, 0, 0Þ,

z ¼ 280

According to parts (a) and (b), calculate the upper bounds U1 and U2, and show U1 ¼ 295, U2 ¼ 285.

Chapter 5

Qualitative Comparison of Models

5.1

Introduction

For most of the programming problems, there may exist several mathematical models which vary in the number of variables and constraints, but they still show a formulation of the same problem. For example, Öncan et al. (2009) presented more than 10 different formulations for the traveling salesman problem (TSP). Different models of a given problem are expected to be different in the formulation, but they agree with the optimal solution. The main question, arising here, is that which model is better when more than one model exists for a given problem? The initial and correct response to this question is strongly related to the solution method used to solve the problem and determining the optimal solution. In other words, the better the formulation, the faster the way to get the optimal solution.

5.2

Model’s Quality

In LP problems, often, the number of variables and constraints indicates the quality of the model. Therefore, in these problems, the model with the least number of variables and constraints is a better formulation. Most of the optimization software, when solving LP problems, utilize preprocessing algorithms to make the most possible reduction in the number of variables and constraints to enter the solving process with a smaller coefficient matrix. However, comparing the quality of MIP models just based on the number of variables and constraints is not correct. With respect to the steps of the B&B algorithm (see Sect. 2.6.2), it can be concluded that in the IP and MIP problems, the continuous and discrete variables have different impacts on the resolution process, and the number of computational operations exponentially increase with the addition of the number of discrete variables. Therefore, the effect of discrete variables on the complexity of the solution © Springer Nature Switzerland AG 2019 S. A. MirHassani, F. Hooshmand, Methods and Models in Mathematical Programming, https://doi.org/10.1007/978-3-030-27045-2_5

205

206

5 Qualitative Comparison of Models

process is so high that the role of continuous variables can be ignored. Of course, this does not mean that the number of discrete variables is an appropriate index to compare the models. In other words, it is not necessarily accurate to say that a model with less discrete variables is always better. The following examples describe this topic. For more details, see Williams (2013). Example 5.1 Addition of binary variables A company plans to set up a new plant. This plant must be located in the north or south of the area and equipped with one of the processes b or c. For this purpose, the following variables are defined: δn, b : A binary variable that is 1 if the plant is placed in the north and equipped with the process b; otherwise 0. δn, c : A binary variable that is 1 if the plant is placed in the north and equipped with the process c; otherwise 0. δs, b : A binary variable that is 1 if the plant is placed in the south and equipped with the process b; otherwise 0. δs, c : A binary variable that is 1 if the plant is placed in the south and equipped with the process c; otherwise 0. Since the company intends to establish only one plant, the following constraint is necessary: δn, b þ δn, c þ δs, b þ δs, c ¼ 1 Now, we define the binary variable γ that is 1 if the new plant is constructed in the north; otherwise 0. The relation between this variable and the previous ones is described by the following constraints: δn, b þ δn, c  γ ¼ 0 δs, b þ δs, c þ γ ¼ 1 Although γ is a non-essential variable, its definition can be useful in the branching process of the B&B algorithm. Similarly, another variable can be introduced to indicate which process should be used in the new plant. For more details, see Williams (2013). Example 5.2 Addition of integer variables P Consider the constraint j aj yj  b in which all variables and coefficients are supposed to be integer. It is possible to introduce the nonnegative integer variables v as a slack variable, and replace the aforementioned constraint by the P constraint j aj yj þ v ¼ b. Usually, solvers adds such a slack variable to the model, but they consider this variable to be nonnegative continuous. While if we define v as an integer variable and prioritize it in the branching process, the constraint

5.3 Comparison of Models Based on LPR

207

can act as a cut and will be able to limit the LP feasible region. For more details, see Williams (2013). The following example shows that in some cases, reducing the number of integer variables leads to a better model. Example 5.3 Reduction of integer variables Assume that  with index i and  with index j are collections of identical-trucks and journeys, respectively. It is necessary to decide on the dispatch of trucks such that each truck can be deployed to a maximum of one journey. Consider a model in which the binary variable δi, j is 1 if the truck i is sent to journey j; otherwise 0. Our variable definition lead to a model with many symmetric solutions. For example, if we have  ¼ f1, 2, 3g and  ¼ f1, 2g, it is clear that the two solutions (δ1,1 ¼ 1, δ2,2 ¼ 1, δ3,2 ¼ 1) and (δ1,2 ¼ 1, δ2,1 ¼ 1, δ3,2 ¼ 1) are symmetric and both refer to the same assignment. Such solutions enlarge the search tree and are not computationally desirable. Therefore, by defining the integer variable yj as the number of trucks sent to journey j, we get a new model with less variables and no symmetric solutions. The symmetric solutions (δ1,1 ¼ 1, δ2,2 ¼ 1, δ3,2 ¼ 1) and (δ1,2 ¼ 1, δ2,1 ¼ 1, δ3,2 ¼ 1) correspond to the solution ( y1 ¼ 1, y2 ¼ 2) of the new model. On the other hand, although the resolution of an MIP problem (especially, solving the LPR problem), is affected by the number of constraints, it is not necessarily accurate to claim that a model with fewer constraints is always better. Because in this category of problems, sometimes a model with more constraints may be solved faster. Examples 5.7 and 5.8 which will be presented later, confirm this. Concerning the above discussion, it can be concluded that the comparison of integer programming models calls for a different criterion. Most algorithms of solving PIPs or MIPs (e.g., B&B, B&C, and cutting plane methods) use the feasible region of LPR problem and need to resolve a large number of LP problems to get the IP/MIP optimal solution. Consequently, the closeness of the optimal objective value of the LPR to that of the original problem has a significant effect on the algorithm convergence rate. Whenever these two values are close, the algorithm convergence rate may also increase. Hence, it seems that this observation is a good foundation for the model’s comparison.

5.3

Comparison of Models Based on LPR

In this section, we describe the manner of comparing the models based on LPR. For this purpose, some terms such as polytope, formulation, and so forth should be defined. Most of the materials of this section have been derived from Chen et al. (2010), Klotz and Newman (2013) and Wolsey (1998).

208

5 Qualitative Comparison of Models

Fig. 5.1 Three formulations associated with Example 5.4

Definition 5.1 Polytope A collection of points  ¼ fxjAx  b, x 2 n g, which are characterized by a set of linear constraints, is called a polytope. Obviously,  is a convex set. Definition 5.2 Formulation Suppose that the set   n is given (note that  may be explicitly introduced as a set of points or implicitly defined by some equations), then,  ¼ fyjAy  b, y 2 n g is a formulation for  if  ¼ . In this case, we define the polytope , in the real n-dimensional space, as the feasible region of the LPR problem corresponding to the formulation . Clearly,  ¼  \ n . Accordingly,  is a formulation for , if the multidimensional space  obtained from the LPR of  has the two following properties: •   . •  contains no more integer solutions than . Similarly, suppose that the set   ðn1  n2 Þ is given (note that  may be explicitly introduced as a set of points or implicitly defined by some equations); then,  ¼ fðx, yÞjAx þ A0 y  b, x 2 n1 , y 2 n2 g is a formulation for  if  ¼ . In this case, we define the polytope  in the (n1 + n2)-dimensional real space equal to the feasible region of the LPR problem corresponding to the formulation . Clearly,  ¼  \ ðn1  n2 Þ. Example 5.4 Three different formulations for a set of integer points Consider the set  as follows:  ¼ fð2, 1Þ, ð1, 2Þ, ð2, 2Þ, ð3, 2Þ, ð1, 3Þ, ð2, 3Þ, ð3, 3Þg In Fig. 5.1, the polytopes 1 , 2 , and 3 represent the feasible region of the LPR problems corresponding to three different formulations of . Example 5.5 Two different formulations for a set of mixed-integer points Consider the set   ð  Þ as  ¼ fðx, yÞjx þ y  1, x  0, y  0, Int:g. As shown in Fig. 5.2, this set corresponds to the point (0, 1) and the points on the line segment AC. Obviously, the polytopes ABC and ADC are corresponding to the feasible region of LPR of two different formulations of .

5.3 Comparison of Models Based on LPR

209

Fig. 5.2 Two formulations associated with Example 5.5

Definition 5.3 Better formulation Suppose that 1 and 2 are feasible regions of LPR problems corresponding to two different formulations of the set . Formulation 1 is better than 2 , if 1 ⊊ 2 . In this way, the comparison of different formulations of a problem would be possible. For example, according to Fig. 5.1, addressed in Example 5.4, since 1 ⊊ 2 and 1 ⊊ 3 , then the first formulation is better than the second and third ones. Nevertheless, note that with this criterion, we cannot necessarily compare any arbitrary two formulations. For example, according to Fig. 5.1, one cannot say which of the second and third formulations is better. Although the area of the feasible region shown by 2 is smaller than that of 3 , the feasible region of an LPR problem corresponding to a formulation, alone, cannot be an upright criterion to determine its quality. Example 5.6 The best formulation among three cases Consider the following set of points:  ¼ fð0, 0, 0, 0Þ, ð1, 0, 0, 0Þ, ð0, 1, 0, 0Þ, ð0, 0, 1, 0Þ, ð0, 0, 0, 1Þ, ð0, 1, 0, 1Þ, ð0, 0, 1, 1Þg It is easy to verify that 1 , 2 , and 3 are different formulations for the set . n o 1 ¼ δ 2 f0, 1g4 : 83δ1 þ 61δ2 þ 49δ3 þ 20δ4  100 n o 2 ¼ δ 2 f0, 1g4 : 4δ1 þ 3δ2 þ 2δ3 þ δ4  4 ( ) δ 2 f0, 1g4 : 4δ1 þ 3δ2 þ 2δ3 þ δ4  4, δ1 þ δ2 þ δ3  1, 3 ¼ δ1 þ δ4  1 Let 1 , 2 , and 3 be the feasible regions corresponding to LPR problems associated with 1 , 2 , and 3 , respectively. ( 1 ¼

δ 2 4 : 83δ1 þ 61δ2 þ 49δ3 þ 20δ4  100, 0  δi  1 ð8i ¼ 1, . . . , 4Þ

)

210

5 Qualitative Comparison of Models

( 2 ¼ ( 3 ¼

δ 2 4 : 4δ1 þ 3δ2 þ 2δ3 þ δ4  4, 0  δi  1 ð8i ¼ 1, . . . , 4Þ

)

δ 2 4 : 4δ1 þ 3δ2 þ 2δ3 þ δ4  4, δ1 þ δ2 þ δ3  1,

)

δ1 þ δ4  1, 0  δi  1 ð8i ¼ 1, . . . , 4Þ

In order to show that the formulation 2 is better than 1 , it is enough to prove that 2 ⊊ 1 . To this end, suppose that δ^ is an arbitrary point belonging to 2 . We have: 4δ^1 þ 3δ^2 þ 2δ^3 þ δ^4  4 By multiplying to 20, we get: 80δ^1 þ 60δ^2 þ 40δ^3 þ 20δ^4  80

ð5:1Þ

On the other hand, the first constraint of 1 can be expressed as: ð80δ1 þ 60δ2 þ 40δ3 þ 20δ4 Þ þ ð3δ1 þ δ2 þ 9δ3 Þ  100 Since 0  δ^i  1, we have: 3δ^1 þ δ^2 þ 9δ^3  20

ð5:2Þ

By summing up both sides of inequalities (5.1) and (5.2), it is concluded that δ^ holds in the first constraint of 1 , and hence, 2  1 . To show that this relationship is strictly established, it is enough to prove that there is at least one point in 1 that does not belong to 2 . Clearly, the point (0.1, 1, 0, 1) has such a condition, and therefore, 2 ⊊ 1 . It is easy to investigate that the formulation 3 is also better than 2 . Example 5.7 Two formulations for un-capacitated facility location problem A company wants to set up several warehouses in a set  ¼ f1, . . . , mg of potential locations to serve a set  ¼ f1, . . . , ng of customers. Suppose that there is no limitation on the capacity of warehouses, fi is the cost of establishing the warehouse i, dj is the demand of customer j, and ci,j is the cost of transferring a unit of goods from location i to customer j. To provide a model meeting the customers’ demands and minimizing the total company’s cost, we define the decision variables as follows: δi : A binary variable that is 1 if a warehouse is set up in location i; otherwise 0. xi,j : A continuous variable indicating the fraction of demand of customer j provided by warehouse i. The following two formulations can be developed for this problem:

5.3 Comparison of Models Based on LPR

211

Model 5.1 Formulation 1 for UFLP min z ¼ s:t: X i2 X

X

f i δi þ

i2

xi, j ¼ 1

XX ci, j d j xi, j i2 j2

8j 2 

xi, j  n  δi

8i 2 

ð5:3Þ

j2

xi, j  0 8i 2 , j 2  δi 2 f0, 1g 8i 2  Model 5.2 Formulation 2 for UFLP min z ¼ s:t: X

X

f i δi þ

i2

xi, j ¼ 1

XX ci, j dj xi, j i2 j2

8j 2 

i2

xi, j  δi 8i 2 , j 2  xi, j  0 8i 2 , j 2  δi 2 f0, 1g 8i 2 

ð5:4Þ

Note that these formulations differ only in constraints (5.3) and (5.4). Since the number of constraints of the second formulation is n times the number of constraints of the first formulation, the size of the second formulation is larger. But we show that the second formulation is better. For this purpose, we define the sets 1 and 2 as the feasible regions of LPR problems associated with the first and the second formulations, respectively, and prove that 2 ⊊ 1 . Note that in LPR problems, the variable δi is assumed to take values belonging to the interval [0, 1]. Obviously, by summing up the inequalities (5.4) for all j 2 , constraint (5.3) is obtained. Therefore, all points belonging to 2 are in 1 , as well; and hence, 2  1 . To show 2 ⊊ 1 , we need to find a point in 1 not in 2 . For simplicity, let n be a number divisible by m. Thus, there is an   integer k  1 such that n ¼ km. Now, we define the point x^, δ^ as follows: k δ^i ¼ , n

 x^i, j ¼

1

j ¼ kði  1Þ þ 1, . . . , kði  1Þ þ k

0

otherwise

  It is not difficult to verify that the point x^, δ^ belongs to the set 1 ∖2 .

212

5 Qualitative Comparison of Models

Example 5.8 Improving formulation by adding new constraints Consider the following model: max z ¼ 3δ1 þ 2δ2 þ δ3 þ 2δ4 þ δ5 s:t: 4δ3 þ 3δ4 þ 5δ5  10 δ1 þ 2δ4  2 3δ2 þ 4δ5  5 δ1 , . . . , δ5 2 f0, 1g We refer to the above formulation as formulation 1 and denote the feasible region of its LPR problem by 1 . Now, consider the constraint δ3 + δ4 + δ5  2. It is clear that all of the feasible solutions to formulation 1 satisfy this constraint. Therefore, formulation 2 which is obtained by adding this constraint to formulation 1, is equivalent to formulation 1. Let 2 be the feasible region of LPR problem associated with formulation 2; obviously, we have 2  1 , and since ð0, 0, 1, 13 , 1Þ 2 1 ∖2 , it can be concluded that 2 ⊊ 1 ; and hence, formulation 2 is better than formulation 1. Now consider the constraints δ1 + δ4  1 and δ2 + δ5  1. One can easily investigate that all of the feasible solutions to formulation 2 satisfy these constraints. Therefore, formulation 3, which is obtained by adding these constraints to formulation 2, is equivalent to formulation 2. Let 3 be the feasible region of LPR problem associated with formulation 3. We have 3  2 and since ð12 , 12 , 12 , 34 , 34Þ 2 2 ∖3 , it can be concluded that formulation 3 is better than formulation 2. Example 5.9 Two formulations for the minimum spanning tree problem In the graph theory, a tree is referred to an undirected graph in which any pair of vertices are connected by exactly one path. Suppose that G ¼ ð, Þ is an undirected connected graph in which, jj ¼ n and with each arc e 2 , an integer cost, denoted by ce, is associated. Assume that the graph T ¼ ð, 0 Þ is a tree containing all vertices of G, and 0  . This tree is referred to as a spanning tree. Define the cost of a tree equal to the total cost of its arcs. To find a spanning tree with the minimum cost, we present two optimization models and compare their quality. In both models, for each e 2 , the binary variable δe is 1 if the arc e is in the tree; otherwise 0. In the first formulation, for each 0 ⊊  with 0 6¼ ∅, the set 1 ð0 Þ contains all arcs both sides of which are in the set 0 . In other words, 1 ð0 Þ ¼ fði, jÞji 2 0 , j 2 0 g Thus, the first formulation is as Model 5.3 in which constraint (5.5) ensures that the tree has precisely n  1 arcs and constraint set (5.6) causes the tree to be acyclic. Such a constraint is known as subtour elimination constraint in the literature. We

5.3 Comparison of Models Based on LPR

213

denote the LPR of this formulation by 1 in which binary variable δe 2 {0, 1} is replaced by continuous variable 0  δe  1. Model 5.3 The first formulation for the minimum spanning tree problem min z ¼ s:t: X

X

ce δ e

e2

δe ¼ n  1

X

ð5:5Þ

e2

δe  j0 j  1

80   : 0 6¼ ∅

ð5:6Þ

0

e21 ð Þ

δe 2 f0, 1g

8e 2 

In the second formulation, for each 0 ⊊  with 0 6¼ ∅, the set 2 ð0 Þ contains all arcs exactly one side of which is within the set 0 . In other words, 2 ð0 Þ ¼ fði, jÞji 2 0 , j 2 = 0 g Thus, the second formulation is as Model 5.4 which is different from Model 5.3 in constraint (5.7) ensuring the absence of any cycle in the tree. Model 5.4 The second formulation for the minimum spanning tree problem min z ¼ s:t: X

X

ce δ e

e2

δe ¼ n  1

X

e2

δe  1

80   : 0 6¼ ∅

ð5:7Þ

0

e22 ð Þ

δe 2 f0, 1g

8e 2 

We denote the LPR of this formulation by 2 in which binary variable δe 2 {0, 1} is replaced by continuous variable 0  δe  1. We need to show that 1 ⊊ 2 and conclude that the first formulation is better than the second one. For each 0  , and 0 6¼ ∅, we have:  ¼ 1 ð0 Þ [ 2 ð0 Þ [ 1 ð∖0 Þ Therefore,

214

5 Qualitative Comparison of Models

X e21 ð0 Þ

δe þ

X

δe þ

e21 ð∖0 Þ

X

δe ¼

e22 ð0 Þ

X

δe

ð5:8Þ

e2

Let δ^ 2 1 , for each 0   with 0 6¼ ∅, we have: X

δ^e  j0 j  1

e21 ð0 Þ

X

δ^e  j∖0 j  1 0

e21 ð∖ Þ

Since

P

^ ¼ n  1, with respect to Eq. (5.8), we have

e2 δe

P

^  1, and

e22 ð0 Þ δe

therefore, δ^ 2 2 . Up to now, we show that 1  2 . Now, consider Fig. 5.3 in which the number placed next to each arc represents the cost of that arc. The solution δ1, 2 ¼ δ1, 3 ¼ δ2, 4 ¼ δ3, 4 ¼ 12 , δ2, 5 ¼ δ4, 5 ¼ 1 belongs to 2 but does not belong to 1 because it violates constraint (5.6) for 0 ¼ f2, 4, 5g. In the examples examined so far, the formulations are the same in the definition of variables, and the difference between formulations was only in the constraints. While two formulations may be different in the definition of decision variables, as well. To compare such formulations, consider the following definition. Definition 5.4 Comparing two formulations having different variables Assume that 1  ðn1  n2 Þ and 2  ðn1  n2  n3 Þ are feasible regions corresponding to two different formulations of a given problem, where compared with the first formulation, the second one has n3 extra variables. The feasible region of LPR problems for the first and the second formulations are denoted by 1 and 2 which are n1 + n2 and n1 + n2 + n3 dimensional polytopes, respectively. Therefore, we have: 1 ¼ 1 \ ðn1  n2 Þ 2 ¼ 2 \ ðn1  n2  n3 Þ Since 1  ðn1  n2 Þ and 2  ðn1  n2  n3 Þ, it is necessary to project 2 on ℝn1  ℝn2 . Thus,

Fig. 5.3 Graph associated with Example 5.9

5.3 Comparison of Models Based on LPR

215

2 n1 n2 n3 Projð n1 n2 Þ ¼ fðx, yÞ 2 ð   Þj∃v 2  : ðx, y, vÞ 2 2 g 2 Now, Projð n1 n2 Þ and 1 are comparable because both of them are subsets of n1 n2 ℝ  ℝ . Therefore, the second formulation is better than the first one if 2 Projð n1 n2 Þ ⊊1 .

Example 5.10 Two formulations for the production planning problem A company is planning to produce a new product during a horizon of n timeperiods. For each period t, assume that ft is the fixed cost, ct is the variable cost of producing one unit of the product, ht is the inventory cost per unit of the product, and dt is the demand. We define the decision variables as follows: δt : The binary variable that is 1 if the product line is active at time-period t; otherwise 0 xt : The production level at time-period t wt : The inventory level at time-period t In this way, the problem is formulated as Model 5.5, which minimizes the total production and storage costs. Constraint set (5.9) ensures that in each time-period, the demand is satisfied and the remaining product is stored. Constraint set (5.10) implies that in any time-period, if δt ¼ 0 then xt ¼ 0. Model 5.5 The first formulation for the production planning problem min z ¼

n X t¼1

f t δt þ

n X t¼1

ct xt þ

n X

ht wt

t¼1

s:t: 8t ¼ 1, . . . , n wt1 þ xt ¼ dt þ wt xt  Mδt 8t ¼ 1, . . . , n 8t ¼ 1, . . . , n w0 ¼ 0, wt , xt  0 δt 2 f0, 1g 8t ¼ 1, . . . , n

ð5:9Þ ð5:10Þ ð5:11Þ

Note that P if we add wn ¼ 0, then, in the constraint set (5.10), the value of M can be set equal to nt¼1 dt . Now, the new variable vt0 , t is defined as the amount of products that are produced in the period t0 to satisfy the demand of period t (assuming that t0  t). In this case, it is no longer necessary to define the variables xt and wt, and the problem is formulated as Model 5.6 in which r t0 , t ¼ ct0 þ ht0 þ ht0 þ1 þ . . . þ ht1 . Constraint set (5.12) guarantees the satisfaction of the demand and constraint set (5.13) implies that if in the period t0, δt0 ¼ 0, then for each t  t 0 , vt 0 , t ¼ 0

216

5 Qualitative Comparison of Models

Model 5.6 The second formulation for the production planning problem min z ¼ s:t: t X

n X

f t δt þ

r t 0 , t vt 0 , t

t 0 ¼1 t¼t 0

t¼1

vt 0 , t ¼ d t

n X n X

8t ¼ 1, . . . , n

ð5:12Þ

t 0 ¼1

8t, t 0 ¼ 1, . . . , n : t 0  t vt 0 , t  d t δ t 0 0 vt 0 , t  0 8t, t ¼ 1, . . . , n : t 0  t δt 2 f0, 1g 8t ¼ 1, . . . , n

ð5:13Þ ð5:14Þ

Optionally, we can add the variables xt and wt to this model. In this case, we need to introduce the following constraints to express the relationship between variables: xt 0 ¼

n X

8t 0 ¼ 1, . . . , n

vt 0 , t

ð5:15Þ

t¼t 0

wt ¼

t X

xt 0 

t 0 ¼1

t X

8t ¼ 1, . . . , n

d t0

ð5:16Þ

t 0 ¼1

Without loss of generality, we add variables xt and wt and the constraints (5.15) and (5.16) to the second formulation, and consider 1  ðn  n  n Þ, 2    nðnþ1Þ as the feasible regions associated with the first and n  n  n   2 second formulations, respectively. ( 1 ¼ ( 2 ¼

ðδ, x, wÞ j Constraints ð5:9  5:11Þ

)

0  δt  18t

ðδ, x, wÞ j Constraints ð5:125:16Þ, ð5:11Þ

)

0  δt  18t

Given the above relations, we have: ( 2 Projð n n n Þ

¼

ðδ, x, wÞ 2 ðn  n  n Þ j nðnþ1Þ 2

∃v 2 

: ðδ, x, w, vÞ 2 2

)

5.4 Ideal Formulation

217

2 It is easy to verify Projð n n n Þ  1 . We only need to show that this relation is   ^ x^, w ^ as follows: strictly valid. To this end, we define the point δ,

X X d ^t ¼ δ^t ¼ t , x^t ¼ d t , w d t0 x^t0  M t 0 ¼1 t 0 ¼1 t

The

point

  ^ x^, w ^ δ,

belongs

t

to

8t ¼ 1, . . . , n

2 1 ∖Projð n n n Þ

and

therefore,

2 Projð n n n Þ ⊊1

meaning that the second formulation is preferred to the first one. Now, the question, appeared here, is that what is an ideal formulation? This topic will be discussed in the next section.

5.4

Ideal Formulation

Before getting involved with the details of this topic, we recall some definitions from optimization theory. Most of the materials of this section have been derived from Chen et al. (2010) and Wolsey (1998). Definition 5.5 Convex hull Suppose that the set  ¼ fa1 , . . . , ak g  n is given. The convex hull of , denoted by ConvðÞ, is a collection of all convex linear combinations of the points of the set . That is, ( ConvðÞ ¼

Xk

λa i¼1 i i

j ai 2 , λi  0,

k X

) λi ¼ 1

i¼1

The convex hull of the set  is the smallest convex set containing . Figure 5.4 exhibits the convex hull of a set of points. Suppose that the set   n is given. Obviously, ConvðÞ is a convex polytope in the n-dimensional real space, so that   ConvðÞ, and the components of any Fig. 5.4 Convex hull of a set of points

218

5 Qualitative Comparison of Models

extreme point of this polytope are integers. If we are able to express the ConvðÞ explicitly as a polytope, i.e., ConvðÞ ¼ fx 2 n jDx  dg, then we can solve the problem min z ¼ cT x s:t: Dx  d instead of min z ¼ cT y s:t: y2 If  is another formulation of the set , and  is the feasible region corresponding to its LPR problem, then we always have ConvðÞ  . According to the above definition, we are ready to define the ideal formulation. Definition 5.6 Ideal formulation The set   n is given. A formulation of  in which, the feasible region of its LPR problem is equal to ConvðÞ represents an ideal formulation. In an ideal formulation, the optimal solution of the LPR problem is the same as that of the IP problem. The totally unimodular models, described in the next section, are in the class of problems whose initial formulation is ideal. In these problems, the solution times of LPR and IP problems are the same.

5.4.1

Totally Unimodular Models

One of the ideal formulations, which is described here, is the totally unimodular models. In order to describe their features, it is necessary to introduce some concepts. Definition 5.7 Unimodular matrix The square matrix B is called unimodular if all of its determinant equals +1 or 1. 2 3 2 1 3 7 1 6 7 6 For example, matrices 4 0 1 2 5 and 4 2 1 0 2 0

entries are integers and its 0 1 0

0

3

7 0 5 are unimodular. 1

5.4 Ideal Formulation

219

Definition 5.8 Totally unimodular matrix The m  n integer matrix A is called totally unimodular if the determinant of all of its non-singular square sub-matrices are +1 or 1. In other words, A is totally unimodular if all of its non-singular square sub-matrices are unimodular. For example, the following matrices are totally unimodular: 2

1

6 1 6 6 4 0

1

1

0 1

0 0

0

1

0

3

2 1 1 7 7 6 7, 4 1 1 5 0 0 0

0

1

1

1 1

0 1

0 0

1

3

7 0 5 1

However, the matrices below, are not. 

2

1 1

 1 6 ,40 1 1

1

1 1 0

0

3

2

1

6 7 61 1 5, 6 40 1 1

1 0 1 1

0 0

3

1 17 7 7 1 05 0 1

Theorem 5.1 Properties of a totally unimodular matrix For any totally unimodular matrix, the following statements are valid: (a) If A is a totally unimodular matrix, then each entry of A is 0, 1, or 1. (b) The matrix A is totally unimodular, if and only if its transpose AT is totally unimodular. (c) The matrix A is totally unimodular if and only if the matrix [A| I] is totally unimodular. Proof The results are established by direct use of the definition of totally unimodular matrix. □ Definition 5.9 Totally unimodular model A mathematical model with a totally unimodular coefficients matrix is called a totally unimodular model. Suppose that the following model is totally unimodular: min z ¼ cT x s:t: Ax ¼ b x0 According to the Cramer’s rule, for a matrix B corresponding to any basic feasible solution to the LPR problem, we have:    1  det Bj xj ¼ B b j ¼ , j ¼ 1, . . . , n det ðBÞ

220

5 Qualitative Comparison of Models

where, Bj is a matrix similar to B with the difference that the column j is replaced by the vector b. If all components of b take integer values, then det(Bj) is also an integer value. Moreover, the totally unimodular property of the matrix A implies that det (B) ¼  1 which ensures the integrality of the x-vector components. In other words, every basic solution to the LPR problem, including the optimal one, has integer components. Concerning the above explanation and Theorem 5.1, it can be concluded that if b has integer components and A is a totally unimodular matrix, then, in each of the three following models, any basic feasible solution to the LPR model has also integer components. Consequently, the IP problem can be solved in the form of an LP problem without enforcing the integrality restrictions. min z ¼ cTx s. t. Ax  b x  0, Int.

min z ¼ cTx s. t. Ax  b x  0, Int.

min z ¼ cTx s. t. Ax ¼ b x  0, Int.

Checking the totally unimodular property of a matrix by calculating the determinant of all of its square sub-matrices is computationally difficult and intangible. The following theorem provides a sufficient condition for the examination of a totally unimodular matrix, which is simple to use. Theorem 5.2 Sufficient conditions for totally unimodularity If matrix A with components 1, 0, +1 satisfies the following properties, it is totally unimodular: (a) Each column of A contains at most two nonzero entries. (b) The rows of matrix A can be partitioned into two distinct subsets 1 and 2 such that – If two non-zero entries with the same sign are in the same column, then the row-index of one of them is in the set 1 and the row-index of the other one is in the set 2 . – If two non-zero entries with different signs are in the same column, then the row-indices of both of them are either in the set 1 or in the set 2 . Proof Refer to Wolsey (1998). □ The coefficient matrices of different problems including transportation problem, shortest path problem, assignment problem, maximum flow problem, and the minimum cost network flow problem are all totally unimodular, and their basic solutions satisfy integrality restriction. All of these problems are special cases of the minimum cost network flow problem which is addressed in the following example. Example 5.11 Minimum cost network flow problem Consider the directed graph G ¼ ð, Þ and assume that ci, j is the cost of transferring one unit of goods through the arc ði, jÞ 2 , and ui, j and li, j, respectively, are the upper and lower bounds of the flow on the arc (i, j). In particular, we may have li, j ¼ 0 and ui, j ¼ 1. Each vertex i 2  is associated with a parameter Pbi, which is positive for supplier vertices and negative for demand vertices such that i2 bi ¼ 0. The goal

5.4 Ideal Formulation

221

is to transport the goods from suppliers to the demand vertices at the minimum cost. Assuming that the variable xi, j shows the amount of goods transferred on the arc (i, j), the problem is formulated as Model 5.7: Model 5.7 Minimum cost network flow problem X

min z ¼ s:t: X

ci , j xi , j

ði, jÞ2

xi, j 

j:ði, jÞ2

X

xj, i ¼ bi

8i 2 

ð5:17Þ

j:ð j, iÞ2

li, j  xi, j  ui, j

8ði, jÞ 2 

ð5:18Þ

A numerical example of the above problem is presented in Fig. 5.5, in which the number next to the vertex i represents bi. The matrix representation of constraints (5.17) for this example is as follows: x1,2 1 1 0 0 0 0

x1,4 1 0 0 1 0 0

x2,3 0 1 1 0 0 0

x3,1 1 0 0 0 0 0

x3,2 0 1 1 0 0 0

x3,5 0 0 1 0 1 0

x3,6 0 0 1 0 0 1

x4,5 0 0 0 1 1 0

x5,1 1 0 0 0 1 0

x5,3 0 0 1 0 1 0

x6,5 0 0 0 0 1 1

¼ ¼ ¼ ¼ ¼ ¼

3 0 0 +2 4 +5

Coefficients of the constraints li, j  xi, j  ui, j can also be added to the above matrix. Therefore, assuming that A is the coefficient matrix of the problem, we can consider it as A¼½AjIjI T , in which D corresponds to the flow equilibrium constraints, the first identity matrix indicates the coefficients of upper bound constraints, and the second one refers to coefficients of the lower bound constraints. By defining 1 as the set of all rows of matrix D and 2 as an empty set, the totally unimodularity of matrix D is implied by Theorem 5.2. Now, the totally unimodularity of matrix A is obtained from Theorem 5.1.

Fig. 5.5 The graph of the minimum cost network flow problem

222

5 Qualitative Comparison of Models

Remark 5.1 The topic of totally unimodularity, discussed here, is valid only for PIP problems. In an MIP model, although similar conditions under which integer variables take integer values in the LPR problem can be provided, they are usually difficult to check. In practice, the presentation of an ideal formulation is not always easy. So far, many attempts have been made to reformulate the problem or rewrite the initial formulation in the form of an ideal formulation. Among the existing approaches, one can use the “lift and project methods” which are capable to obtain the problem’s convex hull by developing the solution space in a higher dimensional space and then projecting it on the initial space. These methods are computationally demanding and cannot be used in practice for large problems. In the next section, the Reformulation and Linearization Technique (RLT), as a “lift and project method”, is introduced to provide the problem’s convex hull for two types of problems BIP and mixed BIP.

5.4.2

Reformulation and Linearization Technique

This section aims to provide a general understanding of RLT addressed in (Sherali and Adams 1999). Before introducing this technique, we examine a variable reduction method, which is known as the Fourier–Motzkin elimination. This method will be used later in the RLT. Fourier–Motzkin Elimination Method Consider a system consisting of m inequalities and n variables as follows: n X

ai, j xj  bi , i ¼ 1, . . . , m

ð5:19Þ

j¼1

To eliminate variable xk from the above system and project the space of variables x1, . . ., xk  1, xk, xk þ 1, . . .xn on the space of variables x1, . . ., xk1, xkþ1, . . .xn, we only need to divide the inequalities in (5.19) into three groups þ ,  and 0 with respect to the sign of the parameter ai, k. Thus, for each i ¼ 1, . . ., m, we perform the following:   P If ai, k > 0, then the ith inequality can be written as xk  ai1, k bi  nj¼1, j6¼k ai, j xj . þ We show the RHS of this inequality with C þ i ðX Þ, and rewrite it as xk  C i ðX Þ, where þ X¼(x1, . . ., xk1, xkþ1, . . . xn), and then, add index i to the set  .  P If ai, k < 0, then the ith inequality can be written as xk  ai1, k bi  nj¼1, j6¼k ai, j xj .  We show the RHS of this inequality with C  i ðX Þ, and rewrite it as xk  C i ðX Þ,  where X¼(x1, . . ., xk1, xkþ1, . . . xn), and then, add index i to the set  .

5.4 Ideal Formulation

223

If ai, k ¼ 0, this variable does not exist in the ith inequality. Therefore, we add the index i to the set 0 and show the ith inequality with C 0i ðX Þ, where X ¼ (x1, . . ., xk1, xk+1, . . . xn). It is evident that any solution of system (5.19) should satisfy the following constraints: þ C i0 ðXÞ  xk  C i ðXÞ

8i 2 þ , i0 2 

Therefore, the system (5.19) can be rewritten as follows, in which the variable xk is not involved: þ C i0 ðXÞ  C i ðXÞ,

8i 2 þ , 8i0 2 

C 0i ðXÞ, 8i 2 0

ð5:20Þ

Assuming that m+, m and m0 are respectively the cardinality of the sets þ ,  and 0 , the system (5.20), consists of n  1 variables and m+  m + m0 constraints. Hence, the original n-dimensional space has been transferred to a space of dimension n  1. The Fourier–Motzkin elimination method can be used to solve LP problems. For this purpose, it is sufficient to implement it repeatedly, to finally obtain a one-variable problem including the variable z (i.e., the variable corresponding to the objective function). By fixing the value of variable z, the value of the other variables are recursively determined. In this way, either the value of all variables is determined or the problem infeasibility or unboundedness is inferred. Example 5.12 Solving an LP by Fourier–Motzkin elimination method Solve the following LP by the Fourier–Motzkin elimination method. max z ¼ 2x1 þ 3x2 s:t: x1  2x2  4 2x1 þ x2  18 x2  10 x1 , x2  0 Answer Because of maximization, the objective function is added to the constraint set as 2x1  3x2  0. Therefore, z  2x1  3x2  0 x1  2x2  4

224

5 Qualitative Comparison of Models

2x1 þ x2  18 x2  10 x1  0 x2  0 To eliminate x1, the above system is rewritten as: 1 3 x1  z  x2 2 2 x1  4 þ 2x2 1 x1  9  x2 2

ð5:21Þ

x2  10 x1  0 x2  0 If we assign the numbers from 1 to 6 to the above inequalities, we get þ ¼ f2, 3g,  ¼ f1, 5g and 0 ¼ f4, 6g. Therefore, we have: 1 3 z  x2  x1  4 þ 2x2 2 2 1 3 1 z  x2  x1  9  x2 2 2 2 0  x1  4 þ 2x2 1 0  x1  9  x2 2 x2  10 x2  0 Which is equivalent to: z  7x2  8 1 z  x2  9 2 2x2  4 1 x 9 2 2

5.4 Ideal Formulation

225

x2  10 x2  0 Since the third and the fourth constraints above are redundant, they are removed, and then, to eliminate x2, the system is rewritten as follows: z8 7 1 x2  z  9 2 x2 

ð5:22Þ

x2  10 x2  0 If we assign the numbers from 1 to 4 to above inequalities, we get þ ¼ f3g,  ¼ f1, 2, 4g, and 0 ¼ f g. Therefore, we have: z8  x2  10 7 1 z  9  x2  10 2 0  x2  10 Consequently, z  78 z  38 Therefore, the boundary obtained for z equals 38. Because the maximum value of z is desirable, we put z ¼ 38. By replacing z in the system (5.22), we have: x2 

30 7

x2  10 x2  10 x2  0 So, x 2 ¼ 10. Now by replacing z and x 2 in the system (5.21), we get: x1  4

226

5 Qualitative Comparison of Models

x1  24 x1  4 x1  0   Which means that the optimal solution is x 1 , x 2 ¼ ð4, 10Þ and z ¼ 38. Example 5.13 Detecting infeasibility by Fourier–Motzkin elimination method Solve the following LP by Fourier–Motzkin elimination method. max z ¼ 5x1 þ x2 s:t: 2x1 þ x2  5 2x1 þ 3x2  6 x2  1 x1 , x2  0 Answer Because of maximization, the objective function is added to the constraint set as z  5x1 + x2. In different iterations, we get the following systems: Iteration 1

2

3

4

System z  5x1  x2  0 2x1 + x2  5 2x1 + 3x2  6 x2  1 x1  0 x2  0 x1   15 x2 þ 15 z 5 x1  1 2 x2 þ 2 x1   32 x2 þ 3 x2  1 x1  0 x2  0 1 1 3 5 z  5 x2  x1  3  2 x2 1 5 3 2 x2 þ 2  x1  3  2 x2 0  x1  3  32 x2 x2  1 x2  0 2z + 13x2  30 x2  12 x2  2 x2  1 x2  0

5.4 Ideal Formulation

227

In the last system, the second and fourth inequalities conflict and the system is infeasible. Reformulation and Linearization Technique (RLT) The RLT method can provide the problem’s convex hull by lifting the solution space to a higher-dimensional space and then projecting it on the initial space. The sets  ¼ f1, 2, . . . , ng and  ¼ f1, . . . , mg are given. Consider a BIP problem with the variables δ1, δ2, . . ., δn and the following constraints: n X

ai, j δj  bi

8i 2 

ð5:23Þ

j¼1

δj 2 f0, 1g

8j 2 

Remark 5.2 In the description of this method, the type of constraint (5.23) is considered as . However, the type of problem constraints can also be  or ¼, and this will not affect the process of implementing the RLT method. Let  and  be the feasible regions corresponding to the above set of constraints and its LPR problem, respectively. In other words, ( ¼

δj

) ai, j δj  bi ð8i 2 Þ, δj 2 f0, 1g ð8j 2 Þ

j¼1

( ¼

n X

δj

n X

) ai, j δj  bi ð8i 2 Þ, 0  δj  1 ð8j 2 Þ

j¼1

In the following, we explain how the RLT produces an ideal formulation corresponding to the convex hull of . Let d 2 f0g [ , and suppose that 1 and 2 are two subsets of  where 1 \ 2 ¼ ∅ and j1 [ 2 j ¼ d. We define the multiplication factor F d ð1 , 2 Þ as:

F d ð 1 , 2 Þ ¼

8 >

:

Q j21

! δj

 Q 1  δj

!

d¼0 d2

j22

Obviously, F d ð1 , 2 Þ is a polynomial of degree d and for every d 2 f0g [ ,

n multiplication factors. The fact that the variables are binary implies there are 2d d that the factor F d ð1 , 2 Þ is always nonnegative. Suppose that d is an arbitrary value belonging to the set f0g [ . We consider the following three steps:

228

5 Qualitative Comparison of Models

Step 1: Reformulation We multiply all factors of level d (i.e., F d ð1 , 2 Þ) in constraints (5.23) to obtain the following constraints: n X

! ai, j δj  bi F d ð1 , 2 Þ  0

j¼1

ð5:24Þ

8i 2 , 81 , 2   : 1 \ 2 ¼ ∅, j1 [ 2 j ¼ d Also, we set d ¼ min fd þ 1, ng, and generate the following constraints which are equivalent to multiplying F d ð1 , 2 Þ to inequalities 0  δj  1 F d ð1 , 2 Þ  0

81 , 2   : 1 \ 2 ¼ ∅, j1 [ 2 j ¼ d

ð5:25Þ

Step 2: Linearization We linearize the nonlinear constraints (5.24) and (5.25). To this end, since δj is a binary variable, the terms δkj and δj(1  δj) are equivalently substituted by δj and zero, respectively. Also, each nonlinear expression obtained by the multiplication of some binary variables is replaced by a new variable, say γ k. Since through the linearization step, some new variables are added to the problem, the problem is lifted into a higher-dimensional space. We denote the feasible region of the lifted problem and its LPR by d and d , respectively. Step 3: Projection Since  is in the space of initial problem (the space of variables δ), and d is in the lifted space (the space of variables δ and γ), we need the Fourier–Motzkin elimination method to project d on the original space. If we show the projected space by d , we have: d ¼ fδj∃γ, ðδ, γÞ 2 d g Since F0(∅, ∅) ¼ 1, therefore 0 denotes the LPR of the problem. Theorem 5.3 Improving the initial formulation According to the above explanations, the following relations would be established: convðÞ ¼ n  n1      1  0 ¼  Also, for every d 2 f0g [ , the set d contains the set . In other words, we have:

5.4 Ideal Formulation

229

  d

8d 2 f0g [ 

Proof Refer to Sherali and Adams (1999). □ Theorem 5.3 shows that for each d, the formulation d is better than the initial formulation, and the higher the level of d, the stronger the formulation d . Finally, at most at level d ¼ n, n corresponds to the convex hull of the feasible region of the original problem (5.23). In other words, as d changes from 0 to n, a hierarchy of different formulations of  is obtained where each one is stronger than the previous one, starting by the LPR problem and ending with the convex hull representation. Hence, the optimal solution to the IP problem can be obtained by solving the problem on the space n . Figure 5.6 shows the process of lifting the solution region to a higher dimensional space, and then, projecting it on the initial space to get the problem convex hull (for case n ¼ 2). Note that the process of determining d for a given level d does not depend on the previous levels, and hence, it is possible to implement RLT for each level independently. Example 5.14 Implementing RLT on a BIP problem The following is the original formulation of a BIP problem:  ¼ fðδ1 , δ2 Þj2δ1 þ 2δ2  1, δ1 , δ2 2 f0, 1gg

ð5:26Þ

Compare the formulations obtained by applying RLT for levels d ¼ 0, 1, 2, and show that the formulation corresponding to level d ¼ 2, matches the convex hull of the problem. Fig. 5.6 Illustration of lift and project method

230

5 Qualitative Comparison of Models

Answer We have:  ¼ fð0, 1Þ, ð1, 0Þ, ð1, 1Þg  ¼ fðδ1 , δ2 Þj2δ1 þ 2δ2  1,

0  δ1  1, 0  δ2  1g

In this example, there are two binary variables, and concerning Theorem 5.3, at most after the execution of level d ¼ 2, the convex hull of the problem is obtained. The multiplicative factors corresponding to levels d ¼ 0, 1, and 2 are as follows: Level d ¼ 0: F0(∅, ∅) ¼ 1 Level d ¼ 1: F1({1}, ∅) ¼ δ1, F1({2}, ∅) ¼ δ2, F1(∅, {1}) ¼ 1  δ1, F1(∅, {2}) ¼ 1  δ2 Level d ¼ 2: F2({1, 2}, ∅) ¼ δ1δ2, F2({1}, {2}) ¼ δ1(1  δ2), F2({2}, {1}) ¼ (1  δ1)δ2, F2(∅, {1, 2}) ¼ (1  δ1)(1  δ2) In the following, we will determine 0 , 1 , and 2 and compare their feasible regions. 0 : The level d ¼ 0 corresponds to LPR problem and we have: 0 ¼ fðδ1 , δ2 Þj2δ1 þ 2δ2  1, δ1 , δ2 2 ½0, 1 g The region associated with 0 is depicted in Fig. 5.7. 1 : To determine 1 , we multiply the factors of level d ¼ 1 to each of the constraints defining (5.26) and all sign restrictions. Note that multiplying the factors of level d ¼ 1 to restrictions δ1  0, δ1  1, δ2  0, δ2  1 is equivalent to the enforcement of nonnegativity restrictions on the factors of level d ¼ 2. Therefore, we have:

Fig. 5.7 Feasible region for 0 in Example 5.14

5.4 Ideal Formulation

231

δ1 ð2δ1 þ 2δ2 Þ  δ1 δ2 ð2δ1 þ 2δ2 Þ  δ2 ð1  δ1 Þð2δ1 þ 2δ2 Þ  1  δ1 ð1  δ2 Þð2δ1 þ 2δ2 Þ  1  δ2 δ1 δ2  0 δ1 ð1  δ2 Þ  0 1  δ1 Þδ2  0 ð 1  δ1 Þð1ð δ2 Þ  0 We replace the nonlinear term δ1δ2 with γ 1, 2 to get 1 as follows: δ1 þ 2γ 1, 2  0 δ2 þ 2γ 1, 2  0 δ1 þ 2δ2  2γ 1, 2  1 2δ1 þ δ2  2γ 1, 2  1 γ 1, 2  0 δ1  γ 1, 2  0 δ2  γ 1, 2  0 δ1  δ2 þ γ 1, 2  1 Above system can be restated as follows: 1 δ 2 1 1 δ  2 2 1 1  δ1 þ δ2  2 2 1 1  δ1 þ δ2  2 2 0

γ 1, 2  γ 1, 2 γ 1, 2 γ 1, 2 γ 1, 2

γ 1, 2  δ1 γ 1, 2  δ2 γ 1, 2  δ1 þ δ2  1

232

5 Qualitative Comparison of Models

Fig. 5.8 Feasible region of 1

By using the Fourier–Motzkin elimination method γ 1,2 is removed and after eliminating redundant constraints, we get the following formulation which is corresponding to 1 : 2δ1 þ 2δ2  1 3δ1 þ δ2  1 δ1  0 δ1 þ 2δ2  1 2δ1 þ δ2  1 δ2  0 δ1 þ 3δ2  1 δ1  1 δ2  1 The feasible region associated  with 1 is depicted in Fig. 5.8. We can see that the point 12 , 0 is in 0 but not in 1 , and hence, we have: 1 ⊊ 0 2 : To determine 2 , we multiply the factors of level d ¼ 2 to each of the constraints defining (5.26), and enforce the nonnegativity restrictions on the factors of level d ¼ 2. Thus, we have: δ1 δ2 ð2δ1 þ 2δ2 Þ  δ1 δ2 δ1 ð1  δ2 Þð2δ1 þ 2δ2 Þ  δ1 ð1  δ2 Þ ð1  δ1 Þδ2 ð2δ1 þ 2δ2 Þ  ð1  δ1 Þδ2 ð1  δ1 Þð1  δ2 Þð2δ1 þ 2δ2 Þ  ð1  δ1 Þð1  δ2 Þ

5.4 Ideal Formulation

233

δ1 δ2  0 δ1 ð1  δ2 Þ  0 ð1  δ1 Þδ2  0 ð1  δ1 Þð1  δ2 Þ  0 We replace the nonlinear term δ1δ2 with γ 1, 2 to get 2 as follows: γ 1, 2  0 δ1  γ 1, 2  0 δ2  γ 1, 2  0 δ1  δ2 þ γ 1, 2 ¼ 1 Above system can be restated as follows: γ 1, 2  0 γ 1, 2  δ1 γ 1, 2  δ2 γ 1, 2  1 þ δ1 þ δ2 γ 1, 2  1 þ δ1 þ δ2 By using the Fourier–Motzkin elimination method, γ 1,2 is removed, and after eliminating redundant constraints, we get the following formulation which is corresponding to 2 : δ1 þ δ2  1 δ1  1 δ2  1 The feasible region associated with 2 is depicted in Fig. 5.9. As can be seen in this figure, the extreme points are vectors with integer components, and 2 represents the problem convex hull. The RLT can be generalized to be used for mixed BIPs. In these problems, the linearization phase of RLT requires two groups of variables. The first group contains the variables that replace the product of binary variables, and the second one involves the variables representing the multiplication of binary and continuous variables. The following example examines the problem of this type.

234

5 Qualitative Comparison of Models

Fig. 5.9 Feasible region of 2

Fig. 5.10 Feasible region of 0

Example 5.15 Implementing RLT on an MIP problem Consider a mixed BIP with the following feasible region:  ¼

ðδ1 , x1 Þ j δ1 þ x1  2,  δ1 þ x1  1, 2δ1  2x1  1, δ1 2 f0, 1g, x1  0

 ð5:27Þ

Compare the formulations obtained by applying RLT for levels d ¼ 0, 1, and show that the formulation corresponding to level d ¼ 1 matches the convex hull of the problem. Answer In this example, we have only one binary variable, and Theorem 5.3 implies that the convex hull representation is obtained after executing the level d ¼ 1. The multiplicative factors corresponding to the levels d ¼ 0, 1 are as follows: Level d ¼ 0: F0(∅, ∅) ¼ 1 Level d ¼ 1: F1({1}, ∅) ¼ δ1, F1({2}, ∅) ¼ δ2, F1(∅, {1}) ¼ 1  δ1, F1(∅, {2}) ¼ 1  δ2 Now, we need to determine 0 and 1 . 0 : The level d ¼ 0 corresponds to LPR problem and we have:  0 ¼

ðδ1 , x1 Þ j δ1 þ x1  2,  δ1 þ x1  1, 2δ1  2x1  1, δ1 2 ½0, 1 , x1  0

The feasible region of 0 is depicted in Fig. 5.10.



5.4 Ideal Formulation

235

1 : To determine 1 , we multiply the factors of level d ¼ 1 to each of the constraints defining (5.27), and enforce the nonnegativity restrictions on the factors of level d ¼ 1. Therefore, we get: δ1 ðδ1 þ x1 Þ  2δ1 δ1 ðδ1 þ x1 Þ  δ1 δ1 ð2δ1  2x1 Þ  δ1 δ 1 x1  0 ð1  δ1 Þðδ1 þ x1 Þ  2ð1  δ1 Þ ð1  δ1 Þðδ1 þ x1 Þ  ð1  δ1 Þ ð1  δ1 Þð2δ1  2x1 Þ  ð1  δ1 Þ ð1  δ1 Þx1  0 δ1  0 δ1  1 By replacing the nonlinear terms δ1x1 and δ21 , respectively, with w1 and δ1, we have: δ1 þ w1  0 2δ1 þ w1  0 δ1  2w1  0 w1  0 2δ1 þ x1  w1  2 δ1 þ x1  w1  1 δ1  2x1 þ 2w1  1 x1  w 1  0 δ1  0 δ1  1 By eliminating the variable w1, 1 is obtained as follows: 2δ1  2x1  1 δ1  2x1  0 0  x1  1 0  δ1  1

236

5 Qualitative Comparison of Models

Fig. 5.11 Feasible region of 1

The feasible region associated with 1 is depicted in Fig. 5.11. As can be seen in this figure, the extreme points are vectors with integer components, and 1 represents the problem convex hull. As mentioned earlier, to obtain d , at the level d ¼ 1, . . ., n, it is necessary to

n d multiplicative factors and multiply them in the problem congenerate 2 d straints. Therefore, it is very difficult to form d for large values of d. However, in some cases, due to the special structure, it is not necessary to form n to obtain the convex hull representation, and d at levels d < n represents the convex hull. This is illustrated in the following example. Example 5.16 Achieving convex hull at levels less than n Consider the following problem and show that the formulation obtained by RLT at level d ¼ 1 coincides with the problem convex hull.  ¼ fðδ1 , δ2 Þj  3δ1 þ 3δ2  1, δ1 þ δ2  1, δ1 , δ2 2 f0, 1gg

ð5:28Þ

Answer The multiplicative factors corresponding to the levels d ¼ 0, 1 are as follows: Level d ¼ 0: F0(∅, ∅) ¼ 1 Level d ¼ 1: F1({1}, ∅) ¼ δ1, F1({2}, ∅) ¼ δ2, F1(∅, {1}) ¼ 1  δ1, F1(∅, {2}) ¼ 1  δ2 We have:  0 ¼

ðδ1 , δ2 Þ j 3δ1 þ 3δ2  1, δ1 þ δ2  1, 0  δ1  1, 0  δ2  1

The feasible region of 0 is depicted in Fig. 5.12.



5.4 Ideal Formulation

237

Fig. 5.12 Feasible region of 0

To determine 1 , we multiply the factors of the level d ¼ 1 to each of the constraints defining (5.28) and enforce the nonnegativity restrictions on the factors of level d ¼ 2. Therefore, we have: δ1 ð3δ1 þ 3δ2 Þ  δ1 δ1 ðδ1 þ δ2 Þ  δ1 δ2 ð3δ1 þ 3δ2 Þ  δ2 δ2 ðδ1 þ δ2 Þ  δ2 ð1  δ1 Þð3δ1 þ 3δ2 Þ  ð1  δ1 Þ ð1  δ1 Þðδ1 þ δ2 Þ  ð1  δ1 Þ ð1  δ2 Þð3δ1 þ 3δ2 Þ  ð1  δ2 Þ ð1  δ2 Þðδ1 þ δ2 Þ  ð1  δ2 Þ δ1 δ2  0 δ 1 ð1  δ 2 Þ  0 ð1  δ1 Þδ2  0 ð1  δ1 Þð1  δ2 Þ  0 By replacing the nonlinear term δ1δ2 by γ 1,2, we get: 4δ1 þ 3γ 1, 2  0 γ 1, 2  0 2δ2  3γ 1, 2  0 δ1 þ 3δ2  3γ 1, 2  1 δ1 þ δ2  γ 1, 2  1 3δ1 þ δ2 þ 3γ 1, 2  1

238

5 Qualitative Comparison of Models

δ1 þ δ2  γ 1, 2  1 γ 1, 2  0 δ1  γ 1, 2  0 δ2  γ 1, 2  0 δ1  δ2 þ γ 1, 2  1 By eliminating variable γ 1,2, 1 is obtained as follows: 0  δ1  1 δ2 ¼ 0 The feasible region associated with 1 is depicted in Fig. 5.13. Sherali et al. (1998) presented an improved version of RLT for problems including constraints with special structures (e.g., general and simple upper bound constraints). The difference between the improved and primary versions is in the way of producing multiplication factors, and its advantage is the possibility of reaching the convex hull at lower levels. Implementation of different levels of RLT will be difficult when the number of binary variables is high. Therefore, a matrix-based approach for RLT implementation is proposed by MirHassani et al. (2015a). Here, the RLT method is described only for BIP problems. In the cases of implementing RLT for IP problems, it is necessary to use Lagrange interpolation polynomials to generate multiplicative factors. See Adams and Sherali (2005) for more details.

Fig. 5.13 Feasible region of 1

5.5 Improving Formulations

5.5

239

Improving Formulations

As mentioned earlier, there usually exist various alternative formulations for any given IP problem, and among them, the better formulation is the one for which the feasible region of the LPR problem is closer to the convex hull representation. The ideal formulation, however, is the formulation for which the feasible region of the LPR problem coincides with the convex hull representation. So far, much effort has been made to close the LPR problem to the convex hull representation. For example, preprocessing tools try to decrease the gap between them by modifying the constraints, removing unnecessary variables, adding new constraints, tightening the RHS values, and adjusting the technical coefficients in the hope of improving the efficiency of the solution algorithms. Also, RLT, described in the previous section, can provide the convex hull representation through lifting the feasible region to a higher-dimensional space and then projecting it on the initial space. Of course, not only this method is useful to find the convex hull representation, but also Theorem 5.3 implies that for each d, the formulation d is better than the initial formulation, and the strength of d increases by increasing the level d.

5.5.1

Improvement by Cutting Planes

Today, much effort has been made to improve the methods of solving IP and MIP problems, and has led to the invention of various algorithms. Most existing methods focus on iteratively solving several LP problems (the LPR problem or other LP problems created by adding the cuts to LPR), and approaching the problem’s convex hull. The cutting plane method is an example of such algorithms that by iteratively generating cuts and adding them to the set of problem’s constraints, restricts the feasible region such that after solving a finite number of LP problems, the optimal solution to the IP problem is obtained. In this method, we try to obtain a sequence of formulations where the last generated LP problem contains the optimal solution of the IP problem as an extreme point. Note that this formulation does not necessarily represent the convex hull of the main problem. The main steps of the cutting plans procedure to solve MIP and IP problems are as follows: Step 1 Let P represent the current LP problem and initialize it as the LPR problem. Step 2 Solve the problem P. If the solution satisfies the integrality restrictions, stop. The current solution is optimal. Otherwise, go to Step 3. Step 3 Create a new cut and add it to problem P (this cut must be valid, meaning that the current solution is eliminated, but none of the solutions to the IP problem is removed), and go to Step 2.

240

5 Qualitative Comparison of Models

x2

Fig. 5.14 General representation of cutting plane method

T

h1 x

≤ r1

h2T x

≤ r2

L1 L2

L3

T

h3 x

≤ r3 x1

L1 = {x ∈ R2⏐Ax ≤ b, x ≥ 0} L2 = L1 ∩ {x ∈ R2⏐Hx ≤ r} L3 = Conu({x ∈ Z2⏐Ax ≤ b,x ≥ 0}) For every i , the i th cut should satisfy the following conditions: ∀x ∈ L3 1) hiT x ≤ ri 2) x ∈ L1:hi T x > ri E

In this way, in Step 3 of each repetition, a new formulation of the problem is obtained that is better than the previous one. In Fig. 5.14, 3 represents the convex hull of all solutions to the IP problem, while 1 is the feasible region of the RLP problem. By adding the cuts, we get the region 2 , which contains all solutions to the IP problem, but involves only a subset of the fractional solutions of 1 . For more details of cutting plans procedure, see Taha (1975). Example 5.17 Improve the following problem by the cutting plane method. max z ¼ 7y1 þ 9y2 s:t: y1 þ 3y2  6 7y1 þ y2  35 y1 , y2  0, Int: Answer The steps of the cutting plane method on this problem can be summarized as follows:   Iteration 1 The optimal solution to the LPR problem is ðy1 , y2 Þ ¼ 92 , 72 , and the objective value equals 63. Accordingly, the valid cut y2  3, as the first cut, is added to the LPR problem.

5.5 Improving Formulations

241

Fig. 5.15 Steps of cutting plane on Example 5.17

Iteration 2 The optimal solution to the LPR problem with the first cut is ðy1 , y2 Þ ¼ 32  7 , 3 with the objective value of 59. On this basis, the valid cut y1 + y2  7, as the second cut, is produced. Iteration 3 The optimal solution to the LPR problem with the first and second cuts is (y1, y2) ¼ (4, 3) with the objective value of 55, which corresponds to the optimal solution to the IP problem. Figure 5.15 shows that by adding each of the above cuts, we get a new formulation of the problem which is better than the previous one. Finally, we emphasize that it is not always easy to present a formulation corresponding to the problem’s convex hull. However, it is important to formulate the problem so that the gap between these two spaces is reduced. For example, selecting an appropriate value for M in a constraint using big M, plays an important role in reducing the gap. To clarify the discussion, consider the following condition in which δ and x are respectively binary and nonnegative continuous variables. δ¼0 ) x¼0 The above if-then constraint is rewritten in the form of a linear constraint as: x  Mδ

ð5:29Þ

Taking the value of M as small as possible (i.e., close to the real upper bound of x) is an important task and is effective in diminishing the feasible region of the LPR problem. For example, assume that by examining the other constraints of the problem, we conclude that the value of x never exceeds 1000. If we fix M at 106, then the following fractional solution is valid for (5.29): x ¼ 700, δ ¼

1 2

242

5 Qualitative Comparison of Models

However, if we fix M at 1000, the above solution violates (5.29), and accordingly, some fractional solutions to the LPR problem are eliminated. On the other hand, considering large amounts for M may also result in computational errors. In fact, many optimization software use a specific precision, namely ε, and accordingly, fractional solutions that their differences with the integer values are less than ε are considered as integer solutions. Thus, in the above example, if the software uses the precision ε > 106, then, by setting M at 106, the fractional solution x ¼ 1, δ ¼ 106, which satisfy (5.29), may be accepted by the software as an integer feasible solution. However, if a more appropriate value is chosen for M, such a computational error does not occur.

5.5.2

Improve by Symmetry Breaking

Assume that  (indexed by i, i0) and  (indexed by j, j0) are, respectively, the sets of customers and homogenous trucks where jj ¼ m, and jj ¼ n. In the VRP (see Exercise 2.6) two types of decisions are made. The first decision assigns each customer to a single truck, and the second one determines the order of meeting customers assigned to each truck. Therefore, in these problems, we encounter with the partitioning the customers of the set  to n pairwise disjoint groups. A common method is to use the binary variable δi, j that is 1 if customer i is in group j; otherwise 0. Since each customer must be included in only one group, we have: X

δi, j ¼ 1

8i 2 

j2

The weakness of this formulation is that there are many alternative solutions that represent the same type of grouping. For example, let  ¼ fa, b, c, dg,  ¼ f1, 2, 3g, and consider the partition {{a}, {c}, {b, d}}. All of the solutions presented in Table 5.1 represent the same partitioning. In other words, associated with any given partitioning, there are n! alternative symmetric solutions. Such a feature raises the difficulty in the branching step of B&B algorithm, and hence, it is necessary to eliminate the repetitive solutions corresponding to different permutations of a given solution. To this end, we consider  as an ordered set and do the partitioning with respect to the following simple rule: Table 5.1 Alternative symmetric solutions

# 1 2 3 4 5 6

Group 1 {b, d} {b, d} {a} {a} {c} {c}

Group 2 {a} {c} {b, d} {c} {b, d} {a}

Group 3 {c} {a} {c} {b, d} {a} {b, d}

5.6 Irreducible Infeasible Subsystems

243

Group j will accept the customer i with the smallest index which has not been included in any of the previous groups 1, 2, . . ., j  1. In this way, there is a one-toone correspondence between the solutions and the different types of grouping. The initial formulation takes all of the permutations of Table 5.1 as different solutions, while, among them, only the solution presented in row #3 satisfies the above rule. In order to comply with this rule, it is necessary to add some constraints to the problem. These constraints which we refer to as symmetry breaking constraints are formulated as follows: Step 1: Group 1/Customer 1 The first customer should be assigned to the first group. Therefore, we set δ1, 1 ¼ 1, and eliminate all variables δ1, j with j > 1. Step 2: Group 2/Customer 2 If the second customer is not in group 1 (i.e., δ2,1 ¼ 0), then it should be assigned to the second group (i.e., δ2,2 ¼ 1). That is imposed by constraint (1  δ2,2)  δ2,1 Group 2/Customer 3 If the second customer is in the first group (i.e., δ2,1 ¼ 1) and the third customer is not assigned to the first group (i.e., δ3, 1 ¼ 0), then it should be assigned to the second group (i.e., δ3,2 ¼ 1) that is guaranteed by (1  δ3,2)  (1  δ2,1) + δ3,1. Group 2/Customer i0 If the customers i ¼ 2, . . ., i0  1 are in the first group (i.e., δi, 1 ¼ 1, 8 i ¼ 2, . . ., 0 i  1) and the customer i0 is not in the first group (i.e., δi0 , 1 ¼ 0), then the customer i0 must be included in the second group (i.e., δi0 , 2 ¼ 1) that is guaranteed   Pi0 1   1  δi, 1 þ δi0 , 1 . by 1  δi0 , 2  i¼2 Step 3: Groups j0 ¼ 3, . . ., n  1 Each group j0 must accept the customer with the smallest index that is not included in the preceding groups (i.e., groups 1, . . ., j0  1). Therefore, for each customer i0, it is necessary to satisfy the following statement: If the customers i ¼ 2, . . ., i0  1 are included in the groups j < j0 and the customer 0 i is not in these groups, then the customer i0 must be placed in the group j. This is ensured by the following constraint set: 1  δi0 , j0 

Xi0 1  Xj0 1  Xj0 1 1  δ δ 0 , 8i0 ¼ j0 , . . . , m þ i¼2 j¼1 i, j j¼1 i , j

In this way, all variables δi0 , j0 with i0 < j0 are unnecessary and should be eliminated Step 4: Group n There is no need to add any constraint for the last group, and this group will automatically include all of the remaining customers.

5.6

Irreducible Infeasible Subsystems

Irreducible Infeasible Subsystem (IIS) is a powerful tool to identify the cause of infeasibility of a problem (see Gleeson and Ryan 1990). An IIS is referred to a subset of constraints of the problem that represents an infeasible subsystem, but if any of the

244

5 Qualitative Comparison of Models

Fig. 5.16 A simple IIS

constraints of this set is eliminated, the system becomes feasible. Figure 5.16 shows a simple IIS. In order to identify the cause of infeasibility of a problem, it is necessary to generate all of its IISs. The Farkas’ lemma based method, the deletion filter, the additive method and the elastic filter (Chinneck 2007) are among the various methods presented to identify an IIS. Here, we focus on a method based on Farkas’s lemma which is applicable to LP problems. For this purpose, we first remind the Farkas’ lemma. Farkas’ Lemma Let A and b be two real matrices with the sizes m  n and m  1, respectively. One and only one of the following two systems is feasible: The first system Ax  b x free

The second system wTA ¼ 0 wTb < 0 w0

Concerning the Farkas’ lemma, it can be verified that there is a one-to-one correspondence between the IISs of the first system and the extreme points of the following system. To prove, refer to Gleeson and Ryan (1990). wT A ¼ 0 wT b ¼ 1

ð5:30Þ

w0 ^ is one of the extreme points of the system (5.30), it corresponds to Therefore, if w an IIS. This IIS contains a subset of constraints of system Ax  b, whose corresponding components in the vector w^ are non-zero (Gleeson and Ryan 1990). For more illustration, consider the following example: Example 5.18 Finding all IISs of an infeasible system Find all IISs corresponding to the following infeasible system: x1  x2  0 2x2  1 x1  x2  2 x2  2 2x1  x2  4

5.6 Irreducible Infeasible Subsystems

245

Answer Assume that the numbers 1–5 are associated with the first through the fifth constraints of the problem, respectively. To find all IISs, we need to specify all extreme points of the following polyhedron: 8 > > > > > >
> 6 > > 4 0 > > : 2

1

3

2

7 6 2 7 6 7 6 T6 ¼ 0, w 1 7 7 6 7 6 1 5 4 1

0

3

9 > > > > > > =

7 1 7 7  27 7 ¼ 1, w  0> > 7 >  25 > > > ; 4

that is: w1 ¼ 1, w2 ¼ 1, w3 ¼ 1, w4 ¼ 0, w5 ¼ 0 1 2 w1 ¼ 0, w2 ¼ , w3 ¼ 0, w4 ¼ , w5 ¼ 0 3 3 4 3 2 w1 ¼ , w2 ¼ , w3 ¼ 0, w4 ¼ 0, w5 ¼ 5 5 5 Therefore, f1 , 2 , 3 g are three IISs of the problem, where k contains the numbers associated with the constraints belonging to the k th IIS. 1 ¼ f1, 2, 3g, 2 ¼ f2, 4g, 3 ¼ f1, 2, 5g Remark 5.3 There are different statements of the Farkas’ lemma. For example, consider the following statement: Let A and b be real matrices of sizes m  n and m  1, respectively. Then, one and only one of the following two systems are feasible: The first system Ax  b x0

The second system wTA  0 wTb < 0 w0

Therefore, it can be verified that there is a one-to-one correspondence between the IISs of the first system and the extreme points of the following system. To prove, refer to Gleeson and Ryan (1990). wT A  0 wT b ¼ 1 w0

ð5:31Þ

246

5 Qualitative Comparison of Models

The Minimal Infeasible Subsystem One of the methods that can be used to identify the cause of infeasibility is to first determine all IISs of the problem; and then, look for a minimal infeasible subsystem (MIS) which is the smallest subset of constraints that should be eliminated from the original system to maintain the problem feasibility. Suppose that the total number of IISs is equal to K, and for each k ¼ 1, . . ., K, let the set k define the constraints in the k th IIS. Now, it is sufficient to solve the following model and determine the MIS: Model 5.8 A set covering problem to determine the MIS min z ¼ s:t: X

m X

δi

i¼1

δi  1

k ¼ 1, . . . , K

ð5:32Þ

i2k

δi 2 f0, 1g 8i ¼ 1, . . . , m In this model, δi is a binary variable that is 1 if the i th constraint is chosen for deletion; otherwise 0. Constraint set (5.32) ensures that at least one constraint of each IIS is removed such that the residual system, obtained after the deletion of selected constraints, is feasible. The above model is a set covering problem and detects the minimum number of constraints that should be eliminated from the original system in order to achieve a feasible system. The subset of constraints selected for deletion by this model is called MIS. Example 5.19 Determining the MIS of a system Consider the following infeasible system, and assume that the numbers 1–9 are associated with the first through the ninth constraints, respectively. 1 1  x1 þ x2  2 2 2x1  x2  3 3x1 þ x2  5 x4  2 3x3  x4  2 x3  5 x1 þ x4  10 x1 þ 2x2 þ x3  14 x2 þ x3  1 xi  0

8i ¼ 1, . . . , 4

5.6 Irreducible Infeasible Subsystems

247

For this system, the following IISs have been identified: 1 ¼ f1, 2, 5, 6, 7g, 2 ¼ f4, 5, 6g, 3 ¼ f1, 2, 3g Where each set k contains the numbers associated with constraints belonging to the kth IIS. Use Model 5.8 to find the minimum number of constraints that should be removed from the original system to get a feasible subsystem. Answer We define the binary variable δi that is 1 if the i th constraint is chosen for deletion; otherwise 0 (i ¼ 1, 2, . . ., 9). Then, we solve the following model: min z ¼

9 X

δi

i¼1

s:t: δ1 þ δ2 þ δ5 þ δ6 þ δ7  1 δ4 þ δ5 þ δ6  1 δ1 þ δ2 þ δ3  1 δi 2 f0, 1g

8i ¼ 1, 2, . . . , 9

In the optimal solution to the above model, variables δ1 and δ5 take one and other variables become zero, which means that the removal of the first and the fifth constraints are sufficient to achieve a feasible subsystem. Now, we discuss another approach to find MIS for a known infeasible system. Consider the following infeasible system: n X

ai, j xj  bi

8i ¼ 1, . . . , m

j¼1

xj  0

8j ¼ 1, . . . , n

It is enough to solve the following model to determine the MIS. Model 5.9 An MIP problem to determine MIS min z ¼ s:t: n X

m X

δi

i¼1

ai, j xj  Mδi  bi

8i ¼ 1, . . . , m

j¼1

xj  0 8j ¼ 1, . . . , n δi 2 f0, 1g, 8i ¼ 1, . . . , m

248

5 Qualitative Comparison of Models

where δi is a binary variable that is 1 if the i th constraint is chosen for deletion; otherwise 0, and M is a sufficiently large positive number.

5.7

Constraint Logic Programming

Constraint logic programming (CLP) is another approach to formulate optimization problems. CLP models are similar to IP models in the sense that they include decision variables, objective function, and constraints, and the value of decision variables should be determined so that the constraints of the problem are met and the objective function is optimized. In a CLP model, the domain of variables is generally limited to a finite set of values, and constraints are expressed as atomic or component logical propositions that are not necessarily mathematical equations or inequalities, while in an MILP model, all constraints are required to be equations or inequalities of types  or . The content of this section has been motivated by Hooker (2012), Balas (2004), Van Hoeve (2001) and Williams (2009).

5.7.1

A Comparison Between CLP and IP

In an IP model, constraints need to be expressed as linear equations or inequalities. Therefore, to express a given concept, it is sometimes necessary to define many constraints and binary variables that lead to a large scale model. While in a CLP, different logical propositions such as “if-then” and “either-or” can be directly included in the model; and it is also allowed to directly use various functions such as modulo, integer division, minimum and maximum as they are, without any replacement. Therefore, a CLP model provides a simpler and more compact representation of the problem, in which fewer variables and constraints are needed, and the testing and debugging processes can be performed in a more straightforward way. An IP model can be generalized to an MIP model, by incorporating continuous decision variables, whereas, in a CLP, the domain of variables is generally restricted to a finite set of values. In general, the optimization methods are based on three processes of searching, inferring, and relaxing. The search process examines the feasible region of the problem, while inference and relaxation processes help the search process to be performed more intelligent. The most common ways to search process are B&C (in the resolution of IP models) and branch and inference (in the resolution of CLP models).

5.7 Constraint Logic Programming

249

The inference process provides implicit information about the regions containing the optimal solutions and accelerates the search process. In the methods of solving IP problems, the inference process is performed by adding cutting plans and implementing pre-processing techniques. While in CLPs, it happens by filtering the domain of variables and bound propagation. Also, the relaxation process replaces the problem with a simpler one which may provide valuable information about the optimal solution. In the methods of solving IP models, the LPR model plays an important role based on which some methods such as B&B and B&C are developed. In this regard, dealing with IP models requires more mathematical skills. However, in the CLP, the use of LPR models is not possible. The main benefit of IP models is the possibility of using linear relaxation that helps to calculate the upper and lower bounds on the optimal objective value and prove the solution optimality, while this is not the case in CLP unless by enumerating all of the solutions. It is easier to recognize the problem structure concerning the CLP model, and such information may help the optimization software to get the optimal solution more quickly. CPLEX is one of the most popular and efficient solvers for IP models (IBM 2011). Also, PROLOG is a logic programming language which is equipped with software and efficient tools for solving CLP models. Recently, the hybrid IP-CLP methods which simultaneously utilize the capabilities of both models to accelerate the problem-solving process have received high attention. ECLiPSe1 (Cheadle et al. 2003) and OPL2 (Van Hentenryck 1999) are among software platforms supporting this hybrid approach. Interested readers are referred to Hooker (2000, 2012) for more information on the CLP models and their solution methods. In the following, we refer to some typical CLP constraints and explain how they can be expressed in IP models as linear equations or inequalities. Such transformations provide the possibility to solve the problems modeled in CLP by the special software of IP. In this regard, if there are different alternative formulations for a given concept, they can be compared concerning the quality of the corresponding LPR problems.

5.7.2

Expressing the Domain of Variables

To express that the variable y takes its value from the set {1, 0, 2, 5, 6}, we can directly insert the following constraint into a CLP model: y 2 f1, 0, 2, 5, 6g

1 2

Constraint logic programming system. Optimization programming language.

250

5 Qualitative Comparison of Models

While, in order to express such a concept in an IP model, it should be consistent with the approach outlined in Sect. 2.2.2.

5.7.3

Not-Equal Constraint (6¼)

The following constraint can be directly inserted into a CLP model: n X

aj xj 6¼ b

j¼1

While in an IP model, the use of the not-equal constraint (6¼) is not permitted, and it is necessary to rewrite the above constraint in the form of linear constraints, according to the method described in Sect. 3.6.6.

5.7.4

The At-Least Constraint

Let a be a fixed number. Suppose that at least m of the n variables y1, y2, . . ., yn must take the value of a. In a CLP model, it is enough to use the atleast constraint as follows: atleast m ðy1 , y2 , . . . , yn jaÞ While in order to observe the above requirement in an IP model, it is necessary to rewrite it in the form of linear constraints. In a particular case, whereP yj is a binary variable and a is equal to 1, it is simply guaranteed by the constraint nj¼1 yj  m.

5.7.5

The Cardinality Rule Constraint

Suppose that δi and γ i are binary variables (i ¼ 1, . . ., m, j ¼ 1, . . ., n). In a CLP model, the “cardinality rule” implies that if at least k of variables δ1, . . ., δm take the value of 1, then at least l of variables γ 1, . . ., γ n take the value of 1. This rule is expressed as the following constraint: ðδ1 , . . . , δm Þk ) ðγ 1 , . . . , γ n Þl

ð5:33Þ

If k ¼ 0, the antecedent is a tautology and the above constraint becomes equivalent to the following constraint:

5.7 Constraint Logic Programming

251

atleast l ðγ 1 , . . . , γ n j1Þ P Which can be rewritten as nj¼1 γ j  l. Thus, from here, we assume that 1  k  m and 1  l  n. To comply with the constraint (5.33) in an IP model, we propose two methods: Method 1: constraint (5.33) is equivalent to: m X

n X

δi  k )

i¼1

γj  l

ð5:34Þ

j¼1

We consider the following three cases: Case 1: k ¼ m. In this case, (5.34) is equivalent to: m X

δi ¼ m )

i¼1

n X

γj  l

j¼1

It can be rewritten as follows: n X

γj  l  l m 

j¼1

m X

! δi

i¼1

Case 2: l ¼ 1. In this case, the contrapositive of (5.34) is: n X

γj ¼ 0 )

j¼1

m X

δi  k  1

i¼1

Which is equivalent to: m X

δi  k  1 þ ðm  k þ 1Þ

i¼1

n X

γj

j¼1

Case 3: A general case, where k and l take arbitrary values. In this case, since the propositions p ) q and ~p _ q are equivalent, (5.34) can be restated as follows: m X i¼1

! δi  k  1

_

n X

! γj  l

j¼1

Now, we introduce binary variable β and add the following constraints:

252

5 Qualitative Comparison of Models m X

δi  k  1 þ ðm  k þ 1Þð1  βÞ

i¼1 n X

γ j  l  lβ

j¼1

Method 2: constraint (5.33) is equivalent to: ð δ1 , . . . , δm Þs _ ðγ 1 , . . . , γ n Þl

ð5:35Þ

where s ¼ m  k + 1 and δi can be replaced by 1  δi. Now, consider the following theorem. Theorem 5.4 Convex hull representation of constraint (5.35) Define the parameters m, n, k, l as stated in the description of constraint (5.33), and let  ¼ f1, . . . , mg,  ¼ f1, . . . , ng, and s ¼ m  k þ 1. (a) In general, the convex hull of the points (δ, γ) 2 {0, 1}m  {0, 1}n, satisfying the constraint (5.35), is described by the following constraints, where 1  s  m  1 and 1  l  n  1. 0  δi  1

8i 2 

ð5:36Þ

0  γj  1

8j 2  ð5:37Þ X X ðj0 j þ l  nÞ ð1  δi Þ þ ðj0 j þ s  mÞ γ j  ðj0 j þ s  mÞðj0 j þ l  nÞ i20

j20

80   : m  s þ 1  j0 j  m, 80   : n  l þ 1  j0 j  n ð5:38Þ (b) If s ¼ m and 1  l  n  1, then to define the convex hull, it is enough to write the inequality (5.38) only for 0   with j0 j ¼ 1. Therefore, when s ¼ m, the index domain of inequality (5.38) can be reduced as follows: ðj0 j þ l  nÞð1  δi Þ þ

X γ j  ðj0 j þ l  nÞ j20

0

ð5:39Þ 0

8i 2 , 8   : n  l þ 1  j j  n (c) If l ¼ n and 1  s  m  1, to define the convex hull, it is needed to write inequality (5.38) only for 0   withj0 j ¼ 1. Therefore, when l ¼ n, the index domain of inequality (5.38) can be reduced as follows:

5.7 Constraint Logic Programming

X

253

ð1δi Þþðj0 jþsmÞγ j  ðj0 jþsmÞ 80 :msþ1j0 j m,8j2 ð5:40Þ

0

i2

Proof See Balas (2004). □ According to Theorem 5.4, constraints (5.36–5.38) represent the convex hull of the points (δ, γ) 2 {0, 1}m  {0, 1}n, satisfying (5.33). Therefore, to express (5.33) in the form of linear constraints, using the second method leads to a better formulation. Of course, the number of constraints of (5.38) is very high. Therefore, generally, these constraints are not added to the model at once, but first, the model is solved without considering these constraints, and it is checked whether or not the obtained solution violate any of the ignored constraints; if no, that solution is considered as an optimal solution. Otherwise, one of the ignored constraints of (5.38) that is violated by the current solution is added to the model, and the new model is solved. This process is repeated until the current solution does not violate any constraint of (5.38). Example 5.20 An application of cardinality rule A company’s manager wants to decide on opening or closing three plants. He should also decide whether or not to produce three products. The following constraints should be observed: Constraint 1: If at least two plants are opened, then at least two products should be produced. Constraint 2: At most one product can be produced unless both plants 1 and 2 are opened (this is equivalent to say that if at least one of the plants 1 and 2 are not opened, then at least two products cannot be produced). Assume that binary variable δi is 1 if plant i is open; otherwise 0 (i ¼ 1, 2, 3). Also, the binary variable γ j is 1 if product j is produced; otherwise 0 ( j ¼ 1, 2, 3). In a CLP model, the above constraints can be expressed as follows: ðδ1 , δ2 , δ3 Þ2 ) ðγ 1 , γ 2 , γ 3 Þ2

ð5:41Þ

ð δ1 , δ2 Þ1 ) ð γ 1 , γ 2 , γ 3 Þ2

ð5:42Þ

Use Theorem 5.4, and rewrite the constraints (5.41) and (5.42) in the form of linear constraints. Answer We have p ¼ 2 and l ¼ 2; and according to Theorem 5.4, constraint (5.41) is guaranteed by restrictions δi, γ j 2 {0, 1} and the constraints provided in Table 5.2. (These constraints are written with respect to (5.38) for the sets 0   and 0   with 2  j0 j  3 and 2  j0 j  3). We have p ¼ m ¼ 2 and l ¼ 2; and according to part (b) of Theorem 5.4, constraint (5.42) is guaranteed by restrictions δi, γ j 2 {0, 1} and the constraints provided in Table 5.3 (These constraints are written with respect to (5.39) for the sets 0   with 2  j0 j  3).

254

5 Qualitative Comparison of Models

Table 5.2 Restatement of constraint (5.41) in linear form 0 {1, 2} {1, 3} {2, 3} {1, 2, 3} {1, 2} {1, 3} {2, 3} {1, 2, 3} {1, 2} {1, 3} {2, 3} {1, 2, 3} {1, 2} {1, 3} {2, 3} {1, 2, 3}

0

Constraints δ1  δ2 + γ 1 + γ 2   1 δ1  δ2 + γ 1 + γ 3   1 δ1  δ2 + γ 2 + γ 3   1 2(δ1 + δ2) + γ 1 + γ 2 + γ 3   2 δ1  δ3 + γ 1 + γ 2   1 δ1  δ3 + γ 1 + γ 3   1 δ1  δ3 + γ 2 + γ 3   1 2(δ1 + δ3) + γ 1 + γ 2 + γ 3   2 δ2  δ3 + γ 1 + γ 2   1 δ2  δ3 + γ 1 + γ 3   1 δ2  δ3 + γ 2 + γ 3   1 2(δ2 + δ3) + γ 1 + γ 2 + γ 3   2 δ1  δ2  δ3 + 2(γ 1 + γ 2)   1 δ1  δ2  δ3 + 2(γ 1 + γ 3)   1 δ1  δ2  δ3 + 2(γ 2 + γ 3)   1 2(δ1 + δ2 + δ3) + 2(γ 1 + γ 2 + γ 3)   2

{1, 2} {1, 2} {1, 2} {1, 2} {1, 3} {1, 3} {1, 3} {1, 3} {2, 3} {2, 3} {2, 3} {2, 3} {1, 2, 3} {1, 2, 3} {1, 2, 3} {1, 2, 3}

Table 5.3 Restatement of constraint (5.42) in linear form 0 {1, 2}

Constraints δ1  γ 1  γ 2   1 δ2  γ 1  γ 2   1 δ1  γ 1  γ 3   1 δ2  γ 1  γ 3   1 δ1  γ 2  γ 3   1 δ2  γ 2  γ 3   1 2δ1  γ 1  γ 2  γ 3   1 2δ2  γ 1  γ 2  γ 3   1

{1, 3} {2, 3} {1, 2, 3}

Example 5.21 Cardinality rule in the at-most case Suppose that δi and γ j are binary variables (i ¼ 1, . . ., m, j ¼ 1, . . ., n). We want to ensure that if at most k of variables δ1, . . ., δm take the value of 1 then, at most l of variables γ 1, . . ., γ n take the value of 1. To ensure the this requirement in IP and CLP models, provide appropriate constraints. Answer The proposition mentioned above can be restated as: m X i¼1

δi  k )

n X

γj  l

j¼1

Thus, in an IP model, it is enough to rewrite the above statement as linear constraints. However, in a CLP model, we can use the cardinality rule. The given

5.7 Constraint Logic Programming

255

proposition is equivalent to say that “if at least m  k of variables δ1, . . ., δm take 0, then at least n  l of variables γ 1, . . ., γ n must be 0”, or equivalently “if at least m  k of terms 1  δ1, . . ., 1  δm take the value of 1, then at least n  l of terms 1  γ 1, . . ., 1  γ n must be 1”. This proposition can be expressed as the following constraint: ð δ1 , . . . , δm Þmk ) ð γ 1 , . . . , γ n Þnl where δi and γ j are equivalent to 1  δi and 1  γ j, respectively.

5.7.6

All-Different Constraint

Suppose that yj be a variable taking its value from the finite set ðyj Þ, j 2  ¼ f1, . . . , ng. The all-different constraint which is defined as follows in a CLP model, implies that the variables yj ( j ¼ 1, . . ., n) take distinct values. alldiff ðy1 , y2 , . . . , yn Þ

ð5:43Þ

In order to comply with the constraint (5.43) in an IP model, we propose three methods: Method 1 Suppose that δj, j0 (for j, j0 2  with j < j0) is a binary variable that is 1 if yj < yj0 ; and 0 if yj > yj0 . Therefore, constraint (5.43) is equivalent to the following constraints: 8j, j0 2  : j < j0

yj  yj0  ε þ Mð1  δj, j0 Þ

8j, j0 2  : j < j0

yj0  yj  ε þ Mδj, j0 δj, j0 2 f0, 1g

8j, j0 2  : j < j0

yj f ree 8j 2  where ε > 0 is a sufficiently small number and M > 0 is a sufficiently large number. This formulation is far away from ideal formulation (corresponding to the convex hull) and therefore, it is not a good formulation. Method 2 Consider the set  as follows: ¼

[

ðyj Þ

j2

Suppose that βj, i is a binary variable that is 1 if yj takes the value i 2 ; otherwise 0. Therefore, constraint (5.43) is equivalent to the following constraints:

256

5 Qualitative Comparison of Models

yj ¼ X

X iβj, i

8j 2 

i2

β j, i ¼ 1

8j 2 

β j, i  1

8i 2 

i2

X j2

βj, i ¼ 0

8j 2 , 8i 2  : i= 2ðyj Þ

βj, i 2 f0, 1g

8j 2 , 8i 2 

yj f ree 8j 2  This is a totally unimodular model, and hence, the feasible region of the LPR model is equal to the convex hull of the integer points. Consequently, the formulation obtained from the second method is better than the first one. Example 5.22 Restatement of all-different constraint Linearize the following model: min z ¼ 2y1 þ 3y2 þ 4y3 s:t: y1 þ 2y2 þ 3y3  32 alldiff ðy1 , y2 , y3 Þ y1 2 f1, 4, 6g, y2 , y3 2 f5, 6g Answer Let  ¼ f1, 4, 5, 6g, and define βj, i as a binary variable that is 1 if the variable yj takes the value i 2 ; otherwise 0. The following model is equivalent to the above model. min z ¼ 2y1 þ 3y2 þ 4y3 s:t: y1 þ 2y2 þ 3y3  32 y1 ¼ β1, 1 þ 4β1, 4 þ 6β1, 6 y2 ¼ 5β2, 5 þ 6β2, 6 y3 ¼ 5β3, 5 þ 6β3, 6 β1, 1 þ β1, 4 þ β1, 6 ¼ 1

5.7 Constraint Logic Programming

257

β 2, 5 þ β 2, 6 ¼ 1 β 3, 5 þ β 3, 6 ¼ 1 β 2, 5 þ β 3, 5  1 β1, 6 þ β2, 6 þ β3, 6  1 βj, i 2 f0, 1g

8j ¼ 1, 2, 3, 8i ¼ 1, 4, 5, 6

y1 , y2 , y3  0 Method 3 Consider the following theorem: Theorem 5.5 Convex hull representation in the original y-space Define ðyj Þ ¼ fa1 , . . . , am g for every j 2  ¼ f1, . . . , ng, and assume that a1 < a2 < . . . < am and m  n. (a) The convex hull of the points (y1, y2, . . ., yn) satisfying (5.43) is described by the following constraint set: 0

j j X

ai 

X

yj 

0

i¼1

m X

ai

80  

0

i¼mj jþ1

j2

(b) If m ¼ n, then some of the above constraints are redundant and can be removed. Therefore, in this case, for the convex hull description, it is sufficient to consider the following constraints: 0

j j X

ai 

80   : j0 j < n

j20

i¼1 n X

X yj

yj ¼

j¼1

n X

ai

i¼1

Proof See Hooker (2012). □ Example 5.23 Represent the convex hull of the following model in original yspace. alldiff fy1 , y2 , y3 g y1 , y2 , y3 2 f7, 11, 13g

258

5 Qualitative Comparison of Models

Fig. 5.17 Convex hull in Example 5.23 (Van Hoeve 2001)

Answer The convex hull of the points satisfying above constraint is depicted in Fig. 5.17 and is described by the following constraints. We have m ¼ n ¼ 3 and 0 2 ff1g, f2g, f3g, f1, 2g, f1, 3g, f2, 3gg. y1 þ y2 þ y3 ¼ 31 y1 þ y2  18 y1 þ y3  18 y2 þ y3  18 y1 , y2 , y3  7 Remark 5.4 If in addition to all-different constraint, the model contains other constraints, the use of Theorem 5.5 is not enough for linearization, and some constraints are required to specify the domain of variables, as well. The all-different constraint is essential and useful in many practical applications. In the following, we present some examples. Example 5.24 CLP model for the assignment problem Suppose that  ¼ f1, . . . , 5g and  ¼ f1, . . . , 6g are the sets of persons and jobs, respectively, and let ci, j be the cost of assigning the job j to the person i. Table 5.4 shows the jobs each person is experienced to do. Each person must be assigned to exactly one job and each job should be assigned to at most one person so that the Table 5.4 Information of Example 5.24 Person Jobs compatible with the person’s skills

1 {1}

2 {2, 3, 5}

3 {1, 2, 3, 5}

4 {1, 5}

5 {1, 3, 4, 5, 6}

5.7 Constraint Logic Programming

259

total cost is minimized. By using the all-different constraint, formulate this problem as a CLP. Answer Suppose that yi represents the job assigned to person i. The domain of each variable is as follows: ðy1 Þ ¼ f1g, ðy4 Þ ¼ f1, 5g,

ðy2 Þ ¼ f2, 3, 5g, ðy5 Þ ¼ f1, 3, 4, 5, 6g

ðy3 Þ ¼ f1, 2, 3, 5g,

The CLP model is formulated as: min z ¼

n X

cyi , i

i¼1

s:t: alldiff ðy1 , . . . , y5 Þ yi 2 ðyi Þ 8i ¼ 1, . . . , 5 Example 5.25 CLP model for TSP By using the all-different constraint, formulate the TSP as a CLP. Answer Suppose that  ¼ f1, . . . , ng is the set of cities and ci, i0 is the cost of traveling from city i to city i0. Let yj be a variable representing the city visited at the jth turn (for j ¼ 1,. . .,n) and taking its value from the set . In this way, the problem is formulated as the following CLP model: min z ¼

n1 X

cyj , yjþ1 þ cyn , y1

j¼1

s:t: alldiff ðy1 , . . . , yn Þ y1 , . . . , yn 2  Example 5.26 CLP and IP models for n-queen problem In the chess game, the queen can be moved horizontally, vertically, or diagonally. For example, Fig. 5.18 shows the permitted moves when the queen is located in the square on the sixth row and the fifth column. It has been proven that if n  4, the maximum number of queens that can be placed on an n  n chess-board so that each one does not threaten the other, is equal to n. Accordingly, the n-queen problem has been introduced with the goal of placing n queens on an n  n (n  4) chess-board so that no two queens attack each other. Provide both CLP and IP models for this problem.

260

5 Qualitative Comparison of Models

Fig. 5.18 Permitted moves for the queen in the chessboard

Answer In each row, exactly one queen should be placed. Therefore, let yi be a variable taking its value from the set {1, . . ., n}, and indicating the column number at which the queen of the ith row is located. The following constraints ensure that no two queens threaten each other. 8i, i0 ¼ 1, . . . , n : i < i0

yi 6¼ yi0 yi  yi0 6¼ i  i0 0

yi  yi0 6¼ i  i

8i, i0 ¼ 1, . . . , n : i < i0 0

8i, i ¼ 1, . . . , n : i < i

yi 2 f1, 2, . . . , ng

0

ð5:44Þ ð5:45Þ ð5:46Þ

8i ¼ 1, . . . , n

Constraint set (5.44) implies that no two queens can be placed in the same column, and constraint sets (5.45) and (5.46) guarantee that no two queens threaten each other by the diagonal move. These constraints can be rewritten as follows: yi  i 6¼ yi0  i0

8i, i0 ¼ 1, . . . , n : i < i0

yi þ i 6¼ yi0 þ i0

8i, i0 ¼ 1, . . . , n : i < i0

Therefore, the n-queen problem is formulated in the form of a CLP model, without any objective function. alldiff ðy1 , . . . , yn Þ alldiff ðy1  1, y2  2, . . . , yn  nÞ alldiff ðy1 þ 1, y2 þ 2, . . . , yn þ nÞ yi 2 f1, 2, . . . , ng

8i ¼ 1, . . . , n

5.7 Constraint Logic Programming

261

To provide an IP model, we define the binary variable δi,j that is 1 if a queen is placed in the square located on the ith row and the jth column; otherwise 0 (i, j ¼ 1, . . ., n). In this way, the problem is formulated as follows: max z ¼

n X n X

δi, j

i¼1 j¼1 n X

δi, j  1

8i ¼ 1, . . . , n

ð5:47Þ

δi, j  1

8j ¼ 1, . . . , n

ð5:48Þ

j¼1 n X i¼1 n X

n X

δi, j  1

8k ¼ 3, . . . , 2n  1

ð5:49Þ

i¼1 j¼1jiþj¼k n X

n X

δi, j  1 8k ¼ 2  n, . . . , n  2

ð5:50Þ

i¼1 j¼1jij¼k

δi, j 2 f0, 1g

8i, j ¼ 1, . . . , n

Constraint sets (5.47) and (5.48) ensure that at most one queen is placed in each row and each column. Also, the diagonal move of the queen indicates that at least one queen is placed on each of the diagonal lines depicted in Fig. 5.19. This requirement is guaranteed by constraints (5.49) and (5.50). Objective function maximizes the number of queens placed on the board. However, since the optimal objective value is proved to be n, the objective function can be removed, but in this case, the constraints (5.47) and (5.48) must be rewritten in the equality form. Example 5.27 CLP and IP models for Sudoku game Consider an n2  n2 matrix divided into n  n blocks. The aim is to fill the matrix entries with numbers 1 to n2 so that in each row, in each column, and in each block, each number appears exactly once. The game begins with a partially filled matrix in which some of the entries have already been fixed to a specific value, namely ai,j. Figure 5.20 shows the initial status of a Sudoku game board (n ¼ 3). Provide both CLP and IP models for this problem (this game can be generalized for any n2  n2 matrix). Answer For the CLP model, we define the variable yi, j taking its value from the set {1, . . ., 9}, and representing the number placed in the entry of the ith row and the jth column. Therefore, the problem is formulated as follows, where constraints ensure that in each row, each column, and each block, every number appears exactly once.

262

5 Qualitative Comparison of Models

Fig. 5.19 Diagonals of the chess-board

Fig. 5.20 An instance of the Sudoku game board

  alldiff yi, 1 , . . . , yi, 9   alldiff y1, j , . . . , y9, j alldiff

8i ¼ 1, . . . , 9 8j ¼ 1, . . . , 9

yi, j , yiþ1, j , yiþ2, j , yi, jþ1 , yiþ1, jþ1 , yiþ2, jþ1 , yi, jþ2 , yiþ1, jþ2 , yiþ2, jþ2

!

5.7 Constraint Logic Programming

263

8i, j ¼ 1, 4, 7 yi, j 2 ai, j for ði, jÞ initialized with ai, j yi, j 2 f1, . . . , 9g for ði, jÞ which is not initialized To provide an IP model, we define the binary variable δi, j, k that is 1 if the entry of the ith row and the jth column, is filled by digit k; otherwise 0 (i, j, k ¼ 1, . . ., 9). Therefore, the problem is formulated as follows without any objective function, where the first three constraints guarantee that in each row, each column, and each block, every number appears exactly once. Constraint set (5.51) indicates that each entry is filed by exactly one number. 9 X

δi, j, k ¼ 1

8j, k ¼ 1, . . . , 9

δi, j, k ¼ 1

8i, k ¼ 1, . . . , 9

i¼1 9 X j¼1 2 X

δiþq, jþr, k ¼ 1

8i, j ¼ 1, 4, 7, 8k ¼ 1, . . . , 9

q, r¼0 9 X

δi, j, k ¼ 1

8i, j ¼ 1, . . . , 9

ð5:51Þ

k¼1

δi, j, k ¼ 1

if ai, j ¼ k

δi, j, k 2 f0, 1g

5.7.7

8i, j, k ¼ 1, . . . , 9

Element Constraint

The element constraint is expressed as follows: element ðk, yjða1 , . . . , am ÞÞ where k is an integer variable and ai is a constant. This constraint implies that the variable y takes the kth element of the vector (a1, . . ., am). The element constraint can be rewritten linearly as: m X i¼1

δi ¼ 1

264

5 Qualitative Comparison of Models

y¼

m X

ai δ i

i¼1

k¼

m X

iδi

i¼1

δi 2 f0, 1g

5.8

8i ¼ 1, . . . , m

Notes and References

Some examples of this chapter and several exercises of Sect. 5.9 have been motivated by Williams (2013), Klotz and Newman (2013), Wolsey (1998), and Bertsimas and Tsitsiklis (1997).

5.9

Exercises

Exercise 5.1 (a) In Example 5.6, prove that formulation 3 is better than 2 . (b) In Example 5.6, examine whether or not 3 is an ideal formulation. Exercise 5.2 In Example 5.10, show that the second formulation presented for the production planning problem is ideal. Exercise 5.3 The feasible region of an IP problem is as follows:

ðy1 , y2 Þ 2 ℤ2 : 2y1 þ 3y2  23, 5y1 þ 5y2  8, y1 , y2  0

Show that y1  4 and y1 + y2  2 are two valid cuts for the above problem and argue that which one is stronger. Exercise 5.4 (a) Suppose that a1, . . ., aK are the points belonging to the real space ℝn. Show that Convfa1 , . . . , aK g ¼ Convða1 , . . . , at , Convðatþ1 , . . . , aK ÞÞ (b) Suppose that 1 ¼ fa1 , a2 , . . . , aK g and 2 ¼ fa01 , a02 , . . . , a0K 0 g are finite subsets of the real space ℝn. Show that if 1  2 , then Convð1 Þ  Convð2 Þ.

5.9 Exercises

265

Exercise 5.5 Which of the following matrices is totally unimodular? (a) 2

3 1 0 1 0 6 1 1 0 07 7 6 7 6 6 0 1 1 0 7 7 6 6 0 1 1 1 7 7 6 7 6 4 0 0 0 15 0 0 0 1 (d) 3 2 0 1 1 0 0 6 0 0 1 1 0 7 7 6 7 6 41 1 0 1 1 5 1 0 0 0 1

(b) 2 1 61 6 6 40 1

(e) 2

1

6 1 6 6 4 0 0

1 0 1 1

0 1 1 0

(c) 3 2 1 0 0 1 7 6 0 1 1 7 6 1 7 6 6 1 1 0 1 7 7 6 7 6 4 0 1 1 0 5 0 1 1 0

3 0 17 7 7 05 1

0

1

1 1

0 0

0

3

0 7 7 7 1 5 1 1 1

(f) 2

3 0 1 1 0 7 6 1 1 1 5 4 0 1 0 0 1

Exercise 5.6 Prove Theorem 5.1. Exercise 5.7 Prove Theorem 5.2. Hint: Suppose on the contrary that A is not totally unimodular, and assume that B is the minimal square sub-matrix of A so that det (B) 2 = {1, 1, 0}. Matrix B cannot contain a column with just one non-zero element, since this contradicts the assumption that B is the minimal sub-matrix with det(B) 2 = {1, 1, 0}. Therefore, B has two non-zero entries per column. Now, show that the rows of B are linear dependent and deduce that det (B) ¼ 0, which is a contradiction. Exercise 5.8 In Sect. 5.7.6, three formulations were proposed for the all-different constraint. Show that the second formulation is totally unimodular. Exercise 5.9 Consider the directed graph G ¼ ð, Þ in which  is the set of vertices (indexed by i, j) and  is the set of arcs. Suppose that ci, j > 0 denotes the length of the arc (i, j) and the vertices O and D, respectively, represent the source and the destination vertices. The goal is to find the shortest path from the source O to the destination D. This is called the shortest path problem. (a) Let δi, j be a binary variable that is 1 if the arc (i, j) is on the shortest path; otherwise 0. Show that the shortest path problem can be formulated as follows:

266

5 Qualitative Comparison of Models

X

min z ¼

ci, j δi, j

ði, jÞ2

s:t:

X

δO , j 

j2:ðO, jÞ2

X

j2:ðj, iÞ2

δj, O ¼ 1

j2:ðj, OÞ2

δj, D 

j2:ðj, DÞ2

X

X X

δD, j ¼ 1

j2:ðD, jÞ2

δj, i ¼

X

δi , j

8i 2  : i 6¼ O, D

j2:ði, jÞ2

δi, j 2 f0, 1g 8ði, jÞ 2  (b) Show that the coefficients matrix of the shortest path problem is totally unimodular and therefore, δi, j can be considered as a continuous variable with the domain [0, 1]. (c) Let P1, P2, . . ., Pk be all paths from O to D and suppose that LPj represents the length of the path Pj, where LP1  LP2  LP3  . . .  LPk In this case, P1 is called the first shortest path, P2 is called the second shortest path, and generally, Pk, is called the k th shortest path. There are efficient algorithms to identify the k th shortest path. Nevertheless, here, we present an algorithm which is based on the iteratively resolution of an IP model and cut generation. The general framework of this algorithm is as follows: An Algorithm to Determine the kth Shortest Path • Call the model of the shortest path problem, presented in part a, as M1 and set t≔ 1. • While t  k do – Solve the model Mt and denote its optimal solution by x . – Produce a cut to eliminate the current solution x from the feasible region of Mt. Add this cut to the problem Mt and refer to the new problem as Mt + 1. – Put t ≔ t + 1. • The last value of x shows the kth shortest path. Formulate the general representation of cuts that should be added to the problem in each repetition of the above algorithm. (d) The problem to find the longest path from O to D is called the longest path problem. If the objective function of the model presented in part (a) is changed from minimization to maximization, can the resulting model be considered as a formulation for the longest path problem?

5.9 Exercises

267

Exercise 5.10 (a) Find all IISs of the following infeasible problem. Then, use Model 5.8 to determine the MIS. (b) Use Model 5.9 to determine MIS. x1  x2  0 2x2  1 x1  x2  2 x2  2 2x1  x2  4 Exercise 5.11 Provide an interpretation of the Farkas’ lemma based on the concept of primal and dual problems in the linear programming context. Exercise 5.12 Find all IISs of the infeasible system addressed in Example 5.19. Exercise 5.13 Use Model 5.9 to find the MIS of the infeasible system addressed in Example 5.19. Exercise 5.14 In the parts (a–c), the feasible regions of BIP problems are presented. In each case, compare the formulations obtained at different levels of the RLT method. (a)  ¼ fðδ1 , δ2 Þj6δ1 þ 3δ2  2, δ1 þ δ2  1, δ1 , δ2 2 f0, 1gg (b)  ¼ fðδ1 , δ2 Þj  6δ1 þ 3δ2  1, δ1  δ2 , δ1 , δ2 2 f0, 1gg (c)  ¼ fðδ1 , δ2 Þjδ1 þ δ2 þ δ3 þ δ4 ¼ 2, δ1 , δ2 , δ3 , δ4 2 f0, 1gg Exercise 5.15 In the chess game, the Rook can move horizontally or vertically. The Bishop moves diagonally. The Knight moves from a square to another one that is two squares away horizontally and one square away vertically, or two squares away vertically and one square away horizontally so that the complete move looks like the letter “L”. In each of the following parts, write an IP model to determine the position of pieces on the chess-board. (a) What is the maximum number of Rooks that can be placed on an 8  8 chessboard so that no one is threaten by the others? (b) What is the maximum number of Bishops that can be placed on an 8  8 chessboard so that no one is threaten by the others? (c) What is the maximum number of Knights that can be placed on an 8  8 chessboard so that no one is threaten by the others?

268

5 Qualitative Comparison of Models

Exercise 5.16 In each case, provide an IP model. (a) What is the minimum number of Queens that can be placed on chess-board so that each square either includes a Queen or is threatened by at least one Queen? (b) How to place n Queens on an n  n chess-board, so that the number of squares which are not under the attack of any Queen is maximized. Exercise 5.17 (a) Let G ¼ ð, Þ be a directed graph; then, G0 ¼ ð0 , 0 Þ is called a subgraph of G if 0   and 0  . A subgraph G0 ¼ ð0 , 0 Þ is called a maximal clique if it has the maximum number of vertices, and any pair of its vertices are connected by an arc. In other words, a maximal clique is a complete subgraph with the maximum number of vertices. Provide an IP model to find the maximal clique of the graph G. (b) The graph G ¼ ð, Þ is given. The aim of the graph coloring problem is to assign colors to the vertices of G so that for eachði, jÞ 2 , the colors of the vertices i and j are not the same. Provide a CLP model (using all-different constraint) and an IP model for this problem.

Chapter 6

Applications of Mathematical Modeling

6.1

Introduction

In this chapter, a set of real-life applications of mathematical programming which are closely related to the methodologies presented in the previous chapters are addressed. For each application, the problem is defined, the model’s components are introduced and then, the model is presented in the general form. Afterward, it is solved on a given set of data with the aid of computer, and the results are analyzed. Although the problems, addressed in this chapter, are described and formulated under certain assumptions, evidently, they are not merely limited to these assumptions and can be extended and generalized from various aspects. Of course, due to the widespread applicability of such problems, the range of assumptions is very extensive, and it is not possible to examine all of them in this book, but an attempt is made to highlight some of the main aspects of the model’s development in the remarks presented within the chapter, or in the exercises provided at the end of the chapter. There is no particular inter-dependency among the practical problems of this chapter; some problems are simple to model and solve; however, others are more complicated. The reader can order his sequence of study in different ways. The key purpose of presenting the model and optimal solution for each problem is to provide a chance for the reader to compare his results with the given solutions. Therefore, it is better for the reader to first formulate and solve the problem for himself before referring to the model proposed in the book, and then, compare the outcomes with the solution presented here. Some problems may have alternative optimal solutions, but their optimal objective values are always the same. If the reader receives a different solution, he can try for validation. In order to solve the models, it is necessary to use optimization software. This software, in addition to the model solving, is helpful in model building and let us focus on the structure of the model and identify possible errors. In this book, the AIMMS optimization software (Bisschop 2012) is used to build and solve the models. © Springer Nature Switzerland AG 2019 S. A. MirHassani, F. Hooshmand, Methods and Models in Mathematical Programming, https://doi.org/10.1007/978-3-030-27045-2_6

269

270

6.2

6 Applications of Mathematical Modeling

Course Timetabling Problem

Today, in educational centers, the system of planning classrooms, courses, laboratories and exams is of great importance in improving the organization’s performance and efficiently using the available resources. Until recently, the process of developing such programs in educational institutions relied on the experience and accuracy of human resources, and rarely there were mechanized facilities. However, with the expansion of institutions and the increase in the number of students and disciplines, manually design of plans becomes a boring and time-consuming process. Mathematical models provide a possibility to do the planning process faster, more efficiently, in a large extent, and within a shorter amount of time, while preventing the occurrence of human mistakes due to the complexity and timing of the process. Furthermore, it provides an appropriate tool for planners to implement official rules and manage rare resources. Additionally, using an appropriate internet platform facilitates the interaction between teachers, students, and the educational institution. In this section, a precise description of the course timetabling problem (CTP) is presented. Then, it is formulated as a multi-objective model with a set of hard and soft constraints.

6.2.1

Description of the CTP

CTP provides a weekly timetable to assign courses to different timeslots and determine a schedule that respects the educational regulations and is feasible regarding the preferences of the institution, teachers, and students. The general needs of a weekly educational program are as follows: 1. The sets of classes, days, and timeslots are known, and it is assumed that the timeslots have the same length (e.g., 2 h) without any overlap. 2. Courses must be scheduled in allowed days and timeslots. 3. The number of weekly sessions required by each course is known in advance, and the number of timeslots will be allocated to a course must be equal to this number. 4. For each discipline, there is a chart describing the prerequisites of each course, and the semester that each course is typically offered. According to this chart, it can be estimated that in a given semester, which courses are simultaneously taken by a given group of students (e.g., the group of students who completed their third semester) and accordingly, the program can be designed so that such courses do not conflict at all. Therefore, in general, we assume that based on the experiences of the educational experts, a few groups of courses are identified, and the weekly program should be set up so that the courses of any given group have no conflict with each other. 5. The number of available classes in the institution is known, and it is assumed that all classes are the same in terms of capacity and educational facilities. 6. The timeslots assigned to an instructor must not conflict with each other.

6.2 Course Timetabling Problem

271

7. At a given timeslot, a classroom cannot be assigned to more than one course. 8. The number of courses scheduled for a given timeslot, cannot exceed the total number of available classes. 9. Some instructors are not available at some timeslots, for example, because of attending weekly meetings. Hence, instructors are asked to specify the timeslots at which they are ready for teaching. The above items represent the essential requirements of the educational institution (hard constraints), and it is necessary for a mathematical model to obey all of them. However, the institution also has preferential conditions (soft constraints) that should be considered appropriately. The preferences of the institution are as follows: 1. Due to the lack of students performance at the first timeslot in the afternoon, instructors prefer to have no course at this time, if possible. 2. The institution prefers that for courses having more than one session within a week, the sessions are not scheduled on the same day or consecutive days and there exists a gap of at least 1 day between them. 3. The institution prefers that the courses with more than one weekly session are offered on different days but at the same timeslot. For example, if a course needs two sessions per week, it is better to assign them to the same timeslots of different days. Remark 6.1 Depending on the needs of educational institutions, other assumptions, such as the ones described below, may also be made. See exercises at the end of the chapter to incorporate such assumptions in the model. 1. The length of the timeslots is not necessarily the same. For example, a four-unit course is generally offered in two 2-h sessions, and a three-unit course needs two sessions of 1.5 h. Also, some timeslots may have intersection. For example, the timeslots 07:45–09:15 and 08:00–10:00 are overlapping, and hence, within a given day, an instructor can provide a course in at most one of these two timeslots. 2. The time of some courses is fixed in advance, and they should be allocated to particular timeslots of given days. 3. Some courses require more than one instructor. 4. Some courses require special educational facilities (such as video projector, microphone, smart board, and so forth) and must be delivered in a classroom that equipped with required facilities. 5. The weekly program should be as compact as possible for students. For example, they do not prefer a long free time between two consecutive courses of a given day. Similarly, the teacher’s weekly program cannot be too scattered. 6. Students should have their consecutive courses in the same building to prevent the congestion caused by unnecessary movements from one building to another. 7. Some courses must be delivered in smaller subgroups (e.g., labs, seminars, and so forth), and the students should evenly be distributed in these subgroups. 8. Classrooms should be filled as much as possible while satisfying their capacity.

272

6.2.2

6 Applications of Mathematical Modeling

Model of the CTP

Sets, Indices, and Parameters ℂ: Set of courses (indexed by c). : Set of instructors (indexed by l). : Set of working days (indexed by d, d0). D: A member of  indicating the last working day of the week. H: Set of available timeslots (indexed by h, h0). h: A member of H indicating the first timeslot in the afternoon. j : A subset of ℂ including incompatible courses ( j 2  ¼ f1, . . . , mg). nc: Number of weekly sessions required by the course c. K: Number of classrooms in the institution ac,l: A binary parameter that is 1 if the course c is assigned to instructor l; otherwise 0. bl,d,h: A binary parameter that is 1 if the instructor l is available at timeslot h of day d; otherwise 0.

Decision Variables δc,d,h: A binary variable that is 1 if course c is offered at timeslot h of day d; otherwise 0 (c 2 ℂ, d 2 , h 2 H)

Objective Function This problem can be considered without any objective function. In this case, there is no particular preference, and we are merely looking for a suitable program in which all of the hard and soft constraints are observed. Thus, when formulating this problem, the objective function can be defined artificially as follows: min z ¼ 0 In practice, however, it is not possible to comply with all soft constraints. Hence, the soft constraints should be allowed to be violated and the level of violations should be minimized in the objective function. In this way, we face with a multi-objective model in which the objectives in the order of the priority are as follows: 1. Minimizing the violation of the first soft constraint (i.e., minimizing the number of sessions that are scheduled to be held in the first timeslot of the afternoon). 2. Minimizing the violations of the second soft constraint (i.e., minimizing the number of sessions of a course scheduled on two consecutive days).

6.2 Course Timetabling Problem

273

3. Minimizing the violation of the third soft constraint (i.e., minimizing the number of courses that their sessions are not assigned to the same timeslots).

Constraints XX

δc, d, h ¼ nc

8c 2 ℂ

ð6:1Þ

d2 h2H

X

δc, d, h  K

8d 2 , h 2 H

ð6:2Þ

c2ℂ

X

δc, d, h  1

8d 2 , h 2 , l 2 

ð6:3Þ

c2: ac, l ¼1

X δ c, d , h  1

8d 2 , h 2 H, j 2 

ð6:4Þ

c2j

δc, d, h  bl, d, h X δ c, d , h  1

8c 2 ℂ, l 2 : ac, l ¼ 1, 8d 2 , h 2 H

ð6:5Þ

8c 2 ℂ: nc > 1, 8d 2 

ð6:6Þ

h2H

XX δ c, d , h  0 c2ℂ d2

X h2

δc, d, h þ

X

X

ð6:7Þ

δc, dþ1, h  1 8c 2  : nc > 1, 8d 2  : d < D

ð6:8Þ

h2

X

d0 2: d 0 6¼d h0 2: h0 6¼h

δc, d, h 2 f0, 1g

δc, d0 , h0  Mð1  δc, d, h Þ8c 2  : nc > 1, 8d 2 , h 2  ð6:9Þ 8c 2 ℂ, d 2 , h 2 H

ð6:10Þ

Constraint sets (6.1)–(6.6) are hard constraints. Constraint set (6.1) ensures that the number of timeslots assigned to a course is equal to the number of required weekly sessions. Constraint set (6.2) requires that the number of courses presented in a timeslot of a given day is not greater than the number of available classrooms. Constraint set (6.3) confirms that an instructor has at most one course at a timeslot of a given day. Constraint set (6.4) ensures that the courses of the set j have no conflict with each other. Constraint set (6.5) ensures that the courses of each instructor are scheduled at the times he/she is available. Constraint set (6.6) indicates that different sessions of a course should not be held on a single day. Soft constraints are expressed by (6.7)–(6.9). Constraint (6.7) requires the first timeslot of the afternoon is left free. Constraint set (6.8) makes it impossible for different sessions of a course to be held in two consecutive days. Constraint set (6.9) ensures that the courses requiring more than one session per week are presented on different days but at similar timeslots. It should be noted that inequality (6.9) is

274

6 Applications of Mathematical Modeling

equivalent to the following statement, where M > 0 is a sufficiently large number and can be set equal to jj  jHj. X

δ c, d , h ¼ 1 )

X

! δc, d0 , h0 ¼ 0 8c 2  : nc > 1, 8d 2 , h 2 

d0 2: d0 6¼d h0 2: h0 6¼h

Thus, the formulation of CTP with no objective is as follows: Model 6.1 Formulation of the CTP (No objective) min z ¼ 0 s.t. (6.1)–(6.10) As already mentioned, it may be impossible to observe all soft constraints, and therefore, Model 6.1 would be infeasible. Hence, by introducing slack variables, we can provide the possibility of violating soft constraints. In order to provide the possibility of violating the soft constraint (6.7), we consider the nonnegative continuous variable w as a slack variable and rewrite (6.7) as follows: XX

δc, d, h  w  0

ð6:11Þ

c2ℂ d2

Similarly, for the soft constraint (6.8), we define the nonnegative continuous variable vc, d (8c 2 ℂ: nc > 1, 8d 2  : d < D) as a slack variable and rewrite the constraint set (6.8) as follows: X h2

δc, d, h þ

X δc, dþ1, h  vc, d  18c 2  : nc > 1, 8d 2  : d < D

ð6:12Þ

h2

Also, associated with soft constraint (6.9), we define the binary variable γ c (8c 2 ℂ : nc > 1) that is 1 if the timeslots of at least one of the sessions of course c is different from that of other sessions; otherwise 0. Thus, the constraint set (6.9) is rewritten as follows: X

X

δc, d0 , h0  ðnc  1Þγ c  Mð1  δc, d, h Þ

d 0 2: d0 6¼d h0 2: h0 6¼h

8c 2  : nc > 1, 8d 2 , h 2 

ð6:13Þ

6.2 Course Timetabling Problem

275

Concerning the priorities given to the objectives, the problem is formulated as the following three-objective optimization model: Model 6.2 Formulation of the CTP (Multi-objective) X X X vc, d , z 3 ¼ min z1 ¼ w, z2 ¼ c2: nc >1 d2: d1

s.t. (6.1)–(6.6), (6.10)–(6.13) w0 vc, d  0 8c 2 ℂ : nc > 1, 8d 2  : d < D γ c 2 f0, 1g 8c 2 ℂ : nc > 1

ð6:14Þ ð6:15Þ ð6:16Þ

The first objective function, z1, indicates the number of sessions offered at the first timeslot of the afternoon. The second objective function, z2, counts the number of times that the sessions of a course are scheduled in two consecutive days, and finally, the third objective function, z3, represents the number of courses that their sessions are not scheduled in the same timeslots.

6.2.3

Solving the CTP

Problem Data Suppose that the number of instructors in the educational institution is 6 (i.e.,  ¼ f1, . . . , 6g), the number of classrooms equals 2 (i.e., K ¼ 2), and 18 courses must be scheduled for the next semester (i.e., ℂ ¼ {1, . . ., 18}). The set of working days is  ¼ f1, . . . , 5g, where d ¼ 1 and d ¼ 5 represent Monday and Friday, respectively. Also, the set of timeslots for each working day is H ¼ {1, 2, 3, 4}, the members of which refer to timeslots 8–10, 10–12, 13–15, and 15–17, respectively. Table 6.1 shows the number of sessions required for each course during the week. The courses of the following sets should not conflict with each other.

Table 6.1 Sessions required for each course c nc

1 1

2 1

3 1

4 1

5 1

6 1

7 1

8 1

9 2

10 2

11 2

12 2

13 2

14 2

15 2

16 2

17 2

18 3

276

6 Applications of Mathematical Modeling

Table 6.2 Courses assigned to instructors l 1

c 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

 

 

2

  

3

18

   

4

 

5

    

6

1 ¼ f1, 2, 8, 9, 18g, 2 ¼ f2, 3, 10, 11, 18g, 3 ¼ f3, 4, 5, 12, 13, 14g, 4 ¼ f7, 8, 16, 17g, 5 ¼ f5, 12, 13, 14, 15, 17g Tables 6.2 and 6.3 present the value of parameters ac,l and bl,d,h, respectively.

Model Size The first model which lacks the objective function (Model 6.1), contains 360 binary variables and 938 constraints (673 hard constraints and 265 soft constraints), and the density of the coefficient matrix is 1.62%.

Problem Solution Model 6.1 is infeasible. Therefore, to determine an appropriate solution, we solve the multi-objective model (Model 6.2). First, we focus on the first objective, add the constraint (6.11) to the set of hard constraints (i.e., (6.1)–(6.6)) and solve the Model 6.3 with the objective function min z1 ¼ w. The number of binary and continuous variables of Model 6.3 are equal to 360, and 1, respectively, and the number of constraints equals 675. Model 6.3 Formulation associated with the first objective min z1 ¼ w s.t. (6.1)–(6.6), (6.10),(6.11), and (6.14) The optimal objective value of Model 6.3 is z1 ¼ 3. Now, consider the following constraint fixing the value of the first objective at z1 .

6

5

4

3

2

l 1



   



   

h Monday 1 2

  

3

 

4

   



   



Tuesday 1 2

Table 6.3 Instructor’s availability time







3





4



    

   

Wednesday 1 2

  

3

 

4

   

    



Thursday 1 2





3



4

   

   

Friday 1 2

  

3

 

4

6.2 Course Timetabling Problem 277

278

6 Applications of Mathematical Modeling

w ¼ z1

ð6:17Þ

To examine the second objective, we constitute Model 6.4 by adding the constraints (6.12) P to the constraints of the Model 6.3, and considering P and (6.17) min z2 ¼ c2: nc >1 d2: d1 d2: d1 d2: d1 γ c as the objective function. This model has 371 binary variables, 45 continuous variables and 941 constraints. Model 6.5 Formulation associated with the third objective X γc min z3 ¼ c2: nc >1

s.t. (6.1)–(6.6), (6.10)–(6.18) The optimal solution to Model 6.5 is a Pareto solution to the multi-objective problem. By solving the Model 6.5, the optimal objective function value is equal to z3 ¼ 1 and the Pareto solution is given as reported in Table 6.4.

Solution Analysis As seen in Table 6.4, all hard constraints are observed. For the first soft constraint, only sessions of courses 7 and 8, which are demonstrated by , are scheduled in the timeslot 13–15. Also, regarding the second soft constraint, except for the first and the second sessions of the course 18, which are presented in two consecutive days (indicated by ), between the sessions of other courses and between the second and the third sessions of course 18, there is a gap of at least 1 day. Also, for the third

9

8

7

6

5

4

3

2

1

15

14

13

12

11

10

c







h Monday 1 2





3

4

 



Tuesday 1 2





3

Table 6.4 Weekly timetable from multi-objective model





4







Wednesday 1 2 3





4









Thursday 1 2 3



4









Friday 1 2



4

(continued)



3

6.2 Course Timetabling Problem 279

18

17

c 16



h Monday 1 2

3

Table 6.4 (continued)



4





Tuesday 1 2 3



4





Wednesday 1 2 3 4



Thursday 1 2 3

4

Friday 1 2 3



4

280 6 Applications of Mathematical Modeling

6.3 Portfolio Selection Problem

281

soft constraint, except for course 14 (indicated by ▪), the timeslots of others are appropriately determined. Note that if Model 6.3 has a unique optimal solution, Model 6.4 cannot give anything new and its resolution is practically useless. Similarly, solving Model 6.5 becomes important when Model 6.4 has alternative optimal solutions. Remark 6.2 It is worth mentioning that in the educational institution, in addition to the weekly course timetable, the preparation of the examination timetable is also a common practice. Most of the educational institutions should schedule a set of exams at the end of each semester or academic year. In a simple form, this problem can be considered as an allocation problem to assign a set of exams to a fixed number of timeslots, so that no student is having more than one exam at a given timeslot. Of course, a series of other constraints and goals should also be respected, depending on the rules and conditions of each educational institution. There are many differences between the course schedule and the exam schedule, which we mention two of them in the following: • In the exam timetable, exams of several courses are allowed to be taken simultaneously in the same classroom; while this is not the case in the weekly course timetable where at most one course can be delivered at any given timeslot of a classroom. • In the weekly course timetable, the compactness of student’s program is an advantage, while this is not preferred in examination timetables, where more gap between consecutive exams is desirable for students.

6.3

Portfolio Selection Problem

In the stock exchange, one of the main concerns of investors is that how to determine an optimal portfolio. A portfolio is a group of financial assets such as stocks of different companies, bonds, commodities, currencies and cash equivalents that are held by investors. Portfolio selection problem (PSP) addresses the selection of an appropriate portfolio as a combination of stocks with a relatively good average return and without a high value of risk. In this regard, extensive research has been carried out, and Markowitz is one of the pioneers in this field, who succeeded in receiving the Nobel Prize (Markowitz 1952) in economics. Markowitz model which uses the variance as an indicator for risk estimation can be expressed in two ways: in the first case, the goal is to maximize the average return, and there is a constraint ensuring that the investment risk does not exceed a predetermined threshold. In the second case, the goal is to minimize the risk, and there is a constraint ensuring that the average return is not lower than a predefined value.

282

6 Applications of Mathematical Modeling

So far, various optimization models have been presented for choosing the best portfolio, and some of them that use nonlinear functions to estimate the risk are difficult to solve. In this section, we introduce a nonlinear model for PSP and then provide a solution method based on the approximation of nonlinear terms with PLFs. The contents of this section are motivated by Bradley et al. (1977).

6.3.1

Description of the PSP

An investor would like to decide on buying the shares of several companies. The average return of each company per unit of investment is estimated by examining the historical records. Also, the risk of investment in terms of the variance of returns is estimated as a quadratic function of the amount of investment. The goal is to maximize the average return and minimize the risk.

6.3.2

Model of the PSP

Sets, Indices, and Parameters  ¼ f1, . . . , ng: Set of companies (indexed by j) b: Available capital for investment rj: Average return of the jth company (in percent) g(x): The quadratic nonlinear risk function ρ: A nonnegative constant expressing the importance of risk versus return. This parameter can be interpreted as the cost of tolerating one unit of risk.

Decision Variables xj: A nonnegative continuous variable representing the amount of investment in the jth company ( j 2 )

Objective Function

max z ¼

1 X  r x  ρ gðx1 , . . . , xn Þ 100 j2 j j

The objective function makes a tradeoff between the return maximization and the risk minimization. It is clear that if we put ρ ¼ 0, we get a linear model in which the

6.3 Portfolio Selection Problem

283

capital is completely invested so that the highest return is achieved, regardless of the risk. On the other hand, taking a large value for ρ is equivalent to ignore the goal of return maximization and only focus on the risk minimization.

Constraints X

xj  b

ð6:19Þ

j2

xj  0

8j 2 

ð6:20Þ

Constraint (6.19) ensures that the amount of investment in companies is at most as high as the initial capital. The restriction (6.20) describes the type of variables. Therefore, the problem formulation is as follows: Model 6.6 Formulation of the PSP 1 X  r x  ρ gð x 1 , . . . , x n Þ max z ¼ 100 j2 j j s.t. (6.19), (6.20) In Model 6.6, we can approximate the nonlinear function g(x1, . . ., xn) with a PLF, and then solve the approximate model.

6.3.3

Solving the PSP

Problem Data In order to graphically demonstrate the results, we focus on a small sized instance of the problem. Assume that the investor has a capital of 5 (mu), and he decides to buy the shares of two companies (b ¼ 5,  ¼ f1, 2g). By examining the records of companies, he concludes that the average returns per unit of investment in companies 1 and 2 are 20% and 16%, respectively (i.e., r1 ¼ 20, r2 ¼ 16). Also, the investments risk is estimated as: gð x 1 , x 2 Þ ¼

  1 2x21 þ x22 þ ðx1 þ x2 Þ2 100

The investor deals with the following NLP model to make a tradeoff between the return maximization and the risk minimization:

284

6 Applications of Mathematical Modeling

Fig. 6.1 Optimal solution to Model 6.7 for ρ ¼ 45 (Bradley et al. 1977)

Model 6.7 Formulation of the PSP for the given data    1 max z ¼ 0:2x1 þ 0:16x2  ρ 2x21 þ x22 þ ðx1 þ x2 Þ2 100 s:t: x1 þ x2  5 x1 , x2  0

Model Size There are two continuous variables and one constraint, and the density of the coefficient matrix is 100%.

Problem Solution As previously stated, ρ is a nonnegative constant and can be interpreted as the cost of accepting one unit of risk. To get insight to the problem solution, we use a graphical method to determine the optimal solution for ρ ¼ 0.8. In Fig. 6.1, the feasible region is highlighted and centralized ellipsoids represents the contour line corresponding to different values of   the objective function. The figure shows that the optimal solution occurs at x1 , x2 ¼ ð2:5, 2:5Þ, and the value of the objective function at this point is equal to 0.55. Similarly, it can be shown that for ρ ¼ 1, the optimal solution and the optimal objective value are equal to (x1 ¼ 73, x2 ¼ 83) and 139 300, respectively. In what follows, we calculate an approximate solution to Model 6.7 for ρ ¼ 1, and compare it with the exact solution. Model 6.7 for ρ ¼ 1 is as follows:

6.3 Portfolio Selection Problem

285

Model 6.8 Formulation of the PSP for ρ ¼ 1   1 max z ¼ 0:2x1 þ 0:16x2  2x21 þ x22 þ ðx1 þ x2 Þ2 100 s:t: x1 þ x2  5 x1 , x2  0

ð6:21Þ ð6:22Þ

For ease of representation, we multiply the objective function of the above model by 100 to get the following equivalent model: Model 6.9 An equivalent restatement of Model 6.8 max z0 ¼ 20x1 þ 16x2  2x21  x22  ðx1 þ x2 Þ2 s.t. (6.21), (6.22) The constraints of Model 6.9 are linear and therefore, separable. However, the objective function is not separable due to the expression (x1 + x2)2. First, we rewrite the model in a separable form and then, by using the PLFs, provide a linear approximation of the model. To this end, we let x3 ¼ x1 + x2 and rewrite the Model 6.9 as the following separable model: Model 6.10 Restatement of Model 6.9 in a separable form max z ¼ 20x1 þ 16x2  2x21  x22  x23 s.t. (6.21), (6.22) x1 þ x2  x3 ¼ 0 The objective function of f(x) ¼ f1(x1) + f2(x2) + f3(x3) where

Model

6.10

can

ð6:23Þ be

considered

as

f 1 ðx1 Þ ¼ 20x1  2x21 , f 2 ðx2 Þ ¼ 16x2  x22 , f 3 ðx3 Þ ¼ x23 Constraints of the problem indicate that 0  x1, x2, x3  5. Therefore, we approximate each of the above functions on the interval [0, 5] with a PLF. For this purpose, we use four breakpoints {0, 1, 3, 5} to approximate f1(x1), three breakpoints {0, 3, 5} to approximate f2(x2) and three breakpoints {0, 2, 5} to approximate f3(x3). See Fig. 6.2 in which piecewise linear approximation of each function is demonstrated by dotted lines. Let f^1 ðx1 Þ, f^2 ðx2 Þ, and f^3 ðx3 Þ be PLFs associated with f1(x1), f2(x2), and f3(x3), respectively. In this way, the model below is an approximation of Model 6.10:

286

6 Applications of Mathematical Modeling

Fig. 6.2 Approximating separable functions of PSP with PLFs Bradley et al. 1977)

Model 6.11 An approximation of Model 6.10 max z ¼ f^1 ðx1 Þ þ f^2 ðx2 Þ þ f^3 ðx3 Þ s.t. (6.21), (6.22) To restate the PLFs in linear forms, the nonnegative continuous variables λi, λ0i and λ00i are defined associated with the breakpoints selected in the domain of functions f1(x1), f2(x2), and f3(x3), respectively. Then, the following substitution is applied in Model 6.11 x1 ¼ 0λ0 þ λ1 þ 3λ2 þ 5λ3 x2 ¼ 0λ00 þ 3λ01 þ 5λ02 x3 ¼ 0λ000 þ 2λ001 þ 5λ002 f^1 ðx1 Þ ¼ 0λ0 þ 18λ1 þ 42λ2 þ 50λ3 f^2 ðx2 Þ ¼ 0λ00 þ 39λ01 þ 55λ02 f^3 ðx3 Þ ¼ 0λ000  4λ001  25λ002

6.3 Portfolio Selection Problem

287

In Model 6.11, constraints are linear, and f^1 ðx1 Þ, f^2 ðx2 Þ, and f^3 ðx3 Þ are concave functions that appear with the positive coefficient in the objective function of a maximization problem. Therefore, the adjacency condition is observed and Model 6.11 can be rewritten in the following form: Model 6.12 Linear representation of Model 6.11 max z ¼ 0λ0 þ 18λ1 þ 42λ2 þ 50λ3 þ 0λ00 þ 39λ01 þ 55λ02 þ 0λ000  4λ001  25λ002 s.t. 0λ0 þ λ1 þ 3λ2 þ 5λ3 þ 0λ00 þ 3λ01 þ 5λ02  5 0λ0 þ λ1 þ 3λ2 þ 5λ3 þ 0λ00 þ 3λ01 þ 5λ02  0λ000  2λ001  5λ002 ¼ 0 λ0 þ λ1 þ λ2 þ λ3 ¼ 1 λ00 þ λ01 þ λ02 ¼ 1 λ000 þ λ001 þ λ002 ¼ 1 λ0 , λ1 , λ2 , λ3 , λ00 , λ01 , λ02 , λ000 , λ001 , λ002  0 The number of continuous variables and constraints of Model 6.12 are equal to 10 and 5, respectively, and the density of the coefficient matrix is 45.5%. The optimal objective value is 44, and the optimal solution is as follows: 1     λ1  ¼ λ2  ¼ , λ0  ¼ λ3  ¼ 0, λ01 ¼ 1, λ00 ¼ λ02 ¼ 0, λ002 ¼ 1, 2 



λ000 ¼ λ001 ¼ 0 The above solution corresponds to the following solution to Model 6.10: x1 ¼ 2, x2 ¼ 3, x3 ¼ 5, z0 ¼ 46 Which corresponds to the following solution to Model 6.8. x1 ¼ 2, x2 ¼ 3, z ¼ 0:46

288

6 Applications of Mathematical Modeling

Solution Analysis The approximate solution to the Model 6.8 is x1 ¼ 2, x2 ¼ 3 with the objective function value of 0.46, and indicates that the amount of investment in companies 1 and 2 is equal to 2 and 3 (mu), respectively, and hence, all initial capital is used in investment. The average return for this solution is equal to 0.88 (mu), which is equivalent to 17.6% (i.e., 0:88 5  100 ¼ 17:6). Also, the risk level is equal to g(2, 3) ¼ 0.42. However, the optimal solution to Model 6.8 is 7 8 139 x1 ¼ , x2 ¼ , z ¼ 3 3 300 indicating that all initial capital is used and the amount of investment in companies 1 and 2 is 73 ¼ 2:33 and 83 ¼ 2:67, respectively. The average yield for this solution is 67 equal to 67 75 (mu), which 7 is8equivalent to 17.9% (i.e., 755  100 ¼ 17:9). Also, the risk level is equal to g 3 , 3 ¼ 0:43. Clearly, the greater the number of breakpoints, the better the approximation we get. Special methods of QP can also solve this problem.

6.4

Traveling Salesman Problem

The traveling salesman problem (TSP) over a given set of cities is defined as seeking a route with the minimum length so that the route starts at the origin city, visits all other cities exactly once and then returns to the origin. The history of TSP dates back to the eighteenth century and its general form, as one of the most used optimization problems, was studied by mathematicians. This problem is also used as a base for many optimization methods. Although this well-known problem is computationally NP-hard, i.e., there is no exact algorithm to solve it in polynomial time, today, many efficient exact algorithms have been presented which can solve instances with many cities. Even, larger instances with tens of thousands cities are also solvable with appropriate approximate algorithms and heuristics. The TSP and its variants have various real-world applications in different areas including transportation and logistics, electronic industry, genetics, astronomy, and so forth. For example, the routing of trucks for parcel pickup and delivery, DNA sequencing, and microchips manufacturing are among the interesting direct applications of this problem. A formulation of TSP was presented in Example 2.2. However, there are different alternative formulations for this problem which are summarized in the review paper addressed by Öncan et al. (2009). In this section, two other formulations of TSP are presented. We refer to the formulation presented in Example 2.2 as the “First Formulation”, and refer to the formulations addressed in this section as “the second and the third formulations”.

6.4 Traveling Salesman Problem

289

The three formulations are compared in terms of the quality of their LPR and the number of variables and constraints.

6.4.1

Description of the TSP

Suppose that  ¼ f1, . . . , mg is a set of cities. A salesman intends to start his trip from city 1, visits each city exactly once and returns to city 1 again. The distance between every pair of cities is known, and the aim is to determine the order of visiting cities so that the total traveled distance is minimized.

6.4.2

Model of the TSP

Sets, Indices, and Parameters  ¼ f1, . . . , mg: Set of cities (indexed by i, j) ci, j: Distance between the cities i, j

The Second Formulation of the TSP Decisions Variables δi, j: A binary variable that is 1 if the salesman travels from city i to city j; otherwise 0 (i, j 2  : i 6¼ j). xi: A nonnegative continuous variable representing the order of visiting city i in the tour (i 2  : i 6¼ 1).

Objective Function

min z ¼

X

ci, j δi, j

i, j2: i6¼j

The objective function minimizes the total distance traveled. X δi, j ¼ 1 j2

8i 2 

ð6:24Þ

290

6 Applications of Mathematical Modeling

Fig. 6.3 Subtour and complete tour in the TSP

X

δi, j ¼ 1

8j 2 

ð6:25Þ

i2

xi  xj þ nδi, j  n  1 δi, j 2 f0, 1g

8i, j 2  : i 6¼ j, i 6¼ 1, j 6¼ 1

8i, j 2  : i 6¼ j

xi  0 8i 2  : i 6¼ 1

ð6:26Þ ð6:27Þ ð6:28Þ

Constraint sets (6.24) and (6.25) ensure that each city is visited exactly once. Constraint set (6.26) prevents the formation of any subtour (i.e., or incomplete tours). See Exercise 6.6 for more details. Thus, the second formulation of the TSP is as follows: Model 6.13 The second formulation of the TSP X min z ¼ ci, j δi, j i, j2: i6¼j s.t. (6.24)–(6.28)

The Third Formulation of the TSP In this formulation, the variable δi, j has the same definition as before, and the difference between the second and the third formulations is in the method of subtour elimination. To see how a subtour is eliminated in the third formulation, consider an example with five cities. The graph depicted in Fig. 6.3a corresponds to the following solution: δ1, 2 ¼ δ2, 1 ¼ δ3, 4 ¼ δ4, 5 ¼ δ5, 3 ¼ 1 This solution satisfies constraint sets (6.24) and (6.25), but it is not a complete tour. On the other hand, the graph depicted in Fig. 6.3b corresponds to the following solution: δ1, 2 ¼ δ2, 3 ¼ δ3, 4 ¼ δ4, 5 ¼ δ5, 1 ¼ 1

6.4 Traveling Salesman Problem

291

Which is a complete tour satisfying constraint sets (6.24) and (6.25). As can be seen, the graph depicted in Fig. 6.3b is connected (i.e., there exists a path from city 1 to all other cities); however, this is not the case for the graph depicted in Fig. 6.3a. This simple example illustrates that to prohibit the formation of any subtour, the variable δi, j should take its value so that the graph whose vertex set is the set of cities and its arc set contains the pairs (i, j) for which δi, j ¼ 1, is a connected graph (i.e., it involves a path from city 1 to other cities). By this introduction, we are ready to describe the third formulation.

Decision Variables δi, j: A binary variable that is 1 if the salesman travels from city i to city j; otherwise 0 (i, j 2  : i 6¼ j). γ i,i0 , j0 : A binary variable that is 1 if an arc (i0, j0) is on the path identified from city 1 to city i; otherwise 0 (i, i0 , j0 2  : i0 6¼ j0 , i0 6¼ i, i 6¼ 1).

Objective Function The objective function of the third formulation is the same as that of the second formulation and minimizes the distance traveled.

Constraints In addition to constraint sets (6.24), (6.25), and (6.27), the following constraints are required: 8i, i0 , j0 2  : i0 6¼ j0 , i0 6¼ i, i 6¼ 1

ð6:29Þ

8i 2  : i 6¼ 1

ð6:30Þ

γ i, i0 , i ¼ 1

8i 2  : i 6¼ 1

ð6:31Þ

X γ i, j0 , 1 ¼ 0

8i 2  : i 6¼ 1

ð6:32Þ

γ i, i0 , j0  δi0 , j0 X γ i, 1, j0 ¼ 1 j0 2

X i0 2

j0 2

X X γ i, i0 , j0 ¼ γ i, j0 , i0 j0 2

8i, i0 2  : i0 6¼ i, i, i0 6¼ 1

ð6:33Þ

j0 2

γ i, i0 , j0 2 f0, 1g

8i, i0 , j0 2  : i0 6¼ j0 , i0 6¼ i, i 6¼ 1

ð6:34Þ

292

6 Applications of Mathematical Modeling

Table 6.5 The distance matrix (km)

City 1 2 3 4 5 6 7 8 9 10

1 – 5 17 23 28 29 28 38 16 39

2 5 – 16 27 28 48 21 45 39 29

3 17 16 – 44 2 41 15 29 23 14

4 23 27 44 – 12 14 32 25 9 11

5 28 28 2 12 – 26 5 41 37 4

6 29 48 41 14 26 – 40 32 17 8

7 28 21 15 32 5 40 – 26 18 15

8 38 45 29 25 41 32 26 – 7 16

9 16 39 23 9 37 17 18 7 – 5

10 39 29 14 11 4 8 15 16 5 –

Table 6.6 The size of different formulations Formulation 1 2 3

Binary var. n2  n (90) n2  n (90) n2  n (90)

Continuous var.  n1 (9) n3  3n2 + 3n  1 (729)

Constraints 2n + n  2 (1032) n2  n + 2 (92) n3  2n2 + 5n  2 (848)

Constraint set (6.29) requires that in identifying a path from city 1 to city i, only the arcs (i0, j0) with δi0 , j0 ¼ 1 are considered. Constraint sets (6.30)–(6.33) guarantee the presence of a path from city 1 to city i. Thus, the third formulation of the TSP is as follows: Model 6.14 The third formulation of the TSP X ci, j δi, j min z ¼ i, j2: i6¼j s.t. (6.24), (6.25), (6.27), (6.29)–(6.34)

6.4.3

Solving the TSP

Problem Data Consider the set of cities as  ¼ f1, 2, . . . , 10g, and define the distance between any pair of them (km) by Table 6.5.

6.4 Traveling Salesman Problem Table 6.7 Comparison of different formulations

Formulation 1 2 3

293 Solution time (s) 0.39 0.61 0.38

LPR bound 109.5 80.4 109.5

Fig. 6.4 Optimal solution of the TSP

Model Size The number of binary and continuous variables and the number of constraints of the three formulations are reported in Table 6.6. Also, the density of the coefficient matrix in the first, second and third formulations for n ¼ 10 is 24.7, 5.24 and 0.46, respectively. Note that in the third formulation, if the variable γ i, i0, j0 is considered as a nonnegative continuous variable, it still takes a value of zero or one in the optimal solution. Thus, the number of variables reported for the third formulation in Table 6.6, is calculated by considering γ i, i0, j0 as a continuous variable.

Problem Solution The optimal solution and the corresponding objective function value are as follows: δ1, 2 ¼ δ2, 3 ¼ δ3, 5 ¼ δ5, 7 ¼ δ7, 8 ¼ δ8, 9 ¼ δ9, 10 ¼ δ10, 6 ¼ δ6, 4 ¼ δ4, 1 ¼ 1, z ¼ 111 The solution time and the bound produced by LPR problem associated with each formulation are reported in Table 6.7.

6.4.4

Solution Analysis

As depicted in Fig. 6.4, the optimal path is 1 ! 2 ! 3 ! 5 ! 7 ! 8 ! 9 ! 10 ! 6 ! 4 ! 1, with a length of 111 km. Also, according to Table 6.7, the bounds of LPR problem associated with the first and the third formulations are better than the second one.

294

6.5

6 Applications of Mathematical Modeling

Disease Control Problem

The disease is one of the main causes of death among people living in developing countries. The mortality rate increases due to the population growth, unhealthy diet, changes in lifestyle and aging, and to overcome this challenge, modern management techniques, and extensive scientific research are necessary. Operations research techniques have particular importance in a wide range of healthcare applications including disease prevention, radiotherapy treatment planning, operating room scheduling, optimal utilization of hospital and medical resources, and so forth. For example, inefficient and inaccurate planning of operating rooms activities may cause delay of surgeries or cancellations, which have serious drawbacks such as patient dissatisfaction and increased hospital costs. Studies show that managers of most hospitals are not satisfied with the status of the operating room scheduling practice. In this section, a disease control problem (DCP), as an example in the field of health-care, is studied which leads to a minimax formulation in which we deal with the multiplication of binary variables. Hence, an appropriate method is utilized for linearization.

6.5.1

Description of the DCP

A city, which is divided into several districts, is exposed to an infectious disease. The Ministry of Health intends to prevent the spread of disease throughout the city. For this purpose, a group of expert teams is considered to be dispatched to the districts for monitoring and investigations. The time needed to check each district by each team is known, and each team can check at most two districts. The time of traveling between every pair of districts is known and when a team is deployed to visit districts, in addition to the time spent to check the districts, the travel time is considered as team’s working hours. When the inspection of all districts is terminated, the disease control center can make the necessary arrangements. Therefore, the aim is to obtain the initial report of the districts in the shortest possible time.

6.5.2

Model of the DCP

Sets, Indices, and Parameters m: Number of districts n: Number of expert teams  ¼ f1, . . . , mg: Set of districts (indexed by j, j0)  ¼ f1, . . . , ng: Set of expert teams (indexed by i)

6.5 Disease Control Problem

295

ti, j: The time of checking the district j by team i dj, j0 : The travel time from district j to district j0(dj, j0 ¼ dj0 , j ).

Decision Variables δi, j: A binary variable that is 1 if team i is dispatched to district j; otherwise 0 (i 2 , j 2 ).

Objective Function

min max i2

8