Mission Oriented Effectiveness Evaluation and Optimization of Complex Systems 1536193801, 9781536193800

Citation preview

COMPUTER SCIENCE, TECHNOLOGY AND APPLICATIONS

MISSION ORIENTED EFFECTIVENESS EVALUATION AND OPTIMIZATION OF COMPLEX SYSTEMS

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.

COMPUTER SCIENCE, TECHNOLOGY AND APPLICATIONS Additional books and e-books in this series can be found on Nova’s website under the Series tab.

COMPUTER SCIENCE, TECHNOLOGY AND APPLICATIONS

MISSION ORIENTED EFFECTIVENESS EVALUATION AND OPTIMIZATION OF COMPLEX SYSTEMS

DEPING ZHANG AND

XUEFENG YAN

Copyright © 2021 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. We have partnered with Copyright Clearance Center to make it easy for you to obtain permissions to reuse content from this publication. Simply navigate to this publication’s page on Nova’s website and locate the “Get Permission” button below the title description. This button is linked directly to the title’s permission page on copyright.com. Alternatively, you can visit copyright.com and search by title, ISBN, or ISSN. For further questions about using the service on copyright.com, please contact: Copyright Clearance Center Phone: +1-(978) 750-8400 Fax: +1-(978) 750-4470

E-mail: [email protected].

NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the Publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. Additional color graphics may be available in the e-book version of this book.

Library of Congress Cataloging-in-Publication Data Names: Zhang, Deping, author. | Yan, Xuefeng, author. Title: Mission oriented effectiveness evaluation and optimization of complex systems / Deping Zhang and Xuefeng Yan. Description: New York : Nova Science Publishers, Inc., [2021] | Series: Computer science, technology and applications | Includes bibliographical references and index. | Identifiers: LCCN 2021015013 (print) | LCCN 2021015014 (ebook) | ISBN 9781536193800 (hardcover) | ISBN 9781536195354 (adobe pdf) Subjects: LCSH: System analysis. | Systems engineering. Classification: LCC T57.6 .Z54 2021 (print) | LCC T57.6 (ebook) | DDC 003--dc23 LC record available at https://lccn.loc.gov/2021015013 LC ebook record available at https://lccn.loc.gov/2021015014

Published by Nova Science Publishers, Inc. † New York

We would like to dedicate this book to our families. Sincere thanks to support from our beloved parents, wives and children.

Contents Preface

xi

Acknowledgments

xiii

1 Overview of Effectiveness Evaluation 1.1. Related Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2. Main Problems and the Trends . . . . . . . . . . . . . . . . . . . . . . . . 1.3. Significance of SoS Effectiveness Evaluation . . . . . . . . . . . . . . . . 2 Basic Concepts 2.1. Effectiveness Measurement . . . . . . . . . . . . . . . . . . 2.1.1. Concept of Effectiveness . . . . . . . . . . . . . . . 2.1.2. Effectiveness Measurement Analysis . . . . . . . . 2.2. System Operational Capability . . . . . . . . . . . . . . . . 2.2.1. Definition . . . . . . . . . . . . . . . . . . . . . . . 2.2.2. Operational Effectiveness and Operational Capability 2.3. Effectiveness Index . . . . . . . . . . . . . . . . . . . . . . 2.4. Process of System Effectiveness Evaluation . . . . . . . . . 2.5. System Effectiveness Evaluation Method . . . . . . . . . . . 2.5.1. Classical Effectiveness Evaluation Method . . . . . 2.5.2. New Method of System Effectiveness Evaluation . .

1 2 3 5

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

7 7 7 10 13 13 14 16 18 19 20 22

3 Construction of Evaluation Index System 3.1. Selection Principle of Evaluation Indexes . . . . . . . . . . . . . . . . 3.2. Construction of Index System . . . . . . . . . . . . . . . . . . . . . . 3.3. Construction of Evaluation Index System . . . . . . . . . . . . . . . . 3.3.1. Construction Model of Evaluation Index System . . . . . . . . 3.3.2. Selection of Construction Pattern . . . . . . . . . . . . . . . . 3.3.3. The Construction Process of Hierarchy Evaluation Index System 3.3.4. Index System and Optimization . . . . . . . . . . . . . . . . . 3.3.5. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4. Evaluation Index System Construction . . . . . . . . . . . . . . . . . . 3.4.1. Characteristics of SoS Effectiveness of Evaluation . . . . . . . 3.4.2. Index System Framework . . . . . . . . . . . . . . . . . . . . 3.4.3. Correlation Analysis of Efficiency Indexes . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

25 26 27 29 29 32 33 36 38 44 44 46 52

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

viii

Contents 3.4.4. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5. Construction of Mission-Oriented Evaluation Index System . . . . . . . . . 3.5.1. Construction of Operational Mission Profile . . . . . . . . . . . . . 3.5.2. Mission Effectiveness Evaluation Index System Based on ProcessFocused Thinking Method . . . . . . . . . . . . . . . . . . . . . . 3.5.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6. Selection of Effectiveness Evaluation Index . . . . . . . . . . . . . . . . . 3.6.1. Indexes Selection Based on Multiple Correlation Analysis . . . . . 3.6.2. Dynamic Index Selection Based on Sensitivity Analysis . . . . . . 3.6.3. Index Selection Based on Kernel Principal Component Analysis . .

4 Mathematical Modeling of Effectiveness Index 4.1. Basic Index Measurement . . . . . . . . . . . . . . . 4.1.1. Index Dimensionless Based on Utility Function 4.1.2. Index Dimensionless Based on Membership . . 4.2. Mathematical Modeling of System Effectiveness Index 4.2.1. System Capability Index Modeling . . . . . . 4.2.2. System Effectiveness Index Modeling . . . . . 4.3. Mathematical Modeling of SoS Effectiveness Index . . 4.3.1. Modeling of Networked Evaluation Index . . . 4.3.2. Modeling of Emergence Evaluation Index . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

5 Classical Method of Effectiveness Evaluation 5.1. Effectiveness Evaluation Based on AHP . . . . . . . . . . . . . . 5.1.1. Basic Principles of AHP . . . . . . . . . . . . . . . . . . 5.1.2. Effectiveness Evaluation Based on AHP . . . . . . . . . . 5.1.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . 5.2. The Comprehensive Effectiveness Evaluation Based on ANP . . . 5.2.1. The Basic Principle of ANP . . . . . . . . . . . . . . . . 5.2.2. Effectiveness Evaluation Based on ANP . . . . . . . . . . 5.2.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . 5.3. Comprehensive Effectiveness Evaluation Based on Factor Analysis 5.3.1. Basic Principles of Factor Analysis . . . . . . . . . . . . 5.3.2. Efficiency Evaluation Based on Factor Analysis . . . . . . 5.3.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . 5.4. Comprehensive Effectiveness Evaluation Based on ADC . . . . . 5.4.1. Basic Principles of ADC . . . . . . . . . . . . . . . . . . 5.4.2. The Effectiveness Evaluation Based on ADC . . . . . . . 5.4.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . 5.5. Comprehensive Effectiveness Evaluation Based on Cloud Model . 5.5.1. Basic Theory of Cloud Model . . . . . . . . . . . . . . . 5.5.2. Effectiveness Evaluation Based on Cloud Model . . . . . 5.5.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . 5.6. Intelligent Effectiveness Evaluation Based on Machine Learning . 5.6.1. Classical Machine Learning Algorithms . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

53 70 70 71 74 80 80 83 84

. . . . . . . . .

89 89 89 92 94 94 98 101 102 111

. . . . . . . . . . . . . . . . . . . . . .

119 119 119 125 127 132 132 138 145 150 150 157 157 160 160 163 165 171 172 180 184 188 189

ix

Contents

5.6.2. Effectiveness Evaluation Based on Machine Learning . . . . . . . . 204 5.6.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 6 Mission Oriented Effectiveness Evaluation 6.1. Mission Based Effectiveness Evaluation . . . . . . . . . . . . . . . . . 6.1.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.2. Mission-Oriented Operation Effectiveness Evaluation Model . . 6.1.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2. Multi-Mission Effectiveness Evaluation Based on Operation Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1. Construction of Operation Loop Model . . . . . . . . . . . . . 6.2.2. Operation Loop Based Multi-Mission Effectiveness Evaluation . 6.2.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Sensitivity of Operational Effectiveness 7.1. Sensitivity Analysis Based on Range Analysis . . . . . . . . . . 7.1.1. Principles . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.2. Range Sensitivity Analysis Based on Agent Model . . . 7.1.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . 7.2. Global Sensitivity Based on Sobol’s . . . . . . . . . . . . . . . 7.2.1. Basic Principle of the Sobol’s . . . . . . . . . . . . . . 7.2.2. Sobol’s-Based System Effectiveness Sensitivity Analysis 7.2.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . .

. . . .

211 211 211 212 223

. . . .

. . . .

226 229 230 238

. . . . . . . .

. . . . . . . .

245 245 245 247 249 253 253 257 259

. . . . . . .

. . . . . . .

263 263 263 265 266 266 269 270

. . . . . .

. . . . . .

. . . . . .

275 275 278 278 279 283 290

. . . . .

. . . . .

. . . . .

291 293 294 294 297

. . . . . . . .

8 Contribution Effectiveness 8.1. Basic Concept of Effectiveness Contribution . . . . . . . . . . . . . . . 8.1.1. Connotation and Classification . . . . . . . . . . . . . . . . . . 8.1.2. Relationship among Contribution, Capability and Effectiveness 8.2. Effectiveness Contribution Evaluation of SoS . . . . . . . . . . . . . . 8.2.1. Contribution Evaluation Based on Operational Effectiveness . . 8.2.2. Systemic Framework based Contribution Rate Evaluation . . . 8.2.3. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Design, Evaluation and Optimization of SoS 9.1. Effectiveness-Based Design, Evaluation and Optimization Framework 9.2. Domain Model-Driven Design Methodology . . . . . . . . . . . . . . 9.2.1. Overview of the SoS Engineering . . . . . . . . . . . . . . . 9.2.2. Domain Model for the SoS . . . . . . . . . . . . . . . . . . . 9.2.3. A Domain Model-Driven Design Methodology for SoS . . . . 9.3. Mission-Oriented Domain Model Driven SoS Modeling Platform . . . 9.3.1. Technology Architecture of Domain Model Driven Modeling Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.2. Main Functions of Domain Model Driven Modeling Platform 9.4. Research and Implementation Complex SoS Simulation Platform . . . 9.4.1. Overall Framework of SoS Simulation Platform . . . . . . . . 9.4.2. Complex SoS Simulation Engine . . . . . . . . . . . . . . . .

x

Contents 9.4.3. Topic Planning . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.4. Scenario Edition . . . . . . . . . . . . . . . . . . . . . . . . 9.4.5. Experiment Design . . . . . . . . . . . . . . . . . . . . . . . 9.5. Implementation of Effectiveness Evaluation and Analysis Platform . . 9.5.1. Technology Architecture of the Evaluation Analysis Platform 9.5.2. Implementation of the Evaluation Platform . . . . . . . . . . 9.6. Mission-Oriented Complex System Design Optimization . . . . . . . 9.6.1. Problem Description . . . . . . . . . . . . . . . . . . . . . . 9.6.2. Mission-Oriented Optimization of SoS . . . . . . . . . . . . 9.7. Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.1. Weapon System of Systems and Operation Scene . . . . . . . 9.7.2. Operation Mission oriented Optimization Design . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

299 300 300 300 301 302 304 304 305 307 307 307

References

311

About the Authors

333

Index

335

Preface With the development and wide application of the new information technology, the trend of informationization, intelligence and integration for complex systems is becoming more and more obvious. Various complex systems promote and restrict each other to form an organic with a feature of system emergence. The goal of these complex systems is to accomplish complex tasks in very complicated and uncertain environments and get better results that people expect. System effectiveness is always used to measure the design quality and capacity of complex systems. It represents the capability of a complex system which performs specific tasks under specific conditions, and it can represent the comprehensive capability of the complex system. To guide the optimization design process, improve the design quality, reduce the design lifecycle and ultimately enhance the overall capabilities of complex systems, it is necessary to find a scientific method to evaluate and optimize the system effectiveness of the complex system. At present, there are many methods to evaluate the system effectiveness, but most of the evaluation objects are mainly single equipment or a subsystem. Complex systems have various typical characteristics of large scale, complex relationships, diverse tasks, and typical uncertainties. It is difficult to achieve comprehensive evaluation and it is not effective to use the existing simple methods directly. Due to the existing researches on comprehensive evaluation of system effectiveness are difficult to meet the emergence requirements of complex systems, it is necessary to research and form a comprehensive evaluation and optimization method of system effectiveness for the complex system. The basic theories and applications of effectiveness evaluation and optimization technology for mission oriented are introduced in this book. The book contains nine chapters, the main contents are as following: Overview of Effectiveness Evaluation, Construction of Effectiveness Evaluation Index System, Mathematical Modeling and Analysis Technology of Effectiveness Index, Analysis Technology of Classical Effectiveness Evaluation, Mission Oriented Effectiveness Evaluation Technology, Operational Effectiveness Sensitivity Analysis Technology, Analysis Technology of System Effectiveness Contribution, Mission Oriented Modeling, Evaluation and Optimization of Complex System. The main contents of this book including the entire analyzing process of system effectiveness and complex system performance, the construction, selection and modeling of the index system, the evaluation of system effectiveness, and the analysis of sensitivity and contribution, optimization etc. Each chapters have a close logical relationship, which can help the readers understand and catch the knowledge in the field easier and better. This book has strong practicality. Based on the basic theories of various methods, we provide many concrete cases. Readers can quickly apply it to the practice of effectiveness evaluation

xii

Deping Zhang and Xuefeng Yan

following the case studies. Not only theoretical knowledge, but also the practical are from the authors research and applications. The latest technological development is also taken into account. This book mainly discusses the performance evaluation of typical complex systems, focusing on new theories and methods in this field. We believe the can help the development and application of effectiveness evaluation of complex system. This book can be used as reference or textbooks for high grade students and graduated students in computer simulation and System Modeling and Simulation, Control Science and Engineering. It also has important reference value for researchers and engineers in professional fields such as ship engineering, aircraft guidance and control, and aircraft design.

Acknowledgments This book embodies the wisdom and hard work of cooperated people, their efforts and contributions make the publishment of this book possible. First and foremost, we would thank Academician Bohu Li, Chinese Academy of Engineering, Professor Yi Pan, Dean of Computer Science and Control Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Professor Xudong Chai, general manager of CASICloudTech Co., Ltd. From China Aerospace Science and Industry Corporation Limited. Thanks to their academic and life guidance, the we have been able to some achievements in SoS evaluation, and due to their encourage and hard work, we can summarize our work and finish this book in good time. Secondly, we would also like to thank staff members at Nova Scientific Publishing, Inc. for their circumspective and earnest work. Thirdly, thanks all the contributors, Professor Liquan Liang, Professor Lei Hu, Professor Zhengxian Wei, Professor Zhe Zhang, Professor Xiaodong Huang, Dr. Baocun Hou, Dr. Guoqiang Shi devoted a great deal of wisdom and effort for the book. Over the past decade, they have guided us in theory, instructed us in technology, helped us in implementation, and assisted us in verification. Finally, many outstanding scholars have made great contributions to the publication of this book. Lecturer Diming Wu, from Bengbu University, responsible for the polishing of the whole book. Master Shasha Cheng, Yuqing Zhang, Weisong Sun, Dr.Yongzheng Wang, YanBiao Niu, Master Jiahao Xu, Weichao Gao, Kai Ye, Guohua Zhang participated in the writing of some part of the book. Because of their efforts, this book can be delivered and published in time. Sincere thanks to their fruitful work. This book is supported by the National Key R& D Program of China, the Key Technology Research and Platform Development for Cloud Manufacturing Based on Open Architecture under Grant No. 2018YFB1702700, the Pre-Research Program under Grant Nos. 61400010111, the 14th Five-Year Planning Equipment Pre-Research Program under Grant No. 2020605C003.

Chapter 1

Overview of Effectiveness Evaluation With the development and wide application of the new information technology, the trend of informationization, intelligence and integration for complex systems is becoming more and more obvious. Various complex systems promote and restrict each other to form an organic with a feature of system emergence. The goal of these complex systems is to accomplish complex tasks in very complicated and uncertain environments and get better results that people expect. System effectiveness is always used to measure the design quality and capacity of complex systems. It represents the capability of a complex system which performs specific tasks under specific conditions, and it can represent the comprehensive capability of the system of systems. To guide the optimization design process, improve the design quality, reduce the design lifecycle and ultimately enhance the overall capabilities of system of systems, it is necessary to find a scientific method to evaluate and optimize the system effectiveness of the complex system. System of Systems (SoS) is one of the most complex and typical complex systems, this book introduces the typical SoS effectiveness evaluation model, method and case. Effectiveness of system of systems (SoS) is a multi-system inter-discipline subject that has gradually matured with the development of aviation science and technology. Research and applications in this area are more advanced in the United States and Russia. The publication of the book “Aerial Shooting” by former Soviet experts Ttyiaueb in 1940 marked the birth of the effectiveness theory of aerial firing. The basic theory of effectiveness analysis for SoS was initially formed in the 1960s. Meanwhile, the effectiveness of air to air firing against single or multiple targets has matured, and also the effectiveness of air to surface firing against the point targets, surface targets, and group targets. The United States, the Soviet Union, and other countries have set up the specialized institutions of operational effectiveness analysis in the early 1960s, the effectiveness analysis of aviation has formed gradually, and its application in aerial firing, bombing, and air battles have been further improved and studied. A lot of research results have been obtained during this period. Related theoretical research on the quantitative methods can be divided into two fields, semi-empirical theory and strict theory. The former includes methods of performance comparison, empirical formula, and expert evaluation. The latter covers approaches of probability statistics, geometric programming, etc. In recent years, methods based on fuzzy technology have also developed, including the analytic hierarchy process (AHP), fuzzy synthetic rating, gray evaluation, and analytical method.

2

Deping Zhang and Xuefeng Yan

The operational effectiveness refers to the expected goal in the development and operational process of the SoS. It directly affects the role of the SoS in the war. Only with high effectiveness the SoS can take the initiative in the modern war, play huge operational capability and finally win the war against a strong enemy. Research on effectiveness analysis in Asia started later than Europe. The systematic analysis of weapon effectiveness began in the late 1980s. Although the research history is not long, it has gotten a rapid progress recently. Several experts in the national defense industry and military have done various fruitful works. The most representative books are “Evaluation of Operational Aircraft Effectiveness” published by Aviation Industry Press [1] and “Military Operations Research” published by Military Science Press in 2006 [2]. Since late 1980s, the national defense science and technology community have attached more and more importance, scientists have made a breakthrough in performance analysis of the performance evaluation of avionics systems. Traditional analysis methods have made many improvements, and some new theories have also been proposed according to national condition. With the development of modern intelligent computing methods, the application of intelligent computing to evaluate system effectiveness has been widely used. Genetic algorithm (GA), neural networks, Petri nets, and various fusion algorithms are gradually being applied in military systems. At present, the combination of scientific theories, experience, and judgments is the trend at home and abroad. Fuzzy mathematics, expert systems, systems engineering, system simulation, big data analysis, artificial intelligence, and other technologies are applied to explore the operational effectiveness of the equipment. With the improvement and development of equipment technology, simulation, virtual technology, big data analysis, and artificial intelligence, the effectiveness evaluation has drawn more attention in recent years. It has gradually become an important research topic in the field of design, acquisition, demonstration, and operation the SoS. The effectiveness evaluation of the SoS can provide a quantitative analysis basis for development demonstration, model demonstration, and technical transformation demonstration. Meanwhile, it is also an important basis to analyze the effectiveness in combination with specified operational backgrounds. The effectiveness evaluation and analysis can provide a quantitative and reliable basis not only for equipment development policy, but also for the research of operational guidelines and tactics. It also plays an important role in improving training methods and benefits. Therefore, the effectiveness evaluation has become a hot topic in the military academia and departments of equipment development.

1.1. Related Research System effectiveness is mainly used to evaluate the effect and capability of equipment at a macroscopic level. The evaluation process does not aim at the enemy situation, the complex operational environments and the enemy confrontation. Research of system effectiveness is relatively easy to implement. There are many mature models and methods that can be referenced. The specialized analysis of operational effectiveness all over the world has begun since the Second World War. The US has conducted lots of research on effectiveness evaluation

Overview of Effectiveness Evaluation

3

issues since the mid-1960s [3]. Various types of models are proposed to estimate different types of equipment. The ADC model, the theoretical lethality indexes and the weapon indexes are the typical system effectiveness model proposed by the US Weapon System Effectiveness Industry Advisory Committee (WSEIAC) for the US Air Force. Typical research results of the former Soviet Union are “The Effectiveness of Air Defense Missile Weapon Systems” by C.H. Petuchov and A.H. Stepanov, and “A Probability Method for Evaluating Weapon Effectiveness” by A. A. Chelvona et al. Research on equipment effectiveness evaluation has become more advanced in recent years. Not only the research scope has expanded, but also the systematic research has increased. Its object has involved various types of equipment and the development, production, and operation. Systematic research refers to two meanings. The first is to take the whole as the object, starting from completing operational missions, the systematic analysis of various equipments involved is carried out. The second one is to evaluate the overall operational capability of the whole army or a army service from a macroscopic level. Research on effectiveness evaluation of the equipment mainly began after the mid1970s and was widely developed in the 1980s. The main methods are systematically summarized in the book “Operational Aircraft Effectiveness Evaluation” by Zhu et al. [1]. A method of effectiveness index is proposed to estimate the capability of operational aircraft quickly and easily, and two aircraft of air battle is simulated in this book. Xu [4] has done a lot of in-depth research in evaluation of naval equipment, and the index method is also suitable for air force equipment. Besides, many colleges, universities, and scientific institutions have also studied the system effectiveness at different levels. The typical methods are AHP [5], multi-targeted decision-making, neural network, fuzzy synthetic rating [6], gray evaluation, etc. However, most of the existing research aimed at single equipment, such as the effectiveness of surface-to-air missiles in a certain type. For the SoS composed of multiple types of equipment, the research also focuses on the effectiveness of a single operational system. Representative research results about the effect of quantitative parameters on system effectiveness are not much for the SoS. In the late 1980s and early 1990s, the operational simulation to evaluate the effectiveness of equipment in air battle was relatively popular in Asia. Hot topics centered on fighter simulations of the one-to-one air battle, two-plane aerial battle, and multi-plane aerial battle. The details is introduced in the book “Evaluation Method of Operational Effectiveness for Surface-to-Surface Missile Weapon System” by Zhen et al. [7]. The theory of effectiveness evaluation, evaluation index system, and system of evaluation methods were presented by using a method of dynamic research and system theories, realized the integration of theories, methods, and engineering. Recently, many kinds of literature also discuss and analyze the concepts and the methods of effectiveness evaluation from different levels and perspectives.

1.2. Main Problems and the Trends In summary, research on the effectiveness evaluation of SoS has made a rapid progress. It has played an important role in all aspects including weapon design and selection, cost analysis, operational application, and optimization configuration. But there are still some problems in the following aspects [8]-[20]:

4

Deping Zhang and Xuefeng Yan (1) Processing of effectiveness data.

The original data used for effectiveness analysis are quite scattered and difficult to collect and process. This is also a major obstacle that affects the credibility of evaluation. (2) Effectiveness evaluation model. Effectiveness evaluation methods are basically static analysis, which cannot reflect the dynamic change process of effectiveness in real-time. In particular, the evaluation model based on the process of operational mission still has some errors with the actual effectiveness of equipment under real confrontation conditions. (3) Pertinence and operational environment. The pertinence and environmental adaptability of effectiveness analysis need to be strengthened, which are mainly reflected in the fact that the effectiveness index cannot fully reflect the quantitative relationship among the operational effectiveness, mission, and the environment. The development trends of the effectiveness evaluation are as follows: (1) New mathematical theory [21]-[30]. It is imperfect to use probability statistics to describe uncertainty in traditional studies. The introduction of fuzzy mathematical is the development trend of effectiveness evaluation for equipment. The gray system believes that the uncertainty of things is not only randomness and ambiguity but also a lack of some key information. With the increasing performance of SoS, there are more and more factors affecting effectiveness. Probability statistics require as many data as possible to establish statistical rules. The introduction of a grey theory is another trend in the development of effectiveness evaluation. (2) Development of intelligent science [31]-[36]. With the rapid development of intelligent control science, fuzzy control, neural networking, GA and other new intelligent control methods have been proposed, which are applied to the effectiveness evaluation and improve the reliability. (3) Requirements of informatization. With the development of operational effectiveness at home and abroad, it will develop in the direction of informatization to meet complex requirements of future war and improve effectiveness. Therefore, combined with real-time operational requirements of the information-based war, effectiveness evaluation will also be a major development trend. (4) Fusion technology of multidisciplinary information [37]-[39]. As a new engineering discipline and design idea, multidisciplinary design optimization (MDO) can handle complex engineering systems, which has been recognized by multiple industries. However, its application in the field of effectiveness evaluation and analysis for equipment started late. There are many research results on the current evaluation, but the research results of optimization are less. Therefore, it is necessary to improve and optimize the effectiveness evaluation, which should develop towards the direction of networking, integration, systematization and function integration, and make a more scientific, compre-

Overview of Effectiveness Evaluation

5

hensive and objective evaluation.

1.3. Significance of SoS Effectiveness Evaluation Operational effectiveness is a dynamic concept, which refers to the effective probability to achieve the expected goal when it is used to perform a specified operational mission under specified conditions. It is indispensable for the demonstration of SoS. The results are an important basis for determining tactical and technical index and operation mode. Enhancing effectiveness evaluation and analysis capabilities is the key point to the improvement of demonstration and decision-making levels. Therefore, the improvement of demonstration and decision-making level requires a higher evaluation capability. The effectiveness evaluation provides support for equipment demonstration in the following aspects: (1) Effectiveness evaluation is used for the optimization of operational scheme. The final result is the effectiveness value corresponding to the operational plan. The effectiveness value is an important basis for decision-makers to choose the reasonable scheme. (2) Effectiveness evaluation is used for the analysis of the key influencing factors. The factor set that affects the effectiveness is composed of multiple-type factors. There must be primary and secondary factors among them. It is a crucial to find out the factors with a great impact and then consider them in the design process of the scheme. (3) Effectiveness evaluation is used for the optimization of operational decision-making. The expected effectiveness value of the operational scheme will be obtained before the decision-makers evaluating the effectiveness of equipment. If the value can not meet the requirement, the evaluation should provide some improvement information on the operational scheme and guide the optimization process. It can be seen that the requirements of demonstrations are of diversification in the operational effectiveness. To access these challenges, it is necessary to solve the following problems effectiveness evaluation: (1) What is the operational effectiveness value of a given scheme? This problem is called a forward problem. (2) If a scheme cannot achieve the expected operational effectiveness, how should the scheme be adjusted? This problem is formally called to as inverse problem. (3) The operational effectiveness is related to the actual situation. Because the state of equipment or the situation during the actual war is often uncertain, it is necessary to answer the question of which solution can achieve the optimal effectiveness for possible changes in uncertainties. This problem is formally referred to as sensitivity analysis. (4) If it is sure that a certain factor in the operational process reaches a certain level, what level effectiveness can achieve under this condition? This problem is called a problem of intermediate interference.

6

Deping Zhang and Xuefeng Yan

An effectiveness evaluation method mainly consists of two parts: the evaluation model and its application pattern. The evaluation model is a computable model that takes the influencing factors as inputs and the focused index as outputs. In other words, the mapping relationship between value of the lower index and the upper index is determined. The main role of the evaluation model is to examine the level of corresponding effectiveness index by adjusting the controllable factors. It is mainly used for scheme comparison and optimization. The above four types of problems are resolved completely by existing evaluation methods from different views. Therefore, it is of great practical significance to study the effectiveness evaluation to meet the increasingly complex and diverse application requirements.

Chapter 2

Basic Concepts 2.1. Effectiveness Measurement The effectiveness measurement is one of the most appropriate quantitative indexes for evaluation of SoS. One of the important missions of effectiveness analysis is to determine the effectiveness measurement. The effectiveness measurement influences the accuracy of the decision directly. According to the requirements of the combat mission, it is very important to select and determine the combat effectiveness index (also called effectiveness measurement) of the weapon system correctly and reasonably, and it is an important reliable basis for decision making of weapon system analysis. The key part of a good effectiveness evaluation is to select effectiveness measurements properly. However, it is not an easy task. In fact, the effectiveness measurement is often a subjective judgment or a “value”. Therefore, a systematic analysis method is of great significance for the equipment acquisition, application and even the whole life cycle.

2.1.1.

Concept of Effectiveness

Effectiveness is a common concept, in the dictionary, it means “the role of things under certain conditions”, “functions of things” or “hidden favorable effects of things”. System effectiveness refers to the degree to which the system achieves a predetermined function under certain conditions. It is an ability or possibility to complete its specified task, which is also called the effectiveness measurement. System effectiveness is generally used to describe the overall capacity of a system to accomplish its missions. To evaluate the system effectiveness, it is necessary to define the measurement indexes. The definition of the Sos effectiveness is the ability to achieve the specified goal under specified conditions. Specified conditions means the factors such as operational environmental conditions, operational time, personnel and adopted methods, etc. Ability is the quantitative or qualitative degree the equipment achieve its goal. From the perspective of probability, weapon effectiveness is the probability that the SoS can meet the requirements of the operational missions under specified conditions and time. Effectiveness can be generally divided into single effectiveness, system effectiveness, and operational effectiveness. Single effectiveness refers to the degree to which a single goal is achieved by the equipment system, such as the effectiveness of firing, detection, command and control commu-

8

Deping Zhang and Xuefeng Yan

nications of the air defense. The operational action corresponding to single effectiveness is a object operation, such as reconnaissance, jamming, minelaying, firing, and other basic steps of firepower support and applications. System effectiveness refers to the probability that can meet a set of specific task requirements under certain conditions. It is a comprehensive evaluation of the effectiveness for the SoS, which is also called comprehensive effectiveness and is mainly considered an effectiveness parameter during the SoS demonstration. The US Weapons System Effectiveness Industry Advisory Committee(WSEIAC) believes that system effectiveness is a measurement of how well a system meets a specific set of mission requirements and is a function of system availability, reliability, and capability. The committee not only made a scientific definition, but also gave a basic framework for evaluating system effectiveness. It means that system effectiveness is composed of three aspects: system availability, reliability, and capability. Sometimes, operational effectiveness is also called military effectiveness. It refers to the probability of the expected goals that can be achieved when using the SoS and their corresponding forces to perform prescribed operational missions under specified operational environments. Here, the implementation of operational missions should cover all the main missions that the SoS may undertake in actual operation and involve the entire process. Therefore, operational effectiveness is the ultimate effectiveness and basic quality characteristic of SoS. Taking the submarine battle as an example, its operational effectiveness mainly depends on the following aspects. Firstly, the submarine must have the ability of anti-ship operations, which is also one of its most important task. The operational effectiveness mainly depends on various sea attack capabilities, such as the operational capabilities of various sea attack missiles and anti-surface ship torpedoes. Secondly, it must have the anti-submarine capability which mainly depends on the performance and number of anti-submarine torpedo systems. Thirdly, it is necessary to have the capability of early warning and acquisition of object information against various threats from the air, surface, and underwater. And, it must have the ability to use water acoustic counter equipment to interfere and deceive enemy detection and guide weapon systems. Finally, the submarine has the adaptability of the operational environment and supporting capability. According to existing studies [40]-[50] the elements of effectiveness evaluation index for the SoS can be divided into four categories, system parameters, performance index, effectiveness index, and operational effectiveness index. The hierarchical relationship is shown in Figure 2.1. The specific elements of each effectiveness evaluation index in Figure 2.1 are described as follows: (1) Dimensional Parameter (DP). The inherent attributes or characteristics of the system. Its value indicates the structure and behavior of the system, such as material, size, weight, and interference power, etc. (2) Measures of Performance (MOP). The quantitative description of system behavior attributes, or the quantitative description of the contribution of a single system factor or attribute to the overall capability, such as the speed, survivability of submarine, and the detection area of the sonar system. Generally the index generally does not take account of

Basic Concepts

9

Figure 2.1. Hierarchical relations of effectiveness index factors. environmental influences. (3) Measures of Effectiveness (MOE). The quantitative description of the function the system can complete, or the quantitative description of the ability of performing its mission in the best way and the expected results. (4) Measures of Force Effectiveness (MOFE). The MOFE, sometimes called the force effectiveness index, is used to measure the probability of a system completing its missions in specific environment. Operations are carried out by certain military forces (including human and the weapon systems) under certain environment according to certain action plans. Therefore, the effectiveness of military forces or action plans under certain conditions can be represented by operational effectiveness. So, the DP and the MOP are the descriptions of the features and capabilities within the system, while the MOE and the MOFE are the descriptions of the features and capabilities of the system in the external environment. The MOE measures the system’s ability to perform its functions or affects other entities in the operational environment. It is a quantitative representation of the system’s ability to achieve specified goals, and it is a basic standard for system analysis and comparison. The MOE is related to some standards, which usually takes ideal systems as reference. Meanwhile, it also describes the ability of system to perform its functions in an operational environment. In particular, MOE depends on specific assumptions. The MOP represents the intrinsic qualities or characteristics of a physical entity. In some ways, it measures the properties of the system behavior, namely the physical and structural behavior parameters and task requirement parameters. It is a measurement of the system effectiveness in performing an function. The performance measurement includes the probability of detection, false alarm rate, signal to noise ratio, baud rate, throughput, frequency range, etc. The MOP reflects the inherent characteristics of the system being an-

10

Deping Zhang and Xuefeng Yan

alyzed and has nothing to do with scenarios. It is one of the basic elements for establishing an index system. High system effectiveness does not mean that operational effectiveness is also high. For example, a communication system can transmit information in time, but the commander does not make good use of this information. Although it can be considered that the effectiveness of this communication system is high, its operational effectiveness at this time is very low. These four categories of indexes have a certain relationship. The system effectiveness measurement depends on the system performance and humans. It reflects the measurement of the system ability to combine with other systems and humans. The operational effectiveness measurement depends on the system effectiveness measurement and environments. It indicates the measurement of the system can accomplish missions in the actual operational environments when the system is combined with other systems and humans.

2.1.2.

Effectiveness Measurement Analysis

It is necessary to establish reasonable requirements (indexes) to evaluate system effectiveness. The description of these requirements must be commensurate with MOP, and it cannot exceed the capability of the system. The effectiveness measurement is obtained by comparing the MOP with the requirements. It means that the effectiveness measurement is a function of the MOP and requirements. Above all, it is necessary to identify the requirements to measure system effectiveness. Requirements are generally proposed and determined by acquisition staff or users. The definition process of requirements is as follows: First, a meaningful attribute for the task is defined. Then, an index (variable) is defined to measure this requirement and a metric (depending on some parametric formula or algorithm) is defined to quantify the index. For a missile, the success rate is defined as 90%, which is a specific requirement. Of course, some requirements are not very specific, but in the specified operational environment, it needs to be abstracted according to the actual situation, and finally this requirement is quantified. The definition process of system performance measurement is as follows: First, an index (concept) is defined which is related to system performance, such as accuracy, time limit, data processing rate, vulnerability, survivability, and reliability. Then, the index can be expressed by a certain formula based on old or new indexes (variables). The result of the measurement index may be a number such as the probability, the expectation value, a number interval, or a multi-dimensional quantitative space. The effectiveness measurement is used to measure how well a system can meet the requirements. The basic process of its definition is similar to performance measurement. First, an index is defined to measure how well the requirement is met. Then, the index is quantified by defining standard formulas or algorithms. Next, it is ensured that the performance measurement is commensurate with the requirement index (variable). The value of effectiveness measurement ultimately depends on the observable and measurable part, requirements parameters in the measurement formula, and the comparison method. Figure 2.2 shows the relationship among the system parameter space, the requirement space, the requirement space, and the effectiveness space. As shown in Figure 2.2, the mapping from the parameter space to the performance space produces the trajectory of the performance measurement (operational capability). Perfor-

Basic Concepts

11

Figure 2.2. Relations among the space of system parameter, performance, requirement, and effectiveness. mance requirements are expressed as the requirement space. If the trajectory of the performance measurement is completely within the requirement space, all system behaviors meet the requirements, the effectiveness is 1.0. If the trajectory is completely outside the requirement space, no system behavior meets the requirements, the effectiveness is 0.0. Let Et represents a subset of the resulting space in which the equipment can complete the operational mission according the general development requirements. Ec represents the subset of the resulting space that can be achieved under actual operational conditions to perform the specified mission. f (X) is used to represent the mapping of the set X. When X is a point set, f (X) represents the number of points in set X. It is assumed that the equipment works normally and makes full use of tactical and technical performance in performing mission. Therefore, the operational effectiveness of the SoS can be defined as: f (Ec ∩ Et ) f (Et )

(2.1)

f (Ec ∩ Et ) f (φ) = =0 f (Et ) f (Et )

(2.2)

Efficiency = If Ec and Ec do not intersect, then, Efficiency =

If Ec ⊇ Et , it means that the equipment has the completed ability to perform the specified missions and achieve expected goals in the specified operational environment, and the ability can be fully used. Therefore, f (Ec ∩ Et ) = f (Et ), then, Efficiency =

f (Et ) f (Ec ∩ Et ) = =1 f (Et ) f (Et )

(2.3)

So, the operational effectiveness of equipment is 0 ≤ Efficiency ≤ 1. The Equation (2.1) shows that the operational effectiveness depends not only on the system performance but also on the actual conditions faced by the operation. In general, the evaluation of performance does not depend on specific scenarios, while the effectiveness does.

12

Deping Zhang and Xuefeng Yan

Figure 2.3. Specific process of formulating MOE. On the basis of clarifying the analysis principle of effectiveness measurement, a specific developing process is given to effectively formulate a MOE as shown in Figure 2.3. It is necessary to determine clear views and missions after having specific requirements of the MOE. This is the basis for formulating MOE. Usually, these contents are generated through repeated interaction with users. For example, in the process of equipment development, the tactical technical performance index and operation requirements proposed by the users and contracts are identified clearly here. In general, it’s not only necessary to determine whether the equipment meets this index and requirements, but also to answer the questions that the decision-makers are concerned in equipment acquisition. These “issues” are the questions that need to be answered when making equipment acquisition decisions, like defense penetration, range, weight, length of the full bomb, and firing accuracy of missiles. “Key issues” are questions that must be studied and answered to evaluate the operation, technology, support, and other capabilities of the equipment. It is the basics for evaluating the equipment and making milestone decisions on equipment acquisition management, such as the maximum range of anti-ship missiles, the number of navigation points in route planning, the ability to resist chaff interference, etc. Usually, it is necessary to classify the key issues related to the use of equipment, namely critical operational issues (COIs). This is also the requirement in previous calculation formulas. The MOE can be initially formulated based on clear requirements. Then the above analytical methods are used to evaluate the MOE. The MOE that meets the needs of the users can be passed, while the MOE does not meet needs to be reformulated. For the passed MOE, it is necessary to be applied specifically. If the MOE has met the expectations user in actual applications, then the whole process stops. Otherwise, it needs to be modified or added.

Basic Concepts

13

2.2. System Operational Capability 2.2.1.

Definition

The operational capability refers to the equipment ability to perform specified missions and achieve expected goals in specified operational environment. It can be reflected by the operational performance (or function) of the equipment [51]. The operational performance mainly includes the technical performance tactics and the operational performance. The technical tactics performance refers to the characteristics and functions determined by factors such as design and manufacturing, etc. It mainly includes the functional characteristics of military and technical performance parameters like mobility, reliability, environmental adaptability, quality parameters, size parameters, service life, and storage life (years), etc. The operational performance refers to the characteristics and functions to meet operational requirements, technical performance, and comprehensive support requirements. It can be seen that the operational capability of equipment is a kind of “comprehensive performance”, which is determined by multiple tactical and technical performances. The definition of “equipment operational capability” has three meanings as follows: (1) Operational capability is an inherent attribute of the equipment, and it depends on structure, quantity and quality of constituent elements and the utilizing way of equipment. (2) The operational capability is aimed at the specified operational missions, environments and objectives. When the operational performance is combined with specified operational missions and environments, the equipment is able to achive the operational capability. (3) As an inherent attribute, the operational capability can change from time to time with the internal factors and external environment. Let Ct represents a subset of the object result space where the equipment executes the specified missions. Cc represents a subset of the object result space achieved by the equip0 ment to perform the specified missions under the specified conditions. Cc is a subset of Cc , and its capability is a sub-capability represented by Cc. f (X) represents the mapping of the set X. When X is a point set, f (X) represents the points in the set X. The operational capability is defined as: Capability = f (Cc ∩Ct )

(2.4)

The operational capability is the intersection mapping of set Cc and Ct . It means that the capability can be measured by the mapping the achieved goal of the equipment to the 0 resulting space. If Cc and Ct do not intersect, it means that the equipment does not have the required capability (ability) to perform specified missions under specified conditions. 0

Capability = f (Cc ∩Ct ) = f (φ) = 0

(2.5)

It means that the equipment cannot perform specified missions and the operational capability required for this task is zero. It is necessary to use a certain quantitative scale to calculate and evaluate the operational capability. The scale is called the capability index. Furthermore, according to the definition, the operational capability index refers to the ability of results achieved by the measurement equipment to carry out the assigned task under specified environment. It can be either

14

Deping Zhang and Xuefeng Yan

the probability that the equipment will complete the specified task or the tactical index of the equipment. If there are multiple object requirements to perform a specified task, a vector composed of a set of capability indexes can be used to characterize the operational capability.

2.2.2.

Operational Effectiveness and Operational Capability

From the definition of operational capacity, it can be seen that the operational capability is the “capacity” to perform specified missions. The operational effectiveness is the degree to which these “capabilities” are used to achieve expected goals. The operational capability and operational effectiveness are two concepts that are related but different. They are closely related. The main connections and differences are as follows: (1) The operational capability is the basis of operational effectiveness. Operational effectiveness is the ultimate manifestation of operational capability for the equipment. The operational capability only indicates the basic subjective conditions required for the equipment to perform specified missions, which is own performance of the equipments. If a communication jamming device can suppress multiple targets based on the comb spectrum pattern, multi-target interference is the “capability” of the device. The “capability” can be described with some performance indexes. For example, interference distance is 10 km, and the number of multiple targets is 3, etc. These values also reflect the operational capability of the equipment. They are used to complete the task of multi-target interference suppression. It is necessary to use operational effectiveness to describe the status of missions. For example, communication interference equipment is used to suppress three radio stations in a certain area. The average accuracy rate of copying message is 5%. Therefore, the operational effectiveness of the communication interference equipment is 0.95. It can be seen that the multi-target suppression task of the three copy stations cannot be completed without the “capability” of multi-target interference, and the operational effectiveness of 0.95 cannot be achieved. (2) The operational capability and operational effectiveness are both related to the implementation of specified missions. Operational capability aims at a certain type (group) of missions, and the “quality” (task type) of the mission must be compatible with its capability. For example, the multi-target interference of the communication is for fixed-frequency communication objects. Its capability can be reflected only by the multi-target interference suppression based on the fixed frequency method. It is not suitable for the equipment to suppress the targets using frequency hopping. The equipment is still used to perform this mission, and its multi-target jamming capability cannot be fulfilled. The operational effectiveness is measured by the compatibility between what can be accomplished and what needs to be accomplished, which means that not only the “quality” but also the “quantity” of missions to be completed is compatible with the ability. That is,   Nf ,1 (2.6) Efficiency = min Nw where N f represents the number of missions that can be completed, and Nw represents the number of missions need to be completed.

Basic Concepts

15

The ratio in Equation (2.6) is a generalized division. It may be an arithmetic division, or it also can be the division of the corresponding components with two vectors. In theory, the complete degree of missions is determined by the existing operational capability. The degree to which the task can be accomplished is a function of the existing operational capability. The greater the operational capability is, the greater the task completing degree is. It can be approximated that the degree is directly proportional to the existing operational capability. Similarly, it is objective that operational capability is required for the mission. The mission completion degree is also proportional to the required operational capability. Efficiency ≈

Cr × Rs × Ra Cn

(2.7)

where Cr is the existing operational capability, Cn represents the required operational capability, Rs represents system reliability, and Ra represents system availability. (3) The operational capability is related to its “ability”, and the measurement of effectiveness is related to the extent to which its capability is utilized. Any value that can distinguish the equipment capability is a measure of operation without the limitation of 0 to 1. For example, the mentioned multi-target interference capability of communication interference can be described by some numerical values such as interference distance of 10 km, and multi-target number of 3, or a collection of these values. Operational effectiveness is the degree to which a task is accomplished. The result must be a value between 0 and 1. For example, the communication interference equipment effectively reduces the accuracy rate of message copy of enemy communication stations under specified operational conditions. Its operational capability is 95%, and its operational effectiveness is 0.95. (4) The operational capability is a static concept, and operational effectiveness is a dynamic concept. Operational capability is an inherent attribute of equipment and is determined by the equipment composition, tactic and technical performance. However, operational capability is not fixed. It will vary with the change of equipment performance and other parameters. For the multi-target interference capability of the communication interference equipment, the operational capability will increase when the transmission power of the equipment becomes larger. The operational effectiveness reflects the performance of equipment to carry out the mission, which not only depends on the operational capability, but also depends on the operational process, the application, the operational object and the whole system. The same equipment can perform different operational missions, or perform the same operational mission in different operational environments. So, operational effectiveness may be completely different. For example, the communication interference equipment mentioned above suppresses multi-target communication based on the fixed frequency mode. When the transmit power of the transmitting party changes, the interference effect of the equipment will change accordingly, and the effectiveness will also change. (5) Both the operational capability and effectiveness are related to the tactical technical performance. The operational capability and operational effectiveness are based on the execution of specified missions. The equipment capability that can be used to perform specified missions must be supported by its tactical technical performance or function. The structural relationship between the three parts is shown in Figure 2.4.

16

Deping Zhang and Xuefeng Yan

Figure 2.4. Relations of the operational effectiveness, the operational capability and the technical performance. Several other important concepts related to effectiveness include operational effect, suitability, inherent capability, system performance, and scale parameters. (1) Inherent capability refers to the ability of the system to complete missions according to its characteristics under the given conditions. (2) The operational effect is the result of the SoS completing the specified operational mission under a certain operational environment. (3) The operational suitability is a comprehensive description of non-operational capability factors such as reliability, maintainability, testability, supportability, availability, safety, compatibility, and interoperability, etc. (4) System performance mainly refers to the characteristics from the perspective of physical parameters such as artillery range and firing rate, etc. The scale parameters mainly describe the external characteristics of the physical geometry.

2.3. Effectiveness Index The effectiveness index is a measure of how well the SoS completes missions in specified conditions. So, the effectiveness criterion corresponds to the objective function. The specific expression of the effectiveness criterion is related to the characteristics and conditions of missions. The effectiveness index is determined by the comprehensive influence of various operational factors in the operation. It changes with these factors. However, the dependence relationship between the operational effectiveness index and these factors is complex and cannot be expressed by simple functional relationship. The operational effectiveness index can be constructed by statistics, analogies, estimates, and theoretical calculations according to the actual operation [52]. If the purpose of the operational mission is obvious and

Basic Concepts

17

can be expressed concretely, the criteria can be expressed in terms of the degree of success achieved for the intended purpose. The effectiveness is often evaluated by multiple indexes due to the complexity of the SoS and the variety of applications in practice. Generally speaking, the probabilistic effectiveness index is more suitable for the SoS whose mission is to destroy specific targets. It is more appropriate to choose the expected effectiveness index for the mission of attacking multiple target or non-specific objects. The rate effectiveness index is more suitable for continuous repetition of a certain operational action. However, no matter which type of effectiveness index is selected, the established model must match the selected effectiveness index and be able to give the quantitative answer according to the selected effectiveness index. (1) Probability of completing an operational mission or achieving a given result. The effectiveness index of operational actions is the probability of obtaining the expected result or completing an operational mission. (2) The specific number of losses. For weapons that cause direct military damage to the enemy, the loss number of enemy operational units can be used as the effective criteria for evaluation, such as the number of enemy aircraft that were shot down and the number of missiles that were intercepted, etc. (3) Mathematical expectation (mean value) of the numerical indexes in the operational process. The physical results of each operational activity are random in each specific case. So, it is necessary to pay attention to the characteristics and values of random phenomena in the effectiveness analysis. If the same activity is repeated many times under the same conditions, the actual average result of each activity should be approximately equal to its mathematical mean value. (4) Relative indexes. Some relative values are commonly used as the effectiveness evaluation index in actual effectiveness analysis. (5) Analogy indexes. They are converted from the parameters of the research object to the reference object. (6) Comprehensive indexes. Comprehensive indexes can be artificially established when the probability or mathematical expectation are used as effectiveness evaluation indexes. (7) Ratio. It is also called fractional indexes. (8) Indexes of weighted sum. This indexes are expressed as a sum. The most important sub-indexes are given the largest weight, while those the-bigger-the-worse indexes are given the negative weight. (9) Vector indexes. When there is only one index that is not enough to evaluate the system, the vector indexes are commonly used. The specific vector indexes are C = C j (N1 , N2 , · · · , Nm ), where j = 1, 2, · · · , m represents the subscript of the index serial number. For the multi-index problem, a compromise method is often used to turn it into one or several single-index problems. (10) Effectiveness index matrix. If the action is carried out in several different conditions while not in a certain specified conditions, then it is assumed that when the conditions can change, several possible solutions A1 , A2 , · · · , Am are compared under various conditions according to a certain effectiveness index. B1 , B2 , · · · , Bn represents various possible conditions, and Ei j indicates the effectiveness index value obtained by the scheme Ai un-

18

Deping Zhang and Xuefeng Yan

der the condition B j . The effectiveness index value can be listed as a matrix, namely the effectiveness matrix E = (Ei j )m×n . The theoretical calculation of the operational effectiveness index can be roughly divided into two ways, simulation and analytical. The simulation method which is developed from actual maneuvers and sandbox operations is commonly used. It calculates the operational effectiveness index by simulating the operational processes on the computer. However, this method is difficult to implement, while the analytical method sometimes only needs some simple calculations to get the result, which is simple and easy to implement and is especially suitable for real time battlefield command and decision. For example, the selection of indexes usually focuses on the operational damage of the two sides in operational effectiveness simulation model. The frequently-used indexes can be divided into the following three categories: (1) Loss, also called damage, is the number or percentage of various weapons destroyed. (2) Result is the number or percentage of various weapons damaged by the main equipment. (3) Loss ratio, also called the exchange ratio, is the ratio between the results and the damage of the main equipment.

2.4. Process of System Effectiveness Evaluation The effectiveness evaluation and analysis of Sos are based on the analysis of each subsystem and the single performance evaluation. It is possible to analyze the operational effectiveness only when the output data and constraints of the subsystem and the single items are obtained. Figure 2.5 shows the process of effectiveness evaluation. To evaluate the operational effectiveness of the SoS, firstly, it is important to determine evaluation objects and confirm the operational mission and evaluation goals. Secondly, the overall evaluation of framework and model architecture should be determined according to the characteristics and objectives of the evaluation, including whether to adopt the classic hierarchical analysis framework or the network analysis framework, whether to adopt the classic effectiveness evaluation models such as AHP, ADC, or big data analysis model, intelligent analysis model, etc. Thirdly, the compositions and functions for the evaluation system and the subsystem are analyzed, so as to carry out accuracy analysis and reliability analysis on the performance of the subsystem. On this basis, the correlation of each index can be constructed; the index related to the evaluation target is extracted; the initial index system can be proposed; and the index system can be verified. After several revisions, the index system of effectiveness evaluation is constructed Finally, the evaluation model and method should be established and determined, which mainly includes single performance analysis and a comprehensive evaluation model. Single performance analysis means that when evaluating the performance of the SoS, single indexes is selected as the evaluation basis according to the purpose of the evaluation, and the results are adopted as the input of the effectiveness evaluation model. For example, the analysis of the hit probability for a single torpedo can be used as the evaluation index of

Basic Concepts

19

Figure 2.5. Analysis process of the system effectiveness evaluation. the attack capability. Therefore, before the system performance is valued, the performance evaluation of a single index should be carried out It is vital to establish an effectiveness measurement model for operational effectiveness analysis. The accuracy of the model directly affects the overall evaluation effect. Therefore, it is necessary to verify and modify the model in the actual use of the system. The more factors are taken into account and the more comprehensive performance is, undoubtedly the more accurate the analysis results are. The calculation will be more difficult at the same time. Therefore, the established model should not only reflect the the essence that the SoS completes the operational mission and achieves the goal under the prescribed conditions, but also be as concise as possible. At the same time, it must be as simple as possible for calculation and analysis. It is necessary to describe the various states of the system in the process of establishing the model, and describe the state change and the possible state transitions during the operational process. After the model is established, the effectiveness evaluation results can be calculated and analyzed according to the effectiveness evaluation method.

2.5. System Effectiveness Evaluation Method The effectiveness evaluation of operational systems has always drawn the attention from the military of various countries. The United States, the Soviet Union, and other countries

20

Deping Zhang and Xuefeng Yan

have set up specialized institutions of operational effectiveness analysis in the early 1960s. In Asia, systematic analysis of operational effectiveness started late and research methods are further improved and developed recently. System effectiveness evaluation methods can be divided into two categories: classic method and the forefront new method, the former including probability and exponent methods, and the later refers to some new effectiveness evaluation methods represented by exploratory analysis, data mining, and artificial intelligence technologies.

2.5.1.

Classical Effectiveness Evaluation Method

Classical effectiveness evaluation methods mainly include the probability method, the exponent method, the ADC method, and other methods, which are described in details as follows: (1) Probability method The probability method is to analyze and predict the effects of the equipment adopted in modern war by using the basic principle and method of probability. It is a commonly used method in equipment demonstration and effectiveness evaluation. In general, the probability method can be used for all problems involving uncertain factors. The advantage is scientific and practical, while the shortage is that most of the data come from shooting range experiments, which is difficult to collect. (2) Exponent method The exponent method is to convert the parameters of various equipment participating in the operation into comparable values according to a certain algorithm. Then these parameter values are calculated according to an algorithm, and finally a value is obtained to represent the effectiveness of the weapon. At present, the exponent methods widely used at home and abroad mainly include the Dubai exponent method, Dunnigan exponent method, relative exponent method, power exponent method and, etc. (3) ADC method The ADC is a system effectiveness index calculation model, E = A × D ×C, developed by WSEIAC for the US Air Force in the 1960s. It takes system effectiveness as a function of weapon system availability, mission credibility, and operational capability. E is the system effectiveness index vector. A is the availability or validity vector, which is a measurement of the system availability at the beginning of the mission and reflects the readiness of the SoS. D is the mission credibility, which indicates the probability that the system will complete the specified function. C is the capability of the system. (4) Monte Carlo method The Monte Carlo method is a standard method for simulating the effect of random factors in operational process, also known as the statistical experiment method. A probability model or a random process is established firstly, and its parameters or numerical characteristics are correspoding to the solution of the problem. Then this parameter or numerical character is calculated by observing the model or process or sampling experiment. Finally, the approximate value of the solution is given, and the accuracy of the solution is expressed

Basic Concepts

21

by the standard error of the estimated value. One difficulty of the Monte Carlo method is the huge dimensionality of the basic state variables, and this method cannot describe the procedure of processing and perceiving information. (5) Lanchester function method The Lanchester function was founded in 1914 by British automotive engineer Lanchester. It is a set of differential equations for the state of operational systems, describing the growth and decline of military forces on both sides of the operation. According to different assumptions, the classic Lanchester equation is divided into first linear law, second linear law, and square law. The square law reveals the important role of the concentrated forces, which has attracted much attention. Since the Lanchester equation was proposed, it has become an important quantitative tool widely adopted in study and analysis war. The disadvantage is that it is difficult to add content related to information acquisition and delivery. (6) System dynamics method The system dynamics method (SD) is a computer simulation technique to study the dynamic behavior of the system, which was proposed by professor Forrest from the Massachusetts Institute of Technology. It applies theories and methods of cybernetics, information theory, and decision theory to build a system dynamics model, and carries on the simulation experiments with the computers as tools comprehensively. System dynamics can be used for long-term, dynamic, and strategic quantitative analysis and research, especially suitable for complex time-varying systems with high-order, nonlinear, and multiple feedback. (7) System effectiveness analysis The system effectiveness analysis (SEA) was proposed by A.H. Levis, etc. The SEA can be summarized as follows: the relationship among component characteristics, system structure, operation method, system availability and performance is studied. The measurement of system effectiveness is given by comparing the capabilities required by the mission with the capabilities provided by the system. The SEA is subjective to the modeling of systems and missions, and the accuracy of the model directly affects the evaluation results. (8) Influence diagram modeling and analysis The influence diagram modeling and analysis is a normalized modeling method of complex systems proposed by James R. Burns from MIT in the 1970s. The method finds out the necessary system parameters which represent system operation process by analyzing the complex system. Then the influence diagram of the system is drawn by analyzing the interaction between the parameters of the system. Finally, according to the actual physical meaning of the influence diagram and system parameters, a certain modeling algorithm is used to obtain the equation of the system state. The method combines qualitative and quantitative methods. But when the system is large , the influence diagram becomes complicated and even difficult to establish. (9) Analytic hierarchy process Analytic Hierarchy Process (AHP) is a decision-making method put forward by Thomes L. Saaty, the famous American operations researcher in the 1970s. The complex problem

22

Deping Zhang and Xuefeng Yan

is first decomposed into constituent elements, and these elements are grouped to form an orderly hierarchical structure according to the dominant relationship. Then the relative importance degree of each element in each level is determined by pairwise comparison and judgment. Finally, the overall ranking of the importance of the decision factors relative to the target level is obtained by synthesizing within the hierarchical structure. Human subjective judgments can be expressed and processed in a quantitative way, AHP is a qualitative and quantitative multi-criteria evaluation method. (10) Petri network The Petri network has both intuitive graphical representations and rigorous mathematical analysis. It has the ability to describe the characteristics of concurrency, synchronization, resource competition and so on, and has its own execution control mechanism, which is suitable to study the distributed concurrency of complex systems [53]. However, the Petri network method also has some shortcomings, such as the inability to perform data processing, no hierarchical design idea, and the inability to describe the timing relationship in the system. (11) Fuzzy comprehensive evaluation By means of fuzzy reasoning, an overall evaluation can be made for system equipment with multiple attributes, or equipment whose overall quality is affected by multiple factors. The overall evaluation can reasonably integrate attributes or factors. The method does not need to know the mathematical model of the object. It is a kind of intelligent activity reflecting human intelligence thinking, and it is also a quantitative and qualitative combined evaluation method. (12) Simulation method The simulation method is an ideal method for evaluating system effectiveness, and it is also the key research method in this book. The advantage of the method is that it can reflect the actual situation truly and dynamically and has higher credibility. The disadvantages are high cost and long cycle of modeling, and high requirements for the designers and users. (13) Meta-synthesis Meta-synthesis is an open complex system and methodology proposed by the famous Chinese scientist Qian Xuesen in 1989. Later, it developed into the Hall for Work Shop of Meta synthetic Engineering (HWSME) from qualitative to quantitative. The method includes the combination of qualitative and quantitative, experts discussion, multimedia and virtual reality, information fusion, fuzzy decision, qualitative reasoning technology and distributed interactive network environment, etc. It is an effective way to evaluate system effectiveness.

2.5.2.

New Method of System Effectiveness Evaluation

With the development of computer, network, and communication technology, parallel computing, distributed computing, grid computing and other large-scale analog computing methods have gradually developed and become more and more mature, and some new system effectiveness evaluation methods have emerged.

Basic Concepts

23

(1) Computational experiment method Computational experiment method takes “simulation” results as an alternative version of the reality, or as a possible reality. Meanwhile, the actual system is also regarded as one of the possible realities, which is equivalent to the simulation results, so as to realize the ideological change from computational simulation to computational experiment. In this method, the traditional calculation simulation has become the “experimental” process in the “computational laboratory”, and has become a way to “cultivate” various complex systems. The actual system is only one possible result of the “computational experiment”. For the evaluation of the complex system, the computational experiment method is considered to be a method with high vitality, at least a useful attempt. (2) Exploratory analysis The Exploratory Analysis (EA) method is one of the hotspots in the research of operation SoS. The EA method conducts a holistic study of the results corresponding to various uncertain elements. In contrast to the synthetic method, this method adopts a top-down, macro-to-micro model. The idea of the method is to determine the top-level operational goal firstly, then determine the relevant elements according to the operational goal, and finally carry out exploratory calculation in the element space. The goal of the EA method is to understand the impact of uncertain factors on the problem studied, and to explore various capabilities and strategies of the system that can meet the task requirements, so as to comprehensively grasp various key elements, obtain flexible, efficient and adaptable problem solutions, and achieve the purpose of capacity planning and scheme optimization. (3) Data farming and data mining technology Similar to the exploratory analysis methods, data farming is another hot topic in the field of operation complex systems. Data farming was jointly proposed by Dr. Alfred G. Brandstein, Chief Scientist of Marine Corps Operational Development Command (MCCDC), and Dr. Gary E. Horne, Chief Scientist of MITRE Corporation in 1996. In the papers the two scientists, data farming is regarded as a meta-technique for research issues in the 21st century. From invention, the data farming has aroused widespread interest from the military forces of various countries and has produced a large number of research results. It is still in further research and development. In contrast, data mining technology is a mature theory and application, and it is also a popular method of processing massive data. (4) Intelligent analysis technology based on simulation big data Intelligent analysis technology based on simulation big data is another topic in effectiveness evaluation and analysis. The simulation big data is obtained through large-scale simulation deductions and experiments. It includes the full-sample spatio-temporal data generated by deduction in sea, land, air, and sky multi-dimensional operation space. The large scale of simulation entity makes it possible to excavate the SoS architecture. This method records the data of a wide variety of entities and the complex interactions with huge data, which provides the possibility for the correlation analysis of the effectiveness evaluation index system. The process and result of the simulation deduction are fully recorded, which provides the possibility to show the uncertainty and emergence that are difficult to see.

24

Deping Zhang and Xuefeng Yan

Therefore, the intelligent analysis technology based on simulation big data provides new ideas for the effectiveness evaluation of the SoS.

Chapter 3

Construction of Evaluation Index System The effectiveness evaluation index system is a series of interconnected essential attribute which forms an integrated organism. It also is the criterion, purpose, direction, the content and the embodiment of the evaluation. The main performance parameters of operational capability of equipment can be described using index system. It is a comprehensive framework of the tactical and technical performance, quality, and support characteristics. In effectiveness evaluation, effectiveness are often measured by multiple indexes. It is almost impossible to design an optimal solution for all indexes in practice. Therefore, the evaluation scheme should be selected based on the comprehensive evaluation of multiindexes. Although it may not be optimal for each single effectiveness index, it is a good choice for comprehensive effectiveness evaluation. It is difficult to evaluate a real system only using single index. It is necessary to apply various evaluation schemes to make scientific and effective evaluation because some conflicts may be caused by the indexes in different schemes. Multi-indexes evaluation means that the evaluation subject collects several indexes information according to its own preference, so as to understand the pros and cons of the evaluation object under a certain standard on the whole. Its basic idea is to reflect the overall picture of the evaluation object, which is necessary to organize multiple indexes to form a comprehensive index system with various aspects. It is represented by the comprehensive index system. The comprehensive index system is composed of a series of indexes in the evaluation activities. And it is the scale set for the comprehensive measurement of the evaluation objects. As we all know, the index system is widely used in the systematic analysis of society, economy and management science. If the evaluated system can reflect the requirements of the each goal objectively and comprehensively, various schemes and projects can be compared and evaluated, then a scientific and reasonable scheme can be selected. In the practical evaluation, the evaluation index system often has a hierarchical structure and includes amount of objects and criterias. The overall goal of the top level of the evaluation index system is only one, for example, the pros and cons of the operational scheme. It is generally vague, ambiguous, and abstract, which is not easy to quantify, measure, compare, and judge. Therefore, the overall goal should be decomposed into criteria and sub-criteria

26

Deping Zhang and Xuefeng Yan

at several levels until it is quite specific, intuitive, and can be represented directly or indirectly by the index attributes and parameters? The lower-level criterion is more specific than the upper-level, which is also convenient to compare, judge and measure. They can be used to achieve the goal of upper level criteria. The lower-level sub-criteria set must ensure the realization of the upper-level. The sub-criteria may be consistent or contradictory, but it must be coordinated with the overall goal and minimize the redundancy.

3.1. Selection Principle of Evaluation Indexes The selection of evaluation indexes is directly related to the conclusion of the comprehensive evaluation. Any index can reflect some information about the evaluation object from one side view. And it is the most important work to decide how much and what kind of index to choose. According to the practice of research and the basic theory of system engineering and operations, when an evaluation index system is constructed for the comprehensive integration level of operational systems, several principles should be followed, which are shown as follows: Completeness: It’s necessary to meet the military requirements under the conditions of winning the informational warfare and the requirements of near, medium and long term planning of equipment system construction. It’s necessary to consider all aspects of the integration of the operational system comprehensively, so as to reflect the comprehensive effectiveness level. Objectivity: The determined evaluation index should reflect the essential characteristics of the impact of operational system integration on system capability. Scientificalness: Because there are many factors that affect the construction of comprehensive integration, it is necessary to grasp the main factor and ignore the secondary. As a result, it not only makes the evaluation index system relatively simple, one while not affect the essence of effectiveness evaluation. Systematicness: According to system theory, the index system should be regarded as an integrated system. The characteristics and status of the evaluated object need to be reflected from all levels and views. And the change and development trend of the object can also need to be considered. Practicality: The difficulty and reliability of index quantification should be fully considered and the calculation data should be taken as the basis. If the index system is not practical but only comprehensive and refined in theory, it is equivalent to nothing. The index system construction is a complicated process. The basic requirements are shown as follows: (1) The index design should cover the main factors that have an impact on the integration of the operational system. And it is closely related to the main performance parameters of the operational system; (2) The index design should be applicable to achieve quantification, departmentalization and industrialization. It should focus on quantitative indexes as much as possible and make qualitative indexes quantitative as far as possible. (3) Generally, the indexes are not allowed to overlap and contain each other.

Construction of Evaluation Index System

27

(4) The essential characteristics of the evaluation objects should be effectively reflected by the index. (5) The index system should be as simple as possible and each index value should easy to be calculated.

3.2. Construction of Index System It is difficult to establish an evaluation index system. Generally speaking, the wider the range of indexes is, the larger the number of indexes is, and the more obvious the differences between the schemes are, which is conducive for judgment and evaluation. At the same time, it becomes more difficult is to determine the content and importance of the indexes, the processing and modelling processes are more complex, then the possibility of misrepresenting the nature of the scheme is greater. The requirements of the each objective of the system can be comprehensively reflected by the evaluation index system. The scientificity and rationality should be achieved as far as possible, and the actual situation should also be consistent. It can be accepted by the relevant personnel and departments basically. Therefore, the establishment of evaluation index system should be based on the comprehensive analysis system. A tender should be drafted at first. Then a series of works will be implemented, including extensively soliciting the opinions from experts and relevant departments, repeatedly exchanging the information, statistical processing and comprehensive induction and so on. Finally, the evaluation index system can be determined. The general idea of the index system construction follows the cycle of ”concreteabstract-concrete”, and its construction is an iterative process. The process is roughly shown in Figure 3.1. The initial construction of the index system mainly includes the following eight steps: Step 1. Evaluation Purpose Analysis. The purpose is the premise of constructing the evaluation index. The hierarchical architecture of the evaluation purpose is the foundation of the hierarchical structure of the index system. The purpose is the goal and expectation the evaluation wants to achieve. The requirements and expectations of the operational system are multifaceted and reflected in the operational goals. Step 2. System Analysis. The systemic viewpoints and methods are adopted in the system analysis. The results of simulations need be analyzed to make clear related factors and their relationships. Step 3. Characteristic Attribute Analysis. The characteristics of each component need to be analyzed. The suitable indexes can be constructed and the essential attributes of each index can be clarified. Therefore, a foundation can be established for the establishment of mathematical models and the acquisition of evaluation data. The index attribute can be used to denote whether each index is qualitative or quantitative, static or dynamic. The qualitative indexes refer to the indexes that cannot be described in quantitative quantities. The quantitative indexes refer to the indexes that can be described in specific quantities through analysis and calculation. The static indexes refer to the indexes that do not change with time, environmental conditions, and other factors. The dynamic indexes refer to the indexes that change with time, occasions and other conditions.

28

Deping Zhang and Xuefeng Yan

Figure 3.1. The flow chart of index system construction. Step 4. Structure Analysis. The structure of different evaluation index systems is decided by the goal architecture. There are two common structure. One is the hierarchical evaluation index system. According to the purpose of the evaluation index system, the corresponding evaluation index system is established by analyzing the functional level, structural level and logical level of the system. The other is networked evaluation index system. In the system with complex structure the network, if it is difficult to separate the evaluation indexes or the system evaluation model has not been determined, a networked evaluation index system could be adopted or partially adopted. Step 5. Analysis of the Information Sources. The sources of index information usually include relevant databases, statistical analysis, expert consultation and subjective statistics. Step 6. Weight Analysis. Weight is a measure of how much an element contributes to the goal of upper level. Through the weight analysis, the status and influence of each index can be obtained by comprehensive evaluation. Step 7. Normalization. Normalization is the foundation of comparison between indexes and the premise of comprehensive evaluation. Step 8. Preliminary Evaluation Index System. After the above works, a preliminary and practical evaluation index system can be formed. After the preliminary index system

Construction of Evaluation Index System

29

is formed by consulting experts and testing in practice, a satisfactory comprehensive index system can be obtained by extensively consulting with the experts, business agencies and relevant personnel and testing in practice. The construction of the index system is usually the foundation for system effectiveness evaluation. At present, many methods for constructing the index system are improved based on the traditional analytic hierarchy process. However, with the new features brought by the networked SoS and new assessment methods brought by big data, traditional methods are facing some challenges[54], [55], which is mainly reflected in the following aspects: Firstly, with the deepening of the understanding of the index system, the principles proposed by researchers have partly reflected the basic characteristics of the index system and the new changes in the SoS. But it is still necessary to further explore more guiding principles that can reflect the characteristics of the networked SoS. Secondly, in recent years, with the introduction of big data, a large amount of data has been accumulated in operational training and equipment development, but the current methods are mainly based on small data and expert qualitative judgment. Thirdly, most of the existing methods are still static analysis based on the hierarchical analysis. It is difficult to represent the basic characteristics such as the correlation between indexes, the aggregation relationship between levels and the comprehensiveness of indexes. Finally, the index system constructed by the existing methods can only meet the unilateral evaluation under static conditions, but cannot meet the evaluation needs of dynamic, overall and confrontational conditions.

3.3. Construction of Evaluation Index System There are many ways to construct the index system, which can be decomposed from the general evaluation purpose to the basic parameter index top-down, or aggregated from the basic index bottom-up, or carried out from both directions at the same time. The method of constructing a system can be referenced by a comprehensive evaluation index system. This includes the selection of core elements, as well as the planning design of the overall framework. Therefore, it is indispensable to choose specific indexes and design the overall framework.

3.3.1.

Construction Model of Evaluation Index System

The effectiveness evaluation index system is often developed based on a certain structure. It is necessary to analyze and summarize the structural elements to construct an abstract effectiveness evaluation index system. Generally, the structural elements of weapon equipment effectiveness are analyzed from the related concepts of the effectiveness measurement. Combined with the expected mission, when the system performs a mission in the expected conditions, the ability of the system can be appropriately measured according to the hierarchical relationship of various index factors and the connections, such as the effectiveness measurement, operational capability, and suitability [56]. The effectiveness can be further defined as:

30

Deping Zhang and Xuefeng Yan

Operational effectiveness is a unified expression of the operational capability, suitability, and effectiveness of the equipment in an operational environment. Therefore, combined with the tree-level evaluation structure, the operational effectiveness structure is formed by the combination of various index factors, which is shown in Figure 3.2.

Figure 3.2. Operational effectiveness structure diagram of the equipment. The operational effectiveness structure covers the main construction pattern of the current effectiveness evaluation indexes. There are several indexes that are related to the effectiveness, such as operational effects, inherent capability, operational capability, operational adaptability, operational situation, and system performance index factors. Consequently, the following six independent construction patterns of the evaluation index is shown in Figure 3.3.

Figure 3.3. The construction pattern of the evaluation index system.

Construction of Evaluation Index System

31

The six construction patterns of effectiveness evaluation index system are introduced. (P1) The construction pattern based on operational effect The construction pattern P1 is based on the operational effect and can be used to obtain the absolute value of effectiveness or the evaluated utility value. In general, the index system used in the operational simulation and the exercise test usually belongs to this pattern. The first advantages is that the evaluation index is easy to understand, so that the evaluation conclusion can be directly corresponding to the operational effect. The second is that the evaluation data can be easily obtained from offensive and defensive simulation or exercise test. However, it is difficult to construct a comprehensive effect-oriented index system, which is usually focus on comprehensive operational effect index under specific circumstances. (P2) The construction pattern based on inherent capabilities The operational effectiveness is analyzed based on inherent capability under the construction pattern P2, from which the relative degree or satisfaction probability of the effectiveness can be studied. The performance evaluation indexes are often calculated by some methods, such as expert evaluation, comprehensive evaluation based on multi-project comparison and fuzzy evaluation method based on experts. The advantage is that the index is comprehensive, simple and shortcut. The disadvantages include more subjectivity and less objectivity, and the evaluation mainly depends on the empirical judgment and speculative estimation of experts. (P3) The construction pattern based on operational capability and suitability The factors of operational capability and suitability are included in P3. The overall value of operational effectiveness can be obtained. The P3 has the advantage of comprehensiveness, which can take into account operational and non-operational capability elements. In fact, the P3 is applicable to availability, dependability and capability (ADC) models and their corresponding improved models. At this time, the operational suitability can be regarded as the operational capability of A and D ·C. The ADC is easy to understand and with distinct physical meaning. However, the computational complexity of operational suitability will be increased sharply with the increase of nodes number. And then the cumulative error is amplified by the multiplications of the ADC matrixes. (P4) The construction pattern based on system performance, operational effect and operational capability In the pattern P4, the operational capability and effectiveness are analyzed by the system performance and effectiveness indexes respectively. It is important to study the relationship between effectiveness and capability, and analyze the key factors that affect the weapon effectiveness. The single operational capability is the key factor for comprehensive effectiveness. The effectiveness evaluation index system can be used to calculate the operational effectiveness and capability. In addition, it is also helpful to study the causality among the factors of operational effectiveness, capability and performance. (P5) The construction pattern based on system performance, inherent capability, operational capability, and operational suitability

32

Deping Zhang and Xuefeng Yan

The purpose of the pattern P5 is to study the influence spreading chair among system performance, inherent capability, operational capability, effectiveness and situation of both sides. Several influences should be analyzed, such as system performance on inherent capabilities, inherent capabilities on operational capabilities, inherent capabilities on operational effectiveness, and single operational capabilities on effectiveness. The index system of this pattern emphasizes the comparative analysis of effectiveness based on the physical characteristics and the prediction of the potential capabilities. There are several characteristics of the pattern P5, such as expert assessment, mixed evaluation of adversarial simulation tests, and multi-level evaluation. The system performance index can be obtained by physical test and the operational capability can be evaluated by simulation and expert. Finally, multiple data sources fusion is adopted to evaluate the comprehensive operational effectiveness. (P6) The construction pattern based on system performance, inherent capability, operational suitability, situation and capability The purpose of the pattern P6 is to study the influence spreading chair among system performance, inherent capability, operational capability and operational effectiveness. When the situation of both sides changes synchronously, there are several influences should be analyzed, such as system performance on inherent capabilities, inherent capabilities on operational capabilities, inherent capabilities on operational effectiveness and single operational capabilities on operational effectiveness. In P6, effectiveness analysis is based on full considering the physical characteristics of equipment and the influence of operational situation. The logical structure and process of operational effectiveness are integrated in the P6, and the comprehensive index system is constructed by taking dynamic and static aspects into consideration. The construction pattern of evaluation index system is not only the abstract class of specific index system, but also the basic framework of detailed index system. According to the above patterns, it is very important to choose the appropriate one to build up evaluation index system according to different backgrounds.

3.3.2.

Selection of Construction Pattern

When constructing the effectiveness evaluation index system, we face not only the problems of rationality, but also time lines. In order to formulate a reliable evaluation index system quickly and reasonably, a set of rational evaluation index system construction pattern should be selected at first, and then the evaluation index system meet the requirements should be refined based on the selected pattern as the basic framework. The selection is important for connecting the same or similar equipment effectiveness evaluation with various construction patterns. What kind of evaluation index system should be adopted for a certain equipment under certain conditions is exactly the selection problem. In order to evaluate the effectiveness with various types and complicated conditions, it is necessary to predict the maturity of each mode based on its own characteristics and evaluation purpose combined with evaluation data and model. Therefore, the construction pattern is selected to guide the refinement of the index system. To evaluate the operational effectiveness, it is necessary to input the characteristics, the requirements of operational mission and the required evaluation maturity based on its

Construction of Evaluation Index System

33

own characteristics. Then it can match with the corresponding evaluation index system construction pattern. Generally, the construction pattern can be selected based on some experience, which is described as follows: Firstly, if the effectiveness is directly analyzed by over all operational effects, the operational effectiveness can be directly reflected according to the final operational results the specific performance and single capability of equipment is not important. Secondly, if the effectiveness is focused by the evaluator, the construction pattern P1 can be selected. If it is necessary to investigate the inherent capability, the construction pattern P2 can be adopted. If it is necessary to examine the operational process and suitability, the construction pattern P3 can be considered. If it is necessary to examine the operational effect, capability and suitability of equipment, the construction pattern P4 can be considered. If there are several influencing factors that are taken into comprehensive account, such as the operational effectiveness, suitability and capability of equipment, the construction pattern P5 can be selected. If the operational situation, the effect, suitability, and capability are the main factors, the construction pattern P6 can be adopted. Finally, if there are other considerations, the pattern can be appropriately adjusted based on the basic construction pattern above.

3.3.3.

The Construction Process of Hierarchy Evaluation Index System

The hierarchical expansion in system engineering can be referenced by the process of instantiated decomposition according to the construction pattern. Based on the requirements of the evaluation system or the construction pattern of the index system, the corresponding effectiveness evaluation index can be decomposed layer by layer. The system hierarchy is usually decomposed by a set of top-down or bottom-up approaches.It usually starts from a world view (expressed as V ) and is refined to the specific interested areas. In specific interested area, the target system elements (such as data) should be analyzed and then the functionality can be decomposed. The hierarchy expansion diagram is formally described as follows: The world view V includes a set of fields Di , where Di is a subsystem, which can be expressed as follows: V = {D1 , D2 , · · · , Di , · · · , Dn }, i = 1, 2, · · · , n

(3.1)

Each field Di is composed by specific elements {Ei j }, the completion of the domain goal role is undertaken by each element: Di = {Ei1 , Ei2 , · · · , Ei j , · · · , Eim }, j = 1, 2, · · · , m

(3.2)

Each element Ei j is implemented by technical component (Ci jk ) that perform the necessary functions of the element: Ei j = {Ci j1 ,Ci j2 , · · · ,Ci jk , · · · ,Ci jl }, k = 1, 2, · · · , l

(3.3)

In order to refine the evaluation index system (also known as instantiation according to the construction pattern), it is necessary to draw lessons from the system engineering hierarchical structure decomposition layer by layer. The process is shown in Figure 3.4.

34

Deping Zhang and Xuefeng Yan

Figure 3.4. The evaluation index construction refinement process. Generally, there are the form of hierarchy and tree in the index system that is obtained by layer-by-layer decomposition. Specifically, in the actual evaluation, the corresponding index system construction pattern is selected by the different evaluation purposes. The effectiveness is expressed by one or several important factors that are concerned in the construction pattern. Take P3 as an example, operational capability and suitability are regarded as comprehensive indexes of equipment effectiveness. Actually, the index system is constructed by two comprehensive indexes in pattern P3, which is based on the idea of the ADC. The operational capability is C and the suitability is the A and D. (1) The Decomposition of Operational Capability Indexes The operational capability of weapon equipment can be regarded as the ability or potential of equipment under certain operational conditions. The equipment system usually has five capabilities including detecting, hitting, directing, defending and resisting, or some of the above five capabilities. “Detecting” means the capability of equipment to detect target, which includes the capability to find, identify, track, and measure targets in general. “Hitting” means the capability to “hit” or “damage” the target. The damage effectiveness is usually adopted as measurement index, which includes target control capability, hit accuracy, target vulnerability, and strike reliability. “Directing” means the the command and control capability of equipment, which usually includes the capabilities of information acquisition, information processing, information monitoring, and information transmission. “Defending” means the survival capability when facing the threat of the enemy, which usually includes the mobility, reaction, anti-detection, anti-destruction ability.

Construction of Evaluation Index System

35

“Resisting” means the electronic countermeasure capability, which usually includes the electronic interference and anti-electronic interference ability. The decomposition map of the operational capability indexes is shown in Figure 3.5.

Figure 3.5. The decomposition of the operational capability indexes. (2) The Decomposition of Operational Suitability Indexes The Test and Evaluation Management Guide, published by the U.S. Defense Acquisition University, defines the operational suitability as “the degree to which the system can be deployed and maintained in the external fields satisfactorily when considering the requirements of availability, compatibility, transportability, interoperability, reliability, operational usage rate, maintainability, safety, human factors, habitability, manpower, logistical supportability, natural environmental effect and impacts, documentation and training”. Generally speaking, operational suitability is the degree to which the equipment meets the requirements of troop training and operational application in the actual environment,

36

Deping Zhang and Xuefeng Yan

including the environment applicability and operational usability. Environment applicability is the applicability of the operational environment including the natural environment, the electromagnetic environment, the transport environment, and the man-machine environment. Operational usability refers to the practicability in the operational environment. It usually includes some properties of equipment, such as reliability, maintainability, testability, standardization, safety and supportability, etc. In summary, the decomposition of the operational suitability indexes is shown in Figure 3.6.

Figure 3.6. The decomposition map of operational suitability indexes.

3.3.4.

Index System and Optimization

The comprehensiveness of the index system is always a key point pursued by the designer. However, on the one hand, excessive evaluation indexes will lead to the illusion and confusion of judgment. On the other hand, it will lead to the weight reduction of other indexes and result in the distortion of evaluation. Different index systems can be designed from different perspectives for the same evaluation objects. And it is difficult to choose an effective index evaluation. The index system is usually selected by researchers based on experience, which is lack of scientificity and strictness. The evaluation results of the same evaluation index system are different because of the different understanding of the evaluation indexes by the experts, which means that the index system lacks stability and reliability. The final evaluation result will be affected by these issues. It is obviously that if the evaluation index system is not set properly, the result will be distorted even with a scientific and advanced evaluation method. Therefore, it is necessary to optimize it after the completion of the index system design. In this section, the screening of the index

Construction of Evaluation Index System

37

system and the evaluation of its effectiveness and reliability will be discussed. (1) Index System Screening It is not better to use more evaluation indexes in practice. The key point is to select proper indexes which can reflect the essence of evaluation. The general principle is to use a few “main” indexes in practice. However, there are some “secondary” evaluation indexes in the index set. It is necessary to distinguish the importance of these indexes, and constitute the evaluation index set reasonably according to the rationality judgment. It is available to use weight judgment in the actual evaluation. According to the index weight, it is necessary to eliminate some indexes with smaller weight, which can not only simplify the decisionmaking problem but also avoid the error and the judgment confusion caused by factors redundancy. The specific steps are as follows: Assuming that the index set in the same layer is {I1 , I2 , · · · , In }, the corresponding weight set {w1 , w2 , · · · , wn } is obtained after considering the importance and weight of each index, where wi ∈ [0, 1], i = 1, 2, · · · , n. Assuming that the tradeoff weight is wk , wk ∈ [0, 1], the index Ii is filtered out when wi ≤ wk , and index Ii is remained when wi > wk . The tradeoff weight depends on the evaluator and the complexity of the evaluation objects. The tradeoff weight is small when there are many factors involved in the evaluation objects, is large when the factors are few. The evaluation weight is objectively utilized by evaluators to appropriately simplify the evaluation index system. (2) Index System Validity In the evaluation, when the same index system is adopted for the same evaluation objects, different values will be obtained due to the difference experts. When the evaluation values are quite different, it is considered that the index system cannot truly reflect the evaluation purpose and should be eliminated. Similarly, the design process of evaluation index system often involves the validity problem. The authenticity and the reflection of the results should be confirmed. For the above problems, the validity coefficient is adopted in this book. It is assumed that the index system is {I1 , I2 , · · · , In }. The number of experts for evaluation is j. The grading set of the expert j for the evaluation object is X j = {x1 j , x2 j , · · · , xn j }, j = 1, 2, · · · , S. The validity coefficient Vi of the index Ii is defined as follows: S

Vi =

|x¯i − xi j | ×M S j=1

∑

(3.4)

where M is the optimal value in the evaluation set of index Ii . and x¯i is the average value of the evaluation index Ii , that is, S

x¯i =

∑ xi j /S

(3.5)

j=1

Then the validity coefficient V of the index system is defined as, V=

1 n ∑ Vi n i=1

(3.6)

38

Deping Zhang and Xuefeng Yan

The statistical significance of the effectiveness coefficient index is that it provides a measure of the cognitive deviation when people evaluate the target with a certain index. The smaller the absolute number of the index is, the more consistent the experts’ understanding of the problem is when evaluating the target, and the higher the effectiveness of the evaluation index system or index is, and vice versa. (3) Stability and Reliability of Index System It is assumed that the nature of the evaluating objects can be completely reflected by a group of ideal evaluation data. If the evaluation data obtained by the index system is more “similar” to these ideal data, it will be closer to the essence of the evaluating objects. And the evaluation index system will be more stable and reliable. Based on this idea, the correlation coefficient in mathematical statistics is used as the reliability coefficient to reflect the reliability and stability of the evaluation index system. It is assumed that the average data set of experts evaluation is represented by Y = S

{y1 , y2 , · · · , yn }, y j = ∑ xi j /S, the reliability coefficient of the evaluation index system is i=1 ρ: ρ=

1 S

S

∑ ρj

(3.7)

j=1

where n

ρj = r

¯ ∑ (xi j − x¯ j )(yi − y)

i=1 n

∑ (xi j − x¯ j

i=1

and x¯ j =

1 n

n

∑ xi j , y¯ = i=1

1 n

)2

n

∑ (yi j − y¯ j

)2

, j = 1, 2, · · · , s

(3.8)

i=1

n

∑ yi . i=1

The average value of the S evaluation results by the index Ii is regarded as expectation value and the variance can be obtained by calculating the difference between evaluation value and average value. The variance can reflect the difference of S evaluation results with the same index system. The larger ρ is, the smaller the difference is and the more reliable the index system is. The reliability coefficient can measure the reliability and stability of the index system evaluation results. The smaller ρ is, the larger the evaluation difference is with the same evaluation index by experts. In index system, it is quite different between the experts’ evaluation results for the same object. Therefore, if ρ is small, it is not suitable to use this evaluation index system, and its reliability is poor.

3.3.5.

Case Study

In this section, the application rationality of the proposed evaluation index system construction is illustrated by the air defense effectiveness evaluation of aircraft carrier formation [57].

Construction of Evaluation Index System

39

(1) Preparation Conditions of the Effectiveness Evaluation The evaluation conditions are as follows. The evaluation object is the aircraft carrier formation SoS composed of aircraft carriers, cruisers, destroyers, submarines, etc. The purpose is to obtain a comparable comprehensive evaluation of the air defense operational effectiveness of aircraft carrier formation. There are several task properties, such as attack, deterrence and interception. However, its fundamental task is to defend the formation or other surface warships in the air defense area by killing incoming air targets. (2) Select the Construction Pattern of Corresponding Evaluation Index System Based on Evaluation Conditions The studied object is the air defense effectiveness of the aircraft carrier formation. Because the aircraft carrier formation system is huge and complex, the air defense effectiveness evaluation index system cannot be constructed by a single operational effectiveness. Therefore, the construction pattern for comprehensive index system should be adopted and the effectiveness logic structure and the aircraft carrier formation’s air defense process should be combined. The comprehensive index evaluation system is constructed from both the dynamic and static aspects. After the analysis, the effectiveness evaluation index system pattern P6 is selected. Then, the air defense effectiveness is analyzed by the index system with the system performance, inherent capability, operational capability, operational suitability and operational situation. (3) Instantiation Based on the Selected Construction Pattern In general, the construction of the effectiveness evaluation index for the aircraft carrier formation can be divided according to the formation, functions, missions, and other way. Here, the indexes are refined according to the process of mission. The description of the given mission can be satisfied in the effectiveness definition. At the same time, the operational process can be analyzed. The situations and results of aircraft carrier formation air defense operational are also analyzed in all stages. It is also beneficial for the development of the simulation test design and the collection of evaluation data. The specific steps are shown as follows: According to the mission and the characteristics of the aircraft carrier formation air defense operation, the effectiveness can be reflected by the process of attack-defense confrontation with air targets. Several capabilities are included, such as reconnaissance and early warning, command and control, fire strike and electronic warfare. The performance is based on the stable and smooth of the operational process. The operational process is a comprehensive process, which is formed by the information flow, the control flow, the energy flow, etc., which is shown in Figure 3.7. The aircraft carrier air defense operational process is divided into three stages including mission, preparation, and implementation. Several events are included in the mission stage, such as receiving the superior operational missions, understanding the superior operational intentions correctly, formulating the aircraft carrier formation missions, analyzing the situation, and making preliminary determinations when the enemy targets invade. After receiving the mission, several activities are included in the operational preparation stage, such as analyzing the operational situation, formulating the action and scenario plan of the aircraft carrier formation based on the mission requirements and the chief’s operational in-

40

Deping Zhang and Xuefeng Yan

Figure 3.7. The operation process of aircraft carrier formation. tentions. Several operational preparations are contained in this stage, such as the action plans, the equipment technology preparation, the logistics support preparation, etc. Several actions are contained in the operational implementation stage, such as reconnaissance and early warning, tracking and identification, formation maneuvering, fire strike (such as, carrier-based aircraft interception, ship-borne weapon interception), electronic counterattacks, and the effectiveness evaluation. (4) Design the Detailed Evaluation Index System According to the construction pattern P6, the operational process described in the Figure 3.7 is supplemented and detailed based on the instantiation of the evaluation index system. The operational situation analysis includes the enemy aircraft and ship attacks. The operational preparations reflect the inherent capability of the aircraft carrier formations including the operational readiness, reliability and formation survivability. The formation detection and identification capability for air, ultra-low altitude, and special targets are checked by the early warning detection and tracking identification. The formation information processing and decision-making capability are checked by the command and control. The response measures include the maneuverability of formation, the fire strike capability, and the electronic warfare capability. In the process of attack and defense, the stability and variability are checked by the formation maneuverability, which is an important parameter for evaluating the aircraft carrier’s air defense effectiveness. It checks the maneuverability and coordination of the formation ship. Therefore, it can be regarded as the inherent capability of the formation. In summary, the hierarchical structure of the air defense effectiveness evaluation index system can be obtained for the aircraft carrier formation [43], which is shown in Figure 3.8. Firstly, the hierarchical structure diagram of the air defense operational effectiveness index system is taken as the basis. Secondly, the upper level index is refined according to the hierarchical decomposition of system engineering. Finally, optimization and improvement are carried out. The details are shown as follows: (i) The operational situation The operational situation mainly includes three aspects, which are the enemy aircraft attack situation, the enemy ship attack situation and the update capability of comprehensive situation. The index factors that affect the operational situation mainly include the attacking direction of the enemy aircraft/ship, the attack batch, the attack interval, the missile launch distance, and the number of missiles launched in one attack. The influencing factors of the

Construction of Evaluation Index System

41

Figure 3.8. The hierarchical structure of the air defense effectiveness index system for the aircraft carrier formation. comprehensive situation update capability include the comprehensive situation generation and update time, whose sub-index composition is concretely shown in Figure 3.9.

Figure 3.9. Composition of the enemy attack situation.

42

Deping Zhang and Xuefeng Yan (ii) The operational suitability

According to the characteristics of the air operational, three aspects are mainly considered for the suitability including the equipment reliability, the operational readiness and the survivability. The reliability mainly focuses on the average failure interval time, the average failure repair time and the continuous failure-free working time. The operational readiness considers the technical and standby readiness rate. The survivability mainly considers the protection capability, the anti-reconnaissance capability, the anti-destroy capability of the formation, etc., whose sub-index composition is concretely shown in Figure 3.10.

Figure 3.10. Composition of the operational suitability. (iii) The formation mobility The formation maneuvering ability is mainly affected by the maneuverability of the carrier and the formation. The influencing factors of the carrier are the maximum speed, the weight of the hull, the height of the gravity center and the maximum steering angle. The influencing factors of formation include of formation keeping ability, adapting to the the operational environment, limiting ability against the maneuverability of the enemy aircraft, etc., whose sub-index composition is concretely shown in Figure 3.11. (iv) The reconnaissance and early warning capability The reconnaissance and early warning capability includes the early warning detection, the tracking and identification. The influencing factors of the early warning detection capability include the position, the detection distance and the anti-jamming capability of the early warning aircraft and the anti-aircraft guard ship. The influencing factors of the tracking and identification capability include two aspects. The first part includes the target processing, the command and guidance, and the anti-jamming capabilities of the early warning aircraft. The second part includes the radar tracking distance, the target processing and the anti-jamming capabilities of air defense warship’s capabilities. The composition of the sub-index is concretely shown in Figure 3.12.

Construction of Evaluation Index System

43

Figure 3.11. Composition of the formation maneuverability. (v) The command and control capability The command and control capability focuses on three aspects including the capabilities of the formation control and decision-making, command and control response, and the assistant decision-making. The first one include the operational plan generation time, the decision response time, and the decision-maker ability. The second one include the issuing time of the super command and control order, issuing time of the synthesis command and control order, and issuing time of the emergency command and control order. The main factors of the assistant decision-making capability are the intelligence processing, the network communication, and the decision-making correctness. The composition of the command and control capability sub-index is concretely shown in Figure 3.13. (vi) The electronic countermeasures capability The electronic countermeasures capability can be mainly analyzed from three perspectives, including the electronic reconnaissance, the electronic interference, and the electronic defense. The main influencing factors of the electronic reconnaissance capability are the detection, transmission and identification of the information. The influencing factors of the electronic interference capability are the information suppression, the information deception, and the radar active/passive jamming effect. The influencing factors of the electronic defense capability include the threat alerting, the self-defense interference, the information system anti-destructivity and etc. The composition of the sub-index is concretely shown in Figure 3.14. (vii) The counterattack capability The counterattack capability can be analyzed from two aspects including the carrierbased aircraft interception and the carrier-based weapon interception. The index factors of affecting the effectiveness include the counterattack capabilities of carrier-based aircraft,

44

Deping Zhang and Xuefeng Yan

Figure 3.12. Composition of the reconnaissance and early warning capability. air-to-air missiles, air missiles and the anti-aircraft artillery, whose sub-index component is concretely shown in Figure 3.15.

3.4. Evaluation Index System Construction 3.4.1.

Characteristics of SoS Effectiveness of Evaluation

At present, the construction and selection of the index system of systems effectiveness evaluation are mostly based on the traditional decomposition and addition reductionism, without fully considering the complex characteristics of the SoS. Therefore, there are some shortcomings in the selection principle, scope and method, which are mainly reflected in the following aspects: (a) In the selection principle, the objectivity, completeness, independence and sensitivity are required by the traditional factors. However, the characteristics of dynamic confrontation, continuous evolution, emergence of effectiveness, and chaotic state are possessed. Therefore, the requirements cannot be satisfied by the traditional selection principles (b) The traditional evaluation is usually based on a hierarchical tree evaluation index system framework that is aggregated from the bottom to the top. However, the decomposition will cause the loss of properties since the complex systems are indecomposable and non-additivity. And the overall property cannot be derived from the local property. Therefore, the requirements for overall evaluation cannot be satisfied by the framework. (c) In the scope, the traditional factors and the index screening scope are limited to the traditional parameter space such as the performance, action and effectiveness of the operational component, and do not take the key factors that lead to the emergence into

Construction of Evaluation Index System

45

Figure 3.13. Composition of the command and control capability. full consideration, including the network properties, structure and evolution of the system. Therefore, there are the problems of incomplete and narrow scope. The complexity of operational system determines that the overall effect must be fully considered in the evaluation, and the characteristics of complex system must be highlighted. The factors and indexes must be selected under the conditions of dynamic, overall and confrontation. The following principle should be highlighted. (1) Emergence Effectiveness The SoS emphasizes on “whole” and “emergence”. In the evolution, the emergence reflects the overall effect. The effect of the SoS is the overall operational capability, which is formed by the interconnection and interaction of various equipment, operational units and capabilities with the support of the complex network. It cannot be reflected by the simple combination of a weapon, an operational unit and one type of operational system capability. Therefore, in the selection of factors, it is necessary to analyze the mechanism of the SoS overall effect, and take the factors that may cause the emergence efficiency into consideration. In the indexes selection, it is necessary to take the indexes which can represent the operational system effectiveness into consideration. (2) Structure Evolution The operational SoS is a “living” system with adaptability and evolution. The structure, function and property are dynamic, and the capability will change dynamically during the confrontation process. Therefore, in the selection of factors and indexes, more attention be paid to the combination of dynamic and static properties. It is necessary to consider not only the changes of static factors, such as the weapon range and the flight speed, but also the changes of the dynamic factors, such as the network structure and coverage, especially the network structure. Also, not only the static indexes, such as the total rate and number of interceptions, but also the dynamic indexes, such as the OODA ring duration and the number of network active nodes, should be considered.

46

Deping Zhang and Xuefeng Yan

Figure 3.14. Composition of the electronic countermeasure capability. (3) Relativity of the Capability The operational capability is demonstrated through the confrontation. The capabilities are also different with different opponents. It is the ability of “a certain moment” and “a certain opponent” under certain conditions. Therefore, in the selection of factors, it is necessary to create conditions for the capability comparison, such as the changes under the different opponents and weapons. It is necessary to select the index that can reflect the capability relativity, such as the destructiveness and vulnerability against the different opponents.

3.4.2.

Index System Framework

The SoS is a typical complex system. From the perspective of the emergence, it mainly comes from several sources, including the component system, the component structure and their interactions. The overall ability evaluation must be based on the component system, but the effective measurement cannot be obtained only by evaluating component system, because the overall emergence effect is generated by the combined influence of the component and networked effectiveness. In this book, the effectiveness evaluation of the component systems is regarded as foundation, the mission measurement is taken as traction, the overall emergence capability evaluation is regarded as the emphasis and the “networked” structure evaluation is taken as the key. Then, the networked capability evaluation framework is constructed and the reference index is proposed. (1) The Index System Framework of Effectiveness Evaluation The evaluation framework of networked capability is shown in Figure 3.16, which mainly includes measures of component performance (MOCP), measures of component effectiveness (MOCE), measures of networked effectiveness (MONE), measures of emer-

Construction of Evaluation Index System

47

Figure 3.15. The index composition of the counterattack capability. gence effectiveness (MOEE), and measures of task effectiveness (MOTE) [140]. In the figure, different shapes indicate different evaluation indexes at each layer. The dashed line represents that the indexes are related to each other. These indexes are depend on each other, which reflects the characteristics of “index network”. The solid line indicates the mapping relationship between the performance indexes at different layers. The MOCP layer refers to the performance indexes of the component system, e.g. the fire range, the hit accuracy, and the kill range of a missile. The MOCE layer is designed to measure the unit effectiveness, such as the fire success rate, the burst rate, the hit rate, and the survivability of the firepower systems. The MONE is used to evaluate the networked interaction degree between the component systems. These indexes include the various properties of the network, such as the degree distribution, the average distance, the agglomeration coefficient, and betweens. The evaluation index framework of the network structure means that the evaluation of capabilities is no longer a bottom-up aggregation. Therefore, the above indexes themselves are not so important in capability evaluation while the association among them are. The effectiveness evaluation should be measured from two aspects, namely, the MOTE and the MOEE. The purpose of the MOTE is to measure the degree to which the system achieves its ultimate goal. It evaluates the overall performance of the mission completion, and is the most direct and fundamental manifestation of the overall emergence of the capability. It is the fundamental criterion for optimization and is also the most concerned factor in the decision makers. It describes the overall performance of mission completion under the specific conditions. Some typical indexes used in traditional evaluation are still very important when studying the effectiveness. There are several indexes at this layer: operation results, operation damage, mission completion, operation loss ratio, advance speed,

48

Deping Zhang and Xuefeng Yan

Figure 3.16. The capability evaluation framework of networked SoS. operation time, and mission completion rate. The overall emergence can be measured by the MOEE. It emphasizes the overall emerged characteristics in structure, function and behavior, such as the robustness and vulnerability of the architecture, the new capabilities that are generated from the functional coupling of the component systems, the adaptation and synchronization behavior, etc. This layer mainly include sensor and attack synergy capability, survivability, fragility, gravity center, adaptability, operational synchronization, and confrontation OODA ring effectiveness. The capability can be deeply analyzed by the MOEE from the mechanism level. It reflects the SoS confrontation and capability generation mechanism. It is the deep exploration for reasons of the mission completion. The more comprehensive overall capabilities evaluation can be obtained by measuring the indexes of above two layers. In the actual experimental, it can also only focus on one layer. For example, if the robustness or the gravity center is emphasized, the capability evaluation can be performed by the indexes selected from the MOEE. (2) The Effectiveness Evaluation Index of Component System Although the SoS is not reducible, the capability of each component system is an indispensable organic component for the emergence. The effectiveness evaluation index of component system can be divided into the measures of component performance the MOCP and the measures of component evaluation MOCE. For the SoS, the performance of component system is the minimum element that is considered in capability evaluation. Each component performance of a system is analyzed and modeled by the MOCP. The performance index reflects a certain attribute of the system unit and is generally unrelated to the environment. In Figure 3.16, each triangle is corresponded

Construction of Evaluation Index System

49

to a MOCP index of a certain system unit. For example, several performance indexes are included in the sensor system, such as detection range, positioning accuracy, azimuth distance resolution, etc. The MOCE is used to measure the function performance of each system unit. The circle is corresponded to a component in that is generally measured by multiple effectiveness indexes. For example, there are several effectiveness indexes that can be adopted to evaluate the sensor system, including the detection effectiveness, the information processing effectiveness, the anti-interference effectiveness and the survival effectiveness. The system effectiveness Indexes MOCE i of any component si are shown as follows: MOCE i = {MOCE1i , MOCE2i , · · · , MOCE ij }

(3.9)

where the dimension of MOCE i is represented by j. For each specific dimension, the corresponding index is shown as followings: MOCE ij = f i, j (MOCPi) = f i, j (MOCP1i , MOCP2i , · · · , MOCPki )

(3.10)

where MOCPki represents the kt h performance index of the system unit si . The Equation(3.10) shows that the effectiveness of sub-system only depends on its owner system performance. And it is independent with the other components in the SoS. The MOCE ij of the sub-system are interrelated. For example, the stronger the antiinterference capability of the sensor system is, the higher the detection effectiveness may be. The effectiveness Indexes MOCE i of different components are also interrelated. For example, for the air defense and anti-missile, the effectiveness of the firepower interception system can be directly affected by the effectiveness of the early warning detection system. (3) The Measures of Task Effetiveness (MOTE) The MOTE is used to measure the degree to which the system completes its mission under specified conditions. The MOTE can measure the degree to which the SoS achieves its ultimate goals, and evaluate the overall mission. The hexagons in the Figure 3.16 correspond to different aspects of the missions. The MOTE is the most direct and fundamental manifestation of the overall capability emergence, and is the key concerned content for the decision makers. Any evaluation results must be summed up to the overall effectiveness of the whole mission. The MOTE also establishes the guidelines for the rationality and correctness of the evaluation index at other layers. It means that the conclusions obtained from evaluating capabilities at any level should be linked to the MOTE. If the effectiveness evaluation is analyzed without considering the mission, the conclusions must be incomplete and unconvincing. The evaluation mainly focuses on the mission-achievement effectiveness, e.g. the communication blocking rate, the active/passive defense rate, the mission completion probability, and the time required to complete the mission. The index is determined by the type and function of the SoS. Different SoS categories correspond to different effectiveness evaluation items. For example, perceptual effectiveness include positioning accuracy, target size, vulnerability discovery rate, key node discovery rate, etc. The attack indexes include penetration rate, interference rate, blocking rate, capacity, effective control volume, and etc. The defense indexes include the active/passive defense rate, the critical system and service recovery time, etc.

50

Deping Zhang and Xuefeng Yan

Taking combat SoS as an example, MOTE can adopt the typical indexes such as operational result, operational loss, operational loss ratio or exchange ratio, advance speed and operational time. In most cases, the mission effectiveness cannot be simply measured by the above-mentioned action effect indexes, but must be determined according to the purpose and the final system confrontation situation. For example, the mission completion degree and probability can be used as two effectiveness indexes mission in general. For the construction of MOTE, the core idea is to select the description component that can reflect the overall state of the operation as the index. The effectiveness is characterized from a macro perspective, and the index synthesis is not needed. The features are as follows: (a) The selection of indexes are based on the overall operation condition. The indexes are often a collection of operational data of both sides which have a decisive effect on the battlefield situation. (b) The action process and mission completion are important to the evaluation. According to the battlefield superiority theory proposed by the US military, i.e., counter effectiveness, protection effectiveness, battlefield superiority and mission completion. The evaluation focus on the ability to obtain the operational advantages and the ability to complete missions. (c) Generally, the indexes are not combined. The statistical analysis method is used to obtain the evaluation indexes results, and the evaluation indexes are not weighted and synthesized. (d) Combination of qualitative analysis and quantitative analysis. Some qualitative indexes can be selected to simplify the description. These indexes requires a combination of qualitative and quantitative methods with the wisdom and experience of evaluators. (e) There are two important data sources: The final battlefield situation and the operational process. The first one mainly focus on qualitative data. The evaluation experts can draw some qualitative conclusions based on the process and the final situation of the operation. However, these qualitative conclusions cannot be described with the quantitative data, or hard to be synthesized by the lower layer indexes, such as the adaptability. (4) The Measures of Emergence Effectiveness (MOEE) The MOEE is used to evaluate the emergence during the evolution process. The emergence is reflected in structure, function and behavior, e.g. the robustness and vulnerability of its architecture, the adaptive and the synchronous behavior, etc. The pentagons in the Figure 3.16 represent different emerging properties of the networking system. The MOEE analyzes the effectiveness on the mechanism level deeply, reflecting the confrontation and ability generation mechanism, and exploring the deep-seated reason for the mission completion. Therefore, it can be considered that emergence effectiveness is the “cause” and the mission effectiveness is the “result”. In fact, the emergence effectiveness determines the final effect in its mission. From the evaluation index system framework shown in Figure 3.16, it can be seen that the MOTE is a function of the MOEE. These two indexes have integral characteristics, which cannot be synthesized and obtained through the aggregation of indexes at other levels.

Construction of Evaluation Index System

51

As a complex system composed of multiple sub-systems which are independently and interconnected. Therefore, in the development and evolution of the SoS, the high-strength emergence characteristics are inevitable. The relationship between the emergence and the operational capability can be described by the reasonable analysis and the quantitative evaluation. It can provide the necessary guiding principles and the decision-making basis for how to optimize the allocation of various operational resources within the aircraft carrier formation, and how to improve the operations effectively. The emergence is mainly reflected by the operational capability [57]-[62]. Therefore, the emergence can be evaluated by the system-level operational capability. The operational capability is determined by the level of operational equipment, the number of personnel, performance, organizational management, command control and management capabilities, and various logistics support capabilities. It is also related to the terrain, sea conditions, weather and other objective conditions. From the perspective of the emergence, the operational capability is analyzed by a series of dynamic processes, such as weapon equipment, establishment system, battlefield information, early warning detection and tracking, command decision-making, and fire strike. Therefore, the system-level operational capability mainly includes early warning and detection capability, command and control capability, and fire strike capability. It can be decomposed to the subsystem level and can be described by the subsystem-level capability index. While the system-level operational capability is the result of the interaction between the sub-systems, which mainly including two categories. The first is the inherited emergence indexes such as early warning coordination capability, coordinated command capability, and firepower strike coordination capability. These indexes inherit the system-level operational capability, and they are the result of the combined effects of several related systems. Therefore, although their functions are similar with the system-level operational capabilities, they are not a simple linear superposition of the related system operational capability indexes. The other category is the non-inherited emergence indexes, including battlefield situational reasoning capability, survivability and adaptability of SoS. They are the result of the comprehensive effects of the constituent units. A single system cannot perform these capabilities independently, so it is a new operational capability index emerging at SoS level. The inherited and non-inherited indexes are the basis for evaluating emergence. The better the emerged SoS capability indexes are, the better the emergence effect will be. (5) The Measures Networked Effectiveness (MONE) Considering the networked characteristics of the SoS, the emergence effectiveness comes from the complex networked interactions. Therefore, the MOEE has to pay special attention to the network-based coupling interactions among the sub-system within the SoS. To address the challenge, the MONE was introduced. The MONE is used to evaluate the degree of the network interaction between the sub-systems during the SoS evolution, which can be fully used in evaluation. The complex network based MONE abstracts the SoS into a complex network model composed of the factors and relationships, and then use complex network theory to analyze the special properties, including network structure and dynamic properties, such as propagation, synchronization, control, and gaming. These properties correspond to certain characteristics of the SoS, which have the practical physical meanings, and are the em-

52

Deping Zhang and Xuefeng Yan

bodiment of MONE. The characteristic parameter describing these properties is exactly the MONE. The MONE includes two aspects, one is the complex network modeling of the SoS, the other is the index construction based on the network model, i.e., MONE = (GSOS , M)

(3.11)

where, GSOS represents the SoS network model, and M = {Metric1 , Metric2, · · · , Metricn } represents the networked effectiveness index set, and then, Metrici = f i (GSOS )

(3.12)

The square in the Figure 3.16 corresponds to the two-tuple as shown in the Equation (3.11). Multiple SoS network models and their corresponding networked effectiveness indexes can be established according to the research goals and requirements. The MOWE describes the properties of the SoS network models, which can be designed based on the typical statistical characteristic parameters in the graph theory and the complex network theory, such as degree distribution, length of the critical-path, network efficiency, aggregation coefficient, betweenness and centrality, etc. The evaluation index of the network model should be selected, designed and constructed according to the physical meaning of the SoS network model. When the traditional complex network parameters are not enough to describe the specific properties, new networked effectiveness index needs to be added. Especially for heterogeneous network models, the meaning of the traditional complex network parameters is not clear enough, so it is necessary to design an appropriate index.

3.4.3.

Correlation Analysis of Efficiency Indexes

In the effectiveness evaluation modeling framework, the correlations of effectiveness indexes at each level is very important to the effectiveness evaluation. (1) The Correlation between MOTE and MOEE For scientific and reasonable MOEE, their changes will inevitably be reflected in some effectiveness indexes at the mission level. For example, a higher decision and action synchronization will inevitably achieve better operational results. Therefore, according to the correlation, the rationality of the MOEE can be verified, and the overall capability can be optimized by improving the system emergence effectiveness. (2) The Correlation between MOEE and MONE MOEE is the key and difficult problem of SoS capacity evaluation, and there is no unified theory and method yet. The MONE provides feasible ideas and methods for the study of MOEE. There are two ways to analyze the correlation. 1) Directly model the MOEE mathematically based on MONE; 2) Measure the actual or simulated operating status of the SoS, and the actual “value” of certain MOEE can be obtained directly, such as the OODA cycle period of the SoS. At the same time, the relevant MONE is constructed and calculated, The data analysis method

Construction of Evaluation Index System

53

is adopted to investigate which indexes of the MONE and MOEE have a close relationship, so as to choose effective MONE. In addition, the MOEE is also closely related to the MOCE. But it is not a simple aggregation relationship so that is difficult to investigate directly. It can be realized by networked modeling. Concretely speaking, the MOCE will be incorporated into the design of MONE, and then the relationship between the MONE and the MOEE will be investigated. It means that the MONE sets up a “bridge” between the MOCE and the MOEE.

3.4.4.

Case Study

The aircraft carrier formation effectiveness capability index is used to measure the resources that the formation can control in operation, and it also measures the way and effects when using these resources. It is the basis for the evaluation capability of the aircraft carrier formation operational. In this section, the anti-submarine operational mission is taken as the research object, a networked operational model based on operation loop is built. With the equipment capability factor indexes, system-level capability indexes, structure characteristics, networked characteristics indexes, and emergence indexes, a dynamic index network can be built by using the correlation with index time series. On this basis, community analysis and clustering of the key indexes can be used to explore and build an aircraft carrier formation anti-submarine networked effectiveness evaluation index system [63]-[66]. The process is shown in Figure 3.17.

Figure 3.17. Effectiveness evaluation index system construction of anti-submarine aircraft carrier formation. (1) Modeling of Formation Anti-Submarine Networked Operational Model The combat organization of the aircraft carrier formation is generally determined by the mission and the threat degree. An aircraft carrier battle group with high effectiveness and strong survivability needs to have several capabilities, e.g. anti-ship, anti-submarine, air defense, and land attack. Supposing an aircraft carrier formation need to anti-submarine. The formation is a single carrier operational group, equipped with a coastal battleship, an anti-submarine aircraft, two anti-submarine helicopters, and two attack nuclear submarines. The mission is to strike against the four attack nuclear enemy submarines. In order to detect enemy submarines in time and effectively, a reconnaissance network need to be deployed,

54

Deping Zhang and Xuefeng Yan

which consists of a unmanned surface vehicle, an unmanned underwater vehicle, a reconnaissance ship, an ocean surveillance ship, a distributed networked system, and a ocean surveillance satellite. The reconnaissance network detects, and transmits the enemy information to the aircraft carrier over a long distance through two communication satellites. Then the carrier strikes the enemy submarine the attack equipment. The anti-submarine operational forces instantly using include anti-submarine aircraft, submarines, anti-submarine helicopters and coastal operational ships, which are deployed in the outer, middle and inner of the anti-submarine surveillance areas respectively, a deep, three-dimensional and multilayer anti-submarine protection system can be formed. Figure 3.18 shows a operational network schematic of the aircraft carrier formation anti-submarine network [67]-[72].

Figure 3.18. Schematic of the anti-submarine operation network. It can be seen, the reconnaissance equipment founds the enemy submarine and uploads the detection information to the communication satellite, which is also the reconnaissance equipment, so as to transmit the information to the command aircraft carrier. Then the carrier makes a decision and sends it to the strike, and strikes on enemy submarines. The equipment in this anti-submarine activity is interwoven into a complex operational network. Therefore, how to evaluate the comprehensive strike effectiveness of this anti-submarine operation comprehensively and effectively is urgent research hotspot. The involved SoS are abstracted into a networked model based on the equipment/subsystems and their complex characteristics. Weapon system-of-systems functional operation network can be expressed as G = (E,V), where V is the meta-functional node in the functional operational network, and E is the edges to connect the meta-function nodes. The following is a detailed modeling description of anti-submarine operational network from the views of node modeling and edge modeling.

55

Construction of Evaluation Index System (i) Node modeling

After the analysis of the operational network nodes, it can be found that there are four types of nodes in the node set: reconnaissance, command, strike, and target, i.e., V = S∩D∩ I ∩ T . Red side reconnaissance (S), command (D), and strike (I)nodes upload information and issue instructions through the data link, and strike on the target (T ). The nodes are shown in Table 3.1. Table 3.1. Node information of the anti-submarine operational network Node type Reconnaissance

Command Strike Target

Equipment Unmanned surface vehicle, Unmanned underwater vehicle, Reconnaissance ship, Marine surveillance ship, Distributed networked system, Marine surveillance satellite, Communication satellite Aircraft carrier Anti-submarine aircraft, Attack nuclear submarine, Antisubmarine helicopter, Coastal operational ship Enemy submarine

(ii) Edge modeling Close cooperation between operational equipments is necessary and significant feature for informationalization operation. The reconnaissance equipment can upload reconnaissance information to the aircraft carrier through the data link, and the aircraft carrier command will further use electromagnetic signals to publish command information to the strike equipment. Eventually, the fire strikes, electromagnetic disturbance and other methods are used to against the target. These materials, energy and information flow form the edges. There are 16 different combinations among the four types of entities in the antisubmarine operational network, excluding 11 kinds of edges that do not appear in the anti-submarine activities, e.g. S → I, S → T, D → S, D → D, D → T, I → S, I → D, I → I, T → D, T → I, T → T . Therefore, the anti-submarine operational network edges are shown in Table 3.2. Table 3.2. Edge type of the anti-submarine operation network Edge type S D I T

S S→S T →S

D S→D

I D→I

T

I→T

56

Deping Zhang and Xuefeng Yan The specific meaning of each edge can be described as following: T → S: Detection link; S → D(S → S → D): Information upload link; D → I: Decision link; I → T : Impact link. (2) Index System Construction of SoS Operational Effectiveness

Based on the anti-submarine operation process, missions and capabilities. A multilevel evaluation model of the anti-submarine operation capability has been comprehensively constructed in accordance with the requirements of the hierarchical model construction of the operation capability [73]-[76]. (i) Establishment of effectiveness index system for component-level sub-system Combined with the decomposition of the operation capability, a multi-level index system of the anti-submarine capability can be constructed, including the detection link, the information upload link, the decision link, and the impact link. By analyzing the index system of anti-submarine from the perspective of the complex network link, the effectiveness evaluation of anti-submarine activities against enemy submarines can be obtained. (a) Index system of the detection links effectiveness evaluation The anti-submarine mission are reconnaissance nodes composed of ocean surveillance satellites, ocean surveillance ships, reconnaissance ships, unmanned surface vehicles, unmanned underwater vehicles, and distributed networked systems. The main task is to search and find enemy submarines and detect the signs and positions in time. The detection capability of the detection link is supported by the capabilities of the above reconnaissance equipment. The construction of the ocean surveillance satellite system effectiveness index system is taken as an example, as is shown in Figure 3.19.

Figure 3.19. Index system of detection link effectiveness evaluation.

Construction of Evaluation Index System

57

The effectiveness of ocean surveillance satellite system U is mainly supported by several capabilities, i.e., system information processing capability U1 , system response capability U2 , scanning infrared camera detection capability U3 , and staring infrared camera monitoring capability U44 . Each capability is supported by the corresponding sub-capabilities. For example, in order to evaluate the system response capability U2 comprehensively, four aspects should be taken into consideration: system alarm time U21, response time U22 , communication delay U23 and anti-interference capability U24 . (b) Index system of information upload link effectiveness evaluation In the mission of an anti-submarine, the communication node is composed of the communication satellites, who transfer the detection information, the position, number, and model of the enemy submarine, to the command equipment through a wide-range communication satellite. Their ultimate purpose is to improve the communication capabilities, expand the communication coverage and enhance anti-submarine operations capabilities. The communication capability of the information upload link is mainly supported by the communication satellites. The communication satellite system effectiveness index system is shown in Figure 3.20.

Figure 3.20. Index system of information upload link effectiveness evaluation. The communication satellite system effectiveness I is mainly supported by four capabilities, i.e., communication coverage capability I1 , transmission capability I2 , information processing capability I3 , and security protection capability I4 . Each capability is supported by a corresponding sub-capability. Taking the evaluation system communication transmission capability I2 as an example, the transmission capability of the communication satellite system should be evaluated from four aspects: the continuous communication capability I21 , the quality of service I22 , and the service capability I23 , and mobile communication capability I24 , etc. (c) Index system of decision link effectiveness evaluation In the anti-submarine SoS, the command node is composed of aircraft carriers, who makes operational plans based on the information of the location, number and type of

58

Deping Zhang and Xuefeng Yan

enemy submarines discovered by the detection link and communication upload link, and makes fire attack strategies and effective manner according to its own equipment capability, and issues command orders to fire-strike equipment. The decision link is the key for the effective implementation. The operational management, as well as the command and control capability are important capabilities to ensure the completion of anti-submarine mission. These capabilities are supported by the capability of the aircraft carrier system. The decision link effectiveness index system is shown in Figure 3.21.

Figure 3.21. Index system for decision link effectiveness evaluation. The effectiveness of aircraft carrier system C is mainly supported by 5 capabilities: the control decision capability C1 , command response capability C2 , communication capability of the network C3 , information processing capability C4 , and comprehensive situation capability C5 . Each capability is supported by the corresponding sub-capability. Taking the control decision capability C1 as an example, in order to evaluate the control and decisionmaking capability of the aircraft carrier command and control system, it should take 3 aspects into consideration: generation time of operational plan C11 , decision response time C12 , and competence of decision maker C13 . (d) Index system of the impact link effectiveness evaluation In the operational mission of the anti-submarine, the strike node is constructed by equipment systems such as attacking nuclear submarines, anti-submarine aircraft, coastal oper-

Construction of Evaluation Index System

59

ational ships, and anti-submarine helicopters. The main task of these nodes is to receive commands and conduct effective and timely fire strikes against enemy submarines, which is the last link of anti-submarine operations. The strike capability of the impact links in the anti-submarine is supported by the above equipment system capabilities. Now taking the construction of the anti-submarine aircraft system effectiveness index system as an example, it can be described as shown in Figure 3.22.

Figure 3.22. The impact on link effectiveness evaluation index system. The effectiveness of anti-submarine system T is mainly supported by four capabilities: the operational capability of air anti-submarine platform T1 , the submarine searching capability T2 , the attack submarine capability T3 , and the operational capability of command system T4 . Each capability is supported by the corresponding sub-capability. Taking the submarine searching capability T2 as an example, the submarine searching capability of the anti-submarine aircraft should be comprehensively evaluated from four aspects: the sonar submarine search capability T21 , the magnetic detection submarine search capability T22 , the radar submarine search capability T23 , and the infrared submarine search capability T24 . (ii) SoS mission effectiveness index The indexes of the aircraft carrier formation anti-submarine operation are based on the analysis of the overall operational effectiveness, which are shown in Figure 3.23. Although the operational effectiveness and the mission completion capability are used as secondary indexes, they are not integrated from lower-level indexes. They can only indicate the attribution or classification of the lower-level indexes. The third level indexes, i.e., the next-level indexes of operational effectiveness and the mission completion capabilities, are all descriptive indexes. If there is no clear data source, they can be further decomposed, or evaluation experts will make a qualitative conclusions through analyzing the entire operational situation.

60

Deping Zhang and Xuefeng Yan

Figure 3.23. Index composition of SoS operational effectiveness. Operational effectiveness mainly includes operational damage, operational results, advance speed, and operational time. Operational damage refers to the loss of equipment and personnel in SoS operations. Operational result represents the number of enemy targets destroyed during the operation. The advance speed indicates the advancing speed of the operation process. Operational time refers to the time of operational activities, i.e., the time that the formation or equipment is directly used to carry out the mission. The mission completion capability is to describe the overall situation of the SoS completing a specific operational mission. The purpose is to draw the overall conclusion on the effectiveness of the SoS. The completion degree of the mission is evaluated based on the operation objective and the final operation situation. It is difficult to draw conclusions only by quantitative means, and sometimes the qualitative judgment of evaluators is needed. The main content of mission completion capability includes two parts: mission completion degree and mission completion probability. For a specific mission, the completion degree describes the situation of a single simulation or exercise, and the mission completion probability is to describe the situation of multiple simulations or exercises. (iii) SoS network structure characteristic index system The aircraft carrier formation is a complex system composed of huge amount of sensors, command and control, communications and other entities or sub-systems connected by various wire or wireless methods. Because of the natural structural similarity between the complex networked model and the SoS, it is possible to use network model to evaluate the SoS, which has become a research hotspot. After constructing the weighted network or complex networked model of the aircraft carrier formation, the SoS networked structure characteristic indexes are shown in Table 3.3. The characteristics indexes are given in Table 3.3. According to Cares J’s description of the basic operational capabilities of distributed networked indexes, combined with the characteristics of aircraft carrier formation anti-submarine operations, the networked effectiveness indexes of the aircraft carrier formation system are defined as following capabilities: the indestructibility of the SoS R1 , reorganization R2 , dispersion R3 , concealment R4 , proximity R5 , flexibility R6 , adaptive R7 and efficiency R8 etc., as is shown in Figure 3.24. Where, (a) Indestructibility R1 refers to the difficulty of destroying the SoS structure, that is, the ability of the network can maintain a certain connectivity when it is damaged.

Construction of Evaluation Index System

61

Table 3.3. Characteristic indexes for networked structure Index name Number of the active nodes Degree Level Functional focus Connectivity Link node ratio Betweenness

Neutral rate Clusters Divergence

Index content Number of the active nodes Node degree, average degree, maximum degree, degree distribution Average level, maximum level Number and interconnection of the command centers, command center vulnerability indexes, number and interconnection of the operations centers, vulnerability indexes for operations centers Natural connectivity of the network Number of edges / Number of nodes Interconnection situation of m nodes with the largest betweenness, betweenness distribution, betweenness center vulnerability index (Number of edges - number of nodes + 1) / Number of nodes Number of clusters, cluster size Node dispersion

(b) Reorganization R2 is the capability to redeploy or assemble factors quickly, adaptively to evolve into a certain topology, and to generate valuable adaptive behavior with multi-scale. (c) Dispersion R3 refers to the divergency degree of space, information, and logic, avoiding producing centra structures. (d) The concealment R4 refers to the concealment ability of the node, which can improve the cohesion ability and reduce the probability of being discovered by aggregating smaller units. (e) Proximity R5 refers to the information and logical completeness of the networked structure. It highlights the informational and logical contribution of a large number of small hidden targets to the operational capability. (f) Flexibility R6 refers to the command interoperability, i.e., the capability to adapt to fierce competition and great environment changes , R6 is independent from the reorganization. (g) Adaptability R7 refers to the adaptive capabilities of network. (h) Efficiency R8 refers to obtain maximum operational effectiveness with the least resources. (iv) SoS emergence indexes From the perspective of emergence, the operational capabilities of the aircraft carrier formation can be analyzed by the following factors, e.g., equipment, organization, battlefield information, early warning detection and tracking, command decision-making, and fire strike. Therefore, system-level operational capabilities mainly include three types of

62

Deping Zhang and Xuefeng Yan

Figure 3.24. Composition of SoS networked operational effectiveness index. indexes, i.e., early warning detection capabilities, command and control capabilities, and fire strike capabilities. The system-level operational capabilities can be decomposed further to the subsystem level, which can be further described by subsystem-level capability indexes. While the overall operational capabilities of SoS are the results of the interaction and mutual influence between the sub-systems, and it mainly includes inherited emergence indexes and non-inherited emergence indexes [83]-[83], as shown in Figure 3.25. In Figure 3.25, the inherited emergence indexes include the early warning coordination capability and firepower coordination capability. They inherit the system-level operational capabilities, but are the result of a combination of two related systems. Therefore, although the function of these indexes is similar to the system-level operational capability, they are not a simple linear superposition of related system operational capability indexes. Non-inherited emergence indexes, including battlefield situation reasoning capability, survivability and adaptability. They are the result of a combination of the components, and a single system cannot have these capabilities independently. Therefore, they are the new indexes of operational capabilities emerging at the SoS level. The inherited and non-inherited indexes are the basis for evaluating the emergence. The better the emerging SoS-level operational capability indexes are, the better the emergence effectiveness is.

Construction of Evaluation Index System

63

Figure 3.25. Composition of emergence indexes for aircraft carrier formation. Collaborative attack capability is an index to measure the overall attack effectiveness. The attack capability index determines whether the combat entity can achieve the hard killing ability, which is related to the survivability, the timeliness of the coordinated attack, the effectiveness index, and the damage evaluation capability. The operational environment of adaptability refers to the electromagnetic environment and natural environment in the operational space specifically. It mainly reflects that the air defense system has improved its adaptability to the electromagnetic environment and the natural environment through networked operations. For example, networking can improve the detection ability of stealthy targets and interference targets, and can overcome the influence of terrain shadowing. The adaptability is a comprehensive index, which is closely related to the operational effectiveness and target characteristics. According to the requirements of networked operations, the adaptability can be measured by anti-jamming capabilities, anti-cruise capabilities, and anti-stealth capabilities. Invulnerability is a comprehensive reflection of system structure heterogeneity, network relevance, dynamic adaptability, and inherent protection ability. It is also the result of the overall emergence, and it is a new characteristic in the evolution of the SoS. The evaluation basis has changed from the separable and independent indexes of traditional reductionist hypothesis to the overall networked indexes. The invulnerability can be reflected by the space scale, the time scale, the overall protection and the command effectiveness. The influencing factor indexes include four invulnerability indexes, such as connectivity, timeliness, protection and command chain integrity. In this case, the massive data generated by the aircraft carrier formation is recorded in the form of index parameters of each component system. The index parameters can be roughly divided into several indexes: search and detection indexes (e.g. air defense system detection and identification of the number of enemy submarines, distance time, etc.), information processing indexes (e.g. the information collection, processing, and distribution time), fire strike indexes (e.g. the operational damage and the number of submarines hit during the whole operation), electromagnetic countermeasure indexes (e.g. the quantity and performance changes of various equipment interfered by the aircraft carrier formation) and communication indexes (e.g. the number of communication networks and corresponding communications). Meanwhile several typical capabilities indexes are considerated (e.g. the number of various equipment and the performance comparison). The networked structure

64

Deping Zhang and Xuefeng Yan

indexes were considered as well, such as networked efficiency, accessibility, the number of different networked models, aggregation coefficient of the network and average time efficiency. Totally, 283 indexes are selected, of which (Y ) is used to determine whether the enemy submarine can be detected and destroyed as the anti-submarine mission. The remaining 282 indexes are used to construct the initial sample set T = {X1 , X2 , · · · , X282}. (3) Dynamic Index Network Based on the Correlation Analysis of Time Series The index correlation analysis is used to discover the evolution law of the indexes and their correlation. The traditional evaluation index system is composed of the index values and correlation which describes the causal relationship, while the indexes and their correlation in the dynamic networked index system are derived from the actual operational data, and the correlation only describes the relationship between these indexes. There are two important steps to construct the network, and the first is to analyze the temporal evolution of these indexes. Then, the characteristic parameters of various types of indexes will be analyzed in time series, which can provide a reference for discovering the evolution law in different operational stages. The second is to analyse multiple types of indexes correlation. The correlation between indexes is unknown in advance, it may be linear or non-linear, and may also have time delays, and changes continuously. Therefore, all indexes are assumed to be related. Next, the appropriate number of windows and correlation analysis method are selected to analyze the sliding-window correlation of all index in time series, and the evolution law of the correlation relationship between any two indexes is obtained. Finally, a fully connected dynamic index network can be constructed, in which index values and correlations are constantly evolving with time. The networked SoS determines the networked correlations between the indexes, and these correlations are usually non-linearity and uncertainty. Therefore, the maximum information interaction algorithm is adopted for the correlation analysis between the initial indexes. Compared with algorithms such as Pearson and Spearman, the maximum information interaction algorithms has no preset parameters, and has two important properties, generality(i.e., strong applicability to any relationship type) and equitability (i.e., robustness to noise). The basic idea is: if there is a correlation between the index pair (Xi ; X j ), all the data points will be distributed in the cells of the grid. By continuously increasing the resolution, the mutual information values of all grids in different grid division scheme will be compared and normalized. The largest mutual information standardized value is the maximum information interaction coefficient (MIC), and 3 steps are as following: 1) If there is a correlation between two indexes, there will be a “most suitable” grid division scheme for the scatter plot of this index pair. In this grid division scheme, most of the data points of the index pair are concentrated in a few cells. Therefore, for a x row ×y column grid g of index pair (Xi; X j ) coordinate plane, the probability density of the cell p(x, y) is the ratio of the number of sample points to the total number of samples. In order to measure the concentration of index pairs, the interactive information I(Xi ;Y j ) of the grid g can be defined as follow:   Z Z p(x, y) dxdy (3.13) I(Xi ;Y j ) = p(x, y) log p(x)p(y) where I(Xi ; X j ) represents the correlation strength between index pairs is grid g.

65

Construction of Evaluation Index System

2) Since the grid does not have to be divided into equal widths, there are multiple division scheme for the same x and y, which can be recorded as G = {g1 , g2 , · · ·}. Therefore, the interactive information eigenvalue MI(X,Y ) of G can be defined as MI(X,Y ) =

max(IG ) log(min(x, y))

(3.14)

where, the grid corresponding to max(IG ) is the “most suitable” x row ×y column grid. Thus, all the possible x row ×y column grid should be traversed to divide n groups of scatter plot from Ti j = {(Xi; X j ); (i 6= j)}. The maximum resolution of grid division should satisfy the requirement that 3 < xy < n0.6 , and the eigenvalue matrix M(Xi ; X j ) = (MI(x; y)) of the index pair (Xi; X j ) can be obtained. 3) The maximum information coefficient MIC is calculated. The maximum information coefficient of the index pair (Xi ; X j ) is the maximum one in the eigenvalue matrix, as is shown in the following: MIC = max{MI(X,Y )}

(3.15)

where MIC ∈ [0; 1], the closer the value is to 1, the stronger the correlation between the indexes is. Therefore, the strength of correlation between each pair of index parameters can be calculated and the initial index network F = (V ; E) is constructed on this basis. The node is V = (X1 , X2 , · · · , Xk ) and the edge weight is E = (MICi j )k×k. The complex mapping relationship between the initial indexes is reflected in the aircraft carrier formation, as is shown in Table 3.4. Table 3.4. The maximum interactive information intensity between indexes Index X2 X3 X4 .. .

X1 0.312 0.281 0.187 .. .

X2

X3

X4

X5

0.566 0.721 .. .

0.815 .. .

.. .

.. .

··· ··· ··· ··· .. .

(4) Key Indexes Mining Based on Community Analysis and Clustering In aircraft carrier formation operations, a complete set of operation data often includes data from multiple operation phases. The community structure is generally stable in the same operation but vary greatly of the index network in different operation. Therefore, the key indexes may also be completely different. It needs to be mined for different operation phases. The basic process is shown as follows: First, based on the fully connected index network, the effective index network is constructed by setting a threshold (not considering the case of too small correlation) and analyzing the time delay characteristics. Second, the community mining is performed on the

66

Deping Zhang and Xuefeng Yan

index network at different times. Then, principal component analysis and centrality analysis are adopted to extract the community characteristic indexes of each moment. Finally, the characteristic indexes at each time are aggregated to obtain key indexes for the whole operation. (i) The community algorithm based on the shortest path The complex networks are often composed of several communities. The connections between nodes within a community are relatively tight, but are relatively sparse between different communities. The initial index network is the basis for measuring the capability of the SoS. The initial community network can be considered as the functional attributes emerging from specific mission of the SoS. The functional attribute represented by the community is defined as the functional indexes. Therefore, in the initial index network, several functional communities can be aggregated to several high-level functional indexes. The partition of functional community is carried out based on the shortest path characteristics. The basic principle is shown as follows: The intermediary coefficient of each node is calculated according to the characteristics of the shortest path number. Then the community center can be obtained and the similarity between the nodes is calculated by the characteristics of length. The average similarity of all nodes is taken as the threshold for dividing the community to form a model clustering. Every time, the node with the largest intermediary coefficient value is taken as the cluster center, and the comparison result between the similarity of the remaining nodes and the threshold is taken as the condition for clustering. After each clustering, the nodes corresponding to the cluster center are deleted. The cyclic clustering is carried out until the division is completed. According to the shortest path algorithm, two different concepts are defined. The first is the feature number Nsp of shortest paths. The second is the feature length Nsp . A simple topology graph in a complex network is shown in Figure 3.26, which is used to clarify the two different concepts above.

Figure 3.26. The simple topology of complex network. The path from node 1 to node 6 is taken as an example: 1) The number feature of the shortest path Nsp In Figure 3.26, only one shortest paths exists from node 1 to node 6 is, that is 1-3-6. The number of shortest paths is represented by Nsp = 1. The feature is used to calculate the

67

Construction of Evaluation Index System intermediary coefficient: ∑ qijk /p jk

Bi =

i6= j6=k

(3.16)

(n − 1)(n − 2)/2

where the feature number Nsp between node j and node k is represented by p jk . 2 In the shortest path of all network node pairs, Cn−1 = (n − 1)(n − 2)/2 represents the maximum of Nsp that might pass through the node i. In the p jk shortest paths between node j and node k, the Nsp shortest paths that passes through the node i is represented by qijk . In the shortest path of each nodes pair in the network node set, ∑ qijk /p jk represents Nsp i6= j6=k

that passes through the node i actually. 2) Length feature of the shortest path According to Figure 3.26, there is one and only one shortest path for nodes 1 to 6, which is 1-3-6. The length of this path is 2(2 edges). The length of the shortest path is defined as Lsp = 2. The similarity of the nodes is obtained by the feature. S( j,k) =

1+

r

1

(3.17)

∑ (d ji − dki )2

i6= j6=k

where d ji and dki are the Lsp from the node j, k to node i, and the range of S( j,k) is (0, 1]. 3) The partition threshold of community structure,

∑ S( j,k)

λ= where Cn2 = pair.

n(n−1) 2

j6=k

n(n − 1)/2

(3.18)

represents the number of all node pairs and ( j, k) represents certain node

The partition of community nodes is clustered based on the shortest path features. The importance and influence of a node in the network topology are characterized by the number of shortest paths through the node. Accordingly, the intermediary coefficient of each node is obtained and sorted in descending order. The similarity of the node pair is judged by comparing the length difference of shortest path between any two nodes to other nodes in the network. The average of similarity for all node pairs is taken as the partition threshold. According to the concept of cluster, the cluster partition of community nodes can be obtained. The main steps are shown in the Algorithm 3.1. The nodes will be continuously removed from the set of the community (the connection edge with the largest number of boundaries in the network) until there is no node in the network. In this case, the final community partition result is shown in Figure 3.27.

68

Deping Zhang and Xuefeng Yan

Algorithm 3.1. The community clustering algorithm based on the shortest path (1) In the network, the intermediary coefficient Bi of each node is calculated by the Equation(3.16) and stored in descending order in the array Bi . (2) The similarity S( j,k) of all node pairs is obtained by the Equation(3.17) and stored in the similarity matrix S. (3) The partition threshold λ of the community structure is calculated by the Equation(3.18). (4) The node corresponding to the largest element in the array B is taken as the community center. (5) According to the partition rules, the similarity of node (excluding the node that has been identified as community centers) is compared with the threshold. The nodes similarity larger than threshold are clustered and partitioned nodes are removed from the node set. (6) If the node set is non-null, steps (4) to (6) are repeated. Otherwise, the cluster partition is completed and the algorithm is finished.

(ii) The characteristic index set based on PCA According to the community partition algorithm, there is strong correlation between the initial indexes of the community. In order to describe the aggregation relationship more clearly between community indexes and emerging system function indexes, and reduce the coupling between initial indexes, the community characteristic index is constructed between the community function index and the initial index, and the paradigmatic relation between them is expressed more closely. The community characteristic index is defined as the eigenvector of each community, which is calculated by principal component analysis (PCA). The vectors in the original data are expressed by linear transformation with relatively less number of linearly independent vectors. At the same time, the main information is not reduced in the original data. It is also an index aggregation method. PCA can transform the high-dimensional space into low dimensional. Moreover, the obtained characteristic indexes are linearly independent, and it can provide most of the main information, and avoiding subjective judgment. It can be calculated by the following steps: 1) The correlation coefficient matrix of k mation interaction algorithm.  r11 r12 r21 r22  R= . ..  .. . rn1 rn2

indexes are obtained by the maximum infor · · · r1n · · · r2n   .  .. . ..  · · · rnn

(3.19)

Construction of Evaluation Index System

69

Figure 3.27. The result of community partition for aircraft carrier formation.

2) The eigenvalues and eigenvectors can be obtained by decomposing the correlation coefficient matrix. The characteristic roots of the characteristic equation |λI − R| = 0 can be calculated and arranged in descending order. λ1 > λ2 > · · · > λn

(3.20)

The corresponding eigenvector of λ j is C j = (C1 j ,C2 j , · · · ,Cn j )T . 3) Calculate the principal components. The principal component is composed of the eigenvectors is as follows. B j = C1 j X1 +C2 j X2 + · · · +Cn j Xn , j = 1, 2, · · · , n

(3.21)

The n principal components are linearly independent and the variance decreases gradually. 4) Select the principal components. The first p(1 ≤ p ≤ n) principal components are selected as the characteristic indexes of the community. The selected p value will have a direct impact on the evaluation. If p is exceedingly large, the data compression ratio will be very low. If p is small, the loss of data characteristic information may be increased. Generally, it can be judged by the cumulative contribution rate of the principal components (the ratio of the sum of the variances corresponding to the first p principal components to the sum of the n variances), which is shown in the following: p

∑ j=1 λ j wp = n ∑ j=1 λ j

(3.22)

Generally, it is required that the sum of variances of B1 , B2 , · · · , Bn accounts for more than 85% of the total variance, so that the original n indexes can be converted into p indexes. According to the above steps, under the guidance of specific mission index, the index system structure model of the hierarchical network can be constructed by analyzing the

70

Deping Zhang and Xuefeng Yan

initial index set, extracting community characteristic indexes and defining emerging system functional indexes. It can be formally expressed as follows: H = (V, E)

(3.23)

where the node is V = (X, B,C,Y), the edge is E = (EX , EXB , EBC , ECY ). X, B, C and Y represent the initial index layer, the characteristic index layer, the system function and the mission index layer respectively. EX represents the connectional edge of the initial index layer. EXB , EBC and ECY represent the connections between the indexes of each layer respectively. The index system network architecture model H not only represents the complex relationship between the initial indexes, but also reflects the cascading emergence relationship from the initial indexes to the community characteristic indexes, then to the system function indexes and finally to the mission indexes. It provides the foundation for emergence the effectiveness evaluation and the mechanism research.

3.5. Construction of Mission-Oriented Evaluation Index System 3.5.1.

Construction of Operational Mission Profile

(1) Basic Concepts of Operational Mission Profile (i) Mission profile The mission profile [84]-[89] is the basic events and their sequential relationships at each layer of each mission phase in certain time, and also includes other possible basic events, and sequential relationships. Mission profile also illustrate the success conditions of these events. The definition consists of three basic elements: main basic event at each level of each mission phase and all other possibility basic elements (including the environment), basic sequential relationship of events; success description of basic event and each phase. (ii) Meta-mission In order to facilitate the analysis and modeling of operational mission processes, the concept of meta-mission is given below. meta-mission is a minimal independent mutually uncorrelated unit which can complete a mission with stable corresponding relationship. According to the definition, the properties are as follows: (a) Limitation. The limited objective can be achieved after the mission have been successfully completed. (b) Independence. There is no inclusive relationship, owner-member relationship, or subordinate relationship between meta-missions. (c) Atomicity. The corresponding system capacity of the meta-mission is fixed. The objectives are relatively clear. The execution status is deterministic whether it is success or not.

Construction of Evaluation Index System

71

(2) Modeling of Mission Process The operational process of equipment is very complicated, which is tactically divided into several different stages, and then decomposed gradually until the appropriate granularity [90]-[94]. Taking the attack process of the submarine torpedo as an example, according to the tactical rules, the attack process is divided into several stages including target search, and identification, target element resolve, and position occupation shooting, and evading and evacuation, as shown in Figure 3.28.

Figure 3.28. Anti-submarine operational task flow diagram. The corresponding meta-mission can be obtained by analyzing the main actions in each stage. Different from the modeling of tactics based operational mission, the modeling for mission profile does not need to study specific tactical details, but only needs to consider the temporal and logical relations among different meta-missions. In order to describe the randomness and uncertainty of different operational phases, the IDEF3 based operational mission profile construction is adopted here. First of all, a formalized method is used to describe the operational mission, actions and the relationships between actions. This relationship is defined as follows: T R = {SeqR,CndR, AndR, OrR,ConqR,SynR,CycR} ⊂ TA × TA

(3.24)

where TA = {TA1 , TA2 , · · · , TAn } is the set of operational actions in an mission. The relationships between operations consist of seven basic logical relationships: sequential (SeqR), conditional (CndR), and (AndR), or (OrR), concurrence (ConcR), synchronization (SynR), and cyclic (CycR). Because of the transmissibility of the relationship, various new relationships can be formed between the operations through reasoning, which can be adopted to confirm the correctness of the relationship between the operations. Then, the main purpose and content of each operational phases are analyzed, including the overall target, the requirements, and the problems. Finally, according to the operational mission profile and the meta-mission definition of the equipment system, a meta-mission based operational process model is built in accordance with the construction process of IDEF3. For example, an anti-submarine operational mission process model of submarine torpedo can be constructed based on IDEF3, as is shown in Figure 3.29. In Figure 3.29, Ti represents the meta-mission i, where T1 and T2 both represent the meta-mission 1.

3.5.2.

Mission Effectiveness Evaluation Index System Based on ProcessFocused Thinking Method

The process focus thinking(PFT) is a method to establish a comprehensive process-centered effectiveness evaluation model, in which the process refers to the whole procedure of a specific mission [95]-[97]. The PFT is proposed as a comprehensive effectiveness evaluation

72

Deping Zhang and Xuefeng Yan

Figure 3.29. Torpedo anti-submarine mission process. method aiming at the systems which focuses on the executing procedure. One of the basic ideas is to express the complicated execution process as several independent sequential modules with single function, based on the analysis of the process profile diagram for a specified mission of a target system. In other words, the PFT converts completion rate of the target mission into a comprehensive performance measurement of relevant missionsupporting modules. The performance of the upper level modules will affect the input data quality of the low level, and further affect the final output of the system. Another basic idea of PFT is to use the bottom index contained in the bottom module to constrain the index value range of the upper module. The measurement standard is provided for the transformation from the bottom index value to the utility value. The index value range of the upper module is constrained by the lower module index design requirements. First of all, the indexes are based on the mission profile. The weapons or software modules to be evaluated, often have various capabilities, from which multiple indexes can be derived. The indexes in the index system reflect the profile of an operational mission at a certain time, and do not involve other indexes of the unrelated evaluated module. For example, an amphibious vehicle can not only sail at sea, but also complete landing operations. If the current operational mission is water patrol, the landing aperation related indexes, such as fire suppression, should not be present in the evaluation index system. From the perspective of the construction process, PFT can instruct the design and analysis of the system, reasonably decompose and allocate the overall capability layer by layer, and finally assign the specific capability requirements on each module. So as to ensure that the entire system can meet the specified effectiveness requirements. The first step is to analyze the system environment and goal, that is, the analysis of operational scenarios and missions. It can be seen from the system design process introduced above. The system is analyzed focusing on the operational mission. In order to fulfill a specific operation, the supported functional modules can be obtained by decomposed the mission layer by layer. The analysis process of the functional modules is also the construction process of the index system. Finally, the functional modules obtained from the analysis of multiple missions

Construction of Evaluation Index System

73

are summarized and organized. The components will be designed and the system will be assembled. The construction of the index system also focuses on the operational mission, so the indexes should only reflect the profile of the mission. On the other hand, the index system can be used for decision make. Each scheme is analyzed, synthesized and evaluated to get the result. Then introduce the decision rules selected based on the label information of the evaluation subject is used to sort the alternative schemes. The scheme that can achieve the same strategic and tactical purpose is selected for implementation. Therefore, from evaluation perspective, indexes should only reflect the current profile of the mission, otherwise it is difficult to ensure the comparability between different schemes. The construction of effectiveness evaluation index system based on PFT focuses on the complete process of the operation missions, so as to select index and construct the SoS, including decomposition, transformation, constraint, and aggregation, as is shown in Figure 3.30.

Figure 3.30. Construction process of effectiveness index SoS based on PFT. 1) Decomposition: Decomposition is to separate and transform the complex process of task execution on the timeline into the ordered work of multiple single functional and relatively independent subsystems, which lays a foundation for successive decomposition and quantification of effectiveness. 2) Transformation: The operational effectiveness of the weapon equipment is essentially a comprehensive description based on the information capability and the missionsoriented effectiveness. The information ability is transformed from the discomposed subsystems mission effectiveness in order to convert the completion rate into the system performance metrics. 3) Constraint: The Information acquisition and transmission ability directly affect the quality of the input reconnaissance data of the processing terminal, and further affect the production and distribution of the information. Because of the causality between the subsystems, the effectiveness of the back-end subsystem will be constrained by the front-end subsystems, which is the constraint of each subsystem on the overall operational effectiveness. 4) Aggregation: By decomposing, transforming and constraining the effectiveness, the index of each subsystem and its acquisition and quantification methods can be obtained. A hierarchical index system, the operational effectiveness can be obtained by aggregating related subsystems capabilities.

74

Deping Zhang and Xuefeng Yan

3.5.3.

Case Study

This section takes the submarine torpedo anti-submarine operational mission as an example to illustrate the modeling process of the effectiveness evaluation indexes on the PFT. (1) Requirement of Anti-Submarine Mission The anti-submarine mission includes the process of searching enemy submarine, underwater acoustic detection, target detection, making attack plan, solving target, launching guided torpedo, and evacuating. The purpose of anti-submarine is to determine sea deterrent and eliminate the blue submarine forces. In operation, the detection both sides mainly rely on sonar. The response time of both sides is relatively short, the operation system is required quickly reaction in torpedo attacking or underwater acoustic countermeasure defense. It is also necessary to prevent the attack of wire-guided torpedo or homing torpedo launched by blue anti-submarine, and prepare for torpedo defense at the same time. (i) Hydrological conditions of sea battle field The operational sea battle field includes all the sea areas within red submarine operational radius. Recently, according to the threat, we can choose a certain operational sea area. The hydrological conditions change with the season. For the convenience, hydrological conditions in particular or different seasons can be selected to make comprehensive analysis. (ii) Initial situation The red submarine is limited within a designated sea area and the length and width of the area are known. The speed and heading of red submarine are known, the blue side is unknown. Assuming that the detection distance of the blue sensors is known, the sonar of red submarine is under full-time surveillance, and bow search is adopted tactically. Under the condition of ensuring the safety of red submarine, The blue submarine can be found and destroyed. (iii) Action rules Search: The submarine uses an synthetic sonar to search for blue submarines in the operational sea battle field. The sonar searches after the submarine reaches the safe depth. Approach: After the target is found, the accusation system should take a series of actions including the underwater acoustic environment analysis, the noise estimation, the calculation of the target motion elements, and the judgment of threat and aggressiveness. If the target threat degree is above the threshold, red submarine should immediately turn into defense or evacuation state. If the threat of medium threat degree is below the standard, it will switch between attack or evacuation. Attack: If target can be attacked, the occupancy maneuver should be carried out according to the performance of red torpedo and submarines. The submarines calculate the torpedo firing data, launch the torpedo, and maintain the position. Then, the submarines observe the firing effect and prepare for attack again or evacuation. Evacuation: The submarines should move apart from the target as soon as possible to make the threat degree below the threshold.

Construction of Evaluation Index System

75

(2) Typical Mission Scenario The anti-submarine mission consists of three typical missions: the surface ship formation escorting the ballistic missile nuclear submarines escorting, and blue submarine in specific battle field by itself. As a member of formation, the submarines use the external tactical and detection information from other platforms while escorting the formation. At the same time, the submarine monitors the battlefield and against the attacks from air and sea. To ensure the concealment of ballistic missile nuclear submarines, the submarine participate in other operation platforms to carry out anti-submarine alert and attack missions. If the submarines are in front of the ballistic missile nuclear submarines or in the sea area outside the standby zone, they are in charge of the warning or searching the enemy. If necessary it can evaluate or launch torpedo ensure the safety of ballistic missile nuclear submarine. When the submarine itself performs the missions of anti-submarine in specific area it is responsible for guarding and attacking independently. The submarines use sonar to search targets in the specific sea area and defend against the torpedo or underwater acoustic countermeasure after the discovery of target. The success criterion of the mission is to eliminate the blue submarines and ensure the safety of red. (3) Operational Activity Anti-submarine mainly relies on the detection and attack capability. Under the command of formation system, it is possible to obtain the active area of the enemy submarines. Because of the concealment and indistinguishability of submarines, there will be no more than two submarines in the same anti-submarine area. The anti-submarine operational missions include navigation, search, occupation, attack, anti-submarine, underwater acoustic countermeasure, and defense. Among them, the attack or the defense should be decided on the battlefield condition. The operational activities are shown in Figure 3.31.

Figure 3.31. Operational activity of anti-submarine mission.

76

Deping Zhang and Xuefeng Yan

Where, Navigation: detection system is working, while accusation system, launching system and torpedo are not working. Search: detection system is working, while accusation system, launching system and torpedo are not working. Maneuver Occupancy: detection system and accusation system are working, while launching system and torpedo is not working. Attack: detection system, accusation system and torpedo are working, while launching system is not working; Confrontation and Defense: detection system and accusation system are working. whether the launching system and torpedo work or not depends on the situation; Evacuation: detection system and accusation system are working, while launching system and torpedo are not working. (4) Effectiveness Index System Based on the PFT Anti-submarine takes the probability of completing the anti-submarine as the system effectiveness based on PFT. The operational effectiveness index system is modeled by means of “decomposition”, “transformation”, “constraint” and “aggregation”. The details are shown as follows. (i) Navigation preparation The navigation preparation is carried out at the wharf and does not serve as a typical operational mission. In general, the effectiveness index can be measured by the availability of submarines. Asub = a × Pk

(3.25)

where a represents the underway rate of submarine, that is, the ratio of the ready to use time in one year; Pk is the probability that the submarine leaves the port. The effectiveness evaluation index for successfully completing mission in the navigation preparation is shown in Figure 3.32.

Figure 3.32. The evaluation index in the navigation preparation.

Construction of Evaluation Index System

77

(ii) Navigation phase Cruise duty is a typical mission during the navigation phase. In addition, if the operational sea area is in deep sea, the mission of breaking through island chain blockade is also important. While escorting formation and ballistic missile nuclear submarines, according to the operational plan, the conceal reconnaissance, searching the enemy targets should be carried out. To complete the mission during navigation, it is necessary for a submarine to have the ability to break through the enemy defense system and find enemy. Therefore, the submarine operational effectiveness index mainly includes the capabilities of penetration and target detection, which are described by the penetration probability Pen and the target detection probability Prob respectively, as is shown in Figure 3.33.

Figure 3.33. Evaluation index in the navigation. The red submarine the search targets in the designated sea area. The length and width of the area are known, and the speed and course of red submarine are known as well. The speed and direction of the blue submarine entering the area are unknown. Assuming that the detection distance of the blue sensors is known, red submarine sonar is monitored in full time and bow search is adopted in tactics. The operational actions are quite different because of the difference in the detection distance of both sides. According to the different distance of the target is discovered and the distance that the target begins to evade, the submarine search model is studied in the following two cases [1]: Case one: the distance that the target is discovered is greater than the distance that the target begins to evade the submarine; Case two: the distance that the target is discovered is less or equal to the distance that the target begins to evade; With the above cases being modeled and analyzed respectively. The factors which affect the target detection probability Prob can be acquired by modeling of the two cases, such as the detection distance of passive sonar, area of search sea and so on. The penetration capability is described by its penetration probability Pen , which refers to the probability of breaking through the enemy long-range defense system (including the blue submarine, the anti-submarine airplane, and the surface ship) in the process of crossing

78

Deping Zhang and Xuefeng Yan

the blockade as show in Equation (3.26). Pen = P1 × P2 × P3

(3.26)

where p1 , p2 , p3 represent the probabilities that the submarine is not detected by the blue submarines, the blue anti-submarine aircrafts, and the blue surface warships respectively. The formula is given below.   P1 = 1 − P11 P12 P = 1 − P21 P22 (3.27)  2 P3 = 1 − P31 P32

where, P12 , P22 , P32 represent the probabilities that the breakthrough submarines are detected by blue submarines, anti-submarine helicopters, and surface ships in the straits respectively within blockade area. They can be estimated by the width of the breaking sea area, the sonar range of the blue submarine and the search method. Thus, the influencing factors of penetration capability can be determined, such as the number of blockade areas and the total number of blue submarines, anti-submarine aircrafts, and surface ships, and the proportion of blockade force in the total force. P11 , P21 , P31 are respectively the probabilities of anti-submarine search of the blue submarine, anti-submarine aircraft and surface ship in the sea area where the red side will breakthrough. The probability depends on the number of blockade area, the number and proportion of blue anti-submarine forces. (iii) Standby phase In standby phase, it is necessary to carry out the concealed reconnaissance task. If the enemy forces are discovered, the captain will decide whether to evade or prepare to attack. The standby phase includes several activities, The captain makes the decision according to the mission target and the current situation of both sides. In order to make the mission successful, several capabilities, including correct decision making, maneuver occupation survivability are important. Those capabilities can be described by the probabilities of correct decision making, encountering blue submarine, occupying the best launch position and the submarine survival. The effectiveness index of the task completion in the standby phase are shown in Figure 3.34. The decision-making capability depends on the extent to which the submarine can give full play to the capability of the command platform it belongs to. It is finally reflected by the comprehensive operational effectiveness. Therefore, the decision-making capability can be evaluated by the influence of various factors on operational efficiency. Many factors play key roles in anti-submarine operational strategy, such as the ability of the commanders, information accessing, targets fusion and processing, and the effective operation of command system. According to the characteristics of submarine operational command process and structure, the main factors affecting the decision-making include the stability, continuity, concealment, correctness and control ability of the command. Those factors can be measured by the perceptible target type, the correctness of perception, the stability of command, and control efficiency of detection system. The encounter probability is mainly related with the distance, the position and course detection and calculation error the blue submarine. Therefore, the encountering probability of blue submarine can be measured by these three factors.

Construction of Evaluation Index System

79

Figure 3.34. Evaluation index in the standby. The probability of occupying the best launch position refers to the probability of condition that the submarine can calculate the best launch position through the fire control system and occupy the position in time after the target is found. When the target motion factors are given and the torpedoes are ensured with a high hit rate (such as: ≥ 0.8), it is unnecessary to consider the counter weapon threat during the position occupation. It is necessary to consider the probability of the submarine occupying the best launch position based on the turning speed, voyage limitation of the torpedo and the speed of submarine. The submarine survivability is the probability that the submarine can survive against blue firepower in the process from discovering the target to approaching and occupying its launching position. The probability should be analyzed and calculated separately according to the types and the performance of the targets counterattack weapon. In general, the counterattack weapons of the target include torpedo, guide depth-charge, and rocket depth charge, etc. (iv) Operational phase While evaluating the effectiveness of anti-submarine mission in operational phase, the attack is a dynamic and antagonistic process. When the red submarine enters the operational sea battle field, it may be attacked by the blue anti-submarine. Therefore, while constructing effectiveness, the ADC should be considered. The effectiveness evaluation index of the antisubmarine mission is composed of several capabilities: torpedo availability A, credibility D and torpedo operational capability C, which is shown in Figure 3.35. In the operational phase, in order to successfully complete the mission, the torpedo must first be able to work normally, and has the ability to break through various countermeasures during the attack, as well as the ability to capture, track, hit and destroy the target. When the torpedo is found by the target, the evasive maneuver can be performed and the

80

Deping Zhang and Xuefeng Yan

Figure 3.35. Indexes of operational phase. hydroacoustic jamming device can be released. Therefore, the torpedo should also have the anti-interference capability. This process can be described by several factors including penetration probability, capture target probability, track target probability, hit target probability, damage probability, and torpedo anti-interference capability respectively. (v) Back phase The typical operational missions in the back phase are the same as the navigation phase. It is necessary to be able to break through the enemy defensive system. Therefore, the indexes mainly contain the penetration capability, which is described by penetration probability. The specific influence factor indexes are the same as the penetration probability of the navigation phase.

3.6. Selection of Effectiveness Evaluation Index 3.6.1.

Indexes Selection Based on Multiple Correlation Analysis

The key point to establishing reasonably index system and effectiveness evaluation is the index selection based on multiple correlation analysis [98]. From the perspective of operational effectiveness and system performance. And the index with larger multiple correlation coefficient in criterion layer can be eliminated by using the multiple correlation coefficient. With the relative discrete coefficient as the information content of the index, the index with

Construction of Evaluation Index System

81

small information content is eliminated through the cumulative information sensitivity analysis, so as to achieve the optimal target of the effectiveness evaluation index. (1) The Cyclic Screening of Multiple Correlation Coefficient Index Multiple correlation coefficient is an index to measure the degree of multiple correlation, which can be calculated by the single correlation coefficient and the partial correlation coefficient. The larger the multiple correlation coefficient is, the closer the linear correlation between the factors or variables is. The essence of the multiple correlation is the correlation between the actual observed value Y and the predicted value of p independent variable. The multiple correlation coefficient is an index to measure the degree of linear correlation between one variable and the others. It can only be calculated indirectly by certain methods. The variables x1 , x2 , · · · , xk cannot be measured directly but the correlation of variable y and linear combination constructed from multiple variables x1 , x2 , · · · , xk can simple. Therefore, it is feasible to construct a linear combination of x1 , x2 , · · · , xk and calculate the simple correlation coefficient. The detail is shown as follows. The first step is to make a regression with y to x1 , x2 , · · · , xk , and get the equation, yˆ = β0 + β1 xˆ1 + β2 xˆ2 + · · · + βk xˆk

(3.28)

The second step is to calculate the simple correlation coefficient, which is the multiple correlation coefficient between y and x1 , x2 , · · · , xk R= p

¯ yˆ − y) ¯ ∑(y − y)( 2 ¯ ∑(yˆ − y) ¯2 ∑(y − y)

(3.29)

The difference between multiple and simple correlation coefficient is that the value range of the latter is [−1, 1], while the value range of the former is [0, 1]. In the case of two variables, the regression coefficient can be positive or negative. Therefore, there are also positive and negative correlations when studying correlation. However, in the case of multiple variables, there are two or more partial regression coefficients, which cannot be distinguished by positive and negative. Therefore, the multiple correlation coefficient only takes positive values. Multiple correlation coefficient focuses on the correlation degree between single index and multiple indexes. It has some advantages comparing with of Pearson or partial correlation coefficient. The smaller the multiple correlation coefficient is, the smaller the overlap of information reflected by other indexes is. The indexes with small multiple correlation coefficient should be retained. In the screening, the maximum of the multiple correlation coefficient will be compared with the given critical value in the same level. If the maximum value is greater than the critical value, the index will be eliminated. The remaining indexes are screened according to the above-mentioned process and will stop when the maximum value is less than the critical value. The cyclic screening for the correlation coefficient index is shown in Algorithm 3.2. (2) The Index Selection Based on the Variation Coefficient In statistical analysis, while comparing the variation degree of two or more variables, the standard deviation can be used directly for comparison if the measurement unit of the

82

Deping Zhang and Xuefeng Yan

Algorithm 3.2. Cyclic screening for the correlation coefficient index algorithm Step 1. The max-min-normalization algorithm are often adopted for the data standardization xi j − min xi j 1≤ j≤n

xi j =

max xi j − min xi j

1≤ j≤n

(3.30)

1≤ j≤n

Step 2. Setting the multiple correlation coefficient e. if e > 0.9, it means the correlation between indexes is strong. Generally, e can be set equal to 0.9. Step 3. The multiple correlation coefficients of the indexes in the same criterion layer can be calculated respectively. Step 4. Index screening. In a criterion layer, the maximum of all the multiple correlation coefficient is ρmax . If ρmax > e, it indicates that the indexes corresponding to ρmax have maximum information overlaps with the others, and the index should be deleted. For the remaining indexes, Step3 and Step4 will be repeated. The screening will be stopped when the maximum multiple correlation coefficient is less than the threshold e.

index is the same and mean. If not, the ratio of standard deviation to the mean (relative value) should be used for comparison. CV = σ/µ

(3.31)

where µ and σ are the mean and standard deviation of variable X respectively. In practice, sample mean x¯ = r

1 n−1

1 n

n

∑ xi and sample standard deviation s = i=1

n

¯ 2 are commonly used to approximate the mean and standard deviation. ∑ (xi − x)

i=1

Therefore, the variation coefficient can be approximated by: c=s CV x¯

(3.32)

The variation coefficient can be used to compare the discrete degrees of two variables with different overall mean, because it can eliminate the impact on variables with different unit and(or) mean. The ratio of standard deviation to its mean value is a relative index to measure the discrete degree of measured data, and it is used to the discrete degree of different data. Based on PCA [99], [101], the variation coefficient can be regarded as the contribution of the variation degree of the variable xi to the sample x. Therefore, the variation coefficient can be defined as the information content of the index variables in the effectiveness evaluation index selection. The index selection method based on cumulative information sensitivity is to choose related indexes with less contribution from the indexes of upper layer.

83

Construction of Evaluation Index System

The ratio of the standard deviation to the mean value can be defined as relative dispersion coefficient to represent index information content. The cumulative information sensitivity is the ratio of the summation of top p indexes to overall indexes. When informational sensitivity of top p indexes reach a threshold, it can be considered that the top p indexes have great effect on comprehensive evaluation and should be maintained. The index selection algorithm based on the variation coefficient is shown in Algorithm 3.3. Algorithm 3.3. The index selection algorithm based on the variation coefficient Step 1. The information content CVi of the index Xi is calculated by the Equation(3.32). Step 2. The n original indexes X1 , X2 , · · · , Xn are sorted in descending order according to the information content. The X(1), X(2), · · · , X(n) is the sorted result. That is, the corresponding information content satisfies CV(1) ≥ CV(2) ≥ · · · ≥ CV(n)

(3.33)

Step 3. the cumulative information sensitivity r p is calculated in the step. The information sensitivity r p indicates the ratio of the top p indexes to the total information content after the data was sorted in descending order in Step 2, and the calculation is as follows: p

n

r p = ∑ CV(i)/ ∑ CVi i=1

(3.34)

i=1

Step 4. Index Screening For a given threshold r0 , if r p ≥ r0 ≥ r p−1 , the top p indexes with large information content are reserved, and others are removed.

In PCA, the principal components with more than 85% contribution rate are kept to make sure that the combined variables can reflect the most information of origins. The threshold r0 of the cumulative information sensitivity is set to be 85%. That is, when r p ≥ 85%, it is considered that most of the initial index set information can be reflected by the selected p indexes and the index of the SoS is reasonable.

3.6.2.

Dynamic Index Selection Based on Sensitivity Analysis

The sensitivity analysis [98][99][100] includes analytical solution method and simulation experiment methods. For indexes with explicit analytic relationships, the mathematical analysis (called local sensitivity analysis) should be conducted. For indexes that are implicit in analytic relationships or indexes with strong nonlinear, the sensitivity analysis (called the global sensitivity analysis) should be used based on the digital simulation. Considering the complexity of the equipment system, the simulation is generally used for the sensitivity analysis to grasp a more accurate and comprehensive relationship between the actual operation elements and the evaluation index. The specific steps are as

84

Deping Zhang and Xuefeng Yan

follows. In actual external operational conditions a large number of Monte Carlo simulation experiments are executed based on the simulation model or the agent-based model (such as support vector regression machine, GMDH algorithm). The statistics analysis for simulation experiment results can be carried out. The essence of the experiment design is to select representative points to reduce experiments time and improve efficiency. There are many methods for the experiment design such as single factor experiment, two factors, random block, incomplete block, Latin square, orthogonal experiment, optimization, robustness, uniform, etc. There are multiple experimental factors and horizontal values because of the complexity of operational condition. The bias experiment is used for the single factor sensitivity analysis. The curve graph and the sensitivity coefficient are used for the analysis of the single factor experiment. The horizontal axis is the factor, and the vertical axis is the evaluation index in curve chart. The sensitivity analysis results can be reflected by the curve trend and the correlation correctness can be verified. Based on the simulation data, through the single factor analysis, caused by the external different actual conditions can be get the slight disturbance to the input of a certain index element, while the other parameters is fixed. The corresponding output can be obtained by the system simulation and the sensitivity is calculated by the calculus of differences. The sensitivity S is as follows. 4I (3.35) 4C where, 4I is the increment of the ability index triggered by operational elements change; 4C is the operational element change. Considering that the dimensions of each operational factor vary greatly, the relative change sensitivity can be used. S=

Sr =

4I

I1 +I2 2 4C

(3.36)

where, 4I is the increment of the ability index triggered by operational elements change. 4C is the operational element change. I1 is the original value of the capability indexes; I2 is the new value for the capability indexes. The basic idea of the dynamic index selection method [99] based on sensitivity analysis is as follows: (a) According to the characteristics of the operational mission, the effectiveness evaluation index considering all the evaluation results is constructed and a complete index is built up. (b) The system effectiveness is evaluated based on this index system. (c) The sensitivity analysis is used to calculate the influence of a single index on the effectiveness. (d)The indexes with little impact on effectiveness will be erased to build up adaptive evaluation index.

3.6.3.

Index Selection Based on Kernel Principal Component Analysis

The index selection methods based on principal component analysis (PCA) is efficient to transform many linearly correlated indexes into linearly unrelated indexes with less number

85

Construction of Evaluation Index System

[101]-[105]. It transforms complex related indexes into a few comprehensive indexes. The indexes, which can reflect the most important information in the system, are regarded as the first principal component followed by the second principal component. The number of principal components is determined according to the percentage of information quantity. Each principal component is linearly independent. Therefore, the multivariate high-dimensional space problem is transformed into a low-dimensional comprehensive index problem. Assuming that E is the set of all unit vectors, X1 , X2 , · · · are in n dimensional space, n R , and Xn are a linear corelated m dimension random variable. Then a linear combination Z = AX T of Xn is constructed, where Xm×n = (X1 , X2 , · · · , Xn ), A ∈ E. If Z1 is the random variable with the largest variance Var(Z1 ) = max{Var(Z)}, then Z1 is the first principal component of vector set X, regarded as Z1 = β1 X T , β1 ∈ E. The second principal component of X is linearly independent with Z1 and has maximum variance in Z = AX T , regarded as Z2 = β2 X, β2 ∈ E. Similarly, several other levels of principal components can be defined. Assuming that the value of n index is obtained through m samples X1 = (y11 , y21 , · · · , ym1 )T , X2 = (y12 , y22 , · · · , ym2 )T ,· · · , Xn = (y1n , y2n , · · · , ymn )T , then the main steps of PCA are as follows: The first step is to standardize the raw data. It includes the adjustment of indexes and the standardization of data. The standardization of data mainly focus on non-dimensionlization and satisfies E(X) = 0. Let xi j =

yi j −y¯ j Sj ,i

= 1, 2, · · · , m; j = 1, 2, · · · , n, where y¯ j =

y¯ j )2 . Then the normalized matrix X can be obtained.   x11 x12 · · · x1n  x21 x22 · · · x2n    X = . .. ..  ..  .. . . . 

1 m

m

∑ yi j , s2j =

i=1

1 m−1

m

∑ (yi j −

i=1

(3.37)

xm1 xm2 · · · xmn

The second step is to calculate the correlation coefficient and get the correlation coefficient matrix R, which is a real symmetric matrix:   1 r12 · · · r1n  r21 1 · · · r2n  1   R= XT X =  . (3.38) .. ..  . . . m−1  . . . .  rm1 rm2 · · · rmn

The third step is to solve the characteristic equation |λI − R| = 0 to get the eigenvalues λ1 , λ2 , · · · , λn (λ1 ≥ λ2 ≥ · · · ≥ λn ≥ 0) of R and the corresponding linearly independent unitized eigenvectors Ai = (ai1 , ai2 , · · · , ain ) i = 1, 2, · · · , n. Then the orthogonal matrix A can be formed.   a11 a12 · · · a1n a21 a22 · · · a2n    A= . (3.39) .. ..  . . .  . . . .  an1 an2 · · · ann

86

Deping Zhang and Xuefeng Yan The fourth step is to calculate principal components.      z11 z12 · · · z1n x11 x12 · · · x1n a11 a12 · · · a1n  z21 z22 · · · z2n   x21 x22 · · · x2n  a21 a22 · · · a2n        .. .. ..  =  .. .. ..   .. .. ..  .. .. ..  .     . . . . . . . . . . .  zm1 zm2 · · · zmn xm1 xm2 · · · xmn an1 an2 · · · ann

(3.40)

Let Z j = (z1 j , z2 j , · · · , zm j )T be the jth principal component, Z = (Z1 , Z2 , · · · , Zn ), then Z = XAT . The fifth step is to select principal components, and calculate the contribution ratio and cumulative contribution ratio of each component. n

The contribution ratio of the jth principal component Z j is p j = λ j / ∑ λi . Then the i=1

cumulative contribution ratio of the first r principal components is as follows. r

r

∑ pj =

j=1

n

∑ λ j / ∑ λi

j=1

(3.41)

i=1

The cumulative contribution ratio indicates that the ability of the first r principal components can reflect the information in the original sample. When it reaches a certain level, the first r principal components can be used to represent the information from the original sample. In PCA, it is assumed that the size of the characteristic root determines how much information we are interested in. That is, small eigenvalue is often the noise. In practice, the projection to the direction of a smaller eigenvalue may also include the data we are interested in. The orientation of the eigenvectors is required to be orthogonal to each other in PCA, and it makes PCA vulnerable to the abnormal points. The kernel principal component analysis (KPCA)[106][107] is a non-linear extension of the PCA. The insufficient feature extraction problem of PCA is overcomed in KPCA. The steps of KPCA are the same as PCA except that the original data is replaced by kernel function. There are two innovations in KPCA. 1) In order to handle non-linear data better, a non-linear mapping function φ(·) is introduced to transform the data from original space into a high-dimensional space. φ(·) is implicit and does not have the specific form. 2) A theorem is introduced: any vector (even the basis vector) in a space can be linearly represented by all samples in the space. For a linearly inseparable dataset, the nonlinear mapping function φ(·) is introduced. The data set can be mapped to a higher dimension and then be partitioned.   φ(x1 )T 1 N 1 (3.42) C= ∑ φ(xi)φ(xi)T = N [φ(x1), · · · , φ(xN )]  · · · T  N − 1 i=1 φ(xN ) Let X T = [φ(x1 ), · · · , φ(xN )], then

C=

1 XT X N −1

(3.43)

Construction of Evaluation Index System

87

where, φ(x) is unknown and the above equation cannot be solved. Even if φ(x) is known, its calculation cost is too large. Therefore, the kernel function is introduced. The theory of kernel function is as follows:    κ(x1 , x1 ) · · · κ(x1 , xN ) φ(x1 )T   .. .. .. K = XX T =  · · ·  [φ(x1 ), · · · , φ(xN )] =   . . . T φ(xN ) κ(xN , x1 ) · · · κ(xN , xN )

(3.44)

(XX T )u = λu

(3.45)



The above K can be calculated according to the property of the kernel function. The following part focuses on the relationship between K and C. If the eigenvalues and eigenvectors of K need to be calculated

where u is the eigenvalue of the matrix, and λ is the eigenvector of the matrix K. Both sides of the Equation (3.45) will be multiplied by X T , then X T (XX T )u = λX T u

(3.46)

that is, (X T X)(X T u) = λ(X T u). Since (N − 1) ·C = X T X, we find that the eigenvalues of the matrices K and C are both λ and the eigenvectors of C are X T u. The eigenvectors need to be transformed to unit vector. v = =

1 1 1 XT u = √ XT u = √ XT u T T T ||X T u|| u XX u u Ku 1 1 √ XT u = √ XT u uT λu λ

(3.47)

where λ and u can be obtained by the matrix K, but X T is still unsolvable and the eigenvectors of C cannot be calculated. In fact, the projection of x on v is important and can be calculated, 1 1 vT φ(x j ) = ( √ X T u)T φ(x j ) = √ uT Xφ(x j ) λ λ     φ(x1 )T κ(x1 , x j ) 1 1     .. = √ uT  ...  φ(x j ) = √ uT   . λ λ T φ(xN ) κ(xN , x j )

(3.48)

(3.49)

All variables in the above formula can be solved. That is, under the condition no eigenvectors are solved, the projections of the sample on the eigenvectors can be directly calculated. The process of KPCA is shown in Algorithm 3.4. The (kernel) principal component analysis is used to select the evaluation index. The selected principal components can be regarded as the emergence indexes in comprehensive evaluation. The contribution rate of each principal component is used as the weight of the emergence index, and it will be aggregated into a comprehensive effectiveness index.

88

Deping Zhang and Xuefeng Yan

Algorithm 3.4. Kernel principal component analysis algorithm 1) Data X of n indexes (each index contains d samples) is written into a matrix of d × n. 2) Kernel matrix will be calculated. The parameters in the Gaussian radial kernel function need to be determined at first, then the kernel matrix K can be calculated according to the formula k(xi , xi ) = φT (xi ) · φ(xi ). 3) The eigenvalues of K, (λ1 , λ2 , · · · , λn ), (u1 , u2 , · · · , un ) will be calculated.

and its corresponding eigenvectors

4) The eigenvalues are sorted in descending order (by selection sort), and the eigenvectors are adjusted respectively. 5) The eigenvectors (u1 , u2 , · · · , un ) are obtained by the unit orthogonalization according to the Schmidt orthogonalization. 6) Calculate the contribution rate and cumulative contribution rate of each feature. 7) Select the first p principal components, let cumulative contribution rate to be bigger than 85%.

Note: the (kernel) principal component analysis is used to execute the effectiveness index selection and comprehensive effectiveness evaluation, which can avoid the subjectivity of AHP and ANP.

Chapter 4

Mathematical Modeling of Effectiveness Index 4.1.

Basic Index Measurement

4.1.1.

Index Dimensionless Based on Utility Function

There are two concepts involved in the effectiveness evaluation [108]-[116]. One is the actual value of each index, that is, index attribute value. The other is the evaluation value, that is, the index value. The physical meaning of each index is different, and there is a dimensional difference. As a result, the value and range of different index attribute are greatly different and the relative size of different attribute values is not clear in the same index. This problem can be solved by the index dimensionless. Dimensionless is a method that the dimensional influence of original variables can be eliminated by mathematical transformation, including two non-quantitative methods based on utility function and membership function. (1) The Application of Utility Function in the Index Dimensionless The attribute values of lower-level indexes are monotonous in the SoS. Therefore, the dimensionless processing can be performed by using the utility function for lower-level indexes. The curve dimensionless function is a kind of utility function when the function value is [0, 1] and the monotone condition can be satisfied. (i) Basic utility function In the multi-attribute decision making, the basic utility function can convert the attribute value into the mapping relationship of the objective utility measure, which is also called attribute value function or attribute transformation function. In the effectiveness evaluation, the index value is converted into the effectiveness value by the utility function, which is called evaluation index utility function. Different utility functions can be selected for the different types of indexes and attributes. The common utility functions include Sigmoid, linear and Gaussian functions. The common forms of utility function are shown in Figure 4.1. In order to accurately express the purpose and intention of operational effectiveness

90

Deping Zhang and Xuefeng Yan

Figure 4.1. The common forms of utility function. evaluation, the utility interval [xl , xr ] should be determined at first and it can be used to determine the parameters of utility function. The utility function may vary significantly in the interval. According to the property and numerical type of evaluation index, it is possible to determine which utility function is more suitable and the parameters of the function can be determined. In Figure 4.1, the benefit Sigmoid utility function corresponds to the benefit indexes and it is monotonic increasing. If x ∈ [xl , xr ], then 0.1 ≤ f (x) ≤ 0.9. The benefit linear utility function corresponds to the benefit indexes and it is monotonic increasing. If x ∈ [xl , xr ], then 0 ≤ f (x) ≤ 1. The cost sigmoid utility function corresponds to the cost indexes and it is monotonic decreasing. If x ∈ [xl , xr ], then 0.1 ≤ f (x) ≤ 0.9. The cost linear utility function corresponds to the cost indexes and it is monotonic decreasing. If x ∈ [xl , xr ], then 0 ≤ f (x) ≤ 1. The interval benefit Gaussian function corresponds to the interval benefit indexes and it is non-monotonic. If x ∈ [xl , xr ], then 0.2 ≤ f (x) ≤ 1. The general formula of the Sigmoid utility function is shown as follows: fs(x) =

1 1 + exp(−a(x − b))

(4.1)

where, the parameters a and b can be determined by the utility interval and b = 12 (xl + xr ). Assuming that f s(x1 ) = 0.1, a = x2r ln9 −xl can be determined by the Equation (4.1). The utility function f s (x) and (1 − f s(x)) can be directly taken as the benefit index and the cost index respectively. The general formula of linear utility function is shown as follows:   0, x < a, fl (x) = (x − a)/(b − a), a ≤ x ≤ b, (4.2)  1, b < x

where a = xl , b = xr , f l (x) and (1 − f l (x)) are taken as utility function of benefit index and cost index respectively. The general formula of Gaussian utility function is shown as follows:   (x − b)2 fg (x) = exp − (4.3) 2a2

Mathematical Modeling of Effectiveness Index

91

where, the parameters a and b can be determined by the utility interval and b = 12 (xl + xr ). b−xl Assuming that f g (xl ) = 0.2, a = √−2 can be determined by the Equation (4.3). f g (x) ln 0.2 and (1 − f g (x)) are taken as utility function of benefit index and cost index respectively. (ii) The utility function of index classification The evaluation indexes can be preliminarily divided into five types, such as success index, failure index, proportion index, time index and utilization rate index. The utility functions of index classification are designed respectively for the numerical types of five kinds of indexes. (a) The utility function of mission success index The mission success index refers to the index denoting the capability measure of completing operational mission. Generally, this kind of index belongs to benefit index, the greater its value is, the stronger the capability is. For example, the values of indexes such as “sonar target search capability” and “missile seeker recognition probability” may be 0.99, 0.999 or 0.9999. The value of the mission success index is generally nonlinear and should be represented by the Sigmoid utility function. Firstly, the index value is transformed by logarithmic function nr = − log(1 − x), then it is substituted into the basic utility function. The Sigmoid utility function of mission success index is shown as follows: fr (nr ) =

1 1 + exp(−a(nr − b))

(4.4)

where nr is the result of logarithmic transformation of index x, a and b are unknown parameters, which can be determined by the utility interval. (b) The utility function of mission failure index The mission failure index refers to the index represented by the capability measure of the mission that cannot be completed. Generally, this index belongs to cost index, the smaller the value is, the stronger the capability is. For example, the values of indexes such as “failure rate of equipment” and “bit error rate of communication equipment” may be 0.01, 0.001 or 0.0001. The mission failure index has nonlinear characteristics that should be represented by the Sigmoid utility function. Firstly, the index value is transformed by logarithmic function n f = − log(x), then n f is taken as a variable and the parameters of utility function are determined. In this way, the index can be converted from the cost type into the benefit type. The basic form of utility function of mission failure index is shown as follows: f f (n f ) =

1 1 + exp(−a(n f − b))

(4.5)

where n f is the result of logarithmic transformation of index value x. a and b are undetermined parameters, which can be determined by utility interval. (c) The utility function of proportion index The proportion index can be obtained based on the statistical results of proportion. For example, underwater acoustic signal connectivity rate, data throughput rate, signal-to-noise

92

Deping Zhang and Xuefeng Yan

ratio, etc. The value range of this index is [0, 1], which has the linear variation, and most of them are benefit growth type. It can be expressed as follows:   0, x < xl , fb (x) = (x − xl )/(xr − xl ), xl ≤ x ≤ xr , (4.6)  1, xt < x

where xl and xr are the upper and lower limits of the utility interval. (d) The utility function of time index

In the time index, time measure is used to express the quality of completing an operational mission. For example, “decision-making process delay” is expressed as the command decision-making time delay for typical operational mission. The shorter the delay time is, the stronger the decision-making capability is; otherwise, the worse the capability is. The communication delay between nodes refers to the communication delay between each network node. The shorter the delay is, the better the timeliness of information communication is. The time index is a type of cost index and the value range is [0, +∞]. Its Sigmoid utility function is expressed as follows: ft (x) = 1 −

1 1 + exp(−a(x − b))

(4.7)

where a and b are determined by the utility interval. (e) The utility function of utilization rate index In the utilization rate index, the usage of operational resources is represented by percentage. For example, the availability of equipment, mission execution rate (MCR), sortie generation rate (SGR), utilization and satisfaction of maintenance equipment and appendage, usage rate of port, utilization of channel capacity, etc. The value range is [0, 1]. If the utilization of resources is very small, the efficiency of the equipment resources is low. If the resources utilization rate is big, the performance-cost rate is low. In order to balance them, the resource utilization is generally within a proper range based on the top planning design principle. It can be seen that the utilization index is a type of interval benefit type. The index value shows favorable utility in a certain range, and the utility is poor is outside the range. The Gaussian utility function of utilization index can be expressed as follows:   (x − b)2 fu (x) = exp − (4.8) 2a2 It is assumed that the utility interval of utilization index is in [xl , xr ]. The parameter value depends on the property of the index itself and different indexes have different parameters.

4.1.2.

Index Dimensionless Based on Membership

The membership theory is the core of fuzzy mathematics. Fuzzy comprehensive evaluation is based on fuzzy mathematics and it transforms qualitative evaluation into quantitative evaluation. That is, fuzzy mathematics is adopted to make a comprehensive evaluation

Mathematical Modeling of Effectiveness Index

93

for things or objects that are restricted by many factors. It has clear results and strong systematicness. The fuzzy and unquantifiable problems can be solved preferably. And it is suitable to solve various uncertainty problems. The indexes are generally divided into qualitative and quantitative indexes. When fuzzy mathematics is adopted for evaluation, the membership of the indexes should be determined at first and the details are shown as follows: (i) The membership of qualitative index The qualitative index usually used for the objects which cab not be expressed by quantitative method, such as reasonable, more reasonable, general, poor, very poor, etc. The options are ranged from positive to negative and the scale is called Likert. A prepared scale is used for Likert to measure people’s attitude towards products or services. It allows respondents to give varying degrees of answers for each survey item. The qualitative index option can be designed as a 5-level Likert scale. The percentage statistics method is used to carry out percentage statistics on the opinion of the experts. The results is taken as the membership degree of the index. (ii) The membership of quantitative index Firstly, a series of values is selected from a continuous interval as the demarcation points. Then, the actual index value is processed by linear interpolation formula. The corresponding membership of the index values can be obtained. The other choice is that the values with obvious boundary are artificially selected from all index values [117]. And then all indexes values are divided by this index value. The obtained value are normalized, which can be taken as the membership of each index value. It is assumed that a certain index value is x and the membership function is u(x). Then the membership u1 (x), u2 (x), · · · , uL+1 (x) of the factor in the L + 1 level interval is shown as follows:   1, x ≤ s1 u1 (x) = (s − x)/(s2 − s1 ), s1 ≤ x ≤ s2 (4.9)  2 0, x ≥ s2   1, x ≤ s1 u2 (x) = (s − x)/(s2 − s1 ), s1 ≤ x ≤ s2 (4.10)  2 0, x ≥ s2 ··· (4.11)   0, x ≤ sL uL+1 (x) = 1 − uL (x), sL ≤ x ≤ sL+1 (4.12)  1, x ≥ sL+1

There is another form of the method supposing the evaluation index ui , i = 1, 2, · · · , m has different quantitative values. The maximum one should be selected to divide with each quantitative value, which can be regarded as the membership of the corresponding index value. And it should be noted that “positive indexes” and “negative indexes” are different at this moment. Since there may be different dimensions between the indexes, the collected data should be dimensionless before data analysis, which is to make it possible to compare indexes in

94

Deping Zhang and Xuefeng Yan

different units and directions. In dimensionless, the concept of membership function in fuzzy mathematics is adopted is fuzzy quantification for positive and negative indexes. (a) Fuzzy quantification of positive index The positive index means that the index contribution rate to the overall objective increases with the evaluation result. The fuzzy quantification of the positive index is shown as follows: h (
i xi max +xi min 1 1 π + sin x − , xi min ≤ xi max i 2 2 xi max −xi min 2 (4.13) Ri (x) = 0, xi ≤ xi min or xi ≥ xi max where, Ri is the evaluation value of the evaluation index i after dimensionless. xi is the original value of the index i. xi max is the maximum value and xi min is the minimum value. (b) Fuzzy quantification of negative index The negative index means that the index contribution rate to the overall objective decreases with the increase of the evaluation result. The fuzzy quantification of the negative class is shown as follows: h (
i xi max +xi min 1 π 1 − sin x − , xi min ≤ xi max i 2 2 xi max −xi min 2 Ri (x) = (4.14) 0, xi ≤ xi min or xi ≥ xi max where, the meaning of each symbol is the same as above. (c) Fuzzy model of moderate index Different from the positive and negative indexes, the moderate index requires a “temperate” value. When the index value is lower than the moderate value, the function increases monotonically. When greater than the moderate value, the function decreases monotonously.  h
i xi max +xi min 1 π 1  + sin x − , xi min ≤ ximod  i 2  2 2 h xi max −xi min i
xi max +xi min 1 1 π Ri (x) = (4.15) , ximod ≤ xi max 2 − 2 sin xi max −xi min xi − 2    0, xi ≤ xi min or xi ≥ xi max

where, ximod is the most moderate value of the index i. Then, the evaluation indexes are between [0, 1]. After multiplying with the weight coefficient, the evaluation result should also be between [0, 1], which can be directly compared.

4.2. Mathematical Modeling of System Effectiveness Index 4.2.1.

System Capability Index Modeling

The calculation method of system capability index is always the hot topics in the research field. The qualitative or quantitative evaluation of the capability can provide capability input for command and information system for SoS. It can also supports the operational deployment and configuration optimization. There are different methods to calculate operational

Mathematical Modeling of Effectiveness Index

95

capability in different situations, including formula, index aggregation and prediction methods. (1) Formula Method The operational capability has explicit data standard definition that can be applied directly as algorithm specification. For example, the calculation formula of aircraft take-off and landing capabilities T L is shown. TL =

f (R1 , R2 , · · · , R5 )

= k(ω1 eθ1 + ω2 eθ2 + · · · + ω5 eθ5 )

(4.16)

In the Equation (4.16), ωi ≥ 0, i ∈ [1, 5] represents the weight of each input component and 5

∑ ωi = 1, θi , i ∈ [1, 5] is the adjusting factor. R1 ∼ R5 and k are the equipment performance

i=1

and the overall adjusting parameters respectively. (2) Index Aggregation Method The campaign (operational) objectives (functions and missions) of SoS are hierarchical. Therefore, the operational capability also is the hierarchical. The system operational capability is aggregated by the indexes and quantity of single equipment. The capability of single equipment is aggregated by the tactical and technical indexes. The SoS structure is complex and the operational capability of any level may be aggregated by several lower-level indexes (or tactical and technical indexes). The essence of index aggregation is to reduce the dimension of multiple indexes. There are two typical methods, model-based and data-based aggregation further, according to different reduction dimension way. The model-based aggregation into includes analytic aggregation and synthetic evaluation aggregation according to the different models. Mathematical analysis is an index aggregation based on analytic model. Computer simulation evaluation is an extension of model-based aggregation. The aggregation based on synthetic evaluation model is related to the structure of index system closely. For hierarchical structure, the index aggregation is usually executed from bottom to top whether tree or network structure. Typical aggregation methods include sum method, product method, proportion method, exponential method, linear and nonlinear regression method, penalty function, logical operation and so on. The linear weighting is widely used because it is easy to operate and simple in logic, especially in the joint operation evaluation and the systems testing of SoS. At present, when the operational capability is evaluated by analytical method, the AHP is adopted for index aggregation. It can decompose complex problems into hierarchical ordered structures and make it simplify. The expert evaluation or investigation is used to construct the judgment matrix and then the weight can be determined. It not only integrates the expert experience effectively, but also combines the advantages of qualitative and quantitative. Therefore, AHP is applicable mothed for aggregation of the operational capability. However, the relationship at all levels is oversimplified and the index aggregation is simplifying. Therefore, the nonlinear aggregation method is generally adopted in the practice, whose basis is the weighted sum aggregation and weighted product aggregation.

96

Deping Zhang and Xuefeng Yan (i) Weighted sum aggregation

The weighted sum aggregation means that the lower-layer indexes are aggregated into the upper-layer in cooperative and complementary manner according to their respective weights, which is corresponded to “OR” gate in the logic gates. Each lower-layer index is a component of the upper-layer and the same level indexes are relatively equal and independent. The specific description is shown as follows: m

xˆi =

∑ wi j xˆi j

(4.17)

j=1 m

where, the weight wi j satisfies ∑ wi j = 1. j=1

In general, g is the nonlinear aggregate function, xˆi is the upper-layer index value, xˆi1 , xˆi2 , · · · , xˆim are the lower-level indexes values. wi1 , wi2 , · · · , wim are the weights that are corresponded to the lower-level indexes. Then the principle of the weighted sum aggregation can be expressed as follows: xˆi = g(xˆi1 , xˆi2 , · · · , xˆim ; wi1 , wi2 , · · · , wim )

(4.18)

For example, the auxiliary decision-making capability of the SoS includes response time, degree and quality. They are relatively equal and independent and the lack of any one will not have a decisive effect on the effectiveness. And they are aggregated to the upper-level indexes by using the weighted sum. The linear weighted sum is adopted to aggregate all the lower-level indexes into the upper-level. And the decisive influence of individual lower-level indexes on upper-level is ignored. Therefore, the actual situation of the evaluation object cannot be reflected truthfully. Some operational capability indexes cannot be aggregated by the weighted sum. For example, some lower-level operational capability indexes are aggregated into upper-level by the “AND” relationship. That is, each lower-level operational capability index is the critical factor of upper-level. As long as one of them is zero, the upper-level operational capability is zero. As is shown in Figure 4.2, the attack capability, maneuverability and guidance capability are the critical factors for over-the-horizon strike capability of air-to-air missile. If the missile does not has any of the three abilities, it will not be able to strike over-the-horizon. It is obvious that the weighted sum cannot describe this “AND” relationship. Therefore, the weighted product aggregation is introduced. (ii) The weighted product aggregation The weighted product aggregation, also known as the power function method, is suitable for scenarios where the importance and weight of the lower-level indexes are different, but are indispensable in the upper-level indexes. If the first is that the index of the same level is highly dependent on each other, or the any lower-level index value is zero, the upper-level indexes will be zero. This aggregation method is corresponded to the “AND” in the logic gates. The principle can be described by the following: m

w

xˆi = ∏ xˆi ji j j=1

(4.19)

Mathematical Modeling of Effectiveness Index

97

Figure 4.2. Over-the-horizon air operational capability (Partial). m

where, the weight wi j satisfies ∑ wi j = 1. j=1

For example, in the command information system effectiveness the command quality is composed of the generation time of the operational plan and the success probability of the guidance. They are interdependent highly and the absence of any one will lead to the failure of the entire information support capability. Therefore, these three capabilities are indispensable for the information support capability and are aggregated using the weighted product. The consistency of the index value is required by the weighted product aggregation. It highlights the index which is small but important. It is suitable for the scenarios where there is a strong correlation between the indexes. This method is inferior to the weighted sum aggregation in highlighting the index weight. And it is relatively complex and has large amounts of computation. (iii) Nonlinear aggregation of index system The hierarchical structure is adopted by most evaluation index systems at present. The lower-level indexes are often aggregated into the upper-level through simple linear weighted sum aggregation. This single aggregation ignores the importance differences between the lower-level indexes. Therefore, the actual condition of the evaluation object cannot be reflected exactly. And the guiding significance of obtained evaluation results is slightly insufficient in practice. In contrast, the nonlinear aggregation can overcome the over-simple affiliation between indexes, highlight the importance of indexes, and describe the aggregation relationship between the lower-level index and the upper-level index. It also reflects the contribution degree of different types of indexes to the whole system, and is convenient for index aggregation. L J It is assumed that “ ” and “ ” represent that the lower-level indexes are aggregated to upper-level by using weighted sum and weighted product aggregation respectively. The affiliation between each index is judged through evaluation of relevant experts. After nonlinear aggregation, the evaluation index system can be obtained as shown in Figure 4.3.

98

Deping Zhang and Xuefeng Yan

Figure 4.3. Comprehensive evaluation index system based on nonlinear aggregation. (3) Prediction Method The operational capability is driven by specific mission and closely related to objective factors, such as environment. In this case, the self-learning and adaptive prediction model are adopted for calculate the capability of equipment. For a certain mission, the capability calculation model can be constructed by analyzing the operational capability and elements. According to the data that are extracted from various relational databases, XML documents, unstructured databases, sensor stream files and logs, a multidimensional dataset is formed through of the cleaning, conversion, deduplication, integration and loading. Based on the output requirements of different capabilities, the basic capability prediction can be completed by the model and the calculated results can be preserved in the capability set. Finally, the mission capability is predicted according to the different capabilities required by specific mission. The architecture of capability prediction model is shown in Figure 4.4. Neural network, support vector machine and Adaboost algorithms can be adopted for the operational capability prediction. These models have strong computational and abstract capabilities. They can be used to deal with nonlinear problems. They are easily combined with several intelligent optimization algorithms to improve their accuracy.

4.2.2.

System Effectiveness Index Modeling

The system effectiveness is the capability that equipment executes the mission. It is comprehensively determined by capability factors and affected by objective conditions such as natural environment. If focus on changing the motion state of the combat objects. The effectiveness evaluation generally includes two aspects. The first is the comprehensive evaluation for SoS, without consideration the influence of the of human, operational environment and situation. The other is the mission-oriented effectiveness evaluation. The human, operational environment and situation have an impact on the effectiveness. There-

Mathematical Modeling of Effectiveness Index

99

Figure 4.4. The architecture of system capability prediction. fore, the modeling of system effectiveness index generally includes two aspects, that is, the comprehensive effectiveness evaluation and the mission-oriented effectiveness evaluation. The comprehensive effectiveness evaluation modeling can also adopt the system capability modeling. It also includes the common formula method, prediction method and index aggregation method (AHP, ANP, index evaluation, ADC, etc.), which won’t go into details here. The following focuses on the mission-oriented effectiveness evaluation index modeling. The mission model is mainly used to describe the hierarchical structure and temporal relationship of the mission. Since the missions are complex, it is necessary to describe the mission profile according to the operational scheme. Then each profile can be refined into different mission stages and units. Therefore, a three-level progressive relationship of “mission profile → mission stage → mission unit” is formed. Mission profile refers to the overall mission requirements, events, environment and participants in a certain period of time from macroscopic view. The mission stage is the development of profile. And the operating conditions or functions are described concretely. The mission unit is the basic component of the warship mission, which describes in detail the name, quantity and application. In each mission unit, the equipment involved and the way it is used are unique. The modeling is shown as follows: (a) Mission stage in the mission profile The mission profile of equipment need to be analyzed and decomposed to concrete stages based on different functional requirements in different time segments. The start and end time of each stage need to be defined. (b) Mission units in the mission stage The mission stage can be refined to mission unit by analyzing the equipment requirements in the mission. And the start and end time of each unit need to be defined. In each unit, the maintenance event will be started when the equipment failure. It should be illustrated that the mission unit cannot exceed the specified termination time, even if there is maintenance events, otherwise, or the mission will be judged as failure. The mission model of equipment is shown in Figure 4.5.

100

Deping Zhang and Xuefeng Yan

Figure 4.5. The mission model of equipment. The mission is always a typical multi-stage mission. The operational effectiveness E is taken as the probability of completing T stage mission, that is, E = Pr{s(1) = 1, s(2) = 1, · · · , s(T ) = 1}

(4.20)

It is assumed that events s(t) = 1 and s(t) = 0 indicate that the system has or has not completed t stage mission Mt . The effectiveness model of SoS can be established according to the mission relationship in each stage. First, Mt1 , Mt2 are inter-independent of each other for any t1 ,t2 ∈ {1, 2, · · · , T }. A group of independent mission are allocated in T phases. Therefore, there are: E = Pr{s(1) = 1, s(2) = 1, · · · , s(T ) = 1} T

=

∏ Pr{s(t) = 1} = ∏ E(t) t=1

(4.21)

T

(4.22)

t=1

Second, the missions in each stage have Markov property for any t ∈ {1, 2, · · · , T }. The mission completed by the system in a certain stage is only related to the completion of mission in the previous stage, that is: Pr{s(t) = i|s(t − 1) = i1 , s(t − 2) = i2 , · · · , s(1) = it−1 }

= Pr{s(t) = i|s(t − 1) = i1 }, i1 , i2 , · · · , it−1 ∈ {0, 1}

(4.23) (4.24)

And Pr{s(t) = 1|s(t − 1) = 0} = 0. That is, if the mission needs to be completed in a certain stage, the missions of previous stage must be completed. Therefore, there are: E = Pr{s(1) = 1, s(2) = 1, · · · , s(T ) = 1} =

∑

i1 ∈{0,1}

(4.25)

Pr{s(1) = i1 } · Pr{s(2) = 1|s(1) = i1 } · Pr{s(3) = 1|s(2) = 1}

· · ·Pr{s(T ) = 1|s(T − 1) = 1} = Pr{s(T ) = 1|s(T − 1) = 1}

(4.26) (4.27)

Mathematical Modeling of Effectiveness Index

101

The mission completion analysis in each stage include completion degree and probability. For a specific mission in SoS, the completion degree is used to describe the mission completion during a single simulation or exercise, and completion probability is used for during multiple simulations or exercises respectively. The completion degree analysis is mainly divided into the following steps: (1) Define the success standard of mission, according to the operational scenario. The mission is refined into a series of operational goals and then define the success standard of each goal. (2) Quantify the success standard. (3) Collect relevant data. The relevant data or evidence are collected according to the defined success standard for each operational goal. (4) Evaluation conclusion. The completion situation of operational goal is analyzed based on the collected data. If there are goals that cannot or difficult to be quantify, the evaluators can make qualitative judgments according to the situation and process. The calculation of completion degree is shown as follows: n

Tsucess = ∑ wi Oi

(4.28)

i=1

where, Tsucess represents the mission completion degree. wi is the weight of the mission i, Oi is the completion situation of the goal i, and n is the number of mission. The mission completion probability is based on a large number of simulations and can be calculated by the frequency estimation according to the classical model. PT j =

NT j N

(4.29)

where, PT j and NT j represent the mission completion probability and simulation times with mission completion degree of T j respectively. And N is the total simulation time.

4.3. Mathematical Modeling of SoS Effectiveness Index Networked sub-systems is the foundation of the SoS. Under the support of the information system,the SoS integrates various information networks (including physical and logical networks) and has significant networked characteristics, which is called the networked SoS. It can be regarded as the integration of various information network [118], [119]. The complex interaction relationships among sub-systems can be embodied by these networks, which are the source of SoS capabilities. The SoS effectiveness evaluation should focus on the network and pay special attention to the network-based coupling interaction. And then the SoS overall effectiveness will be generated. The effectiveness evaluation should be measured from two aspects, which are the MOTE and the MOEE. The MOTE aims to measure the extent that the system achieves the ultimate goal. The key points are evaluating the overall situation how the mission be completed. It is the most direct and fundamental manifestation of the overall emergence

102

Deping Zhang and Xuefeng Yan

capabilities. It is the fundamental criterion for optimization and also is the most concerned content for the decision maker. It describes the overall situation of mission completion under specific conditions. Some typical indexes used in traditional effectiveness evaluation are important for studying mission effectiveness evaluation. The indexes of this level mainly include operational results, operational damage, mission completion, operational damage ratio, advance speed, operational time, and mission completion rate. The MOEE measures the overall emergence. It pays special attentions on the whole characteristics that come from the structures, functions, and behaviors, such as the robustness and fragility of the architecture, new capabilities that are generated by functional coupling, adaptation and synchronization behavior, etc., [120]-[123].

4.3.1.

Modeling of Networked Evaluation Index

The relationship among various indexes is essentially interconnected network structure rather than tree-like. There is complex correlation between the indexes rather than interindependence. The improvement of some indexes may cause the decline of other related indexes, which reflects the ”non-reductive”. It is necessary to envisage the complex correlation between the system effectiveness indexes. The networked index system should be established and the evaluation should be studied based on it [124]-[129]. The basic evaluation index describes the overall performance of the SoS. It can be constructed based on the characteristic parameters of the weighted hyper-network model. These indexes can be modeled from the views of individual behavior and overall SoS behavior respectively, which is shown in Figure 4.6.

Figure 4.6. The basic evaluation index. In Figure 4.6, the main evaluation objects of individual behavior indexes are the structure and behavior characteristics, composed by the individuals themselves and their interactive behaviors. Several indexes are needed to be modeled, such as the number of active nodes, degree, level, the number and distribution of functional gravity center, and betweenness. The main evaluation objects of the overall behavior indexes are the overall behavior and

Mathematical Modeling of Effectiveness Index

103

structural characteristics. Several indexes are needed to be determined, including connectivity, link-to-node ratio, neutrality rate, number of clusters, and cluster size. These indexes is introduced respectively as follows: In the SoS network, vi represents the i(i = 1, 2, 3, · · · , N) node and V = {v1 , v2 , · · · , vN } is the node set. N is the number of active nodes. The interconnection relationship el represented by the l(l = 1, 2, 3, · · · , M) edge and E = {e1 , e2 , · · · , eN } is the system network edge set. M is the number of connection edges. Node Degree. Assume vi is a node in the network G, the degree of vi is represented by 0 the number of all edges connected to it, which is recorded as Ki . The node degree includes the in-degree, out-degree, average-degree, maximum-degree and degree distribution index. Among them, degree distribution is the most important structural property of complex network. Generally, the larger the node degree is, the greater the effect is. The node degree in the command and control network can also be represented by the degree distribution function P(k). It is the probability of selecting a node with exactly k edges randomly, it also represents the proportion of the number of nodes with degree k in the total number of nodes. The node degree refers to the number of edges connected to the node. The averagedegree is the mean value of all node degree, as follows: 1 N 0 K¯ = ∑ Ki N i=1

(4.30)

In a directed network, according to the direction of the link passing through the node, each node degree is a out-degree (expressed as Kout ) or in-degree (expressed as Kin ). The out-degree denotes the number of links from the node i to other nodes. The in-degree refers to the number of directed links to the node i from other nodes. Adjacent Degree. Assume vi is a node in the network G, the adjacent degree of vi is 00 defined as the sum of the degrees of its neighbor nodes, which is recorded as Ki . Polymeric Degree. The polymeric degree of vi is represented by the algebraic addition 00 0 of the adjacent degree Ki and the degree Ki of this node, which is recorded as Ki , namely: 0

00

Ki = Ki + Ki

(4.31)

The polymeric degree Ki will take into account the impact of the importance of its neighbor nodes on this node. Therefore, the importance of nodes can be further distinguished. For example, a (neighbor node is c) and b (neighbor node is d) are the same degree and the degree of node c is greater than node d. If the degree is adopted for comparison, a and b have the same importance. However, if the polymeric degree index is adopted for comparison, a is more important than b. Betweenness. The Betweenness represents of how much information and resources pass through a node and it is the “bridge” attribute. σ(vs , vt |vi ) s 1, the survivability is strong. Otherwise, if L < 1, the survivability is poor. The average aggregation coefficient ratio F is an index that measures the reliability of network. It is assumed that the connectivity of node vi is Di . A subnet is composed of node vi and the connected Di nodes. The number of edges is Ei in the subnet, then the aggregation coefficient of vi is shown as follows: fi = 2Ei /Di(Di − 1)

(4.51)

The average aggregation coefficient f is the average value of f i of all nodes in the network. f and f 0 represent the average aggregation coefficient before and after the SoS is attacked respectively. Then F can be defined as: F=

f0 f

(4.52)

If F > 1, the network structure has high reliability. Otherwise, the network structure has poor reliability. Network Structure Stability. The natural connectivity describes the connection relationship among the internal structures. It can reflect the stability of network structure to some extent. The natural connectivity λ is defined as follows: ! 1 N λi λ = ln (4.53) ∑e N i=1 where, λi represents the eigenvalue of network G corresponding to adjacency matrix A(G). Therefore, the stability of the network structure Hi is represented by the relative value between the natural connectivity after the network is attacked and the initial natural connectivity. H1 =

λ λ0

(4.54)

where λ represents the natural connectivity after the network is attacked. λ0 represents initial natural connectivity and H1 ∈ [0, 1]. Obviously, the larger H1 is, the stronger the structural stability is. The function usability of the SoS network can be measured by the number of integrated operational chains. That is, after the SoS network is attacked, the more the integrated operational chains reserved, the stronger functional usability is. Therefore, the functional usability H2 can be evaluated by the following formula: H2 = k1

B G + k2 B0 G0

(4.55)

where B0 , B represent the number of integrated operational chains before and after the SoS network is attacked respectively. G0 , G represent the number of the generalized integrated operational chains before and after the attack respectively. k1 , k2 represent the proportion of the standard and generalized operational chains respectively.

Mathematical Modeling of Effectiveness Index

4.3.2.

111

Modeling of Emergence Evaluation Index

Emergence is an important concept in and complex science and its quantitative evaluation is particularly difficult. The SoS is a complex and large system composed of multiple sub-systems. Each sub-system that makes up the system operates independently and is interrelated. Therefore, it is inevitable that emergence is important for the development and evolution of the system. If these characteristics are reasonably analyzed and quantitatively evaluated, several benefits can be obtained. Firstly, the relationship between the emergence and the operational capability can be described. Secondly, it can provide useful guiding principle and decision basis for optimizing various operational resources and effectively improving the operational capability. The emergence is mainly reflected in the behavior of operational capability. Several factors and dynamic processes are taken into account for the operational capability and emergence evaluation. The factors include equipment, organizational establishment and battlefield information, etc. The dynamic processes include early warning and tracking, command decision and fire strike, etc. Therefore, three kinds of indexes are included in the operational capability, such as the capabilities of early-warning detection, command and control, and fire striking. The operational capability is decomposed into sub-systems which can be further described by the operational capability indexes. The operational capability is the result of the interaction and mutual influence among the components. It mainly includes two kinds of indexes. The first is the inherited emergence indexes that include the cooperative capabilities of early warning and fire, which are inherited from the operational capability of system. Although it is similar to the operational capability of system on the function, it is not simple linear superposition of system-level capability. The other is the non-inherited emergence indexes that include the capabilities of battlefield situation reasoning, survivability and adaptability, etc. They are the integrated effect among the constituent units. These capabilities cannot be possessed by a single system independently. Therefore, the new indexes of operational capability are emerged at the SoS-level. The inherited and non-inherited indexes are the basis for evaluating the system emergence. The better the emerging capability indexes are, the better the system emergence evaluation will be. The system emergence should be judged according to the operational capability and whether the capability is inherited from each component system [130]-[144]. The details are shown as follows: (1) Inherited Emergence The operational capability of the SoS is mainly realized by each subsystem, such as early warning and detection, command and control, and fire attack, etc. Therefore, the operational capability must include some indexes that are inherited from the subsystems. However, there is great difference between the capability index and single system due to the information exchange and sharing among subsystems, and also due to the networking optimization of their internal subsystems. In general, the following principles are used to judge the emergence that is caused by the inheritance. If an operational capability is derived from a certain subsystem and

112

Deping Zhang and Xuefeng Yan

the index of the operational capability is not the sum of the relevant capabilities. The operational capability composed of system inheritance presents nonlinear change in the overall capability, that is inherited emergence. (2) None-Inherited Emergence In the process of self-organization evolution, some operational capabilities that are not available in some sub-systems will emerge at the overall SoS. The following principles are used to judge the system emergence that is caused by the none-inheritance. If a certain operational capability is not specific to a single system, that is non-inherited emergence. The emergent index is comes from the depth analyze of the capability from the mechanism aspect. It refers to the confrontation mechanism and capability generation. The relationship between “micro-macro” of effectiveness can be explored. The SoS has new capability that is not provided by sub-systems and composition system. Although the capability cannot be obtained by accumulation of discrete subsystems, the “micro-macro” relationship between subsystems is an internal mechanism that implies the capabilities of the emergence. The correlation between the interaction of subsystems and the overall effect of the SoS need to be explored by the effectiveness evaluation. Then the mechanism of emergence can be revealed. From the perspective of indexes, it is necessary to explore the effectiveness index of subsystems and the relationship between the sub-system and the SoS. The comprehensive evaluation of the overall capability can be obtained by measuring the indexes at the two layers. However, according to the evaluation purpose, it is possible to focus on the index of one layer during the experiment. For example, the robustness of the SoS and the gravity center can be evaluated by the index from the SoS emergence effectiveness index layer. Several indexes are included in this layer, such as network connectivity, invulnerability, operational elasticity, adaptability, battlefield situation reasoning capability, early warning cooperative capability, attack cooperative capability and survivability, etc. (3) Network Connectivity of SoS The connectivity focus on the probability that the network is still connected after being attacked. For the communication network G(N, b), consists by of N nodes, b communication links. The nodes are reliable and links fail independently with probability p. The network connectivity P1 (G) can be expressed as follows: b

P1 (G) =

∑

i=n−1

Ai (1 − p)i pb−i

(4.56)

where Ai is the number of the connected network (subnet of G) with i edges and N nodes. Similarly, for an aircraft carrier complex network G(N, b), consists of N nodes and b communication links. The node reliability is 1. The nodes are reliable or links fail independently with probability q. Then, the network connectivity can be expressed as follows: b

P1 (G) = 1 −

∑

i=n−1

Ni qi (1 − q)n−i

where Ni is the number of unconnected graphs after the node i is removed.

(4.57)

Mathematical Modeling of Effectiveness Index

113

(4) Invulnerability SoS Invulnerability is an important basis for SoS optimization. The invulnerability is a comprehensive indexes of system structure heterogeneity, network correlation, dynamic self-adaptability and inherent protection. It is also the overall emergence and new feature in evolution. The evaluation is transformed from the separable independent index based on traditional reductionist assumption to the comprehensive correlative network index based on complex network theory. To evaluate the invulnerability from the aspects of spatial scale, time scale, overall protection and command effectiveness, four evaluation indexes of invulnerability are desrcibed, such as connectivity, timeliness, protection and the integrity of command chain. In graph G, in all the paths with length less than k(k > 1), the correlation coefficient of local degree is defined as the sum of the ratio of the degree of node v in each path, as Tk (v). Tk (v) =

dv d s,t∈V,d(s,t)≤k l∈L(s,t|v) j

∑

∑

(4.58)

Let p(v) represents the number of paths through the node v with length less than k, σ(s,t) indicates the number of the shortest paths from node s to node t, σ(s,t|v) represents the number of the shortest paths from node s through node v to node t. Then the local betweenness-degree centrality Ck (v) of the node v can be determined by the following formula. Ck (v) =

Tk (v) σ(s,t|v) ∑ p(v) s,t∈V,d(s,t)≤k σ(s,t)

(4.59)

The node invulnerability is defined as the correlation degree of the destruction between node v and the entire network, S(v). The greater the value is, the easier the node is to be destroyed, which leads to the destruction of the whole network. The calculation is as follows: S(v) = CLv

Ck (v) ∑ Ck (i)

(4.60)

i∈V

where CLv is the accumulation coefficient of node v,Ck(v) is the index of the local betweenness-degree centrality of the node v. (5) Elasticity of the Operational SoS The invulnerability is represented by Rb , which indicates the degree of the SoS’ capability to be maintained after being attacked. It can also be represented by the exponential function of the decreased degree b = [P(ta) − P(td )]/P(t0) and the degraded speed vb = (ts − ta )/(td − ta ) of the SoS capability, as the following. Rb = e−bvb

(4.61)

The restoring ability is represented by Rr , which indicates the degree of the capability recover by the redundancy backup and reorganization. It also be represented by the

114

Deping Zhang and Xuefeng Yan

exponential function of the recovery degree r = [P(ts) − P(tr )]/P(t0) and recovery speed vr = (ts − ta )/(t)s − tr), as is shown in the following. Rr = er·vr

(4.62)

The operational capability can be measured by various indexes. The elasticity of certain capability is defined as Ri , as is shown in the following. Ri = Ra · Rr

(4.63)

The elasticity R can be represented by the set of Ri . The changes of the SoS capability can be obtained through dynamic monitoring. Then, the comprehensive evaluation of elasticity can be obtained, as is shown in the following: R = f (R1 , R2 , · · · , RN )

(4.64)

(6) Adaptability Under the complex conditions, the adaptability can reflect the probability of the ability to work normally. Therefore, the adaptability index Ea can be described by the non-failure work probability under certain conditions. Assuming that for the N incoming target in the M anti-missile operation environment, SoS has L anti-missile operation schemes; For the j incoming target under the i operational environment, if the k anti-missile operation scheme is adopted, the non-failure work probability of the SoS is Pi jk . Then, the adaptability can be represented as following. M

N

Ea = ∑ φ i ∑ φ i j i=1

j=1

L

∑ (ηi jk Pi jk )

(4.65)

k=1

where φi is the probability of the i anti-missile operational environment. φi j is the probability of the j target in the i anti-missile operation environment. If the j target appears in the i anti-missile operation environment, ηi jk is the probability that the k anti-missile operation scheme is adopted. The greater Ea is, the stronger the adaptability is. The battlefield adaptability will be more obvious, which is reflected in the system emergence. (7) Reasoning Capability of Battlefield Situation The reasoning of battlefield situation must meet the requirements of accuracy, completeness and timeliness. Therefore, the reasoning capability Er can be described by the accuracy Pa , completeness Pc and information processing efficiency Pt , as is shown in the following. Er = ωa Pa + ωc Pc + ωt Pt

(4.66)

where ωa , ωb , and ωt are the weights of Pa , Pc , Pt respectively. The greater Er is, the stronger the reasoning capability is, then the characteristics of the battlefield situation awareness and reasoning will be more obvious, which are reflected in the system emergence. In the specified anti-missile mission, the accuracy Pa denotes the degree that the characteristics of obtained blue missile target consistent with the characteristics of real targets.

Mathematical Modeling of Effectiveness Index

115

It is assumed that u j (−1 ≤ u j ≤ 0) is the weight coefficient that is jammed by false target information. s j (0 < s j < 1) is the weight coefficient of the influence for the target characteristics on the performance index; oi j is the objective eigenvalue j of the target i, ri j is the awareness and reasoning eigenvalue j of the target i. After the reasoning, for the target i, the average deviation degree between the awareness reasoning eigenvalue and the objective eigenvalue is as follows. n

Vi =

|ri j − oi j |(u j + s j ) oi j j=1

∑

(4.67)

Therefore, for N attacking targets that have been discovered, ω1 , ω2 , · · · , ωN are used to N

represent the targets weight coefficients and ∑ ωi = 1. The accuracy of the battlefield i=1

situation reasoning Pa can be represented as follows: N

Pa = ∑ ωi (1 −V j )

(4.68)

i=1

In the specified anti-missile mission area, the situational awareness and reasoning completeness Pc denotes the degree to which the type and number of detected targets consistent with the real battlefield. It is assumed that Pc1 , Pc2 and Pc3 are the completeness of the target type, target quantity and detection range respectively. Then the following formula can be obtained. Pc = Pc1 Pc2 (1 − exp(−Pc3 ))

(4.69)

where Pc1 refers to the ratio of the number of discovered targets types to the actual number of target types. Pc2 refers to the ratio of the discovered targets number to the actual number of targets. Pc3 refers to the ratio of the detection range of sensor to the actual range. The battlefield situation processing efficiency Pt refers to the ratio of the targets number that can be obtained correctly within unit time interval to the total number of targets, which is shown as follows: Pt =

N0 (T ) T N(T )

(4.70)

where T is the specified time interval, N0 (T ) is the number of targets and N(T ) is the total number of incoming targets. (8) Collaborative Early Warning Capability Collaborative early warning capability can be reflected by the probability how the SoS can acquire, process and transmit the target information and effectively. Therefore, the collaborative early warning capability Ew can be described by the probability Pf of target discovery, the probability Pr of target information fusion and processing, and the probability Pl (i, j) of target information transmission, as is shown in the following: Ew = Pf Pr Pl (i, j)

(4.71)

116

Deping Zhang and Xuefeng Yan

The greater Ew is, the stronger the collaborative early warning capability is, and then the collaborative early warning capability of the system emergence will be more obvious. Pf is the synthetic probability that the targets are found by space-based satellites, space-based early warning platforms and ground-based radar. Pr can be represented as follows: n

Pr = 1 − ∏(1 − Pri (t))

(4.72)

i=1

where, in the n early warning information sources, Pri (t) represents the probability that information source i is fused and processed: Pri = αi + βi (1 − exp(−ωi t))

(4.73)

where, when the information processing and fusion system deals with data from earlywarning information source i, αi, ωi and 0 < αi + βi ≤ 1 represent the processing capability, processing accuracy and maximum processing capability respectively. Pl (i, j) can be represented by the connectivity of terminal, that is, the probability that there are at least 1 routes between two specified nodes after the network is attacked. In the network consisting of N nodes and b links, it is assumed that each link and each node are destroyed independently with probability P and Q respectively. Pl (i, j) is the connectivity probability between any two nodes i, j in the network. When the P  Q, there is b

Pl (i, j) =

∑ Aei, j (k)(1 − P)kPb−k

(4.74)

k=1

If A  P, then n−2

Pl (i, j) =

∑ Ani, j (k)(1 − Q)k Qn−2−k

(4.75)

k=0

where Aei, j (k) represents the number of sets that consist of k links, and assuming in each set, when the k links are normal and the b − k links are destroyed, there is still at least 1 paths between i, j. Similarity, Ani, j (k) represents the number of sets that consist of k nodes meeting and assuming on each set, when the k nodes work and the remaining n − 2 − k nodes do not work, there is still at least 1 paths between i, j. (9) Collaborative Attack Capability The collaborative attack capability can be reflected by the probability that the attack instruction can be transmitted smoothly, and kill the target effectively when the attack condition meets the attacking requirement. Therefore, the collaborative attack capability Ec can 0 be described by the transmission probability Pl (i, j) of attack instruction, target attackable probability Ps and target damaged probability Pk , as follows: 0

Ec = Pl (i, j)PsPk

(4.76)

The greater Ec is, the stronger the attack cooperative capability is. Then the attack cooperative capability will be more obvious, which are one of concrete aspects of the system

Mathematical Modeling of Effectiveness Index

117

0

emergence. Pl (i, j) represents the probability that there are at least one route surviving between two specified nodes after the network is attacked. Therefore, the connectivity can be represented by the terminal. 0

n

Pl (i, j) = 1 − ∏(1 − Pki )

(4.77)

i=1

where Pki is the probability that the interception missile i will destroy the incoming targets. (10) Survivability The survivability can reflects by the survival probability in the battlefield. Therefore, the survivability Es can be described by the camouflage success probability Ph , the network anti-damage probability Prd and the network connectivity P1 (G) , which is shown as follows: Es = Ph + (1 − Ph )Prd + (1 − Ph )(1 − Prd )P1 (G)

(4.78)

where the greater Es is, the stronger the attack survivability is. Then the battlefield survivability will be more obvious, which are an important factor of the system emergence. Ph is mainly measured by the success probabilities of position camouflage, electromagnetic deception and so on. Prd can be measured by the probabilities of anti-hard damage and anti-soft damage.

Chapter 5

Classical Method of Effectiveness Evaluation 5.1. Effectiveness Evaluation Based on AHP Analytic Hierarchy Process (AHP), proposed by the American operations researcher T. L. Saaty in the 1970s, is a multi-objective decision-making method that combines qualitative and quantitative analysis. It quantifies the experienced judgment of decision-makers based on characteristics of behavioral science. In the case of complex structure of target (factor) and lack of necessary data, AHP is a more practical than others [145]-[147].

5.1.1.

Basic Principles of AHP

The basic idea of AHP is to divide the problem to be analyzed into several layers. It divides the problem into different components (factors) according to the characteristics of the problem and the overall goal to be achieved. The factors are aggregated at different layers to form a multilayer analysis structure model according to the related influence of these factors and their affiliation. (1) Establish a Hierarchical Structure Model AHP divides the goals, considered factors (decision criteria) and objects of decisionmaking into the top, middle and bottom layers according to their relations, and forms a hierarchical structure diagram, as is shown in Figure 5.1. The decision scheme is decomposed into three layers. The top layer is the target layer, which is the purpose of the decision and problems to be solved. For example, the goal of the decision is to get high comprehensive effectiveness. The middle layer is the criterion layer, which is the criterion for factors and decisions, such as abilities of concealment, perception, accusation, attraction, and adaption. The bottom is the alternative scheme layer, such as the alternative P1 , P2 , and P3 . The problem solved by AHP is concerned with the relative weight of the bottom layer to the top. Various schemes and measurements in the bottom layer can be sorted according to the weights. Therefore, different schemes can be selected based on specific principles.

120

Deping Zhang and Xuefeng Yan

Figure 5.1. Hierarchical structure model of AHP. (2) Construct the Judgment Matrix When the weights of different factors at each layers are determined, if it is only a qualitative result, it is often difficult to be accepted by others. Therefore, a consistency matrix is proposed by T. L. Saaty et al. This method has two major rules. 1) It does not compare all factors together but compares them with each other in pairs. 2) Relative scale is used in comparison factors with different properties to minimize the difficulty and improve the accuracy. The matrix of pairwise comparison is generally used to construct the judgment matrix. The purpose is to compare the influence of n factors C1 ,C2 , · · · ,Cn in a certain layer on the factor O of the upper layer. For example, in decision-making of the comprehensive effectiveness evaluation, the importance of the criteria is compared. Two factors Ci and C j are used at a time to compare their influence on the target factor O, and the influence is represented by ai j . Results of all comparisons are represented by a pairwise comparison matrix, that is, A = (ai j )n×n , ai j > 0, a ji = 1/ai j

(5.1)

The scale method of the element ai j in the judgment matrix is shown in Table 5.1. For example, for the target O each element of the judgment matrix can be calculated after determining the importance of each criterion according to experts experience, as show following: a12 =

C1 1 = C2 2

(5.2)

It means that the importance of C1 to the target O is 1, and the importance of C2 to the target O is 2. Similarly, the judgment matrix A can be obtained by using pairwise comparison matrix.   1 1/2 4 3 3  2 1 7 5 5     A = 1/4 1/7 1 1/2 1/3 (5.3)  1/3 1/5 2 1 1  1/3 1/5 3 1 1

Classical Method of Effectiveness Evaluation

121

Table 5.1. The scale method of the element in the judgment matrix Intensity of Definition importance 1 Equal importance 3 Somewhat more important 5 Much more important 7 Very much more important 9 2, 4, 6, 8

Explanation

Two factors contribute equally to the objective Experience and judgement slightly favour one over the other Experience and judgement strongly favour one over the other Experience and judgment very strongly favour one over the other. Its importance is demonstrated in practice Absolutely more The evidence favouring one over the other is of the important highest possible validity Intermediate values When compromise is needed

Problems in the pairwise comparison matrix are as follows: 1) There are inconsistencies in various elements, for example, C1 1 = 1/2, a21 = =2 C2 a12 C1 1 = = 4, a31 = = 1/4 C3 a13

a12 =

(5.4)

a13

(5.5)

Then it can be deduced that, a23 =

C2 a21 C2 /C1 = = =8 C3 a31 C3 /C1

(5.6)

It is inconsistent with a23 = 7 in the matrix A. 2) The times of pairwise comparison matrix is too much. Comparison times of n elements are Cn2 = n(n − 1)/2!. To address this challenge of inconsistencies, the weights of each factor to the factor (upper factor) O is proposed by Saaty, and the allowed error range for this inconsistency is determined. (3) Single Layer Sorting and Consistency Checking If the pairwise comparison matrix A described by Equation (5.1) satisfies the following rules: 1) Positive reciprocity, that is ai j > 0, a ji = 1/ai j . 2) Consistency, that is ai j = CCij = aaihjh , i, j, h = 1, 2, · · · , n. If the symmetric matrix of positive reciprocity A meets the above conditions, it can be called the consistency matrix, or the uniform matrix for short. In general, the consistency matrix can be looked as given n objects M1 , M2 , · · · , Mn , and their weights are W1 ,W2 , · · · ,Wn . Their weights are compared in pairs, their ratio forms a

122

Deping Zhang and Xuefeng Yan

consistency matrix A. Matrix A is the comparison matrix and reciprocals are automatically assigned to each pair wise comparison. Here, ai j is the comparison values. Next is calculating the vectors of priorities by dividing the elements of each column by the sum of the column and then add the element in each resulting row. Next is to divide this sum by the number of the elements in the row.     a11 a12 · · · a1n a11 /W1 a12 /W2 · · · a1n /Wn a21 a22 · · · a2n  a21 /W1 a22 /W2 · · · a2n /Wn      A= . (5.7) .. ..  →  .. .. ..  .. ..  ..   . . . . . . .  an1 an2 · · · ann

where Wi =

ai j n

∑ ai j

an1 /W1 an2 /W2 · · · ann /Wn

n

, i, j = 1, 2, · · · , n, ∑ ai j is the summation of the column matrix A, and Wi i=1

i=1

n

is the priority values for the criteria i and satisfies ∑ Wi = 1. i=1

The vector W = (W1 ,W2, · · · ,Wn)T represents the weights of the factors C1 ,C2 , · · · ,Cn to the upper target O, which is called the weight vector. This process is called the single layer sorting. This method of determining the weight vector with eigenvectors is called the eigenvalue method. The basic idea of AHP is to calculate the weight vector of each element in the upper layer to each element in the next layer (That is the eigenvector W = (W1 ,W2, · · · ,Wn )T corresponding to the maximum eigenvalue λmax ), as well as combined weight vectors and consistency checking problems. It is not necessary to pursue higher accuracy when calculating the maximum eigenvalue and corresponding eigenvector. Because the judgment matrix has a certain range of error. Moreover, the numerical value of the priority is also an expression of the qualitative concept. Therefore, a simpler approximation algorithm is expected to be used from the perspective of applicability. The common approximate methods for finding eigenvalue are “sum-based method”, “root-based method” and “power-based method”. This chapter mainly introduces the power-based method to solve the judgment matrix. It is easy to implement on the computer since iteration is used in the power-based method. Suppose A = (ai j )m×m, A > 0, then Ak · E = CW k→∞ E T · Ak · E

(5.8)

lim

where C is a constant. (0) (0) (0) Step 1. Initialize positive vectors X(0) = (x1 , x2 , · · · , xm )T , and m0 = (0) (0) max{xi },Y (0) = xm0 is selected when k = 0. i

(1)

x(1) m1 i (k+1) (k) A ·Y , Mk+1 = max{xi },Y (k+1) i

Step 2. Iterative calculation: X (1) = A ·Y (0), M1 = max{xi },Y (1) = Step 3. Step 2 is being repeated, until: X (k+1) = x(k+1) mk+1 , k ← k + 1.

=

Accuracy checks are performed after each iteration. When Mk+1 − Mk < ε, the next step can be taken to get the eigenvalue. Or, let k = k + 1, and the iterative continues.

Classical Method of Effectiveness Evaluation The formula for calculating eigenvalues is as follows:   λmax = mk+1 Y (k+1)  W= m

123

(5.9)

∑ Y (k+1)

i=1

The corresponding eigenvalues and eigenvectors can be got. The eigenvectors are used as the weights of the parameters in the effectiveness evaluation. The larger the comprehensive value is, the better the solution is under the given weights. The decision-maker can subjectively judge the ratio by comparing the relations of objects M1 , M2 , · · · , Mn in pairs when the weight vector W = (W1 ,W2, · · · ,Wn )T is unknown, or Delphi (survey method) can be used to determine these ratios, so that matrix A (with consistency or not) is known. The matrix made by the subjective judgment is taken as the ¯ and A¯ (inconsistent) is within the tolerance of inconsis(subjective) judgment matrix A, tency. The eigenvalue λ and the eigenvector W of A depends on the elements ai j of the matrix continuously. It means that the eigenvalue and the eigenvector of A¯ are not much different from the matrix A when ai j is not too far from the requirement of consistency. The method of obtaining the weight vector W from the eigenvector W = (W1 ,W2, · · · ,Wn ) called the eigenvector method, which used in consistency checking. (4) Random Consistency Checking Index When adopting pairwise comparisons for each factor, no one can guarantee, the obtained subjective judgment matrix A¯ is exactly the consistency positive reciprocal matrix A, so there are errors (and error estimation problems). This error will inevitably lead to the ¯ − λ)|) and the difference between the eigenvectors difference between the eigenvalues (|(λ ¯ −W )|), which results in a difference between the problem A¯W ¯ = λmaxW ¯ and the prob(|(W ¯ lem AW = nW . Where λmax is the maximum eigenvalue and W is the relative weight vector ¯ This is caused by inconsistencies in the with the bias of the subjective judgment matrix A. judgment matrix. Therefore, it is necessary to measure the consistency of the subjective judgment matrix A¯ to avoid too many errors. 1) When the subjective judgment matrix A¯ is the consistency matrix A, then,

∑ λ¯ k =

k=1

n

n

∑ λk =

k=1

n

∑ akk =

k=1

∑1=n

(5.10)

k=1

When A is the consistency matrix, aii = 1. There is only one λ = λmax = n in this time. 2) When the subjective judgment matrix A¯ is not the consistency matrix, there will be λmax ≥ n, and then, n

λmax +

∑

k6=max

λk = ∑ aii = n

(5.11)

i=1

That is, λmax − n = −

∑

k6=max

λk

(5.12)

124

Deping Zhang and Xuefeng Yan

Therefore, the mean value can be taken as the criterion for checking the subjective judgment matrix. The consistency index CI is, λmax − n = CI = n−1

− ∑ λk k6=max

(5.13)

n−1

There is complete consistency when CI = 0. There is satisfied consistency when CI is close to 0. The inconsistency is not good when CI is large importance, it was found that the larger dimension of the subjective judgment matrix A¯ is, the worse consistency is. Therefore, the consistency requirements of the high-dimensional matrix should be loosened. The correction value RI is introduced to correct the consistency checking index. The value is defined as shown in Table 5.2. Table 5.2. Stochastic consistency index RI n RI

1 0

2 0

3 0.58

4 0.90

5 1.12

6 1.24

7 1.32

8 1.41

9 1.45

10 1.49

11 1.51

The new consistency checking index CR is, CI (5.14) RI It is generally believed that when the consistency ratio CR < 0.1, the inconsistency of A is within the allowable range. There will be satisfied consistency, and consistency checking is passed. The normalized eigenvector can be used as the weight vector, otherwise, the pairwise comparison matrix A must be reconstructed, and ai j needs to be adjusted. CR =

(5) Total Layer Sorting and Consistency Checking The total layer sorting is to calculate the weight of the relative importance of all factors at a certain layer to the highest layer (overall goal). This process is performed from the highest layer to the lowest layer sequentially. For example, the hierarchical structure model is shown in Figure 5.2.

Figure 5.2. Hierarchical structure model of AHP. If the sorting of m factors A1 , A2 , · · · , Am in the A layer to the overall goal Z is a1 , a2 , · · · , am. To the factor A j in the upper layer A. The single-level sequencing of n factors in B layer is b1 j , b2 j , · · · , bn j ( j = 1, 2, 3, · · · , m).

Classical Method of Effectiveness Evaluation

125

The total layer sorting of the B layer, that is, the weight of the i factor of the B layer to m

the overall goal is Bi = ∑ a j bi j , i = 1, 2, · · · , n, is j=1

 B1 : a1 b11 + a2 b12 + · · · + am b1m    B2 : a1 b21 + a2 b22 + · · · + am b2m ··· ···    Bn : a1 bn1 + a2 bn2 + · · · + am bnm

(5.15)

The consistency ratio of the total layer sorting is as follows. CR =

a1CI1 + a2CI2 + · · · + amCIm a1 RI1 + a2 RI2 + · · · + am RIm

(5.16)

The total layer sorting is considered to pass the consistency checking when CR < 0.1.

5.1.2.

Effectiveness Evaluation Based on AHP

AHP is a basic method of effectiveness evaluation. A clear hierarchy structure is established to decompose complex problems. Then, the measurement theory is introduced. Through pairwise comparison, the relative scale is used to quantify the judgment of humans, and a judgment matrix layer by layer is established. Then, the weights of the judgment matrix are solved, as is shown in Figure 5.3. Specific steps are as follows: (1) Objects confirmation evaluation. It is necessary to mak the evaluation objects clear and study the mission. The operating environment and evaluation purpose of the object need to be confirmed, which is the foundation for further work. (2) Initial the index set of the system effectiveness evaluation. It is important to find out the indexes that affect the system effectiveness through the analysis of the evaluation objects and the operating environment. The index set of effectiveness evaluation is developed through the quantitative process and the correlation analysis of indexes [148]. Comprehensive considerations must be taken into account when initializing the index set because a large system has many influencing factors, especially the operating environment. (3) Experts invitation evaluation. Experts in related fields are invited according to the professional knowledge according to the evaluation objects. The invited experts must have a solid theoretical foundation, rich practice, and evaluation experience, etc. This is a key step to establish an rational index system. The scientific rationality of the index system is ensured by the correct selection of relevant experts. Each invited expert in this field will receive a consultation form which contains the feasibility and weights of indexes. Experts fill in the consulting form based on their professional knowledge and previous evaluation experience. Experts can add an index and give the weight when they believe the index set is not complete.

126

Deping Zhang and Xuefeng Yan

Figure 5.3. Process of comprehensive effectiveness evaluation based on AHP. According to statistics and summary of the feedback information, indexes with low feasibility considered by most experts are deleted, and reasonable indexes suggested by experts are added. The collated set of indexes is sent to the experts for consultation. The final index system is formed when the opinions of the experts are generally consistent after multiple rounds of consultation and screening. (4) Hierarchy structure construction. The key of the AHP is to construct a reasonable and concise hierarchical structure model. When using AHP, the goals should be determined first. Then, the system evaluation index system is established based on expert opinions, and a structural model of the system effectiveness evaluation is also constructed. The construction criterion is that the elements in the structural model only belong to one layer, and it is not necessary for every element to be associated with the lower layer elements. The hierarchies of AHP should not be too much, with no more than 9 elements in each group. The psychology shows that most people have a clear judgment with no more than nine. Too many elements will make subjective judgments difficult. In that case,

Classical Method of Effectiveness Evaluation

127

logical errors will occur, and the model will be inaccurate. If multiple elements are needed, the number of elements in the layer can be reduced by increasing the number of layers. (5) Judgement matrix construction. The pairwise comparison matrix is used to compare the elements in each layer by pair, so as to construct a judgment matrix. It is a key step of the AHP. After the hierarchical model is established, the affiliation of every element is determined. The judgment matrix starts at the target layer. Then, pairwise comparison on its associated lower layer elements is performed. The importance of the layers is compared from the top to the bottom to form the judgment matrix. The importance judgment of elements adopts the pairwise comparison matrix that uses the natural number between 1 ∼ 9 and their reciprocal to assign the importance. The measurement standard is shown as in Table 5.1. The judgment matrix A can be formed by the pairwise comparison for n elements. Because of the pairwise comparison, the judgment matrix is a reciprocal matrix. (6) Consistency checking. After the judgment matrix is constructed, it cannot be directly applied. The consistency checking needs to be performed for the pairwise comparison matrices. It is reasonable to calculate the single-level sequencing results after the consistency checking. λmax = m is the sufficient and necessary conditions for the consistency checking of the matrix. The maximum eigenvalue λmax is slightly larger than the cardinality m, and the remaining eigenvalues are close to 0 to avoid inconsistencies. When the situation is satisfied, it is called satisfactory consistency. If the consistency checking is passed, step (7) is executed. Or, step (5) is executed to modify the judgment matrix until the consistency checking is passed. (7) Relative importance calculation. The weight of each element target is calculated in this step. The corresponding eigenvalues and eigenvectors can be obtained, and the eigenvectors are used as the evaluation weights of the effectiveness evaluation parameters. (8) The importance of each scheme. The evaluation results under the index system are obtained by calculating the importance of the scheme. The larger the comprehensive value is, the better the scheme is.

5.1.3.

Case Study

The effectiveness evaluation of the submarine torpedo is taken as an example to illustrate the method of the AHP. The submarine torpedo system is composed of several subsystems such as submarine detection, fire control, launch and torpedo, as is shown in Figure 5.4.

Figure 5.4. Composition of the submarine torpedo system.

128

Deping Zhang and Xuefeng Yan

According to the composition and evaluation goal of the submarine torpedo, the index system construction pattern P2 is chosen now. The evaluation index system is determined based on the inherent capability. The effectiveness evaluation index system is shown in Figure 5.5.

Figure 5.5. The evaluation index system of the submarine torpedo system. The importance of each index is judged and evaluated by the expert group to construct the judgment matrix. As is shown in Figure 5.5, the expert sorting method of 1 ∼ 9 scale is adopted to compare each element by pair in the same layer with respect to each upper element, so as to determine the relative importance of the lower layer element to the upper layer element. Based on this, the judgment matrices of pairwise comparison are constructed respectively, such as A, B1 , B2 , B3 , B4 , B5 . The judgment matrix A is shown in Table 5.3, and other judgment matrices can be obtained in the same way. The geometric mean value of all elements in each row in A is calculated, and the approximate value of the eigenvector is obtained after normalization, such as W = (W1 ,W2, · · · ,W5 )T . It is also the relative weight of each element, namely W = (0.2474, 0.4601, 0.0808, 0.1052, 0.1066)T .

129

Classical Method of Effectiveness Evaluation Table 5.3. Judgment matrix A A B1 B2 B3 B4 B5

B1 1 2 1/3 1/2 1/3

B2 1/2 1 1/5 1/4 1/5

B3 3 5 1 2 1

B4 2 4 1/2 1 2

B5 3 5 1 1/2 1

The maximum eigenvalue of the judgment matrix is obtained like:      1 1/2 3 2 3 0.2474 1.2498  2     1 5 4 5    0.4601 2.3123 T      AW = 1/3 1/5 1 1/2 1  0.0808 = 0.4144  1/2 1/4 2 1 1/2 0.1052 0.5587 1/3 1/5 1 2 1 0.1066 0.5722

(5.17)

we have, n

(AW)i = 5.172 i=1 nWi

λmax = ∑

(5.18)

The consistency index CI of judgment matrix is calculated, where n is the matrix order. When n > 1, then, CI =

λmax − n = 0.0430 n−1

(5.19)

Further, the average random consistency index RI can be got, as is shown in Table 5.4. Table 5.4. The average random consistency index RI Matrix order RI

1 0

2 0

3 0.52

4 0.89

5 1.12

6 1.26

The consistency ratio CR is calculated: CR = CI/RI = 0.0430/1.12 = 0.0384 < 0.1

(5.20)

Therefore, the judgment matrix passes the consistency checking. The same method can be used to obtain the weight of each element in the third layer, and the consistency of its judgment matrix is carried out respectively. The results show that the consistency of each judgment matrix is acceptable. Finally, the weights of every evaluation index are obtained, which are shown in Table 5.5.

130

Deping Zhang and Xuefeng Yan Table 5.5. Element weights in the third layer R11 0.2431 R23 0.5396 R42 0.2970

R12 0.0848 R31 0.4747 R43 0.5396

R13 0.3437 R32 0.1630 R51 0.6250

R14 0.3284 R33 0.2551 R52 0.1365

R21 0.2970 R34 0.1072 R53 0.2385

R22 0.1634 R41 0.1634

Furthermore, the composite weight of every element to the target can be calculated as follows: ai j = Ri j ·Wi

(5.21)

Thus, the weights of each index in the evaluation index system based on AHP can be obtained. According to the opinions of the commanders and experts, it is assumed that the fuzzy comments on the solution relative to each index are shown in Table 5.6. The index weight vector Ai = (ai1 , ai2 , · · · , aik j ) and the comment set V = {v1 , v2 , · · · , vm } are obtained. The evaluation object is equivalent to the fuzzy comments of the indexes. It means that the fuzzy matrix on Ui ×V is as follows: i Ri = [rgh ]k j ×m

(5.22)

The fuzzy transformation is performed as follows: Bi = Ai · Ri = (bi1 , bi2 , · · · , bim ), i = 1, 2, · · · , n

(5.23)

Then the evaluation result of each Bi can be achieved. According to the data shown in Table 5.6, the fuzzy comments Bi (i = 1, 2, 3, 4, 5) relative to each index A of the first layer can be calculated.

B1 = A1 · R1



 0.3 0.3 0.2 0.2   0.4 0.3 0.2 0.1  = 0.0601 0.0210 0.0850 0.0812  0.3 0.3 0.2 0.2 0.5 0.5 0 0   = 0.0922 0.0902 0.0330 0.0225

It can be normalized

  B1 = 0.3874 0.3790 0.1389 0.0947

(5.24) (5.25)

(5.26)

131

Classical Method of Effectiveness Evaluation Table 5.6. Fuzzy evaluation table of effectiveness index level Fuzzy evaluation Less Normal Good V2 V3 0.3 0.2 0.3 0.2 0.3 0.2 0.5 0

Poor V4 0.2 0.1 0.2 0

a21 =0.1366 0.3 a22 =0.0752 0.4 a23 =0.2483 0.3

0.3 0.3 0.4

0.3 0.3 0.2

0.1 0 0.1

a31 =0.0384 a32 =0.0132 a33 =0.0206 a34 =0.0087 a41 =0.0172 a42 =0.0312 a43 =0.0568 a51 =0.0666 a52 =0.0146 a53 =0.0254

0.2 0.4 0.4 0.3 0.4 0.4 0.3 0.2 0.3 0.5

0.4 0.1 0.2 0.2 0.1 0.2 0.2 0.3 0.2 0

0.1 0 0.1 0.2 0.1 0.1 0.2 0.1 0.1 0

Lower-level index

Weight Ai

Radar detection capability R11 Sonar detection capability R12 Detection ability of periscope R13 Detection capability of towed antenna R14 information support capability R21 Command device effectiveness R22 Command decision-making capability R23 Noise jammer capability R31 Acoustical decoy capability R32 Air curtain missile capability R33 Torpedo anti-jamming capability R34 Torpedo damage capability R41 The power of torpedo warhead R42 Single torpedo hit rate R43 Submarine maneuverability R51 Submarine stealth capability R52 Submarine survivability capability R53

a11 =0.0601 a12 =0.0210 a13 =0.0850 a14 =0.0812

Good V1 0.3 0.4 0.3 0.5

0.3 0.5 0.3 0.3 0.4 0.3 0.3 0.4 0.4 0.5

Similarly, the values of B2 , B3 , B4 , B5 can be obtained. B2 = [0.3163 0.3540 0.2460 0.0837]

(5.27)

B3 = [0.3326 0.2943 0.2786 0.0944]

(5.28)

B4 = [0.3163 0.3460 0.1837 0.1540]

(5.29)

B5 = [0.4238 0.2852 0.2148 0.0762]

(5.30)

It can be seen that in the detection capability, charge capability, underwater acoustic countermeasure capability, anti-submarine torpedo operational capability and adaptive capability, the membership degrees of good and less good are 0.7664 (0.3874+0.3790), 0.6703, 0.6269, 0.6623 and 0.7090 respectively, which shows that the evaluation results are relatively reasonable. At this time, the index set is U = {U1 ,U2 ,U3 ,U4,U5 }. Bi is the fuzzy comment of the evaluated question for the index Ui , and each Bi is combined into a fuzzy matrix R composed by U ×V , then, R = [BT1 BT2

· · · BTn ]T = (bi j )n×m

(5.31)

132

Deping Zhang and Xuefeng Yan

The total fuzzy comments of the evaluated problem relative to all indexes obtained by the fuzzy transformation are as follows (calculated by the weighted algorithm). B = W × R = (0.3467 0.3472 0.2123 0.0939) 0.3472 0.2123 0.0939 0.3467 + + + = good less good normal bad

(5.32) (5.33)

To illustrace the operational effectiveness more concisely and intuitively, the comprehensive fuzzy comment B must be expressed by a number. The weight W can be determined for each level of comments B according to the industry experience and expert opinions, such as W1 = 100,W2 = 80,W3 = 60,W4 = 40, and then, 4

Sw =

∑ b j ·W j = 78.94

(5.34)

j=1

5.2. The Comprehensive Effectiveness Evaluation Based on ANP The analytic network process (ANP) is a decision-making method suitable for a nonindependent recursive hierarchical structure proposed by Professor T.L.Saaty of Pittsburgh University in 1996. It is a new practical decision-making method developed from the AHP and it presents complex problems in the form of a network [149][150]. Its basic principle is the same as the AHP. The difference between ANP and AHP is the model structure and the supermatrix in the ANP.

5.2.1.

The Basic Principle of ANP

The ANP network is composed of components and their influence between connected components. Components are further composed of elements. There is mutual influence between elements. Elements of one component can interact with element of other components. The various interaction relations are denoted by “→”, while “ A → B ” represents that the component (or element) A is affected by the component (or element) B, or the component (or element) B affects the component (or element) A. The influence relation of the component on itself is called feedback. The ANP generally divides system elements into two parts, the control layer and the network layer. The control layer includes the problem goals and the decision criteria. All decision criteria are considered to be independent of each other and are governed only by the goals. There can be no decision criteria in the control element, but at least one goal. The weight of each criterion in the control layer can be obtained by the traditional AHP. The network layer is composed of all the elements controlled by the control layer. The elements are interdependent on each other and dominate each other. The elements and layers are not internally independent. Each criterion in a hierarchical structure governs an interdependent and feedback network structure not rather than a simple, internally independent element. The control and the network layers are composed of the typical hierarchy structure, as is shown in Figure 5.6.

Classical Method of Effectiveness Evaluation

133

Figure 5.6. Typical structure of ANP. Using ANP to analyze the problem can be roughly divided into four steps, as is shown in Figure 5.7.

Figure 5.7. Typical analysis process of ANP. Step 1. Analyze the structure of the problem. To analyze the interrelation between elements and elements as well as between element groups and elements, so as to describe the relevance of elements. Step 2. Establish the judgment matrix of pairwise elements comparison, and to construct the judgment matrix between element groups and elements. Step 3. Calculate the relative weights of the compared elements based on the judgment matrix and to construct the initial supermatrix. Step 4. Calculate weights among groups, construct the limit supermatrix, and calculate the final ranking results. An important step of the AHP is to perform a pairwise comparison of the dominated elements under a criterion to obtain the judgment matrix. However, the compared elements in the ANP may not be independent and maybe interdependent, so this comparison will be carried out in two ways.

134

Deping Zhang and Xuefeng Yan

(1) Direct degree of dominance. The importance degree of the two elements are compared using a given criterion. (2) Indirect degree of dominance. The two influence of elements are compared by a given criterion on the third element. For example, the degree of dominance of performance indexes of A and B for weapon search capability is to be compared, one method is that it can be indirectly obtained by comparing their ability of find targets. The former comparison applies to the case where the elements are independent with each other, which is also a traditional judgment comparison of the AHP. The latter applies to the case where elements are interdependent, which is the difference between ANP and AHP. (1) Construction of the Supermatrix It is assumed that there are elements P1 , P2 , · · · , Pn in the control layer of ANP, and element groups C1 ,C2 , · · · ,CN in the network layer under the control layer, where there are elements ei1 , ei2 , · · · , ein (i = 1, 2, · · · , N) in Ci . The elements Ps (s = 1, 2, · · · , m) in the control layer are taken as the criterion. The elements e jl (l = 1, 2, · · ·n j ) in the C j are taken as the secondary criterion. The elements in Ci are compared for indirect degree of dominance according to their influences on e jl . The scoring criteria of AHP will be adopted, which is shown in Table 5.1. If the factor i compares ai j with the factor j, then the comparative judgment of the factor i and the factor j is 1/ai j . (a) According to the pairwise comparison of the above scale meanings, the judgment matrix is constructed, as is shown in Table 5.7, using the control layer criterion Ps . The ( jl) ( jl) ( jl) priority vector W = (wil , wi2 , · · · , wini )T can be obtained by the eigenvalue method. Table 5.7. Judgment matrix of pairwise comparison e jl ei1 ei2 .. .

ei1

ei2

···

eini

Normalized eigenvector ( jl) wil ( jl) wi2 .. . ( jl)

eini

wini

(b) The judgment consistency is not required in the construction of the judgment matrix. However, general consistency of judgment is required to prevent the situations where A is extremely important than B, B is extremely important than A, and C is extremely important than A. Therefore, after getting the priority vector, consistency must be checked. The calculation steps are as follows: (1) Calculating the consistency index CI CI =

λmax − n n−1

(5.35)

where n is the order of the judgment matrix, λmax is the eigenvalue of the judgment matrix.

Classical Method of Effectiveness Evaluation

135

(2) Calculating the average random consistency index RI The average random consistency index is obtained by taking the arithmetic mean after repeated calculation of the eigenvalues of the random judgment matrix for multiple times (more than 500 times). Table 5.2 shows the average random consistency index is repeatedly calculated 1000 times for the 1st to 11th order. (3) Calculating consistency ratio CR CR =

CI RI

(5.36)

when CR < 0.1, it is generally considered that the consistency of the judgment matrix is acceptable. (4) Wi j is expressed as:  ( j1)  ( jn ) ( j2) wi1 wi1 · · · wi1 j  ( j1) ( jn )  ( j2) wi2 wi2 · · · wi2 j   Wi j =  .. ..   .. .. .  . . .  ( jn ) ( j1) ( j2) wini wini · · · wini j

(5.37)

The column vector of Wi j is the priority vector of influence of elements ei1 , ei2 , · · ·, eini in Ci on the elements e j1 , e j2 , · · · , e jn j in C j . If elements in C j are not affected by the elements in Ci , Wi j = 0. In this way, the supermatrix W under Ps can be achieved:

1  .. .   n1   1  ..   .  W=  n2  ..   .   1  ..  . nN

1, · · · , n1 1, · · · , n2 · · ·

1, · · · , nN

W11

W12

···

W1N

W21

W22

···

W2N

.. .

.. .

..

.. .

WN1

WN2

···

.

WNN

                

(5.38)

The construction process of the above supermatrix has the following features: (a) Each column in each sub-block of the supermatrix is a priority vector, which consists of the elements in the same layer calculated by the pairwise comparison in judgment matrix; (b) Regardless whether the overall system can be aggregated into a hierarchy or elements groups, the supermatrix W can be directly derived from the pairwise comparison between the elements. At this time, each column in the matrix is a priority weight based on an element. However, the grouping method has the advantages of efficiency, stability, and systematicity, and is more consistent with human thinking. Therefore, the grouping method is often applied.

136

Deping Zhang and Xuefeng Yan

(c) Each sub-block in the supermatrix (except the zero sub-blocks) is composed of a normalized column vector. But the sum of any column may not be normalized (unless there is only one nonzero sub-block in the column). In order to use a supermatrix to sort a complex system, each column of the supermatrix needs to be normalized, which can be implemented with a weighted matrix. (2) Construction of Weighted Supermatrix There are m hypermatrices constructed by adopting the above method. They are all nonnegative matrices. Although the sub-blocks Wi j of the supermatrix is column-normalized, W is not column-normalized. It is necessary to construct a weighted matrix to normalize the columns of the supermatrix. Taking Ps as the criterion, the importance of each group of elements to the criteria C j ( j = 1, 2, · · · , N) is compared using Ps , as is shown in Table 5.8. Table 5.8. Judgment matrix of pairwise comparison Cj C1 .. . CN

···

C1 .. .

Normalized eigenvector (a11 , a12, · · · , a1N )T .. .

CN

j = 1, 2, · · · , N

(aN1 , aN2 , · · · , aNN )T

The Component of the sorting vector corresponding to the groups of elements independent with C j is zero, so the weighted matrix A is obtained:   a11 a12 · · · a1N  a21 a22 · · · a2N    A= . (5.39) .. ..  ..  .. . . .  aN1 aN2 · · ·

aNN

N

where ai j ∈ [0, 1] satisfies ∑ ai j = 1, j = 1, 2, · · · , N. i=1

¯ = (W ¯ i j ), where element is W ¯ i j = ai jWi j (i, j = 1, 2, · · · , N). The constructed matrix is W W = (Wi j )N×N is the supermatrix of the system, and A = (ai j )N×N is the weighted matrix ¯ is called the weighted supermatrix . Any column of the weighted superof the systems. W matrix is normalized and its column sum is 1, which is called a column random matrix. (3) Construction of Limit Supermatrix For most ANP systems, there is no longer a single element or the highest level of overall domination in the system. Therefore, the synthesis order of schemes similar to the recursive hierarchical structure will be meaningless. In this case, the following two types of sorting will be more focused: (i) Relative ranking (or influence ranking) Relative ranking is that the element j is used as the criterion, and other elements are sorted based on its importance. Because the influence between elements in the ANP system

Classical Method of Effectiveness Evaluation

137

is interactive and cyclic. For example, A affects B, B affects C,C influences A. Therefore, it is necessary to find out the limitation of this influence, which is to figure out the limiting relative priority. It can be described as a matrix, which is abbreviated as LRP. (ii) Absolute rank Considering the cumulative influence of elements in the system, after the initial importance sort of all elements is given, all the elements can be sorted. Because this sorting is for the entire system, not for an element, it is called absolute rank. In particular, it is necessary to find the limiting absolute rank, which is abbreviated as LAR. (k) It is assumed that W is the supermatrix of the system. The k power of W is W k = (Wi j ), N

(2)

W (1) = Wi j , Wi j =

(1)

(1)

∑ Wim Wm j

(5.40)

m=1

In general, the cumulative influence in K steps of element i on the element j is considered as follows: (k)

Wi j =

N

(1)

(k−1)

∑ Wim Wm j

(5.41)

m=1

So the matrix W k reflects the cumulative k steps relative rank. When the limit of the W k at k → ∞ exists, there is LRP = W ∞ = lim W k

(5.42)

k→∞

Similarly, W k v(0) represents the influence of the cumulative k steps of the supermatrix on the initial absolute order. When the limit of W k v(0) at k → ∞ exists, there is LAR = v∞ = W ∞ v(0) = lim W k v(0)

(5.43)

k→∞

If W is a nonnegative irreducible column random matrix, the sufficient and necessary condition for the positive existence of W ∞ = lim W k is that W is a prime matrix. Prime k→∞

matrix is nonnegative irreducible matrix,and its largest eigenvalue (primary eigenvalue) is unique, the multiplicity of a weight is equal to 1. At this time, both LAR and LRP exist, and LAR is exactly a limit supermatrix composed of LRP (as columns). (4) Weights of ANP In summary, taking the typical networked structure as an example, the basic steps of determining the weights of ANP model are as follows: (i) The element supermatrix N

According to the definition of the scale in Table 5.1, N × ∑ n j judgment matrices can be j=1

obtained by taking element Ps (s = 1, 2, · · · , m) of the control layer as the criterion, element e jl (l = 1, 2, · · · , n j ) of C j ( j = 1, 2, · · · , N) as the secondary criterion, and comparing the dominance degree of other elements in group Ci (i = 1, 2, · · · , N) according to their influence

138

Deping Zhang and Xuefeng Yan

on C j ( j = 1, 2, · · · , N). The eigenvector corresponding to the maximum eigenvalue of each matrix is calculated, and then the consistency checking is carried out. If the checking is passed, these eigenvectors will be normalized. If the checking fails, the comparison matrix needs to be reconstructed. These normalized eigenvectors are formed into a supermatrix of N

∑ n j order, which is composed of N × N block matrices.

j=1

(ii) The weighted matrix of the element group The element Ps (s = 1, 2, · · · , m) in the control layer is taken as the criterion. An element group of C j ( j = 1, 2, · · · , N) is used as the secondary criterion. The other element groups respect to this criterion are compared with relative influence degree. N comparison matrices are constructed. It is also necessary to perform consistency checking on these judgment matrices and to obtain eigenvectors. These normalized eigenvectors form a weighted matrix with N order. (iii) The weighted supermatrix The weighted supermatrix is constructed by multiplying each element of the weighted matrix in (2) with the blocks of the supermatrix in (1). It reflects the control and feedback of the element group on the element. (iv) Solving the weights of the indexes According to the type of the supermatrix in (3), the corresponding calculation method N

is used to determine the relative order vectors of the elements, namely, the weights of ∑ n j j=1

elements. (v) The total target weight of the elements The weights of the network layer are calculated according to the criteria at different layers, namely, the above steps are repeated with the criteria P1 , P2 , · · · , Pm . Then the element weights calculated using each criterion are synthesized by multiplying the local weights with the corresponding secondary criteria weights. Then the weight vectors of all the basic elements under the overall target can be obtained.

5.2.2.

Effectiveness Evaluation Based on ANP

The effectiveness evaluation analysis of the ANP is the same as the decision-making principle of the AHP. The only difference is that the former takes into account the interaction and dependence between indexes, which can solve the problem of the networked evaluation index system. At the same time, according to the expert experience and the characteristics of simulation data, the multiple evaluation information will be integrated to evaluate the operational effectiveness, so as to make the evaluation results more reasonable and credible. The basic framework of the system comprehensive effectiveness evaluation based on the ANP is shown in Figure 5.8. It can be generally divided into three stages, the construction and optimization of the networked index system, the supermatrix of the multi-information fusion, and the effectiveness evaluation model based on the index weight.

Classical Method of Effectiveness Evaluation

Figure 5.8. Basic framework of the effectiveness evaluation based on ANP.

139

140

Deping Zhang and Xuefeng Yan (1) Construction and optimization of networked index system (i) Preliminary scheme of the evaluation index system

The construction and optimization of the networked index system are mainly based on the required evaluation object. The operational mission and evaluation scenario, specifying the evaluation conditions, and clarifying the evaluation. The construction pattern of the evaluation index system is need to be selected and adopted preliminary scheme is design. (ii) Index screening based on Delphi The screening of evaluation index system is mainly to further improve and optimize the evaluation index system, so the index system construction couldn’t be too complex and redundant. In general, the index system screening can be carried out according to the opinions of experts and based on the importance of each expert. The importance of the indexes can be obtained by the Delphi method. According to the requirement, the indexes with low importance can be eliminated, and the evaluation index system of hierarchical structure can be refined. (iii) Correlation analysis of indexes Correlation analysis of indexes is to determine the mutual influence relationship between indexes and establish a correlation model. The index correlation matrix R = (ri j )n×n is constituted by the index correlation coefficient between every two pairs of indexes in n indexes. ri j ∈ [0, 1] indicates the influence of the index Ii on the index I j . (iv) Design the index system The index system is networked the evaluation index system with a hierarchical structure based on the correlation matrix. The indexes I1 , I2 , · · · , In are the bottom indexes of the index system, without considering its own influence, then rii = 0 in the correlation matrix R. Since it is only necessary to judge whether there is a mutual influence relationship between indexes. When ri j ≤ 0.5, it will be replaced with 0, which means no influence. When ri j > 0.5, it is replaced with 1, which means there is an impact. A 0-1 matrix R∗ = (ri∗j )n×n is obtained. First of all, the graphical represent rules of the indexes and their relationships need to be defined. In the networked index system diagram, the index Ii is called a node and represented by a small circle, as is shown in Figure 5.9.

Figure 5.9. Influence relationship between indexes. The n indexes are divided into m groups according to the attributes of indexes. Each m

group contains ni , n2 , · · · , nm indexes. ∑ ni = n means that the m groups of indexes are m i=1

index clusters, called clusters C1 ,C2 , · · · ,Cm for short. The index clusters are represented by ellipses. The relationship between clusters and nodes is shown in Figure 5.10 (left).

Classical Method of Effectiveness Evaluation

141

Figure 5.10. Relationship between clusters and nodes. When ri∗j = 1, the index Ii has an influence on the index I j , which is represented by a directed connecting line. At this time, the index Ii is called a child node. The index I j is a parent node, and the direction arrow is from the parent node to the child node; If r∗ji = 1 at the same time, it means that I j has a feedback influence on Ii , as is shown in Figure 5.10 (right). When there is a correlation between clusters and internal nodes, it is said that cluster Ci has an internal dependency, and its graphical description is shown in Figure 5.10(right). When there is a correlation between the indexes in cluster Ci and the indexes in cluster C j , it is said that there is an external dependency between the cluster Ci and the cluster C j , and its graphical description is shown in Figure 5.11 (the dotted line indicates the correlation between the indexes, Which is not marked in Figure 5.11 due to simplification).

Figure 5.11. Correlations between the clusters. Then, according to the value of ri∗j , a networked evaluation index system is constructed and described. The network structure of the evaluation index system is consistent with the description model of ANP. A typical index system of the networked evaluation is shown in Figure 5.12. (2) Supermatrix Based on Multi-Information Fusion The construction supermatrix based on the multi-information fusion of the experts and the simulation data is mainly completed by steps 5-8 in Figure 5.8. The details of the consistency analysis can be found in the previous chapters, and the detailed procedures for the remaining steps are as follows: (i) Collecting the experts experience data The judgment matrix mainly based on expert experience. It is assumed that there are n evaluation indexes I1 , I2 , · · · , In in the primary scheme of the index system. There (k) are m experts p = {p1 , p2 , · · · , pm}.M j is used to express the influence of the index I j ( j = 1, 2, · · ·n) given by the k expert on the evaluation purpose. The total importance

142

Deping Zhang and Xuefeng Yan

Figure 5.12. Networked index system. m

(k)

of the index I j is defined as v j = ∑ M j . After consulting experts and making statisk=1

tics, the total importance of consolidated expert opinions is v1 , v2 , · · · , vn respectively, and the total importance of indexes is sorted and normalized according to the formula n

w j = v j / ∑ v j . The normalized priority vectors W of n evaluation indexes are obtained, j=1 n

namely, W = (w1 , w2 , · · · , wn ) and ∑ wi = 1. i=1

(ii) Basic evaluation index of dynamic measurement The judgment matrix based on the simulation data is constructed by simulating and analyzing the correlation between some quantitative indexes. Firstly, according to the evaluation index system and based on the simulation system, a set of index parameters should be input for a given mission. Then the upper effectiveness evaluation index value affected by this set of parameters can be output. If there is a linear relationship between this group of indexes and the upper layer indexes, regression analysis can be adopted to realize this process. It is assumed that the input parameter value of a group of indexes is X = (x1 , x2 , · · · , xn ), where xi is the value of the index i.y is the upper layer operational capability evaluation index affected by this parameter. x j ( j = 1, 2, · · · , N) is the experimental index value of group j; y j is the upper layer operational capability evaluation index value corresponding to the experiment of group j. The fitting relationship regression model between each index value and operational capability evaluation index value is established using the least square method. n

y = b0 + ∑ bk xk

(5.44)

k=1

Furthermore, the influence coefficient vector B = (b1 , b2 , · · · , bn )T on y is obtained for the parameters of n indexes. A judgment matrix composed of relative importance can be

Classical Method of Effectiveness Evaluation formed by pairwise comparison of these influence coefficients.   bi = (ai j )n×n A= b j n×n

143

(5.45)

Based on the Equation (5.46), the matrix A can be adjusted to A∗ = (a∗i j )n×n, which can satisfy the 1-9 scale judgment requirements. dai j e is a rounding operation of ai j .  9, ai j ≥ 9    da i j e, 1 ≤ ai j < 9 a∗i j = (5.46) 1/dai j e, 1/9 ≤ ai j < 1    1/9, 0 < 0 < 1/9

If there is a non-linear relationship among the index parameters, a data-driven method can be adopted to analyze the correlation of each index by the machine learning or deep learning methods, such as SVR, radial basis neural network or DNN algorithm. (iii) Multi-evaluation source data

The judgment matrix of fusion experts and the judgment matrix based on simulation can be fused into a comprehensive matrix. According to the ANP, if each sub-column vector of the ANP supermatrix is the eigenvector of the corresponding judgment matrix, multiple judgment matrices can be fused by synthesizing the judgment matrix or the priority vector of each judgment matrix directly. Due to the differences in the status, knowledge, experience,and preferences of experts (simulation is taken as virtual experts), the importance of evaluations and judgments may also be different. Therefore, the importance (weight) of different experts is considered synthesizing expert opinions. There are four common methods for determining the weight of experts: 1) Weights are determined by the dictatorship of “authoritative experts” who is overall and impartial, but such a expert is hard to find, so it is not a practicable. 2) Self-evaluation. 3) Collective evaluation of the group members; 4) Evaluation based on the previous information. Here, the collective evaluation method of group members based on the delegation process is adopted to determine the weights. The specific description is as follows: It is assumed that A = (ai j )n×n and B = (bi j )n×n are the positive reciprocal judgment matrix. The addition operation of A and B is defined as C = A ⊕ B, where C = (ci j )n×n satisfies the following requirements:  ai j + bi j , i ≤ j (5.47) ci j = 1/ci j , i > j Let D = (di j )n×n and di j = ai j · bi j , the Hadamard product of A and B is defined as following: D = A⊗B It is assumed that the evaluation criteria is C, and there are l influence indexes which are (k) x1 , x2 , · · · , xl . A(k) = (ai j )l×l is the judgment matrix for the k (k = 1, 2, · · · , m) evaluation expert pk to compare the influence indexes in pairs with criterion C. Its eigenvalue is w(k) .

144

Deping Zhang and Xuefeng Yan m

The weights of each expert are expressed by λ1 , λ2 , · · · , λm , where ∑ λi = 1. The weighted i=1

arithmetic mean or the weighted geometric method will be adopted to fuse multiple judgment matrices. The formulas to calculate the comprehensive judgment matrix A¯ or A˜ can be respectively described as follow: A¯ = λ1 A(1) ⊕ λ2 A(2) ⊕ · · · ⊕ λm A(m) A˜ = (A(1))λ1 ⊗ (A(2))λ2 ⊗ · · · ⊗ (A(m) )λm

(5.48) (5.49)

Similarly, the formulas for calculating the comprehensive priority vector w¯ or w˜ are respectively as follows: w¯ = λ1 w(1) ⊕ λ2 w(2) ⊕ · · · ⊕ λm w(m) (1) λ1

(2) λ2

(5.50)

(m) λm

w˜ = (w ) ⊗ (w ) ⊗ · · · ⊗ (w

)

(5.51)

It can be seen that, after adjustment, the eigenvectors of A¯ or A˜ and w¯ or w˜ can be used as the sub-column vectors of the supermatrix. (iv) Construction of (weighted) supermatrix Unlike AHP, supermatrix is adopted by the ANP for storing the data that represents the correlation of the indexes. The weighted matrix expresses the degree of interaction between clusters. The weighted supermatrix is adapted for storing the correlations among all the indexes, and its power limit is the index weight obtained by the comprehensive analysis of all the mutual influence relations. According to the networked evaluation index system, the ANP model can be constructed for obtaining the number of judgment matrices, and the criteria and indexes. These judgment matrices can be constructed by using expert experience or the simulation data, and the original supermatrix W can be established by its eigenvectors. Similarly, according to the mutual influence relationship between index clusters, the relationship judgment matrix of these clusters is constructed, and the weighted matrix A can be constructed with the eigen¯ = A ◦W = (ai jWi j ) can be vector. Based on the theory of ANP, the weighted supermatrix W ∞ ¯ calculated, and the limit supermatrix (W ) can also be obtained. Then, in the effectiveness evaluation index system, the relative importance of each evaluation index can be sorted, and its weights can be calculated. (3) Effectiveness Evaluation Model Based on Index Weight The weighted sum and the power exponent are generally adopted to construct the effectiveness evaluation model based on index weight. (i) Weighted sum method The “weighted sum method”, also known as the linear weighted synthesis method or accumulation method, is the most common evaluation method adopted in the military field for evaluating the equipment effectiveness and the inherent operational capability of arms. The principle is to apply the arithmetic average operator, multiply the index weight calculated by ANP and the quantitative value of each layer to get the evaluation value of the operational effectiveness (capability) in the upper layer index.

Classical Method of Effectiveness Evaluation

145

(ii) Power exponent method The “power exponent method”, also known as the weighted product method, is the most commonly adopted evaluation method in the military field for evaluating equipment effectiveness and the inherent operational capability of arms. The principle of this method is to apply geometric arithmetic average operator, the index weight and the power index of the quantized value at each layer, and then multiply and aggregate them to obtain the evaluation value of upper level operational effectiveness (ability) study.

5.2.3.

Case Study

In this section, the operational capability evaluation of the infrared air-to-air missile system is taken as an example [68] to illustrate the availability of the comprehensive effectiveness evaluation method based on ANP. (1) Background The airborne missile, as the key equipment for precision strikes and air control has the capabilities of infrared guidance and radar guidance. The short-range operational missile is guided almost entirely by infrared, while medium and long range air-to-air missiles use radar guidance. The effectiveness evaluation of the infra-red air-to-air missiles in the shortrange is taken as an example. Based on the analysis of simulation experimental data, the measurement of the mission is inllustrcted under a certain operational situation. (2) Design of Networked Evaluation Index System Based on the characteristics of the airborne missile system, the structure and function of the infrared air-to-air missile are analyzed through analysis of the short-range operations. By referring to the evaluation indexes proposed in relevant studies, the existing indexes are improved and integrated, and the preliminary scheme operational capability evaluation index system is presented as is shown in Table 5.9. Table 5.9. Influencing factors table of the effectiveness Element group name Situational awareness capability C1

Control decision capability C2 Fire strike capability C3 Electronic warfare capability C4

included element radar detection capability Photoelectric radar detection capability Data link support capability Pilot’s visual capability Information processing and fusion capability Weapon system control capability Aircraft mobility Pilot control capability Missile guidance capability Missile damage effect Missile anti-jamming capability Electronic information attack capability Anti interference ability of electronic information

number C11 C12 C13 C14 C21 C22 C23 C24 C31 C32 C33 C41 C42

146

Deping Zhang and Xuefeng Yan

The comprehensive effectiveness evaluation method of ANP is adopted and five experts in related fields are invited to participate in consultation. The expert consistency coefficient CI can be calculated after each consultation. After three rounds of feedback consultation, it is obtained that CI=0.8531, indicating that the expert opinions tend to be consistent. The importance of these 13 indexes (data omitted) is normalized and sorted. According to the indexes ranking, the importance of each index can be calculated by the accumulation and 11

normalization. When the first eleven items are added, there is N = ∑ Ni = 0.9623 > 0.95. i=1

Therefore, the first eleven items are selected for constructing the operational capability evaluation index system and the other indexes are deleted. Based on the simulation data, the interactional relationships among the elven indexes are judged respectively, and the index correlation matrix (ri j )11×11 of the evaluation index system is obtained. The mutual influence relationship among indexes is represented by graphics in the evaluation index system, the same as the relationship between clusters and clusters. The operational capability evaluation index system is established based on the method described in the previous section, as is shown in Figure 5.13.

Figure 5.13. The index system of networked airborne infrared air-to-air missile. (3) Solution of Networked Evaluation Index System Firstly, the networked operational capability evaluation index system in a short-range need to be constructed. Then experts are invited to construct the experience based-judgment matrix. Next, the simulation system established, and the judgment matrices of the index set which can be quantitative analysis are created by collecting the simulation data. Finally, the supermatrix is constructed by fusing multiple judgment matrices, and the weight of each index is obtained by calculating the limit supermatrix. The details are described as follows: The operational effectiveness of infrared air-to-air missile in the control layer is taken as the criterion, and the damaging effect (C32 ) of the element missile in fire strike capability

Classical Method of Effectiveness Evaluation

147

(C3 ) in the network layer element group is taken as the secondary criterion. The indirect dominance is compared by considering the influence of the elements in the element group C1 on (C32 ). The judgment matrix can be constructed as is shown in Table 5.10. Table 5.10. The judgment matrix of C1 element group with C32 C32 C11 C12 C13

C11 1 5 1

C12 1/5 1 1/5

C13 1 5 1

Normalized eigenvector 0 .14286 0 .71429 0 .14286

Similarly, the judgment matrix can also be constructed by taking the elements C31 and C33 in C3 as the secondary criteria respectively. Due to the limitation of space, the normalized eigenvectors obtained from the above three judgment matrices are given directly. And a correlation matrix W13 of element group C1 to element group C3 is constructed with the objective criterion, which is shown as follows:   0.13153 0.14286 0.44332 W13 = 0.69403 0.71429 0.16920 (5.52) 0.17444 0.14286 0.38748

Similarly, W12,W14 ,W23,W31,W32,W33 ,W34,W41,W42,W43 can be calculated according to the relationship among these elements. Other correlation matrices that not listed here are zero, which indicates that there is no correlation between the two element groups. The weightless supermatrix W is calculated with the objective effectiveness criteria, as is shown in Table 5.11. Table 5.11. The weightless supermatrix W with the objective effectiveness criteria

C11 C12 C13 C21 C22 C23 C31 C32 C33 C41 C42

C11 0.000 0.000 0.000 0.000 0.000 0.000 0.081 0.188 0.731 0.250 0.750

C12 0.000 0.000 0.000 0.000 0.000 0.000 0.105 0.637 0.258 0.833 0.167

C13 0.000 0.000 0.000 0.000 0.000 0.000 0.333 0.333 0.333 0.250 0.750

C21 0.185 0.741 0.074 0.000 0.000 0.000 0.067 0.219 0.715 0.250 0.750

C22 0.142 0.678 0.179 0.000 0.000 0.000 0.258 0.637 0.105 0.167 0.8333

C23 0.075 0.324 0.602 0.000 0.000 0.000 0.143 0.429 0.429 0.500 0.500

C31 0.132 0.694 0.174 0.333 0.333 0.333 0.114 0.481 0.405 0.500 0.500

C32 C33 C41 C42 0.143 0.443 0.156 0.281 0.714 0.169 0.659 0.135 0.143 0.387 0.185 0.584 0.143 0.33333 0.000 0.000 0.714 0.333 0.000 0.000 0.143 0.333 0.000 0.000 0.081 0.150 0.073 0.179 0.188 0.106 0.205 0.113 0.731 0.744 0.722 0.709 0.500 0.250 0.000 0.000 0.500 0.750 0.000 0.000

In the eighth column of the Table 5.11, the first three values are the weights of the elements calculated in Table 5.10 (the last column).

148

Deping Zhang and Xuefeng Yan

Since only the objective criterion is taken into account in the structural model, there is only one weightless supermatrix, which is a non-negative matrix. The sub-blocks Wi j of the supermatrix is column normalized, but the W is not. Therefore, the target is taken as a criterion, the importance of each element group with the objective criterion is compared. And the weighted matrix can be obtained, which is shown in Table 5.12. Table 5.12. Weighted matrix between element groups with the objective criteria

C1 C2 C3 C4

C1 0.0000 0.0000 0.1667 0.8333

C2 0.3090 0.0000 0.1095 0.5816

C3 0.1900 0.380081 0.0664 0.3635

C4 0.7500 0.0000 0.2500 0.0000

¯ can be obtained From the weighted matrix obtained above, the weighted supermatrix W by weighting the elements of the supermatrix W , as is shown in Table 5.13. ¯ Table 5.13. The weighted supermatrix W

C11 C12 C13 C21 C22 C23 C31 C32 C33 C41 C42

C11 0.000 0.000 0.000 0.000 0.000 0.000 0.013 0.031 0.122 0.208 0.625

C12 0.000 0.000 0.000 0.000 0.000 0.000 0.017 0.106 0.043 0.694 0.139

C13 0.000 0.000 0.000 0.000 0.000 0.000 0.056 0.056 0.056 0.208 0.625

C21 0.057 0.229 0.023 0.000 0.000 0.000 0.007 0.024 0.078 0.145 0.436

C22 0.044 0.210 0.055 0.000 0.000 0.000 0.028 0.070 0.011 0.097 0.485

C23 0.023 0.100 0.186 0.000 0.000 0.000 0.016 0.047 0.047 0.291 0.291

C31 0.025 0.132 0.033 0.127 0.127 0.127 0.008 0.032 0.027 0.182 0.182

C32 0.027 0.136 0.027 0.054 0.271 0.054 0.005 0.013 0.048 0.182 0.182

C33 0.084 0.032 0.074 0.127 0.127 0.127 0.010 0.007 0.049 0.091 0.273

C41 0.117 0.494 0.139 0.000 0.000 0.000 0.018 0.051 0.181 0.000 0.000

C42 0.211 0.101 0.438 0.000 0.000 0.000 0.045 0.028 0.177 0.000 0.000

After 2k + 1 times evolution of the weighted supermatrix, k will approach infinity. The results will be consistent and a long-term stable matrix can be formed. In this case, the supermatrix will have the same values for each row. As is shown in Table 5.14. There is only one criterion for the target. Therefore, each column of the stability limit supermatrix obtained above is the relative weight of each element with respect to the target. The details is shown in Table 5.15. (4) Power Exponent Evaluation Model In the power exponent evaluation model, the power exponent reflects the relative importance of the parameters affecting the operational capability. The calculated results of

149

Classical Method of Effectiveness Evaluation ¯ ∞ under objective criterion Table 5.14. Limit supermatrix W

C11 C12 C13 C21 C22 C23 C31 C32 C33 C41 C42

C11 C12 C13 C21 C22 0.082 0.082 0.082 0.082 0.082 0.132 0.134 0.134 0.134 0.134 0.139 0.139 0.139 0.139 0.139 0.019 0.019 0.019 0.019 0.019 0.029 0.029 0.029 0.029 0.029 0.019 0.019 0.019 0.019 0.019 0.027 0.027 0.027 0.027 0.027 0.045 0.0452 0.045 0.045 0.045 0.106 0.106 0.106 0.106 0.106 0.173 0.173 0.173 0.173 0.173 0.227 0.227 0.227 0.227 0.227

C23 0.082 0.134 0.139 0.019 0.029 0.019 0.027 0.045 0.106 0.173 0.227

C31 0.082 0.134 0.139 0.019 0.029 0.019 0.027 0.045 0.106 0.173 0.227

C32 0.082 0.134 0.139 0.019 0.029 0.019 0.027 0.045 0.106 0.173 0.227

C33 0.082 0.134 0.139 0.019 0.029 0.019 0.027 0.045 0.106 0.173 0.227

C41 C42 0.082 0.082 0.134 0.134 0.139 0.139 0.019 0.019 0.029 0.029 0.019 0.019 0.027 0.027 0.045 0.045 0.106 0.106 0.173 0.1730 0.227 0.227

Table 5.15. The results of analysis C11 0.082

C12 0.134

C13 0.139

C21 0.019

C22 0.029

C23 0.019

C31 0.027

C32 0.045

C33 0.106

C41 0.173

C42 0.227

ANP are normalized and the power exponent coefficients are obtained. The operational capability power exponent evaluation model can be constructed. W Q = (C1 )u1 (C2 )u2 (C3 )u3 (C4 )u4

(5.53)

where C1 ,C2 ,C3 and C4 represent situation awareness, control decision, fire strike and electronic warfare capabilities respectively. u1 , u2 , u3 , and u4 are their power exponent coefficients respectively. Similarly, for each single operational capability, the power exponent evaluation model can be established. Limited by space , only the power exponent evaluation model of the situation awareness capability is given below: C1 = (C11 )u11 (C12 )u12 (C13)u13

(5.54)

where C11 ,C12 and C13 represent the radar detection, photoelectric radar detection capability and data link information support capabilities respectively. u11 , u12 and u13 are respectively the influence indexes of the C11 , C12 , C13 on the situational awareness capability.

150

Deping Zhang and Xuefeng Yan

5.3. Comprehensive Effectiveness Evaluation Based on Factor Analysis 5.3.1.

Basic Principles of Factor Analysis

The selection and establishment of evaluation index system is the important foundation of comprehensive evaluation and the guarantee of effectively evaluating system. The evaluation index should cover the main attributes and reflect the essential attribute of the evaluated object accurately. The evaluation index system should have strong logic completeness. The lower-level indexes are subordinate to the upper-level indexes and cover their attributes. The indexes of the same level are independent of each other and do not intersect with each other, reflecting one aspect of the attributes of the evaluated object independently. However, it is usually difficult to divide the attributes of the evaluated objects strictly, and it is inevitable that there are various links between the indexes at the same level, which leads to certain correlations among the evaluation results and makes it difficult to reflect the objective reality correctly and appropriately In fact, the evaluation index system establishment is a process of analyzing problems using the system ideology. If there are many index influence factors, it is a general and difficult problem to screen the indexes properly and establish a “comprehensive” and “unique” index system. The credibility and effectiveness of the evaluation index system can be improved by researching the index correlation and using the factor analysis method [151]. (1) Basic Principles of Correlation Analysis of Evaluation Index In the comprehensive evaluation system, there are generally qualitative index and quantitative index, the unit and the magnitude of each index are different. There is incommensurability between the indexes. Therefore, the indexes should be standardized before the correlation analysis and then the incomparable indexes should be converted into the comparable. The correlation degree between indexes is usually expressed by the correlation coefficient. It is assumed that there are n evaluation indexes. The index vector is X = (X1 , X2 , · · · , Xn)T , and the mean vector of X is E[X] = (E(X1 ), E(X2), · · · , E(Xn))T , its covariance matrix Σ is shown as follows: Σ = Cov(X, X) = E[(X − E(X))E(X − E(X))T ]   Cov(X1, X1 ) Cov(X1 , X2 ) · · · Cov(X1 , Xn ) Cov(X2, X1 ) Cov(X2 , X2 ) · · · Cov(X2 , Xn )   =   .. .. .. ..   . . . . Cov(Xn, X1 ) Cov(Xn , X2 ) · · · Cov(Xn , Xn ) = (σi j )n×n

(5.55)

(5.56)

(5.57)

where, Cov(Xi, X j ) = σi j is called the covariance of the i-th component Xi and the j-th component X j in X. The covariance matrix Σ is a symmetric matrix. The correlation coefficient r between the two indexes Xi and X j is shown as follows: ri j =

Cov(Xi, X j ) σi σ j

(5.58)

Classical Method of Effectiveness Evaluation

151

where, ri j and the covariance Cov(Xi, X j ) represent the correlation coefficient p and the correlation degree between the two indexes Xi and X j respectively. The σi = Var(Xi) is the standard deviation of the index Xi , which indicates the dispersion degree of the index value. The correlation coefficient is between -1 and 1. Its properties are shown as follows: If r >0, the two variables are positively correlated. They are correlated negatively when r < 0. If |r| = 1, it indicates that the two variables are completely linearly dependent, namely, a function relationship exists. If r = 0, the two variables are linearly independent. If 0 < |r| < 1, it means that there is a certain linear dependence between the two variables. And the closer |r| is to 1, the closer the linear relationship is. The closer |r| is to 0, the weaker the linear relationship is. Generally, it can be divided into three levels: |r| < 0.4 is low linear dependence, 0.4 < |r| < 0.7 is significant dependence, and 0.7 < |r| < 1 is high linear dependence. For n evaluation indexes, the correlation coefficients between the two variables can be calculated separately to form a correlation coefficient matrix.   r11 r12 · · · r1n r21 r22 · · · r2n    R= . (5.59) .. . . ..   .. . . .  rn1 rn2 · · · rnn

Factor analysis is a process of reconstructing a few representative factor variables from a large number of original variables. Its underlying requirement is that the original variables must have relatively strong correlation. Therefore, factor analysis requires the correlation analysis and then the correlation coefficient matrix between the original variables can be calculated. Before analyzing the correlation of the original variables, it is necessary to conduct standardized calculations on the original data. It is assumed that there are p original variables X1 , X2 , · · · , Xp and they may be dependent or independent. Then Xi can be standardized to a new variable Zi : Xi − E(Xi ) Zi = p Var(Xi)

(5.60)

The correlation coefficient matrix is R. The characteristic equation |R − λIn | = 0 of R can be calculated to obtain p eigenvalues λ1 , λ2 , · · · , λ p and corresponding eigenvectors β1 , β2 , · · · , β p . Then the factor analysis model can be established as follows: Zi = ai1 F1 + ai2 F2 + · · · + aim Fm + εi i = 1, 2, · · · , p

(5.61)

where Fj ( j = 1, 2, · · · , m), called common factor, appears in the formula of each variable. Its implication should be interpreted according to the specific problem. εi (i = 1, 2, · · · , p), is only related to the variable Zi . The coefficient ai j (i = 1, 2, · · · , m; j = 1, 2, · · · , m) is called load factor and A = (ai j ) is called load matrix.

152

Deping Zhang and Xuefeng Yan The Equation (5.61) can be expressed as the following matrix form: Z = AF + ε

(5.62)

where Z = (Z1 , Z2 , · · · , Z p )T , F = (F1 , F2 , · · · , Fm )T , ε = (ε1 , ε2 , · · · , ε p )T , and A = (ai j ) p×m

(5.63)

It is assumed that the special factors are independent of each other and independent of all common factors, namely, Cov(ε, ε) = diag(σ21 , σ22 , · · · , σ2p )

Cov(F, ε) = 0

(5.64) (5.65)

It is further assumed that all common factors are independent normal random variables with mean 0 and variance 1. Its covariance matrix is the unit matrix Im , namely F ∼ N(0, Im). When the components of factor F are dependent, Cov(F, F) is no longer a diagonal matrix. This model is called the oblique factor model, which is not considered here. The contribution rate of the i factor is calculated and the number of common factors m should be determined. The factor contribution rate is defined as follows: m

γ i = λi / ∑ λi

(5.66)

i=1

where m represents the number of common factors, which is determined by the number of eigenvalue roots greater than or equal to 1, or by the factor contribution rates greater than or equal to 85%. Then the contribution of m common factors to the variance of the variable i is called the i communality, which is recorded as h2i . Therefore, h2i = a2i1 + a2i2 + · · · + a2im

(5.67)

However, the variance of specific factor is called specific variance or value, which is = 1, 2, · · · , p) in the Equation (5.64). Then, the variance of the variable i has the following decomposition.

σ2i (i

Var(Zi) = h2i + σ2i

(5.68)

A basic problem of factor analysis is estimating factor loading, that i how to calculate the factor model given by the Equation (5.61). Intuitively, the following deformation computation is attempted to apply to the Equation (5.62): Z T Z = (AF + ε)T (AF + ε)

(5.69)

Classical Method of Effectiveness Evaluation

153

The non-public parts can be removed form the right part and the results are shown as follows: Z T Z ≈ (AF)T (AF)

(5.70)

It is noticed that the approximate equal is not caused by extracting common factors (with a few number of common factors), but by removing the specific factors. The eigenvalue decomposition of real symmetric matrix Z T Z is executed as follows: Z T Z = V ΛV T

(5.71)

It is assumed that λ1 ≥ λ2 ≥ · · · ≥ λ p are the eigenvalues of the sample correlation coefficient matrix R. β1 , β2 , · · · , β p are the corresponding standard orthogonal eigenvectors. Assuming that m < p, the result is shown as follows:    λ1 β1     λ2   β2  (5.72) Z T Z = [β1 , β2 , · · · , βn ]     ··· · · · βn λn √  λ1 β1 √ p p p  λ2 β2   = [ λ1 β1 , λ2 β2 , · · · , λn βn ]  (5.73)  ···  √ λn βn

Through the comparison of the above formulas, Z T Z ≈ (AF)T (AF) is consistent in form and conforms to the definition of common factor. Therefore, the load matrix A of the sample correlation coefficient matrix R factor analysis can be obtained, which is shown as follows: p p p A = ( λ1 β1 , λ2 β2 , · · · , λm βm )

(5.74)

The variance of specific factors can be estimated by diagonal elements of R − AAT , namely, m

σ2i = 1 − ∑ a2i j

(5.75)

j=1

The residual matrix is expressed as R − AAT − Cov(ε, ε). Therefore, when AAT + Cov(ε, ε) is relatively close to the correlation coefficient matrix R, it can be considered intuitively that the factor model has better fit to the data. The parameters of the factor model are generally taken as the emphasis of the factor analysis, which is the load matrix. Sometimes, it is necessary to estimate common factors (factor score). The factor score can be used for model diagnosis or as raw data for the further analysis. It should be pointed out that the calculation of factor score is not the parameter estimation, but an estimation value of unobservable random vector Fi .

154

Deping Zhang and Xuefeng Yan

The specific data values of each sample data on different factors are expected to obtain after the factor variables are determined, which is called factor score. There are several methods to estimate factor scores, such as regression, Bartlett, etc. The factor variable should be expressed as a linear combination of the original variables at first, which is shown as follows:  F1 = a11 x1 + a12 x2 + · · · + a1p x p    F2 = a21 x1 + a22 x2 + · · · + a2p x p (5.76) ··· ···    Fm = am1 x1 + am2 x2 + · · · + amp x p

In general, factor scores can be estimated by weighted least squares and regression methods. Regression is also called Thomson method. The factor score is derived from the Bayes’ theorem and there is the partial derivation in the obtained factor scores, but it is small. Bayesian discrimination means that the posterior probability is calculated according to the prior probability and statistical inference can be executed according to the posterior probability distribution. Bartlett factor score is the maximum likelihood estimation and also the weighted least squares regression. There is no partial derivative in the obtained factor scores, but the error of calculation results is large. The principal component decomposition is not unique, because any orthogonal transformation of A will not change the original AAT . It is assumed that Q is an orthogonal matrix of m-order and B = AQ, then BBT = AAT . This non-uniqueness of the load matrix is unfavorable, but it can be used to make the new factors with more distinct practical significance or interpretability through appropriate factor transformation. For example, the orthogonal transformation is used to obtain as many elements in B equal to or close to 0 as possible. Therefore, the structure of factor load matrix is simplified and more practical explanation can be made. Orthogonal transformation is a kind of rotation transformation. If the orthogonal rotation with large variance can be selected, that is, each factor is rotated to a certain position with the largest or smallest projection on the factor axis. Therefore, the high load of each factor will appear on only a few variables. In the rotation factor load matrix, except for a few values, all the elements in each column are close to zero. The principle of the orthogonal transformation is illustrated by taking the plane orthogonal rotation of two factors as an example. It is assumed that the factor load matrix is a = (ai j ), i = 1, 2, · · · , p; j = 1, 2 and the orthogonal matrix is shown as follows:   cos φ − sinφ (5.77) Q= sinφ cos φ The rotation is counterclockwise for the matrix Q. If it is clockwise, the two elements can be swapped on the secondary-diagonal line of the Equation (5.77) and recorded as follows: B = AQ = (bi j ), i = 1, 2, · · · , p, j = 1, 2

(5.78)

Classical Method of Effectiveness Evaluation

155

B is the rotation factor load matrix and the Equation (5.62) will be changed, as is shown in the following: Z = B(QT F) + ε

(5.79)

At the same time, the common factor F becomes QT F. According to rotation, the variables can be divided into two parts that are mainly explained by different factors. Therefore, the variances calculated by the two columns (b211, b221 , · · · , b2p1 )and (b212 , b222 , · · · , b2p2 ) respectively should be as big as possible. The relative variance is considered and its calculation formula is shown as follows: !2 !2 2 2 1 p bi j 1 p bi j − (5.80) Vj = ∑ ∑ h2 , j = 1, 2 p i=1 h2i p i=1 i where, b2i j is used to eliminate the influence of the symbol of bi j . The purpose of dividing by h2i is to eliminate the influence of how the variables depend on the common factor. The purpose of orthogonal rotation is to maximize the total variance V = V1 +V2 . And dV dφ = 0, φ should be satisfied the following formula after calculation. tan 4φ =

D0 − 2A0 B0 /p C0 − (A20 − B20 )/p

(5.81)

where, p

A0 =

p

∑ ui, B0 = ∑ vi

i=1 p

C0 = ui =

p

∑ (u2i − v2i ),

i=1



ai1 hi

(5.82)

i=1

2



D0 = 2 ∑ ui vi

ai2 − hi

2

(5.83)

i=1

, vi =

2ai1 ai2 h2i

(5.84)

When m = 2, the coordinate axis can rotate intuitively an angle φ by using graphic method. The general approach is to cluster the variables at first. Then it is easy to determine new common factors by using clustered variables. When the number of common factors satisfy m > 2, two factors can be selected from the m factors each time for rotation. There are Cm2 = m(m−1) rotation way. One time of 2 m(m−1) the cycle can be completed by executing rotations. Then the second cycle will be 2 started. The relative variance of each column of the A matrix and V will become larger after each cycle. After the k cycles, if V (k) is few different from V (k−1) of the previous cycle, the rotation will stop. In summary, the flow of factorization is shown in Algorithm 5.5. The basic principle of factor analysis is based on linear relationships among variables. Most of the observed data are summarized into few common factors using correlation coefficient matrix and covariance matrix. The remaining variation can be regarded as a special factor. The latent prerequisite for factor analysis is that there is a strong correlation

156

Deping Zhang and Xuefeng Yan

Algorithm 5.5. Factoring algorithm (1) The original data are standardized to eliminate the differences in magnitude and dimension between variables. (2) The correlation matrix of the standardized data is calculated. (3) The eigenvalues and eigenvectors of the correlation matrix are calculated. (4) The contribution rates of variance and cumulative variance are calculated. (5) Determination factor. It is assumed that F1 , F2 , · · · , Fp are p factors, when the overall data information (cumulative contribution rate) contained by top m factors is not less than 85%, the top m factors can be used to reflect the original evaluation index. (6) Factor rotation. If obtained m factors cannot be determined or its practical significance is inconspicuous, the factors should be rotated. (7) The linear combination of the original indexes is used to obtain the score of each factor, such as regression and Bartlett estimation method. (8) Comprehensive factor score. For each index, the variance contribution rate is regarded as the weight, and the comprehensive evaluation index function can be obtained from the linear combination: F=

m γ1 F1 + γ2 F2 + · · · + γm Fm = ∑ ωi Fi γ1 + γ2 + · · · + γm i=1

(5.85)

where ωi is the variance contribution rate of the factor before or after rotation (9) Score ranking. The score ranking can be obtained by using the comprehensive score analysis.

between original variables, which is an essential condition for synthesizing the common factors that can reflect the common characteristics of some variables from the original variables. Therefore, the premise conditions should be tested before factor analysis. There are three methods to test the original variables, such as KMO test, Bartlett sphericity test and variable commonality method. In the KMO test, the value of KMO is used to indicate whether the original variable is suitable for factor analysis. The closer the KMO value is to 1, the stronger the correlation is and the more suitable for factor analysis is. Otherwise, it is not suitable. In general, the judgment criteria of KMO is shown in Table 5.16. Bartlett sphericity test assumes that the correlation coefficient matrix is the identity matrix. If the calculated results are relatively large and the corresponding significance probability is less than the given probability, the original hypothesis is rejected and the original variable is suitable for factor analysis. Variable commonality refers to the variation part of original variable that can be explained by the common factors. The stronger the explanatory

Classical Method of Effectiveness Evaluation

157

Table 5.16. KMO criteria for judging statistics KMO statistics ≥ 0.90 ≥ 0.80 ≥ 0.70 ≥ 0.60 ≥ 0.50 ≤ 0.50

Suitability of factor analysis excellent favorable moderate mediocre lamentable unacceptable

power of common factors is. When the value of variable commonality is between 0 ∼ 1, the larger the value is, the more suitable original variable are for factor analysis. Otherwise, it’s not suitable.

5.3.2.

Efficiency Evaluation Based on Factor Analysis

The factor analysis can be used to evaluate the effectiveness of equipment. The construction pattern of evaluation index need be determined based on the evaluation object and purpose. And the evaluation index can be selected to construct the effectiveness evaluation index system of equipment. In the evaluation, there are many indexes about the performance, tactics and effect of SoS. Therefore, the independent index with great influence on the effectiveness should be selected. After the evaluation parameters are obtained, in order to eliminate the influence of different dimensions of variables, it is necessary to standardize and establish correlation coefficient matrix R of the original data. Then, according principal component analysis, the eigenvalues and variance contribution rate of the correlation coefficient matrix R can be calculated by taking the eigenvalues greater than 1 as the criterion. Therefore, the common factor can be calculated. The factor load matrix can be established and orthogonal rotated. Finally, the factor score is calculated to obtain the effectiveness evaluation conclusion.

5.3.3.

Case Study

In this section, the land attack of cruise missile is taken as an example to evaluate the operational effectiveness. The index system is constructed through the operational purpose. And the operational effectiveness is evaluated by using factor analysis. Four kinds of common cruise missiles are selected, such as AGM-129 stealth cruise missile of the United States, 3M-54 cruise missile of Russia, storm shadow cruise missile of Europe, and “ Scalp EG” cruise missile that is jointly researched and developed by Britain, France and Italy. By referring to the relevant data, the performance parameters are shown in Table 5.17. In the Table 5.17, x1 , x2 , x3 are the firing range (km), the trajectory length (m) and the flying velocity (Mach) respectively. x4 is the operating distance (km) of guidance platform, x5 , x6 and x7 represent the penetrability (mm), the hit accuracy (%) and the maneuvering radius of terminal trajectory (m) respectively. The specific evaluation steps are shown as follows:

158

Deping Zhang and Xuefeng Yan Table 5.17. Performance of four cruise missiles cruise missile AGM-129 3M-54 Storm shadow Scarp EG

x1 3200 5000 250 400

x2 1,534 1,472 1,522 1,188

x3 0.8 1.2 1.5 0.8

index x4 8 12 6 5

x5 600 550 450 480

x6 96 94 98 90

x7 2500 1800 2000 1500

(1) Standardization of raw data. (2) The correlation coefficient matrix of the raw data R is calculated, which is shown in Table 5.18. After verification, KMO = 0.732, which indicates that there is a certain correlation between the observed variables. Then, the factor analysis can be carried out. Table 5.18. The correlation coefficient matrix R of raw data Index x1 x2 x3 x4 x5 x6 x7

x1 1.0000 0.6513 0.8912 0.9814 0.1123 0.4301 0.9865

x2 0.6513 1.0000 0.9103 0.5162 0.4721 -0.0521 0.7892

x3 0.8912 0.9103 1.0000 0.8671 0.1902 0.3503 0.9781

x4 0.9814 0.5162 0.8671 1.0000 0.0287 0.5892 0.9702

x5 0.1123 0.4721 0.1902 0.0287 1.0000 -0.8703 0.0328

x6 0.4301 -0.0521 0.3503 0.5892 -0.8703 1.0000 0.6109

x7 0.9865 0.7892 0.9781 0.9702 0.0328 0.6109 1.0000

(3) The eigenvalue of R and the corresponding variance contribution rate are calculated, which is shown in Table 5.19. Table 5.19. Eigenvalue and variance contribution rate of R Component

Eigenvalue

x1 x2 x3 x4 x5 x6 x7

4.678 2.012 0.513 1.32 × 10−15 −7.51 × 10−16 −2.13 × 10−15 −8.65 × 10−15

Variance contribution rate 66.125 28.153 5.722 1.27 × 10−14 −8.33 × 10−15 −4.79 × 10−14 −1.12 × 10−14

Cumulative contribution rate 66.125 94.278 100 100 100 100 100

Classical Method of Effectiveness Evaluation

159

Obviously, the variance contribution rate of the first two eigenvalues is more than 85 %. It means that the most information of the row data has already contained in the first two principal components of factor analysis, which can also be illustrated by the Figure 5.14.

Figure 5.14. Eigenvalue curve of correlation coefficient matrix R. As the cumulative contribution rate of the first two principal components has reached 94.278%, the row variables can be analyzed by using this components. The unit eigenvector corresponding to the eigenvalue can be calculated to determine the factor load and establish the factor loading matrix. (4) The factor loading matrix orthogonal rotation. The comparison before and after rotation is shown in Table 5.20. Table 5.20. Factor loading matrix after the rotation Variable x1 x2 x3 x4 x5 x6 x7

Factor loading matrix 0.981 0.768 0.964 0.953 0.975 0.949 0.998

0.003 0.517 0.143 -0.175 0.139 -0.401 -0.513

Factor loading matrix before and after rotation 0.983 -0.072 0.798 0.435 0.986 0.061 0.945 -0.208 0.305 0.957 0.398 -0.943 0.993 -0.138

From the Table 5.20, x1 , x2 , x3 , x4 , x7 can be mainly explained by the first principal component, which are the firing range, the trajectory length, the flying velocity, the operating distance of guidance platform, the maneuvering radius of terminal trajectory respectively, and can also be called the cruise missile performance factors. The second principal component mainly explains x5 and x6 , namely the two parameters of penetrability and hit accuracy, which can be called as the firepower strike effect factor. It shows that the actual meaning of the factor becomes clearer after rotation.

160

Deping Zhang and Xuefeng Yan

(5) The factor scores are calculated and effectiveness evaluation is executed, which is shown in Table 5.21. where F1 , F2 are the linear functions that is corresponded to each Table 5.21. Operational capability evaluation effect

F1 F2 F

AGM-129 0.6871 -0.7685 0.2524

3M-54 0.9703 0.1517 0.7259

Storm Shadow -0.3836 1.3952 0.1475

Scarp EG -0.1427 0.0632 -0.0811

column of the factor loading matrix after orthogonal rotation. The following formula can be obtained based on the calculated factor loading matrix. F1 = 0.231x1 + 0.187x2 + 0.245x3 + 0.197x4 + 0.074x5 + 0.067x6 + 0.238x7 F2 = −0015x1 + 0.236x2 + 0.072x3 + 0.0907x4 + 0.5074x5 − 0.417x6 − 0.046x7 In the comprehensive evaluation formula, α1 and α2 are the variance contribution rates of the corresponding principal components respectively. The cumulative contribution rates are 66.125% and 28.153% respectively after rotating the two principal components. From the Table 5.19, the cumulative contribution rate is 94.278%. α1 = 66.125/94.278 = 0.7014

(5.86)

α2 = 28.153/94.278 = 0.2986

(5.87)

Therefore, there are F = 0.7014F1 + 0.2986F2. The standardized raw data can be calculated to obtain the evaluation effect of the cruise missile operational capability. The evaluation result can be clearly seen from the evaluation value in Table 5.21. That is, the Russian 3M-54 cruise missile has the best operational effectiveness, and the Scarp EG cruise missile is relatively poor. It is basically consistent with the operational effect of the weapons in practice.

5.4. Comprehensive Effectiveness Evaluation Based on ADC The ADC is a common and mature model of system effectiveness evaluation. The model is proposed by the US Weapon System Effectiveness Industry Advisory Committee (WSEIAC). The system effectiveness is a measurement for the degree to which a system is expected to meet specified mission requirements. It is a function of system availability, dependability and inherent capability [152]-[156].

5.4.1.

Basic Principles of ADC

The ADC is the most widely used model for the effectiveness evaluation [157]-[162]. The indexes such as reliability, maintainability and inherent capability are transformed into three comprehensive indexes of availability (A) , dependability (D) and inherent capability (C).

Classical Method of Effectiveness Evaluation

161

The system effectiveness is mainly based on the three indexes. Availability is the measurement of the system state at the beginning of the mission. Dependability is the measurement of the system state at one or more instants during the mission execution of the system. Inherent capability is the measurement of the ability to achieve mission purpose during the execution of the system. It is necessary to address the following three issues when using ADC for the effectiveness evaluation. (1) What is the system state at the beginning of the mission? How much is the availability? (2) How will the system state change during the mission execution? What is the transition probability? How about considering maintenance? (3) What is the probability that the system can complete the mission at each state or each state transition during the mission execution? Accordingly, the expression of the system effectiveness is, E = A · D ·C

(5.88)

where E is the system effectiveness, A = [a1 a2 · · · an ] is the availability vector, n is the number of the system states at the beginning of the mission. D = (di j )n×n is a dependability matrix, and di j is the transition probability from the initial system state to the state j during the mission execution. C is the capability vector. The system effectiveness E can be calculated in two cases based on the form of C. Case 1: if C is the vector C = [c1 c2 · · · cn ]T , the calculation of the system effectiveness E is shown as follows.    d11 d12 · · · d1n c1 d21 d22 · · · d2n  c2     (5.89) E = [a1 a2 · · · an ]  . .. ..   ..  . . .  . . . .  .  dn1 dn2 · · · dnn cn Case 2: if C is the matrix C = (ci j )n×n , the calculation of the system effectiveness E is shown as follows.   d11 c11 + d12 c12 + · · · + d1n c1n d21 c21 + d22 c22 + · · · + d2n c2n    E = [a1 a2 · · · an ]  (5.90)  ..   . dn1 cn1 + dn2 cn2 + · · · + dnn cnn The system effectiveness index E can be regarded as the probability of successful completion of the mission, which is illustrated by the Equation (5.88). It is assumed that the system is always in one of the n states at any time. It is assumed that the initial system state is the B1 , B2 , · · · , Bn and the final system state is the E1 , E2 , · · · , En , where ∪i Bi = ∪i Ei = S and S is the sample space. It is supposed that the event C represents the mission that need to be completed. According to the calculation rules of probability events, there will be, C = C ∩ S ∩ S = C ∩ (∪i Bi ) ∩ (∪ j E j ) = ∪i j (CBi E j )

(5.91)

162

Deping Zhang and Xuefeng Yan Therefore, P(C) = ∑ P(CBi E j ) = ∑ P(Bi)P(E j |Bi )P(C|Bi, E j ) ij

(5.92)

ij

where P(C) is the probability that the mission is completed by the system. P(Bi ) is the probability of the system in state i at the beginning. P(E j |Bi ) is the transition probability from the initial system state i to the final system state j. P(C|Bi, E j ) is the conditional probability of completing the mission from the initial system state i to the final system state j. Compares the Equation(5.92) and the Equation (5.88), in the equation of the system effectiveness evaluation, E is P(C). Therefore, E is a probability index for effectiveness measurement. P(Bi) is the availability ai of the initial system state i. P(E j |Bi ) is the transition probability di j from the initial system state i to the final system state j. P(C|Bi, E j ) is the conditional probability c j or ci j of completing the mission from the initial system state i to the final system state j. If P(C|Bi, E j ) = c j is adopted in the Equation (5.92), then E = p(C) = ∑ ai di j c j

(5.93)

i, j

The expansion of the Equation (5.93) is the Equation (5.89). It shows that the E is the unconditional probability of completing the mission in the Equation (5.88). c j is the conditional probability of completing the mission. It emphasizes the effect of the final system state j on the mission completion, which is expressed in probability. It is believed that the initial system state and the final system state are independent. The initial state has no influence on the completion and the final system state has the largest effect on the mission completion. Therefore, c j is calculated based on the probability that the system can complete the mission at the end of the mission. If P(C|BiE j ) = ci j is adopted in the Equation (5.92), then E = P(C) = ∑ ai di j ci j

(5.94)

i, j

The expansion of the Equation (5.94) is the Equation (5.90). As mentioned above, ci j is the conditional probability that the system completes the mission. It emphasizes that the system has effects on the mission completion from the initial system state i to the final system state j, which is expressed in probability. It is considered that the initial system state is related to the final system state and has effects on the mission completion. ci j is calculated based on the probability of mission completion from the state i to the state j. Therefore, the Equation (5.90) can be applied to a system with a longer working time, and it reflects the probability that the system completes the mission from start to finish. The choice of c j or ci j does not depend on whether the system works continuously or not throughout the mission, but on the effect of the actual situation on the completion of the mission. The inherent capability matrix C is determined by the objects, missions and system characteristics of the evaluated system. E = ADC is the application of the total probability formula. In fact the multiplication formula and E is a probability index. c j and ci j are also the probabilistic effectiveness

Classical Method of Effectiveness Evaluation

163

indexes, but they are the conditional probability. Based on this idea, it is possible to provide new thinking for the large system synthesis, the dynamic process synthesis and the effectiveness evaluation of the mission-oriented system.

5.4.2.

The Effectiveness Evaluation Based on ADC

The equipment systems have been fully demonstrated and reviewed by multiple parties during the top-level design. Its sub-systems have specified missions and play an important role in the operational process. The system effectiveness will be definitely affected when the sub-system failure. The relationships between each sub-system and the overall system are basically summarized into three types: series, parallel, and series parallel. Series means that the mission cannot completed by the system when a sub-system fails to work. Parallel means that the mission can be completed by the system when a certain sub-system fails to work. However, its capability is reduced. For example, once the chassis sub-system of the wheeled equipment fails, the mission cannot be completed. The relationship between the chassis sub-system and the wheeled equipment system is series. The combination of inertial navigation and satellite positioning is adopted in the orientation of reconnaissance vehicles. Both of them can work in combination or independence. When inertial navigation or satellite positioning fails, the orientation missions can be completed. Therefore, the relationship is parallel. The series parallel relationship refers to the combination of series and parallel by multiple subsystems. Two sub-systems seldom fail at the same time. So it is unnecessary to consider the failure of multiple sub-systems during mission execution. If a system has m + n sub-systems, in which m sub-systems are serial systems and n sub-systems are parallel. It is assumed that there is at most one sub-system failure during the mission executing. The ADC is adopted to analyze the system effectiveness evaluation, as follows: ai =

MT BFi MT BFi + MT T Ri

(5.95)

where, the MT BFi is the average time between failures of the sub-system i, MT T Ri is the average time of repairing sub-system i. When the sub-system effectiveness is calculated in units of work times or work miles. The availability is calculated based on the mission profile and its corresponding mean fault interval, the average usage times, the whole system can still ensure the mission completion even though there are n + 1 faults in sub-system, that is, the entire system is failure-free. If a certain parallel sub-system fails (totally n sub-systems, regardless of multiple system failures), the mission n + 2 cannot be completed, which means that the series system fails. Therefore,  a1 = a1 × a2 × · · · × am+n     a  2 = (1 − a1 ) × a2 · · · × an × · · · × am+n ··· (5.96)    a = a × a × · · · × (1 − a ) × · · · × a 1 2 n m+n   n+1 an+2 = 1 − a1 − a2 − · · · − an+1

164

Deping Zhang and Xuefeng Yan

where, a1 is the availability when the system is faultless. a2 is the availability when the first parallel sub-system fails. an+1 is the availability when the last parallel sub-system fails. an+2 is the availability that the mission cannot be completed by the system. In summary, the availability matrix A = [a1 a2 · · · an+2] is obtained. According to the system mission profile and the combination of the system failure rate and the average failure time etc., the dependability of each sub-system is as follows:   T Ri = exp − (5.97) MT BFi where MT BFi is the average time between failures of the sub-system i. It is assumed that the fault state cannot be transferred to a non-fault state, and there is at most only one sub-system failure. Therefore, the elements of the dependability matrix are shown as follows:  d11 = R1 × · · · × Rn+m     d 12 = (1 − R1 ) × R2 × · · · × Rn+m     ···     d1(n+1) = R1 × · · · × (1 − Rn ) × · · · × Rn+m  d1(n+2) = 1 − R1 − · · · − d1(n−1)    d 21 = 0     d 22 = R1 × R3 × · · · × Rn+2     d =0   23 ···  d2(n+2) = 1 − d22     d 31 = 0     d32 = 0    d33 = R1 × R2 × R4 × · · · × Rn+2  ···     d(n+2)1 = 0     ···   d(n+2)(n+2) = 1

(5.98)

The state 1 in the matrix means the system is faultless. The failure rates of n parallel systems are indicated by the states 2 to n + 1 respectively. The failure rate of the series system is represented by n + 2. It is assumed that the system has one sub-system failure at most and the fault state cannot be transferred to the non-fault state. At this time, the dependability is 0. Therefore, the dependability matrix is obtained by calculation, which is shown as follows:   d11 d12 ··· d1(n+2)  d21 d22 ··· d2(n+2)    (5.99) D= .  .. .. ..   .. . . . d(n+2)1 d(n+2)2 · · · d(n+2)(n+2)

165

Classical Method of Effectiveness Evaluation

There are several methods to model the capability C in the ADC metod. For example, a combination method of the comprehensive system effectiveness index evaluation and the experts consulting can be used. The system evaluation is mainly determined by the effectiveness index. However, the effectiveness indexes have different weights for different mission. Therefore, the indexes can be weighted and assigned to the system capability. Specifically speaking, the system is divided into some sub-systems, which is further decomposed according to the effectiveness index And then the evaluation values and weights are generated from indexes and experts experience. Finally, the capability matrix is determined, as follows:   S(c1 ) S(c2 )   Ci = [QZ c1 QZ c2 · · · QZ ci ]  .  (5.100)  ..  S(ci )

where QZ ci is the weight of the index ci . S(ci ) is the evaluation value of the index ci . The capacity matrix is obtained as follows: C = [C1 ,C2 , · · · ,Cn+1 ,Cn+2]T

(5.101)

where C1 is the capacity when the system is faultless. C2 is the capacity when the first parallel sub-system fails. Cn+2 is the capacity when the serial system fails, and Cn+2 = 0. According to the definition of the ADC, the final effectiveness of the system is obtained as follows: E = ADC



d11 d21 .. .

··· ··· .. .

d1(n+2) d2(n+2) .. .



    = [a1 a2 · · · an+2 ]   [C1 , · · · ,Cn+2]T   d(n+2)1 · · · d(n+2)(n+2)

5.4.3.

(5.102)

Case Study

The comprehensive effectiveness evaluation based on the ADC is illustrated by the example of effectiveness evaluation of the mission-oriented manned/unmanned aerial vehicle (MAV/UAV) cooperative attack on the sea [?]. (1) Background A mission-oriented cooperative attack on the sea is performed collaboratively by MAV and UAV, in which the cooperation operation are undertaken by the UAV. The formation is constituted by one MAV as the leader aircraft and two UAVs for cooperative attack. The operational mode is shown in Figure 5.15.

166

Deping Zhang and Xuefeng Yan

Figure 5.15. The MAV/UAV cooperative attack mode on the sea. Under the guidance of the early warning aircraft and the ship command system, various sensor information must be transmitted by the MAV/UAV formation based on the battlefield information communication system. After the information fusion, the battlefield situation and the threat of the objects can be estimated. At the same time, the tactical decision of the system cooperative operation is obtained and the mission/route can be planned. Therefore, the aiming targets calculation and the launch guidance of the weapon can be completed to implement the final attack. The operational process is shown as follows: (1) After data binding of the mission/route task, the MAV/UAV formation will be guided to the operational zone by the ship command center. (2) The battlefield surveillance, the reconnaissance and the target detection missions are performed by the UAV in the operational zone. The battlefield situation information is transmitted from the UAV to the MAV through the data link. The UAV is controlled by MAV to complete the target search. At the same time, real-time battlefield information is received by the MAV, which is transmitted by the communication system of ship command center. (3) The MAVs will merge the received information to complete the evaluation of the battlefield situation and the threat. The mission/route planning and the re-planning are completed for the UAVs according to the battlefield situation changes. (4) The MAVs assign tasks based on the battlefield information to the UAVs. After receiving the attack command, the UAVs start to carry out fire control calculation and weapon management, and finally carry out the attack. (5) The battlefield is monitored by the UAVs during the attack. The picture of the attacked target is taken by the airborne television or camera system. The attack effect is analyzed and evaluated. The overall attack effect and the damage effect of self-side are analyzed and evaluated by the MAVs. (6) According to the damage evaluation, if the target is not destroyed, the MAVs will decide whether to re-attack based on the battlefield information and the survival status of the UAVs. If the target is destroyed, the aircrafts will return to the ship according to the predetermined route. (2) The construction of the index system Based on the design criteria of MAV/UAV formation cooperative attack on the sea including operational process, influencing factors and evaluation indexes, the effectiveness

Classical Method of Effectiveness Evaluation

167

evaluation indexes are divided into four layers, as is shown in Figure 5.16.

Figure 5.16. The Effectiveness Evaluation Index System of the MAV/UAV Formation Cooperative Attack on the Sea. (1) The overall effectiveness layer index. It reflects the effectiveness of the MAV/UAV formation cooperative attack on the sea comprehensively. (2) The operational process layer index. It indicates that the effectiveness of the MAV/UAV formation changes with the advancement of the operational process. It is a measurement of completing the sea attack mission of the UAV/UAV formation in different operational stages. (3) The local effectiveness layer index. It is a sub-index of the operational process layer and the refinement and decomposition for the operational process layer index. (4) The equipment performance layer index. It is a quantitative description of the important behavioral property of the system, which is related to the system’s physical, structural and operational skills, etc. (3) Index Analysis (i) The availability vector A For the MAV/UAV system, it can be divided into the available and the unavailable state. However, due to the parallel relationship of the UAV system, there are three states for the MAV/UAV formation system. Therefore, the availability vector of system A is determined,

168

Deping Zhang and Xuefeng Yan

as is shown in the following. A = [a1 a2 a3 ]

(5.103)

where, a1 is the probability that the MAV system and two UAV systems work correctly. a2 is the probability that the MAV system and one UAV system work normally while other UAV system fails. a3 is the probability that the MAV fails or two UAVs fail, or the MAV and UAVs all fail. Assumed that MT BF1 and MT BF2 respectively represent the average failure probability time of MAV and UAV. MT T R1 and MT T R2 are the average failure repair time of MAV and UAV. λ1 and λ2 represent the failure probability before the mission execution respectively. µ1 and µ2 are the repair probability respectively. Then the probabilities of the MAV/UAV in correct working state can be obtained respectively, which is shown as follows. λ1 MT BF1 = MT BF1 + MT T R1 λ1 + µ1 MT BF2 λ2 P(U) = = MT BF2 + MT T R2 λ2 + µ2

P(M) =

(5.104) (5.105)

According to the three states of the MAV/UAV formation system, the availability vector A is obtained, which is shown as follows. λ1 λ2 ·( )2 λ1 + µ 1 λ2 + µ 2 2λ1 λ2 µ2 = 2P(M)P(U)(1 − P(U)) = · · λ1 + µ 1 λ2 + µ 2 λ2 + µ 2 = 1 − a1 − a2

a1 = P(M) × P(U)2 =

(5.106)

a2

(5.107)

a3

(5.108)

(ii) The dependability matrix D The MAV/UAV formation system only has two states of normal and failure during the mission execution. According to the possible system state transition from initial state, its dependability matrix is:   d11 d12 d13 D = d21 d22 d23  (5.109) d31 d32 d33

where, di j (i = 1, 2, 3; j = 1, 2, 3) means that the formation system is at state ai (i = 1, 2, 3) at the beginning. The formation system is at state a j ( j = 1, 2, 3) when the mission is completed. Obviously, it is impossible to repair the MAV/UAV formation system during the mission execution if there is a failure. Therefore, if an operational platform system fails at the beginning of a mission, the system will still be in the failure state even the mission is completed. Then the d21 = d31 = d32 = 0, d33 = 1 can be manifestly obtained.

Classical Method of Effectiveness Evaluation

169

The failure distribution is assumed to be exponential distribution, the dependability matrix of the MAV/UAV formation system is shown as follows.
 −λ T −2λ T  e 1 1 2 2 2e−λ1 T1 1 − e−λ2 T2 1 − 2eλ1 T1 + e−λ1 T1 −2λ2
T2 D= (5.110) 0 e−λ1 T1 −λ2 T2 1 − e−λ1 T1 1 − e−λ2 T2  0 0 1

where, λ1 , λ2 are the failure probability of the MAV and UAV formation system respectively. T1 and T2 are the working time of the mission execution of both respectively. (iii) The capability matrix C

The capability of the MAV/UAV cooperative attack on the sea is the degree to which the formation system can complete the attack mission against maritime target. It is generally expressed by the damage probability of the attacked target. This probability is closely related to the formation system state during the cooperative operation. In the same MAV/UAV formation system, if the system is in different states, the probability of completing the attack mission is different as well. In this section, the capability matrix C = [c1 c2 · · · c j · · · cn ]T can be constructed for the MAV/UAV cooperative attack on the sea, where c j is the energy value that the system completes the mission in the j state. During the execution of the attack mission, the MAV/UAV formation may be in three states, the MAV/UAV system works normally, one MAV system or one UAV system work normally, the MAV system fails or two UAV systems fail. The corresponding operational capability matrix is C = [c1 c2 c3]T . Obviously, when the formation system is in the third state, c3 = 0. According to the effectiveness evaluation index system of the MAV/UAV formation, the operational capabilities is mainly composed by the early warning detection Cd , the mission planning Cp , the cooperative command Cc and the cooperative attack Ca . The operational capability matrix of the MAV/UAV formation can be expressed as follows.     c1 Cd1 ·Cp1 ·Cc1 ·Ca1 C = c2  = Cd2 ·Cp2 ·Cc2 ·Ca2  (5.111) c3 0

where, the early warning detection, the mission planning, the coordinated command and the coordinated attack capabilities are represented by Cd1 ,Cp1 ,Cc1 and Ca1 respectively under the first state of the MAV/UAV formation. And these capabilities are represented by Cd2 ,Cp2 ,Cc2 ,Ca2 under the second state. There are three cases for completing the MAV/UAV cooperative attack mission. In case 1, the MAV system and the two UAV systems are in normal working state. In case 2, the MAV system and UAV1 are in normal working state, while UAV2 is in failure state. In case 3, the MAV system and the UAV2 are in normal working state, while UAV1 is in failure state. Without loss of generality, it is assumed that two UAVs are the same, the relationship between the capabilities of each stage can be calculated in these three cases, as is shown below. Cd1 = 1 − (1 −Cd2 )2 , Cp1 = 1 − (1 −Cp2 )2 Cc1 = 1 − (1 −Cc2 )2 , Ca1 = 1 − (1 −Ca2 )2

(5.112) (5.113)

170

Deping Zhang and Xuefeng Yan

In different sea conditions, the MAV/UAV cooperative formation attacks against large, medium, and small targets on the sea, corresponding mathematical model of the local efficiency indexes level can be built based on simulation data. The indexes can be calculated by the simulation data of the leaf index of equipment performance layer and can be synthesized by index aggregation after the lower index is quantized by utility function and fuzzy mathematics. The operational process layer index is calculated by the index aggregation or construction of the mathematical model. Therefore, the value of early warning and detection capability, mission planning capability, cooperative command capability and cooperative attack capability under the two conditions are obtained. Finally, the comprehensive effectiveness evaluation of the MAV/UAV cooperative attack capability on the sea is obtained based on the ADC. (iv) Case study It is assumed that the average time between the MAV failures is 1000h and the average repair time is 50h. The average time between the UAV failures is 2000h and the average repair time is 100h. Under the class III sea state, the mission time for the medium-sized maritime target is 6min. The corresponding simulation model is constructed to simulate the operational mission that the MAV/UAV formation attacks a medium-sized maritime target with one MAV and one UAV cooperation (that is, under the condition of a2 ). The relevant performance index data are collected and the local effectiveness layer index is calculated. After several rounds of expert polling, the AHP is used to obtain the weight of index in the local effectiveness layer corresponding to the index in the operational process layer, as is shown in Table 5.22. Table 5.22. The capability index and the weight of One MAV and One UAV Formation cooperative attack on the sea The operational process effectiveness index Detection and early warning capability Cd Mission planning capability C p Cooperative command capability Cc Cooperative attack capability Ca

The local effectiveness index Target discovery capability x11 Target recognition capability x12 Target selection capability x21 Target allocation capability x22 Route planning capability x23 Planning algorithm effectiveness x24 Cooperative decision capability x31 Situation renewal capability x32 Survivability x41 Flight performance x42 Damage probability x43 Hit target probability x44

Index value 0.9345 0.9830 0.9401 0.9712 0.9667 0.9843 0.9348 0.9403 0.9322 0.9501 0.9763 0.9511

Weight 0.64 0.36 0.24 0.23 0.34 0.19 0.33 0.67 0.27 0.18 0.21 0.34

Classical Method of Effectiveness Evaluation

171

The calculation steps of the MAV/UAV formation effectiveness are shown as follows: (1) Calculate the availability matrix of the formation system. P(M) = P(U) =

1000 = 0.95 1000 + 50 2000 = 0.95 2000 + 100

(5.114) (5.115)

Therefore, the availability vector A can be obtained, which is shown as follows: A = [a1 , a2 , a3 ] = [0.86, 0.09,0.05]

(5.116)

(2) Calculate the dependability matrix of the formation system. Before the mission execution, the failure rate λ1 , λ2 of MAV and UAV are shown as follows: 50 = 0.05 1000 + 50 100 λ2 = = 0.05 2000 + 100 λ1 =

(5.117) (5.118)

Therefore, the dependability matrix can be obtained, which is shown as follows:   0.9851 0.0099 0.0050 D= 0 0.9900 0.0100 (5.119) 0 0 1 (3) Calculate the capability matrix of the cooperative attack on the sea. According to the preset sea situation and the target to be attacked, the data of the single UAV operational system are obtained by simulation (as is shown in Table 5.22). The weighted sum is adopted by the AHP to aggregate the index. And then the following can be obtained: [Cd2,Cp2 ,Cc2 ,Ca2 ] = [0.9520, 0.9647, 0.9385, 0.9511]

(5.120)

According to the Equation (5.111), the operational capability data can be obtained under the normal working state of the MAV/UAV formation system, which is shown as follows: C = [c1 , c2 , c3 ]T = [0.9903, 0.8198, 0]T

(5.121)

It can be further obtained that E = A · D ·C = 0.9227.

5.5. Comprehensive Effectiveness Evaluation Based on Cloud Model The physical world is a multi-parameter, nonlinear, and time-varying unstable system. The basic elements such as things, concepts, entities, or phenomena in the system have multiple uncertainties at the same time. It is common for fuzziness and randomness to appear

172

Deping Zhang and Xuefeng Yan

simultaneously. Therefore, it is necessary to study the randomness and fuzziness to explore the uncertainty of things. The effectiveness evaluation of equipment systems also has these problems, such as the fuzziness of each evaluation index. For qualitative indexes, the definition of each index is fuzziness in nature and not completely accurate. For quantitative indexes, the determination of each evaluation is usually affected by subjective and objective factors and also has certain fuzziness. The effectiveness is affected by a large number of factors and each factor may not play decisive role, so the evaluation index also has a certain randomness. In order to solve the problems of fuzziness and randomness, the uncertainty conversion model of qualitative and quantitative analysis (the cloud model) can be adopted [163]-[166]. Based on the combination of qualitative and quantitative analysis, it organically integrates the randomness and fuzziness of qualitative concepts in natural language. Therefore, the natural transformation between qualitative linguistic values and quantitative values is realized to provide systematic and high-level tools for description of uncertainty.

5.5.1.

Basic Theory of Cloud Model

(1) The Concept of Cloud The cloud is an uncertain transformation model between a certain qualitative concept described by the linguistic value and its numerical representation. In short, the cloud model is an uncertainty model of qualitative and quantitative inter-conversion. The definition of the cloud model is shown as follows: It is assumed that U is the quantitative domain that is represented by a value. C is the qualitative concept on U. If x ∈ U is specified as a random implementation of qualitative concept C, the certainty µ(x) ∈ [0, 1] of x to C is a random number µ : U → [0, 1], ∀x ∈ U, x → µ(x) with a stable tendency. Therefore, the distribution of x on the universe is called as the cloud, and represented by C(x). Each x is called as a cloud droplet. The cloud is composed of a large number of droplets. The important characteristics of qualitative concepts is reflected by the overall shape of the cloud. The cloud droplet is a quantitative description of qualitative concept.Thecloud droplet generation process represents the uncertain mapping between qualitative concepts and quantitative values. According to the dimension of the universe U, the cloud can be one-dimensional, two-dimensional, multidimensional and so on. The cloud model has the following properties: (1) The mapping of each x ∈ U to the interval [0, 1] is the one-to-many conversion. The membership of x to C is a probability distribution rather than a fixed value. Therefore, the cloud rather than a clear membership curve is generated. (2) The cloud is composed of a large number of droplets. The cloud droplet is a quantitative realization of the qualitative concept. A single cloud droplet may be insignificant. The details of clouds generated at different times may be different. However, the basic characteristics of qualitative concept are reflected by the overall shape of the cloud. (3) The unordered cloud droplets are generated by the cloud model, which is the realization of the qualitative concept. That is, the more the cloud droplets are, the more it can reflect the overall situation of the qualitative concept. At the same time, the “Gaussian cloud distribution” is formed.

Classical Method of Effectiveness Evaluation

173

(4) The mathematical expectation curve of a cloud can be regarded as its membership curve from the view of fuzzy set theory. (5) The “thickness” of the cloud is uneven. The middle part is the most dispersed and its “thickness” is the largest, while the top and bottom are convergent and the “thickness” is small. The randomness of the membership is reflected by the “thickness” of the cloud. The greater the cloud droplet membership degree is, the greater the cloud droplet occurrence probability is and the greater the contribution to the concept is. When it is close to or far from the concept center, the membership randomness is small. When the location is neither too far nor too close from the concept center, the membership randomness is large. These are consistent with human subjective feelings. The Figure 5.17 shows the membership cloud with the linguistic value of “about 30 meters”. The geometry of the cloud is very helpful for understanding the uncertainty of the transformation between qualitative and quantitative analysis.

Figure 5.17. The membership cloud with the linguistic value of “about 30 meters”. (2) The Digital Features of the Cloud The digital features of the cloud are represented by three values including the expectation Ex, the entropy En, and the hyper entropy He. The fuzziness and randomness are completely integrated to form a mapping between qualitative and quantitative. The digital features of the cloud are the numerical basis for describing the cloud models, generating the virtual clouds, implementing the cloud computing and transformations, which is shown in Figure 5.18. The expectation Ex: Generally, the distribution expectation is best points which can represent the qualitative concept. The quantitative data can be transformed to qualitative data based on the index. The position of the cloud in the universe is marked with Ex and this is the gravity center of the cloud. The Entropy En is the uncertainty measurement of qualitative concepts. It is used to

174

Deping Zhang and Xuefeng Yan

Figure 5.18. The features of the cloud. measure the ambiguity and probability in the cloud comprehensively, which reflects the uncertainty. En is a measurement of the randomness, which reflects the discreteness of the cloud that can represent this qualitative concept. The acceptable cloud droplet range is reflected by En in the domain, called ambiguity. The correlation between randomness and fuzziness is represented by the same digital feature in the cloud. The hyper entropy He: It is the uncertainty measurement of entropy, called the entropy of entropy. It is an important index to form the concept of sample data. The thickness of the cloud is indirectly reflected by the value of the hyper entropy. The greater the hyper entropy is, the greater the cloud droplet dispersion is, the greater the randomness of membership is and the greater the cloud thickness is. For example, the qualitative linguistic value of “the speed of the vehicle” is described by the concept of cloud, as is shown in Figure 5.19. Generally, the standard range is 40 ∼ 60 km/h and it is indisputable that 50km/h is the “standard”. According to probability and statistics, in the equation Ex = 50, En = 2, the left and right 3En range of Ex should cover 99% elements that can be accepted in the concept. He can be roughly assigned to 0.2. The cloud comparison for different parameters is shown in Figure 5.20. The basic features of the cloud can be summarized from the three numerical characteristics. Combined with the mapping between qualitative variables and quantitative variables in the effectiveness evaluation process, the concept of cloud can be re- as follows: (1) In the cloud model, the membership of each element is a random number that follows the normal distribution, rather than a unique value. The uncertainty of the mapping between qualitative and quantitative variables is reflected by this characteristic. That is, the mapping of the concept C from the domain U to the membership interval [0, 1] is a one-to-

Classical Method of Effectiveness Evaluation

175

Figure 5.19. The schematic diagram of the cloud. many relationship, rather than the one-to-one relationship of traditional fuzzy membership function for a specific qualitative variable, in the effectiveness evaluation a certain random margin is contained in the corresponding quantitative mapping. The inherent “softness” of the qualitative variables is maintained by the cloud definition. (2) Although the mapping of each element is always changing slightly, these changes are only a flutter within a certain range and will not affect the overall characteristics of the cloud. The true meaning of the mapping from qualitative variables to quantitative variables is reflected by the overall shape of the cloud drastically. In the effectiveness evaluation process, the characteristic of cloud can be understood as the expert consensus of this qualitative variable after multiple evaluations and transformations. (3) In the generation of cloud, if the number of droplets is too small, the overall shape and cohesion characteristics of the cloud is not clear. With the increase of droplets number, the overall shape of the cloud gradually becomes clear and the aggregation degree of the droplets near the expected value is higher. According to this characteristic, it is meaningless to discuss the membership of single element. The fuzziness and randomness of the mapping can be reflected by the distribution characteristic of a large number of droplets. In the effectiveness evaluation, the accuracy of mapping results can be guaranteed by multiple samplings, which include multiple experts’ assignment of a qualitative variable and multiple simulations. Therefore, the credibility of the results can be assured. (4) The thickness of the cloud is uneven. The top and bottom are the narrowest and the middle is the thickest. The randomness of the membership is reflected by the “thickness”. If it is close to or far from the concept center, the randomness of membership will be relatively small. If the location is neither too far nor too close from the concept center, the randomness of membership will be relatively large. This characteristic reflects the randomness that the understanding of qualitative variables. However, the whole evaluation process shows certain overall regularity. In conclusion, the fuzziness and randomness in qualitative and quantitative mapping

176

Deping Zhang and Xuefeng Yan

Figure 5.20. The comparison for different cloud parameters. are effectively integrated by the cloud to study the universal laws of uncertainty, It possible to obtain the range and distribution of quantitative data from the qualitative information expressed by linguistic values, and also possible to convert the precise data into the appropriate qualitative linguistic values effectively. Therefore, the cloud model is used to study the transformation of the qualitative and quantitative variables in the effectiveness evaluation of the command information system. (3) Gaussian Cloud and Comprehensive Cloud Different types of cloud models are implemented based on different probability density distribution functions, such as normal cloud model, triangular cloud model and trapezoidal cloud model. The normal cloud model evolved from Gaussian distribution is very suitable to express relative concepts. The definition of the normal cloud model is shown as follows: It is assumed that an exact value represents the quantitative domain U and qualitative concept C in the U. For x ∈ U, and x is a random implementation of C. If x satisfies 0 0 x ∼ N(Ex, En 2 ), En 2 ∼ N(En, He2 ), the membership of x to C will satisfy:   (x − Ex)2 µ = exp − (5.122) 0 2(En )2 Then the distribution of x in the universe is called the normal cloud (or the Gaussian cloud). Equation (5.122) indicates that the quantitative values of the cloud droplets can be determined by the standard normal distribution function. The certainty degree of the cloud droplets can be calculated by the normal fuzzy membership. Therefore, the normal cloud has both fuzziness and randomness features. The cloud model can describe the qualitative concepts that can be expressed through a large number of quantitative concept values and their certainty degree. The transformation between qualitative concepts and quantitative values can be realized based on the cloud generation algorithm. The qualitative concepts

Classical Method of Effectiveness Evaluation

177

will be transformed into quantitative values using normal cloud generation algorithm, that is, the required number of droplets will be generated based on the three known numerical features. The algorithm of cloud model is called cloud generator, which establishes the relationship between the qualitative values and the quantitative. There are the normal generator, the reverse generator, and the index approximation method and so on. The cloud droplet drop(xi , µi ) is several two-dimensional points is produced by the normal generator based on 3 numerical characteristics of the cloud. The normal cloud generation algorithm can be described as follows: (1) Generate a normal random number xi ∼ N(Ex, En) with the expectation Ex and the standard deviation En; 0 (2) Generate a normal random number Eni ∼ N(Ex, He) with the expectation Ex and the standard deviation En are generated; h i 0 (3) Calculate the membership µi = exp −(xi − Ex)2 /2(En )2 , (xi , µi ). The normal cloud is composed by the cloud droplet drop(xi , µi). (4) Repeat steps 1)∼ 3) until the desired cloud is formed. For example, if the cloud C(0.8, 0.15,0.005) with the “good” qualitative evaluation, 1000 cloud droplets are generated, as is shown in Figure 5.21.

Figure 5.21. The membership cloud with the “Good” qualitative evaluation. The cloud droplet distribution is obtained by the normal cloud model through the generation algorithm and approximately follows the Gaussian distribution. It has the feature of “two ends are small, the middle is large” and “pike apex and thick trail”. These can be used to describe the “28 law” of the power-law. Therefore, the normal cloud model is an important and universal cloud model. If the random variables of the cloud model follows

178

Deping Zhang and Xuefeng Yan

the Gaussian distribution, the corresponding relationship between the cloud model and the related concepts in statistical theory can be established, as is shown in Table 5.23. Table 5.23. The corresponding relationship between cloud model and concepts in statistical theory Cloud model Cloud droplet Cloud model Expectation (Ex) Entropy (En) Hyper entropy (He) Reverse cloud generation algorithm Normal cloud generation algorithm

Concepts in statistical theory Random variable Probability measure space Posterior estimate expectation Variance in likelihood distribution Super-parameter Point estimation method (Pseudo) Random number generation algorithm

However, if quantitative values need to be transformed into qualitative concepts, reverse cloud generation algorithm can be used, namely, the reverse process of normal cloud generation algorithm. According to the reverse cloud generation algorithm, the three numerical characteristics are obtained from a given cloud droplet sample. Therefore, the qualitative evaluation of the sample data can be achieved. The observed value of a certain concept can be obtained from finite limited samples according to the idea of inferring the whole from some samples in statistical theory. In the reverse cloud generation algorithm, the estimated values of the numerical characteristics of the cloud model can be calculated by the cloud droplet sample {xi }nI=1 . The idea of the reverse cloud generation algorithm is shown as follows: Step 1. The cloud droplets {x1 , x2 , · · · , xn } is used for calculation 1 n Sample mean X¯ = ∑ xi n i=1 First-order sample absolute central distance m = Sample variance S2 =

1 n ∑ (xi − X¯ )2 n − 1 i=1

(5.123) 1 n ¯ ∑ |xi − X| n i=1

(5.124) (5.125)

Step 2. Calculate the estimate of the expectation, the entropy, and the hyper entropy ˆ = X¯ Ex r π 1 n ˆ ˆ En = × ∑ |xi − Ex| 2 n i=1 q ˆ 2| ˆ |S2 − En He =

(5.126) (5.127) (5.128)

ˆ En, ˆ He) ˆ of cloud C(Ex, En, He) can be According to this idea, the point estimate C(Ex, obtained based on the samples.

179

Classical Method of Effectiveness Evaluation

The index approximation method is suitable for the case where the boundary is known. For the index with bilateral constraint [Cmax ,Cmin], the calculation steps are shown as follows: 1 (Cmax +Cmin ) 2 1 (Cmax −Cmin ) En = 6 He = k Ex =

(5.129) (5.130) (5.131)

where k is a constant and can be adjusted based on the randomness of the comment. According to Equation (5.122), a normal cloud composed of any number of cloud droplets can be generated based on three numeric characteristics (Ex, En, He). The comprehensive cloud model and the combined cloud model can also be generated. The comprehensive cloud model refers to the integrative calculation of two or more cloud models, so as to evolve into a higher-level cloud and become the linguistic value with broader concept. Its evaluation set is obtained by comprehensive calculation of all subsets. Based on the operator adapted by the normal cloud, four arithmetic operators are carried out on the numerical characteristics of a given cloud. And then the cloud can be constructed by the new numerical characteristics. It is assumed that there are two clouds C1 (Ex1 , En1 , He1 ) and C2 (Ex2 , En2 , He2 ), λ ∈ N. The algebraic operator of the cloud model is shown as follows: 1) The addition and subtraction operator q q C1 ±C2 = C(Ex1 ± Ex2 , En21 + En22 , He21 + He22 )

(5.132)

2) The scalar multiplication operator

√ √ λC1 (Ex1 , En1 , He1 ) = C(λEx1 , λEn1 , λHe1 ) 3) The multiplication operator  C1 ×C2

= C Ex1 Ex2 , |Ex1 Ex2 | s

×



4) The division operator C1 C2

He1 Ex1

2



s 

He2 + Ex2

2

En1 Ex1 

2



En2 + Ex2

2

(5.133)

, |Ex1 Ex2 |



(5.134)



s    Ex1 Ex En1 2 En2 2 Ex1 1  = C , + , Ex2 Ex2 Ex1 Ex2 Ex2  s  2  2 He1 He2  × + Ex1 Ex2

(5.135)

180

Deping Zhang and Xuefeng Yan 5) The power operator √ √ C1λ = C(Exλ1 , λExλ1 1 En1 , λEx1λ−1 He1 )

(5.136)

However, more than two clouds with the same type Ci (Exi , Eni , Hei ), (i = 1, 2, · · ·) will be generated in the same universe. After clustering, a comprehensive cloud will be generated. Actually, the essence of the comprehensive cloud is to integrate the multiple qualitative concepts into a broader concept. After the integration of the several clouds, the numerical characteristics of the weighted average comprehensive cloud model can be expressed as follows. CWAAw(C1 ,C2 , · · · ,Cn ) = C(Ex, En, He) (5.137) r r n n n where, Ex = ∑ wi Exi , En = ∑ wi En2i , He = ∑ wi He2i , wi represents the weight, i=1 n

which satisfies ∑ wi = 1. If wi = i=1

i=1

1 n,i

CWAAw(C1 ,C2 , · · · ,Cn ) = C

5.5.2.

i=1

= 1, 2, · · · , n, then 1 n 1 Exi , ∑ n i=1 n

s

n

1 n ∑ En2i , n i=1

s

n

n ∑ He2i i=1

!

(5.138)

Effectiveness Evaluation Based on Cloud Model

The effectiveness evaluation based on the cloud model is to describe qualitative indexes by using the cloud model. According to the hierarchical structure of the system index and the relevant knowledge of cloud theory, the gravity center of multi-dimensional weighted comprehensive cloud can be derived for each index. The weighted deviation degree is used to measure the change of cloud gravity center and to activate the cloud generator. Therefore, the evaluation value of the object can be obtained and the system effectiveness can be comprehensively evaluated. The comprehensive evaluation method based on cloud model has three elements, including index set U, weight set W , and evaluation set V . 1) The index set can be represented by U0 ,U1 , · · · ,Um, in which U0 is the target index and the remaining are the sub-index i. 2) The weight set can be represented by W = (w1 , w2 , · · · , wm ), which satisfies wi ≥ 0 m

and ∑ wi = 1; i=1

3) The evaluation set can be represented by V = (V1 ,V2, · · · ,Vm). The evaluation index can be divided into multi-level hierarchical structures if it is needed. Based on the cloud model, the gravity center is used in the evaluation, starting from the layer n of the index hierarchy. And the evaluation results are transferred to the (n − 1) layer. Then the evaluation is carried out layer by layer in turn until the result of the specified layer is obtained. The specific steps can be described as follows: Step 1. Determine the index set and the weight of each index The index system is a set of related indexes that reflect the various attributes of the evaluated object. However, an index system can only satisfy the characteristics of one aspect of the object needed to be evaluated. Therefore, in order to obtain reasonable results, it is

Classical Method of Effectiveness Evaluation

181

necessary to make clear the purpose of evaluation and then establish corresponding index system based on certain principles. It is important and difficult to select an appropriate index system. The index weight is the quantitative importance of an index. It is an important part of the index system and also known as weight value and weight coefficient. The status and importance of each evaluation index are different. In order to reflect these differences, each evaluation index should be set a weight so as to be objective and comparable. Step 2. Represent the indexes using the cloud model In the effectiveness index system, there are precise numerical representation and linguistic value description. The decision matrix can be formed by the n samples. Then n precise numerical indexes can be represented by a cloud model. Ex = En =

1 n ∑ xi n i=1 1 [max(Ex1 , Ex2 , · · · , Exn ) − min(Ex1 , Ex2 , · · · , Exn )] 6

(5.139) (5.140)

At the same time, the index of each linguistic value can be represented by a cloud model. Therefore, an index is represented by n linguistic values (cloud model), which can be also represented by a one-dimensional comprehensive cloud. Ex1 En1 + Ex2 En2 + · · · + Exn Enn En1 + En2 + · · · + Enn En = En1 + En2 + · · · + Enn Ex =

(5.141) (5.142)

When the index is a precise numerical value, Ex1 ∼ Exn is the value of each index. When the index is a linguistic value, Ex1 ∼ Exn is the expectation of the index cloud model and En1 ∼ Enn is the entropy [34]. Step 3. Represent the system status As n performance indexes can be described by n cloud models, the system status reflected by n indexes can be represented by the n dimensional comprehensive cloud. When the system status changes, the cloud shape will change and also its gravity center. That is, the change of the system status information can be reflected by the change of the cloud gravity center. The gravity center T of n dimensional comprehensive cloud is represented by n dimensional vector, that is, T = (T1 , T2 , · · · , Tn ), where Ti = (ai × bi ) (i = 1, 2, · · · , n), a is the position of the cloud gravity center. The information center of the corresponding fuzzy concept is reflected by the expectation value. b is the height of the cloud gravity center, which is also called the weight. The importance of the corresponding cloud is reflected by the height of the cloud gravity center. When the system changes, the change of 0 0 0 0 the gravity center can be described as T = (T1 , T2 , · · · , Tn ).

182

Deping Zhang and Xuefeng Yan

Step 4. Measure the change of cloud gravity center based on the weighted deviation degree Each index value is known in the ideal state of a system. It is assumed that the position vector and the height of the n dimensional comprehensive cloud gravity center are represented by a = (Ex01 , Ex02 , · · · , Ex0n ) and b = (b1 , b2 , · · · , bn ) respectively. Then in the ideal state, the vector of the cloud gravity center is T 0 = a × b = (T10 , T20 , · · ·Tn0 ). Similarly, the vector T = (T1 , T2 , · · · , Tn ) of the n-dimensional comprehensive cloud gravity center can be calculated in a certain state for the system. The weighted deviation degree θ can be used to measure the difference of the comprehensive cloud gravity center. Firstly, the vector of the comprehensive cloud gravity center is normalized in this state and the vectors T G = (T1G , T2G , · · · , TnG ) are obtained.   Ti −Ti0 , Ti < T 0 i G Ti0 (5.143) Ti =  Ti −Ti0 , T ≥ T 0 Ti

i

i

After normalization, the vectors of the comprehensive cloud gravity center that represent the system state are the values with size, direction and dimensionless. The weighted n

deviation θ, θ = ∑ w∗i TiG is obtained by multiplying the normalized vector weight of each i=1

index and then adding them. w∗i is the weight of the single index i. The effectiveness can be obtained by comparing the ideal state with the result gotten by input θ into the cloud generator. Step 5. Get evaluation comment set based on the cloud model The more comments in the comment set is, the more accurate the result is. Here, one comment set is composed of eleven comments, such as extremely poor, very poor, large poor, poor, little poor, normal, good, better, very good, best and excellent. The eleven comments are listed on the continuous linguistic value scale and each comment is implemented by a cloud model. Therefore, a cloud generator with qualitative evaluation can be constituted, as is shown in Figure 5.22. For a specific scheme, the calculated θ is input into the evaluated cloud generator, which may have two activation situations. Firstly, if the degree of an activated cloud object with a comment value much greater than the others, the comment value can be regarded as an evaluation result for the scheme. Secondly, if the cloud object with two comment values is activated and the activation degree is not very different, a new cloud object is need to be generated using the comprehensive cloud principle. Its expectation will be output as the quantitative evaluation result. Then, the qualitative expression of this expectation can be given by experts or users additionally. For example, if the result is 0.62 for an index, it reaches the degree of “good”. If the result is 0.65, it is between good and better. Then the qualitatively expression can be given additionally by experts or users. The effectiveness evaluation process based on cloud model for the SoS can be shown in Figure 5.23. (1) Determination of the Evaluation Purpose The overall effectiveness is taken as the starting point. The main factors that influence

Classical Method of Effectiveness Evaluation

183

Figure 5.22. The cloud generator with qualitative evaluation. the overall effectiveness of the evaluation object are comprehensively taken into consideration. Then the evaluation purpose can be determined. (2) Analysis of the Effectiveness Factors According to the determined evaluation purpose, the main influencing factors for the evaluation effectiveness are analyzed and summarized. Several important effectiveness factors and their correlative relationships are extracted. These extracted elements will be regarded as the main basis for constructing the evaluation index system. (3) Construction of the Evaluation Index System The key effectiveness factors are taken as the benchmark with the assistance of the evaluation system and the index system framework. A suitable evaluation index system is constructed according to the relevant principles. The factors should be representative and indispensable, and should be as comprehensive and concise as possible. (4) Calculation of the State Value of Each Index There are several ways to obtain the evaluation data. Some can be summarized from previous examples, some can be obtained by expert consultation and other methods, and some index data can be obtained by simulation. (5) Evaluation of the Single Effectiveness Each index is represented by the cloud model. And the expectation and entropy of the cloud model are calculated. The single effectiveness of the system is evaluated by the changes of the gravity center. (6) Evaluation of the Comprehensive Effectiveness The final analysis and evaluation is carried out comprehensively and systematically based on the single evaluation. The synthesized evaluation results can be used as a measure for the evaluation purpose.

184

Deping Zhang and Xuefeng Yan

Figure 5.23. The flow chart of the effectiveness evaluation based on cloud model. (7) Output of the Evaluation Result The evaluation results are output to the user in character, language, or chart forms.

5.5.3.

Case Study

It is assumed that the relevant information about blue naval targets has been obtained through reconnaissance. The carrier-based UAV formations are used to conduct sea assault operations. Three types of UAV formation schemes A, B,C can be used. The cloud evaluation model is used to simulate and analyze the effectiveness to illustrate comprehensive effectiveness evaluation based on cloud model. (1) Influencing Factors of the Effectiveness Evaluation The carrier-based UAV formation has the following characteristics for sea assault operations. The operation base is the depot ship. The UAV requirements are high for the launch, recovery, and mission planning. At the same time, the operational activities are affected by natural conditions such as the depot ship, hydrology, weather, and geographical environment. In addition, the wide region of naval operations will increase the difficulty for target search and selection. The carrier-based UAV formations can be divided to different see assault phase. Therefore, in the construction of the effectiveness evaluation index system, the stage division is not only help to the traceability analysis of influencing factors, but can also help to further analyze the operational process, the operational effect and single capability of the carrierbased UAV formation in each stage. The sea assault processes can be divided into five stages and the details are described as follows:

Classical Method of Effectiveness Evaluation

185

1) Battlefield preparation stage. The aviation command post receives the operational missions issued by the aircraft carrier formation. It formulates and reports the operational plan. After getting the permission, it allocates the mission planning to the carrier-based UAV, including the target allocation, the route planning and so on. The carrier-based UAV is ready to take off. 2) Flying to battlefield stage. The carrier-based UAV takes off from the carrier, flies to the operational sea area according to the planned route. 3) Penetration stage. When the UAV approaches the planned area, if the weapons can not be launched outside the blue defense region, it is necessary to penetrate the blue air defense system before reaching the launch position. 4) Naval assault. After the carrier-based UAV formation approaches the target area, it searches the targets immediately. The attack should be triggered as early and far as possible. After the launch, the decision is made based on the damage results. When additional strikes are needed, it should be executed. 5) Return stage. The formation flies away from the operational area, and returns and lands according to the operational plan or under the guidance of the early warning aircraft. (2) Construction of Index System Combined with the sea assault operation of the carrier-based UAV formation, in the construction pattern P2, the inherent capability and operational suitability of the effectiveness evaluation index system is selected. The comprehensive effectiveness evaluation index system can be constructed, as is shown in Figure 5.24.

Figure 5.24. The effectiveness evaluation index system of the UAV formation.

186

Deping Zhang and Xuefeng Yan (3) Determination Weight Cloud of the Index

According to the established effectiveness evaluation index system, the judgment matrix for each index set is constructed and the weight of each index can be calculated and converted into the weight cloud. First, according to the AHP and the expert evaluation, the judgment matrix for the index set can be constructed and the weight W of each index can be obtained. The mission planning capability of the UAV formation is taken as an example. Experts evaluate the four indexes of the lower layer to get the index judgment matrix. The index weight can be obtained after the consistency check. W3 = [w31 , w32 , w33 , w34 ]T = [0.44, 0.32, 0.11,0.13]T Similarly, the weights of all the index and criterion layers can be obtained.     0.23 0.44 0.21 0.12 W1 W2  0.28 0.16 0.38 0.18     W3  0.44 0.32 0.11 0.13     W = =  0.31 0.23 0.34 0.12 W 4     W5  0.19 0.11 0.35 0.35 0.21 0.21 0.26 0.32 W6

(5.144)

(5.145)

Compared with the “the effectiveness of the carrier-borne UAV formation sea assault” on the purpose layer, the weight of the six indexes in the criterion layer is shown as follows. W0 = [w1 , w2 , w3 , w4 , w5 , w6 ]T = [0.09, 0.09, 0.13,0.22, 0.34,0.13]T

(5.146)

The weight W of each index is obtained according to the evaluation by expert scoring. It is converted into the weight cloud Cw (Exw , Enw, Hew ) by using the following equation. W

= [W1,W2 , · · · ,Wi ]   Exw1 Enw1 Hew1 Exw2 Enw2 Hew2    =  . .. ..   .. . .  Exwi Enwi Hewi = Cw (Exw , Enw , Hew )T

(5.147)

(5.148)

(5.149)

where Exw is the weight. Enw = 1/3Ew, Ew is the maximum expectation deviation the expert evaluation. Hew represents the randomness of expert evaluation. The mission planning capability is taken as an example. Based on the expert evaluation for the UAV formation weight in the scheme A, the weight cloud can be calculated by the Equation (5.148).   0.44 0.04 0.005 0.32 0.04 0.005  C3w (Ex3w, En3w , He3w)T =  (5.150) 0.11 0.04 0.005 0.13 0.04 0.005

187

Classical Method of Effectiveness Evaluation

The following formula is used to convert the qualitative evaluation given by experts to the normal evaluation of each index in each scheme. The numerical characteristics of the cloud are expressed as Ce (Exe , Ene, Hee ). R = [R1 , R2 , · · · , Ri]   Exe1 Ene1 Hee1 Exe2 Ene2 Hee2    =  . .. ..   .. . .  Exei Enei Heei

(5.151)

(5.152)

= Ce (Exe, Ene , Hee)T

(5.153)

Its index cloud can be calculated by the Equation (5.148) based on its expert evaluation in the scheme A.   0.6 0.06 0.008 0.5 0.06 0.008  (5.154) CA3e(ExA3e , EnA3e, HeA3e )T =  0.3 0.06 0.008 0.4 0.06 0.008 Similarly, the evaluation clouds of the remaining indexes in the three UAV formation schemes of A, B and C can be obtained, as is shown in Table 5.24. Table 5.24. The weight cloud and the evaluation cloud at the criterion layer Index Flight performance

Weight Cloud (0.09, 0.04, 0.005) Operational (0.09, 0.04, suitability 0.005) Mission planning (0.13, 0.04, capability 0.005) Detection and early (0.22, 0.04, warning capability 0.005) Survivability (0.34, 0.04, 0.005) Cooperative Attack (0.13, 0.04, Ability 0.005)

Scheme A (0.582, 0.0526, 0.0065) (0.565, 0.0575, 0.0076) (0.509, 0.0505, 0.0065) (0.687, 0.0521, 0.0074) (0.662, 0.0568, 0.0075) (0.786, 0.0683, 0.0088)

Scheme B (0.512, 0.0632, 0.0072) (0.431, 0.0767, 0.0078) (0.367, 0.0683, .0087) (0.552, 0.0796, 0.0092) (0.571, 0.0825, 0.0097) (0.721, 0.0987, 0.0121)

Scheme C (0.435, 0.0835, 0.0166) (0.356, 0.0963, 0.0267) (0.213, 0.0886, 0.0183) (0.479, 0.0987, 0.0252) (0.431, 0.0979, 0.0238) (0.625, 0.1328, 0.0823)

(4) Determination of the Comprehensive Operational Effectiveness Cloud According to the operational rules of the normal cloud, the evaluation cloud of the criterion layer is obtained by using the index weight cloud and the evaluation cloud. Similarly, the comprehensive evaluation cloud of the target layer can be obtained to evaluate the ef-

188

Deping Zhang and Xuefeng Yan

fectiveness sea assault. C(Ex, En, He) = CwT Ce  T   Exw1 Enw1 Hew1 Exe1 Ene1 Hee1 Exw2 Enw2 Hew2  Exe2 Ene2 Hee2      =  . .. ..   .. .. ..   ..   . . . . .  Exwi Enwi Hewi Exei Enei Heei s s " i

=

i

i

∑ Exwk · Exek , ∑ (Enwk · Enek )2, ∑ (Hewk · Heek )2

k=1

k=1

k=1

(5.155)

(5.156) #T

(5.157)

The comprehensive evaluation cloud of the mission planning capability index of the UAV formation scheme A can be obtained by using the Equation (5.157), which is shown as follows. T CA3 (ExA3 , EnA3 , HeA3 ) = C3w ×CA3e = CA3 (0.509, 0.0505, 0.0065)

(5.158)

Similarly, as is shown in Table 5.24, the evaluation clouds of the remaining indexes in the three UAV formation schemes of A, B, and C can be obtained. After weighted fitting operation on the weight cloud and the evaluation cloud of each index in the criterion layer, the comprehensive operational effectiveness clouds of the three UAV formation schemes A, B, and C can be obtained by using the Equation (5.157), which are respectively shown in the following. CA (ExA , EnA , HeA ) = CA (0.6478, 0.0590, 0.0075) CB (ExA , EnA , HeA ) = CB (0.5419, 0.0583, 0.0071)

(5.159)

CC (ExA , EnA , HeA ) = CC (0.4321, 0.0581, 0.0154) The operational effectiveness cloud can be obtained by simulation. The operational effectiveness cloud of UAV formation scheme A is shown in Figure 5.25. The comparison diagram of the operational effectiveness cloud of three UAV formation schemes of A, B, C is shown in Figure 5.26.

5.6. Intelligent Effectiveness Evaluation Based on Machine Learning For many problems of effectiveness evaluation, simulation experiments are used to evaluate the influence of different performance indexes on system effectiveness. For example, in order to find the best scheme for a certain operational mission, simulation operations are often conducted for different weapon attack schemes etc. For many simulations, single simulation may be completed with minutes, hours, or even days. It is impossible to solve the original model directly, and often needs thousands or even tens of thousands of simulation. In order to improve this situation, the approximate model is used to replace highprecision models, which is also called the proxy model, the response surface model and

Classical Method of Effectiveness Evaluation

189

Figure 5.25. The operational effectiveness cloud of UAV formation scheme A.

Figure 5.26. The comparison of the operational effectiveness cloud of three schemes. the meta-model. The complex numerical calculations or physical experiments can be replaced by mathematical models meeting the requirement. It is a modeling technique which includes experimental designs and approximate algorithms. The approximate model can improve the efficiency of simulation and reduce the cost of algorithm. The results are very close to original model, but the computation quantity is small.

5.6.1.

Classical Machine Learning Algorithms

(1) The Back Propagation (BP) Neural Network The BP neural network algorithm is widely used in the effectiveness evaluation [167][172]. It was proposed in 1986 by a scientific research group led by Rumelhart and Mc-

190

Deping Zhang and Xuefeng Yan

Celland. It is a multi-layer feed forward network based on error back propagation algorithm. The three-layers BP neural network is the most commonly used one. A large number of input-output relationships are stored in the memory. It is unnecessary to construct the mathematical model of these relationships in advance. When the variables with the same characteristics are input, the output can be automatically obtained by using the existing corresponding relationships in the BP neural network. The BP neural network is trained by the gradient descent algorithm. Before the error can be accept, the training error are fed forward continuously and the weights and thresholds of the network elements are adjusted to effect the input-output relationships. The more accurate output of re-input variables can be calculated can be using the existing memory. Therefore, the BP neural network is also called the multi-layer feed forward network. The complex mathematical calculation model is replaced by the memory mapping in the BP neural network. Its most important advantage is that it can feed back the error of the network in real time and predict the future results by analyzing of the known data. The weights of the internal neurons are adjusted automatically in the whole process rather than adjusting the external structure. The basic structure of the BP neural network includes the input layer, the output layer and the hidden layer. The structure is flexible and diverse, which satisfies not only one input and multiple output, but the multiple input and multiple output. It can be applied flexibly according to the actual situation. The basic model is shown in Figure 5.27.

Figure 5.27. The structure of three-layers BP neural network. The BP is a neural network learning algorithm based on the δ learning rule. Its core operation is to continuously modify the network weight and threshold based on the back propagation of the prediction data error. And then the fast convergence of the model can be realized. The input vectors are defined as X = (x1 , x2 , · · · , xn )T . The output vectors of the hidden layer and the output layer are regarded as Y = (y1 , y2 , · · · , ym )T and O = (O1 , O2 , · · · , Om )T respectively. And the expected output index vectors are regarded as d = (d1 , d2 , · · · , dm )T . The unipolar/bipolar Sigmoid function is generally selected as the activation function

Classical Method of Effectiveness Evaluation

191

for the hidden layer. f (x) =

1 1 − e−x or f (x) = 1 + e−x 1 + e−x

(5.160)

The profile function (pureline transfer function) is selected for the output layer, That is y = x. The objective function is the error function between the actual output value and the expected output value. E = =

=

=

1 l 1 (d − O)2 = ∑ (di − Oi )2 2 2 i=1

(5.161)

" !#2 m 1 l 1 l 2 ∑ (dk − f (netk )) = 2 ∑ dk − f ∑ w jk y j 2 k=1 j=1 k=1 ( " #)2 1 l ∑ dk − f ∑ w jk f (net j) 2 k=1 j=0 " " !##2 m m 1 l ∑ dk − f ∑ w jk f ∑ vi j xi 2 k=1 j=0 i=0

(5.162)

(5.163)

(5.164)

The rules to change weights are shown as follows in the BP learning algorithm. ∆wi j = η(dk − ok )ok (1 − ok )y j l

∆vi j = η

∑ (dk − ok )ok(1 − ok )w jk

k=1

!

(5.165) y j (1 − y j )xi

(5.166)

where, η is the learning efficiency of the network. The inputs are several indexes which can affect the evaluation objects. The hidden layer is the bridge to adjust the relationship between the inputs and outputs. The weights and thresholds are adjusted continuously until the error is within the acceptable range. The outputs are the final evaluation values of the indexes. The input signal is input forward and the error signal is transmitted backward. At the hidden layer, the thresholds and weights of each neuron are adjusted continuously until the network achieves the expected output. The stored memory will be used to predict or revise the sample results when the input variables are similar with the previous variables. Some application fields are covered by the BP neural networks, such as the psychology, medicine, prediction, actual production, effectiveness evaluation, intelligent control and optimization. In particular, it is widely used in the prediction field and has good application effects. The complex model calculations are avoided and many modeling problems are solved. The reliable reference is provided for the optimization, analysis and decisionmaking of the evaluation purpose. The BP neural network has the advantages of its accurate prediction and high reliability. In the process of application, the algorithm can be constantly improved and optimized to obtain more accurate results. The BP neural network refers to the training process that the calculation error is constantly compared with the expected error based on the error back propagation and then the

192

Deping Zhang and Xuefeng Yan

weights and thresholds of each neuron are constantly adjusted. The expected error is the basics of the training. The comparison result between the actual calculation error and the expected error of the network is transmitted by the back propagation in the training process. The training will be finished when the results meet the requirements. Otherwise, the training will be repeated. The general training process of the BP neural network is shown in Figure 5.28.

Figure 5.28. The training of the BP neural network.

Step 1. Initializing the network. The initial parameters are set, such as the learning efficiency, error accuracy, initial weight and threshold, and training accuracy of the network. Step 2. Setting the number of nodes in hidden layer. The training samples with the clear input-output relationship are input in batch, and the error function is recorded in real time. Step 3. Calculating the error of the output, and comparing with the expected error. Step 4. The thresholds and weights of the neuron are adjusted continuously by using the back propagation. Step 5. The network training will be finished when the output error is smaller than the expected error. Otherwise, the calculated error will be propagated backward and the process will return to step 4. If the output error is smaller than the expected error, the training will be finished. The smaller the error is, the more accurate the result is. Otherwise, the weights and thresholds of each neuron will be adjusted until the output results meet the requirements. Error setting is not the smaller the better. It should consider the network convergence speed comprehensively. In the next section, combined with the simulation results, the influence of the

Classical Method of Effectiveness Evaluation

193

network error setting on the network convergence speed will be analyzed in details. (2) The Support Vector Regression (SVR) Algorithm The SVM is proposed for the binary classification problem and an important application branch is the support vector regression (SVR) [175]-[177]. The difference between SVR and SVM is that SVR sample points are only classified into one category finally. Its optimal hyper-plane is not the most obvious one between two or more sample points as the SVM, but it minimizes the total deviation of all the sample points from the hyper-plane. Given a set of data {(x1 , y1 ) , (x2 , y2 ), · · · , (xm , ym )}, where xi ∈ Rd , yi ∈ R, the regression problem is hoped to learn a model, f (x, w) = wT x + b

(5.167)

which makes f (x) and y as close as possible. The nonlinear problems cannot be represented by Equation (5.167). The improved method is to introduce the nonlinear mapping function Φ(x). And then the input space is mapped to a feature space with a higher dimension. Φ(x) can reach infinite dimensions and then f can be approximate to any nonlinear function. The infinite dimensions cannot be treated in the actual calculation. It is unnecessary to calculate Φ(x) and it is enough to calculate the inner product Φ(xi )T Φ(x j ). The linear Equation (5.167) can be extended as a nonlinear equation with the introduction of Φ(x). f (x, w) = wT Φ(x) + b

(5.168)

where, w is the weight vector with the same dimension of vector Φ(x). The losses of the traditional regression models are usually based on the difference between the output of f (x, w) and the real output y. The loss is zero if and only if f (x, w) and y are exactly the same. But, the SVR assumes that it can tolerate a deviation of ε between f (x, w) and y. That is, the losses need calculated if and only if | f (x, w) − y| > ε. As is shown in Figure 5.29, the SVR construct an interval with the width ε take f (x, w) as the center. The prediction is regarded as validity when the samples fall within this interval zone. Therefore, the loss function of the SVR is called ε − insensitive error, which is, ( 0 if |y − f (x, w)| ≤ ε L(y, f (x, w)) = (5.169) |y − f (x, w)| − ε otherwise All model outputs f (x) are expected to fall within the ε interval zone. Therefore, the optimization object of the SVR can be defined as follows: 1 min ||w||2 w,b 2

(5.170) T

s.t. |yi − w Φ(xi ) − b| ≤ ε,

i = 1, 2, · · · , m

The slack variable ξ > 0 can be introduced for each sample point. That is, some samples are allowed to fall out of the interval zone and then the model have the better robustness.

194

Deping Zhang and Xuefeng Yan

Figure 5.29. The SVR. In fact, there are two inequalities since the absolute value is used. It means that it is nec(L) (U) essary to use the slack variable in both sides, which is defined as ξi , ξi . Therefore, the optimization object is shown as follows: 1 2 2 ||w|| +C

min

w,b,ξ(L) ,ξ(U)

s.t.

(L)

−ε − ξi (L)

ξi

m

(L)

(U)

∑ (ξi + ξi

)

(5.171)

i=1

(U)

≤ yi − wT Φ(xi ) − b ≤ ε + ξi (U)

≥ 0, ξi

≥ 0 i = 1, 2, · · · , m

(5.172) (5.173)

where, C and ε are parameters. The larger the C is, the greater the penalty for outlier points will be. Eventually, there will be fewer points cross the interval boundary and the model will become complex. If C is small, more points will cross the interval boundary, and the model will become smoother. However, the larger ε is, the higher the tolerance for outlier points is and the smoother the final model will be. This parameter is unique in the SVR and is not available in the SVM. For the Equation (5.171), the Lagrange multiplier is introduced for each constraint, as is shown as follows: L(w, b, α(L), α(U) , ξ(L) , ξ(U) , µ(L) , µ(U) ) m m 1 (L) (U) (L) (L) = ||w||2 +C ∑ (ξi + ξi ) + ∑ αi (−ε − ξi − yi + wT Φ(xi ) + b) 2 i=1 i=1 m

(U)

(U)

+ ∑ αi (yi − wT Φ(xi ) − b − ε − ξi i=1

m

(L) (L)

m

(U) (U)

) − ∑ µi ξi − ∑ µi ξi i=1

i=1

Classical Method of Effectiveness Evaluation

195

The dual problem is shown as follows: max min L(w, b, α(L), α(U) , ξ(L) , ξ(U) , µ(L) , µ(U) ) α,µ w,b,ξ

(L)

(U)

s.t. αi , αi (L)

≥ 0,

i = 1, 2, . . .m

(U)

≥ 0,

i = 1, 2, . . .m

(L)

(U)

µi , µi

If the partial derivative of w, b, ξi , ξi as follows:

(5.174)

is zero in the above formula, it can be obtained

m ∂L (U) (L) = 0 =⇒ w = ∑ (αi − αi )Φ(xi ) ∂w i=1

∂L = 0 =⇒ ∂b ∂L ∂ξ(L) ∂L ∂ξ

(U)

= 0 =⇒

m

(U)

∑ (αi

i=1

(L)

− αi ) = 0

(L) (L) C − αi − µi (U)

= 0 =⇒ C − αi

=0

(U)

− µi

(5.175)

=0

The Equation (5.175) is substituted into Equation (5.174) and the Equation (5.175) is used to obtain C − αi = ui > 0. Therefore, when 0 6 αi 6 C, the simplified optimization can be obtained as follows: m

max

α(L) ,α (U)

(U)

∑ yi(αi

i=1

− m

s.t.

(U)

(L)

− αi ) − ε(αi

(U)

+ αi )

1 m m (U) (L) (U) (L) ∑ ∑ (αi − αi ) · (α j − α j )K(xi, x j ) 2 i=1 j=1 (U)

∑ (αi

i=1

(L)

(5.176)

(L)

− αi ) = 0 (U)

0 6 αi , αi

6 C,

i = 1, 2, . . .m

where K(xi , x j ) = Φ(xi )T Φ(x j ) is the kernel function. It is necessary that the KKT condition need to be satisfied and the complementary slack condition is shown as follows: (L)

(L)

(U)

(U)

αi (ε + ξi + yi − wT Φ(xi ) − b) = 0 αi (ε + ξi

− yi + wT Φ(xi ) + b) = 0

(L) (L)

µi ξi

(U) (L)

µi ξi

(5.177) (5.178)

(L)

(L)

=0

(5.179)

(U)

(U)

=0

(5.180)

= (C − αi )ξi

= (C − αi )ξi

196

Deping Zhang and Xuefeng Yan (L)

If the sample is in the interval zone, ξi = 0,|yi − wT Φ(x) − b| < ε. Only let αi = (U) 0, αi = 0, the complementary slack can be hold. Then w = 0, based on Equation (5.175). It indicates that none of the samples are the support vectors in the interval zone. If the (L) (U) samples is in or out of the interval zone, the corresponding αi or αi can be non-zero. In addition, a sample cannot be above and below the f (x, w) at the same time. Thus, the Equations (5.177) and (5.178) cannot be held at the same time. Therefore, at least one of (L) (U) αi and αi is zero. The quadratic programming or the SMO algorithm can be used to calculate the α in m

(U)

the optimization problem (5.176). Then, the model parameters w = ∑ (αi i=1

(L)

− αi )Φ(xi )

can be obtained according to the Equation (5.175). If any samples satisfy 0 < αi < C, it is possible for the model parameter b to obtain ξi = 0 from the Equations (5.179) and (5.180). According to the Equations (5.177) and (5.178), there will be, m

(U)

(L)

b = ε + yi − wT Φ(xi ) = ε + yi − ∑ (α j − α j )K(x j , xi )

(5.181)

j=1

Then the final SVR is shown as follows. m

(U)

f (x, w) = wT Φ(x) + b = ∑ (αi i=1

(L)

− αi )K(xi , x) + b

(5.182)

According to the KKT (Karush-Kuhn-Tucker) condition, only some of the coefficients are non-zero in the above planning problem, and the corresponding input vector has an approximate error that equal to or greater than ε, which is called the support vector. The structure of the SVR output function is shown in Figure 5.30.

(U) (L) αi − αi

Figure 5.30. The structure of the SVR output function. Generally, some meta-heuristic algorithms are adopted in the regression analysis based on the SVR, such as GA, particle swarm optimization (PSO), and etc. The parameters of the SVR are optimized to improve the accuracy of the algorithm. The flow is shown in Figure 5.31.

Classical Method of Effectiveness Evaluation

197

Figure 5.31. The process of prediction and analysis of the SVR algorithm. The specific steps of the SVM (ε-SVR) algorithm for the regression problems are shown as follows: 1) The historical data are input and preprocessed to form the training and test dataset. 2) The parameters of the SVR are initialized. The appropriate kernel function is selected and the random initial values are assigned to the Lagrange multipliers αˆ and b. 3) The objective function of the SVR can be established based on the training data. Then it can be solved to obtain the values of αˆ and b. 4) The values are substituted into the prediction function and the training data are used to calculate the predicted value at a certain time in the future. 5) The error function is calculated. The learning process will be finished when the absolute value of the error is less than the predetermined positive number. Otherwise, after the parameters are optimized by heuristic method, the process will return to step (3). The main steps of ε-SVR are shown in the Algorithm 5.6. (3) The Group Method of Data Handling (GMDH) The group method of data handling (GMDH) is a polynomial neural network or statistical learning method, which was proposed by A.G. Ivakhnenko et al. It is used to solve some complex and high-dimensional nonlinear problems, such as the road rutting depth detection and so on. It can take the relationship between the input and output as the part of/all of the input of the polynomial function. These polynomial functions are constructed by the linear and nonlinear regression. In the selection of the polynomial type, there are many neurons

198

Deping Zhang and Xuefeng Yan

Algorithm 5.6. The ε-SVR Algorithm 1) It is assumed that the known training sample set is {xi , yi , i = 1, 2, · · · , l} and the expected output is yi ∈ R, xi ∈ Rd . 0

2) The appropriate parameters ε, C and the kernel function K(x, x ) are selected to construct and solve the optimization problem given by the Equation (5.176). The optimal (L) (U) (L) (U) (L) (U) solution αˆ = (α1 , α1 , α2 , α2 , · · · , αl , αl )T can be calculated. m

(L) (U) 3) A positive components 0 < α j < C of αˆ is selected to calculate bˆ = yi − ∑ (α j − j=1

(L) α j )K(x j , xi ) + ε. Or a positive component m (U) (L) bˆ = yi − ∑ (α j − α j )K(x j , xi ) − ε. j=1 m

0data − < vi h j >recon ∆ai =< vi >data − < vi >recon (5.198)  ∆b j =< h j >data − < h j >recon

where < · >data is the distribution of the original observation data model and < · >recon is the distribution of the reconstructed model. Considering learning rate ε the parameter update criterion is shown as follows:  k+1 k  wi j = wi j + ε∆wi j k+1 (5.199) ai = aki + ε∆ai  k+1 b j = bki + ε∆b j

BP is used for the training in the traditional neural networks. However, with the increase of the hidden layers, there are several problems in the BP, such as the gradient gradually sparse and easy convergence to the local minimum. For the deep learning-based DBN, network parameters are trained through the pre-training and the fine-tuning, which can better solve the existing problems of the BP. The algorithm is as following steps: Step 1. Pre-training In the unsupervised condition, each layer of the network is trained separately. The outputs of the upper layer are taken as the inputs of the lower layer. When the eigenvector is mapped to different feature space, the feature information can be retained as much as possible. Step 2. Fine adjustment The BP network is set in the last layer of the DBN and the outputs of the RBM are taken as its inputs. The parameter fine adjustment from top to bottom is achieved based on the supervised network training. Each layer of RBM network can only ensure that the weights in its own layer are optimized. Therefore, the bias information is also propagated by the BP network from top to bottom to each layer of the RBM, which is used to fine adjust the whole DBN network. The whole training process can be taken as the initialization of the weight of deep BP network, so as to overcome the shortcoming of BP network that fell into the local optimal. The training time and the convergence speed are also improved significantly. The common machine learning algorithms can be used for the predictive analysis of various effectiveness evaluation indexes [178]. Meanwhile, the correlation analysis and sensitivity analysis between indexes can also be performed based on these models to improve the evaluation accuracy with effect.

204

Deping Zhang and Xuefeng Yan

5.6.2.

Effectiveness Evaluation Based on Machine Learning

The effectiveness refers to the degree the equipment can achieve the expected goals under specified condition on a particular mission. It is also the specific performance of the tactical and technical index. The key task of the effectiveness evaluation is to construct the model. The traditional model is mainly constructed in analytical way. The modeling process mainly includes the construction of the index system, the determination of weight coefficient for each index, and the comprehensive evaluation of the effectiveness index, in which the weight coefficient is the key. The weight determination methods include subjective weighting and the objective weighting. The subjective weighting method is gotten the weight by expert estimation, such as Delphi method, AHP, ANP, and so on. The objective weighting is gotten by the dispersion degree (standard deviation) or correlation degree (correlation coefficient) of each scheme. The weights are only for the selection of the schemes. These methods can not reflect the impact of the evaluation index itself on the physical mechanism objectively. The typical methods include the entropy weight method, the standard deviation score, PCA, KPCA and the factor analysis. Among them, PCA gives weights to the principal component indexes after the (kernel function transformation) linear combination of the original indexes. The interpretability of the physical meaning for the original indexes is lost. On the basis of PCA, the factor analysis tries to strengthen the interpretability by rotating the principal component factors. However, it is very difficult to fully return to the original index level to interpret the weights. Although the equipment effectiveness evaluation problems are solved by the analytic method at some extent, there are obvious subjective factors interfering in the evaluation. In particular, it is not objective to assign weights to evaluation indexes. To address these challenges, it is necessary to adopt the simulation analysis data, the machine learning algorithms and the data-driven thinking. The equipment effectiveness evaluation model can be established based on the meta-model [179]-[181]. On this basis, the meta-heuristic optimization algorithms, such as GA, PSO, and so on, can be used to improve the fitting model accuracy of the machine learning [182]-[184]. Then the effectiveness evaluation meta-model is constructed and the correlation analysis and sensitivity analysis can be performed. The relationship of the evaluation index system and the objective weight between the indexes are mined. This method is the foundation for the rapid optimization design of the top level SoS and the prediction of effectiveness evaluation. (1) Effectiveness Evaluation Based on Machine Learning The essence of the effectiveness evaluation is to solve multi-objective input nonlinear equation, which can be described as follows.   max( or min) F(X) s.t. gi (X) ≤ 0, i = 1, 2, · · · , m (5.200)  h j (X) = 0, j = 1, 2, · · · , k

where X = (x1 , x2 , · · · , xn )T ∈ Rn is the evaluation index set. Rn is the real vector space of n dimensions. F(X) is the non-linear objective function of the n dimension index space. gi (X) and h j (X) are the inequality and equality constraints of the objective function respectively.

Classical Method of Effectiveness Evaluation

205

For the equipment, the objective function space F(X) of effectiveness evaluation is very complicated and usually cannot be described by the specific mathematical expressions. It is a typical “black-box system” that cannot be accurately described by the traditional mathematical analysis method. It is necessary to consider other methods with scientific feasibility and high confidence. (2) Effectiveness Evaluation Based on Meta-Model The meta-model is a simplified model that fits the algorithm based on the I/O data and obtain the internal law of the problem. It provides the internal mechanism description of the complex systems. The construction of effectiveness evaluation the meta-model is shown in Figure 5.35, mainly includes the following four steps.

Figure 5.35. Construction of the evaluation meta model. Step 1. Simulation scenario design According to the requirements of the effectiveness evaluation object, the evaluation

206

Deping Zhang and Xuefeng Yan

task, and the operation scenario, the operation simulation scenario is designed to simulate the mission of the evaluation equipment, which lays a foundation for generating effectiveness evaluation data. Step 2. Evaluation index system construct The significance of the evaluation index system lies in guiding the experimental design of the simulation evaluation including the input and output. Through analyzing the evaluation mission and the object as well as the the operation confrontation, the index that affect the effectiveness are determined under the conditions of confrontation, so as to construct a hierarchical evaluation index system. Step 3. The simulation experiment design The experimental design is obtaining a set of evaluation index parameters with uniform distributed and traversing all possible situations in effectiveness evaluation. The training data and test samples containing a series of universal laws can be constructed using evaluation meta-model. Step 4. Meta-model selection and regression analysis Considering the efficiency of the meta-model algorithm for regression analysis, the suitable meta-model is selected based on the characteristics of the effectiveness evaluation and the structure of index system[185]. The evaluation meta-model can be constructed by several methods, such as BP, SVR, GMDH, and DBN.

5.6.3.

Case Study

The long-rang precision sea strike SoS (LPSS) will be taken as an example in this section. The effectiveness evaluation process will be described based on the DBN network. In order to obtain the accurate evaluation result, the data generation model is constructed according to the rule of index data value. The sample indexes are shown in Table 5.25. Based on the experimental design, the corresponding output values are obtained through the simulation of the LPSS. Five thousand schemes are generated by simulation, which contain the input factors and the output factors that can be used to constitute the experimental data. The index data value and its corresponding effectiveness value are selected as a sample under one strike. For the qualitative index in the samples, the “expert” questionnaire is adopted and the data are quantitatively processed at 9 levels. The reference standard of the evaluation index can be obtained through research and experience. The dimensions and units are inconsistent in the different indexes. Therefore, the range transformation method is adopted for the standardization, so as to eliminate the difference between indexes. For the benefit index y, the formula is shown as follows: y=

x − xmin xmax − xmin

(5.201)

For the cost index y, the formula is shown as follows: y=

xmax − x xmax − xmin

(5.202)

207

Classical Method of Effectiveness Evaluation Table 5.25. Sample indexes and types in LPSS Index name Satellite revisit cycle x1 Satellite orbit altitude x2 Instantaneous frequency coverage x3 Instantaneous frequency coverage angle x4 Imaging satellite revisit cycle x5 Imaging satellite orbit altitude x6 Single scene coverage x7

Mean 12 4.3 × 105 120

Variance 2 25 12

270

10

6

2

270

25

Imaging resolutionx8

0.85

0.15

5.8 × 106 50

Maximum endurance time x9 40 Mission flight altitude x10 5 × 104 Cruise speed x11 8

5 35 4

Maximum detection range x12 Number of simultaneous tracking objects x13 Number of guided aircrafts x14 Missile positioning accuracy x15 Radar detection accuracy x16 Radar scan range x17

1550

18

5

3

4

2

0.75

0.3

0.90 330

0.3 10

SAR imaging time x18

5

2

SAR imaging range x19

300

12

Bit error rate x20 Communication delays x21

0.1 3

0.15 2

Data throughput x22

0.95

0.25

Point-to-point transmission 0.92 rate x23 Network data throughput x24 0.95

0.15

Underwater Acoustic Signal 0.86 Connectivityx25 Target RCS x26 0.87

0.4

0.20

0.35

Index name Signal-to-noise ratio x27 Response time x28 Info processing quantity x29 Info fusion accuracy x30

Mean 0.6 4.5 8

Variance 0.34 0.8 2

0.9

0.25

Aux-decision-making capability x31 Acc-decision accuracy x32 Decision process delay x33 Missile operational performance x34 Target type x35 Search total azimuth x36 Missile fuze performance x37 Missile seeker recognition probability x38 Missile ballistic characteristics x39 Missile body flight speed x40 Missile end penetration speed x41 Missile cruise speed x42 SAR imaging resolution x43 Missile stealth capability x44 Missile target interception capability x45 Connectivity x46 Missile seeker scan angle x47 Missile seeker detection range x48 Underwater signal bit error rate x49 Missile seeker detection accuracy x50 Communication delay between nodes x51

0.89

0.15

0.90

0.16

15

3

0.90

0.3

5 320 0.86

3 18 0.5

0.80

0.5

0.90

0.36

5.2

0.45

2.5

1.2

2.4 0.85

0.8 0.38

0.85

0.5

0.90

0.18

0.93 227

0.5 18

2.5 × 103 25 0.05

0.02

0.90

0.4

4

2

208

Deping Zhang and Xuefeng Yan

After preprocessing the index data, the samples are normalized by the scale transformation method. The formula is shown as follows: z = a+b×

y − ymin ymax − ymin

(5.203)

where a and b are constants. ymin and ymax represent the minimum and maximum of the same index data y respectively. y is the original data. z is the normalized data. After the quantization, standardization and normalization, the sample can be taken as the input of the effectiveness evaluation analysis model. For the BP, SVR, and GMDH, the analysis accuracy can be affected by the excessive input variables. Therefore, variables with great influence on the effectiveness value are kept based on mutual information theory, which is used to represent the amount of shared information among multiple variables. Therefore, it is generally adopted in variable selection. The mutual information between the discrete random variables X,Y is defined as follows: n

m

I(X,Y ) = ∑ ∑ p(xi , y j )log2 ( i=1 j=1

p(xi , y j ) ) p(xi )p(y j )

(5.204)

where n and m are the sample number of the random variables X and Y respectively. The greater the mutual information between two random variables is, the stronger the correlation between them is. When the amount of information is smaller or close to 0, the correlation between variables is weaker or independent. Based on the simulation data, 51 indexes in the index system are screened by the mutual information. The value 0.7 is taken as the mutual information threshold. 23 performance indexes with the mutual information greater than 0.7 are obtained. Some data are adopted as training data of BP, SVR and GMDH, such as the 23 performance index values and the effectiveness index values obtained by simulation. Then the effectiveness evaluation model of the LPSS can be obtained. The BP model is taken as an example. When faced with large-scale data calculations, the BP algorithm has low computational efficiency and is easy to fall into local optimum. Therefore, initial weights and thresholds of BP neural network can be optimized by the particle swarm optimization (PSO), so that the calculation efficiency can be improved. The PSO is derived from the birds foraging behavior. The standard PSO evolution formula is shown as follows: vik+1 = wvkij + c1 r1 (pkij − xkij ) + c2 r2 (pkg j − xkg j ) j

xik+1 = xkij + vik+1 j j

(5.205) (5.206)

where w is the inertia weight, c1 and c2 are acceleration factors. r1 , r2 ∈ rand[0, 1]. For the j dimension variable of the parameter i in the k-th iteration, vkij , xkij , pkij , and pkg j are the speed, position, optimal position of individual extreme value and optimal position of group extreme value respectively. The initial values and thresholds of BP can be calculated by the improved PSO. The local optimization ability can be enhanced by the Equation (5.208). The search ability of the particle and the cognitive ability of all particle groups can be developed by the Equation

Classical Method of Effectiveness Evaluation

209

(5.209). w = wmax − (wmax − wmin )k/T C1 = Cmax − (Cmax −Cmin )k/T

C2 = Cmax + (Cmax −Cmin )k/T

(5.207) (5.208) (5.209)

where wmax and wmin are the maximum and minimum of initial inertia weights respectively. Cmax and Cmin are the maximum and minimum of initial acceleration factors respectively. C1 and C2 are the values of inertia weights and acceleration factors respectively in the k-th iteration. The fitness function of the PSO is the training error of the BP, as is shown in the following: N

Ffitness =

m

2 ∑ ∑ (pi j − ti j )

i=1 j=1

(5.210) N where N and m are the number of samples and the dimension of the observation data respectively. For the j dimension observation data of the i sample, pi j and ti j are the output value and true value of the model trained by the BP algorithm respectively. The BP structure parameters can be described as 23-35-1. 23, 35 and 1 represent the neuron number of the input layer, the hidden layer and the output layer respectively. The learning rate is 0.001. The learning target is 0.01. The number of iterations is 5000. The learning parameters of the SVR are C and ε, which are optimized and selected by the grid search. The range of parameters is [-8, 8] and the iteration step is 1. In the GMDH, the quadratic K-G polynomial is adopted as the reference model. The minimum deviation criterion is adopted as the external criterion and 4-layers network structure is selected. For the DBN, according to the complexity and the number of the evaluation indexes, the 5-layers DBN network structure is selected for the 51 indexes in the index system. And the three hidden layer network structure is adopted to represent the non-linear relationship between the data. The number of neurons in each hidden layer is gradually decreased and finally encoded into the required number of dimensions. The first hidden layer usually takes an integer within 10 times of the initial data dimension. The number of neurons in each layer of the network structure is shown as follows. The neuron number of the input layer is 51. The neuron numbers of the three hidden layers are 200, 100, and 50 respectively. The neuron number of the output layer is one. According to the simulation experiment, the 5000 sets of data for 51 performance indexes and their corresponding effectiveness evaluation values are obtained. The 125 sets of data and the remaining are respectively taken as the test set and the training set. The training results obtained are shown in Figure 5.36. The box plot of the training error for the four machine learning algorithms can be described as shown in Figure 5.37. Based on the construction method of the effectiveness evaluation model, the metamodel is fitted by the machine learning algorithms (such as BP, SVR, GMDH, and DBN). The association relationship of evaluation index is explored and the objective weight is assigned to each index. The objective modeling method can be formed which become the foundation for the rapid optimization design, evaluation prediction.

210

Deping Zhang and Xuefeng Yan

Figure 5.36. The comparison of LPSS based on four machine learning algorithms.

Figure 5.37. The training error comparison of four machine learning algorithms.

Chapter 6

Mission Oriented Effectiveness Evaluation 6.1. Mission Based Effectiveness Evaluation 6.1.1.

Introduction

As a comprehensive assessment of the whole SoS, the mission-oriented operational effectiveness refers to the probability that SoS can successfully complete a certain mission at a given time in specified conditions. It is based on effectiveness evaluation and single performance analysis of SoS in each operational stage with specific mission. In other words, it is a process to analyze the effectiveness of the whole SoS through analyzing the single effectiveness of each operational stage, and obtaining the relevant output data and constraints. The purpose of effectiveness evaluation of each operational stage is to determine the composition and execution relationship of each mission, as well as the effect and status. Single effectiveness analysis refers to single index is selected as the evaluation object and its results are taken as the input of overall effectiveness evaluation model. For example, the hit probability of a single torpedo can be regarded as the evaluation index of the overall submarine system effectiveness. Therefore, before the overall effectiveness evaluation, it is necessary to evaluate the single index. The construction of the effectiveness measurement model for each single index is important for analyzing the SoS effectiveness. For the evaluation model, multiple factors should be taken into consideration. The model not only should reflect the essence of the SoS, but also should be as simple as possible for calculation and analysis. In establishing the mission-oriented SoS effectiveness evaluation model, various states should be described in every stage. The state possible changes and transitions should be described in detail. When the model is constructed, the results can be calculated and analyzed. Therefore, the steps of operational effectiveness analysis can be summarized, as shown in Figure 6.1.

212

Deping Zhang and Xuefeng Yan

Figure 6.1. The framework of the SoS effectiveness evaluation analysis.

6.1.2.

Mission-Oriented Operation Effectiveness Evaluation Model

(1) Activity Diagram of Mission-Oriented Operation The activity diagram constructed mainly analyzes the whole operational process according to the characteristics of the mission based on UML or domain meta-model [186]-[188]. The specific steps are shown as follows: Step 1. The use-case diagrams should be established step-by-step to find all the use cases involved in the model. Step 2. Entities involved in the first step should be identified and described by static diagram: a class diagram is used to provide a detailed description of the entity. Step 3. The activities and behaviors of the entities should be defined and described by dynamic diagram. Several ways can be used to improve the use-case diagram: sequence diagram, activity diagram, interaction diagram and so on. UML activity diagram mainly contains several basic elements which include initial node, termination node, node set and control edge set. In order to describe these elements better, the UML activity diagram is formally defined as follows: Activity diagram is a quadruple G =< in, F, A, E >. (a) in is the initial node. There’s always a path from the initial node in to all the other nodes. (b) F represents the all termination nodes set. (c) A =< AN, ON,CN > is a node set, where AN represents the active node set, ON represents the object node set. CN = DN ∪ MN ∪ FN ∪ JN is the control node, and satisfies

Mission Oriented Effectiveness Evaluation

213

DN is the Branch node . MN is the merge node. FN is the fork node. JN is the join node. (d) E is the control edge set and satisfies E = {(x, y)|x, y ∈ A}. From the definition of the UML activity diagram, the control nodes in the activity diagram include branch node, merge node, fork node and join node. Branch node is very common branch node in the SoS operational flow, which usually represents condition and behavior of the equipment in different operational situation. The branch node contains one inbound migration and two or more outbound migrations with mutually-exclusive conditions. The merge node includes one outbound migration and two or more inbound migrations. The operational control flow represented by merge node does not need to occur at the same time, when the requirements of the merge node are satisfied by the single operational control flow, the process can be executed continually. Branch node and merge node are expressed by hollow diamond in the activity diagram. The join node and the fork node are introduced into the UML activity diagram to realize the modeling of the concurrent operational control flow. Both fork node and join node are represented by the thickened horizontal segments. There may be two or more concurrent running control flows when at running time. There is no time constraint between the concurrent control flows at a macro level. The operational behavior flow is divided into two or more concurrent operational control flows, which can be described by the fork node. Each fork node can have one inbound migration and two or more outbound migrations, and each concurrent output is executed independently and does not interfere with each other. The join node can have one outbound migrations and two or more inbound migrations, represents the synchronous occurrence of two or more concurrent control flows. When all the control flows reach the join node, the control process can perform continually. It is adopted to synchronize these concurrent branches to accomplish mission together. In practical, the analysis of the whole operational process becomes very complex because of the various control structures and the nested structures, which brings great difficulties to the effectiveness evaluation of the SoS. In order to evaluate and analyze the effectiveness of the mission-oriented SoS based on UML, it is necessary to further analyze the execution mode of operational process. But it is difficult to determine the operational scenes of the SoS based on UML. Therefore, various operational control structures of the UML activity diagram can be adopted for automatic identification. The operational scenes can be described by the activity diagram. In order to facilitate the system effectiveness evaluation, a virtual node is used to represent a basic operational activity structure, which is called the operational combination point. Next, the common basic control structure in the UML activity diagram will be introduced. (i) Sequential structure In UML activity diagram, sequential structure means that operational activity nodes take place according to time sequence, which is an ordered sequence of operational activity nodes. This kind of nodes can be basic nodes or combined nodes. It can be described by the chain structure, as is shown in Figure 6.2. The sequence structure in the activity diagram corresponds to the sequence structure of SoS, which indicates that each operational activity is executed in turn.

214

Deping Zhang and Xuefeng Yan

Figure 6.2. Sequential structure. (ii) Selection structure A selection structure in an operational activity diagram can be regarded as a set of nodes contained in a branch node and a merge node. The inbound migration interface of the selection structure is the branch node and the outbound is the merge node. The merge nodes can be divided into matched selection, unmatched selection and K-out-of-N selection based on the number of its inbound migration. If the Branch node has N(N 6= 0) outbound migration, the merge node aggregates K(0 ≤ K ≤ N) control flows. If all the outbound migration of a branch node can match the inbound migration of the merge node, that is, when K = N, it is called matched selection. if K < N, it is called K-out-of-N selection. Furthermore, if the outbound migration of the branch node does not aggregate to the merge node, it is called unmatched selection. Three different types of selection structures are shown in Figure 6.3.

Figure 6.3. Selection structure. (a) matched selection (b) K-out-of-N selection (c) unmatched selection. The selection structure indicates that one of several candidate operational activities (paths) will be selected for execution during the operational process and the specific choice depends on the current situation and operational command and control. Obviously, this behavior will have a distinct impact on the success of the whole SoS mission. Generally, it cannot be predicted before the operation. But the probability of each branch being executed can be estimated, these probabilities are sufficient to evaluate the operational effectiveness. (iii) Loop structure A loop structure is a set of operational activities that can be repeatedly carried out. It is modeled by a branch node and the loop structure has two branches (outbound migration). In both branches, one branch exits the loop by connecting to a node outside the loop and the other branch continues the loop by connecting to the node in the loop. If the branch node is the first node of the loop (i.e., the entrance node), then the structure is called the forward

Mission Oriented Effectiveness Evaluation

215

loop, as is shown in Figure 6.4. The branch node also can be the last node (i.e., the exit node), which is called the backward loop.

Figure 6.4. Loop structure. (a) backward loop (b) forward loop. In the process of mission-oriented operation, the loop structure is often used to indicate that a certain operational activity will be carried out repeatedly. There is also some probability information, which represents the probability of the control flow leaving and returning to the loop body. (iv) Branch structure In UML activity diagrams, the branch node is often used to describe two or more parallel operational activities. These operations may need to be combined into a single operational control flow. For this purpose, the join node or merge node will be selected. Join nodes are often used to describe the next activity which need to wait for all parallel operational activities before proceeding. This is a synchronous branch structure, as is shown in Figure 6.5. Sometimes, one operational activity does not need to wait for the completion of all pre-activities. In this case, merge nodes are often used. This is called a merge and branch structure. Similar to the selection structure, the branch structure also has different types, such as matching fork structure, non-matching fork structure and K out of N fork structure. (2) Data of Operational Effectiveness (i) Data of operational activity For the effectiveness evaluation, it is an important prerequisite to accurately estimate the index value of operational activity nodes. It depends on a certain sub-system or the performance and combat conditions of a certain equipment. There two methods to obtain the effectiveness index data of operational node activities. The first is that several missions completing probabilities of one node can be analyzed by traditional analytical method, such as the search probability, the position occupation probability. The second is based on the system modeling and simulation. The probability can be obtained through the results of the model analysis and simulation, so as to support the effectiveness data acquisition and calculation.

216

Deping Zhang and Xuefeng Yan

Figure 6.5. Fork structure. (a) synchronous branch (b) merge and branch Structure (c) K out of N branch structure (d) non-matching branch. (ii) Transition probability labeling of activity diagram The decision nodes in the activity diagram indicate different activities sequences or path during the execution, which usually correspond to different operational scenarios. Therefore, each outgoing edge of the branch node is marked with a branch execution probability in current activity. There are three common methods for determining the migration probability. The uninformed approach assigns the same probability to each outgoing edge of the branch node. The informed approach calculates the probability of each outgoing edge, based on the system with similar functions or history sequence. The intended approach give different probabilities with specific purpose assuming that the commanders focus on certain outbound migrations. The following methods can be adopted to obtain the migration probability. 1 Hypothesis based migration probability.

It is assumed that all n operational activities in a certain state have the same occurrence probability of 1/n in different operational situations. 2 Historical operational data based migration probability.

The historical operational data can be recorded and analyzed according to the historical or similar SoS, which is used to calculate the migration probability. 3 Evaluation and analysis based migration probability.

The migration probability can be obtained through the analysis, prediction and evaluation by experienced experts or commander based on possible operational requirements and scenarios. (3) Combination and Simplification of UML Activity Diagram Considering the uncertainty in environmental and operational activities, it is necessary to reasonably combine some operational activities to reduce the computing dimension and the important of uncertainty. In UML activity diagram, an operational scenario corresponds to an operational activity

Mission Oriented Effectiveness Evaluation

217

sequence. However, for a complex system, its activity diagram is more complex. Especially, for some cases with concurrent nested structure, it is very difficult to generate and understand each scene of activity diagram correctly. In order to make the complex activity diagram clearer and simpler, and facilitate the evaluation, it is necessary to further simplify the UML activity diagram. Considering the characteristics of activity diagram, the repeated iteration is adopted to identify the basic structure automatically. The virtual composite activity nodes is adopted circularly, until the activity diagram becomes sequential structure. The combination and simplification of the activity diagram is shown in the following. (i) Recognition and combination of loop structure It is assumed that x is a decision node and the node y (y 6= x) is the predecessor node of x. If there is a branch b that is out from node x and merged into node x after a series of nodes, then node x is regarded as a branch node. If the predecessor node y is on the branch b, the loop is a backward circuit, otherwise it is a forward circuit. If the loop structure is a forward circuit, the branch node x is the entry node and the exit node at the same time. On the other hand, if the loop structure is a backward circuit, in branch b, the node after x is the entry node, and x is the exit node. Once the entry node and exit node of a loop structure are identified, which is a loop structure in the control structure diagram can be determined and can be replaced by a combine node. As is shown in Figure 6.6.

Figure 6.6. Recognition and combination of loop structure. (ii) Recognition and combination of selection structures A selection structure can be adopted to describe two or more mutually exclusive and selective paths. It can be described by a branch node with two or more outbound edges. The execution of each path depends on the value of the conditional expression. Based on the number of outbound edges, the selection structure can be divided into two or multidimensional selection. A selection structure may be a matched selection structure, a mismatched selection structure or a K-out-of-N selection structure. Therefore, different types of selection structures should be identified according to their characteristics. Case 1: Matched selection structure It is assumed that x represents a branch node, if there is a merge node y and all branches

218

Deping Zhang and Xuefeng Yan

from the node x can be merged into the y, x is called a branch node matching the selection structure, If x is an entry node and y is an exit node, match selection structure can be formed by the entry node, the exit node, and the nodes and edges between them, which can be replaced by a combined node, is shown in Figure 6.7.

Figure 6.7. Recognition and combination of matched selection structure. Case 2: Mismatched selection structure In the mismatched selection structure, there is no aggregation nodes at the end of the control flow branch. However, if the termination nodes of the branches are known, the control flow can be simplified as a combine node. Therefore, the mismatched selection structures can be identified. It is assumed that x represents a branch node. If all branches from x terminate at the exit node of the control flow, the control flow structure starting from the node x can be judged as a mismatched selection structure. When each branch terminates, there is no activity any more. Therefore, a combine node can be adopted to replace this kind of control structure, as shown in Figure 6.8.

Figure 6.8. Recognition and combination of mismatched selection structures. Case 3: K-out-of-N selection structure In the K-out-of-N selection structure, K branches are converged and its exit node is the merge node. The remaining N − K branches are not matched and its termination node is the exit node, which satisfies K < N. Therefore, it can be identified using the rules in case 1 and

Mission Oriented Effectiveness Evaluation

219

case 2 respectively and then replaced by a combine node. The recognition and combination of selection structure of “three out of four” are shown in Figure 6.9.

Figure 6.9. Recognition and combination of K-out-of-N matched selection structure. (iii) Recognition and combination of branch structure It is difficult for the component-based software system to study the reliability of branch structure. Because the branch structure has no clear definition. In some cases, all or a few components are required to be completed successfully. Therefore, it is necessary to identify these branch structures before the reliability evaluation of component-based software system. Case 1: Matched branch structure It is assumed that x is a branch node, if all branches from x can be converged with a join node or merge node y, this structure is called matched branch structure. The x is the entry node and y is the exit node. The structure is formed by the node x, node y and the nodes and edges between them. The recognition and combination of matched synchronization branch structure are shown in Figure 6.10 respectively.

Figure 6.10. Recognition and combination of branch structure.

220

Deping Zhang and Xuefeng Yan Case 2: Mismatched branch structure

The different execution paths from node x may not be converged to a merge or convergence node. On the contrary, each path will end up with different termination node of the control flow. This is called mismatched branch structure. It is assumed that x is a branch node. All the branches from x terminate at the termination node of different control flow. If there are multiple termination nodes, only one termination node is retained and the others are removed. Then the control flow can be redirected to this termination node. The mismatched branch structure is formed by the all nodes and edges between the entry node and the termination node and can be replaced by a combined node. The recognition and combination of mismatched branch structure are shown in Figure 6.11.

Figure 6.11. Recognition and combination of mismatched branch structure. Case 3: K out of N matched branch structure Some branch structures are not matched perfectly. Some of the paths from node x end up at termination node, and others are gathered at the merge node or the join node. This type of structure is called K out of N matched branch structure. In this case, K paths are gathered into a single control flow by using merge (or join) nodes and N − K paths are mismatched paths. Therefore, the methods of Case 1 and Case 2 can be used for recognition and finally a combined node can be used to replace then, as shown in Figure 6.12.

Figure 6.12. Recognition and combination of the K out of N merged branch structure.

Mission Oriented Effectiveness Evaluation

221

The recognition and combination of the K out of N merged combined branch structure are shown in Figure 6.13.

Figure 6.13. Recognition and combination of the K out of N combined branch structure. For a given UML activity diagram, an evaluation model with only sequence structure can be generated by identifying and replacing each specific basic structure in the activity diagram. (4) Effectiveness Analysis of Virtual Combination Active Node The operation process of the entire SoS can be summarized into six basic structure: Sequence, AND-merge, OR-merge, AND-branch, OR-branch and Loop iteration. The six basic structures are further explained and the corresponding calculation formulas of effectiveness evaluation are given. (i) Sequence structure It composes of n activities. A certain mission can be completed based on continuous execution of n activities. That is, if and only if all n activities complete the mission successfully, the SoS can complete the mission, any failure of an active node will lead to the failure of the entire mission. It is assumed that the probability of completion of the active node i is pi , then the operational effectiveness calculation formula of the sequential structure composed of n active nodes is shown as follows: n

Es = ∏ pi

(6.1)

i=1

(ii) AND-merge structure A certain mission can only be successfully completed by n active nodes in cooperation with a certain activity node. The SoS can complete the mission successfully, if and only if all the n + 1 active nodes success. The failure of operational activities for any node will result in the failure of the mission. The probability of completion of each active node is defined as PI , i = 1, 2, · · · , n + 1.

222

Deping Zhang and Xuefeng Yan

Then the effectiveness is: n

Ea = ∏ pi

(6.2)

i=1

(iii) OR-merge structure A certain mission can be successfully completed by the cooperation of any operational activity i of the node n and the operational activity n + 1. In other words, this mission can only be completed when both the activitiy i and n + 1 complete the node mission successfully. n

Eu = (1 − ∏(1 − pi ))pn+1

(6.3)

i=1

(iv) AND-branch structure A certain mission is composed of n + 1 active nodes. It needs to be completed through one operational activity C0 together with n other operational activities Ci . In other words, if and only if all the n + 1 operational activities are completed successfully, the mission can be finished. n

Ew = p0 × (1 − ∏(1 − pi ))

(6.4)

i=1

(v) OR-branch structure The mission can only be completed after an operational activity C0 is successfully completed together with a certain activity node a of other n operations. In other words, The SoS can complete the mission only when the active nodes Ci and C0 complete their mission. n

Eo = p0 × ∏ pi

(6.5)

i=1

(vi) Loop iteration structure It is assumed that a certain mission is composed of one active node. It can be successfully completed by executing this node with n times. In other words, the system can complete the mission if and only if the mission is completed by operational activity C and the migration between active nodes. ER = p0 × pni

(6.6)

(5) System Effectiveness Analysis of Loop Step 1. The UML operational activity diagram can be constructed based on the mission of the equipment. The smallest loop in the diagram can be found and replaced by an equivalent virtual combined node. Then the loops in the diagram are continually searched and replaced until there is no smallest loop.

Mission Oriented Effectiveness Evaluation

223

Step 2. The basic operational activity path can be generated based on simplified activity diagram. It may contain zero or multiply (nesting) virtual combined operational activity nodes. Then each activity node is expanded and replaced by its corresponding internal operational activity process until no virtual combine operational nodes exists. Therefore, the initial set of operational activities is formed. Step 3. The initial operational activities set is simplified and the invalid operational activity process (operational scenarios) is deleted. Step 4. The operational effectiveness value of each scenario is calculated according to the information of each activity node in each scenario (operational activity sequence). First, in the scenario i(i = 1, 2, · · · , k), the effectiveness value of each virtual combined activity node is calculated and denoted as Ei,1 , Ei,2 , · · · , Ei,n . Then the effectiveness of the basic operational activity path is calculated. It is assumed that the path contains m activity nodes including n virtual operational activity nodes. The effectiveness of non-virtual operational activity nodes are respectively Ei,n+1 , Ei,n+2 , · · · , Ei,m . Therefore, the effectiveness of the entire operational scenario is shown as follows: m

Ei = ∏ Ei, j , i = 1, 2, · · · , k

(6.7)

j=1

where k is the total number of the scenarios in the mission. Step 5. Calculating effectiveness of the mission-oriented SoS. According to the occurrence probability pi of each operational scene ih the overall mission and the effectiveness of each operational scene in step 4, the effectiveness of the mission-oriented SoS can be calculated as follows: k

E = ∑ pi Ei

(6.8)

i=1

6.1.3.

Case Study

In this section, the mission of an air defense and anti-missile system is taken as an example to illustrate the mission-oriented effectiveness evaluation method [111]-[114]. (1) The Composition and Mission Flow The air defense and anti-missile system are composed of long-range early warning radar, operational command vehicle, fire-control radar, launch vehicles equipped with various types of missiles. The process mainly consists of the following steps, as is shown in Figure 6.14. The first is the alert stage. The early warning information is mainly acquired through the detection by remote warning radar and other early warning radar. Then, the obtained information will be transmitted to the command center for analysis and processing. In the stage of making interception plan, the target is recognized and classified through the information and the database obtained from the command center. Then the interception plan is formulated based on the location and distance of the fire unit. The targets with greater threat should be firstly intercepted if the interception capability is insufficient. In the stage of interception, fire control radar is adopted for target tracking, so the command center will send the target information to the fire control radar of each fire unit. The

224

Deping Zhang and Xuefeng Yan

Figure 6.14. Air defense and anti-missile operation command flow chart. missile launch vehicle deployed in a fire control unit contains missiles with various ranges. The short, medium and long-range missiles will be selected according to the distance from the target. Since a specific missile has its own optimal interception effect. Therefore, the flight path of the target can be calculated by the fire control radar. All the information will be bundled into navigation and control system of the missile. The missile will be launched at the scheduled time. Two missiles are generally used to intercept each target. The second missile is usually launched within the time period of 1500ms-5000ms after the first is launched. After that, different guidance modes are needed adopted for different types of missiles. The fire control radar can be used to correct the command for short and long-range missiles until the active radar seeker guidance of the missile starts to work, while the medium range missile needs continuous tracking guidance before hitting the target. After the strike is completed, the target signals are collected by radar to access the damage assessment. If the target is not intercepted, the second round of strike will be launched. The main behaviors of the anti-missile system can be obtained from the use case diagram, such as alert, planning, launch, and guidance. The static structure and behavior information can be obtained from the objects relation in the class diagram. On this basis, the dynamic model can be established to describe the special behavior of the system. In this way, the dynamic interaction can be shown in the different stages of system execution. To model the dynamic activity interactions in UML, activity, sequence, or collaboration diagrams can be used. The three diagrams have different emphasis, and they focus on job description, time and space respectively. In practice, air defense and anti-missile operation system focuses on time, while in simulation, the real focuses on the work flow. Therefore, two kinds of diagrams, the sequence diagram and the activity diagram are used to describe the system interaction, as is shown in Figure 6.15. The back up warning radar has no iden-

Mission Oriented Effectiveness Evaluation

225

tification in the sequence diagram, because it is mainly used when losing contact with the command center.

Figure 6.15. Sequence diagram of a certain type of air defense and anti-missile operation system. The workflow in activity diagram contains activity states, control flow, initial and end states, which are represented by the system rounded rectangle, the solid arrow line, and the default circular icon respectively. When the workflow needs to be forked or concurrent, diamonds and synchronization bars are introduced. The diamond corresponds to fork and represents OR-branch. Then, a negative or positive workflow will be generated. The synchronization bar corresponds to concurrency and split, which indicates a interactional or simultaneous splitting workflow. The interception process involves different operation modules which are distinguished by swim lanes, as is shown in Figure 6.16. Based on a certain air defense and anti-missile operation activity diagram, different activity scenarios are extracted. After screening, the following five scenarios are obtained 1 : S → C1 → C2 → C3 → C5 → F; 2 : S → C1 → C2 → C3 → C4 → C5 → F; 0

3 : S → C1 → C2 → C3 → C4 → C6 → C7 → C8 → C10 → C2 → C15 → F; 0

4 : S → C1 → C2 → C3 → C4 → C6 → C7 → C9 → C10 → C2 → C15 → F; 0

0

5 : S → C1 → C2 → C3 → C4 → C6 → C7 → C1 → C3 → F.

226

Deping Zhang and Xuefeng Yan

Figure 6.16. Activity diagram of a certain type of air defense and anti-missile system. 0

0

0

where C1 , C2 and C3 represent the virtual node of the selection structure, the branch structure 0 and the loop structure respectively. C3 virtual node of loop structure composed of a basic 0 node and a branch structure virtual node C2 . The completion probability of each basic and virtual node can be obtained through simulation and calculation, as is shown in Table 6.1. The occurrence probabilities of five operational scenarios are 0.05, 0.11, 0.12,0.4, 0.32 and 0.1 respectively in an air defense and antimissile operational activity. Therefore, its operational effectiveness is 0.8736.

6.2. Multi-Mission Effectiveness Evaluation Based on Operation Loop In order to complete a specific combat task, the equipment system, such as reconnaissance, decision-making and attack, form a closed loop with enemy targets. The operational process is composed of the observation, orientation, decision and action (OODA) according to the operation loop theory [189][190][191] . That is, the enemy targets are discovered and the relevant information is transmitted to the decision nodes by the reconnaissance

227

Mission Oriented Effectiveness Evaluation Table 6.1. The probability of each basic and virtual node mission Node name

Variable Probability Node name of success Target detection C1 0.9871 Information reception Information fusion C3 0.9983 Threat estimation Tracking recon- C5 0.9887 Command decision naissance Situation analysis C7 0.9921 Missile Selection Target allocation C10 0.9868 Missile strike Target assignment C15 0.9778 Missile repeated strike

Variable Probability of success C2 0.9912 C4 C6 0

C1 0 C2 0 C3

0.9865 0.9982 0.9952 0.9508 0.9894

nodes. Then the commands are issued to the attack nodes by the decision nodes after detailed analysis. Finally, the targets are attacked after receiving the corresponding command. The operational units with different functions in SoS are divided into three categories: reconnaissance, command and control, and attack nodes. The enemy targets are generally introduced into the operation loop by considering the source and flow direction of information so the research process will be closer to the actual operation. Therefore, the integrated operation loop includes four types of nodes: reconnaissance, command and control, attack and target. An entire operational activity can be completed by multiple reconnaissance and decision-making entities in practice [193]. First, the enemy target information is obtained by the reconnaissance nodes and transmitted to the command and control nodes after brief processing. Then, the commands and schemes can be formulated. Finally, the enemy targets are effectively attacked by the attacking nodes. The reconnaissance nodes include satellite, surveillance aircraft and radar detection and there is information sharing between these nodes. The command and control nodes also contain multiple command entities. The complex operational network SoS is formed by multiple operation loops. In order to complete specific mission, the closed loop is composed of reconnaissance, decision-making, strike equipment and the enemy targets. Because each equipment plays different roles in operation, it can be divided as follows: 1) The reconnaissance, surveillance, and early warning equipment. The sensor is adopted by the equipment to collect target and battlefield information. The main functions are the target reconnaissance, the intelligence acquisition and the battlefield surveillance. 2) The communication, command and control equipment entities. It refers to the equipment with the abilities of information processing and analysis, assistant decision-making, command and control on jamming entities. 3) The joint strike and interference equipment.

228

Deping Zhang and Xuefeng Yan

The equipment mainly execute operational damage activity, which have the functions of precision strike, firepower damage and electronic jamming. 4) Enemy target: the attack target. The most basic process of operational network can be described by the standard operation loop, and four relationships among reconnaissance, decision, command and control, and attack entities. During the operational process, there is also information sharing between reconnaissance entities, and cooperative command relationship among decision entities. The most complicated process is the information-sharing relationship between multiple reconnaissance entities, and the cooperative command between multiple decision entities at the same time. The operation loop is shown in Figure 6.17.

Figure 6.17. The schematic diagram of the operation loop. The multiple operation loops will be formed by the several equipment and the interaction between equipment in the actual operational scenarios. The equipment can be shared among operation loops. Therefore, a multi-layer complex operational network will be established, which is shown in Figure 6.18. The most basic operational process can be represented by a standard operation loop in the complex operational network. The operation loop number reflects the number of attack plans being executed by the SoS. The number can also be used as an index for evaluating the operational system. The more operation loops are, the greater the operational potential is, the more attacking plans are and the stronger the survivability is.

Mission Oriented Effectiveness Evaluation

229

Figure 6.18. The multi-layer complex operational network.

6.2.1.

Construction of Operation Loop Model

The operation loop is used to describe the relationship of the information and material energy transition between the operational units using the nodes and edges. In essence, the SoS model can be abstracted based on the complex network modeling principle. And then the model is further deepened and expanded based on the operation loop. The construction of the operation loop has the following four steps. Step 1. The operational goals are determined according to the missions. The SoS constitution of red side should be confirmed and the entire operational process should be clarified. Step 2. Extraction of the operation loop nodes. According to the function differences of each operational unit, platform or system, it can be abstracted into four types of nodes such as reconnaissance, command and control, attack and target nodes. The network node set is constituted by all operational unit nodes. Step 3. Extraction of the operational network edges. The relationship among the nodes in the operation loop is analyzed after the nodes are determined. The relationships with information flow are abstracted as the connecting edges in the network. Step 4. Construction of operational network model. The information flow edges are used to connect each operational unit to generate. And the operational network, as is shown in Figure 6.19. The operational network model can be expressed by the following formula. G = (N, E)

(6.9)

where N is the set of all operation unit nodes and E is the set of edges. All the equipment in SoS is used to attack the blue target entities directly or indirectly. Therefore, all kinds of

230

Deping Zhang and Xuefeng Yan

Figure 6.19. The construction of operational loop model. equipment entities and relationships will be contained in the operational network G which is formed by different operational loops. The modeling of the nodes is based on the equipment functions in operational activity. The entities of operational loops can be abstracted into four categories, which is shown as follows: |VNodeType| ∈ {T, S, D, I}

(6.10)

where T is the target node, S is the reconnaissance and surveillance node, D represents the command and control node, and I is the attack node. The attribute of different nodes are shown in Table 6.2.

6.2.2.

Operation Loop Based Multi-Mission Effectiveness Evaluation

In order to analyze the interaction between the nodes in the operation loop, the tactical and technical index of each node need to be described at first. The index is related with capability of the equipment unit system with independent function. (1) Description Nodes It is assumed that a certain type equipment node contains n capability indexes, it can be represented as the capability vector CNodeType, that is CNodeType = (c1 , c2 , · · · , cn )

(6.11)

Mission Oriented Effectiveness Evaluation

231

Table 6.2. Four common node types in operation loop Node type Node name T class Target class node S class Reconnaissance and surveillance node

D class

I class

Node attribute features description Enemy operational entities or battlefield facilities. The equipment entities collect of blue army information in battlefield and support for command and decision-making, which include reconnaissance satellites, radars, early warning aircraft, etc. Command and con- For the command platform, command and control center, the trol node information from satellite entities can be received, the analysis and decision can be made, and the operational commands can be issued. Attack node The weapon platform or SoS, includes fighter, missile, etc., which destroy or make mterfere to the blue targets.

(i) Reconnaissance and surveillance node S The main functions of reconnaissance and surveillance nodes acquire information of both sides in operation space and complete the reconnaissance and searching task. The equipment with one or multi-functions, such as reconnaissance, identification, tracking and search, is collectively called reconnaissance and surveillance node. Therefore, there are two tasks for the node in the network. The enemy objective is scouted, tracked, located, identified by node S. The information and data are transmitted to another equipment node through communication data link. The main capabilities of node S are the information reconnaissance, target recognition, and tracking and searching.The capability vector of reconnaissance and surveillance node is represented as follows: CS = (cS1 , cS2 , · · · , cS6 )

(6.12)

where cS1 , cS2 , cS3 , cS4 , cS5 , cS6 represent the reconnaissance coverage, the maximum detection distance, the recognition probability, the resolution, the maneuvering speed and the scanning frequency respectively. (ii) Command and control node D In the constructed operation loop, the command and control nodes mainly integrate the information from different reconnaissance nodes. According to the integrated the information, the battlefield situation is analyzed, the enemy intention is judged. Then the corresponding decision is made and the operational command is issued. The node D is the core equipment for informatization warfare. And it is the key to ensure the integrity, security and availability of information. Therefore, the main capabilities of the node D are the command decision, the information processing. The node D is represented as follows: CD = (cD1 , cD2 , · · · , cD7 )

(6.13)

where cD1 , cD2 , cD3 , cD4 , cD5 , cD6 and cD7 represent the commander knowledge level, the emergency response capability, the collaborative planning capability, the command deci-

232

Deping Zhang and Xuefeng Yan

sion time, the information processing rate, the information visualization capability and the accuracy of information analysis respectively. (iii) Attack node I The main functions of attack node I is to execute the specific attack operations blue targets, including against blue targets, precise attack, electronic jamming and destroy the blue targets. In order to describe operational capability of the attack node I, it is necessary to consider three attribute indexes: the operational coverage radius (cI1 ), the hit accuracy (cI2 ) and the maneuvering speed (cI3 ). The capability vector of attack node is represented as follows: CI = (cI1 , cI2 , cI3 )

(6.14)

(iv) Objective node T The objective node refers to the equipment unit and infrastructure of blue side. The objective of blue side is abstracted as node in operational network, which is an important part of the operation loop. It is necessary to consider their capability index that can affect the attack effectiveness of red side. The anti-reconnaissance and anti-strike capability are mainly concerned, that is, it is not easy to be detected by red side reconnaissance during the operation, not easy to be hit after being found, and will not easily be destroyed after being hit.The capability vector of target node T is represented as follows: CT = (cT1 , cT2 , · · · , cT9 )

(6.15)

where cT1 , cT2 , cT3 and cT4 represent the jamming interception capability, the rapid restructuring capability, the maneuvering capability and the warning time respectively. cT5 , cT6 , cT7 and cT8 refer to the invulnerability coefficient, the anti-radar coefficient, the camouflage stealth capability, the anti-optical coefficient and the anti-infrared coefficient. (2) Edge Description and Modeling In operation loop, each equipment unit in joint operation is abstracted as different nodes in network according to the different operational functions. The interaction relationship between nodes among reconnaissance information flow, control flow, coordination relationship flow and fire attack flow is abstracted into edges. There are 16 possible connection modes between the four types of nodes, which is shown in Table 6.3. Table 6.3. Sixteen possible connection modes between nodes Edge type S D I T

S S→S D→S I→S T →S

D S→D D→D I→D T →D

I S→I D→I I →I T →I

T S→T D→T I→T T →T

Mission Oriented Effectiveness Evaluation

233

The connection between equipment entities of joint operation is complex and involved the command, communication, coordination, sharing and support and so on. Therefore, it is difficult to consider all operation detail and constraints. If we only consider the operational capability of SoS against the blue side, the constructed node function is and the information stream is transmitted specifically between nodes. Typucally, there are seven types of node relationships that need to be considered emphatically, such as reconnaissance (T → S), decision-making (S → D), command (D → S), strike (I → T ), information sharing (S → S) and collaborative (D → D). The edge sets are as follows respectively:  ET S = {eTi S j }, Ti ∈ T, S j ∈ S, i, j = 1, 2, · · ·     ESD = {eS jDm }, S j ∈ S, Dm ∈ D, j, m = 1, 2, · · ·      EDS = {eDm S j }, Dm ∈ D, S j ∈ S, j, m = 1, 2, · · · EDI = {eDm In }, Dm ∈ D, In ∈ I, m, n = 1, 2, · · · (6.16)   E = {e }, I ∈ I, T ∈ T, n, i = 1, 2, · · ·  IT In Ti n i     ESS = {eSk Sl }, (k, l) ∈ j   EDD = {eDr Ds }, (r, s) ∈ m where there are E = ET S ESD EDI EIT EDS ESS EDD . The simple operational network structure model is shown in Figure 6.20. At the same time, the node type and edge type are described. S

S

S

S

S

S

Figure 6.20. Operation loop structure model Combined with the operation activities completed by each edge in the network, the tactical and technical indexes of different edges in the operation loop are analyzed and measured, and the vector is as follows: CEdgeType = (c1EdgeType, c2EdgeType, · · · , cnEdgeType)

(6.17)

where EdgeType represents the type of edge relationship; C denotes the capability index. In operation activities, the edge between nodes is mainly used for transmission and distribution of operational information such as reconnaissance, command and control, intelligence sharing, environment support and fire attack among different nodes. Therefore, it is necessary for both ends of information link to have communication capability and the

234

Deping Zhang and Xuefeng Yan

nodes are within the range of communication. The measurement indexes of the edge include the link transmission time c1 , the information loading capacity c2 , the transmission quality c3 , the transmission resistance capability c4 , the transmission delay c4 , and the transmission security capability c6 . The vector is represented as follows: Ce = (c1 , c2 , c3 , c4 , c5 , c6 )

(6.18)

(i) Reconnaissance information sharing link (S → S) The S → S edge represents the information sharing link between two different reconnaissance nodes. For the same target, the information quality in different operational nodes is different. Their respective advantages should be integrated and then the operational requirement can be satisfied by the detected target information as much as possible. If there is an information sharing relationship in the operation loop, the edge number will be increased and the reaction time will be delayed. However, the accuracy of information is enhanced after information sharing. (ii) Reconnaissance information uploading link (S → D) The S → D edge represents the information uploading link from reconnaissance node to the command and control node. The reconnaissance nodes transmit operational target information through the reconnaissance intelligence uploading link. (iii) Instruction uploading link (D → S) The D → S edge denotes the instruction uploading link from command and control node to reconnaissance node. The instruction is transmitted to reconnaissance node by command and control node. For example, in order to acquire more target information, orbital maneuver instruction is issued, then the target can be searched accurately. (iv) Command and control cooperation link (D → D) The D → D edge refers to the link between two command and control nodes, which is a process of cooperative command. The accuracy and efficiency of operational command can be improved. Meanwhile, the links and response time of operation loop will be increased. (v) Operational command issue link (D → I) The D → I edge represents the link from command and control node to fire attack node. (vi) Reconnaissance link (T → S) The T → S edge represents the activity of reconnaissance nodes against blue targets. According to the relationship between node S and node T , the unidirectional link from target node to reconnaissance node can be generated. The blue target information can be detected by the reconnaissance node with different ways (infrared reconnaissance, radar detection). It can be acquired by the operation activity of identification, tracking, discovery, etc. And the information foundation for command and decision is provided. (vii) Attack link (I → T ) The I → T link denotes the fire attack or electromagnetic interference activities from attack node to blue target. If the target node within the attacking range, the attack node

Mission Oriented Effectiveness Evaluation

235

can select different ways to attack. In the process of the attack or interference, several information services are provided by the navigation satellite, such as precise positioning, precise velocity measurement and precise time. The equipment have to have the operational capability to complete respective task in the whole operation activity. For each operation link, it is also necessary for the equipment entities to provide corresponding support capability. (3) Operational Capability Evaluation of Single Operation Loop In current research, more attention is paid to the network topology. The edge probability in operation loop is given directly without too much explanation. In this book, the operational effectiveness of each link is calculated based on the capability requirement satisfaction analysis. The operational effectiveness of edge is represented as the efficiency value. That is, Pr ∈ [0, 1], r ∈ {T → S, S → D, D → I, I → T, S → S, D → D}

(6.19)

If the operation edge r is completed by n capabilities, the effect provided by different capabilities is different for the whole operation process. Assuming that the weight of can

pability i is wi , ∑ wi = 1, the operational effectiveness of r can be obtained by using the i=1

weight method. n

Pr = ∑ wiC(ei )

(6.20)

i=1

The operation loop is directed and closed, so each edge will affect the next edge. Then the overall operational capability will be affected. After analyzing four aspects of the standard operation loop, the operation effectiveness can be measured by the capability index Eop . The formula is shown as follows: Eop = PT →S · PS→D · PD→S · PI→T

(6.21)

If there is information sharing between the reconnaissance entities, the number of edges in operation loop will increase, and the information accuracy is improved after information sharing. Therefore, The information sharing and auxiliary decision links can be taken as parallel system, which is shown in Figure 6.21(a). In the similar way, if there is a cooperative command relationship between decision-making entities, the number of equipment and edges will be increased. The cooperation command and control process improves not only the accuracy and real-time performance, but also the flexibility and effectiveness. Therefore, the command relationship between decision equipment and influence equipment and collaborative relationship among decision equipment can also be taken as parallel system, which is shown in Figure 6.21(b). According to features of parallel system, the equivalent decision efficiency can be obtained by merging the information sharing among reconnaissance equipment into the decision-making process. 0

N

i PS→D = 1 − (1 − PS→D ) · ∏(1 − PS→S ) i=0

(6.22)

236

Deping Zhang and Xuefeng Yan

Figure 6.21. The equivalent structure of the complex loop. i where N represents the number of collaborative command and control edges. PS→S represents the synergetic effect of the decision-making edge i. When i = 0, it means that there is 0 no cooperative command and control in the operation loop. That is PS→S = 0, which also returns to the standard operation loop. The cooperative command and control relationship of decision-making equipment is merged into command relationship, according to the calculation of parallel system, the equivalent decision effect can be obtained as follow: N

0

i PD→I = 1 − (1 − PD→I ) · ∏(1 − PD→D )

(6.23)

i=0

i where N represents the number of cooperative command and control links. PD→D represents the cooperative effect of decision node i. When i = 0, it means that there is no coordinated 0 command and control in the operation loop. That is PD→D = 0, which also returns to the standard operation loop. In summary, there are two relationships of information sharing and cooperation command control in operation loop. The operation capability index is shown as follows: 0

0

Eop = PT →S · PS→D · PD→S · PI→T

(6.24)

For the SoS effectiveness evaluation, the equipment is connected through interaction during the process of operation. The equipment and the connection between them constitutes the operation loop. Different operational loops are interlaced with each other and then a complex SoS network is finally formed. Its evaluation is very important to optimize the configuration of equipment. (4) Capability Evaluation of Multiple Operation Loops In general, the more the operation loops are, the more ways to complete mission, and the stronger the invulnerability and capability of SoS are, and the higher the completion rate is. Therefor the SoS can be comprehensively evaluated by operational capability E f and operation loops Nop .

Mission Oriented Effectiveness Evaluation

237

(i) Operational capability calculation Many operation loops may share the same blue targets in the network. Assuming that each equipment entity can only respond to the information of one operation loop at the same time. The operational capability E f can be represented by the maximum capability in all loops, which is shown as follows: E f = max{Eop1 , Eop2 , · · · , Eopn }

(6.25)

When the E f is strong, it shows that the probability of mission completion is greater and it is easier to destroy the blue targets. (ii) Calculation of the number operation loops The number of operation loops can be specifically calculated by the power of the adjacency matrix A which represents the connection relationship of network nodes. Firstly, the adjacency matrix A = (ai j )N×N of the network is defined. If there are N equipment nodes in the SoS network, the adjacency matrix is a matrix of order N. The elements are defined as follows:  1, has an arc between a node i and j (6.26) ai j = 0, has no arc between node i and j (k)

According to the power operation of adjacency matrix, each non-zero element ai j of the matrix Ak indicates that there is a direct path with length k from node i to j. Diagonal (k) element aii represents the number of loops with a length of k passing through i. Therefore, the number of operation loops is shown as follows: N

∞

(k)

Nop = ∑ ∑ aii

(6.27)

i=1 k=1

The formula is simple, but there is still a same loop that is repeatedly calculated i and the loop doesn’t contain blue targets. In order to obtain the correct number, the method is improved. k = k0 = 4 is initialized since the minimum length of operation loop is 4. The number of operation loops with different length is calculated first and the judgement whether to the same loop is repeatedly calculated is added. If it is satisfied, the computing process should be stopped. The maximum length of operation loop is k − 1. The detail of the calculation process is shown in Figure 6.22. If the number of operation loops Nop is large, there will be multiple choice for operation mode. Even though one or more loops are destroyed, the operation can still continue, that is, the SoS has strong invulnerability. (iii) Multiple mission-oriented SoS evaluation For mission Mi , there are many operation loops. The more number of operation loops are, the more attacking methods against the blue targets are, the stronger the operational capability is. Therefore, the operational capability of mission-oriented SoS is shown as follows: Nop

E Mi = ∑ Eopi i=1

(6.28)

238

Deping Zhang and Xuefeng Yan

Figure 6.22. Calculation process of the number of operation loops. There are many kinds of missions for SoS. Assuming that M1 , M2 , · · · , ML represent the operation missions and p1 , p2 , · · · , pL represent the probability of missions occurrence in operation. Then the operational effectiveness is as follows: L

E f = ∑ pi E Mi

(6.29)

i=1

where L is the number of operation missions. The SoS effectiveness evaluation process for specific operation loop is shown as follows: Step 1. Construction of the network topology and index system. The nodes and edges in operation loops are abstracted and modeled according to the operation activities. Therefore, the capability index system of whole operation network can be constructed. Step 2. Calculation of the operation effectiveness of each operation link. If the operation link r is supported by n capabilities, different capabilities have different degree of influence on the whole operation process (one operation loop). If the weight of capability i is wi , the effectiveness value can be calculated by Equation (6.20). Step 3. The integrated effectiveness of each operation loop is calculated by Equation (6.21) or Equation (6.24) based on the effectiveness of each operation link. Step 4. Considering the cooperation combat of multiple operation loops, the operational capability of the whole SoS in a specific mission can be calculated by Equation (6.28). Step 5. Based on the effectiveness of all operation missions, the effectiveness evaluation of multi operation missions can be obtained by weighted aggregation of the Equation (6.29).

6.2.3.

Case Study

This chapter takes a missile SoS as an example to illustrate the feasibility and validity of the proposed method. The modeling is constructed based on the principle of operation loop,

Mission Oriented Effectiveness Evaluation

239

and the effectiveness of the whole operational activity is evaluated. The index of a missile SoS is constructed, and the relevant parameters are set, so as to carry out the case study. According to the mission requirements, the operational process is summarized into three main stages including detection and early warning detection, operation decision and combat operation, as shown in Figure 6.23.

Figure 6.23. Operational process of the missile equipment SoS. 1) Emergency detection stage. The main task at this stage is to transmit information to the military intelligence center, as well as the joint operation command center. 2) Operation decision stage. The main task at this stage is to make the operation plan and decision. 3) Combat operation stage. The main task this stage is to make the operation scheme and attack on the blue targets. Combined with the specific task of each stage in operation, the main equipment involved in the missile operation are scheduled. There are five kinds of detection and early warning surveillance including radar S1 , radar S2 , radar S3 , space-based surveillance system S4 , and intelligence center D1 . There are three decision control nodes including general command center D2 and operational command center D3 , D4 . The attack nodes include 6 attack units which are equipped with M weapon. It is considered that the attack units I1 , I2 are directly controlled by the operational command center. I3 , I4 are controlled by the operational command center D1 and I5 , I6 are controlled by the operational command center D2 . The target nodes include four important blue targets. The emergency detection system is used to discover and identify the incoming blue target T1 . The attack units I1 , I2 are used to against the target T2 ; I3 , I4 are used to attack the target T3 ; I5 and I6 are used to strike the target T4 . Each equipment is abstracted as a node and the operational activities between equipment are abstracted as edges. The Figure 6.24 shows a certain type of operational network model of the SoS.

240

Deping Zhang and Xuefeng Yan

Figure 6.24. Operational network. Table 6.4. Emergency detection activity T → S Operational activities

Detection distance

T1 → S1 T1 → S2 T1 → S3 T1 → S4

3 800 2 000 3 200 4 000

Recognition probability 0.6 0.9 0.6 0.9

Antiinterference 0.6 0.8 0.65 0.55

Table 6.5. Information transmission activities between S equipments Operational activity S1 → S2 S3 → S2

Transmission rate 1.4 1

Channel width 2.4 2

Bit error rate 0.05 0.1

There are six kinds of operational activities in the above model. The key indexes for an specific operational activity are taken as the main evaluation objects. The evaluation indexes for the above operational activities are as shown in Table 6.4 to Table 6.8. In order to avoid qualitative indexes, only the communication connection between the command decision node and the attack node is considered here. The weight of descriptive indexes for operational activities are calculated, as is shown

241

Mission Oriented Effectiveness Evaluation Table 6.6. Information sharing activities in S → D Operational activity S 1 → D1 S 2 → D1 S 3 → D1 S 4 → D1

Transmission rate 1.6 1.2 1.4 1

Channel width 3 2 2.4 2

Bit error rate 0.1 0.05 0.05 0.1

Table 6.7. Information transmission activities between D equipments Operational activity D1 → D2 D2 → D3 D2 → D4

Transmission rate 1.6 1.4 1.4

Channel width 3 2.4 2.4

Bit error rate 0.05 0.1 0.1

Table 6.8. Information sharing activities in D → I Operational activity I1 → T2 I2 → T2 I3 → T3 I4 → T3 I5 → T4 I6 → T4

Transmission rate 550 450 500 400 450 500

Channel width 90 % 85 % 90 % 90 % 95 % 80 %

Bit error rate 300 300 200 200 350 350

Table 6.9. Indexes weights of the edge relationship in the operation loop Operational activities T →S S→S S→D D→D D→I I→T

Index 1 weight 0.324 0.33 0.4 0.33 0.33 0.326

Index 2 weight 0.38 0.33 0.375 0.33 0.33 0.352

Index 3 weight 0.296 0.34 0.225 0.34 0.34 0.322

in Table 6.9. (k) According to the power operation of the adjacent matrix, each non-zero element ai j in the matrix Ak indicates that there is a directed path from node i to j with a length of k in the (k) network. The diagonal element aii represents the number of loop of length k through node

242

Deping Zhang and Xuefeng Yan Table 6.10. Indexes weights of the edge relationship in the operation loop

NO.

Operational loop

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

T1 → S1 → D1 → D2 → I1 → T2 T1 → S1 → D1 → D2 → I2 → T2 T1 → S1 → D1 → D2 → D3 → I3 → T3 T1 → S1 → D1 → D2 → D3 → I4 → T3 T1 → S1 → D1 → D2 → D4 → I5 → T4 T1 → S1 → D1 → D2 → D4 → I6 → T4 T1 → S1 → S2 → D1 → D2 → I1 → T2 T1 → S1 → S2 → D1 → D2 → I2 → T2 T1 → S1 → S2 → D1 → D2 → D3 → I3 → T3 T1 → S1 → S2 → D1 → D2 → D3 → I4 → T3 T1 → S1 → S2 → D1 → D2 → D4 → I5 → T4 T1 → S1 → S2 → D1 → D2 → D4 → I6 → T4 T1 → S2 → D1 → D2 → I1 → T2 T1 → S2 → D1 → D2 → I2 → T2 T1 → S2 → D1 → D2 → D3 → I3 → T3 T1 → S2 → D1 → D2 → D3 → I4 → T3 T1 → S2 → D1 → D2 → D4 → I5 → T4 T1 → S2 → D1 → D2 → D4 → I6 → T4 T1 → S3 → S2 → D1 → D2 → I1 → T2 T1 → S3 → S2 → D1 → D2 → I2 → T2 T1 → S3 → S2 → D1 → D2 → D3 → I3 → T3 T1 → S3 → S2 → D1 → D2 → D3 → I4 → T3 T1 → S3 → S2 → D1 → D2 → D4 → I5 → T4 T1 → S3 → S2 → D1 → D2 → D4 → I6 → T4 T1 → S3 → D1 → D2 → I1 → T2 T1 → S3 → D1 → D2 → I2 → T2 T1 → S3 → D1 → D2 → D3 → I3 → T3 T1 → S3 → D1 → D2 → D3 → I4 → T3 T1 → S3 → D1 → D2 → D4 → I5 → T4 T1 → S3 → D1 → D2 → D4 → I6 → T4 T1 → S4 → D1 → D2 → I1 → T2 T1 → S4 → D1 → D2 → I2 → T2 T1 → S4 → D1 → D2 → D3 → I3 → T3 T1 → S4 → D1 → D2 → D3 → I4 → T3 T1 → S4 → D1 → D2 → D4 → I5 → T4 T1 → S4 → D1 → D2 → D4 → I6 → T4

Operational loop effectiveness eMi 49.97 49.87 63.10 63.02 63.20 63.17 56.83 56.73 69.96 69.88 70.06 70.03 44.24 44.14 57.37 57.29 57.47 57.44 56.60 56.50 69.73 69.65 69.83 69.80 48.22 48.13 61.35 61.28 61.46 61.43 40.75 40.66 53.88 53.81 53.99 53.96

probability occurrence 0.025 0.017 0.018 0.023 0.032 0.021 0.011 0.026 0.018 0.033 0.022 0.015 0.030 0.017 0.029 0.015 0.023 0.021 0.037 0.018 0.019 0.023 0.028 0.013 0.037 0.013 0.018 0.019 0.024 0.029 0.025 0.017 0.031 0.028 0.026 0.028

of

i. Therefore, the number of operation loop is N

∞

(k)

Nop = ∑ ∑ aii = 36 i=1 k=1

It can be seen, the total number of closed operational loop is 36.

(6.30)

243

Mission Oriented Effectiveness Evaluation

The weights of the operational loop edges could be obtained according to the effectiveness evaluation index system, combined with expert opinions. The edge effectiveness indexes of different operational activities could be obtained by constructing an analysis model or simulation model for each operational activity. For each connected edge in the operation loop, the effectiveness could be calculated by multiplying its weight and the edge effectiveness index of the loop. In this specific operation loop, three cases are taken into account, that is, the total number of the edges is 5, 6 and 7, respectively. It is necessary to calculate the actual effectiveness of the equipment participating in a single operation loop, then the pros and cons of the equipment in operational loop can be compared and analysed. The Equation (6.21) and Equation (6.24) are used to calculate the actual effectiveness of operation loop in the 3 operations loops and estimate the occurrence probabilities of each operational loop for different mission, as is shown in Table 6.10. The effectiveness value of SoS can be calculated as 57.8 by the Equation (6.29). For the missile SoS, according to the different number of edges, when the numbers of the operation loops edges are 5, 6, 7, the effectiveness of attack nodes I1 , I2 , I3 , I4 , I5 and I6 can be calculated according to the effectiveness weighted averages of the operational activities. The detail is as follows. Table 6.11. Effectiveness of attack nodes 5 edges loop 6 edges loop 7 edges loop

I1 3.1 3.1 3.1

I2 3.05 3.05 3.05

I3 0 2.94 2.94

I4 0 2.91 2.91

I5 0 3 3

I6 0 2.98 2.98

It can be shown from the Table 6.11 that the number of the attack node in the operational loop is the same, but the effectiveness is different because of the tactical indexes. The effectiveness of attack node I4 is relatively low because its hit accuracy and probability are low compared with other equipment. Therefore I4 is one of the weak equipment and is necessary to improve the index 1 and index 2 of the attack and confrontation activities. From the analysis, the operational effectiveness of equipment in SoS is related to the quantity of the operation loop and corresponding tactical and technical indexes. The more the number of operation loop involves, the better the tactical and technical indexes are, and the stronger the operational effectiveness is. Moreover, the effectiveness of the same equipment is different in operational activities. Therefore, the operational activities with weak effectiveness should be optimized from two aspects. One is to improve the structure of the SoS, another is to promote the equipment tactical and technical indexes.

Chapter 7

Sensitivity of Operational Effectiveness 7.1. Sensitivity Analysis Based on Range Analysis 7.1.1.

Principles

The purpose of the sensitivity analysis is to determine the influence of the agent model variables on the output [194]-[197]. It can make clear that the agent model is sensitive to which input variables. Then the influence of each input variable on the model can be obtained. The following formula denotes the mathematical expression of the operational effectiveness index for the SoS: ~y = (y1 , y2 , · · · , yn )T

(7.1)

The performance parameters are expressed as follows: ~x = (x1 , x2 , · · · , x j )T

(7.2)

Then the agent model used to describe the relationship between the effectiveness index ~y and the performance index ~x is shown as follows: yi = f i(~x) = f i(x1 , x2 , · · · , x j ), i = 1, 2, · · · , n

(7.3)

It is assumed that the normal weight of each system parameter is ~x0 and the normal system output is ~y0 . When the system parameters are perturbed, it is as follows: xi = xi0 + ∆xi , i = 1, 2, · · · , j

(7.4)

The corresponding output is changed as follows: yi = =

f i (x10 + ∆x1 , x20 + ∆x2 , · · · , x j0 + ∆x j )

f i (~x0 + ∆~x), i = 1, 2, · · · , n

(7.5)

The results obtained by Taylor expansion are shown below. ∆yi =

d f i (~x0 ) 1 d 2 fi (~x0 ) 2 1 d n fi(~x0 ) n ∆~x + ∆~ x + · · · + ∆~x + Rn (~x), i = 1, 2, · · · , n dx0 2! dx20 n! dxn0

(7.6)

246

Deping Zhang and Xuefeng Yan Then the sensitivity vector ~Si of yi near ~x0 can be expressed as follows: ~Si = d f i (~x0 ) , i = 1, 2, · · · , n dx0

(7.7)

Equation (7.7) gives the parameter sensitivity expression in analytical form, but it cannot be directly applied to a non-continuous, non-analytic system. The discrete approximation formula of parameter sensitivity is derived by range analysis. The range Rij for the parameter x j is calculated as follows: Rij = max{y¯ij1 , y¯ij2 , · · · , y¯ijk , · · ·} − min{y¯ij1 , y¯ij2 , · · · , y¯ijk , · · ·}

(7.8)

where y¯ijk represents the mean value of output of n tests at the k level of the parameter x j . The calculation is shown as follows: y¯ijk =

1 n i ∑ y jkm n m=1

(7.9)

where yijkm represents the output yi of the experiments m when the parameter x j is at the level k. The range Rij represents the effects of the parameters on the output. It can reflect the sensitivity of the output to the parameters qualitatively. It also can be used as the robustness index for effectiveness evaluation. But the quantitative sensitivity should be considered based on the two aspects of the range and the parameters. The sensitivity calculation is shown as follows: Sij =

Rij x j max − x j min

(7.10)

where x j max represents the x corresponding to the maximum yi , x j min represents the x corresponding to the minimum yi . Sij indicates the sensitivity of the performance yi to the parameter x j . The higher the value is, the more sensitive the parameter x j is. In addition, the symbol Sij indicates the sensitive direction of the parameter yi to x j . If Sij < 0, yi is negatively sensitive to x j . If Sij > 0, then yi is positively sensitive to x j . In addition to the sensitivity of single performance to single parameter, the output is also affected by the coupling effects between different parameters. Therefore, an approximate formula for the coupling sensitivity is given here. The coupling range of the output yi on xi and x j to the coupling range Rijk is defined in the formula as is given below. Rijk

1 1 i i i i = (∑ y jh,kh + ∑ y jl,kl ) − (∑ y jh,kl + ∑ y jl,kh ) 4 4

(7.11)

where h indicates the maximum value of j or k. l indicates the minimum value of j or k. yijh,kh represents the output yi corresponding to the parameters xi and x j at the highest level. yijl,kl represents the output yi corresponding to the parameters xi and x j at the lowest level.

Sensitivity of Operational Effectiveness

247

The coupling range Rijk denotes the effect of the coupling between the parameters xi and x j on the output yi . It can reflect the sensitivity of the output to the parameter coupling qualitatively. The calculation of the coupling sensitivity is shown as follows: Rij,k Sij,k = p (x j max − x j min )2 + (xk max − xk min )2

(7.12)

where x j max value represents the maximum value of the parameter x j . x j min represents the minimum of the parameter x j . xk max and xk min is the same as the parameter above. Sij,k indicates the coupling sensitivity of the performance yi to the parameters x j and xk . The higher the value is, the higher the coupling sensitivity of yi to the parameter x j and the parameter xk is, and vice versa. Sij,k facilitates the horizontal comparison between different coupling sensitivities, and it also helps to determine the order of the influence by the parameter coupling.

7.1.2.

Range Sensitivity Analysis Based on Agent Model

The agent model is constructed in a bottom-up data-driven approach. In general, it is assumed that the exact internal process of the original model is unknown (and sometimes may be known). However, the input-output behavior of the model is very important. The response (output) of the original model is calculated by selecting limited input carefully. The input-output pairs are regarded as modeling data. The agent model is built by the means of fitting (such as machine learning) or the interpolation algorithm. Therefore, from the view of mathematics, the agent model actually uses the sample points to construct a function and predict the response of the unknown point through the fitting or the interpolation algorithm. The process of the range sensitivity analysis based on agent model is as follows: (1) Construct Samples Set Based on the Effectiveness Index System The operation feature is analyzed aiming at the mission of the equipment system. The construction pattern for effectiveness index system is selected and the index system is constructed. Each index and sub-index of index system are to be modeled. If the correlation between performance index and parameter index can be clarified, the mathematical and physical model and simulation model can be established. If it is not clear, the index aggregation method can be used to do modeling such as AHP, ANP, ADC, etc. (2) Construct Training Set Based on the Samples The value range and rule for each effectiveness index is determined. The suitable sampling method is chosen to generate N group of input variables. The index is calculated to obtain corresponding machine learning label. Then, the initial N group of samples set is obtained. Furthermore, the sample set is preprocessed. Each qualitative performance index in the sample set is processed quantitatively by different methods, such as the utility function, the fuzzy mathematic, the rough set, etc. After dimensionless process, the sample value is normalized in the range [0,1]. The non-linear S-type derivable function is adopted in normalization according the physical significance of the performance parameters to emphasize the saturation characteristics. On the one hand, a certain performance parameter

248

Deping Zhang and Xuefeng Yan

has the limitation in terms of physical significance or current technical realization capability. On the other hand, the benefits can be brought from improving a certain performance, which also has the trend of S-type curve essentially. Therefore, the evaluation model can be normalized by S-type curve. f (x) =

1 1 + Te−Ux

(7.13)

where T,U are the parameters for the adjustment curve. Thus the normalization criterion of each index is determined. For the benefit-type index, x¯i =

1 n o xi max +xi min 6 1 + exp − xi max −x × [x × ] i 2 i min

(7.14)

where x¯i is the normalized value of each index, xi represents the value of the original index. xi max and xi min are the maximum and the minimum values of the index parameters respectively. The denominator is xi min − xi max for the cost-type index. (3) Construct Agent Model There are many agent models in equipment effectiveness model analysis,such as the Kriging, the Radial Basis Function (RBF) interpolation, the RSM polynomial response surface model, the BP neural network, the support vector machine (SVM) and so on. The initial generated samples are used to train the agent model. If the model’s fitting accuracy reaches the preset threhold, the training stopped. The fitting accuracy is computed by the root mean square error (RMSE), is shown as follows: s 1 n (7.15) RMSE = ∑ (yi − yˆi )2 n i=1 where yi is the true value and yˆi is the fitted value. Therefore, the agent model can be constructed for the effectiveness sensitivity analysis. (4) Sensitivity Analysis Based on Range Analysis According to the agent model and the orthogonal testing, the sensitivity analysis process can be summarized as follows: 1)The interested objective function, constraint function and the design variables involved in the analysis need to be determined at first. 2) The appropriate two-level orthogonal table is selected according to the number of variables (factors) in the analysis. If the variable number is n, the selected orthogonal table La (2c) should satisfy c ≥ n. And the constructive models is shown as follows:  a = 4×i (7.16) c = a−1 where c = 4 × i − 1, i ≥ (n + 1)/4, i ≥ 0. For example, when the variable number n is 14, i = 4, the orthogonal table should be selected as L16 (215 ).

Sensitivity of Operational Effectiveness

249

3) After each variable (factor) level is determined, the scheme can be designed according to the selected orthogonal table, similar to the experimental scheme in the orthogonal experiment. 4) The values of objective and (or) the constraint functions for different design schemes are calculated by using the machine learning-based agent models , such as SVR, BP, GMDH, etc. 5) For each variable in each level, the mean values y¯ jk ( f ), y¯ jk(gi ) of the objective and constraint functions are calculated. 6) The objective function, the range Rij of constraint function and coupling range Sij,k are calculated based on Equation (7.8) and Equation (7.11). 7) The results are analyzed by the range analysis. According to y jk , the restraint of each variable on the objective function and constraint function can be judged and its monotonicity can be obtained. The sensitivity Sij and the coupling sensitivity Sij,k of the objective and constraint function to each design variable are calculated based on Rij and Sij,k . Based on the sensitivity analysis, the influence of each index or multiple indexes on the effectiveness can be calculated. And the bottleneck factors can be found, which is used to construct the effectiveness evaluation index system and optimize the top-level design scheme.

7.1.3.

Case Study

In this section, the anti-ship missile effectiveness evaluation will be taken as an example to illustrate the sensitivity analysis based on the agent model. Firstly, the suitable evaluation indexes and influence factors are selected to construct the SVR model of the anti-ship missile. Secondly, the SVR model is trained according to the existing data of range test or simulation experiment. Meanwhile, the parameters of SVR are refined to obtain the best performance of the agent model. Finally, the sensitivity analysis is carried out by the range analysis based on the SVR-based agent model. (1) Selection of the Evaluation Indexes and the Influencing Factors Based on the operation objectives and task of the anti-ship missile, the operation phases include launching, cruising and the self-seeking attack. With the support of the informationized SoS, the reliable launch and stealthy engagement tasks can be easily completed by the anti-ship missile. However, the search, track and attack targets mission need to be completed in the self-seeking phase. This phase is short and fast-paced and it is difficult for the informationized SoS to support efficiency in the assault process. The anti-ship missile is facing a huge threat and is easily intercepted by the blue anti-missile defense system. Therefore, the anti-ship missile assault capability researched in this section focuses on the self-seeking attack phase, is shown as follows. The seeker is turned on to search the target after self-controlled flight. After suspected target is found, it will be identified. And the final attack target will be selected. With the guidance of the seeker, the missile penetrate the defense system of the surface ship using its tactical capability and hit the target eventually. Generally, the distance to turn on seeker of anti-ship missile is 30 ∼ 70 km. At this time, the anti-ship missile may enter the effective anti-air damage area of blue surface ship. And it will face the fire strikes from far, medium, close distance and the interference from

250

Deping Zhang and Xuefeng Yan

the active and passive equipment. In this section, the affecting indexes of the anti-ship missile, including the capture target, the penetration, the hit and damage are summarized. The missile penetration process is analyzed in detail and the main factors affecting the subindexes are further proposed. There are several factors that affect the target capture, such as self-controlled terminal dispersion error of anti-ship missile, the coverage capability of searching sector, the target discovery, the identification and the selection capabilities. The smaller the dispersion error is, the larger the sector coverage is, and the stronger the target discovery, identification and selection capabilities are. Then the target capture capability of is stronger. The factors that affect the penetration capability of the missile include the stealth performance, the penetration capability of shipborne air defense fire system, the active and passive interference. The factors that affect the hit capability includes the radar tracking capability, the maneuverability and the anti-target tactical evasion capability. The main factors affecting the damage capability are the warhead type, the working model and the encounter position of the missile, and the parameters of the blue ship. Since the anti-ship missile attack is a dynamic confrontation process, the reactions and countermeasures of the blue surface ships play a key role in penetration. The parameters of the enemy ship include the radar detection range, the reaction time, and the single hit probability of intercept missile. The penetration effectiveness evaluation index system of the anti-ship missile is established based on these influencing factors, which is shown in Figure 7.1.

Figure 7.1. The penetration evaluation index system for the anti-ship missile.

251

Sensitivity of Operational Effectiveness (2) Construction of the Training Set

SVR is used to evaluate the operational effectiveness of anti-ship missile. It can be trained by test data samples. Due to lack of a large number of anti-ship missile range test data, the experiment data are generated by self-developed simulation, which are used for training and verification. The system is composed by various components, such as the simulation engine, radar, ship, and anti-ship missile.In this section, 500 different factor sets of anti-ship missile effectiveness index are selected and analyzed statistically and the effectiveness value of each intermediate node is calculated. The effectiveness value of antiship missile penetration operation is calculated by using the following formula: β

γ

ζ

α δ E = ξ ·CAcquisition ·CPenetration ·CHit ·CDamage ·CEemy ship

(7.17)

where, ξ is the correction coefficient. The weight indexes are represented by α, β, γ, δ, ζ, which are determined by the their fucnction in the anti-ship missile assault process. In this case, ξ = 1.2, α = 0.18, β = 0.22, γ = 0.2, δ = 0.26, ζ = 0.14. After quantitative and normalized processing of qualitative indexes on obtained 500 samples, 340 samples are selected as the training set for the SVR and the remains are used as the verification set. The corresponding samples are shown in Table 7.1. Table 7.1. Training Sample Dataset x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 x12 x13 x14 x15 x16 x17 x18 1 0.31 0.92 0.95 0.98 0.91 0.93 0.95 0.73 0.92 0.67 0.56 0.82 0.84 0.68 0.88 0.71 0.63 0.75 2 0.43 0.89 0.93 0.90 0.87 0.90 0.93 0.81 0.88 0.73 0.67 0.76 0.79 0.71 0.79 0.67 0.68 0.83 3 0.42 0.95 0.97 0.93 0.86 0.85 0.88 0.67 0.89 0.81 0.55 0.83 0.77 0.78 0.85 0.83 0.52 0.88 ··· ··· ··· ··· ··· ··· ··· ··· ··· 500 0.33 0.96 0.98 0.95 0.95 0.95 0.92 0.91 0.87 0.79 0.71 0.73 0.83 0.86 0.91 0.78 0.73 0.87

Through the simulation, the intermediate index variables from the statistical analysis and the final effectiveness obtained by calculation are shown in Table 7.2. Table 7.2. The anti-ship missile sub-index and penetration operation effectiveness Order number 1 2 3 ··· 500

Acquisition Capability

Target

Hit Capability

Damage Capability

Capability 0.81 0.83 0.92

0.95 0.93 0.97

0.98 0.90 0.93 ··· 0.95

0.76

Penetration

0.92 0.89 0.95 ··· 0.96

0.98

Blue Ship

Operational

Information Effectiveness 0.91 0.87 0.86 0.95

E 0.93 0.90 0.85 ··· 0.95

252

Deping Zhang and Xuefeng Yan (3) Construction of the Agent Model

The self-developed effectiveness evaluation analysis platform is used to train and predict the SVR in the experiment. The machine learning algorithms in the platform including SVM, BP, ELM have the features of the fast speed and high calculation accuracy. In order to obtain the suitable parameters for SVR of anti-ship missile effectiveness evaluation, 340 samples are selected to train. The mean square error (MSE) and the average absolute error (MAE) are used to evaluate the fitting effect of SVR. Based on the fun-tuned SVR, 160 samples can be used to verify the agent model. Then the error can be calculated and the prediction error of the model can be obtained. The results and the errors of each sample are shown in Figures 7.2(a) and (b) respectively.

Figure 7.2. The Training Error RMSE of the SVR. (4) Sensitivity Based on Range Analysis From the above evaluation scheme, 19 factors and 5 levels for each factor are considered in this case. Therefore, the orthogonal table L25 (519 ) is adopt to design the sensitivity analysis scheme. And the anti-ship missile operational effectiveness for each scheme can be calculated based on SVR. The factor in each level, the average vale, the range y jk and coupling range Sij,k are calculated. At the optimal level, the monotonicity and the order of the sensitivity of each factor of the system effectiveness can be judged based on the average value. The sensitivity and coupling sensitivity of the objective function and constraint function on different variables can be calculated based on the y jk and Sij,k , as shown in Figure 7.3 (a) and (b).

Figure 7.3. The anti-ship missile operational effectiveness sensitivity analysis.

253

Sensitivity of Operational Effectiveness

7.2. Global Sensitivity Based on Sobol’s 7.2.1.

Basic Principle of the Sobol’s

The sensitivity analysis has the following characteristics. 1) It studies the global impact of various factors on the model. 2) The range of factors can be extended to the entire definition domian. Various factors can change at the same time, which can be used to research the non-linear, non-superimposed, non-monotone models. At present, the Sobol’s is the most common global sensitivity analysis method [198]-[206]. The basic idea of the Sobol’s is the variance decomposition. First, the studied model is decomposed into functions composed by single input variables and multiple input variables. Then, the sensitivity coefficient is obtained by calculating the influence of variance introduced by these single input variables or multiple input variables on the total variance. The Sobol’s sensitivity analysis is a variance-based Monte Carlo method. The k dimensional unit Ωk is defined as the input factor of the spatial domain, which is expressed as follows: Ωk = {x|0 ≤ xi ≤ 1, i = 1, 2, · · · , k}

(7.18)

The key of the Sobol’s is to decompose the function f (x) into the sum of sub-terms. k

f (x1 , x2 , · · · , xk ) = f 0 + ∑ fi (xi ) + i=1

∑ 1≤i< j≤k

fi j (xi , x j ) + · · · + f 1,2,··· ,k (x1 , · · · , xk )

(7.19)

There are 2k sub-items in Equation (7.19). At present, the frequently used decomposition method is a multi-integrals based method proposed by Sobol ’in 1990. The characteristics of this decomposition method are shown as follows: (1) f 0 is the constant term. The integral of each sub-term to any factors is zero. That is Z 1 0

fi1 ,i2 ,··· ,is (xi1 , xi2 , · · · , xis )dxi j = 0, (1 ≤ j ≤ s)

(7.20)

(2) The sub-terms are orthogonal. If (i1, i2 , · · · , is) 6= ( j1 , j2 , · · · , jl ), there will be, Z 1 0

fi1 ,i2 ,··· ,is · f j1 , j2 ,··· , jl dx = 0

(7.21)

(3) The decomposition form of Equation (7.19) is unique and each order sub-term can be calculated by the multi-integrals. For example, f0 =

Z

Ωk

f (x)dx

fi (xi ) = − f 0 +

Z 1 0

···

(7.22) Z 1 0

f (x)dx(−i) (1 ≤ i ≤ k)

fi j (xi , x j ) = − f 0 − f i (xi ) − f (x j ) +

Z 1 0

···

Z 1 0

(7.23)

f (x)dx(−i j) (1 ≤ i < j ≤ k) (7.24)

where x(−i) and x(−i j) represent the input factors except xi , except xi and x j respectively. Similarly, the other higher-order sub-terms can be obtained.

254

Deping Zhang and Xuefeng Yan

According to the statistics, the total variance of the model output f (x) is shown as follows: D=

Z

Ωk

f 2 (x)dx − f 02

(7.25)

The variance of each sub-term in the Equation (7.19) is called the partial variance, that is the s order partial variance. Di1 ,i2 ,··· ,is =

Z 1 0

···

Z 1 0

fi21 ,i2 ,··· ,is (xi1 , · · · , xis )dxi1 · · ·dxis (1 ≤ i1 < · · · < is ≤ k)

(7.26)

Equation (7.19) is squared and calculated the integral on overall Ωk . The relationship between the total variance and the partial variance can be obtained with Equation (7.21). The total variance is equal to the sum of each order partial variance. k

D = ∑ Di + i=1

∑ 1≤i< j≤k

Di j + · · · + D1,2,··· ,k

(7.27)

The sensitivity coefficient of each order is defined as the ratio of the partial variance to the total variance. The s order sensitivity Si1 ,i2 ,··· ,is is defined as follows: Si1 ,i2 ,··· ,is =

Di1 ,i2 ,··· ,is (1 ≤ i1 < · · · < is ≤ k) D

(7.28)

where Si is called the first-order sensitivity coefficient of the factor xi , which represents the main influence of xi on the output. Si j (i 6= j) is the second-order sensitivity coefficient, which represents the mutual influence between two factors. Similarly, S1,2,···,k is the k-order sensitivity, which means the mutual influence between k factors. Based on the Equation (7.27), it can be seen that, k

∑ Si + ∑

i=1

1≤i< j≤k

Si j + · · · + S1,2,··· ,k = 1

(7.29)

In the Sobol’s, each integral can be calculated by the Monte Carlo. Therefore, f i , D and Di can also be estimated. fˆ0 =

1 n ∑ f (xm ) n m=1

(7.30)

Dˆ =

1 n 2 ∑ f (xm ) − fˆ02 n m=1

(7.31)

Dˆ i =

1 n (1) (1) (1) (2) ∑ f (xim , x(−i)m ) f (xim , x(−i)m) − fˆ02 n m=1

(7.32)

1 n (1) (1) (1) (2) ∑ f (xi jm , x(−i j)m) f (xi jm , x(−i j)m ) − fˆ02 n m=1 = Dˆ cij − Dˆ i − Dˆ j

Dˆ cij =

(7.33)

Dˆ i j

(7.34)

Sensitivity of Operational Effectiveness

255

where n is the sampling size. Superscript (1) and (2) in Equation (7.32) are the two n × k dimension arraies of the input variables (x1 , x2 , · · · , xk ). In effect, the parameter xi is sampled only once, while the others are sampled twice. Then, the two sets of sampling values are substituted into the model for calculation and the corresponding variance can be determined. According to Equation (7.32), Dˆ −i can also be calculated based on the following formula. 1 n (1) (1) (1) (2) Dˆ −i = ∑ f (x(−i)m , xik ) f (x(−i)k , xik ) − fˆ02 n m=1

(7.35)

Therefore, the total sensitivity coefficient of the parameter xi , SˆT (i) is shown as follows: Dˆ −i SˆT (i) = 1 − Dˆ

(7.36)

The Monte Carlo integral requires a large amount of samples to approximate the variance of each factor in Equation (7.19). It requires thousands of samples and a large amount of calculation. It is unpractical to calculate all the interaction effects among all the factors in turn. For the factors of the interaction effect are unconspicuous, it is not important to calculate their interaction influence. However, if only the main and total effect of factors are calculated, many important information will be lost. The significance of considering the sensitivity coefficients of Sobol’s is that the interaction effects between factors can be selectively analyzed. The threshold is introduced to make judgement. If the interaction effect is less than the threshold, the factor interaction effect is unconspicuous. Then, it is unnecessary to consideration and calculation. If the interaction effect is greater than the threshold, it can be calculated continually. Some redundant sampling calculation can be removed by adopting the threshold in the Sobol’s. At the same time, the result roughness can be controlled by setting the threshold, which makes the method more truthfulness. Based on the combination of the Monte Carlo integral and the threshold, the calculation steps for Sobol’s are shown as follows: Step 1. Two independent sampling.  x11 x12 x21 x22  A= . ..  .. . xn1 xn2

sample matrices A, B are generated based on the random   0 0  0 x11 x12 · · · x1k · · · x1k 0 0  x0 · · · x2k    21 x22 · · · x2k  ..  , B =  .. .. . . ..  ..  . . .  . .  . 0 0 0 · · · xnk xn1 xn2 · · · xnk

(7.37)

Each row of the matrix is a set of sampling of the model, where n is the sampling size and k is the number of factors. The matrices A, B are substituted into the simulation model to obtain the output vectors f (A), f (B), which are all n dimensional column vectors. Step 2. The column i of the matrix A is replaced with the column i of the matrix B to

256

Deping Zhang and Xuefeng Yan (i)

obtain the transformation matrix AB .  x11 x12 · · · x21 x22 · · ·  (i) AB =  . .. ..  .. . . xn1 xn2 · · ·

0

x1i · · · 0 x2i · · · .. . . . . 0 xni · · ·

 x1k x1k   ..  . 

(7.38)

x1k

(i)

The matrix AB is substituted into the simulation model to obtain the model output (i) vector f (AB ) with column n. Step 3. According to the improved Sobol’s method proposed by Saltelli et al., the calculation formula of the first-order sensitivity coefficient Si and the total sensitivity coefficient ST (i) is shown as follows: Si =

Di 1 ≈ D n

ST (i) = 1 −

n

(i)

∑ f (B) j( f (AB ) j − f (A) j )/D

(7.39)

j=1

D−i 1 ≈ D 2n

n

(i)

∑ ( f (A) j − f (AB ) j )/D

(7.40)

j=1 k

Step 4. All interaction effects ∑ Sin = 1 − ∑ Si are calculated and the threshold Ω is i=1

set. it is considered that there is significant interaction effect among system variables when ∑ Sin ≥ Ω. The difference between ST (i) and Si represents the impact of the interaction effect between the variable Xi and other variables on the effectiveness output. Let Sin i = ≥ Ω , it is considered that there is significant ST (i) − Si and the threshold Ωi is set. When Sin i i interaction effect between the variable Xi and other variables. Therefore, the variable Xi requires the interaction effect analysis. Step 5. It is assumed that the second-order sensitivity coefficient Si1 ,i2 = ST (i1 , i2 ) − Si1 − Si2 of the variables Xi1 , Xi2 needs to be calculated, where ST (i1 , i2 ) = n

(i i )

(i i )

( n1 ∑ f (A) j f (BA 1 2 ) j − f 02 )/D. BA 1 2 represents that the columns i1 , i2 of the matrix B are j=1

replaced with the columns i1 , i2 of matrix A. ST (i1 , i2 ) describes the impact of the variables Xi1 , Xi2 as the main effect on the model output. Step 6. The sum Si + ∑ Si j of the obtained first-order and second-order sensitivity coefficients is calculated. When Si + ∑ Si j ≥ Ω, it is unnecessary to continually analyze because the factors influencing the output have been analyzed. Otherwise, the third-order and above interaction sensitivity SI (i) − Si − ∑ Si j of variable Xi needs to be calculated and the thresh(2) (2) old Ωi is set. When SI (i) − Si − ∑ Si j ≥ Ωi , it is considered that there is unconspicuous interaction effect of the third-order and above of Xi . Then it need not to be taken into consideration. Step 7. The high order sensitivity coefficients that cannot be ignored need to be calculated. Continuously, until the sum of the each order sensitivity is bigger than the threshold.

Sensitivity of Operational Effectiveness

257

The Sobol’s can calculate the each order sensitivity coefficients of variables. However, it is limited by the computation quantity, integrated computations are impossible. Moreover, there are errors in the calculation results based on Monte Carlo. The relative error is large when the sensitivity coefficient is small. In general, only several effects of each variable need to be calculated, including the main effect, total effect, and second-order interaction effect. According to the practical requirements, it is necessary to calculate the high-order sensitivity coefficient. When the Sobol’s is applied to the SoS, the quantitative sensitivity analysis results are obtained and indicate the influence of the equipment and the performance on the effectiveness. The analysis have significant guiding significance for the military operations. The application of sensitivity analysis in the operational effectiveness is analyzed as follows: (1) In the analysis of the influence of variable Xi on effectiveness Y , the sensitivity coefficient related with Xi includes Si , ST (i) and Si j . ST (i) is the total effect of the variable Xi , including the main effect from Xi and all the interaction effect related with Xi . ST (i) should be taken as evaluation standard while sorting the importance of variables. If ST (i) is very small, it represents that the change of Xi not only has a small influence on the simulation output, but also has a small interaction effect with other factors. In the SoS simulation, a fixed value can set to the factors with small total effect index, so as to reduce the model variables and simplify the model. (2) Si j is used to measure the influence of interaction effect between the variables Xi and X j . If Si j is larger, it means that there is a strong interaction effect between the Xi and X j . Different numerical combinations of the two variables have a great impact on the system effectiveness. They need to cooperate with each other to play a better role in the combat. (3) If the values of Si j , Sik , Sil are large, the variable Xi is an important factor that affects the SoS effectiveness. The variables X j , Xk, Xl have a great impact on the performance of the corresponding equipment. (4) In the analysis of the impact of equipment sub-systems on the SoS effectiveness, that is the influence of single factor set on effectiveness, if the main effectiveness index of equipment is composed of r, s,t, it only needs to analyze S{r,s,t} = Sr + Ss + St + Srs + Sst + Srt + Srst . This expression describes the impact of the variables r, s,t on the model output, which can be regarded as the influence of the equipment sub-system on the overall effectiveness. The Sobol’s is independent and it is unrelated with the input form of the model. The sensitivity coefficient of the factor can be obtained quantitatively to analyze the main effect, total effect and interaction effect of each order. The limitation of the Sobol’s is that there are many sampling. It has a large amount of calculation when the simulation model is very complicated and contains many analysis factors. Therefore, the factor screening is usually adopted.

7.2.2.

Sobol’s-Based System Effectiveness Sensitivity Analysis

Based on the effectiveness sensitivity analysis of Sobol’s, the evaluation index system can be constructed according to requirements. The analysis model can be constructed based on the evaluation index system, such as the parametric model, the utility-based analysis model

258

Deping Zhang and Xuefeng Yan

or the agent model. The index system can be constructed based on the characteristics of the evaluation object and purpose. However, the traditional statistic and analysis method are not effective because of too many levels, diverse attributes, and commensurability in evaluation index. The effectiveness evaluation of the SoS is centered on operation; it is appropriate to adopt the multi-properties decision-making method based on machine learning algorithm or utility preference. For generalized research, it is assumed that there are m evaluation schemes for a given mission, such as the anti-submarine of submarine torpedo or submarine missile attack against warship. The evaluation index system of the SoS’s operational capability can be expressed by mathematical symbols, which is shown in Figure 7.4.

Figure 7.4. Evaluation index system based on the mathematical symbols. The utility preference-based SoS effectiveness evaluation is taken as an example. The utility aggregation can be used to construct capability evaluation model in the following form. The operational allocation scheme effectiveness is shown as follows: n

E a = ∑ wiUia

(7.41)

i=1

The utility function of each sub-index Uia is shown as follows: Uia = f i (xi1 , xi2 , · · · , xik ) = f i (xi1 ) · f i (xi2 ) · · · f i (xik ) where, for benefit-type indexes, the utility function is shown as follows: ( 0 0 0 eci (xik −Xik /Xik ) , 0 < xik < Xik fi (xik ) = 0 1, xik > Xik For the cost-type index, the utility function can be shown as follows: ( 0 0 0 eci (xik −Xik /Xik ) , 0 < xik < Xik fi (xik ) = 0 1, xik > Xik For the neutral-type index, the utility function is shown as follows: ( 0 0 0 eci (xik −Xik /Xik ) , 0 < xik < Xik fi (xik ) = 0 1, xik > Xik

(7.42)

(7.43)

(7.44)

(7.45)

Sensitivity of Operational Effectiveness

259

where ci is the constant coefficient that is related to index 1 and its defaults is 1. On this basis, the global sensitivity analysis of Sobol’s can be executed. Several properties of the sensitivity analysis should be determined at first, such as the parameters, the value range, and the probability distribution. And the Monte Carlo sampling is adopted to calculate the effectiveness evaluation of each sampling. Then, for the the constructed model, the global sensitivity analysis is performed by adopting the Sobol’s. It should determine the sensitivity coefficients of the first-order, the cross-term sensitivity coefficient, the total, and the contribution rate of each parameter. The first-order (main effect), cross-term (interaction effect) and total effect sensitivity coefficients reflect the influence of individual parameter, coupling factors and all factors on the model respectively. The contribution rate for each parameter can be obtained after normalizing the overall sensitivity of each parameter. Finally, the determined decision-making variable is taken as the foundation for the subsequent optimization of SoS. The specific sensitivity analysis process based on the Sobol’s is shown in Figure 7.5.

Figure 7.5. Global sensitivity analysis based on Sobol’s.

7.2.3.

Case Study

The submarine operational system is taken as an example in this section. The utility function is combined with the ADC to model each effectiveness sub-index and value. The submarine operational system is divided into the submarine platforms system and the weapon system. It also can be divided into different function modules according to different functions. The initial state of each system is fault or normal. The evaluation index of the submarine operational capability is shown in Figure 7.6. In the submarine effectiveness evaluation model, the utility function is combined with the ADC to comprehensively calculate the each sub-index and operational effectiveness indexes. The results are taken as the training set of operation agent model. The operational

260

Deping Zhang and Xuefeng Yan

Figure 7.6. Evaluation index of the submarine operational capability. effectiveness is taken as example to illustrate the sensitivity analysis when each system of the submarine works normally. Due to the large number of the effectiveness indexes, the attack capability is selected as the sensitivity analysis object in this experiment. There are six sub-indexes, which is shown in Table 7.3. According to the attributes of each index, the value range can be determined. Table 7.3. Sub-indexes of the attack capability

Launch speed (km/h) Range range (km) Guidance accuracy (0 ) Weapons quantity Weapon type Damage radius

Minimum 50 60 1 3 1 20

Maximum 120 110 40 10 3 60

Where the values of weapon type are discrete and corresponding to three different weapon types, 1, 2, 3. The agent models are adopted for comparative experiments, including the Latin hypercube sampling Extreme Learning Machine (L-ELM)[207]-[209], the Latin hypercube sampling BP (L-BP), the random sampling ELM (R-ELM), and the random sampling BP neural network (R-BP). The number of the hidden neurons is 20 and the activation function is sigmoid function. 90% of the samples and the remaining are taken as the training set and the testing set respectively. The sample size is 300 and two models are run respectively. The results are shown in Table 7.4. According to Table 7.4, Compared with the random sampling model, the Latin hyper-

261

Sensitivity of Operational Effectiveness Table 7.4. Experiment results of the agent model

Mean square error (MES) Training time (s) Prediction time (s)

L-ELM 0.0767 0.0012 0.0020

L-BP 0.1670 0.4047 0.0140

R-ELM 0.0822 0.0015 0.0030

R-BP 0.0941 0.3875 0.0148

cube sampling model significantly reduces the training and prediction time. Therefore, Latin hypercube sampling model training is reasonable. Figure 7.7 is the fitting effect of L-ELM and L-BP, the ADC curve is the real effectiveness value. The fitting effect is similar and the curves coincide basically between L-ELM and L-BP, but the L-ELM has better fitting effect than L-BP model at the extreme point.

Figure 7.7. Comparison of fitting results. Two groups of contrast tests were set to compare with the real sensitivity coefficient in efficiency sensitivity analysis, the sampling method is Sobol’s sequence with low difference and the sample size is 10000. The comparative experiments are as follows: one is sensitivity analysis based on L-BP, the other is L-ELM. The experimental results are shown in Table 7.5. According to Table 7.5, the sensitivity coefficient can be calculated based on the L-ELM is generally close to the real one. However, the L-BP is unstable. Some coefficients are very close to the true value such as S3 , which is the same as the true value. Some coefficients are far away from true value such as s4 , s5 . The total sensitivity coefficient of the attack capability index is shown in Table 7.6. From Table 7.6, the total sensitivity coefficient calculated based on the L-ELM is closer

262

Deping Zhang and Xuefeng Yan Table 7.5. Sensitivity coefficient of attack capability index

real value L-ELM L-BP

S1 0.0907 0.0850 0.0691

S2 0.0449 0.0478 0.0432

S3 0.4830 0.5155 0.4830

S4 0.0001 0.0004 0.0243

S5 3.95e-05 0.0002 0.0043

S6 0.2344 0.2449 0.1880

Table 7.6. The total sensitivity coefficient of the attack capability index

True value L-ELM L-BP

ST (1) 0.1391 0.1143 0.1494

ST (2) 0.0730 0.0819 0.1094

ST (3) 0.6004 0.5932 0.5771

ST (4) 3.09e-05 0.0079 0.1179

ST (5) 9.70e-05 0.0065 0.0875

ST (6) 0.3346 0.2999 0.2770

to the true value and the overall effect is better. However, the result of L-BP is quite different from the true value, especially ST (4) and ST (5) are significantly different from the true value. According to Table 7.5, the sensitivity coefficient Si is sorted in descending order, S3 > S6 > S1 > S2 > S4 > S5 . The indexes that affect the attack capability are obtained, which are sorted in descending order like guidance accuracy, damage radius, launch speed, range distance, weapon type and weapon quantity. According to Table 7.6, the total sensitivity coefficient ST (i) is sorted in descending order, ST (3) > ST (6) > ST (1) > ST (2) > ST (4) > ST (5). The degree of mutual influence among the indexes of attack capability are obtained, which are sorted in descending order like guidance accuracy, damage radius, launch speed, range distance, weapon quantity and weapon type. It is found that the key factors of the attack capability are the guidance accuracy, the damage radius and the launch speed. The capability can be improved by enhancing the corresponding indexes of the equipment. However, the changes of range distance, weapon quantity and weapon type have little influence on the effectiveness value and other indexes. Therefore, these indexes can be set as a fixed value to reduce the calculation cost and time.

Chapter 8

Contribution Effectiveness With the rapid development of equipment in Asia, the new operational system has been established and developed. Whether these operational systems can be use-friendly, effective, durable and practical needs to be evaluated. The US military mainly tests the operational effectiveness and adaptability of the equipment by constructing realistic operation environment, setting up appropriate opponents and simulating operational application and confrontation process. Therefore, the real operational capability of the new operational system and the degree to meet the battlefield requirement can be evaluated before using in practical battlefield. The purpose of suitability evaluation is to answer the question whether the operational system can satisfy the requirement in the battlefield, while the effectiveness evaluation answers whether the operational system can achieve the anticipated operational effect. The purpose of above two evaluations are aiming at the capability of the operational system itself. At present, the system operational capability has become the fundament of the force. In the SoS, the equipment, the sub-system and the coupling relationship among them become more and more complicated. If single equipment cannot be integrated into the whole system, it cannot provide any capabilities. In order to study and address this problem, the concept of the effectiveness contribution is proposed to evaluate the promotion of operational system performance on operational capability. As a new concept, the connotation must be clarified at first. The evaluation theory and method must be explored, to support the effectiveness contribution evaluation development of equipment SoS.

8.1. Basic Concept of Effectiveness Contribution 8.1.1.

Connotation and Classification

The effectiveness contribution is a measurement of the impact or emergence for each equipment, and sub-system and overall operational capability. The effectiveness is defined as follows: the weapon system is composed by different operational element, unit and sub-system which provides organic capability such as real-time perception, covert maneuver, efficient command, precise attack, full-dimensional protection [210]-[221]. Therefore, considering the new technology and equipment adopted in the operational system, the effectiveness contribution evaluation focuses on the improvement of each equipment, sub-system and the

264

Deping Zhang and Xuefeng Yan

whole operation system. The effectiveness contribution can focuses on the degree of requirement satisfaction and the effectiveness promotion because of new introduced and equipment technology. (1) Requirement Satisfaction The sub-systems or capability elements interact with each other in the operational system. Especially, in tightly coupled condition, contributor is unique to their beneficiary. That is, the beneficiary cannot complete the mission without the support of the contributor. For example, the target calculation system provides data fusion and target recognition capability. The sub-systems of weapon must depend on the target information provided by the calculation system to execute precise attacks. In this case, the effectiveness contribution of contributor can be measured by the requirement of the beneficiary, it can be used to evaluate the mutual contribution of various systems in the operational system. (2) Effectiveness Promotion of the Introduced Technology and Equipment In general, the operational system is composed of equipment and command-control system with various capabilities. The capability will change after new technology, equipment and sub-system are adopted. The effectiveness contribution can be measured by the improvement of operational effectiveness. That is, it can be defined as the improvement of the operational capability (operational effectiveness) of original operational system after the equipment or technology is introduced. The fundament of contribution evaluation is the operational effectiveness. The effectiveness contribution of the SoS can be obtained through the comparative analysis of the effectiveness changes before and after the use of the new technology and equipment. Due to complicated and various coordination or relationships among the different equipment and sub-systems in the operational system, the effectiveness contribution, both of the requirement satisfaction and the effectiveness promotion can be classified into different aspects, which is shown in Figure 8.1.

Figure 8.1. Classification of the effectiveness contribution. According to the relationship between the contributor and the beneficiary,there are two

Contribution Effectiveness

265

categories of contributions, the direct and the indirect contribution. The contribution of the new technology and the new weapon sub-system to the operational effectiveness can be measured and evaluated from the direct and indirect ways. The direct contribution refers to the military benefits that is generated from the new technology and equipment sub-system. For the lethal weapons in given mission, it is the ratio of the actual target damage quantity to required quantity. The indirect contribution refers to the indirect military benefits. The lethal weapon is also taken as an example, the indirect contribution refers to the measurement of the survivability changes because of the capability promotion of target damage, such as the blue electronic jamming equipment. In other words, the indirect contribution is reflected by cascading effect of the capability. There are two categories of contributions, the absolute and relative contribution, based on the measurement method. The contribution reflects the change degree of the operational capability brought by the operational system. The measurement of change degree can be directly reflected in the variation (the difference) or changing rate (ratio), which is the absolute and relative contribution respectively. The absolute contribution refers to the direct change of the operational capability. Through the comparison of the absolute contribution, the contribution of each equipment and sub-system can be obtained in the change of the system operational capability. The relative contribution refers to the change rate of system operational capability brought by the new operational system. Namely, it is the proportion of the improvement of system operational capability brought by the new operational system. Through the comparison of the relative contribution, the contribution proportion of each system can be obtained in the change of the operational capability. For example, the submarine weapon system has its operation mission and purposes. The mission is collaboratively completed by the equipment and sub-systems with similar functions and different capabilities. One or more sub-tasks can be completed by the single equipment or sub-system. Therefore, the absolute contribution is the amount of tasks completed by the operational system, while the relative contribution is the proportion of the completed missions to the total mission purpose. In the effectiveness contribution evaluation, the requirement satisfaction or the effectiveness promotion follow the above classification criteria. In practice, the four contribution should be integrated to evaluate according to the correlation of the evaluation purpose to the sub-system and overall operational capability.

8.1.2.

Relationship among Contribution, Capability and Effectiveness

The concepts of the system effectiveness and the operational effectiveness are easy to confuse with the effectiveness contribution. These three concepts are not only different from each other, but also related to each other. The system effectiveness, called the comprehensive effectiveness, is the possibility of satisfying a set of specific mission requirements under certain conditions, which is the comprehensive evaluation of the weapon system. The operational effectiveness refers to the extent to which operation forces play an effective role in process of operation, which reflects and evaluates the measure and criterion of system operational capability. The essential difference between the system and operational effectiveness is that the system effectiveness does not consider the human factors against the blue side. Both concepts are used to evaluate the capability of the operational system and

266

Deping Zhang and Xuefeng Yan

describe the result of a specific mission. However, the effectiveness contribution considers not only the capability of the operational system, but also the contribution to the system operational capability. It can be adopted to describe the effect of the operational system on the beneficiary, which can be the specific equipment, system or the whole system. The effectiveness contribution evaluation is based on the results of the effectiveness evaluation, but focuses on the requirement or the capability changes (increments) of the beneficiary. Only if all operational elements are coordinated closely and all kinds of equipment support each other, an operational capability with “exponential” emergence can be achieved.

8.2.

Effectiveness Contribution Evaluation of SoS

8.2.1.

Contribution Evaluation Based on Operational Effectiveness

For SoS effectiveness contribution evaluation based on operational effectiveness, there are four contribution measurement methods, which are method based on increment, ratio, satisfaction and cost-efficiency ratio [215]-[224]. The details are described as follows: (1) Increment-Based Measurement The increment-based measurement is to measure the change of the operational effectiveness (operational capability) caused by the new equipment in the operational system. The contribution evaluation should take the following conditions into considering: 1) There is no such equipment in the original SoS, the new developed equipment is added. For example, in the traditional equipment SoS, the contribution of new concept weapon need to be evaluated. 2) The old equipment is replaced by the improved ones. 3) The different types of old equipment with similar functions is replaced by new developed ones. 4) The allocation quantity and the application pattern of the new developed equipment in SoS. The SoS with and without new developed equipment is called new SoS and original SoS respectively. The measurement principle of the increment-based SoS contribution rate is to compare the new and original SoS. Then the increment of operational capability or effectiveness of the new SoS is the contribution rate, which can be described as follows: It is assumed that E1 and E2 represent the effectiveness of the new and original SoS respectively. f 1 , f 2, · · · , f n correspond to each effectiveness index. Then the SoS contribution rate can be expressed as follows: 4E = E1 ( f 1 , f 2 , · · · , f n ) − E2 ( f 1 , f 2 , · · · , f n )

(8.1)

Obviously, 4E may be positive or negative. If the contribution 4E is positive, the system effectiveness is improved after adopting the new equipment. That is, this equipment has contribution to the SoS. Otherwise, if the system contribution 4E is negative, the system effectiveness will decrease. That is, the

Contribution Effectiveness

267

equipment hinders the SoS operation. Specially, 4E = 0 means that this equipment has no contribution to the SoS. SoS operational effectiveness is the comprehensive embodiment of each single effectiveness, such as the fire attack, the information processing, the comprehensive support, and so on. It is assumed that E1i and E2i are a single effectiveness of the new and original SoS. If i = 1, 2, · · · , k, the single effectiveness contribution rate in the SoS is shown as follows: 4Ei = E1i ( f i1 , f i2 , · · · , f ik) − E2i ( f 1i, f 2i, · · · , f ik)

(8.2)

After adding new equipment, the SoS structure may be more complex and the new equipment also needs the resources. Therefore, some single effectiveness may be improved, while others will be decreased. The contribution can also be represented by a function of the single effectiveness contribution. 4E = Q(4E1 , 4E2 , · · · , 4Ek )

(8.3)

If the influence of each single effectiveness is superposition effect, 4E can be expressed as follows: 4E = w1 4E1 + w2 4E2 + · · · + wk 4Ek

(8.4)

where W = (w1 , w2 , · · · , wk ) corresponds to the weight of each single effectiveness. If some single effectiveness has a significant impact on the SoS effectiveness, Q(4E1 , 4E2 , · · · , 4Ek ) are non-linear functions. Q(4E1 , 4E2 , · · · , 4Ek ) = m1 4E1n1 + m2 4E2n2 + · · · + mk 4Eknk

(8.5)

where m1 , m2 , · · · , mk , n1 , n2 , · · · , nk are constants. In order to simplify the calculation, the overall contribution rate is calculated by the linear function usually. However, some single effectiveness often has significant influence on the effectiveness. For example, the attack type SoS focuses on the system fire attack capability, so the fire attack effectiveness has significant influence on the effectiveness. In this case, it is suggested to use nonlinear function to calculate the contribution. (2) Ratio-Based Measurement The completion probability is taken as the index of the SoS completion a specific mission. According to the definition of the contribution rate, the ratio-based measurement is used for analyzing the contribution rate of a certain SoS. That is, the relative changes of the mission completion probability are calculated by adding, deleting, replacing and modifying the corresponding equipment on the original SoS. The capability is considered as the static and inherent attribute. The operational effectiveness is the exertion of operational capability, which is the dynamic attribute. Therefore, the contribution rate can be measured by static and dynamic attributes. 1) The contribution rate of new developed equipment to the overall capability (static attribute). Assuming that the operational capability of the new SoS is E1 and the operational capability performed by the equipment is E2 , the contribution rate of the equipment can be measured as follows: E2 Cj = × 100% (8.6) E1

268

Deping Zhang and Xuefeng Yan

Obviously, C j ≥ 0. If C j = 0, it means that this equipment has no contribution. In general, the operational capability is a comprehensive reflection of the reconnaissance, the early warning, the command and control, the fire attack, the comprehensive support and other single capabilities. Therefore, the contribution rate of the equipment to each SoS capability can be described as follows: E1i(i = 1, 2, · · · , k) represents a single capability of SoS. E2i (i = 1, 2, · · · , k) refers to this single capability of new developed equipment or system. Then, the contribution rate of the equipment or system to single capability of a SoS is shown as follows: Ci =

E2i × 100% E1i

(8.7)

It is considered that the SoS operational capability can be expressed as a function of each single capability. Assuming that E1 = f (E11 , E12, · · · , E1n ), E2 = f (E21 , E22 , · · · , E2n ), the contribution rate of the equipment can be described as follows: Cj =

f (E21, E22 , · · · , E2n) × 100% f (E11, E12 , · · · , E1n)

(8.8)

2) The contribution rate of new developed equipment to the effectiveness (dynamic attribute) of the SoS. Similarly, the contribution rate of operational effectiveness can be defined as follows. It is assumed that the effectiveness of the new developed SoS is Et . The effectiveness of the new developed equipment is Ez , then the contribution rate is shown as follows: Cd =

Ez × 100% Et

(8.9)

(3) Satisfaction-Based Measurement This method measures the satisfaction rate of operational requirement after the new equipment developed in the SoS. There is some special equipment act as key nodes in the SoS. The lack of these equipment quit, will lead to the mission of the SoS failure. For example, the systems cannot communicate with each other and receive the command and control information without the communication sub-system. According to the definition above, the equipment contribution rate will reach 100%, it is inappropriate obviously. The ultimate purpose of communication sub-system is to promote the functionality (mission requirements). Therefore, quantitative contribution rate can be achieved from the perspective the operational requirements. The satisfaction rate is proposed for the key equipment, which is used to measure the contribution rate in the joint operations. It is assumed that the capability of the equipment is Ei and the ideal value for capability requirement is Ei∗ . The satisfaction rate can be represented as follows: C=

Ei × 100% Ei∗

(8.10)

where Ei∗ can be obtained by analyzing the missions and the functional requirements.

Contribution Effectiveness

269

(4) Effectiveness-Cost Ratio Based Measurement Effectiveness-cost ratio is the ratio of the increased operational effectiveness to the increased cost of the new developed weapon. It is mainly used in the economic demonstration of the equipment development. After the new developed equipment is incorporated into the SoS, in addition to its own cost, the equipment also takes up the resources and increases the SoS consumption. The concept of the effectiveness-cost ratio is extended to the measurement of the contribution rate, which is composed of five parts including the reconnaissance judgment E1 , the accusation decision E2 , the comprehensive defense E3 , the fire operational E4 , and the evaluation analysis E5 . It can be described formally. 5

REC = ∑ wi Ei

(8.11)

i=1

where wi represents the weight of each effectiveness contribution rate, which is calculated based on the analytic hierarchy process. The formula of Ei is shown as follows: Ei =

EiN − EiC × 100% EiC

(8.12)

where EiN represents the effectiveness of the SoS with new equipment i and EiC represents the corresponding effectiveness cost.

8.2.2.

Systemic Framework based Contribution Rate Evaluation

According to the definition of the contribution rate, the analysis and evaluation of the contribution rate in this section is based on the operational effectiveness. The effectiveness measures the completion degree of the expected mission in specific conditions dynamically. The overall framework of the contribution rate evaluation is shown in Figure 8.2. From Figure 8.2, the contribution rate evaluation is divided into three steps including the mission scenario description, the operational effectiveness and the contribution rate analysis. The operational effectiveness analysis is based on the operation loop model in specific mission. The contribution rate evaluation takes the operational effectiveness analysis as the evidence so as to compare and analyze the effectiveness changes in different conditions and quantitatively calculate SoS contribution rate. The details are shown as follows: Step 1. Mission scenario description. The evaluation scenario of the contribution rate is generated according to the mission. Step 2. Analysis of operational effectiveness in specific mission scenarios. Step 2.1. The systems and equipment involved in SoS will be analyzed. The network topology of the operational effectiveness analysis is constructed according to the logical relationship among the sub-systems. The nodes and edges are abstracted and the capability index system of the entire operational network can be constructed. The main tactical and technical indexes are determined and then the relevant qualitative and quantitative data are collected. Step 2.2. Calculation of the operational effectiveness of each operational. If the operational phase is supported by multiple capabilities and different capabilities have different

270

Deping Zhang and Xuefeng Yan

Figure 8.2. The framework of contribution rate evaluation. effect on the operational process (the operation loop), the weighted method is applied to obtain operational effectiveness. Step 2.3. The overall effectiveness of each operation loop is calculated according to the effectiveness of each operational phase. Step 2.4. Considering the synergistic effect of multiple operation loops, the overall effectiveness evaluation value is calculated for every mission. Step 3. Element contribution rate evaluation. Step 3.1. The equipment and its input data upgrading from e0 to e1 will be clear and definite. In this mission scenario, the operational effectiveness Ee0 and Ee1 are calculated by repeating step 2 after updating or upgrading. Step 3.2. The contribution rate formula is selected according to the requirement. Ee0 and Ee1 are the operational effectiveness before and after the sub-system updated. Both of them are substituted into the selected contribution formulas. And then the contribution rate can be obtained.

8.2.3.

Case Study

Taking the anti-ship equipment system of aircraft carrier formation as an example. The method proposed in this chapter are illustrated. The formation is composed of the aircraft carrier, the frigate, the missile destroyer, the fighter bomber, the reconnaissance aircraft, the early warning aircraft, the anti-submarine aircraft and etc. According to the framework of contribution rate analysis evaluation, the “operation loop” model is constructed based on the mission from the scenario of warship formation, which is shown in Figure 8.3. The aircraft carrier formation effectiveness evaluation index system is constructed based on the method mentioned above. For simplicity, some key tactical and technical indexes are selected. For example, T → S represent the detection distance, the anti-jamming performance and the identification probability in early warning reconnaissance activities, S → D

Contribution Effectiveness

271

Figure 8.3. The warship formation operation loop model. represents the information sharing activity, D → I represent data transmission rate, the information processing delay (s), the communication quality (bit error rate) in the information sharing activity, and I → T represent the penetration probability, the hit accuracy, the hit probability, the target anti-destructive value in the strike and confrontation activity. The relevant parameters of each equipment in the operational activities are obtained through the system simulation and the analytic hierarchy process. For example, penetration probability simulates the probability of breaking through the side 3-layer air defense of blue system under the strategy of hitting 1 out of 5 missiles. The damage probability simulates the probability when the damage area of the blue target is more than 50%. Plan A and plan B with different types of tactical and technical indexes are designed for each equipment, which is shown in Table 8.1. Based on the experience and knowledge, network structure, combat technology index and combat theory of experts, the system and equipment of SoS can be analysed. According to the logical relationship among the elements of equipment system, the network topology model of effectiveness analysis is constructed. This chapter abstracts the nodes and edges in the operation loop, models the nodes and edges according to the combat activities, constructs the capability index system of the whole combat network, determines the main combat technical indexes of the equipment, and collects the relevant qualitative and quantitative data. The equipment composition system in scheme A is recorded as system A, and the corresponding renewal equipment is replaced in system A to build a new system, which is recorded as system B. the operational effectiveness of system A and system B is analyzed respectively, and the operational effectiveness value of each operational link is calculated. If the operation coop is supported by multiple capabilities, and different capabilities have different effects on the whole operation (i.e., one operation loop), coop the weighted method is used to obtain the operation effectiveness the combat link, and the result is shown in Table 8.2. From Table 8.2, before the equipment update, the effectiveness of the battlefield maneuvering is the most optimal, the early warning reconnaissance is more optimal and the fire attack is the weakest. After update, early warning reconnaissance is rank one, the battlefield maneuvering is rank two, and the fire attack is the third. From the view of the effectiveness,

272

Deping Zhang and Xuefeng Yan Table 8.1. The main tactical and technical of A and B plans

Equipment Missile Destroyer Aircraft carrier (information system)

Early warning machine

Fighter Bomber

Performance index Index A Sailing speed 32 Maximum Range 150 Warhead weight 232 Destroy probability 0.82 Accused time 13 Information transmission 15 rate Information transmission 15 rate Bit Error Rate 0.12 Intelligence monitoring 0.95 rate Data processing rate 0.96 Navigation accuracy Flight Time Mobility range Penetration Probability Anti-interference ability Destroy probability

0.94 13 45 0.95 0.80 0.85

Equipment B 45 120 Frigate 295 0.87 15 19 Anti-submarine aircraft 19 0.05 0.99 0.93 0.99 15 50 0.90 0.85 0.80

Reconnaissance aircraft

Performance index Index A B Sailing speed 25 30 Warhead Weight 300 210 Maximum Range 65 130 Destroy probability 0.75 0.82 Detection distance 53 68 Detection accuracy 45 50 Detection accuracy

0.95 0.90

Destroy probability Target detection ability Position recognition ability Maneuverability

0.85 0.80 0.93 0.85 0.95 0.90 0.95 0.90

the early warning and battlefield maneuvering benefit update greatly, but the improvement of fire attack is not obvious. In the future, it is necessary to increase the research and upgradation of the fire attack and try the best to achieve the comprehensive balanced of operational effectiveness. From Table 8.2, the effectiveness of battlefield maneuvering system benefits from the updated weapon system. For the fire attack system, the effectiveness improvement from the bomber update is larger than other equipment. For the early warning and reconnaissance system, the update of missile destroyer and reconnaissance plane has obvious effect on the improvement of ieffectiveness. Each equipment update can improve the overall operational effectiveness, where the improvement from the fighter bomber, the missile destroyer and the frigate is larger than the others obviously. In general, whether to consider three subeffectiveness or the whole effectiveness of SoS, the contribution rate from single updated equipment is limited, which is much lower than all the equipment is updated. If the multiple equipment update is taken as integration, the effectiveness can be improved greatly and the update plays the role of “1 + 1 > 2”. Therefore, it is necessary to balance the development of all kind of equipment as a whole in the SoS, then all the equipment can be a organic whole and play a maximum force. Further study the system contribution rate of each equipment based on Table 8.2. To provide decision support for equipment development planning, the system contribution rate of the equipment involved in the mission scenario is evaluated according to the calculation process described in the previous section, the conclusion is shown in the Figure 8.4. From Figure 8.4, the contribution rate of equipment supporting one system, such as the aircraft carrier information system, the early warning aircraft and the anti-submarine air-

273

Contribution Effectiveness

Table 8.2. Summary of operational effectiveness before and after equipment update in plan A and B Equipment be replaced Missile equipped destroyer Frigates Fighter bomber Aircraft carrier (Information System) AWACS Anti submarine aircraft Reconnaissance aircraft SoS

Battlefield maneuver Fire attack A 56.152

B 64.876

A 53.492

B 54.416

Early warning reconnaissance A B 54.954 54.954

Overall operation effectiveness A B 55.339 62.832

56.152 56.152 56.152

66.392 72.404 56.152

53.492 53.492 53.492

58.058 65.492 55.304

54.954 54.954 54.954

54.954 54.954 54.954

55.339 55.339 55.339

69.947 69.663 56.212

56.152 56.152

56.152 56.152

53.492 53.492

53.492 53.492

54.954 54.954

58.091 58.623

55.339 55.339

57.020 57.534

56.152

56.152

53.492

53.492

54.954

70.433

55.339

63.076

59.152

92.340

56.492

74.100

57.954

99.532

58.339

96.874

Figure 8.4. Contribution rate of each SoS. craft, is lower than the one with supporting multiple systems, such as the fighter bomber, the missile destroyer and the frigate . The result shows that the equipment should contain the capabilities to support as many systems as possible in the equipment research and development. Although the missile destroyer supports three effectiveness sub-indexes at the same time, the system contribution rate is lower than the fighter bomber and the frigate which only support two effectiveness sub-capabilities. The reason may be that the improvement degree of the missile destroyer operational indexes is low and cannot effectively improve the system operational effectiveness when the investment is the same.

Chapter 9

Design, Evaluation and Optimization of SoS At present, the equipment research and development focus on the independent and systematic innovation. The evaluation also develops from the single performance and effectiveness index of equipment to the overall SoS. Similarity, the optimization design has changed from the single tactical and technical index to integrated tactical and technical index. The SoS is a “complex system”. Each sub-system operates independently but depends on each other. The interaction of various factors will affect the overall capability. Several general characteristics of complex systems are emerged, such as nonlinearity, open, dynamic, diversity, emergence, self-adaptability, self-organization, and so on. Therefore, it is very difficult to optimization the SoS.

9.1. Effectiveness-Based Design, Evaluation and Optimization Framework The SoS is a complex “big system” that is composed of many “small systems” in the way with loose coupling and networked system. It is a kind of complex system relative to the general “monomer” complex system. In other word, it is a kind of “multi-body network system”. Therefore, no matter how complex an aircraft is, it cannot be called a SoS. No matter how small the area of an airport is, it should be treated as a SoS. The difference between SoS and system mainly lies in its complexity. The SoS can also be understood as a kind of “Super-system”. The component systems could be independent a complex systems that can fulfill their own mission independently. More complex mission can be accomplished by the interaction of these complex systems. The complexity of the SoS mainly comes from several typical characteristics, such as functional interleaving, heterogeneous evolution, fuzzy requirements, boundary floating, dynamic inflow and outflow, and multilevel emergence. In order to complete certain mission, the SoS is a higher-level system that consists of various interrelated sub-systems, which is a typical complex system. The system analysis, design and evaluation in different fields have some commonalities. However, due to the differences of object ontology, description language, industry standard and application logic, the common commercial methodology and platforms are difficult to meet the application

276

Deping Zhang and Xuefeng Yan

requirements in different fields. The design, simulation, evaluation and optimization are separated in the design process. As a result, the design results cannot be verified in time, which affects the design quality of the system seriously. At present, the equipment research and development has entered a new stage. The design pattern has changed from bottom-up to up-bottom model-based design. The optimization design of SoS is a process of seeking overall optimization of structure, proportion, technical level, quantity, allocation, etc. In this work, the optimization is oriented to the missions, including the strategy, campaign and tactical mission. Combined with the domain characteristics and research and development organization, the mission-oriented and efficiency-based optimization framework for SoS, is proposed based on the theories of systems engineering, requirements engineering, ontology, MDA, system modeling, simulation and evaluation. As regard to the multi-view modeling in specific fields, the modeling framework and methodology are proposed based on model extension mechanism. Domain meta-model is defined to meet the requirements of system analysis, design, modeling and simulation of weapon system. The key activities, input/output ports, association and constraint of each views are proposed to supporting iterative simulation and evaluation. The equipment requirements analysis need capability to be enabled to improve the system design efficiency and quality. Considering the difficulties of effective verification and evaluation research and development of SoS such as the complex architecture, numerous components and parameters, a seamless integrated and an integrated design, evaluation and optimization methodology platform are to ensure verify and optimize at the early design of SoS. Based on the DoDAF, MoDAF, UML, SYSML, UPDM, and general solutions, as is shown in Figure 9.1.

Figure 9.1. The mission-oriented and effectiveness-based design evaluation optimization framework for SoS. Where, the unified meta-model of the design platform, simulation environment and effectiveness evaluation for the SoS is used to achieve seamless data connection. According to the whole process design methodology of SoS that is driven by domain model, the inte-

Design, Evaluation and Optimization of SoS

277

grated platform of complex SoS is constructed, as is shown in Figure 9.2.

Figure 9.2. The connotation and iterative process of design, simulation and optimization of SoS. The design platform of the complex SoS is based on model-based systems engineering (MBSE). Aiming at the research and development of multi-domain complex equipment demand requirements and integrated design, combined with domain modeling, model expansion and other technologies, a domain model-driven and mission-oriented system engineering methodology is proposed to realize a visual analysis and design platform based on UML/SYSML, The methodology can guide the users in system requirement analysis, comprehensive index construction, functional analysis, architecture design, man-machine interaction design and componentization design in a top-down pattern. It can help to improve the design efficiency, ensure the design quality and exert the comprehensive advantages of the system. After the one round design is completed, the high-level concept model, operational activity model, operational interaction model and operational capability model can be imported into the simulation and evaluation platform for further study. For the inspection and evaluation of SoS, a service-oriented complex equipment simulation platform is design and deployed. It can not only verify the integrity of system design, but also generate important input data for effectiveness evaluation. Therefore, a complex simulation system development and integration can be simplified and the efficient. After the simulation, the simulation configure file and corresponding data are imported into the evaluation platform. When the SoS is operated in specified conditions, the effectiveness evaluation platform can estimate the degree that the system meets the operational requirements within the specified time. The index of the SoS can be constructed to represent the capability of specified missions. The effectiveness evaluation is an important means to verify the operational capability. The inputs of the evaluation platform include index model established in the design environment and the simulation scenario and data. Several applicable evaluation algorithms are adopted, such as the most widely used analytic hierarchy process (AHP), network analytic hierarchy process (ANP) or mission-specific evaluation algorithms. The

278

Deping Zhang and Xuefeng Yan

basic attributes of each index are described and the index system analysis is developed. According to the existing evaluation model and the imported simulation data, the effectiveness evaluation can be developed and the contribution analysis can be completed so that to guide the designer to optimize the system.

9.2. Domain Model-Driven Design Methodology 9.2.1.

Overview of the SoS Engineering

The SoS engineering is a method of planning, analyzing, organizing and integrating capabilities. The capabilities of existing and new added systems are to merge and transform into capability of the SoS. It tries to achieve the effect of “1+1>2”. Boeing has a unique insight about the application of the SoS engineering. Researchers define the SoS engineering as a process of strictly regulated system engineering. The capabilities of the SoS, and the infrastructure networked architecture are defined in this process. Then these capabilities are decomposed and asigned to the systems and sub-systems, which effectively guides the design, production, and maintenance for the whole lifecycle of each system. The architectural model is taken as major production for the conceptualization, construction, management and evolution of the system and the SoS, which can be defined as description for constituent components, the relationships between components and the capabilities assigned to components. The SoS engineering are supported by many key technologies, such as modeling, requirement analysis, design, management, integration, optimization, test, evaluation and so on, as is shown in Figure 9.3. The requirement analysis is the description of the purpose, function and structure of the SoS to be developed. The design is the top-level planning of the method, architecture and management mode, and the overall scheme of the development. The integration refers to the integrating principle and method of component systems to realize the overall goal. The management, including relative methods and theory of development and operation, is the key point for the overall benefits. The optimization is to explore the way of adjusting the architecture and functions so that the behavior and performance meets the requirements. The evaluation is a comprehensive assess of behavior and performance to judge if the final effectiveness of the developed SoS meets the military requirements. The development should be carried out under the guidance of the architectural framework. The evolution reflects the long-term dynamic change of SoS. If the evolution mechanism and law are studied successfully, the behavioral pattern and structural gradual progress will be clearly understood, which is great significant for design and development of SoS. The SoS requirements engineering is the process of implementing and completing requirements analysis. Generally, it contains requirements development and management. The process of requirement development mainly includes acquisition, modeling, analysis, verification, evaluation etc. In the field of military information system, a large number of requirements analysis methods have emerged, such as requirement development methods based on system engineering, based on conceptual model, based on capability, and multiviews requirement engineering methodology. In 2014, the detailed defense architecture framework, the DoDAF 1.0, is published by

Design, Evaluation and Optimization of SoS

279

Figure 9.3. The basic research framework of on SoS engineering. the US Department. A new framework of requirements analysis is proposed by summarizing and improving the existing methods based on multi-view, capability or concept. It is emphasized to design the requirement description framework of military information system from six kinds of views, including capability, operational, system, service, technology view and full-view. Then the requirement acquisition, analysis and modeling can be performed. So far, it has been renewed to DoDAF 2.x and is gradually becoming the national standard for research on C4 ISR. However, due to the lack of consistent requirements description criterion and guidance methods in different fields, disciplines and departments, in the process of requirements development for C4 ISR system based on multi-views, it is extremely difficult to be integrated.

9.2.2.

Domain Model for the SoS

In the initial stage of C4 ISR system requirement engineering, the requirements products are described by the normalized graphics, text, or tables. The product-centric requirement design is highlighted. The product-based requirements analysis and decision-making activities as well as system development and design are emphasized. However, the views in the requirement description framework are not isolated and have mightily relevance, such as high-level conceptual model and operation activity model. Therefore, it is difficult to ensure the consistency of data and the accuracy of grammar and semantics only focus on the requirement of the products itself. The accuracy of the requirements description is affected seriously, leading to the difficulty in understanding, exchange and integrating requirements,

280

Deping Zhang and Xuefeng Yan

and inefficiency of execution of analysis and decision. MDA is a strategic direction identified by the Object Management Organization (OMG) in early 2002. It defines a standard for developing systems and the key point is the separating system functionality from implementation. In the process of development, a higher level of abstraction is provided by MDA and it is easy to integrate different systems on different middleware platforms. Therefore, interoperability and inter-transplantation between systems can be improved and the system can be flexible regardless the software infrastructure changing. The OMG has defined two approaches for creating meta-models based on MDA, that is, MOF (heavyweight extension) and UML Profiles (lightweight extension), both approaches have their own characteristics. Three core extension mechanism are included in the UML 1 Stereotype, which is the language facility of Profile. 2 Tags, the tag in the Profile. 3 Constraints, it is used to define the semantics Stereotype is the tag value of an instance. accurately. The extended UML meta-model in the 4th generation language is similar to the user-defined class library in the 3th generation. These above extensions can be implemented on the current UML tools and are not technically based on MOF. MOF extension is used to define modeling language. The MOF-based extension defines a new meta-model to which new meta-class and meta-tectonic structure are added. It is a much essential extension mechanism because new semantics is added at the meta-model layer. The difference between the two extension approaches is the different restrictions at the meta-model levels when extending. Based on the extension requirement of the UML Profile, the extended meta-model must be completely consistent with the semantic standard of UML meta-model. The essence of MOF-based extension is to define a new meta-model. This methodology is based on model-driven architecture (MDA) and is aimed at the multi-views system design in specific domains. Based on model extension mechanisms, the domain-oriented architecture meta-model is defined. The corresponding modeling framework and method are proposed. The associations and constraints on models for each view and between views are put forward. While meeting the requirement of domain modeling, the main models are ensured to meeting industry standards, facilitating the interchange and integration of model data. Finally, according to the research results, a graphical modeling platform is developed, and the requirements analysis is mainly oriented to the military industry. To meet the efficient design of domain and customization as well as the standardization, the domain modeling and modeling extension are adopted in this methodology. The function, purpose and meta-model definition of each model are shown in Table 9.1. The system is described from different levels and perspectives in the proposed methodology, so as to realize the optimal design of the SoS. Therefore, the models between different views are not isolated from each other, but are related or derived. The model relationship between two different meta-models can be established in the following ways: 1) Association. When there are two objects, the designer establishes the association between them. Then, the current property parameters can be get from one object to another. 2) Derivative. A model object can be generated from another and it is unnecessary for the two model objects to have the same meta-model. 3) Inclusion. It is a combinatorial relationship that one model object contains another. When the parent object does not exist, the included parts also cannot exist. Conversely, the absence of the included parts does not affect the parent.

Function The top-level model of system requirement is presented, and the mission, mission structure and relationship are described The operation concepts (interveners, objects) of the system-related and their relationships are described and the operation scenes information is described The identification, range, target, viewpoint and tool of the system are described by words, tables and graphs. The command structure, relationship and cooperation relationship between the personnel, organizations or the operational concepts that play a key role in the system are described The main activities and the input-output information flow relationship between them are described The information exchange order between operation concepts is described OV-4

Mission analysis

The missions and business scenarios of OV-5 mission are analyzed The tracking description of mission and OV-6c mission event is analyzed

AV-1

Mission analysis

Application Standard The entry points of mission, mission modeling and analysis The high-level conceptual analysis of OV-1 assistant and mission

The system capability index system is CV2 defined, and the capability distribution is performed to realize the design tracking in the design Use case view The system use cases, boundaries, and users of the mission are de- The system use cases of mission are described signed System workflow view The main processes are described in the use case execution scenario The use cases are analyzed under different missions The tracking descrip- In use case execution scenarios, the information exchange order be- The use cases are analyzed under dif- SV-10c tion view of system tween functional modules is described ferent missions event Function module view The multilevel functional set of the system, functional control re- The system functions under different SV-4 quirement, processing requirement, capability requirement and in- missions are designed terface are described System logical deploy- The logical composition of the system and the allocation of func- The system architecture is designed unment view tions within the logical modules are described der different missions Human-computer The definition of human-computer interaction interface, touch The human-computer interaction is deinteraction View screen and special keys are described signed under different missions Component view The components set of the system is described The system component is designed Data Interactive model The rules, metadata and types of information interaction are de- The analysis of auxiliary operation ac- SV-11 scribed tivity, workflow and function module, and the description of operation and system event tracking System physical The integrated assembly scheme of the system is described The integrated assembly design of the EV-1 deployment view system

Operational activity view The tracking description view of operation event The capability view The system capability index system is described by the tree structure, so as to describe the user’s original demand point or effectiveness index, which is the original input of the system design.

Overview and summary information Organization relationship view

Model Top-level requirement view High-level conceptual view

Table 9.1. The definition of domain modeling

Weak

Derivative

System physical deployment: De- System component vice component

view: Inclusion

Strong

Strong

Inclusion

System use case: use case Derivative Interactive data model:struct Inclusion Interactive data model: struct Inclusion

Task operation activity: action Operation activity:flow Description of the operation event tracking message Function model: infomationFlow System logic view: Node

The requirements of class operation, processing, interface, capability are converted to the calculation, display, interface, performance of the component The component is manually allocated to the Device

The model of human-computer interaction interface, touch screen and special keys are manually established for the Node

The flow is manually associated with the struct The flow is manually associated with the struct

Strong Strong

Inclusion Inclusion

Interactive data model: struct System function module view: class System logic view: Node Human-computer interaction model:interaction interface, touch screen and special keys System function model view: class System component view: component

Description of the mission Correlation event tracking LifeLine

Task concept: Operation concept

Mission concept: Operation con- Description of mission event Correlation cept tracking LifeLine

Mission concept: Operation con- Task concept: Operation con- Correlation cept cept Partition

Mission concept: Operation con- Mission operation activity Correlation cept Partition

Correlation Description Strong The class objects in the organization diagram can be automatically generated from the high-level concept diagram Weak The subsets are manually selected from the index system and assigned to special task Weak The subsets are manually selected from the mission index System and assigned to classes Weak The mapping relationship of the operation concept and the Partition is created manually.The newest attribute parameters of the operation concept can be seen Weak The mapping relationship of the operation concept and the Partition is created manually.The newest attribute parameters of the operation concept can be seen Weak The mapping relationship of the operation concept and the LifeLine is created manually. The newest attribute parameters of the operation concept can be seen Weak The mapping relationship of the operation concept and the LifeLine is created manually. The newest attribute parameters of the operation concept can be seen Weak The action can be mapped to use case Strong The flow is manually associated with the struct Strong The flow is manually associated with the struct

View Model I View Model II Correlation way High-level concept: Operation con- Organization relationship: Derivative cept class Top level requirement: task Capability index system Ca- Correlation pability Function module: class Capability index capacity Correlation

Table 9.2. The model constraint relationships between various views

Design, Evaluation and Optimization of SoS

283

According to the different relationship between the two models, there are two kinds of association, which are strong association and weak association: 1) Strong Association. It means that two objects are essentially one model. When the original object is destroyed, the derived object is also destroyed automatically. The combinatorial relationship must be a strong association. 2) Weak Association. It means that two objects have certain kind of mapping relationship, but are two separate models in essence. The weak association comes from correlations or derivations. When the original object is destroyed, the derived one is not affected. The model constraint relationships between the various views are shown in Table 9.2.

9.2.3.

A Domain Model-Driven Design Methodology for SoS

The weapon SoS is a typical complex SoS whose sub-systems operate independently and interdependently. In recent years, the unified modeling languages, such as UML and SYSML, are introduced to verify the design. Some specification, such as UPDM (Unified Profile for DoDAF and MoDAF), CADM (Core Architecture Data Model) are proposed. However, those imported tools define analysis and design process with pure computer professional vocabulary in general view, without domain-specific modeling languages, methods, and tool chains, which encounter serious problems in analysis and design process. In particular, due to the limitation of major and division, the granularity of system decomposition is limited and the reusability is low. The commercial tools focus on the design results rather than the design process, which leads to the low efficiency of researchers. As a result, the complex system design is uncontrollable, difficult and inefficient, and the design achievements and experience cannot be accumulated effectively. According to the current situation of system engineering, a new model-driven design methodology is proposed, which takes operational capability index as main principle line, operation mission as profile, domain model as core. And the innovation process of inheritance design of complex system is proposed, which is shown in Figure 9.4. The method includes expressive domain modeling language, transformable modeling process, verifiable modeling method and applicable modeling tools. Several elements of complex system are organically integrated into the design methodology, such as the business rules, design specifications, processes, constraints, control requirements and so on. And it is based on the domain model to carry out operation competency-based modeling. Compared with other method, it is easy to use and reuse. The design cost and cycle can be reduced and the design quality can be improved. These are the following terminologies in the methodology: 1) Mission. The scope, usage, background and organization relationship of SoS is described by words, tables, and views etc. The integrated dictionary of the unified system is the basis and prerequisite of the design. 2) Operation Concept. It can be looked as all the objects and their relationships participating in the operation. 3) Mission. From the perspective of requirement, the mission and system objectives can be decomposed according to the principle of cohesion. It focuses on the description of the participating objects, business processes, interaction relationship and prototype interaction interface in each scenario.

284

Deping Zhang and Xuefeng Yan

Figure 9.4. Domain model driven methodology for complex SoS. 4) Task. From the operation perspective, the business of the system is recombined according to principle of functional cohesion. It focuses on the description of participants, activities, interaction and cooperation among participants, as well as the capability to complete the mission in each scenario. Note: Mission can be distinguished with mission task from perspective of decomposition. One is from the view of users focusing on business cohesion, the other is from the view of system focusing on functional cohesion. Taken ship formation as an example, the mission can be divided into formation escort and area search and the mission task can be divided into anti-submarine and anti-ship in short range. 5) Capacity. System capacity can be used to describe the requirement point and the performance index. 6) Capacity Index System. According to AHP, the relationship of the capacity is organized using tree structure, and each capacity node has their own weights. 7) System Use Case. The system use case, refers to the application scenarios for users. 8) Logic Node. The logic node is used to define the architecture and can also be viewed as the container that manage a group of functions in a task. 9) Function Module. It refers to the capacity of the system and corresponding requirement including the operation, computing (preprocessing), capacity and interface. 10) Component. It is the minimal software unit at runtime. The basic information of components is composed of name, type, version, display resolution and overview. Component entities consist of the program files and runtime environment. The format of the files can be selected based on the system requirement, such as exe, dll, war, jar and so on. 11) Device. the physical node and runtime environment of the system, such as computer, servers and so on. (1) Requirement Analysis The requirement analysis takes the military requirement and mission as input. Several models need to be constructed based on the analysis of mission, mission task, and task, such as high-level concepts, organization relationship, scenario and objective interaction.

Design, Evaluation and Optimization of SoS

285

And then the ability requirements and comprehensive ability requirements are formed in different mission for each object, shown in Figure 9.5.

Figure 9.5. The requirement analysis. 1) Mission Analysis. The target is to define the general information for SoS, which is the foundation of the system requirement analysis and design. The main activities is shown as follows. Firstly, define the mission model. Secondly, build up system summarization and abstract information based on chart and table tools and describe system identification, scope, purpose, background, used tools, file format and conclusion. Thirdly, construct a integrated dictionary for system. Finally, an organization relationship view is formed. 2) Mission Task Analysis. Mission task is the achievements based on mission analysis. The mission and system target can be decomposed according to business cohesion thinking, which consists of the following steps. Firstly, the mission task model is established. Secondly, the relationship of the objective, area and concept in current mission task scenario is described in high-level concept diagram. Thirdly, the operation activity analysis view describes a series of activities in execution of the mission task, and also the input and output information flow among the activities. Then, the operation event tracking description view is used to describe the information exchange and temporal relationship in the process of mission task execution. Finally, the interaction design view is used to describe the main mode and interaction that is expected by users. 3) Task Analysis. Based on the achievements of mission task analysis above, the typical mission scenarios can be extracted according to functional cohesion thinking, which consists of the following steps. Firstly, the task model is established and high-level concept diagram is used to describe the relationship of the objective, area and concept in current mission scenario. Secondly, based on the high-level concept analysis and combined with

286

Deping Zhang and Xuefeng Yan

the attribute of participated concept, the capability requirements related to the mission are extracted from the capability index system. Thirdly, the operation activity view describes a series of activities in the process of mission task execution of each concept, and also the input and output information flow among the activities. At the same time, the capability requirements are decomposed and assigned to various activities. Finally, the temporal graph describes the information exchange and temporal relationship in the execution mission task of each concept model. (2) Operational Capability Analysis The purpose of operational capability analysis is to construct SoS capability model according to the requirements of capability and performance index in military. And it can be used throughout the whole process of the requirement analysis, which is shown in Figure 9.6

Figure 9.6. Operational capability analysis. 1) Capability conception modeling describes the main operational support requirement and strategic development concept of SoS. 2) Capability structure modeling describes the capability constitution structure. And the analytic hierarchy process or network analytic hierarchy process is adopted to describe the capabilities and their structures. 3) Capability relationship modeling describes the relationship among various capabilities. 4) Capability tracking modeling describes the support relationship between military capability and operation activity and also describes which operation missions or activities are used to implement the capability.

Design, Evaluation and Optimization of SoS

287

(3) System Function Analysis System function analysis mainly includes the design the use case model and function module. From this section, the design will be performed according to the mission profile. In this methodology, parallel design process can be adopt and the consistency of design results are ensured through comprehensive optimization in each design step. If the serial design process is adopt, the design results of previous missions can be reused continually. The use case analysis and the function module design of the system are parallel, cyclic and iterative processes. Based on the system flow and tracking description in use case, the control, processing and interface requirements of the system can be extracted continuously. The system function is decomposed layer by layer to refine the precision of use case. Through the continuous iteration, functional module set of the system can be form, which is shown in Figure 9.7.

Figure 9.7. System function analysis. 1) System Use Case Analysis. Firstly, the system boundary range is delimited based on the activity analysis of the current mission. The system actor and use case are extracted and the system use case is established. Secondly, the system white box is developed through the system workflow view. The swim lane is constructed based on the current use case scenario and the system function module. The activity model is constructed in the each swim lane. It describes a series of activities of the system function module in the process of the use cases execution and the input and output information flow among activities. Thirdly, the system white box is fulfilled through the system event tracking description view. Lifeline model is constructed based on the current use case scenario and the system function module. It describes information exchange and temporal relationship between the function modules in the process of the current use cases execution.

288

Deping Zhang and Xuefeng Yan

2) System Function Module Design. Firstly, the top-down hierarchical system function module is established based on the experience of designers and the requirements in use case. Secondly, a state transition model is established to describe the function state transition event for the function module. Thirdly, the function module requirements of the control, calculation (processing) and interface are defined according to the description of activity, capability requirement, interaction and state transition. Then, the current mission capability is decomposed and assign to each function module to form the function-capability tracking matrix. Finally, if the current mission is parallel designed, the comprehensive optimization between the current mission and other missions is executed. And the same function modules are summarized and merged to ensure the integrity, consistency and accuracy of system functions. (4) System Architecture Design The system architecture design constructs the overall architecture of the system based on the design principle of open architecture and combined with the system function composition, which is shown in Figure 9.8.

Figure 9.8. System architecture design. Firstly, system logic node is defined. Secondly, function modules are allocated to each logical node, including control, calculation and processing, capability and interface requirements. Thirdly, the attribute parameters of each logic node are defined based on the assigned function modules, such as average CPU utilization, memory utilization, bandwidth utilization, I/O interfaces and parameters. The load, bandwidth and cost of the current system are calculated based on the above attribute parameters. And the function allocation scheme

Design, Evaluation and Optimization of SoS

289

can be constantly adjusted based on the calculation results to achieve the optimal scheme. Finally, if the current mission is parallel designed, the comprehensive optimization of function module design between the current mission and other missions is executed. And the same logical modules are summarized and merged to ensure the integrity, consistency and accuracy of system functions. (5) Human-Computer Interaction Design The human-computer interaction design proposes a unified interaction standard, layout and effect for a equipment or modules with display and control requirements. At the same time, it considers the reuse comprehensively and supports further division of interface, which is shown in Figure 9.9.

Figure 9.9. Human-computer interaction design. Human-computer interaction design is developed for logic nodes with control requirement. Firstly, the prototype of human-computer interface is designed and the schematic or effect diagram is drawn. Secondly, the control design prototype is devised, such as touch screen, private key. Thirdly, the connection between control requirements, display and control components is established. (6) Components Design The components design proposes the development requirement for component according to the service-based/component-based design principle and the requirement of the system architecture and human-computer interaction, which is shown in Figure 9.10. According to the requirements of control, processing, interface, capability that are allocated to each logic nodes. The system component model is defined and the relevant requirements are mapped to the development requirements. The entity file, dynamic library

290

Deping Zhang and Xuefeng Yan

Figure 9.10. Components design. and configuration file which are depended on the public support or running environment are defined for components. If current mission is parallel designed, the comprehensive optimization of components between the current mission and other missions is executed. And the same components are summarized and merged to avoid duplicate construction. Finally, the designed components are uniformly uploaded to the component library as the input of component development.

9.3. Mission-Oriented Domain Model Driven SoS Modeling Platform For a long time, many commercial modeling tools are introduced into the design of SoS, and some valuable engineering experience are gained in the process of implementation. However, there are still many shortages. 1) Most of the modeling tools are oriented to the general field and do not provide specific services such as modeling process, model elements for specific domain. Therefore, these tools are complex, not easy to use and inefficient in modeling. 2) Due to the lack of independent intellectual property, as well as technology blockade or preset security holes, data leakage and other issues, it has brought major potential risks. 3) The interface of commercial modeling tools is self-contained, so it is difficult to integrate with the mainstream independent simulation and evaluation tools. In the development process of SoS, the design results cannot be directly simulated, evaluated and verified conveniently, which affects the iterative optimization, the collaboration and traceability of the

Design, Evaluation and Optimization of SoS

291

results of each stage in the development process in specific domain. The domain-oriented of design tools has become more and more important. It is the general trend to develop a set of independently controllable and efficient design platform. On the basis of long-term theoretical research of system engineering and overall design experience, a domain model driven modeling platform with independent intellectual property is developed, which is combined with the domain characteristics and organizational structure of SoS, and is oriented to different professional teams. The platform addresses the practical problems encountered in the complex equipment engineering. It is based on the theories of system engineering, requirements engineering, ontology, model driven system design, system modeling and simulation, system evaluation, etc. The platform is designed to help overall design institution launch requirement analysis, system function and operation process analysis, equipment selection, unified human-computer interaction design, component design and integrated assembly design. It really realizes the integration of requirement analysis, system design and application integration, covering the whole life cycle of system design. It improves design efficiency, ensures design quality and gives full play to the comprehensive advantages of the system.

9.3.1.

Technology Architecture of Domain Model Driven Modeling Platform

Under the guidance of the methodology above, the modeling platform with independent intellectual property is designed and developed. The structure can be divided into system modeling specification layer, system resource layer, modeling framework layer, and application layer, which is shown in Figure 9.11.

Figure 9.11. Architecture of modeling platform. 1) Model Specification Layer: Meta-model/model of the SoS is defined. The four modeling views of dynamic, static, deployment, and service are provided and analysis modeling is supported.

292

Deping Zhang and Xuefeng Yan

2) System Resource Layer: The underlying resource and fundamental component in the modeling of complex system analysis are defined and especially the domain model and data are formed based on the domain knowledge and modeling constraint. 3) Modeling Framework Layer: It includes modeling of business scenario, top-level concept, comprehensive dictionary, organizational relationship and configuration. 4) Application Layer: The requirements analysis, functional decomposition, architecture design, and simulation model design are implemented based on the visual humanmachine interface. The platform implementation follows the standard modeling language SysML, and provides graphical modeling functions such as use case diagrams, activity diagrams, sequence diagrams, state diagrams, module definition diagrams, internal module diagram, and so on. Meanwhile, it has been extended according to domain characteristic, such as high-level concept map, organization chart, and deployment diagram, etc. The models can be exchanged based on XML and the data and models can be shared with mainstream modeling platforms such as Rhapsody, simulation platform and evaluation tools. During modeling, the platform takes the mission as input and supports the whole life cycle of system requirements analysis, system function analysis, system architecture design, unified human-machine interaction design, component design and integrated assembly design. In order to analyze the problems from different aspects by the designers, and to ensure the consistency of information and models between different diagrams, the model has strict correlation and constraint relation, which helps to analyze and check the inconsistency and ensure the design quality. The basic model library with domain characteristics can reduce design difficulty of the architecture, and improve modeling efficiency. The consistency of information and model among different views in each stage solves and ensures the consistency of requirement tracking in each stage of system design. The platform is based on a soft-bus architecture, and the open source eclipse workbench platform is used. The software implementation architecture is shown in Figure 9.12.

Figure 9.12. Implementation architecture of modeling platform. Based on the Eclipse Modeling Framework (EMF), meta-model of the extended DoDAF

Design, Evaluation and Optimization of SoS

293

and the general model framework are designed, and the meta-model is established in the specific field; Meanwhile, concepts (such as capabilities) of domain model is customized according to user requirements; The visual modeling portal is developed based on SWT/JFace, and the association between data model (EMF) and domain primitives is realized based on GEF, so as to realize a domain model-driven SoS modeling platform.

9.3.2.

Main Functions of Domain Model Driven Modeling Platform

(1) System Requirement Modeling Tool With the system requirements modeling tool, the developer takes mission task as the entrance and takes the mission as a profile. According to the top-down design pattern, several models can be used to analyze the system mission task, such as operational concept model, mission process model and interaction model. The mission task includes operational scenarios, battle areas, objects, main weapons, etc. The capability requirements of each mission are extracted, summarized and integrated to form itemized system requirements. (2) System Function and Architecture Modeling Tool With the system function and architecture modeling tool, system boundaries and constraints can be defined and the hierarchical system function models can be constructed from top to down. The use case analysis is applied in information interaction modeling, system state transition modeling. The requirements of control, process, interface and capacity within each functional module are formed. Meanwhile, the effectiveness indexes possessed by the system are decomposed. And the mapping between system effectiveness and functional model is established. Finally, according to space and operational position constraint, the preliminary equipment is selected, including special equipment and general equipment. Then the equipment list is formed and the function decomposition on each equipment is completed.

Figure 9.13. Sequential modeling.

294

Deping Zhang and Xuefeng Yan

Figure 9.14. Behavior-based state modeling. (3) Unified Human-Computer Interaction Design The auxiliary system and UI designers focus on interface prototype, human-computer interaction, the sub-system/equipment interface etc. The page and view modeling is also provided according to the function-concentrated and service-relevant principle. Then the design of display interface and manipulation interaction is completed. The code framework is generated automatically based on the designed interface, finally. (4) Deployment Design and Integrated Assembly Tools After completing each mission design, the deployment design tools construct “physical equipment” model and describe equipment attributes according to practical battlefield position. And the attributes include equipment name, IP address, process ability, etc. The components, special machine or display console is taken as units to complete the deployment of “physical equipment”. Based on system deployment design, the integrated assembly tool deploys component entities and relevant files from the component library to remote physical stations (assembly platform) automatically according to project and mission in distributed Ethernet environment. Therefore, the complex work caused by manual deployment can be avoided to improve the deployment efficiency. Meanwhile, the deployment state of each equipment can be queried to keep system state stable.

9.4. Research and Implementation Complex SoS Simulation Platform 9.4.1.

Overall Framework of SoS Simulation Platform

The simulation platform is composed of simulation engine and tools. The engine is the core of the platform. It is not only the “engine”, but also the “neural center” and “skeleton”

Design, Evaluation and Optimization of SoS

295

of the simulation system. It usually solidifies or predefines the SoS structure and integration strategy in the software framework. Meanwhile, the support and services of reuse are provided for communication, interaction, scheduling and synchronization between models, which is shown in Figure 9.15.

Figure 9.15. Overall framework of simulation platform. The simulation resource database is built into the simulation engine to manage and maintain the simulation models, data and unstructured documents. And it is also a public modules for information exchange between other simulation tools. The simulation tools include resource management tool, system modeling tool, entity assembly tool, scenario editing tool, experimental design tool, guidance control tool, simulation evaluation tool, etc. During the running preparation, the simulation resources are activated and loaded by the engine from the resource database. In the run time, simulation objects are scheduled by the scheduler uniformly. The input simulation object comes from external simulator, guidance system, actual installation system or monitoring system. The simulation results can be directly serialized to the resource database by the engine and can also be output to the visualization system. The whole life cycle process model of SoS simulation is proposed according to the analysis and summarization of the development and application typical simulation platform. The whole process is divided into five phases, such as resource development management, experiment theme scheme, simulation mission planning, simulation experiment running, evaluation and visualization. Each phase can be divided into several sub-phases, as is shown in Figure 9.16. Each phase and its sub-phases correspond to the specific functions of platform. 1) The functions of resource development management include model development, data management, and resource deployment and so on. In a visual integrated development environment, the use can construct the model system required by the application, generate the model code. Data management refers to the integrated management and access of model

296

Deping Zhang and Xuefeng Yan

Figure 9.16. The development and implementation of simulation system. parameters, system parameters, experimental parameters, simulation results and other data. Resource deployment means that the simulation resources can be copied to the simulation computer quickly according to simulation task requirement. 2) The concept of “theme” is a series of simulation task that can be supported by the same simulation entity, model and basic data parameters. The purpose of “theme” is to better realize combinatorial reuse of resources. Theme scheme includes component selection and loading, entity assembly, system or model parameter configuration and so on. 3) The task planning is based on the simulation theme. According to the experiment task, the initial state and operational plan of simulation objective can be determined in the experiment. The experiment factors and evaluation indexes can be planned and the experiment subjects can be configured. Finally, different kinds of experimental schemes (Word and XML) are generated according to the planning content. 4) When all types of resources are ready, simulation is scheduled and the simulation scene is managed by engine. 5) The system is analyzed and evaluated, and the simulaiton data is visualization after simulation begins. The following is the nonfunctional features of SoS simulation platform: 1) Model Combination. That is, the system is composed of sub-equipment or submodules. Multi-level combination is supported, such as “aircraft-sensor-antenna+ transmitter + receiver”, “aircraft-borne radar + motion + command + airborne weapons · · ·”, etc. The mechanism includes “scenario object + affiliated object” (STK), “entity + state pool + model”, “equipment + sub-equipment” (Flames). 2) Object/Service-Oriented Design. The concept, method and element support of object/service-oriented design are provided. The management and access of model object and entity object are offered. It includes the class-based, component-based, service-based, interface-based, event-based and dataflow-based design pattern. 3) Message/Event Interaction. Based on the response mode of message and event notification, the message/event receive-send and transmitting mechanism are provided and the receive-send is supported within the model or between models and object. 4) The Simulation Support Environment is Separated from Model. The simulation support environment and service are loosely coupled with the model. The model configuration and dispatch running are supported and the model can also be easily transplanted to other simulation environments through service and interface mechanisms. 5) Super real time simulation/fast computation.

Design, Evaluation and Optimization of SoS

297

6) The parameters of model, equipment and scenario can be configured flexibly. 7) Scalability. Model and other auxiliary functions can be extended.

9.4.2.

Complex SoS Simulation Engine

The functions of the simulation engine mainly include system, communication, component, object, event, time, scene, resource managements, as is shown in Figure 9.17.

Figure 9.17. The functions simulation engine. (1) System Management 1) The system management provides the construction, initialization, configuration, management and organization for the engine process and object. 2) It provides the engine global state maintenance, management and the notification of important state switch. 3) Simulation control service realize the functions such as star, suspend, recovery, replay, stop etc. 4) It supports local or remote displaying and controlling of the object organization, system configuration and running state. 5) The remote monitor service can obtain several information of engine running state by heartbeat message, such as computer state information, member status, component, simulation object. And remote-control command is received to control the engine state and simulation scene. (2) Communication Management 1) Communication management implements subscription and publishing of object classes and events using the configuration file. And the transparent communication among

298

Deping Zhang and Xuefeng Yan

simulation objects is realized in the distribute environment. 2) The statement management of middleware (such as RTI, DDS) is encapsulated and is compatible with the original middleware. 3) It provides the subscription and publishing mechanism to the simulation engine without such mechanism (such as TCP/IP, shared memory). 4) It provides the message encapsulation, combination, distribution, receiving, resolving, and publishing functionality. (3) Component/Service Management 1) The loading and initialization component/service is function provided, also the construction and organization for the component/service model. 2) A alarm is sent if any abnormality is detected. 3) The meta-object model is dynamically generated according to the description files of the component/service model. 4) It provides the functions of search, serialization of component/service. (4) Object Management 1) Object management provides the organization and management using reflective object tree for the object instances. 2) The construction, naming, ID management, search and delete services, and callback notification is designed for sophisticated object. 3) The objects aggregation and assembly is provided, and it also supports for dividing simulation entities into multiple simulation objects. 4) It combines with the communication management and encapsulates the communication function of the middleware, at the same time maintenances the states of global object automatically. 5) It monitors and manages the allocated memory block of the object and prevents memory leak. 6) It supports encapsulating the batch objects to perform the management and communication for stream object. 7) It supports updating the object attributes of dynamic length. 8) It supports configuring the updating strategy of object data. (5) Event Management 1) A unified interface for the implicit function call and data propagation are provided to the Federal, member and entity. 2) It transfers event according to event propagation range and description information. 3) It supports the management and coding of events in static and dynamic ways. 4) It supports event scheduling in synchronous or asynchronous ways. 5) It supports transferring parameters with dynamic length of the event. (6) Time Management 1) A unified logic time promotion and synchronized mechanism is provided for the entire federal, which ensures the causality correctness. 2) It integrates with time management service of lower layer middleware and supports all time management strategies (TC/TR/NTC/NTR) of HLA.

Design, Evaluation and Optimization of SoS

299

3) The logic step length of the simulation and the interval of physical trigger can be changed dynamically. 4) It provides default trigger clock. (7) Scene Management 1) A set of description language based on SRML is defined to support the describing, editing and modifying of simulation object, event, initial parameters, aggregation modes, behavior scheme. The scene files can be loaded, explained and executed by the simulaiton engine. 2) The scene scheduling is used to scan all activities of simulation object, dispatch simulation event, cooperate simulation time process and allocate computing resource. 3) It supports configuring and optimizing the scheduling time and function of model. 4) The different scheduling strategies and cycles is designed to support different models. 5) It supports parallel computing and scheduling on the multi-core or multi-thread. 6) It supports two scheduling modes including time cycle and “as fast as possible”. 7) The times can be set for specific simulation experiments. 8) New simulation scenario can be loaded on-demand in specified simulation time. 9) It supports producing objects, events and deleting object in specified time. And the scene events can be added and deleted dynamically. 10) It supports recording and replaying of scenes. (8) Resource Management 1) Resource management provides a unified interface, which is used to describe, store, access, link and load the parameters of system, component, determined, model and experiment, and simulation results. 2) It provides parameters description method according to SRML system and model. It can also import and export the data with the database. 3) The parameterized system with four layers is provided for object model systems, including class parameters, type static parameters, object instance parameters, object initial attributes

9.4.3.

Topic Planning

The concept of “topic” refers to a series of simulation missions that can be supported by the same simulation entities, models and basic data parameters. The purpose is to achieve the combinatorial multiplexing of resources better. Several functions are included in the topic planning, such as component selection and loading, entity implement, system or model parameters configuration and so on. The topic planning tools are used to define the following issues: which members and entities are necessary for simulation experiments topic? How about the relationship among entities. Which basic parameters are contained by the entity? Which kind of equipment, model and components are assembled in the entity? Which parameters are contained by the specific equipment, model and components? How to import parameters? It is responsible for instantiating the abstract simulation models as parameterized and combinatorial simulation entities, which can be edited and run. From the aspect of simulation platform, it is enough to use xml file or database to support the entity model and data.

300

Deping Zhang and Xuefeng Yan

However, when the entity model and the amount of data increase greatly, the direct operation through files or databases will become inadequate, the efficiency will reduce and the probability of error will increase greatly. Accordingly, it is necessary to design and develop a management tool which cover these functionality in an intuitive, humanized, flexible and use-friendly way. “Equipment database” is the achievement of the topic planning tool, which is composed of a series of parameters. Different with practical equipment database used for query, the software is a model-related database because it connects with the basic simulation model. Therefore, the parameters can be directly used in the subsequent simulation calculation. Topic planning tool has great benefit in guiding the subsequent model mechanism design, model system design, parameter design, model function design and so on.

9.4.4.

Scenario Edition

Several basic description information of scenario can be edited using this tool, such as scenario background, basic content; the simulation time including the start and end time, year, month, day, hour, minute, second, the operation area including the maximum rectangle operational area, random seed, the moment T . The force edition contains the description of formation, grouping, equipment and armament, initial position and state attributes of the warring parties, and simulation model and parameters. The created and edited instances are the entities such as aircraft and vessels etc. Other ammunition entities are created by model during simulation runtime. The contents of the armed force entity edition contain the basic motion state (position, velocity, direction), basic information (entity id, enemy-us attribute id, entity type, appearance type), characteristics (RCS and infrared characteristics), resources (entity weapon and fuel). In addition, it also contains the command and control relationship, armed force formation, carrying relationship. The others include logic editing, battlefield environment editing and so no.

9.4.5.

Experiment Design

The experimental scheme includes target, scene, object, factor, index, parameters etc. Experiment edition should provide visualization to connect and edit different parts of the experimental scheme. It provides the functions to classify, query and update experiment plan as well as object selection and factor design.

9.5. Implementation of Effectiveness Evaluation and Analysis Platform The effectiveness evaluation and analysis platform is an independent intellectual property software system orient to SoS analysis and optimization design. Based on experimental data from various sources including simulation, target test, rehearsal, the platform will perform the evaluation throughout the whole life cycle of SoS. It provides visual index system construction and evaluation for system capability and typical operation mission. The platform provides classic AHP, ANP ADC, TOPSIS models and big-data simulation-oriented

Design, Evaluation and Optimization of SoS

301

evaluation model. It also provides automatic modeling function. And automatic modeling analysis is performed for specific operation mission effectiveness index and sub-index. The platform analyzes the effectiveness contribution, designs the optimization scheme and generates the evaluation scheme and analysis report. It is helpful to analyze and evaluate the effectiveness evaluation modeling of various equipment systems, and to design and optimize the upper-level of auxiliary SoS.

9.5.1.

Technology Architecture of the Evaluation Analysis Platform

The evaluation analysis platform for operational is a new developed software used to evaluate the effectiveness in different stages including demonstration, development, test and usage. It provides quantitative basis to evaluate and optimize the SoS. The system effectiveness evaluation is performed throughout the life cycle of equipment. The general technical framework is shown in Figure 9.18.

Figure 9.18. General technical architecture of effectiveness evaluation analysis platform. (1) Data Source The simulation, experimental data are read from the interface of the simulation platform, which specifically include sonar, radar, photoelectric magnet, satellite acquisition, underwater sound, situation, environment, weather, etc. (2) Data Processing The data processing mainly includes data configuration, data cleaning, data generation, heterogeneous resource scheduling, model and parameters configuration, data backup, data preprocessing, data fusion and stream data processing.

302

Deping Zhang and Xuefeng Yan (3) Algorithm Framework

The algorithm framework contains classical model and index analysis model. The classical effectiveness evaluation used for SoS capability modeling includes analytic hierarchy process (AHP), network analytic hierarchy process (ANP), grey whitening weight function clustering[225][226], TOPSIS method[227][228], ADC method, sea method and machine learning model, etc. The mission oriented effectiveness evaluation models mainly include warning area range model, detection probability model, target recognition model, cooperative detection model, command decision control model, tracking target model and anti-interference model. (4) Evaluation Analysis The evaluation analysis includes index system construction, operation process design, effectiveness evaluation, automatic modeling, sensitivity analysis, optimization design and analysis report, etc.

9.5.2.

Implementation of the Evaluation Platform

In general, it is necessary for the platform to provide integrated effectiveness evaluation environment and apply unified management of index system, evaluation scheme, and evaluation task and data. The evaluation data with different sources can be used and the requirement of the complex SoS integral effectiveness evaluation can be supported. The analysis procedure is shown in Figure 9.19.

Figure 9.19. The technical framework of SoS effectiveness evaluation platform. According to the SoS effectiveness evaluation analysis requirement, the whole process includes index selection and evaluation index system construction. The algorithm is selected to build evaluation scheme according to the characteristics of index system. Therefore, evaluation missions can be formed. On this basis, the life cycle of effectiveness evaluation is carried out, including correlation analysis, index sensitivity analysis, optimization analysis for SoS structure. At the same time, the final report is generated.The primary func-

Design, Evaluation and Optimization of SoS

303

tional requirements of the effectiveness evaluation through the whole life cycle are shown in Figure 9.20.

Figure 9.20. The functional effectiveness evaluation platform. The main functions and system interface are constructed based on system requirements. (1) Evaluation Scheme Construction The evaluation scheme includes basic information of scheme, the evaluation index selection, and the formation of the operational effectiveness index system. According to the difference of evaluation task, the system capability and the index system construction of mission-oriented effectiveness can be evaluated. The function of evaluation scheme is responsible to create, edit and save the scheme, which is the process of evaluating single or multiple objects once. The evaluation object should be set for evaluation scheme at first and then index system or operational process are configured. (2) Evaluation Task The evaluation task is adopted to constructs different effectiveness evaluation, which include classical model and self-service modeling. The classical model is used to configure different classical evaluation models, including AHP, ANP, and ADC. It is also used to configure the consistency and effectiveness of basic attributes and model. For the specific analysis requirement of the users, the self-service modeling function mainly includes the modeling for different phases and effectiveness indexes. (3) Data Configuration The data configuration is to import evaluation analysis data from database through file or interface for each evaluation task. The mapping relationship between analysis data and evaluation index is determined and bound. The preprocessing is performed for the data, such as missing data process, normalization, feature decomposition and extraction. (4) Evaluation Analysis The effectiveness evaluation analysis function focuses on evaluating each configured evaluation task, including comparison analysis, accuracy analysis, credibility analysis for

304

Deping Zhang and Xuefeng Yan

the evaluation results of different models in the same mission. And the final report is generated. (5) Contribution Analysis In the effectiveness evaluation index system, the contribution analysis is used to analyze the contribution based on each basic index to the entire SoS and find the key index. It mainly analyzes the contribution of marginal index and interaction elements. And the optimal configuration scheme is generated for the performance index. (6) SoS Optimization It is mainly used to obtain key index factors which affect the overall effectiveness and the value range of different elements based on the contribution analysis. The multiple design schemes can be formed and then the optimal one can be selected. (7) Model Base Management The model base management is to manage each algorithm in life cycle of the effectiveness evaluation system, including DLL files, JavaScript scripts, R language etc. The algorithm contains factor extraction class, evaluation class, classification algorithm, verification, validation and determination relevant algorithms, contribution analysis algorithm, correlation and independence analysis algorithm, weight calculation method, fitting algorithm and different index analysis model for the mission oriented effectiveness evaluation, etc. The functions of model base management provides insert, delete, update, query and import for all kinds of algorithm model base and configures relevant parameters. (8) System Management System management provides authority management, user management, system help and other fundamental functions.

9.6. Mission-Oriented Complex System Design Optimization According to the effectiveness evaluation framework of simulation, the effectiveness-based SoS optimization design model is constructed based on evaluation and intelligence optimization. And the multi-objective solving problems can be addressed [229]-[233]. The meta-model technology is introduced and then the evaluation meta-model is constructed to solve the problem that the computation consumption is too high in the optimization design[234][235].

9.6.1.

Problem Description

In general, the optimization design problem can be defined as multi-objective optimization problem [236] .

s.t.

min f (x)

(9.1)

ci (x) = 0, i = 1, 2, · · · , mI

(9.2)

c j (x) ≥ 0, j = 1, 2, · · · , mJ

(9.3)

Design, Evaluation and Optimization of SoS

305

where x = (x1 , x2 , · · · , xn )T ∈ Rn are optimization design variables. f (x) = f1 (x), f 2(x), · · · , f l ((x))T ∈ Rn is the objective function vector (l ≥ 2). If k = 1, 2, · · · , l, each f k (x) represents a single objective function of the design variable x, ci (x) is constraint function where ci (x) = 0 is the equality constraint and c j (x) ≥ 0 is inequality constraint. Both are called constraint condition. For the SoS optimization problem, the objective function vector f (x) is usually called “black-box” function, which greatly limits the application scope of the multi-objective solving algorithm. The black-box problem can be solved by some intelligent optimization algorithms. However, when those algorithms is used to address the multi-objective optimization problem, the convergence process is essentially a stochastic process with uncertainty, which is difficult to control and slow. The simulation model is complex and the computation consumption is large. The slow convergence rate of multi-objective optimization results in the expensive calculation of optimization process, which limits the application of optimization design.

9.6.2.

Mission-Oriented Optimization of SoS

According to the definition of effectiveness and evaluation framework, the essence of effectiveness is to integrate objective function vectors of the optimization design base on the evaluation purpose. The effectiveness is taken as the optimization objective function of optimization design. The transformation and solution of multi-objective optimization problem is realized by effectiveness evaluation model. And the complexity of optimization can be reduced. However, the simulation system should be run in each iteration when the method is applied to optimization design, which will lead to excessive consumption of calculation, and even cause the infeasibility of optimization. The meta-model is taken as a simplified agent model of simulation model. The approximate and simplified mathematical model is obtained by fitting the simulation input and output data. The meta-model is used for simulation experiment, which can greatly reduce the computational cost and improve the simulation efficiency while ensuring the accuracy. In summary, this section provides a optimization design model and effectiveness meta-model based on effectiveness evaluation framework, and then the optimization design can be executed. The difficulty and computation consumption of multi-objective optimization problem are caused by the complexity of SoS. A mission-oriented and effectiveness-based complex system design simulation optimization framework is proposed. The optimization design is achieved according to simulation-based evaluation technology and heuristic optimization theory. The multi-objective transformation model based on effectiveness evaluation metamodel is constructed according to the heuristic optimization algorithm. Therefore, the fast solving multi-objective of optimization design can be realized, which is shown in Figure 9.21. The mission-oriented and effectiveness-based optimization design is described in detail as follows: Step 1. The effectiveness evaluation index system is constructed and confirmed according to the SoS military mission and operation task. Step 2. The simulation model is constructed by using model-driven modeling tools.

306

Deping Zhang and Xuefeng Yan

Figure 9.21. Effectiveness-based optimization design process.

Design, Evaluation and Optimization of SoS

307

Furthermore, simulation deduction system is constructed. Then according to experiment design, the simulation experiment scheme is designed and executed. Step 3. The samples are obtained based on operational experiment. The simulation model is designed and the contribution is computed. Then the experiment samples are obtained and divided into training and test samples. Step 4. The suitable meta-model satisfying fitting precision is constructed based on the model characteristic and training samples. Its prediction accuracy is evaluated by test samples. When the accuracy satisfies the requirement, the meta-model is considered to be completed and previous model can be replaced. Otherwise, another model is selected or more training samples are added. Then model is re-constructed until the accuracy is satisfied. Step 5. The simulation experiment is executed on the basis of operation simulation scenario and simulation model. Then the effectiveness can be evaluated. Step 6. The shortcomings and bottleneck of current SoS are searched through causal and sensitivity analysis. Step 7. Simulation optimization. The equipment attributes in a SoS are adjusted based on research idea, including performance, quantity, architecture, deployment and the application of operation, etc. The scenario space is formed up based on fundamental scenario. The optimization scheme can be solved by artificial intelligence algorithm.

9.7. Case Study A hypothetical optimal design of medium range surface-air missile defense is taken as an example, the proposed mission-oriented and effectiveness-based optimization for the SoS is demonstrated and verified.

9.7.1.

Weapon System of Systems and Operation Scene

In consideration of medium range surface-air missile air defense SoS, the equipment of the defense side is composed of surveillance and alertness system, command and control system, fire interception system, communication system and relevant maintenance support systems. The surveillance and alertness system includes multi-type air searching radar, which is responsible to monitor air target and process detection information initially. The command system is divided into two levels and is responsible to generate battlefield situation, provide decision-making assistance for commander and allocate operation mission for each operational unit. Assuming a air defense missile SoS is composed of three medium range air defense missile system. The blue target group is approaching at a certain height, the interception area of the air defense missile is shown in Figure 9.22.

9.7.2.

Operation Mission oriented Optimization Design

Combing with the optimization design requirement, bottom layer index of the evaluation index system is formed by considering the factors in the mission operational process of the

308

Deping Zhang and Xuefeng Yan

Figure 9.22. The interception diagram of the air defense missile. surface-air missile defense SoS. Then the operational effectiveness index system is constructed as shown in Figure 9.23.

Figure 9.23. The evaluation index system of the air defense missile SoS.

309

Design, Evaluation and Optimization of SoS

The index system is composed of three layers. The upper layer is the air defense missile SoS effectiveness. The medium layer is the second-level indexes obtained by decomposing the effectiveness of air defense missile, which includes fire coverage, flight performance, early warning detection, control, damage capabilities as well as the research manufacturing cost. The lower layer is the bottom indexes obtained by decomposition indexes in medium layer, which is also called SoS design index. In general, it is considered that indexes in different layer have same weight.

Figure 9.24. Result of the Sobol’ global sensitivity. The Sobol’ global sensitivity analysis is adopted to perform sensitivity analysis for 23 indexes of the bottom layer. Several indexes have been determined that influence the effect of missile interception, such as the maximum interception slant-range, single shot killing probability, the number of fire channels, system reaction time, hit accuracy and tracking target distance, as is shown in Figure 9.24. The relationship among the screened six indexes and final interception effect is shown in Figure 9.25. Table 9.3. Effectiveness optimization result Hit Accuracy Consider cost Nonconsider cost

0.92

Single Shot Killing Probability 0.912

System Reaction Time 6

Tracking Target Distance 35

Number of Fire Channels 2

Maximum Interception Interception Effectiveness Slant-range 7.13 × 103 0.9574

0.93

0.927

5

30

4

8.05 × 103

0.9813

The optimization design is performed on the screened indexes according the metamodel based SoS optimization design. The meta-model between the optimization index and effectiveness is constructed at first. The training and test sample sets can be obtained by using the Latin hypercube experiment. The SoS effectiveness evaluation can be executed

310

Deping Zhang and Xuefeng Yan

Figure 9.25. The relationship between main design index and interception effect. based on the training sample set and then result set can be obtained. The SVR meta-model between the sample set and evaluation result set can be constructed by SVR modeling. The genetic algorithm (GA) is adopted to optimize meta-model. And then the optimization process and result are given respectively whether to consider research manufacturing cost or not, which are shown in Figure 9.26 and Table 9.3.

Design, Evaluation and Optimization of SoS

311

Figure 9.26. Genetic algorithm optimization. From the result, it can be seen that, there is a certain compromise between the optimization results of each index and the results without considering the development and production costs. The best optimization result can be obtained under the limitation of cost.

References [1] Zhu, Baoliu, Rongchang Zhu, and Xiaofei Xiong. (2006). Operational Airplane Effectiveness Evaluation. Beijing. Aviation Industry Press. [2] Li, Zongcheng. (2004). Military Operations Research. Beijing. Chinese People’s Liberation Army Publishing House. [3] Lambeth, Benjamin. (2006). American Carrier Air Operation Power at the Beginning of New Century. Beijing. Comprehensive Institute of Naval Equipment Research Institute, 2: 47-53. [4] Xu, Zhiqiang. (2013). Research on Command and Control Effectiveness Evaluation Method of Task-oriented Joint Operational Systems. National University of Defense Technology. [5] Meng, Yalei. Effectiveness Evaluation of Integrated Avionics System Based on AHP Method. Xian Shiyou University, 2015. [6] Zhang, Yongli, Zhishui Sun, and Rongkun Zhou. (2015). Efficiency Evaluation of Manned/Unmanned Aerial Vehicles Coordinate Operation Based on AHP-Fuzzy Synthetic Evaluation Method. Shipboard Electronic Countermeasure, 38(06): 8084+92. [7] Zen, Tao, PingJun Wang, XingMing Zhang. (2005). Evaluation Method of Operational Effectiveness for Surface-to-Surface Missile Weapon System. Beijing. National Defence Industrial Press. [8] Jamison K D, Lodwick A. Fuzzy linear programming using a penalty method. Fuzzy Sets and Systems, 2001, 11(1): 97-110. [9] Gao, Xiaoying, Chongyang He. (2005). Fuzzy Evaluation Method of the Electronic Warfare System Effectiveness. Fire Control & Command Control, 30(1): 69-72. [10] Chen, Guangting, Zheyong Qiu. (2010). Mathematical Modeling. Beijing. Higher Education Press, 5: 125-131. [11] Zhang, Zuiliang. (1993). Military Operations Research. Beijing. Military Science Press.

314

References

[12] Luo, Pengcheng, Jinglun Zhou, Guang Jin, etc. (2014). Evaluation Analysis Method of Operational Effectiveness and Operational Capability for Equipment System of Systems. Beijing. National Defense Industry Press. [13] Hu, Lei, Hao Li, Shiqiang Yan, etc. (2015). Building of Operational Effectiveness Evaluation Index System of Early Warning Satellite System. Fire Control & Command Control, 40(5): 46-73. [14] Xu, Xiangli, Xiaofeng Hu. (2014). A Complex Network Based Network Space Operational Effectiveness Evaluation Index Framework, 28(1): 33-41. [15] Lu, Yanjing, Ben Chen, Yingwu Chen, etc. (2012). Capability weightiness degree analysis of weaponry system-of-systems based on BN theory. System Engineering and Electronic, 34(8): 1601-1612. [16] Luo, Pengcheng, Panfeng Fu, Jinglun Zhou. (2005). Framework to Evaluate the Combat Capability of Weapons SoS. System Engineering and Electronic, 27(1): 1072-1075. bibitemLuY2010 Lu, Yanjing, Xiaoke Zhang, Yingwu Chen, etc. (2010). Incomplete Weight Information Area Weapon System of Systems Capacity Evaluation Method. System Engineering, 28(4): 94-100. [17] Fan, Ke, Xuanhong Fan, Wenhong Dong, etc. (2015). Operational Effectiveness Assessment of Tanker Airplane Based on Multi-index Comprehensive Evaluation Method. Journal of Naval Aeronautical and Astronautical University, 20(2): 191196. [18] Du, Hongmei, Hongfa Ke. (2015). Connotation Analysis of Equipment Combat Capability and Equipment Combat Efficiency. Ordnance Industry Automation, 34(04): 23-27. [19] Wu, Qibo, Fangchao Ye, Xuejun Wan. Study on Measures of Effectiveness of Weapon System. Electronic Product Reliability and Environmental Testing. 2010, 28(04): 27-31. [20] Chen, Wenqi. Assessment and Analysis on the Operational Efficiency of Air Defense Missile Weapon System. Xiamen University. 2006. [21] Tang, Linjie, Jingzhuo Jiang. (2007). Region of war Force Planning Simulation Method in Informational Condition. Military Operations Research and Systems Engineering, 21(1): 32-36. [22] Chen, Qi, Ning Jiang, Mingshan Lv, etc. (2013). Navy Exercise Effects Evaluation Methods and Key Technology based on Combat Effectiveness. System Engineering and Electronic, 35(6): 1226-1230. [23] Zhang, Liang, Zhennan Zhao, Li Wang, etc. (2015). Evaluation of Weapon Systemof-Systems Based on Combat Effectiveness. Journal of Sichuan Ordnance, 34(3): 50-58.

References

315

[24] Yu, Qinizhang, Yingchao Zhang, Jing Zhang, etc. (2011). Synthetic Evaluation Methodology of Operational Effectiveness for System of Armament Systems Based Overall Effect. System Simulation Technology, 7(3): 183-189. [25] Li, Xiaoxi, Haoguang Chen, Daxi Li, etc. Study on Combat Effectiveness Prediction Model Using Elman Feedback Network. Journal of System Simulation, 27(1): 183189. [26] Lan, Yushi, Kan Yi, Heng Wang, etc. (2013). Delay Assessment Method for Networked C4ISR System Architecture. Systems Engineering and Electronic, 35(9): 1908-1914. [27] Zang, Lei, Xiaoyuan Jiang, Yu Wnag, etc. (2008). Research on Metrics of Measure of Force Effectiveness for C4ISR System. Journal of System Simulation, 20(3): 574611. [28] Wang, Yu, Xiaoyuan Jiang, Chunjiang Wang, etc. (2006). Research on Measures of Merit of C4ISR System. Computer Simulation, 23(9): 1-3. [29] Liu, Junxian, Zhiping Jiang, and Zhen Shu, et al. (2009). Weighting Determination Method of C4ISR System Effectiveness Indices under System Confrontation. Systems Engineering and Electronics, 31(10): 2418-2421. [30] Jia, Zi ying, Feilong Yan, and Haisheng Wang. (2015). Evaluation of Combat Effectiveness of Antiaircraft System based on Net Effectiveness. Fire Control & Command Control, 38(5): 82-85. [31] Hou, Lei, Wang Zeng, and Haiwen Fan, et al. (2019). Research of Integrated Intelligent Flexible Simulation Evaluation Technology. Fire Control & Command Control, 44(11): 174-179+185. [32] Weng, Xinjin. (2019). Effectiveness Evaluation of Radar Jamming Based on Machine Learning. University of Electronic Science and Technology of China, 2019. [33] Yang, Xin. (2015). Research and Realization of Weapon Effectiveness Evaluation Model and ITS Self-learning. Southeast University of China. [34] Zhang Le, Zhong Liu, and Jianqiang Zhang, et al. (2013). Reduction Method of Weapon System-of-systems Assessment Index System Based on Autoencoder. Journal of Central South University (Science and Technology), 44(10): 4130-4137. [35] Zhang, Yong. A Study on Theory and Method in Intellectualization of Evaluation. Zhejiang Gongshang University. 2014. [36] Xu, Xiangyi, Liang Lei, Daohua Liu and Yun Liu. (2013). Agent Model Based on Non-radial Symmetric Generalised Radial Basis Function Neural Network. Computer Application and Software, 30(05):171-173. [37] Pang, Yanjun, Limin Liu, and Kaidi Liu. (2010). New method for effectiveness evaluation on air defense communication system. Journal of Liaoning Technical University (Natural Science), 29(05): 998-1001.

316

References

[38] Jia, Ziying, Songhui Chen, and Defei Wang. (2013). Research on Survivability Indexes of Networked Anti-aircraft System. Fire Control & Command Control, 38(10): 84-87. [39] Tin, Yingyuan, and Chong Ren. (2011). Evaluation Methods for Operational Effectiveness on Guided Depth-charge. SICHUAN Ordnance Journal, 32(08): 42-44+51. [40] Du, Hongmei, Ke Hongfa. (2015). Connotation Analysis of Equipment Combat Capability and Equipment Combat Efficiency. Ordnance Industry Automation, 34(4): 23-27. [41] Wu, Qibo, Fangchao Ye, Xuejun Wan. (2010). Study on Measures of Effectiveness of Weapon System. Reliability and Environmental Engineering Center, 28:27-31. [42] Luo, Congwen. Analysis of Air Defense Warfare Efficiency of Warship Formation. Nanjing. Naval Command Academy. 2007, 35-40. [43] Meng, Qingcao, Guang Yan. (2015). Design of Effectiveness Estimation Index System of Air Defense of the Aircraft Carrier Formation. Ship Electronic Engineering, 35(10): 1-4. [44] Zhang, Xudong, Jie Peng, Jian Wang. (2011). Evaluation of Air Defense Combat Capacity in Surface Ship Formation Based on Fuzzy Theory. Ordnance Industry Automation, 30(10): 39-42. [45] Sun, Zhiwei, Tao Zhang. (2016). Assessment of Anti-air Defense Formation for A Group of Surface Warships Based on Fuzzy Evaluation & TOPSIS Method. Ship Electronic Engineering, 34(4): 33-37. [46] Ma, Yulin, Xiang Luo, Wanhai Guo. (2007). Warship Formation Air Defense Capability Evaluation. Ship Electronic Engineering, 27(1): 50-53. [47] Kuang X H.(2009). U.S navy aircraft carrier of operational capability analysis. Journal of National Defense Science and Technology, 30(6):58-63. [48] Zhuang, Shijie, Dishan Qiu, Jichang Sha. (1998). Study on the Operational Effectiveness Criteria System Design of Submarine Anti-vessel Missile Weapon System. Fire Control & Command Control, 20(4): 68-73. [49] Wu, Tao, Huicheng Xia. (2006). Research of Air Defense Warfare Capability of Surface Vessels Formation System. Journal of Naval University of Engineering, 18(5):108-110. [50] He, Jiazhou. (2009). Study of Multi-Platform Cooperative Engagement Data Fusion. Ship Electronic Engineering, 28(3), pp. 40-43. [51] Liu, Changyu, Xiaofeng Hu, and Pi Luo. (2007). The Systematic Thought of the Theoretical Basis of War Engineering. Defence Technology, 8(1): 48-53.

References

317

[52] Zhao, Wenting. (2013). Operational Effectiveness Measurement Indexes and Validation Methods for Air Defense Missile Network-centric Systems. Modern Defence Technology, 41(2): 6-11. [53] Liu, Yuan, Hao Jingjing, and Zhigeng Fang. (2014). Petri net model for performance evaluation of manufacturing system under uncertain information. Computer Integrated Manufacturing Systems, 20(05): 1237-1245. [54] Xu, Linsheng. (2010). Research on Multi-attribute Evaluation Method Used in the Process of Weaponry Argumentation. Nanjing University of Science and Technology. [55] Cao, Shucheng and Bo Wang. (2011). Commanding Effectiveness Evaluation of Systematic Artillery Battle Based on Information System. Ship Electronic Engineering, 31(05): 57-58+123. [56] Yin, Chun, Yanyan Huang, Jianyu Wang, and Zhiquan Wang. (2009). Guidance Mode of Evaluation Index Systems of Weapon Equipment Operational Effectiveness. Journal of Nanjing University of Science and Technology, 33(06): 779-784. [57] Wei, Xiulei, and Xi Zhang. (2019). Index System Design and Evaluation for Air Defense Capability of the Aircraft Carrier Formation. Ship Electronic Engineering, 39(01): 14-17+26. [58] Zhang, Le, Zhong Liu, Jianqiang Liu, and Xiongwei Ren. (2013). Index Space Modeling of Capability Assessment for Long-Range Precision Sea Strike System SoS. Journal of Ordnance Engineering College, 25(3): 5-10. [59] Ha, Junxian, and Jinsong Wang. (2016). Index System of Cyberspace Operation Command Efficiency Assessment Based on Social Network Analysis. Journal of Armored Force Engineering College, 30(06): 1-5. [60] Ba, Hongxin, Bifeng Xue, Sha Li, and Jianhua Ye. (2016). Index System and Modeling Constructing of Operational Effectiveness Evaluation for Air Force Weapon Control Network. Command Information System and Technology, 7(04): 41-48. [61] Meng, Qingcao, and Guang Yang. (2015). Design of Effectiveness Estimation Index System of Air Defense of the Aircraft Carrier Formation. Ship Electronic Engineering, 35(10): 1-4+61. [62] Hu, Lei, Hao Li, Shiqiang Yan, and Li Zhang. (2015). Building of Operational Effectiveness Evaluation Index System of Early Warning Satellite System. Fire Control & Command Control, 40(05): 65-68+73. [63] Hu, Xinwu, Pengcheng Luo, and Xiaonan Zhang, et al. (2020). Effectiveness Evaluation Method Based on System Simulation Big Data. Fire Control & Command Control, 45(01): 7-11+17. [64] Wu, Wenfeng, and Xiaofeng Hu. (2016). Capability Evaluation Framework of Networked Combat System Based on Big Data. Military Operations Research and Systems Engineering, 30(02): 26-32.

318

References

[65] Li, Yuanzuo, Xiaoduan Yang, Xiangmin Yin and Shuyun Liu. (2009). Research on the Operational Ability Evaluation of MSOAMFSOS. Command Control & Simulation, 31(04): 5-9. [66] Lu, Yingbo, Xiaochao Qian, and Wei Chen, et al. (2018). Research on Construction Method of Data-driven Equipment Effectiveness Evaluation Model. Journal of System Simulation, 30(12): 4587-4594+4601. [67] Qin, Maosen, Danling Zhao, and Kewei Yang. (2018). Effectiveness evaluation of anti-submarine activities based on combat network. Systems Engineering and Electronic, 40(7): 1513-1520. [68] Li, Huatao. (2019). Research on Combat Effectiveness Evaluation of Infrared Air-toAir Missile Weapon System. Nanjing University of Aeronautics and Astronautics. [69] Cao, Fangfang, Weidong Liu, and Juanli Li. (2011). Efficiency Evaluation of MultiUUVs Coordinated Operation Based on Underwater Network. Computer Measurement & Control, 19(06): 1397-1399+1402. [70] Cheng, Cong. (2013). Task allocation and attack effectiveness evaluation of UAV cooperative operation. Nanjing University of Aeronautics and Astronautics. [71] Cong, Hongri, Tian Jin and Zheng Chu. (2018). Search Method and Effectiveness Evaluation of Enti-submarine Patrol Aircraft Ahead of Voyaging Aircraft Carrier Formation. Journal of Naval Aeronautical and Astronautical University, 33(06): 560-566. [72] Ba, Hongxin, Zheng Fang and Yafei Chen. (2014). Construction of Index System of Air Force Networked Command Information System Operational Effectiveness. Command Control & Simulation, 36(06): 21-26. [73] Si, Guangya, Xiang Gao, Yang Liu, and Lin Wu. (2016). Method for Building Effectiveness Evaluation Index System Based on Big Simulation Data. Big Data Research, 2(04): 57-68. [74] Wu, Wenfeng and Xiaofeng Hu. (2016). Capability Evaluation Framework of Networked Combat System Based on Big Data. Military operations research and system engineering, 30(02): 26-32. [75] Xu, Xiangli, and Xiaofeng Hu. (2014). A Framework of Operational Effectiveness Evaluation Index System in Cyberspace Based on Complex Network Theory. Military Operations Research and Systems Engineering, 28(01): 33-37+41. [76] Zhang, Zhuang, Linlin Li, and Yunfei Lu. (2018). Establishment of Evaluation Index System for C4ISR Based on Multiple Correlation Sensitivity of Information. Fire Control and Command, 43(10): 68-73 [77] Pan, Changpeng, Yulong Han, and Yifu Zhuang. (2019). Research on the Operational Effectiveness Evaluation Indices System of Ship-based UAV Formation Cooperative Air-to-sea Attack. Tactical Missile Technology, 2: 25-32.

References

319

[78] Li, Wen, and Jian Chen. (2017). Review and Prospect of Cooperative Combat of Manned/Unmanned Aerial Vehicle Hybrid Formation. Aerospace Control, 35(03): 90-96. [79] Xu, Guanhua, Yanfei Dong, Yuan Yue, and Chao Wang. (2016). The R/S-mission of UAV Operational Effectiveness Assessment. Fire Control & Command Control, 41(07): 60-64. [80] Qin, Yingxiang. (2015). Research on Reconnaissance Mission-Oriented Effectiveness Assessment of UAV Formation. Hefei University of Technology. [81] Liu, Xiaoli, Wei He, and Jun Huang. (2013). The Attack Effectiveness Evaluation of Two-UAVs Formation. Aircraft Design, 33(02): 64-67. [82] Li, Kun, Jie Peng, and Shuang Song. (2018). Evaluation Index System of Communication Equipment Operation Test. Communications Technology, 51(07): 1649-1655. [83] Pan, Changpeng, Yulong Han and Yifu Zhang. (2019). Research on the Operational Effectiveness Evaluation Indices System of Ship-based UAV Formation Cooperative Air-to-sea Attack. Tactical Missile Technology, 2: 25-32. [84] Chen, Xia. (2017). Architecture Oriented Modeling and Simulation Method for Combat Mission Profile. Chinese Journal of Ship Research, 12(03): 135-141. [85] Xu, Yong, Cuirong Yang, and Zichen Yang. (2013). Structural Optimization Method of the Mission Profile for the Combat System of Surface Warships. Chinese Journal of Ship Research, 8(04): 97-102+122. [86] Xiong, Hua, Xiangling Ma, and Po Gao. (2013). Analysis of Typical Combat Mode and Operational Flow of Intelligent Anti-ship Missile. Maneuverable missile, 6: 1719. [87] Sui, Yuanli, Wei Wang, and Tao Hu, et al. (2010). Modeling and Simulation of Warship Mission Flow Based on Mission Profile. Ship Science and Technology, 32(04): 117-120. [88] Yang, Chunhui, Jun Wei, and Fang Li. (2009). On Technology of Constructing Success Oriented Mission Profile of Conventional Submarine. Journal of Naval University of Engineering, 21(04): 98-102. [89] Xu, Yong, Cuirong Yang, and Zichen Yang. (2013). Structural Optimization Method of the Mission Profile for the Combat System of Surface Warships. Chinese Journal of Ship Research, 8(04): 97-102+122. [90] Yu, Zhouchuan, Gaoping Hu, and Wu Zhan, et al. (2019). Modeling and Simulation Based on UPDM of the Information Flows of Cooperative Assault Operations to Ship. Ship Electronic Engineering, 39(09): 81-85+145. [91] Liang, Zhenbin, Jinghui Li, and Jun Jiang. (2019). Research on the Simulation and Optimization of Aircraft Carrier Formation’s Surface Warfare Process. Computer & Digital Engineering, 47(03): 562-566.

320

References

[92] Zhang, Yi, Wu Zhan, and Yinghui Guo, et al. (2019). Validation Method of Operational Architecture Design Based on Information Process Simulation. Command Control Simulation, 41(01): 131-135. [93] Yuan, Zong. (2016). Warship’s RMS Modeling and Simulation Based on the Task Process. Chinese Journal of Ship Research, 11(02):5-11+26. [94] Wei, Yong, Yanxue Xu, Junyuan Gu, and Renbo Yu. (2011). Modeling and Simulating for Mission Completion Success Probability Based on Phased-Mission System. Computer Simulation, 28(3): 5-10. [95] Wang, Chen, Jun Jiang, and Jinghui Li. (2017). Optimization Design of the Formation Command Post’s Position Based on Operational Process Simulation. Command Control Simulation, 39(04): 96-100. [96] Fan, Pengcheng, Li Zhu, Yongwang An, and Changbo Yang. (2017). Constructions of Operational Effectiveness Index System of Aerospace Electronic Reconnaissance System Based on PFT. Aerospace Electronic Warfare, 33(04): 26-30. [97] Cai, Qiufang, and Guohui Li. (2013). PFT-Based Comprehensive Effectiveness Evaluation Method for the Space Early Warning System. Digital Technology and Application, 5: 126-128. [98] Zhang, Zhuang, Linlin Li, Yunfei Lu. (2018). Establishment of Evaluation Index System for C4ISR Based on Multiple Correlation Sensitivity of Information. Fire Control & Command Control, 43(10): 68-73. [99] Li, Chengyang, Xiaowu Zhang, Lining Xing. (2005). The Method of Dynamic Selecting Measures Based on Sensitivity Analysis. Computer Simulation, (05): 120123+136. [100] Li, Bo, Haoran Luo, and Linyu Tian, et al. Sensitivity Analysis of Ship Formation Operational Effectiveness Based on DBN Effectiveness Fitting. Acta Aeronautica ET Astronaution Sinica, 40(12): 156-166. [101] Zhi, Tao, Fuming Wang, Songliang Zhang. (2008). Valuation system of Formation Network Warfare based on Principal Components. Aerospace Electronic Warfare, 2: 55-58. [102] Ju, Rusheng, Zimin Cai, Mei Yang, and Song Wang. (2019). Evaluation Method of Simulation Results Based on Principal Component Analysis. Journal of System Simulation, 31(12): 2678-2684. [103] Zhou, Lifeng. (2019). Method of Measurement Radar Effectiveness Based on Principal Components Analysis. Modern Radar, 41(02): 7-9+25. [104] Liu, Dan. (2015). The Assessment of Airspace Utilization Efficiency Based on Primary Component Analysis. Civil Aviation Flight University of China.

References

321

[105] Wang, Penghua, Yuan Li, and Wenge Wang. (2009). Combat Effectiveness Evaluation of Self-Propelled Artillery Based on Principal Component Analysis. Ordnance Industry Automation, 28(05): 24-25. [106] Li, Lei, Juliang Jin, and Zhongming Liang. (2009). Parameters Optimization of Kernel Principal Component Analysis and its Application in Evaluation of Water Resources Sustainable Utilization. Advances in Science and Technology of Water Resources, 29(04): 36-38+66. [107] Ji, Ming, and Li Ma. (2014). Research on Simulation Experiment Factors and Index Selection for System Effectiveness Evaluation. Military Operations Research and Systems Engineering, 28(03): 61-65. [108] Gao, Fei, Fuxiang Gao, Yu Wang, and Shaoqing Chen. (2012). Construction of Measures of Effectiveness System for Digitized Army’s C2 System. Ship Electronic Engineering, 32(03): 1-3+16. [109] Wang, Rui, and Ning Jiang. (2018). Evaluation Method for Operational Effectiveness of Warship Formation Communication. Modern Defence Technology, 46(01): 108114. [110] Zhang, Lijun, and Yeping Wang. (2011). Research on Relative Effectiveness Measurement Method of Comprehensive Evaluation Model. Statistics & Decision, 20: 18-21. [111] Ma, Li, Yunxiang Ling, and Yaohong Zhang. (2016). Modeling Method of Air Defense and Antimissile Process in Uncertain Battlefield Environment. Ship Electronic Engineering, 36(9): 17-21+61. [112] Guo, Qisheng, Kang Wang, Yanping Fan, and Ge Mu. (2014). Amount Demand Analysis Method Based on Operational Activity Decomposition for Equipment System. Journal of Academy of Armored Force Engineering, 28(03): 1-5+22. [113] Zhang, Yi, Guohua Yang, and Zhiwei Li. (2017). Testing Method of Submarine Combat System Operational Capability Based on Information Flow. Ship Electronic Engineering, 37(07): 25-28+69. [114] Wei, Zhengxian, Ming Song, Guisheng Yin, Zhe Zhang, Hongbin Wang, and Wansong Zhang. (2012). Event-active Flow Oriented Naval Vessel Combat Mission Effectiveness Evaluation Model. Journal of System Simulation, 24(07): 1438-1442. [115] Zhao, Wenting. (2013). Operational Effectiveness Measurement Indexes and Validation Methods for Air Defense Missile Network-Centric Systems. Modern Defence Technology, 41(02): 6-11+29. [116] Cheng, Ben. (2012). Capabilities-based Weapon System-of-Systems Assessment Methods and Application Research. National University of Defense Technology.

322

References

[117] Wang, Yanhui, Guotong Lu, and Jiaxiang Li. (2019). Application of S type membership function in the reliability evaluation of model. Ship Science and Technology, 41(01): 76-79. [118] Wu, Donghai, Qi Tan, and Qingsong Wang, et al. (2017). Connotation and Frameworks of Mission Profile for Military Communication Systems Based on Complex Networks Theory. Communications Technology, 50(11): 2527-2534. [119] Wang, Nanxing, Jinsong Wang, and Yulin Ji. (2017). Analysis of Working Process of Cyberspace Operational Commanding Institution Based on Petri Net. Journal of Academy of Armored Force Engineering, 31(04): 10-13. [120] Sun, Yuanshun. (2017). Research on Theoretical Model and Simulation Algorithm of Communication Network Efficiency Evaluation. Beijing University of Posts and Telecommunications. [121] Li Chao. (2011). Research on the Robustness Evaluation Methodology & Analysis of Military Communication Network Effectiveness. National University of Defense Technology. [122] Hu, Xiaofeng, Xiaoyuan He, and Dehu Rao. (2015). A Methodology for Investigating the Capabilities of Command and Coordination for System of Operation Based on Complex Network Theory. Complex Systems and Complexity Science, 12(02): 9-17. [123] Jiang, Zhipeng, Duolin Zhang, Le Wang and Xiangfeng Shi. (2015). Evaluation Method for Node Importance of Command Network with Multiple Constraints. Journal of PLA University of Science and Technology (Natural Science Edition), 16(03): 294-298. [124] Shang, Huilin. (2013). Research on Modeling and capability evaluation method of weapon equipment system of systems operational network. National University of Defense Science and Technology. [125] Yang, Cheng and Zeping Liu. (2013). Study of Evaluation Based on Information System Operational Command and Control Capacity. Computer Engineering and Design, 34(03): 1087-1091+1107. [126] Teng, Kenan and Andong Sheng. (2010). Research on Metric of Network Effect in Ship Formation Cooperation Anti-missile Operation. Acta Armamentarii, 31(09): 1247-1253. [127] Wang, Zhijian. Organization and Efficiency Evolution on Cooperative Operation of Missile Force. Harbin Institute of Technology, 2010. [128] Gao, Xueqiang, Rijie Yang and Linlin Zhang. (2010). Quantitative Evaluation Models for Countermeasure Effectiveness of Anti-torpedo Solution. Journal of detection and control, 32(02): 88-91.

References

323

[129] Yang, Xiaoduan, Yuanzuo Li, Xiangmin Yin, Shuyun Liu. (2009). System Effectiveness Evaluation Model for Army Materal Facility Meta-Synthesis System. Command Control & Simulation, 31(06): 1-5. [130] Li, Jun, Jiangang Yan, and Shifeng Yang, et al. (2016). Optimization Research of Combat Mission Distribution Procedure Based on Complex Net. Ordnance Industry Automation, 35(11): 84-89. [131] Gao, Xiang, Lin Wu, and Yang Liu, et al. (2015). Weapon System-of-Systems Effectiveness Analysis Based on Weighted Network of Networks Model. Journal of Command and Control, 1(02): 175-180. [132] Zhang, Xin, and Yang Guo. (2020). Betweenness-Degree Entropy Invulnerability Measurement Method in Complex Networks. Computer Engineering and Applications, 56(12): 105-111. [133] Rong, Ming, Xiaofeng Hu, and Jingyu Yang. (2019). Resilience Analysis and Evaluation of Combat System of Systems Architecture Based on Dynamic Super Network. Journal of System Simulation, 31(06): 1055-1061. [134] Gang, Janjun, Xiongbing Ye, and Wei Wang. (2019). Hypernetwork Modeling Research on Carrier Formation Fighting SoS. Ship Science and Technology, 41(09): 6-11. [135] Liang, Dejun, Gangning Song, and Ming Zhao. (2019). Evaluation Method for Node Importance of Communication Network Based on Complex Network Analysis. Communications Technology, 52(03): 674-679. [136] Gang, Jianxun, Xiongbing Ye, and Hongyuan Yu. (2018). Modeling and Analysis of Carrier Formation Combat SoS Base on Hypernetworks. Command Control & Simulation, 40(05): 6-10+78. [137] He, ZhenAn. (2018). Simulation Research on A Command Information System Based on Complex Network Theory. Xian Institute of Optics & Precision Mechanics, Chinese Academy of Sciences. [138] Ma, Longbang, and Ping Guo. (2016). Exploratory Analysis of Invulnerability Optimization Scheme of Command Information System Structure Based on Super Network. Military Operations Research and Systems Engineering, 30(04): 32-39. [139] Shi, Bingzheng. (2016). Cooperative Track Procedure Design Based on Distributed Network Command & Control System. Command Control & Simulation, 38(03): 102-107. [140] Ma, Li, and Mingzhi Zhang. (2016). Research on Network System Effectiveness Evaluation Modeling. Military Operations Research and Systems Engineering, 30(01): 12-17.

324

References

[141] Hu, Xiaofeng, Xiaoyuan He, Dehu Rao, and Shengming Guo. (2015). Research on Mechanism of Command and Coordination for System of Systems Operation Based on Complex Network Theory. Journal of Command and Control, 1(01): 5-13. [142] He, Baowei. Complex Networks Based Research on Invulnerability of Command and Control System. Xidian University, 2014. [143] Li, Renjian, Guangya Si, Yu Zhang, and Fei Wang. (2014). Visualizing Analysis for Effectiveness of System of Systems Based on Network of Networks. Journal of System Simulation, 26(09): 1944-1949+1955. [144] Liu, Yue. (2012). Research on Modeling and Evaluation Methods of Command and Control Organization Based on Complex Networks. National University of Defense Technology. [145] Ding, Yuanming, Liangchen Jia, Dapeng Jiang, and Lei Qiu. (2012). Effectiveness Evaluation for Submarine Communication System Based on Analytic-Exploring Hierarchy Process. Fire Control & Command Control, 37(05): 195-198. [146] Huang, Bo, Yong Zhang, Jinda Chang, and Liang Li. (2011). Fuzzy Multi-factorial Evaluation Model of Torpedo Weapon System Effectiveness in Submarine Based on AHP. Fire Control & Command Control, 32(152): 33-35+41. [147] Liang, Dong, Changyi Chen, Yanping Fan, and Yanqi Bai. (2010). Research on Armored Equipment System Warfare Efficiency Evaluation Based on ADC-AHP. Microcomputer Information, 26(03): 15-17. [148] Cui, Yanming. (2009). Research on Evaluation Index System of Urban Traffic Based on Correlation Analysis. Forest Engineering, 25(04): 55-57. [149] Han Wei, Zhenyang Li, and Xichao Su. (2019). Effectiveness Evaluation of Carrier Aviation Support System Based on Improved ANP and Extension Theory. Journal of Ordnance Equipment Engineering, 40(08): 100-105+197. [150] Shi, FuLi, Feng Yang, Yongping Xu, and Guodong Wu. (2011). Power Index Method for Operational Capability Evaluation of Weapon Equipment Based on ANP and Simulation. Systems Engineering-Theory & Practice, 31(06): 1086-1094. [151] Zheng, Baohua, Zhaofeng Zhang, and Guangtong Fang, et al. (2012). Influence Factor Analysis of Command and Control Cooperation Efficiency in Operating System. Fire Control & Command Control, 37(07): 70-73. [152] Wu, Chao, Hongfei Xu. (2005). Effectiveness Evaluation Method of QADC for Operational Project of which Submarine Attack Naval Ship. Fire Control & Command Control, 2: 75-77+80. [153] Liang, Guilin, Xiaoji Zhou, and Yaqiong Wang. (2018). Efficiency Evaluation Research on Ground System of Remote Sensing Satellite Based on ADC Model. Command Control & Simulation, 40(05): 62-68.

References

325

[154] Wang, Min, Yu Tao, Danying Fan. (2016). Research on Effect Valuation to the Complicated Weapon System Based on ADC. Fire Control & Command Control, 41(02):113-116. [155] Huang, Guo Qing, and Mingxu Wang. (2013). Effectiveness Evaluation of Multifighter Cooperated Combat Based on ADC Model. Fire Control & Command Control, 38(10): 126-129. [156] Yang, Xiaoxiao, Hui Qi, and Lei Chen. (2014). Effectiveness Evaluation of Submarine Anti-Submarine Combat Scheme Based on Improved ADC Method. Journal of Naval Aeronautical and Astronautical University, 29(03): 285-290+300. [157] Huang, Guo Qing, and Mingxu Wang. (2013). Effectiveness Evaluation of Multifighter Cooperated Combat Based on ADC Model. Fire Control & Command Control, 38(10): 126-129. [158] Yan, Yongling, Qingbo Zhang, and Yongping Xin. (2011). The Effectiveness Evaluation of the C4ISR System Based on the ADC Method. Fire Control & Command Control, 36(03): 71-73+77. [159] Luan, Xiaofeng, and Rui Wen. (2010). Evaluation for Ship-to-Ship Missile Weapon System Effectiveness Based on UML and ADC Method. Computer & Digital Engineering, 38(08): 135-137+149. [160] Wang, Shixue, and Zhaogeng Li. (2009). Tank Operational Effectiveness Evaluation Based on Improved ADC Model. Ordnance Industry Automation, 28(07):14-16. [161] Zhao, Decai, Luping Wang, and Ji Li, et al. (2009). Efficiency Evaluation of Battlefield Communication Systems Based on ADC Model. Ship Electronic Engineering, 29(06): 96-98. [162] Wang,Wei, and Wewei Yuan. (2008). The effectiveness appraisal of C4ISR communication system based on the modified ADC method. Shanxi Architecture, 24: 21-22. [163] Liu, Jun. (2015). Research on the Comprehensive Evaluation of the Level of Electric Power Enterprise Information Based on Cloud Model. North China Electric Power University. [164] Wang, Jian, Wenjie Xiao, Shuwen Wang, Fuxian Zhao, and Zhihuai Li. (2010). An Improved Effectiveness Evaluation Method based on Cloud Model. Fire Control & Command Control, 35(07): 139-142. [165] Chen, Guilin. (2010). Uncertain Model of Qualitative/Quantitative Information Transformation Cloud Model. Application Research of Computers, 27(06): 20062010. [166] Zhang, Haifeng, Fanglin Han, and Changpeng Pan. (2019). Operational Effectiveness Evaluation of UAV to Sea Assault Based on Cloud Model. Ordnance Industry Automation, 38(04): 57-61.

326

References

[167] Zhu, Miaomiao, Weijie Pan, Xiang Liu, Jian Lv and Huiliang Zhao.(2020). Interactive Genetic Algorithm Based on BP Neural Network and User Cognitive Surrogate Model. Computer engineering and Application, 56(02): 146-151. [168] Li, Jian, Haibing Liu, and Di Hu. (2020). Weapon System Effectiveness Evaluation Based on Generative Adversarial Network and BP Neural Network. Computer Applications and Software, 37(02): 253-258. [169] Yujun Zheng, Jinlin Zhang, and Yuehua Li. (2012). Operational Effectiveness Evaluation of A Certain Equipment Based on GA-BP Neutral Network. Journal of Air Force Radar Academy, 26(05): 346-348. [170] Wang, Zhen. (2015). Application of the Multi Index Comprehensive Evaluation Model Based on Factor Analysis and BP Neural Network. Yunnan University. [171] Tang, Ke, Cunwei Wang, and Jinjin Lu. (2010). Research of Factor of Influencing Shrapnel Spreading Precision Based on Neural Network and Matlab. Ordnance Industry Automation, 29(07): 90-93. [172] Wang, Chao, Chi Ma, and Junjie Chang. (2020). Cooperative Engagement Capability Evaluation Based on Improved Wavelet Neural Network. Command Information System and Technology, 11(01): 41-45. [173] Qi, Zongfeng, Huabing Wang, and Jianxun Li. (2020). Combat Capability Evaluation Method Based on Deep Learning for Radar Reconnaissance System. Command Control & Simulation, 42(02): 59-64. [174] Ren, Jun, Ning Li. (2017). Prediction of Weapon System Effectiveness Based on Stack Self-coding Dimension Reduction. Military Operations Research and Systems Engineering, 31(01): 61-67. [175] Liu, Minghui, Feng Yang, and Lei Wang, et al. (2009). An Operational Effectiveness Exploratory Evaluation Method of Anti-ship Missile Based on SVR. Computer Simulation, 26(8): 9-12. [176] Liu, Minghiu, Feng Yang, Lei Wang, and Hong Wu. (2009). An Operational Effectiveness Exploratory Evaluation Method of Anti-ship Missile Based on Support Vector Regression. Computer Integrated Manufacturing Systems, 26(08): 9-12+44. [177] Wu, Liang, Xiangmei Zuo and Yong Qiu. (2020). Study of surrogate model technology in multidisciplinary design optimization based on SVR. Flight mechanics, 38(02): 23-28+33. [178] Ren, Jun, Xiaofeng Hu, and Feng Zhu. (2017). Effectiveness Prediction of Weapon Equipment System of Systems Based on Deep Learning Feature Transfer. System Engineering and Electronics, 39(12): 2745-2749. [179] Xiao, Ningcong, Kai Yan and Yongshan Wang. (2019). Structural Reliability Analysis Using Sequential Surrogate Models. Journal of University of Electronic Science and technology, 48(01): 156-160.

References

327

[180] Zhao, Zhimei. (2014). Feature Selection Algorithm Based on Surrogate Model and Artificial Immune System. Computer engineering and design, 35(06):2174-2178. [181] Zhang, Yanxiao, Danqi Chen, Ying Han and Daohua Liu. (2014). Surrogate Model of Radial Basis Function Networks Based on Width Factor Sensitivity Analysis. Journal of Intelligent Systems, 9(02): 259-264. [182] Gan, Xudong. Evaluation and Analysis of Swarm Intelligence Algorithms. Shanghai Jiao Tong University. 2011. [183] Qi, Yun. (2012). Construction of Command Information Process Optimization Capability Based on Information System Operation. Journal of Sichuan Ordnance, 33(03): 107-108+128. [184] Hong, Liming. (2011). A Study on Optimization of Combat Process Based on Information. National Defense Science & Technology, 32(05): 63-65. [185] Gao, Changyuan, Dinghong Peng, and Yi Zhong. (2011). Research on Automatic Comprehensive Evaluation Model Selection. Journal of Harbin University of Science and Technology, 16(06): 118-123. [186] Dinggang, Wu. (2012). Research on Operation Cooperated Flow Modeling of AEWA Based on UML. Ship Electronic Engineering, 32(05): 8-11. [187] Qiao ,Yongjie, Liang Sun, Xingbo Chen, and Wei Huo. (2011). Effectiveness Evaluation of Antimissile-warning-system Based on Operational Activity. Fire Control and Command, 36(06): 120-123. [188] Zhao, Xin. (2011). Dynamic-Stochastic Influence Diagrams Modeling and Simulation Approach for Naval Theater Analysis. National University of Defense Technology. [189] Tian, Yunfei, Jiangang Yan, Shiling Li, and Lei Chu. (2011). Research and Effectiveness Evaluation of Air Anti-Submarine System Based on Net-Ring Models. Ship Electronic Engineering, 31(02): 31-34. [190] Yu, Jie, Wei Wang, and Guoning Zhang. (2011). Research on Joint Operations Command System Based on Complex Network. Fire Control & Command Control, 36(02): 5-10. [191] Kai, Luo, Mingzhi Zhang and Xi Wu. (2016). Key Nodes Analysis Model for Space Information Temporal Network Based on Operation Loop. System Engineering and Electronic Technology, 38(07): 1572-1576. [192] Wang, Tao, Liuying Wang, and Leilei Chang, et al. (2019). Dynamic Evaluation of Missile Combat System Effectiveness Based on Operation Loop. Modern Defence Technology, 47(03): 42-49. [193] Luo, Jinliang, and Xi Wang. (2011). Research on Effectiveness and System Structure of Air-defense Missile Network Operation. Tactical Missile Technology, 5: 15-18.

328

References

[194] Luo, Chengkun, Yunxiang Chen, Bailiguo Zhang, and Lili Wang. (2016). Sensitivity Analysis on Influencing Factors of Fighter’s Demand Based on Sobol’Method. Fire Control and Command, 41(10): 122-125+129. [195] Zhou, Ding, An Zhang, and Huan Chang. (2016). Evaluation of Operational Effectiveness of Space-based Strike Weapon and Sensitivity Analysis. Journal of Ballistics, 28(02): 5-11+52. [196] Ruan, Wenbin, Yang Liu, and Lei Xiong. (2016). Influence of side aerodynamic derivative uncertainty on side flight load based on global sensitivity analysis. Acta Aeronautica ET Astronautica Sinica, 37(06): 1827-1832. [197] Yan, Congxhi, Feng Zhao, and Jin Liu. (2013). Sensitivity Analysis of Practical Factors for Radar Based on Simulation System. Modern Defence Technology, 41(02): 147-154+166. [198] Zhou, Zhichao. (2013). Sensitivity Analysis of Oriented to System Effectiveness of Weapons and Equipment. Fire Control and Command, 38(02): 98-102. [199] Chai, Tong, Zhongxiang Tong, Yanlong Lu, and Chaozhe Wang. (2012). Comparison of Air Combat Effectiveness Assessment Methods Based on Sensitivity Analysis of Indicator. Fire Control and Command, 37(06): 21-24. [200] Hu, Kun, and Bintao Liu. (2011). Research on Damage Effect of Spacecraft Based on the Sensitivity Analysis. Computer Integrated Manufacturing Systems, 28(10): 52-56. [201] Li, Dong, Chaohui Qu, and Haiyang Ma. (2011). Sensitivity Analysis of Operational Effectiveness Based on Interval Method. Electronics Optics & Control, 18(07): 1-4. [202] Li, Dong, Lu Wang, and Wenzheng Kang. (2011). Sensitivity Analysis of Operational Effectiveness Based on SVM. Ordnance Industry Automation, 30(01): 35-38. [203] Lu, Lingyun. (2013). Research on Sensitivity Analysis Method and Tool Based on Simulation Experiments. Harbin Institute of Technology. [204] Wang, Xiaodi. Global Sensitivity Analysis for High-dimensional Complex Models. East China Normal University, 2012. [205] Jiang, Zhenyu and Weihua Zhang. (2008). Prediction Model-based Sensitivity Analysis in Virtual Experiment. Journal of system simulation, 11: 3059-3061+3064. [206] Han, Linshan, Xiangyang Li and Dakao Yan. (2008). Several Mathematical Methods of Sensitivity Analysis. Water Transportation in China (Second Half of the Month), 4: 177-178. [207] Ren, Tianzhu, Wanqing Xin, and Xijun Yan, et al.(2019). A Method for Establishing the Combat Capability Evaluation of SoS Based on the Extreme Learning Machine. Missiles and Space Vehicles, 6: 107-111.

References

329

[208] Guo, Shengming, Xiaoyuan He, Lin Wu, and Xiaofeng Hu. (2018). Backtracking analysis approach for effectiveness of air defense operation system of systems based on force-sparsed stacked-autoencoding neural networks. Science China Information Sciences, 48(07): 824-840. [209] Zhai, Huawei, Licheng Cui and Weishi Zhang. (2019). Novel Online Adaptive Algorithm of Extreme Learning Machine Based on Improved Sensitivity Analysis. Minicomputer system, 40(07): 1386-1390. [210] Li, Jichao, Kewei Yang, Xiaoke Zhang, Jiang Jiang, and Duzheng Qing. (2016). Equipment Contribution Degree Evaluation Method Based on Combat Network of Weapon System-of-System. Complex Systems and Complexity Science, 13(03): 1-7. [211] Zhang, Feng, and Xiao Liu. (2019). Research on contribution rate evaluation of anti missile early warning equipment system. China CIO News, 10: 139+142. [212] Wang, Ying, Shuai Shi, and Guangzhi Bu, et al. (2019). Model of Contribution Rate of Equipment to System of Armament Systems. Modern Defence Technology, 47(04): 102-107. [213] Luo, Chengkun, Yunxiang Chen, and Huachun Xiang, et al. (2019). Review of the Evaluation Methods of Equipment’s Contribution Rate to System-of-Systems. Systems Engineering and Electronics, 41(08): 1789-1794. [214] Li, Xiaobo, Weiping Wang, and Lin Lin, et al. (2019). The Research Framework, Progress, and Key Directions of System-of-Systems Contribution Ratio Evaluation. Systems Engineering-Theory & Practice, 39(06): 1623-1634. [215] Yin, Rongrong, Xueliang Yin, and Mengkai Cui, et al. (2019). Node Evaluation Method Based on Importance Contribution in Scale-free Networks. Journal of Software, 30(06): 1875-1885. [216] Yin, Xiaojing, Xiaofeng Hu, and MingRong, et al. (2019). Review of Evaluation Methods of Contribution Rate to System of Systems. Journal of System Simulation, 31(06): 1027-1038. [217] Peng, Geng, Shaoping Zhou, and Xuming Zhang, et al. (2019). Contribution Rate Calculation of Weapon System. Fire Control & Command Control, 44(04): 3336+43. [218] Song, Jinghua, Liang Li, and Qisheng Guo. (2019). Research on Weapon Equipment System Contribution Rate Evaluation Method. Fire Control & Command Control, 44(03): 107-111. [219] You, Yaqian, and Jiang Jiang, et al. (2019). Evidential Network-based Evaluation Method of Contribution to Weapon System-of-Systems. Systems Engineering and Electronics, 41(08): 1780-1788.

330

References

[220] Yang, Kewei, Zhiwei Yang, Yuejin Tan, and Qingsong Zhao. (2019). Review of the Evaluation Methods of Equipment System of Systems Facing the Contribution Rate. Systems Engineering and Electronics, 41(02): 311-321. [221] Li, Xiaobo, Mu Lin, and Zhe Shu, et al. (2018). Synthesized Capability-Effectiveness Evaluation Method of Contribution Ratio to System-of-Systems. Journal of System Simulation, 30(12): 4520-4528+4535. [222] Cheng, Lixing. (2018). Thoughts on the Equipments’ Contribution Rate to System of Systems. Military Operations Research and Systems Engineering, 32(03): 37-43. [223] Chen, Xiaowei, Junqi Zhang, and Yongzhi Yang. (2018). Analysis on Measurement Methods of New Equipment System-of-Systems Contribution. Journal of Ordnance Equipment Engineering, 39(04): 19-22. [224] Luo, Xiaoming, Yanlei Zhu, and Rong He. (2017). Research on Evaluation Method of Contribution to System War Fighting for Weapons and Equipment System Based on Complex Network. Fire Control & Command Control, 42(02): 83-87. [225] Wang, Shuangchuan, Xisheng Jia, Qiwei Hu, and Qiang Wang. (2019). Effectiveness evaluation for equipment maintenance support system based on normal grey cloud model. Journal of Systems Engineering and Electronics, 41(07): 1576-1582. [226] Li, Gang, Yafei Jiao, Fuyan Liu, Min Yu, Yu Song, and Fuzhen Wen. (2016). Comprehensive Evaluation of Big Data Quality in Power Systems with Entropy Weight and Grey System Theory. Electric Power Construction, 37(12): 24-31. [227] Xu, Cundong, Donghui Yao, Suo Zhang, and Dan Hu. (2013). Application of Improved TOPSIS Comprehensive Evaluation Model to Optimization of River Regulation Schemes. Journal of Hohai University (Natural Sciences), 41(03): 222-228. [228] Lin, Mu, Xiaobo Li, and Yanfeng Wang, et al. (2019). Capability-effectiveness evaluation of contribution ratio to system-ofsystems based on QFD and combination weights TOPSIS. Systems Engineering and Electronics, 41(08): 1802-1809. [229] Yue, Lin, Guofu Xiang, and Weihao Hu. (2015). A Mission-Oriented Simulating and Optimizing Method for the Shipborne System Information Flow. Chinese Journal of Ship Research, 10(06): 114-119+125. [230] Zhang, Lei, Lin Zhu, and Huaping Zhang. (2013). Ship Formation Air-defence Operation in Resource Management Based on Operation Flow. Ship Electronic Engineering, 33(02): 1-3+16. [231] Zheng, Xiao. (2016). Research on Simulation Analysis Optimization Method and Tool for System of Systems Effectiveness. Harbin Institute of Technology. [232] Xing, Huange, Yibo Peng, and Yuan Qiu. (2014). Research on Optimization Algorithm for Command Architecture Based on Order Degree of Network Organization. Command Control & Simulation, 36(01): 19-24.

References

331

[233] Xu, Yuguo, Jing Qiu, and Guanjun Liu. (2012). Optimization Design on Cooperation Effectiveness of Equipment Maintenance Support Network Based on Complex Network. Acta Armamentarii, 33(02): 244-251. [234] Miu, Xudong. (2010). The Self-organization Decision-Making Model and Decision Support System of the Warship Formation Cooperative Operation. Military Operations Research and Systems Engineering, 24(01): 48-52. [235] Wang, Pengfei. (2014). Research on the Multi-fighter Corporative Combat Effectiveness Evaluation and Uncertainty Problem. Zhengzhou University. [236] Guo, Xi. (2011). Research on Risk and Effectiveness Evaluation Technology on Crime Prevention System Based on Risk Entropy Model. Wuhan University.

About the Authors Deping Zhang

Deping Zhang received his bachelor’s degree in Computational Mathematics and master’s degree in Applied Probability and Statistics from Lanzhou University, China, in 1997 and 2000 respectively, and his Ph.D. degree in computer software and theory from Southeast University, China, in 2009. Currently, he is associate professor, master supervisor in College of Computer Science and Technology at Nanjing University of Aeronautics & Astronautics, China. His main research interests include big data mining and data analysis, algorithm design and analysis. He has published more than 30 papers in journals and conferences, a postgraduate textbook and a monograph. Email: [email protected].

334

About the Authors

Xuefeng Yan

Xuefeng Yan received his B.Eng. and M.Eng. Degrees in mining engineering from China University of Mining Technology, China, in 1998 and 2001 respectively, and his Ph.D. degree in computer science and technology from Beijing Institute of Technology, China, in 2005. He was a visiting scholar in the Department of Science at Georgia State University, from 2008 to 2009 and 2012 to 2013. Currently, he is the vice dean and professor in College of Computer Science and Technology at Nanjing University of Aeronautics & Astronautics, China. Dr. Yan has published 3 monographs and more than 50 papers in various journals and conferences. His recent research focuses on the modeling, simulation and evaluation of complex products, especially the domain-oriented modeling methodology and platform, mission-oriented evaluation and application. Email: [email protected].

Index A access, 5, 224, 295, 296, 299 actual output, 191 adaptability, 4, 8, 13, 30, 45, 48, 50, 51, 62, 63, 104, 106, 107, 111, 112, 113, 114, 263, 275 adjustment, 85, 144, 203, 248 aggregation, 29, 47, 50, 52, 64, 68, 73, 76, 95, 96, 97, 98, 99, 107, 110, 170, 175, 218, 238, 247, 258, 298, 299 Air Force, 3, 317, 318, 326 algorithm, 2, 10, 20, 21, 64, 66, 68, 82, 83, 84, 88, 95, 121, 132, 143, 156, 170, 176, 177, 178, 189, 190, 191, 196, 197, 201, 202, 203, 205, 206, 208, 209, 247, 258, 302, 304, 305, 307, 310, 311, 333 analysis factor, 257 architecture design, 277, 288, 292 arithmetic, 15, 135, 144, 145, 179 artificial intelligence, 2, 20, 307 assessment, 29, 32, 211, 224 awareness, 114, 115, 145, 149

B bandwidth utilization, 288 base, 184, 304, 305 basic research, 279 behaviors, 11, 102, 212, 224 benefits, 2, 111, 248, 265, 272, 278 bias, 84, 123, 200, 201, 203 Boltzmann distribution, 200

C classification, 59, 91, 106, 193, 265, 304 coding, 298, 326 column vectors, 144, 255

communication, 10, 14, 15, 22, 49, 54, 57, 58, 63, 91, 92, 112, 166, 231, 233, 240, 268, 271, 297, 298, 307, 315 comparative analysis, 32, 264 compatibility, 14, 16, 35 complex interactions, 23 complex numbers, 106 complex system performance, viii complex systems, viii, 1, 21, 22, 23, 44, 205, 275 complexity, 17, 31, 37, 45, 83, 84, 198, 209, 275, 305 composition, 15, 41, 42, 43, 47, 60, 112, 128, 211, 271, 288 computation, 97, 152, 189, 257, 296, 304, 305 computer, viii, 18, 21, 22, 122, 283, 284, 289, 291, 294, 296, 297, 329, 333, 334 computer simulation, viii, 21, 95 computing, 2, 22, 173, 216, 237, 284, 299 concurrency, 22, 225 configuration, 3, 94, 236, 290, 292, 296, 297, 299, 301, 303, 304 connectivity, 60, 61, 63, 91, 103, 104, 105, 107, 108, 109, 110, 112, 113, 116, 117 construction, viii, 26, 27, 28, 29, 30, 31, 32, 33, 34, 38, 39, 40, 44, 50, 52, 53, 56, 59, 71, 72, 73, 126, 127, 128, 134, 135, 138, 140, 141, 157, 166, 170, 184, 185, 198, 204, 205, 209, 211, 229, 230, 247, 277, 278, 290, 297, 298, 300, 302, 303 correlation analysis, 23, 64, 80, 104, 125, 150, 203, 204, 302 correlation coefficient, 38, 68, 80, 81, 82, 85, 113, 140, 150, 151, 153, 155, 156, 157, 158, 159, 204 correlation(s), 18, 23, 29, 38, 52, 53, 64, 65, 68, 69, 80, 81, 82, 84, 85, 97, 102, 104, 112, 113, 125, 140, 141, 142, 143, 144, 146, 147, 150, 151, 153, 155, 156, 157, 158, 159, 174, 203, 204, 208, 247, 265, 292, 302, 304 critical value, 81 cruise missiles, 157, 158

336

Index

D data analysis, 2, 18, 52, 93, 333 data generation, 206, 301 data mining, 20, 23, 198, 333 data processing, 10, 22, 301 data set, 38, 86 database, 223, 295, 299, 300, 303 decision control, 239, 302 decomposition, 33, 34, 35, 36, 40, 44, 56, 73, 76, 152, 153, 154, 167, 253, 283, 284, 292, 293, 303, 309 depth, 3, 74, 79, 112, 197 destruction, 34, 113 detection, 7, 8, 9, 34, 40, 42, 43, 49, 51, 54, 56, 57, 58, 59, 61, 62, 63, 74, 75, 76, 77, 78, 111, 115, 127, 131, 145, 149, 166, 169, 170, 197, 207, 223, 227, 231, 234, 239, 240, 250, 271, 272, 302, 307, 309, 322 detection system, 49, 76, 78, 239 development policy, 2 deviation, 38, 115, 180, 182, 186, 193, 199, 209 differential equations, 21 dimensionality, 21 discrete random variable, 208 dispersion, 60, 61, 83, 106, 107, 151, 174, 204, 250 distributed computing, 22 distribution, 47, 52, 61, 63, 73, 102, 103, 106, 107, 169, 172, 173, 175, 176, 177, 178, 200, 203, 233, 298 distribution function, 103, 176 dynamic factors, 45

E early warning, 8, 39, 40, 42, 44, 49, 51, 61, 62, 111, 115, 116, 166, 169, 170, 185, 223, 227, 231, 239, 270, 271, 272, 273, 309, 329 effectiveness index, 3, 4, 6, 7, 8, 9, 16, 17, 18, 20, 25, 31, 41, 49, 50, 52, 56, 58, 59, 60, 63, 73, 76, 78, 87, 88, 106, 112, 131, 165, 170, 181, 204, 208, 215, 243, 245, 247, 251, 257, 260, 266, 275, 293, 301, 303 effectiveness measurement, 7, 10, 12, 19, 29, 106, 162, 211 electromagnetic, 36, 55, 63, 117, 234 emergency, 43, 231, 239 energy, 39, 55, 109, 169, 200, 201, 202, 229 engineering, viii, 2, 3, 4, 26, 33, 40, 276, 277, 278, 279, 283, 290, 291, 318, 326, 327, 334 entropy, 106, 173, 174, 178, 181, 183, 204 environment(s), viii, 1, 2, 4, 8, 9, 10, 11, 13, 15, 16, 22, 30, 35, 36, 42, 48, 61, 63, 70, 72, 74, 98, 99,

114, 125, 184, 233, 263, 276, 277, 284, 290, 294, 295, 296, 298, 300, 301, 302 equipment, viii, 2, 3, 4, 5, 7, 8, 11, 12, 13, 14, 15, 18, 20, 22, 25, 26, 29, 30, 32, 33, 34, 35, 36, 40, 42, 45, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 63, 71, 73, 83, 91, 92, 95, 98, 99, 100, 111, 144, 145, 157, 163, 167, 170, 172, 204, 205, 206, 213, 215, 222,226, 227, 228, 229, 230, 231, 232, 233, 235, 236, 237, 239, 243, 247, 248, 250, 257, 262, 263, 264, 265, 268, 269, 270, 271, 272, 273, 275, 276, 277, 289, 291, 293, 294, 296, 297, 299, 300, 301, 307, 322, 329, 330 error estimation, 123 evacuation, 71, 74 evaluation index, 3, 8, 17, 18, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 44, 47, 48, 49, 50, 52, 53, 59, 64, 72, 73, 74, 76, 77, 79, 81, 82, 83, 84, 87, 89, 90, 91, 93, 94, 97, 98, 99, 102, 104, 106, 113, 126, 128, 129, 130, 138, 140, 141, 142, 144,145, 146, 150, 151, 156, 157, 166, 167, 169, 172, 180, 181, 183, 184, 185, 186, 203, 204, 206, 209, 211, 240, 243, 249, 250, 257, 258, 259, 260, 271, 296, 302, 303, 304, 305, 307, 308 evidence, 101, 120, 269 execution, 15, 22, 70, 72, 73, 92, 161, 163, 168, 169, 171, 211, 213, 214, 216, 217, 220, 221, 224, 280, 285, 286, 287 experimental design, 189, 206, 295 extraction, 86, 303, 304

F factor analysis, 84, 104, 150, 151, 152, 153, 155, 156, 157, 158, 159, 204 formation, 38, 39, 40, 41, 42, 43, 51, 53, 54, 59, 60, 61, 63, 65, 69, 75, 77, 105, 165, 166, 167, 168, 169, 170, 171, 184, 185, 186, 187, 188, 189, 270, 271, 284, 300, 303 formula, 1, 10, 78, 87, 88, 90, 93, 95, 99, 105, 110, 113, 115, 123, 142, 151, 155, 160, 162, 187, 195, 201, 206, 208, 221, 229, 235, 237, 245, 246, 251, 255, 256, 269, 270 functional analysis, 277 fusion, 2, 22, 32, 78, 115, 116, 138, 141, 143, 145, 166, 207, 227, 264, 301 fuzzy membership, 175, 176 fuzzy set theory, 173

G geometric programming, 1 geometry, 16, 173 graph, 52, 66, 84, 106, 113, 286

Index gravity, 42, 48, 102, 104, 106, 107, 112, 173, 180, 181, 182, 183 grid computing, 22 grouping, 135, 300 guidance, vii, viii, 42, 69, 96, 97, 145, 157, 159, 166, 185, 224, 249, 262, 278, 279, 291, 295 guidelines, 2, 49 guiding principles, 29, 51

H heterogeneity, 63, 106, 113 historical data, 197 history, 2, 216 hotspots, 23 human, 9, 22, 35, 98, 135, 173, 265, 289, 291, 292, 294 human-machine, 292 hydrological conditions, 74 hypercube, 260, 261, 309 Hypernetwork, 106, 323 hypothesis, 63, 156

I independent variable, 81, 198 industry, 2, 132, 275, 280 inequality, 204, 305 inertia, 208, 209 information exchange, 111, 285, 286, 287, 295 information processing, 34, 40, 49, 57, 58, 63, 114, 116, 227, 231, 232, 271 information sharing, 227, 228, 233, 234, 235, 236, 271 infrastructure, 232, 278, 280 integration, viii, 1, 3, 4, 26, 98, 101, 180, 272, 277, 278, 280, 291, 295 intellectual property, 290, 291, 300 intelligence, viii, 1, 2, 22, 43, 227, 233, 234, 239, 304 interaction effect, 255, 256, 257, 259 interface, 214, 283, 284, 287, 288, 289, 290, 292, 293, 294, 296, 298, 299, 301, 303 interference, 5, 8, 12, 14, 15, 35, 43, 49, 57, 63, 80, 145, 227, 234, 235, 240, 249, 250, 272, 302 interoperability, 16, 35, 61, 107, 280 interpretability, 154, 204 IP address, 294 issues, 3, 12, 23, 36, 58, 161, 290, 299 iteration, 122, 208, 209, 217, 221, 222, 287, 305

337

L Lagrange multipliers, 197 languages, 283 learning, 98, 143, 190, 191, 192, 197, 199, 201, 202, 203, 209, 302, 315 life cycle, 7, 291, 292, 295, 300, 301, 302, 303, 304 Likert scale, 93 linear dependence, 151 linear function, 160, 267 linear model, 198 logistics, 40, 51

M machine learning, 143, 203, 204, 209, 210, 247, 249, 252, 258 manufacturing, 13, 309, 310, 317 mapping, 6, 10, 11, 13, 47, 65, 86, 89, 172, 173, 174, 175, 190, 193, 283, 293, 303 mathematics, 2, 92, 93, 94, 170, 247 matrix, 17, 18, 65, 68, 69, 85, 87, 88, 95, 105, 106, 107, 108, 110, 119, 120, 121, 122, 123, 124, 125, 127, 128, 129, 130, 131, 133, 134, 135, 136, 137, 138, 140, 141, 142, 143, 144, 146, 147, 148, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 164, 165, 168, 169, 171, 181, 186, 200, 237, 241, 255, 256, 288 memory, 190, 191, 288, 298 methodology, 22, 275, 276, 277, 278, 280, 283, 284, 287, 291, 334 migration, 213, 214, 216, 222 military, 2, 8, 9, 13, 17, 19, 21, 23, 26, 50, 144, 145, 239, 257, 263, 265, 278, 279, 280, 284, 286, 305 model system, 295, 299, 300 models, 2, 3, 18, 27, 31, 52, 64, 95, 98, 107, 173, 176, 179, 181, 188, 189, 198, 203, 248, 249, 253, 260, 271, 280, 283, 284, 292, 293, 295, 296, 299, 300, 302, 303, 304 modules, 72, 225, 259, 287, 288, 289, 295 Monte Carlo method, 20, 21, 253 multidimensional, 98, 172, 217 multiple factors, 22, 211

N network elements, 190 network theory, 51, 52, 113 networking, 4, 50, 63, 111 neural network(s), 2, 3, 4, 143, 189, 190, 191, 192, 197, 201, 203, 208, 248, 260, 329 neurons, 190, 197, 199, 200, 201, 202, 209, 260

338

Index

nodes, 31, 45, 54, 55, 56, 59, 61, 66, 67, 68, 92, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 116, 117, 140, 141, 192, 207, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 226, 227, 229, 230, 231, 232, 233, 234, 237, 238, 239, 243, 268, 270, 271, 289

O objective criteria, 148 objective reality, 150 operational capability, 2, 3, 10, 13, 14, 15, 16, 20, 25, 29, 30, 31, 32, 34, 35, 39, 45, 46, 51, 59, 61, 62, 79, 95, 96, 97, 98, 106, 107, 111, 112, 114, 131, 142, 144, 145, 146, 148, 149, 160, 169, 171, 232, 233, 235, 236, 237, 238, 258, 259, 260, 263, 264, 265, 266, 267, 268, 277, 283, 286, 316 operational effectiveness, 1, 2, 4, 5, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 20, 30, 31, 32, 33, 39, 40, 59, 60, 61, 62, 63, 73, 76, 77, 78, 80, 89, 100, 104, 132, 138, 144, 145, 146, 157, 160, 188, 189, 211, 214, 221, 223, 226, 235, 238, 243, 245, 251, 252, 257, 259, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 303, 308 operational mission profile, 71 operations, 8, 18, 21, 26, 51, 57, 59, 60, 61, 63, 65, 71, 72, 118, 145, 184, 188, 215, 222, 232, 243, 257, 268, 318 operations research, 21, 118, 318 optimization, v, vi, viii, 1, 3, 4, 5, 6, 23, 36, 40, 47, 84, 94, 98, 102, 111, 113, 138, 140, 191, 193, 194, 195, 196, 198, 204, 208, 209, 259, 275, 276, 277, 278, 279, 283, 285, 287, 288, 289, 290, 291, 293, 295, 297, 299, 300, 301, 302, 303, 304, 305, 306, 307, 309, 310, 311, 319, 320, 321, 323, 326, 327, 330, 331 optimization method, viii, 276

P parallel, 22, 163, 164, 165, 167, 215, 235, 236, 287, 288, 289, 290, 299 performance measurement, 9, 10, 11, 72 platform, 59, 78, 157, 159, 168, 229, 231, 252, 276, 277, 280, 291, 292, 293, 294, 295, 296, 299, 300, 301, 302, 303, 334 polynomial functions, 197 principal component analysis, 66, 68, 84, 86, 87, 88, 157 principles, 26, 29, 44, 111, 112, 118, 181, 183 probability, 1, 4, 5, 7, 8, 9, 10, 14, 17, 18, 20, 31, 49, 50, 60, 61, 64, 76, 77, 78, 79, 80, 91, 97, 100, 101, 103, 106, 107, 109, 112, 114, 115, 116, 117,

154, 156, 161, 162, 163, 168, 169, 170, 172, 173, 174, 176, 200, 201, 202, 203, 207, 211, 214, 215, 216, 221,223, 226, 227, 231, 235, 237, 238, 240, 242, 243, 250, 259, 271, 272, 300, 302, 309 probability distribution, 154, 172, 201, 202, 259 propagation, 51, 190, 191, 192, 201, 298 protection, 42, 50, 54, 57, 63, 113, 263 prototype, 283, 289, 294

Q quadratic programming, 196 qualitative concept, 121, 172, 173, 174, 176, 178, 180 quantification, 26, 73, 94 quantitative concept, 176 query, 300, 304

R radar, 42, 43, 59, 116, 145, 149, 223, 224, 227, 232, 234, 239, 250, 251, 296, 301, 307 radius, 74, 107, 108, 157, 159, 232, 260, 262 recovery, 49, 114, 184, 297 redundancy, 26, 37, 105, 113 regression, 81, 84, 95, 142, 154, 156, 193, 196, 197, 198, 200, 206 regression analysis, 142, 196, 206 regression equation, 200 regression method, 95 regression model, 142, 193, 198 reliability, 4, 8, 10, 13, 15, 16, 18, 26, 34, 35, 36, 37, 38, 40, 42, 110, 112, 160, 191, 219, 322 residual matrix, 153 resolution, 49, 64, 65, 207, 231, 284 resource management, 295, 297 resource utilization, 92 resources, 51, 53, 61, 92, 103, 111, 269, 295, 296, 299, 300 response, 40, 43, 57, 58, 74, 96, 188, 234, 247, 248, 296 rotation transformation, 154 rules, 4, 68, 71, 73, 74, 119, 120, 140, 161, 187, 191, 198, 218, 283

S scope, 3, 44, 45, 283, 285, 305 semantics, 279, 280 sensor nodes, 107 sensor(s), 48, 49, 60, 74, 77, 98, 107, 109, 115, 166, 227, 296

Index simulation(s), viii, 2, 3, 18, 21, 22, 23, 24, 27, 31, 32, 39, 60, 83, 84, 95, 101, 104, 138, 141, 142, 143, 144, 145, 146, 170, 171, 175, 183, 188, 189, 192, 204, 206, 208, 209, 215, 224, 226, 243, 247, 249, 251, 255, 256, 257, 271, 276, 277, 278, 290, 291, 292, 294, 295, 296,297, 298, 299, 300, 301, 304, 305, 307, 328, 334 software, 72, 219, 280, 284, 292, 295, 300, 301 solution, 5, 20, 25, 83, 123, 130, 198, 201, 305 SoS effectiveness, 1, 5, 44, 101, 211, 212, 236, 238, 257, 258, 266, 267, 302, 309 stability, 36, 38, 40, 78, 110, 135, 148 standard deviation, 81, 82, 83, 151, 177, 204 standardization, 36, 82, 85, 206, 208, 280 statistics, 1, 4, 16, 28, 38, 84, 93, 126, 157, 174, 254 structure, 8, 13, 21, 22, 25, 27, 28, 29, 30, 32, 33, 39, 40, 41, 45, 46, 47, 48, 50, 51, 53, 60, 61, 63, 65, 67, 68, 69, 78, 95, 97, 99, 102, 106, 107, 110, 113, 118, 119, 124, 125, 126, 132, 133, 136, 137, 140, 141, 145, 154, 180, 190, 196, 198, 201, 206, 209, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 224, 226, 233, 236, 243, 271, 276, 278, 280, 284, 286, 291, 295, 302 submarines, 39, 53, 54, 56, 58, 59, 63, 74, 75, 76, 77, 78 synchronization, 22, 48, 51, 71, 102, 104, 219, 225, 295 synergistic effect, 270 synthesis, 22, 43, 50, 136, 144, 163 system analysis, 7, 9, 27, 275, 276, 278, 292 system effectiveness, viii, 1, 2, 3, 7, 8, 9, 10, 19, 20, 21, 22, 29, 45, 49, 56, 57, 59, 76, 84, 97, 98, 99, 102, 125, 126, 160, 161, 162, 163, 165, 180, 188, 211, 213, 252, 257, 265, 266, 267, 293, 301 system operational capability, 51, 62, 95, 263, 265, 266

339

115, 116, 118, 119, 121, 127, 130, 138, 148, 166, 169, 170, 171, 180, 184, 185, 187, 207, 209, 223, 224, 227, 228, 229, 230, 231, 232, 234, 239, 249, 250, 264, 265, 271, 285, 300, 302, 307, 309 technologies, 2, 20, 277, 278 technology, vii, viii, 1, 2, 4, 12, 22, 23, 24, 40, 198, 263, 264, 265, 271, 279, 290, 304, 305, 326, 334 test data, 197, 251 testing, 29, 95, 248, 260 topology, 61, 66, 67, 104, 105, 235, 238, 269, 271 training, 2, 29, 35, 190, 191, 192, 197, 198, 200, 201, 202, 203, 206, 208, 209, 210, 248, 251, 259, 260, 261, 307, 309, 310 trajectory, 10, 11, 157, 159 transformation matrix, 256 transformation(s), 2, 68, 72, 73, 76, 89, 91, 130, 132, 154, 172, 173, 175, 176, 204, 206, 208, 256, 305 transmission, 15, 34, 43, 57, 73, 104, 105, 115, 116, 207, 233, 234, 240, 241, 271, 272

V variables, 10, 21, 81, 82, 83, 87, 89, 151, 152, 154, 155, 156, 157, 158, 159, 174, 175, 176, 177, 190, 191, 198, 199, 200, 208, 245, 247, 248, 251, 252, 253, 255, 256, 257, 305 vector, 14, 17, 20, 84, 85, 86, 87, 98, 121, 123, 124, 130, 134, 135, 136, 142, 143, 144, 150, 153, 161, 167, 168, 171, 181, 182, 193, 196, 202, 204, 230, 231, 232, 233, 234, 246, 248, 256, 305 vehicles, 56, 163, 223 velocity, 157, 159, 235, 300 visualization, 232, 295, 296, 300 vulnerability, 10, 34, 46, 48, 49, 50, 61, 104

W T tactics, 2, 13, 71, 77, 157 target, 14, 15, 17, 18, 22, 33, 34, 38, 42, 49, 55, 63, 71, 72, 74, 75, 77, 78, 79, 80, 81, 91, 107, 114,

war, 2, 4, 5, 20, 21, 284, 314 weapons, 17, 18, 46, 72, 79, 160, 185, 265, 293, 296