MemComputing. Fundamentals and Applications 2021948719, 9780192845320

Citation preview

MEMCOMPUTING

MemComputing Fundamentals and Applications Massimiliano Di Ventra University of California, San Diego

3

3 Great Clarendon Street, Oxford, OX2 6DP, United Kingdom Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries © Massimiliano Di Ventra 2022 The moral rights of the author have been asserted Impression: 1 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by licence or under terms agreed with the appropriate reprographics rights organization. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above You must not circulate this work in any other form and you must impose this same condition on any acquirer Published in the United States of America by Oxford University Press 198 Madison Avenue, New York, NY 10016, United States of America British Library Cataloguing in Publication Data Data available Library of Congress Control Number: 2021948719 ISBN 978–0–19–284532–0 DOI: 10.1093/oso/9780192845320.001.0001 Printed and bound by CPI Group (UK) Ltd, Croydon, CR0 4YY Links to third party websites are provided by Oxford in good faith and for information only. Oxford disclaims any responsibility for the materials contained in any third party website referenced in this work.

To Elena, Matteo, and Francesca undeserved, yet the most beautiful gift of my life

Preface In questions of Science, the authority of a thousand is not worth the humble reasoning of a single individual. Galileo Galilei (1564−1642) MemComputing is a portmanteau word I introduced in a Nature Physics article of 20131 to represent ‘any physical system that computes in and with memory’. In other words, it is a type of computation performed by physical units able to concomitantly process and store information in the same region of space.2 However, although the prefix ‘mem’ stands precisely for mem-ory, I do not necessarily mean ‘long-term memory’ or ‘storage’. Rather, I have in mind the following equivalence: Memory is equivalent to time non-locality. A MemComputing machine then exploits time non-locality (the ability to remember its past dynamics) to perform the necessary tasks involved, whether this memory is short- or long-term.3 As such, the MemComputing paradigm is a radical departure from the accepted Turing paradigm of computation, and from its traditional implementation in a von Neumann architecture, where a central processing unit (CPU) is physically distinct from its storage unit.4 It is also fundamentally different from another computing paradigm of current interest: Quantum Computing. It is loosely inspired by the operation of the brain, although this analogy, at this time, is quite superficial and, as I will discuss in this book, only limited to a few features. The rationale behind MemComputing is simple, and yet very powerful. Unlike the traditional algorithmic/combinatorial way in which problems are solved nowadays, MemComputing represents them first as physical systems, including those combinatorial and optimization problems that seem difficult to imagine in physical terms. In fact, as I will explain at length, this physical representation can preserve the digital structure of the input and output, so that the digital version of MemComputing machines (digital MemComputing machines or DMMs), like our modern

1

M. Di Ventra and Y.V. Pershin, Nature Physics 9, 200 (2013). 2

MemComputing is not synonymous of ‘computing with memristive elements’ as some authors narrowly imply (see Section 4.5). Of course, as I will show in this book, resistive memories may be of use. However, the notion of MemComputing encompasses much more than resistive memories. In fact, it can be realized with many different physical systems and devices, and requires active elements as well. 3

Time non-locality could be due to any type of physical property of the system, e.g., its spin or charge polarization, atomic configuration, geometric shape, etc. (Pershin and Di Ventra, 2011a). 4

Let me also point out that MemComputing is not just ‘in-memory computing’ (or ‘computing in memory’). In fact, the latter typically refers to storing of data in random access memory (RAM), often in a distributed way, across different RAMs of a parallel computer or server. This speeds up the processing of information considerably, compared to processing that requires access of data from a hard disk. However, it still suffers from the limitations of the von Neumann architecture and does not fundamentally change the computing paradigm employed. See Section 4.8 for the consequences of taking away the ‘with memory’ part from MemComputing.

viii Preface

5

Even though quantum features of the underlying devices used in MemComputing may be of help, or even necessary for their operation, the physical systems I will consider in this book exhibit non-quantum dynamics. This means that they do not require the full description of Quantum Mechanics, based on the time evolution of a state vector in a Hilbert space. Rather, the dynamical state of the non-quantum systems representing DMMs resides in a phase space, which has a mathematical structure quite distinct from a Hilbert space. Therefore, MemComputing is fundamentally different from Quantum Computing both in its physical realization and its mathematical formulation. 6

This is somewhat similar to the challenges one faces when explaining Quantum Computing, which is also a physical approach to computation. Shaded boxed Sections The reader will notice that I have put a few Sections of the book in shaded boxes. These are Sections that cover quite advanced material, requiring substantial background in specialized fields. As such, I could only provide a cursory introduction to such subjects. However, these Sections can be skipped at first read without (I hope) much detriment to the understanding of the following material. The interested reader can then return later to those Sections, and use them as a starting point to learn more about these advanced topics with the aid of the suggested literature.

7

Unlike quantum computers that cannot be simulated efficiently on our modern computers, and thus require a hardware implementation to show any advantage.

computers, are scalable. The solutions of the original problem (if they exist) are the (long-time) equilibrium points of an appropriately designed non-quantum dynamical system.5 These dynamical systems can be engineered so that they do not manifest (quasi-)periodic orbits and chaos, and offer several advantages compared to quantum systems. At this point, it shouldn’t come as a surprise that I face a considerable challenge in explaining such a topic. Since efficient computation is our ultimate goal, we want to address a Computer Science audience. However, MemComputing is first and foremost a physical approach, hence some background in Physics and Engineering is necessary.6 To complicate matters further, some Physics aspects that help us understand the operation of these machines—such as (supersymmetric) Topological Field Theory—are not even common knowledge to all physicists, let alone scientists in other fields of research. Finally, determining whether a MemComputing machine actually computes efficiently requires demonstrating a lot of statements, with specific knowledge of diverse subjects such as Dynamical Systems Theory, Functional Analysis, Algebraic Topology, and even some Differential Geometry. The range of topics needed is thus quite broad. Therefore, to make MemComputing accessible to a wide audience, a compromise is in order. This book is my attempt to present this new computing paradigm in relatively simple terms, by keeping the Mathematics to the necessary minimum. The reader who wants to delve deeper into the Physics and Mathematics (often quite heavy) behind these machines will be urged to look into published papers, and references therein. However, I hope that the exposition in this book will be much easier to follow than in the technical papers, and thus provide a much-needed introduction to MemComputing accessible to a wide range of scientists, in a single, comprehensive volume. Since, for practical purposes, we are only interested in scalable machines, most of the book will be devoted to DMMs that map a finite string of symbols (such as 0s and 1s) into a finite string of symbols, so that we can write the input and read the output with finite precision. I will discuss that when implemented in hardware these machines can solve complex (non-convex) problems by employing time, energy, and space resources that scale only polynomially with the size of the problem. Quite unexpectedly, however, since these same machines are represented by non-quantum dynamical systems, their ordinary differential equations of motion can also be efficiently simulated on our modern computers.7 By ‘efficient’ simulations I mean those that require time and memory resources of the computer processor employed, that scale only polynomially with input size. I will indeed present in this book several examples of hard problems solved with polynomial resources by simply

Preface ix

simulating DMMs on standard processors that we find even in our laptops. In general, these simulations outperform, by orders of magnitude, state-of-the-art algorithms specifically designed to solve those problems. Therefore, these results show that a Physics-inspired paradigm of computation, such as MemComputing, offers advantages that are not easily achieved by traditional algorithmic means. This, then, begs the following question I often get asked. Since our modern computers are the closest physical realization of a deterministic Turing machine, and since we have solved some instances of hard problems in polynomial time on these computers, does this mean we have answered the famous question of whether NP = P or not? The answer is a resounding no: one cannot prove a mathematical conjecture using numerical simulations! In addition, the demonstrations we have carried out, and I will point out in this book, pertain to the continuous-time dynamics, not the discrete version one necessarily employs in numerical simulations.8 Therefore, the resolution of such a question would require more rigorous mathematical work, and has not been settled at the time of the writing of this book. Of course, the answer to that question is somewhat irrelevant to the practical application of MemComputing to hard problems of interest to both academia and industry. What matters in that case is whether MemComputing offers any particular advantage compared to other computational methods. As I will amply discuss in this book, this is indeed the case and has already been demonstrated in several published works.9 As a final note, I would like to offer a disclaimer.10 It is na¨ıve to think that a technical book, like the present one, would be completely free of errors, despite the best efforts of the author. Therefore, if I uncover any errors or confusing statements in the book after its publication, I will post corrections on a link to my website: https://diventra.physics.ucsd.edu/ Irrespective, I hope this book will be a useful starting point to think more deeply about the relation between physical systems and computation, with MemComputing an interesting representative example. —per intellectum et voluntatem— Massimiliano Di Ventra Carlsbad, San Diego Some time in 2021

Important remarks The book contains several ‘important remarks’, highlighted in shaded boxes. Unlike the shaded boxed Sections, I strongly suggest the reader to go through these remarks, as they are intended to clarify and stress, in a succinct way, very important points.

Margin notes are not marginal! I have taken advantage of ‘margin notes’ (like this one) to define and clarify concepts I use in the main text. This way, the reader (hopefully) should be able to readily understand most of the jargon before an in-depth study of the relevant literature. These notes are typically within (±) one page away from the place where the concept they expand on is first introduced.

8

Let us also recall that the famous NP vs. P question has a meaning only for Turing machines. MemComputing machines are not Turing machines. 9

As well as in numerous ones performed at the company MemComputing, Inc. (www.memcpu.com). 10

Not intended though as a justification for my mistakes.

Acknowledgements Showing gratitude is one of the simplest yet most powerful things humans can do for each other. Randolph Frederick Pausch (1960−2008) The work on MemComputing could not have been done without the invaluable collaboration of a few people, and to them goes my heartfelt gratitude. First of all, I am grateful to Yuriy Pershin with whom I started exploring the very concept of MemComputing at an early stage, and its application to a variety of optimization problems. I am also greatly indebted to Fabio Traversa, with whom I proposed the digital version of these machines, and its practical realization using self-organizing gates. Igor Ovchinnikov has introduced me to the supersymmetric Topological Field Theory of non-quantum dynamical systems. This powerful theoretical approach has provided us with a better physical understanding of these machines. Some of my students, present and past—Sean Bearden, Haik Manukian, Forrest Sheldon, Yan Ru Pei, and Yuanhang Zhang— have done a lot of research on several aspects of MemComputing, adding considerably to its development. I also want to thank Eric Fullerton, who, through the Center for Memory and Recording Research at the University of California, San Diego, has provided much-needed funding for my research in this field, when I could not get any from either government agencies or private foundations.11 John Beane has supported our efforts in the field with contagious enthusiasm and continuous encouragement, as well as provided seed funding for the company MemComputing, Inc., we have co-founded (together with Fabio Traversa) and which aims to develop this computing paradigm for industrial applications. Finally, no words can fully describe the support and encouragement my family has given me during the most difficult times. I therefore express my deepest gratitude to my wife, Elena, and my children, Matteo and Francesca, for their love, support, and patience toward an often absent husband and father.

.__ - - -

,1 I

I I I

. -------:.....-7>;;;:J"' . .__ .__ ..... '